Trustworthness and Validity in presentation of Research findings.pptx

Education articles

1. Trustworthiness and Validity in Research Clement

2. Reliability and validity The terms reliability and validity are most commonly used in quantitative research In quantitative research you would use research methods that are measurable, numerical and statistical They are usually cause and effect based and therefore reliability can be tested and re-tested

3. Reliability and validity In qualitative research, the purpose is not to prove something to be reliable and valid in a measured way but rather to provide more in- depth understanding of a phenomenon. In qualitative studies, it is argued that using reliability and validity as the way to ensure feasibility does not provide the type of trustworthiness needed

4. Reliability Reliability is about the credibility of your research and it demands consistency. If someone examines your research will it stand up to this scrutiny? In other words will someone to be able to re-test your data and obtain the same results

5. Reliability “Reliability refers to the fact that different research participants being tested by the same instrument at different times should respond identically to the instrument.” (Mouton 2007 p144).

6. Reliability Another term used when looking at reliability is generalisation. Reliability can be seen as the extent to which the results can be generalised and similar results obtained if the research was tried again. This term generalised is important in quantitative studies when looking at reliability because reliability in quantitative research is very much about if your results can be generalised to different measuring occasions (Collis & Hussey 2003: 145; Welman & Kruger 2003:139).

7. Type of Reliability What is it How to Establish it Inter-rater or inter-coder This is a measure of agreement Different participants used but with the same method/tool/instrument administered Test-re-test This is a measure of stability The same participants used but the method/tool/instrument is administered at different times Parallel forms This is a measure of equivalence Different participants are used at the same time and using a different method/tool/instrument   Split-halves This also measures equivalence Since it is often difficult to (like with the parallel form) have a different method/tool/instrument. The split halves instead would take the method/tool/instrument and split it into two equivalent halves and correlate those scores together. Internal Consistency This is about how consistently each item measures the same thing (construct) Correlate the performance on each item with the performance across participants

8. Validity Validity is about finding out if the research method/tool/instrument that you selected obtained the information that it was supposed to. In other words, the extent to which the tool/method/instrument that was selected actually reflects the reality of those constructs that were being measured. The question to ask for seeing if you have validity is do the findings actually reflect what is happening in the given situation? (Babbie & Mouton 2001 p22; Collis & Hussey, 2003 p58-60; Welman et al. 2007 p142-144).

9. Type of Validity Questions you would ask related to this type of validity Content validity Is the test representative? Does the measurement represent the specific content? Sampling validity This is similar to content validity. Face validity Does the test look like what it is supposed to? Is it well designed? Construct validity Does the test correspond to other variables? Does it measure what it is meant to? “Is there agreement between a theoretical concept and a specific measuring device or procedure? For example, a researcher inventing a new IQ test might spend a great deal of time attempting to "define" intelligence in order to reach an acceptable level of construct validity.” (Colorado State University 2013)   Criterion-related validity/instrumental validity Does the test accurately predict future behaviour? How accurate is the measure or procedure when compared to another?

10. Internal Validity Internal validity asks if the method and design will answer the research question that you want it to. In other words there must be no errors in your design of your research and your method must match the question. Internal validity is mainly used in quantitative research. Internal validity also relates to possible errors in the results even though controls were in place to prevent it

11. External Validity External validity is also mainly used in quantitative research It is focused on the ability to generalise findings from a specific sample to a larger population. External validity refers to how whatever results we get from our chosen sample that we need to be able with confidence say we could get similar results from the rest of the population, if we applied our same method and design to our question .

12. External Validity In other words you would ask yourself will I be able to generalise my results outside of my sample to the general population? Neuman (1997:145) gives an example of external validity by explaining that if something happens in a controlled environment such as a laboratory would the findings be able to be generalised when they are tested outside of that controlled environment.

13. Trustworthiness Reliability and validity cannot be measured in the same in qualitative research because the answers are not objectively measurable they are unique to each individual’s experience. Some qualitative researchers do still use the terms reliability and validity but the way in which they show reliability and validity differs. In qualitative research researchers look for trustworthiness, which will represent the same thing for a research project as reliability and validity but it is more suitable for qualitative research methods and studies.

14. Four areas of trustworthiness Lincoln and Guba (1985) state that within qualitative research, there are four criteria that must be used in order to make sure that a qualitative study would be considered to be feasible. These are the four areas of trustworthiness

15. Four areas of trustworthiness Credibility – This is similar to internal validity in quantitative studies. Transferability - This is similar to external validity or generalisability in quantitative studies. Dependability - This is similar to reliability in quantitative studies. Confirmability - This is in preference to what quantitative studies call objectivity. (Collis & Hussey, 2003p278-279; Lincoln & Guba 1985; Shenton, 2004)

16. Credibility Credibility is about how accurate your interpretation of the data is compared to the original data that your participant gave you in your research. Credibility is increased when the researcher spends long periods with the participant and also by using mixed methods (more than one method) of data collection

17. Transferability Transferability is based on the ability of the findings to be applied to a similar situation and then obtain similar results. This would allow for generalisation within an approach that does not lend itself to generalised findings. In other words, it is the degree to which the results and analysis can be applied beyond a specific research project

18. Dependability Dependability is about the quality of the process of integration that happens between the data collection method, data analysis and theory generation

19. Confirmability Confirmability is about how well the data collected supports the findings and interpretation of the study. It examines how well the findings flow from the data and it requires the researcher to have described the research process fully

20. Triangulation defined Triangulation involves using multiple data sources in an investigation to produce understanding. Some see triangulation as a method for corroborating findings and as a test for validity of research through the use of a variety of methods to collect data on the same topic. Rather than seeing triangulation as a method for validation or verification, qualitative researchers generally use this technique to ensure that an account is rich , robust , comprehensive and well-developed . Triangulation is an approach to research that uses a combination of more than one research strategy in a single investigation. Denzin (1978), however, indicated that triangulation should not be confused with mixed methods; but instead, these are two distinct ways to conceptualize interpretations and findings.

21. Rationale for triangulation Denzin proposes four reasons to triangulate: Enriching-outputs of different informal and formal instruments add value to each other by explaining different aspects of an issue and thus reducing sources of error. Refuting- where one set of options disproves a hypothesis generated by another set of options Confirming –where one set of options confirms a hypothesis generated by another set options Explaining –where one set of options sheds light on expected findings derived from one set of options It minimises bias and helps to balance out any of the potential weaknesses in each data collection method The goal in choosing different strategies in the same study is to balance them so each counterbalances the margin of error in the other.

22. Rationale for triangulation (Cont.) Qualitative investigators may choose triangulation as a research strategy to assure completeness of findings or to confirm findings. Assure completeness : The most accurate description of the elephant comes from a combination of all three individuals' descriptions. Confirm findings : Researchers might also choose triangulation to confirm findings and conclusions. Any single qualitative research strategy has its limitations. By combining different strategies, researchers confirm findings by overcoming the limitations of a single strategy. Uncovering the same information from more than one vantage point helps researchers describe how the findings occurred under different circumstances and assists them to confirm the validity of the findings.

23. Rationale (Cont.) Triangulation does not only ensure validity but places the responsibility with the researcher for the construction of plausible explanations about the phenomena being studied. Mathison (1988) mentions three outcomes that might result from a triangulation strategy i.e. Convergence: data from different sources, methods, investigators, and so on will provide evidence that will result in a single proposition about some social phenomenon. Inconsistency: When multiple sources, methods, and so on are employed we frequently are faced with a range of perspectives or data that do not confirm a single proposition about a social phenomenon. Contradiction: When we have employed several methods we are sometimes left with a data bank that results in opposing views of the social phenomenon being studied. Triangulation therefore, is in support of the complementary theorist who carefully considers the outcome in logical sequence and evaluates whether differences in conclusions can co-occur ( DiLoreto and Gaines: 2016)