RELIABILITY ( Sociology Optional)

EN

Introduction

• Reliability refers to the capacity of measurement to produce consistent results.
• Thus, a method is reliable if it produces the same results whenever it is repeated, and is not sensitive to the researcher, the research conditions or the respondents in reliability.
• Internal reliability means consistency of results within the site; and that data are plausible within that site.
• External reliability refers to consistency and reliability of data across sites.
• The purpose of reliability testing is to ensure that the instruments in question are robust and not sensitive to changes of the researcher, the respondent or the research condition.
• Reliability is concerned with objectivity, accuracy, precision, consistency and stability. These criteria are employed as in validity.

PYQs: Reliability and Validity

• Utility of Reliability and Validity in Social Research. (03/20)
• सामाजिक अनुसंधान में विश्वसनीयता और वैधता की उपयोगिता। (03/20)

• Write short note on Reliability and Validity, keeping sociological perspective in view. In about 150 words. (11/12)
• समाजशास्त्रीय परिप्रेक्ष्य को ध्यान में रखते हुए विश्वसनीयता और वैधता पर संक्षिप्त टिप्पणी लिखें। (11/12)

• How can one resolve the issue of reliability and validity in the context of sociological research on inequality? (17/10)
• असमानता पर समाजशास्त्रीय अनुसंधान के संदर्भ में विश्वसनीयता और वैधता के मुद्दे को कैसे हल किया जा सकता है? (17/10)

• What is reliability? Explain the different tests available to social science researcher to establish reliability. (2022/10)
• विश्वसनीयता क्या है? इसे स्थापित करने के लिए सामाजिक विज्ञान शोधकर्ता के लिए उपलब्ध विभिन्न परीक्षणों की व्याख्या कीजिए। (2022/10)

Thinkers’ Views

• Reliability is consistency of measurement. Reliability is that part of a measure that is free of purely random error. – Bollen, 1990.
• Reliability is a major concern when a psychological test is used to measure some attribute or behavior. - Rosenthal and Rosnow, 1991.

Reliability in quantitative research

Three types of reliability are:

• Stability reliability: It relates to reliability across time. Here the question is whether a measure produces reliable findings if it is employed at different points in time.
• Representative reliability: It relates to reliability across groups of subjects. The question here is whether the measure will be reliable if employed in different groups.
• Equivalence reliability: It relates to reliability across indicators. The question will the measure in question produce consistent results across indicators?

The most common methods are:

• Test-retest method: The same subjects are tested and retested with the same instrument. If the same results are obtained the instrument is reliable.
• Split-half method: Responses to the items of an instrument are divided into two groups (e.g. odd/even questions), and the scores are correlated. The type and degree of correlation indicate the degree of reliability of the measurement.
• Inter-item test and item-scale test: These correlations indicate the degree of reliability of the instrument.
• Alternate-form reliability: Reliability is tested by administering two similar instruments in one session. It is assessed by the degree of correlation between the scores of the two groups.

These tests are regularly used and entail a considerable amount of statistical analysis and interpretation. Instruments are tested before they are put to use, and the results are normally disclosed to the academic community every time these instruments are referred to and their findings published. The advent of computer-based statistical analysis has made this task easier, more accurate and more enjoyable than before.

Reliability in qualitative research

• Qualitative researchers give little attention to reliability in the way that quantitative ones do. Objectivity is replaced by confirmability. - Cuba and Lincoln.
• Quantitative researchers are thought to:
  • Control the environment.
  • Employ high levels of measurement and standardization.
  • Restrict the researcher-researched relationship.
  • Create artificial situations which are different from those they intend to study.
  • Alienate the researcher from the research environment, which is counterproductive.
• The quality of qualitative research is assessed in more general terms than that of quantitative research.
• Both aspects of reliability, the internal and the external, are considered in qualitative research.
• Drew and associates (1996) suggest that the following steps to achieve internal reliability:

1. Use low inference descriptors.
2. Use multiple researchers whenever possible.
3. Create a careful audit trail.
4. Use mechanical recording devices where possible (and with permission)
5. Use participant researchers or informants to check the accuracy or congruence of descriptions.

• Regardless of the significance of these approaches to reliability, a number of writers argue that qualitative research does not provide as high a degree of reliability as quantitative research.

Validity and reliability

• Validity and reliability are both quality measures of research instruments, and although they are quite different in their nature and purpose, some students find it difficult to distinguish between the two.
• Messick (1989) transformed the traditional definition of validity, with reliability in opposition to validity, to reliability becoming unified with validity.
• The validity and reliability of a measure are closely interrelated. Reliability without validity is of little use.
• Even the most reliable instrument is useless if it is not valid. A reliable instrument is not necessarily valid. Even the most reliable instrument can be invalid.
• For instance, the scales that show the student's weight is exactly 65 kg every time the student steps on them are of no value. Hence, it is useful to measure and interpret reliability results together with validity scores.