Unit II. Research Methods
Reliability and Validity
ThemeUnderstanding the scientific basis of sociology
DescriptionA brief class exercise to illustrate the concepts of reliability and validity.
Learning GoalsTo help students distinguish between reliability andBack to top
validity and remember the significance of that distinction.
Things Needed5-10 minutes of class time.
A bathroom scale.
ActionsUsually students are capable of memorizing the concepts of reliability and validity. Often, however, they do not truly understand the meanings of these concepts and, as a result, they confuse them. This brief illustration helps lead to a discussion of the concepts which enable them to grasp the significance of both reliability and validity.Back to top
After noting that the students have read about reliability and validity for
that day's class, I ask for volunteers to explain the difference between them.
After we review the correct definitions (given either from the volunteers or
with help from others in the class) I note that students often confuse these concepts and that I will give them an example to help illustrate both
I then explain that, like many people, I do not fit into my clothes the way
that I used to. It seems that my clothes have been shrinking. This has
particularly become a problem during the last year. At this point I produce a bathroom scale and place it on the floor where everyone can see. I step on the scale and say, "Every morning I get on my scale, I look down and I see that I weigh ___pounds." (Inserting humor into the lecture is always helpful, so I usually put my hand over my mouth each time I repeat my weight, and mumble the amount so that it cannot be heard.) I go on to say: "This bathroom scale is a measurement instrument. Just like the questionnaires you have read about in the book, it measures something--in this case only one variable--rny weight. This particular measurement instrument is reliable. When applied to the same situation, it produces the same result. Thus, when I step on the scale tomorrow, I will still weigh pounds. (Assuming that I have not lost or gained any weight in the twenty-four hours since I last stepped on the scale.) This scale is not, however, a valid measurement instrument. It does not give a true measurement of the variable it purports to measure."
At this point I step off the scale, pick it up, and play with the dial which
allows a person to adjust the weight on the scale. I go on to say: "Because of the fact that my clothes have been shrinking (and my weight is now higher than I would like) I used this little dial last week to make the scale read five pounds lighter. Have any of you ever done that? Thus, this scale is still a reliable measure. It is not, however, a valid measure. It yields the same results, but they are not the correct results."
After this brief illustration, I then turn to the class as a whole and ask them for examples from social science data where a measure might be reliable, but not valid. I am always prepared with some examples of my own to back up illustrations from the class. A particularly useful illustration: A questionnaire might ask a parent how many times he or she severely beat his/her child in the last week. The responses would likely be quite reliable--consistent responses of 0. They might not, however, be valid.
More InformationThe discussion which follows the illustration can point out that the bathroom Bcale example is a useful reminder of the distinction between reliability and validity. Problems in measures of social science variables are seldom so clear cut. Carefully examining both reliability and validity of measures is thus even more important.Back to top
It is possible that students will remember the illustration but it will not help them remember which concept is reliability and which is validity. I have not found this to be the case, however, as long as I repeat the distinction at least three times while I am using the bathroom scale. Improvement in understanding of these concepts has been reflected in significant increases in the proportion of the class who can write essay questions on examinations explaining and illustrating the differences between these two concepts.
Creator/SourceEdward L. Kain, Department of Sociology and Anthropology, Southwestern University, Georgetown, TX 78626. From Kain, Edward L. and Robin Neas. 1993. Innovative Techniques for Teaching Sociological Concepts. Washington, DC: American Sociological Association, pp. 75-76.