Around four long years ago, my concluding remarks about the “The Grandmother Problem“, referred to “some real and very valuable lessons to be learnt” that I decided, for reasons best known to myself, to leave unsaid.
Having reflected (slowly, as it turns out) on these things I thought it might be useful to offer-up a few Methods-related pointers as to what some of these lessons might be.
Correlations: First and probably most obviously, there’s a lesson about how to demonstrate correlation is not causation. While the Grandmother Problem addresses this tangentially, the stuff about a “positive correlation between college attendance and divorce rates” probably rams this idea home with a bit more force. There are, however, quite a few further examples of correlation not equalling causation dotted throughout the post.
Data Validity: A bit less obvious, perhaps, is the warning that just because we collect data directly “from the horse’s mouth“, so to speak, doesn’t automatically make it valid. We shouldn’t, in other words, simply accept what people claim uncritically, just because they say it in a questionnaire or interview. The lesson here is that when collecting data directly from people we need to be aware of their agency. That is, we need to be aware of the motives they may have, innocent or otherwise, for providing the information they choose.
Sociology and Science: Thirdly, one of the characteristics of everyday thinking about the world is that commonsense explanations develop out of observing “facts” and then developing some kind of “theory” that explains them. While this is actually okay as far as it goes, one major problem is that without rigorous scientific testing of the theory we have no-way of differentiating between competing theories if both seem to adequately “explain the facts”, something the Grandmother Problem illustrates quite clearly (and which is also a general feature of Conspiracy Theories, many of which are far more pernicious than our Dead Grandmother issue ).
A related problem, again illustrated by the Grandmother Problem, is that this way of constructing theories treats “the facts” uncritically. It fails to ask if they really are “facts” in the first place.
And if they’re not, what happens to our carefully constructed theories that “explain the facts”? Is it simply a case of Garbage In, Garbage Out?
In this respect you can use the Grandmother Problem to introduce and illustrate a different kind of scientific method, such as the Hypothetico-Deductive Model that begins with an hypothesis and tests it against the facts.
And if you’re thinking it would be helpful to develop their knowledge of the Hypothetico-Deductive Model in the context of a role-playing simulation?
You’re welcome.