Do you remember the 100 Acre Wood? In A.A. Milne’s first book, Winnie the Pooh, Pooh Bear and Piglet go hunting a woozle. They follow the woozle’s tracks, and over time find that there are more and more tracks. Christopher Robin eventually explains that Pooh and Piglet have been following their own tracks.
Over time, the woozle effect has become a term that means evidence based on citation, when the evidence in the citation is weak. (In rhetoric, a citation is usually considered an appeal to authority). Over time, we see a citation of a study so many times, that it leads people to believe a fact when it actually lacks evidence. It is a claim not supported by original findings. Often this happens in the popular press, or on the internet where a study is cited to support an argument, yet the original study lacks evidence or has flawed methodology, lack precise definitions, or misrepresent the conclusions.
The mainstream press is often to blame for woozles – you can spot the woozle effect if the languages has changed from “the evidence suggests” to “the evidence shows”, suggesting a firm link between evidence and conclusions. You can sometimes see woozles in popular books, such as Malcolm Gladwell’s book Outliers, where he quotes the famous 10,000 hours to expertise. (In fact, the research on deliberate practice is very mixed, and suggests that practice alone will not create expertise).
For example, I just googled the term “crayola scent study” and found hundreds of references to a Yale University study of scents that showed that crayola is the 18th most recognizable scent to American adults. But when I searched the databases, I could not find this study anywhere. So it is difficult to tell if the study was reliable and valid. At this point, the die is set – we all now believe that crayola scent has strong memory association for American adults. But is this true? If you are making a decision on product development, you might be tempted to place a lot of emphasis on product scent based on this research. But is this a good choice?
Why does all of this matter? When we are making decisions we need to use reliable information. The only way to determine if the information is reliable is to read the original studies, not to rely upon the newspaper or magazine’s interpretation of these studies. We need to ask all the right questions – does the study use a flawed sample? Use a very small sample? Cite original research? Interpret correlation as causation? If the evidence is weak, the decision could be too. So be on the look out for the Woozle Effect.