You probably saw that study that came out by the Innovation Center for US Dairy a few weeks ago: Seven percent of American adults believe that brown cows lactate chocolate milk. Seven percent! Humiliating! Embarrassing! Another strike against us ignorant Americans and our scientific ignorance!
According to an online survey, 7% of adult Americans believe that chocolate milk comes from brown cows. Now that's "Sad!"
— Joe Schwarcz (@joeschwarcz) July 6, 2017
You may have also noticed that there’s no link above to the survey. There’s a reason for that: The survey has not been made public. So while we can learn interesting tidbits like 29 percent of Americans use their kids as an excuse to buy chocolate milk but then drink it all themselves, or that 95 percent of Americans have some kind of cheese in their fridge, we can’t actually know what the questions were that gave us those results.
This really matters. Take, for instance, this survey question:
“Does chocolate milk flow directly from the udders of brown cows? Answer yes or no.” In a clear-cut question like that, I would be sad indeed if seven percent of Americans – roughly 16.5 million – said yes.
But the question, it seems, may not have been quite so clear cut. In an NPR interview with the president of the National Dairy Council, Jean Ragalie-Carr, we get a hint at what the question actually was. In the interview, she said: “Well, there was brown cows or black and white cows, or they didn’t know.”
Does @netflix really have to explain to me what thumbs up and down mean? We think chocolate milk comes from brown cows so maybe so
— Casey Reid (@CReid827) June 28, 2017
This makes the survey look quite a bit different. The general population might not know if dairy cows are black and white or brown or both—or whether the milk we drink can come from either.
And what about the phrasing? Say the question is: “Does chocolate milk come from: (1) brown cows; (2) black and white cows; or (3) I don’t know.” The chocolate milk we drink may have come from a black and white jersey cow before having some delicious Hershey’s syrup stirred into it. It may have come from a brown Jersey cow with some fancy Ghirardelli scooped in. In both of these cases, the chocolate milk would have “come from” one of those color cows – there would just be an extra step mixed in (so to speak).
There is another statistic about chocolate milk in this survey that is not so reported on: An additional 48 percent of respondents did not know where chocolate milk comes from. This answer provides further credence to the theory that there was a problem with validity with the survey—that is, the survey did not measure what it was supposed to measure. Surely it cannot be that a full half of Americans don’t understand that chocolate milk is simply milk with chocolate sauce stirred in?
Without seeing the questions, it’s hard to know for sure. But it seems likely that the survey question was confusing, not that the general American public does not know whether chocolate milk can be squeezed from the udder of a cow. Or if, on a chilly night, chocolate ice cream might come out instead.
A statement by Lisa McComb, senior vice president of communications for Dairy Management, Inc., confirms this. “The purpose of the survey was to gauge some interesting and fun facts about consumers’ perceptions of dairy, not a scientific or academic study intended to be published,” she said.
While this might give us a collective moment of relief, the survey unintentionally revealed a larger problem, as explored thoughtfully in an article by Lauren Griffin and Troy Campbell of The Conversation. Why did the media take the results of this unscientific survey and send it out to the world as fact? The authors argue that the survey was jumped on because it feeds a popular narrative—that Americans are ignorant and scientifically imbecilic. Moreover, they argue, as news stories quickly become viral, these problems are likely to continue.
In the end, we just do not know whether there are millions of American adults out there who do not know if white cows give white milk and brown cows give chocolate milk. What we do know is that before we come to conclusions on that, we need to see the survey.