But all too depressingly often I run into studies that end up revealing the failings of the social sciences themselves.
Case in point: A Chronicle Review article mentioned an incredible story about how easily duped we are by authority -
"In the early 1970s, a group of medical researchers decided to study an unusual question. How would a medical audience respond to a lecture that was completely devoid of content, yet delivered with authority by a convincing phony? To find out, the authors hired a distinguished-looking actor and gave him the name Dr. Myron L. Fox. They fabricated an impressive CV for Dr. Fox and billed him as an expert in mathematics and human behavior. Finally, they provided him with a fake lecture composed largely of impressive-sounding gibberish, and had him deliver the lecture wearing a white coat to three medical audiences under the title "Mathematical Game Theory as Applied to Physician Education." At the end of the lecture, the audience members filled out a questionnaire.
"The responses were overwhelmingly positive. The audience members described Dr. Fox as "extremely articulate" and "captivating." One said he delivered "a very dramatic presentation." After one lecture, 90 percent of the audience members said they had found the lecture by Dr. Fox "stimulating." Over all, almost every member of every audience loved Dr. Fox's lecture, despite the fact that, as the authors write, it was delivered by an actor "programmed to teach charismatically and nonsubstantively on a topic about which he knew nothing."'
Pretty stunning proof that we will blindly believe an "authority" even when he speaks utter nonsense, yes?
For one, the study broke the cardinal rule of scientific study; that is, it didn't have a "control group." In other words, the study looked only at how an audience judged a good lecturer delivering a nonsensical lecture; it did not also look at how a similar audience would judge a good lecturer delivering a sensible lecture. Doing so, the experimenters may have found that the good feedback the phony lecture received was not so good when compared to a non-gibberish lecture.
The other holes hint that that may be the case. First, the feedback forms ask mostly about the style of the presentation, not the content. For some reason the researchers didn't ask "Did the lecturer make a clear (or convincing) argument?" If they had, the experimenters may have found a big drop in positive feedback.
Second, the responders free-form comments suggest that quite a few weren't buying the lecture. Most of the positive comments were, again, about the style of Dr. Fox. But when it comes to content, the responses show signs of bafflement: "Too intellectual a presentation. My orientation is more pragmatic. Did not carry it far enough. Lack of visual materials to relate it to psychiatry. Left out relevant examples. He misses the last few phrases which I believe would have tied together his ideas for me. Interesting, wish he dwelled more on background. Somewhat disorganized. Unorganized and ineffective." (There was one positive comment on content: "Good analysis of subject that has been personally studied before." I think we've all met this delusional guy before.)
But, you say, what can explain the good feedback this impostor got, if people did not actually believe he was legit? A likely explanation was that they were being polite or hiding their confusion. Think about it: how often have you sat through a talk that you didn't understand and then written on the feedback form either "I had no idea what he was talking about" or "That lecture was bullshit." You probably either left the comment section blank or politely alluded to its inanity, saying something like "My orientation is more pragmatic."
Unfortunately, we won't ever have the true explanation because, deep sigh, this potentially brilliant study didn't ask the right questions in the right conditions.
No comments:
Post a Comment