Fair enough. So how hard is it to double word scores in a small group? How many words were they given? What was the interaction like between audiologist and patient? What were the circumstances? What were the expectations?
Patient: "You are one of the first people on Earth to receive a potentially revolutionary drug that will, for the first time in history, restore hearing!"
Audiologist: "We are building a $1b company with some of the most prestigious scientists in the world to achieve the holy grail of regenerative medicine and alleviate the suffering of millions of people, but we do have this little hurdle to overcome first...."
I don't know about you, but I think I'd be all sorts of focused on my follow-up exam.
I'll just provide one more thing to think about and I've outlined this in detail in the past. In particular in
this post, I talk about the game theory, from the perspective of the participant.
The way I see it, and why I'm not overly concerned with the bias either way, is that there's a balancing effect going on that helps regulate the stuff you talked about.
A test is a quick, 50 word thing, but think about all of the time outside of the testing. I am living with this drug/placebo. I probably have an idea as to which one I think I got.
Say I think I got the placebo. Then my motivation is actually to
not score well because by scoring well, I am helping the placebos and reducing the likelihood of me eventually receiving the drug. In other words, the person who goes in gunning for a top score has to believe they got the drug.
Now, granted, there could be a real placebo effect in that they guessed wrong. But given the 3:1 sampling ratio, their likelihood of guessing wrong is a lot lower than in a 1:1 situation. Hence, you are right that there could be more incentive for people who think/did get the drug compared to people who think they didn't, but want to see the drug pass so they can eventually take it.
This is why I argued that the motivation could even favor the treatment group. This is, of course, bad for thee Phase 2 results, as it paints them as even worse on the groupwide level. However, it's confusing at the individual level because if someone is "trying harder" or "studying harder" that also can be correlated with seeing their every day lives improve and want to show it off. Hence, these superstar performances can be a combo effect.
But think about it. If the person doesn't think their life improved at all, why would they want to perform well on the test? Why would the audiologists be "pressuring" them if they are blinded and they could be pressuring a placebo patient, hurting the company? I don't buy this conspiratorial stuff, as it doesn't make sense for a blinded, placebo-controlled study.
Occam's razor is that the drug just isn't that good at a groupwide level, with some individual responders that will play no role in the approval of the drug, but hopefully progress the science in a positive way.