The Hawthorne effect is where when measuring the effect of lighting changes on worker output in an electrical factory any change increased output, even back to the original lighting conditions. I've heard this explained as running the experiment caused the employees to be observed more closely which led them to work harder, and used as a rationale for observing employees more. Except the Hawthorne effect is a myth.

The economists Steven D. Levitt (of Freakonomics fame) and John List have a paper Was there really a Hawthorne Effect at the Hawthorne Plant?. They chased the original data from the study and found that it doesn't at all match the popular story that any change increased output. They found that there may have been a small increase in productivity during weeks they performed experimental changes (a few percent, on the order of difference in productivity between Mondays and Tuesdays). But there's also an open question of what else changed in those weeks.

But the myth lives on; even though this paper was written in 2009 I've heard this effect referenced as late as last year. The New York Times article, Scientific Myths That Are Too Good to Die, talks about another aspect of the Hawthorne experiement where any changes in break time (longer, shorter, more, fewer) increased productivity. But in that study there were only five workers, and two were replaced during the experiment for low output. It explains other similar myths.

How do we avoid these myths and tell fact from fiction? There's a whole heap of popular psychological effects that couldn't be replicated. Things like ego depletion, that smiling makes people feel happier, or the Lady Macbeth effect (that shame makes people want to clean) failed to replicate. But you still hear these things around.

Doing good research is difficult, especially when it comes to people. Communicating that research and pulling it apart from stories that sound good but have no evidence is actually really hard for people. It's easy to just cherry pick studies that support your current point of view.

The Hawthorne effect is just one of many examples of stories that sound good, but have no factual basis.