What bias is, and five biases in research
A bias is a filter between the world and our judgement of it.
Experience, expectation and preference shape what we notice, what we remember and how we interpret it.
Psychology aims at objectivity, but most of its data (ratings, interviews, observations) are gathered and interpreted by people, so the filter never fully disappears.
Bias can be explicit, when a person is aware of a preference and could report it, or implicit, when it operates without awareness.
A researcher who believes they are neutral can still code ambiguous behaviour towards their hypothesis, which is why controls are designed in.
Five biases in the research process
Naming a bias earns little on its own.
What matters is where it enters the study and what it does to the conclusion.
- Researcher bias (running, interpreting): the researcher's expectations change how participants are treated or how data are scored.
- Participant bias (running): people change their behaviour because they are studied, through demand characteristics or social desirability.
- Sampling bias (recruiting): the sample differs from the target population, so findings generalise less than claimed.
- Confirmation bias (analysing, interpreting): evidence that fits the hypothesis is trusted; evidence against it is explained away.
- Publication bias (after the study): significant results are published more often than null ones, so the literature overstates effects.
Two real studies show how serious this is.
Rosenthal and Fode (1963) gave students rats labelled "maze-bright" or "maze-dull", although the labels were assigned at random.
The "bright" rats were recorded as learning the maze faster, an effect produced by the students' expectations.
For publication bias, Turner and colleagues (2008) compared antidepressant trials registered with the US Food and Drug Administration with the published record.
About half the registered trials had positive results, but around 94% of the published trials appeared positive, because negative trials often went unpublished or were written up as positive.
Sampling bias runs through the discipline: Henrich, Heine and Norenzayan (2010) showed that most participants in leading journals come from WEIRD (Western, educated, industrialised, rich, democratic) societies.
- Bias: a systematic distortion in thinking or in research that pushes judgements or results away from an accurate picture.
- Implicit bias: a bias that operates without the person being aware of it.
- Demand characteristics: cues in a study that lead participants to guess its aim and change their behaviour to fit, or to spoil, it.
- Publication bias: the tendency for studies with significant results to be published more often than studies with null results.
A researcher who runs a mindfulness club believes mindfulness improves concentration.
She recruits 30 club members and 30 non-members from her school, gives both groups a concentration task, and scores the answers herself, knowing who is in which group.
The club members score higher.
State two biases and explain how each affects the conclusion.
(Invented study.)
1. Sampling bias: club members chose mindfulness, so they may differ in motivation or interest. The groups were not formed at random, so the difference may not be caused by mindfulness.
2. Researcher bias: she knows each participant's group and expects the club to do better. Ambiguous answers may be scored more generously for club members, inflating the difference.
3. Link back: both biases work in the direction of the hypothesis, so the result overstates any real effect.
Answer: sampling bias (self-selected groups) and researcher bias (unblinded scoring) each offer an alternative explanation for the higher scores, so the study cannot show that mindfulness caused them.
For every bias you name, write one sentence on its consequence: which result it inflates or hides, and why that weakens the conclusion.
A named bias with no consequence reads as a list, and concept answers are rewarded for analysis.
- Treating bias as deliberate cheating. Most bias is unintentional, which is why controls are designed in instead of being left to honesty.
- Confusing participant bias with sampling bias. Participant bias is about how people behave in the study; sampling bias is about who was recruited.
- Treating publication bias as a flaw in one study. It is a property of the published literature as a whole.
Check your understanding
1. State what is meant by bias, and the difference between implicit and explicit bias. [2]
2. Explain how demand characteristics could affect the results of a study on attitudes to recycling. [3]
3. Explain why publication bias can mislead a student writing a literature review. [3]