Hypothesis Testing Lab

The experiment

Each participant was asked: “Is the percentage of UN member countries that are in Africa higher or lower than X?” and then for their best estimate. A computer set X to 10 or 65 at random. The true answer is 54 of 193 countries, about 28%. The number carries no information. Did it move the answers anyway?

Effect of the anchor

Working, step by step

Where the observed statistic falls

Balance check: did randomisation work?

Gender was not used to assign the arms, so the share of women should be similar in both. This is what random assignment buys: the arms are alike in everything except the number shown, so any difference in estimates is caused by the anchor.

Does the effect replicate every year?

Anchoring effect in each year's class, using all responses, with 95% CIs.

What happens across many samples?

Click “Repeat 500 times” to rerun the experiment on 500 fresh samples with the current settings. The share that come out significant is the power at this sample size.

Power and sample size

Effect to detect, δ (points)

Standard deviation, σ (points)

Significance level, α

Target power

Sample size per arm, n

Hypothesis Testing Lab

Test result

Working, step by step

Where the observed statistic falls

What happens across many samples?

Can we say gender causes this?

The experiment

Effect of the anchor

Working, step by step

Where the observed statistic falls

Balance check: did randomisation work?

Does the effect replicate every year?

What happens across many samples?

Power and sample size

Practice set