What is a test hypothesis?
A test hypothesis is a specific, testable prediction about how a change will affect a metric. It's the core input to any A/B test — without a clear hypothesis, you're not running an experiment, you're just making changes. A strong test hypothesis states what you're changing, what effect you expect, and why you expect it.
The If / Then / Because structure
The cleanest hypothesis format experienced CRO practitioners use:
If [we change X], then [metric Y will move in direction Z], because [user-behavior or psychology reason].
The "because" is the critical part. It forces you to articulate why the change should work, not just what you're doing. A hypothesis without a "because" is a guess.
Example hypotheses
Five well-formed test hypotheses from real SaaS scenarios:
- Pricing page: If we reduce pricing tiers from 4 to 3, then trial signups will increase 10%+, because fewer options reduce choice paralysis.
- Signup form: If we remove the phone number field, then signup completion will rise 15%+, because each optional field adds friction.
- Homepage hero: If we replace the abstract illustration with a product screenshot, then click-through to pricing will increase, because product visuals communicate the offer faster than metaphors.
- Checkout: If we add PayPal as a payment option, then checkout completion will rise 8%+, because mobile users prefer one-tap checkout.
- Onboarding: If we make the first onboarding step completable in under 60 seconds, then day-7 activation will increase 20%+, because users who experience value fast come back.
How to build a strong test hypothesis
- Start with data or observation. "Session recordings show 40% of users abandon at the phone-number field" is a better trigger than "I think the form is too long."
- Specify the change. "Remove the phone number field" beats "Simplify the form."
- Predict direction and magnitude. "Signup rate will increase 15%" forces you to pre-commit to what success means.
- State the mechanism. The "because" connects your change to a behavioral principle — friction, trust, cognitive load, social proof.
- Make it falsifiable. If no outcome would change your mind, it's not a hypothesis — it's a belief.
Test hypothesis vs null hypothesis
These are related but distinct:
- Test hypothesis: Your prediction, for example "Removing the phone field will increase signups 15%."
- Null hypothesis (H₀): The statistical default — "There is no difference between control and variation."
- Alternative hypothesis (H₁): The formal counterpart to H₀ — "There is a difference."
Your test hypothesis informs H₁; the experiment tests whether you can reject H₀.
Common mistakes
- Kitchen-sink hypotheses. "If we redesign the pricing page, signups will go up" is too vague — you can't learn what caused the lift.
- No direction stated. "The button color will affect signups" isn't testable — any change satisfies it.
- Unmeasurable outcomes. "Users will feel more confident" can't be measured with an A/B test.
- Testing without a prediction. Some teams skip the hypothesis and "just try stuff." The result: random wins they can't replicate.
Related concepts
Hypothesis, null hypothesis, statistical significance, sample size, A/B testing, multivariate testing.
