Overview
Most email A/B tests are not experiments — they are hunches dressed in data. Two emails go out, one performs better, and the team declares a winner. But without a pre-registered hypothesis, adequate sample size, statistical significance threshold, or a protocol for what the result means, the "winner" is noise. The next test contradicts it, the next one contradicts that, and nothing is ever learned.
The Email A/B Test Design Framework builds a valid experiment: a single variable, a falsifiable hypothesis, a minimum sample size based on the effect you need to detect, and decision rules that produce a true learning — not just a number.
What you get: - Variable selection: the single element to test and why that variable over others - Falsifiable hypothesis: what you expect to change, by how much, and why - Sample size calculation: the minimum list size required for the result to be trustworthy - Test structure: treatment vs. control, send protocol, isolation requirements - Measurement protocol: the metric that answers the hypothesis (not all available metrics) - Decision rules: what each result means and what action it triggers - Learning documentation: how to record the result so it informs future tests
Built for: email marketers, growth teams, and CRM managers who run A/B tests and want results they can actually use.