Overview
A/B test reports fail in two directions: they report statistical significance without business significance (a 0.3% lift is statistically significant at N=500,000 but commercially irrelevant), or they report business significance without statistical validity (a 15% lift from a 3-day test with N=200 is noise, not signal). Both failures lead to wrong decisions — shipping changes that don't work, or killing changes that do.
The right A/B test report answers one question: should we ship this? The answer requires three things: statistical validity (was the test designed and executed correctly), statistical significance (is the result unlikely to be noise), and business significance (is the effect large enough to matter). All three must be true to ship. Any one failing is a reason not to.
The A/B Test Results Report Prompt generates a complete experiment results report: test validity assessment, statistical analysis with effect size and confidence intervals, business impact translation, decision recommendation with explicit confidence level, and a follow-up specification for inconclusive results.
What you get: - Test validity assessment (pre-analysis checks) - Statistical analysis with effect size and confidence intervals - Business impact quantification in revenue/conversion terms - Decision recommendation with explicit confidence level - Follow-up specification for inconclusive results
Built for: analysts, product managers, and data scientists who need to communicate experiment results to decision-makers who will act on the recommendation.