Overview
Experimental results are systematically misinterpreted through two opposing errors: over-claiming (treating a p<0.05 result as proof of a large effect) and under-claiming (dismissing a non-significant result as proving no effect). Statistical significance indicates whether an effect is distinguishable from zero given the sample size — it says nothing about whether the effect is large enough to matter commercially or scientifically. Effect size is the metric that determines whether a statistically significant result is practically significant.
The Experimental Results Analysis Framework combines significance testing with effect size interpretation, diagnoses validity threats that may have compromised the causal inference, and produces findings statements that accurately describe what the data supports and what requires further investigation.