Hypothesis Testing
Scientific hypothesis formulation
Equivalence & Non-Inferiority Testing Framework
Design and interpret equivalence and non-inferiority tests — defining the smallest effect of practical importance as the equivalence bound, applying TOST (Two One-Sided Tests), and producing conclusions that establish active equivalence rather than the mere absence of significant difference.
Statistical Power Analysis Framework
Conduct a priori statistical power analyses — calculating required sample sizes for specific tests and effect sizes, justifying the minimum meaningful effect size from domain knowledge rather than arbitrary convention, and designing studies that can detect effects worth detecting at the desired power level.
Meta-Analysis & Systematic Review Framework
Design and conduct a meta-analysis — defining the inclusion criteria, extracting and standardizing effect sizes across studies, applying random vs. fixed effects models based on heterogeneity, testing for publication bias, and interpreting pooled estimates with explicit heterogeneity acknowledgment.
Non-Parametric Hypothesis Testing Framework
Apply non-parametric hypothesis tests when parametric assumptions fail — selecting between Mann-Whitney U, Wilcoxon signed-rank, Kruskal-Wallis, and Friedman tests based on design, calculating rank-biserial correlation for effect size, and interpreting results without the normality assumption that parametric tests require.
Bayesian Hypothesis Testing Framework
Apply Bayesian hypothesis testing — calculating Bayes Factors to quantify evidence strength for or against a hypothesis, specifying priors from domain knowledge, and interpreting posterior distributions in terms that communicate continuous probability rather than the binary reject/fail-to-reject of frequentist testing.
Regression Hypothesis Testing Framework
Test hypotheses using regression analysis — verifying the four key regression assumptions, interpreting standardized beta coefficients and model fit, conducting incremental F-tests for model comparison, and distinguishing predictive models from causal claims.
Correlation Analysis Framework
Test and interpret correlations between variables — selecting Pearson vs. Spearman vs. Kendall based on data characteristics, reporting effect size alongside significance, visualizing the relationship before testing, and explicitly addressing the correlation-causation distinction in the interpretation.
Chi-Square & Categorical Data Testing Framework
Test relationships in categorical data — selecting between chi-square goodness-of-fit, chi-square independence, and Fisher's exact test based on sample size and design, calculating Cramér's V effect size, and interpreting residuals to identify which specific cells drive a significant relationship.
ANOVA & Multiple Group Comparison Framework
Design and interpret ANOVA tests — applying the correct ANOVA variant (one-way, two-way, repeated measures), conducting valid post-hoc comparisons with appropriate corrections, and reporting effect sizes that communicate variance explained beyond statistical significance.
Null Hypothesis Design & Testing Framework
Design and test null hypotheses — formulating testable H0 and H1 pairs, selecting the correct statistical test, calculating required sample size for adequate power, and interpreting results with correct error probability language that avoids the "proves" and "disproves" mischaracterizations.