Common reporting errors in subgroup analysis: a comparison of interaction and stratified regression models.

backgroundIn public health, epidemiology and medical research, it is considered a relatively effective practice to report treatment effect estimates from stratified regressions alongside significance test results from interaction regressions. However, since the two approaches differ in their theoretical foundations, statistical properties, interpretability, and conditions for appropriate application, combining them in reporting may raise methodological concerns. This study aims to evaluate and compare the effectiveness of these two methods in estimating heterogeneous treatment effects using Monte Carlo simulations.

methodsWe conducted a series of Monte Carlo simulations to compare the performance of interaction regression and stratified regression across various scenarios. These scenarios varied in total sample size, the distribution of subgroup proportions, heterogeneity in the associations between covariates and the outcome variable across subgroups, and the degree of correlation among covariates. We simulated 10 covariates and used a generalised linear model to generate the outcome variable. The performance of both methods was evaluated using mean squared error (MSE), empirical coverage rate (ECR), and empirical statistical power (ESP).

resultsThe results demonstrated that stratified regression based on bootstrap resampling effectively controlled Type I error under large sample sizes and balanced group proportions. Interaction regression struggled to control Type I error when the coefficients of baseline characteristics differed across groups, and covariates were only weakly correlated. In other scenarios, the interaction regression outperforms stratified regression, primarily due to its relatively higher statistical power while maintaining acceptable Type I error rates. As an illustrative application, the two methods were applied to real-world data from the International Social Survey Programme (ISSP) to demonstrate how interaction regression and stratified regression may yield different conclusions regarding the association between fruit or vegetable consumption and body mass index.

conclusionThis study highlights the strengths and limitations of interaction regression and stratified regression in estimating heterogeneous treatment effects. Researchers should avoid adopting a hybrid strategy for reporting subgroup analysis results, such as presenting treatment effect estimates from stratified regressions and significance tests from interaction regressions, because the two methods may exhibit substantial differences in Type I error control under certain conditions.