search for

CrossRef (0)
A sample size calibration approach for the p-value problem in huge samples
Commun. Stat. Appl. Methods, CSAM 2018;25:545-557
Published online September 30, 2018
© 2018 Korean Statistical Society.

Yousung Parka, Saebom Jeonb, Tae Yeon Kwon1,c

aDepartment of Statistics, Korea University, Korea;
bDepartment of Marketing Information Consulting, Mokwon University, Korea;
cDepartment of International Finance, Hankuk University of Foreign Studies University, Korea
Correspondence to: Department of International Finance, Hankuk University of Foreign Studies, 81 Oedae-ro, Mohyeon-eup, Cheoin-gu, Yongin-si, Gyeonggi-do 17035, Korea. E-mail: tykwon@hufs.ac.kr
Received May 8, 2018; Revised June 15, 2018; Accepted June 25, 2018.
The inclusion of covariates in the model often affects not only the estimates of meaningful variables of interest but also its statistical significance. Such gap between statistical and subject-matter significance is a critical issue in huge sample studies. A popular huge sample study, the sample cohort data from Korean National Health Insurance Service, showed such gap of significance in the inference for the effect of obesity on cause of mortality, requiring careful consideration. In this regard, this paper proposes a sample size calibration method based on a Monte Carlo t (or z)-test approach without Monte Carlo simulation, and also proposes a test procedure for subject-matter significance using this calibration method in order to complement the deflated p-value in the huge sample size. Our calibration method shows no subject-matter significance of the obesity paradox regardless of race, sex, and age groups, unlike traditional statistical suggestions based on p-values.
Keywords : huge sample, p-value problem, subject-matter significance, Monte Carlo, sample size calibration