Predicting medical cost for diabetes patients, then turning the model into a savings calculator.
Five years of Korea Health Panel data, a panel-data Random Effects regression on log-cost, careful variable selection across four derived specifications, validated with multicollinearity and autocorrelation diagnostics. The result was a 19-coefficient interpretable model whose actionable signal (exercise reduces cost, smoking compounds it) became the basis for a member-facing lifestyle savings calculator.
Why diabetes, and why interpretability is non-negotiable.
Diabetes is one of the highest-cost chronic conditions in Korea, both in direct healthcare spend and in compounding complications across the cardiovascular, renal, and ophthalmic systems. Managing the cost trajectory of a diabetes cohort is commercially valuable for any healthcare service that integrates with insurance or member-facing wellness products. HD Hyundai Medi Plus needed both a predictor for next-year medical cost and, crucially, a model whose coefficients could be explained back to members.
Cost is structurally hard to model. It is right-skewed (most members generate near-zero cost in any given year, and a small tail dominates total spend), and it is longitudinal (this year's cost correlates with last year's because the underlying health state is persistent). Standard cross-sectional regression discards the longitudinal signal; standard tree ensembles ignore the panel structure entirely. The interpretability requirement also ruled out black-box methods regardless of accuracy.
Korea Health Panel v1.7.2, 2014 to 2018, seven sheets joined.
The Korea Health Panel (KHP) is a longitudinal household survey covering healthcare utilisation, expenditure, and demographics. I used five consecutive annual waves (2014–2018) of KHP v1.7.2, joining seven KHP sheets per year (household, individual, ER use, inpatient, outpatient, supplemental survey, chronic disease) into a unified analytical panel. Across the five years that joined dataset is built from 35 sheet-year combinations.
Diabetes-cohort selection used KCD codes E10–E17. Cohort sizes per year were roughly stable at around 1,400 patients, with the small year-to-year variance reflecting standard panel attrition and addition.
The dependent variable was personal medical spend (out-of-pocket payment plus prescription cost). Its raw distribution is heavily right-skewed, about 1.4% of observations are zero-cost, and about 9.4% sit above the upper outlier threshold (1.5 × IQR + Q3). I applied a log transformation; this is the standard treatment for healthcare cost and stabilises the regression at the cost of needing care on the inverse transform later.
Independent variables: 32 candidates spanning demographics, lifestyle (BMI, smoking-years, sleep hours, exercise minutes), utilisation (outpatient days, inpatient days, ER visits), chronic-disease counts, complication codes, marriage status, education, income proxies, and insurance type. Complication codes for diabetes (KCD E10–E17), hypertension (I10–I15), and hyperlipidemia (E78) were documented up front so the framework could extend to those cohorts in follow-up work.
Random Effects on log-cost. Variable selection. Validated.
Random Effects Linear Regression on log-cost
The model is a Random Effects Linear Regression on log-transformed cost, with an individual-level random intercept that absorbs unobserved heterogeneity (chronic risk profile, healthcare-seeking behaviour, regional access differences). Fixed effects on the observable covariates do most of the explanatory work; the random intercept gives each individual a credible baseline.
For next-year forecasting I assumed a 2% medical inflation rate applied uniformly to predicted costs.
Variable selection: four derived specifications
The full 32-variable specification produced an Adjusted R² of about 0.57, but a sizeable share of variables sat above p > 0.10. Rather than dropping non-significant terms one at a time, I derived four candidate specifications and compared them on AIC and adjusted R²:
- (a) Outpatient utilisation only, single-variable baseline.
- (b) Highly significant variables (p < 0.05).
- (c) Significant plus marginal (p < 0.10).
- (d) Magnitude-based, variables whose coefficient estimate exceeded 1.
Specification (c) won on the joint criterion: 19 variables, Adjusted R² ≈ 0.56, RMSE ≈ 1.39 on the log scale, AIC dropping from about 25,800 in the full model to about 15,600 in the selected one. The magnitude of the AIC drop confirms the trimmed specification was substantially better-supported by the data, not just a parsimony win.
Validation
Two diagnostics ran cleanly. Variance Inflation Factors were all below 5, consistent with no problematic multicollinearity. The Durbin-Watson statistic on the model residuals was approximately 1.98, very close to 2, the assumption of no autocorrelation in the residuals held.
Top drivers, and the lifestyle deltas that became the product.
Top cost drivers in the final 19-variable model:
- Inpatient days and inpatient indicator: by a wide margin the largest coefficients.
- Outpatient days: also dominant.
- General medication use and chronic-disease count.
- ER visit indicator.
- Smoking-related variables (current smoker indicator, daily cigarette count, smoking years).
- Disability status, marriage status, and sedentary behaviour with smaller but stable effects.
Two findings the product team could act on directly
The model's coefficient signs and magnitudes made two lifestyle effects operationally usable:
A gradient-boosted ensemble would likely have predicted slightly better in absolute terms. But the moment we wanted to tell a member "today's exercise saves you about 3.4 million won by year ten," interpretability stopped being a preference and became a hard product requirement. The Random Effects model carried the explanation through to the user; an ensemble would not have.
From regression output to a member-facing savings calculator.
- Productised as a lifestyle savings calculator that surfaces individual, actionable savings projections with a 95% confidence interval. Sample copy: "Today's exercise saves you X won by year 10."
- End-to-end ownership. Spec, code, validation, and productisation handover. 90% contribution from intake to model selection.
- Replaces a cross-sectional baseline that ignored the panel structure and consistently mis-ranked tail-cost individuals.
- Generalises to other chronic-disease cohorts. Hypertension and hyperlipidemia complication codes were documented in the same framework so the methodology could be re-run on those cohorts without re-engineering.