Advanced Matching Techniques for Population Comparisons in RWE Studies: The case of Entropy Balance

Advanced Matching Techniques for Population Comparisons in RWE Studies: The case of Entropy Balance Ramaa Nathan1 PhD; Mostafa Shokoohi2, PhD; Jason Poh1, PhD Pierantonio Russo1, MD 1EVERSANA, 7045 College Blvd, Suite 300, Overland Park, KS 66211 2EVERSANA, 204-3228 South Service Road, Burlington, Ontario L7N 3H8 Abstract Matching techniques are central to ensuring valid inferences in observational and comparative studies. While traditional approaches such as propensity score matching (PSM) and inverse probability treatment weighting (IPTW) have been widely adopted, they exhibit key limitations in covariate balance and sample efficiency. Entropy balancing (EB) offers a robust alternative by directly optimizing covariate balance through moment conditions: reweighted control group has the same moments (means variances, skewness), of covariates as the treated group. In retrospective studies, entropy balancing help achieve covariate balance and reduce confounding. In this study, we compared the performance of IPTW and EB methods using Merative claims data (2006-2024). Three patient cohort groups were balanced on age, sex, insurance type, region and Elixhauser Comorbidity Index (ECI): two with binary treatments using average treatment effect on treated (ATT) and one multinomial treatment using average treatment effect (ATE). Balance was assessed via effective sample size (ESS), weight distribution and absolute standardized mean difference (ASMD). In the first binary group (48 vs. 4,800 patients), both methods achieved balance: IPTW (ASMD <0.01; ESS: 1,545; weights: 0.01-0.1) and EB (ASMD <0.001; ESS: 1,353; weights: 0.01-11.69). In the second binary group (24,423 vs. 16,406 patients), only EB balanced all covariates (ASMD <0.0001; ESS: 5,913; weights: 0.01-24). In the multinomial group (350 vs. 53 vs. 82 patients), only EB balanced all covariates (ASMD <0.001; ESS: 338, 39, 48; weights: 0.01-4.8). These findings suggest that in real-world studies, EB, by requiring the weighted variance of covariates in the control group to match those in the treatment group (second moment constraints) in addition to matching the means, provides better covariate balance. Key Words: IPTW, Entropy Balancing 1. Introduction In a randomized control trial (RCT), the confounders are controlled by randomizing the treatment between the subjects. This automatically ensures that the confounders are equally distributed between the treatment groups. To mimic a RCT in an observational study (Franklin JM 2021), the confounders have to be specifically controlled for to ensure the ignorability and positivity assumptions hold. One way to control for the variables is by using matching or re-weighting techniques to ensure balanced distribution of the confounders. Matching techniques are central to ensuring valid inferences in observational and comparative studies. While traditional approaches such as propensity score matching (PSM) and inverse probability weighting (IPW) have been widely adopted, they exhibit key limitations in covariate balance and sample efficiency. Entropy balancing offers a robust alternative by optimizing covariate balance through moment conditions. This paper reviews population matching methods with a special focus on entropy balancing, highlighting its theoretical foundations, empirical performance, and implications for real-world evidence (RWE) generation, and epidemiological research. We describe three case studies, and evaluate the performance between EB and IPTW and discuss how EB can become a preferred method in modern causal inference frameworks (Miguel A. Hernan 2023)and when randomization is not possible. 2. Methods 2.1 Background on Statistical Techniques Matching techniques like exact matching or stochastic matching like propensity score matching can be used to do a 1:1 matching between the cohort of subjects administered target treatment and the cohort of subjects administered comparator treatment. But, as the number of subjects administered either drug cannot be controlled or randomized proportionally, it could be very difficult to find 1:1 match at the time of each interim report generation resulting in a reduction in the number of subjects included in the study. Instead, re-weighting techniques can be used to balance between the different sized cohorts without a loss in the number of subjects. There are two common techniques used for re-weighting cohorts – Inverse Probability of Treatment Weighting (IPTW) and entropy balancing (EB) (J. 2012) IPTW is a statistical technique based on the Propensity Score model. In this technique, the propensity score (PS) of each patient for receiving the target treatment is first determined using a classification model like logistic regression or a tree-based model. The individual weight associated with each patient is defined as the inverse of the probability of being in the corresponding cohort. For subjects treated with the target treatment, the weight is the inverse of the propensity score and for subjects treated with a comparator treatment, the weight is the inverse of 1propensity score. IPTW is a very iterative process to ensure that the propensity score has high accuracy. Entropy balancing [ (J. 2012) is a statistical technique used to balance the distribution of covariates across different groups or treatments in an observational study. The basic idea is to reweight the observations so that the distribution of covariates in each group is as similar as possible. This is done by assigning weights to each observation based on its entropy score, which measures the degree of imbalance in the covariate distribution. The end result is a dataset where the covariates are balanced across groups, which can help reduce bias in estimating treatment effects. In simpler terms, entropy balancing is a method to ensure that each group in a study has a similar mix of characteristics, so that the effect of the treatment (exposure) can be accurately measured. Entropy balancing is specifically designed to achieve balance across multiple treatment groups. It considers the covariate distributions across all treatment groups simultaneously and optimizes the weights to minimize the differences between them. In this study, IPTW was used as the primary technique for cohort re-weighting, with entropy balancing acting as a secondary sensitivity analysis. In both cases, the balancing of covariates was verified after re-weighting using standard techniques. Given the binary exposure variable, the Standard Mean Differences of each covariate was checked to be below 0.1. 2.1 Data Source Closed claims data from the Merative® MarketScan® Research Databases were used in this retrospective study. This is a nationally representative dataset comprised of large employer-based insurance (including Medicare Advantage) plans consisting of more than 300 unique employers and over 25 different health plans. Data are contributed by employers, states, health plans, hospitals, electronic medical records (EMR) providers, and Social Security Administration. This is the largest collection of privately and publicly insured de-identified patient-level data in the US, with more than 32 billion service records and over 240 million covered lives. The database integrates individual-level enrollment files and paid claims data for inpatient and outpatient care, as well as paid prescription drug claims starting from 2006. Prescription claims include those from brick-and-mortar pharmacies and mail-in prescriptions 2.2 Study Cohorts Three patient cohort groups were included in the study. The first cohort pair (Pierantonio Russo 2025)Error! Reference source not found.8] included adults (age >=18), diagnosed with chronic obstructive pulmonary disease (COPD) with at least two claims for COPD as determined by the clinical codes ICD-10 J44 and ICD-9 codes of 493.2 and 491.2 and were arrhythmia naïve at the time of initial diagnosis of COPD. Patients with diabetes (Type 1 and Type 2) were excluded from the study. The target cohort were patients with COPD and had an arrhythmia (target) and the comparator cohort included patients with COPD, but who had not experienced an arrhythmia. The comparator type here was binary. The second cohort pair included adult patients with Gastroparesis and treated with metoclopramide (MCP) (Russo P 2025)]. The target cohort comprised of these patients treated with nasal MCP (NMCP) and the comparator cohort included patients treated with only oral metoclopramide (OMCP). The comparator type here was binary. The third cohort set comprised of three cohorts and included adult patients diagnosed with Chronic myeloid leukemia (CML) and treated with Tyrosine Kinase Inhibitors (TKIs) (Kilcoyne A 2025). The target cohort comprised of patients treated with only TKI and the comparator cohort pair included patients treated concomitantly with TKI and acid-reducing agents (ARAs). Comparator cohort #1 were patients who were treated with concomitant medications before achieving remissions and the comparator cohort #2 were patients treated with concomitant medications, but after having achieved remission. Each patient was required to have continuous enrollment (CE) 6 months before and 24 months after the date of first TKI prescription. A threeway multinomial comparison was adapted. . In each of the three pairs of cohorts, IPTW and entropy balancing were separately applied to adjust for the baseline covariates of age, sex, insurance type, and Elixhauser comorbidity index (ECI). 2.3 Statistical Analyses The reweighting process is usually controlled by specifying the estimands or comparator types and by the number of moments (first: mean, second: variance, third: skew). In this study, only the first and second moments were used. There are three common weighting approaches used to adjust for differences in characteristics in the treated and comparator groups: • Average treatment effect in the treated (ATT): considers treatment effect in a population similar to the treated population • Average treatment effect in the control (ATC): considers treatment effect in a population similar to the comparator population • Average treatment effect (ATE): considers treatment effect in a combined population In this study, ATT was used for cohort pair #1 and #2 with binary treatments and ATE was used for the third cohort set with multinomial treatments,. Balance achieved via reweighting was assessed via effective sample size (ESS), weight distribution and absolute standardized mean difference (ASMD). Effective sample size (ESS) is the number of non-weighted patients that would produce a treatment effect estimate with the same precision as the weighted sample estimate (Figure 1Figure 1). There is limited HTA guidance regarding minimally accepted ESS. However, ESS should be considered when assessing the comparability of study populations and the overall robustness of estimated treatment effects. A large decrease in ESS is indicative of substantial cross-trial differences. Figure 1: Calculation of ESS Absolute standardized mean difference (ASMD) is the difference in means between groups, divided by the pooled standard deviation. 𝑠𝑚𝑑=% 𝑥'!"#$%! −𝑥'&'(!#') )𝑠!"#$%! *+𝑠&'(!#') * 2 ASMD does not depend on sample size and is calculated for each variable. As a rule of thumb: • Values < 0.1 indicate adequate balance • Values 0.1-0.2 are not too alarming • Values > 0.2 indicate serious imbalance 3. Results 3.1 Case Study #1 In the first binary group, the target cohort comprised of 24,423 patients with COPD followed by arrhythmia and the comparator group had 16,406 patients with COPD and no arrhythmia. In this case only EB balanced all covariates (ASMD <0.0001; ESS: 5,913; weights: 0.01-24). Table 1: Case Study #1 - Effective Sample Size Cohort Effective Sample Size Balanced? IPTW Target (N=24,423) 20,080.27 No (ECI) Control (N=16,406) 1,931.88 EB (1 moment) Target (N=24,423) 21,120.73 Yes Control (N=16,406) 10,329.08 EB (2 moments) Target (N=24,423) 20,102.52 Yes Control (N=16,406) 10,775.32 EB (3 moments) Target (N=24,423) 19,637.79 Yes Control (N=16,406) 10,752.64 Table 2: Case Study #1 - SMD of Baseline Characteristics Characteristic Absolute SMD (Unweighted) Absolute SMD (After IPTW) Absolute SMD (After EBAL 1moment) Absolute SMD (After EBAL 2moment) Absolute SMD (After EBAL 3moment) Age (years) 0.6184 0.0465 <0.0001 <0.0001 <0.0001 Sex (Female) 0.2087 0.0153 <0.0001 <0.0001 <0.0001 Insurance Type Commercial 0.3502 0.0381 <0.0001 <0.0001 <0.0001 Medicaid 0.1287 0.0215 <0.0001 <0.0001 <0.0001 Medicare 0.4916 0.0155 <0.0001 <0.0001 <0.0001 Region North Central 0.2229 0.0124 <0.0001 <0.0001 <0.0001 Northeast 0.0265 0.0011 <0.0001 <0.0001 <0.0001 South 0.1162 0.0318 <0.0001 <0.0001 <0.0001 Unknown 0.1274 0.0207 <0.0001 <0.0001 <0.0001 West 0.0583 0.0044 ≤0.0001 <0.0001 <0.0001 ECI Score (Continuous) 0.8336 0.4509 <0.0001 <0.0001 <0.0001 Abbreviations: SMD = standardized mean difference; IPTW = inverse probability of treatment weighting; EBAL = empirical balancing; ECI = Elixhauser Comorbidity Index. Among 40,829 adults with COPD, 24,423 had a diagnosis of arrhythmia and 16,406 did not (Table 1b). Before weighting, several baseline variables were imbalanced between the two study cohorts, including age (SMD: 0.62), ECI score (SMD: 0.83), and insurance type (commercial: 0.35; Medicare: 0.49). Imbalance was also observed by region and sex (SMDs: 0.20–0.22). After IPTW, covariate balance improved for all covariates ( absolute SMD < 0.05) other than ECI score (SMD: 0.45). Using EBAL weighting (1-3 moments) urther improved covariate balance, achieving complete alignment for all covariates (absolute SMD ≤ 0.0001). ESS were comparable in the two methods (target ≈ 19,600-21,100; control ≈ 10,300-10,750). 3.2 Case Study #2 In the second binary group of patients with gastroparesis treated with Nasal (N=48) vs. oral metoclopramide (N=4,800), both methods achieved balance: IPTW (ASMD <0.01; ESS: 1,545; weights: 0.01-0.1) and EB (ASMD <0.001; ESS: 1,353; weights: 0.01-11.69). Table 3: Case Study 2: Effective Sample Size Reweighting Technique Effective Sample Size (N=4,800) Balanced? IPTW 1,545.35 Yes EB (1 moment) 1,551.18 Yes EB (2 moments) 1,353.49 Yes EB (3 moments) 1,234.99 Yes Table 4: Case Study #2 - SMD of Baseline Characteristics Characteristic Absolute SMD (Unweighted) Absolute SMD (After IPTW) Absolute SMD (After EBAL 1moment) Abs. Weighted SMD (EBAL moment 2) Abs. Weighted SMD (EBAL moment 3) Age (years) 0.387 0.0013 <0.0001 <0.0001 <0.0001 Sex (Female) 0.2478 0.0028 <0.0001 <0.0001 <0.0001 Insurance Type <0.0001 <0.0001 Commercial 0.8876 0.0003 <0.0001 <0.0001 <0.0001 Medicaid 0.8307 0.0004 <0.0001 ≤0.0001 <0.0001 Medicare 0.1848 0.0002 ≤0.0001 <0.0001 <0.0001 Elixhauser Comorbidity Index (ECI) <0.0001 <0.0001 ECI <2 0.4341 0.0016 <0.0001 <0.0001 <0.0001 ECI 2–5 0.2620 0.0005 <0.0001 <0.0001 <0.0001 ECI 6–10 0.5998 0.0011 <0.0001 <0.0001 <0.0001 ECI 11–15 0.0452 0.0016 <0.0001 <0.0001 <0.0001 ECI >15 0.1634 0.0026 <0.0001 <0.0001 <0.0001 Gastroparesis Severity <0.0001 <0.0001 Mild 0.1405 0.0026 <0.0001 <0.0001 <0.0001 Moderate 0.1246 0.0032 <0.0001 <0.0001 <0.0001 Severe 0.0452 0.0005 <0.0001 <0.0001 <0.0001 Among 4,848 patients with gastroparesis, 48 received nasal metoclopramide and 4,800 received oral metoclopramide (Table 1a). Before weighting, a few covariates showed imbalance between the two groups, including insurance type (commercial SMD: 0.89; Medicaid SMD: 0.83) and comorbidity burden (ECI 6-10 SMD: 0.60). Other covariates with imbalance distribution included age (SMD: 0.39), sex (SMD: 0.25), and gastroparesis severity (SMDs: 0.12-0.14 across categories). After weighting, covariates were balanced using the two weighting approaches (all absolute SMD < 0.004 for IPTW and ≤ 0.0001 for EBAL method). ESS were similar in approaches (≈ 1,235 to 1,551). 3.3 Case Study #3 In the multinomial group of three cohorts of patients with CML treated with Tyrosine Kinase Inhibitors (TKIs), the first cohort comprised of patients treated with only TKI (N=350), the second cohort had patients treated with TKIs and Concomitant with acid reducing agents after remission (N=53) and the third cohort had patients treated with TKIs and oncomitant with acid reducing agents before remission (N=82). Only EB balanced all covariates (ASMD <0.001; ESS: 338, 39, 48; weights: 0.01-4.8). Table 5: Case Study #3 - Effective Sample Size Reweighting Technique Cohort Effective Sample Size Balanced? IPTW TKIs only (N=350) 342.05 No Con after Rem (N=53) 48.35 Con before Rem (N=82) 68.97 EB (1 moment) TKIs only (N=350) 342.36 Yes Con after Rem (N=53) 49.41 Con before Rem (N=82) 49.90 EB (2 moments) TKIs only (N=350) 339.16 Yes Con after Rem (N=53) 47.76 Con before Rem (N=82) 49.44 Table 6: Case Study #3 - SMD of Baseline Characteristics Characteristic Max Abs. Unweighted SMD Max Abs. Weighted SMD (IPTW) Max Abs. Weighted SMD (EBAL 1moment) Max Abs. Weighted SMD (EBAL 2moment) Age (years) 0.3911 0.0873 <0.0001 <0.0001 Sex (Female) 0.2374 0.0245 <0.0001 <0.0001 Insurance Type – Commercial 0.1054 0.0342 <0.0001 <0.0001 – Medicaid 0.2245 0.1088 <0.0001 <0.0001 – Medicare 0.0956 0.0950 <0.0001 <0.0001 ECI Score (Continuous) 0.2192 0.1207 <0.0001 <0.0001 First/Second Generation Treatment 0.3855 0.1178 <0.0001 <0.0001 Abbreviations: SMD = standardized mean difference; IPTW = inverse probability of treatment weighting; EBAL = empirical balancing; ECI = Elixhauser Comorbidity Index. Among 485 patients with chronic myeloid leukemia (CML), 350 received TKI therapy only, 53 received concomitant therapy after achieving remission, and 82 received concomitant therapy before remission (Table 1c). Before weighting, following covariates showed imbalance among the study groups: age (SMD: 0.39), first/second generation treatment (SMD: 0.39), and insurance type (Medicaid SMD: 0.22). Comorbidity burden (ECI score, SMD: 0.22) and sex (SMD: 0.24) also showed imbalance. After IPTW, most covariates improved but residual imbalance remained for Medicaid insurance (SMD: 0.11), ECI score (SMD: 0.12), and treatment generation (SMD: 0.12). Using EBAL weighting (1or 2-moment) achieved covariate balance for all covariates (absolute SMD ≤ 0.0001). ESS were preserved across the three study groups (≈340-350 for TKIs only, ≈4850 for concomitant groups). 4. Discussion Comparative analyses in clinical research hinge on constructing equivalent patient populations to accurately estimate treatment effects. While randomized controlled trials (RCTs) achieve this through random assignment—balancing both observed and unobserved confounders—real-world evidence (RWE) studies must replicate this balance using statistical methods to reduce bias and ensure credible findings. Propensity score techniques, such as matching and inverse probability of treatment weighting (IPTW), are standard tools in observational research. These methods attempt to adjust for differences in baseline covariates by modeling the probability of treatment assignment. However, they come with limitations: residual imbalance, sensitivity to model specification, and potential loss of sample size due to unmatched patients. These issues are magnified in high-dimensional datasets like electronic health records (EHRs) or administrative claims data. Entropy balancing (EB), introduced by Hainmueller in 2012, offers a compelling alternative. Rather than modeling treatment probabilities, EB directly incorporates covariate balance into the weighting process. It does so by solving an optimization problem that assigns weights to control units so that their covariate distributions exactly match the treatment group on specified moments, typically the mean, variance, and skewness. This ensures balance in a single step without excluding data, preserving statistical power and simplifying the workflow. EB aims to recreate the statistical properties of an RCT within observational data by reweighting the sample to achieve covariate balance. The researcher imposes balance constraints, and EB finds weights that satisfy these constraints while staying as close as possible to uniform base weights. This method bypasses the iterative modeling and balance checking required in IPTW, where one must first estimate the propensity score and then assess balance, often tweaking models and applying calipers or trimming to achieve acceptable results. A key distinction between the methods lies in their structure. In IPTW, weights are estimated for both treatment and control groups based on the predicted propensity scores (PC 2011) (Franklin JM 2021). EB, on the other hand, typically assigns a weight of one to treated observations and computes balancing weights only for the control group. This streamlines the process and avoids the instability associated with extreme propensity score weights. Once computed, EB weights can be applied in standard statistical analyses—ranging from weighted mean comparisons to regression model, allowing researchers to estimate treatment effects with improved balance and reduced bias. EB is also versatile: it accommodates binary, multinomial, and continuous treatment types. In cases of continuous treatment, the method balances the moments of the treatment variable and covariates, ensuring robust adjustment across a range of applications. The strengths of entropy balancing are especially pertinent in clinical and real-world settings. Its use is growing in the medical literature. In oncology, EB has been employed to adjust external control groups drawn from real-world data to better match RCT populations, such as in studies on advanced non-small cell lung cancer. In public health, EB has enabled cross-regional comparisons of interventions—like mask mandates or vaccine rollouts—by adjusting for demographic and health-related differences. It has also shown utility in EHR-based studies evaluating treatments for chronic conditions such as diabetes and cardiovascular disease. In summary, entropy balancing offers a rigorous, transparent, and efficient method for achieving covariate balance in observational research. Its capacity to retain data, reduce modeling complexity, and directly enforce balance makes it a powerful tool for generating credible evidence in settings where randomization is not feasible. 5. Conclusion Entropy balancing provides a rigorous practical approach for creating balanced comparison groups in observational research, helping bridge the gap between real-world studies and the methodological rigor of randomized trials. References Ben-Michael E, Feller A, Rothstein. 2021. "The augmented synthetic control method." Journal of the American Statistical Association, 116 (536): 1789-1803. Franklin JM, Patorno E, Desai RJ, Glynn RJ, Martin D, Quinto K, Pawar A, Bessette LG, Lee H, Garry EM, Gautam N, Schneeweiss S. 2021. "Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results." Circulation 143 (10): 1002-1013. Hung AM, Roumie CL, Greevy RA, Grijalva CG, Liu X, Murff HJ, Ikizler TA, Griffin MR. 2016 Dec 7. "Comparative Effectiveness of Second-Line Agents for the Treatment of Diabetes Type 2 in Preventing Kidney Function Decline." Clin J Am Soc Nephrol. 11 (12): 21772185. J., Hainmueller. 2012. "Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis." Political Analysis 20(1): 25–46. Kilcoyne A, Nathan R, Anderson C, Moore A, Russo P. 2025. " Impact on Cost of Care of Concomitant Prescribing of Acid Reducing Agents with Tyrosine Kinase Inhibitors in Chronic Myeloid Leukemia Patients: A US Payer Perspective." Journal of Health & Medical Economics 11 (1:102). Kosuke Imai, Marc Ratkovic. 2014. "Covariate balancing propensity score." Journal of the Royal Statistical Society: Series B 76 (1): 243-263. Miguel A. Hernan, James M. Robins. 2023. Causal Inference: What If. P Russo, Ramaa Nathan, Daniel Pfeffer, Jason Poh, Ken Boyle, Brent Wright, Erik Hendrickson. 2025. "Clinical and Economic Burdennof Cardian Arrhythmias in Patients with Type 2 Diabetes." Journal of Health & Medical Economics 11 (1): 1:151. PC, Austin. 2011. "An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies." Multivariate Behavioral Research 46(3):399– 424. Pierantonio Russo, Ramaa Nathan, Jason Poh, Harjeet Singh, Brent Wright, Ken Boyle & Erik Hendrickson. 2025. "Real world evidence on health care resource utilization and economic burden of arrhythmias in patients with COPD." Journal of Medical Economics 28 (1): 1564-1573. Russo P, Ramaa Nathan, Dan Pfeffer, Mostafa Shokoohi, Christopher Quesenberry. 2025. "NMCP delayed time to hospital encounter and reduced cost compared to OMCP in patients with gastroparesis - A Real World Study." ISPOR. Montreal, ON.