Quantifying Structural Selection Bias in Observational Cohort Data: A Ponderation Analysis of Age- Specific Incidence Rates to Inform Vaccine Safety Verification

1 Research Article Quantifying Structural Selection Bias in Observational Cohort Data: A Ponderation Analysis of AgeSpecific Incidence Rates to Inform Vaccine Safety Verification Marco Roccetti Department of Computer Science and Engineering University of Bologna, 40126, Italy [email protected] ORCID: 0000-0003-1264-8595 Abstract Background: A recent nationwide cohort study reported an unadjusted Hazard Ratio (HR) of 2.714 for vitiligo incidence following COVID-19 vaccination, indicating a major safety concern. This finding was based on cohorts with an ≈ 11-year age difference, immediately raising critical concerns regarding extreme structural selection and detection bias. Objectives: We hypothesize that this extreme association is an artifact of a fatal methodological flaw, challenging the study's internal validity and subsequent external validity. We aim to quantitatively separate the HR attributable to the structural age imbalance (HR Structural) from the residual HR (HR Residual) which measures the uncorrected methodological failure. Methods: We performed a stratified ponderation analysis using the age distribution of the scrutinized study’s cohorts (Vaccinated, mean age=56.32 years vs Non-Vaccinated, mean age=45.51 years) and applied established national age-specific vitiligo incidence rates (IR) from external epidemiology. This allowed us to quantitatively separate the HR attributable to the structural age imbalance (HR Structural) from the residual HR (HR Residual), which measures the uncorrected methodological failure. 2 Results: The HR Structural was calculated to be 1.2821. This robust correction demonstrates that the structural age difference explains only 16.43% of the observed excess risk. The remaining HR Residual (2.1168) is the exact measure of the methodological failure caused by the double distortion in cohort design (lowering the baseline risk in the NonVaccine group while maximizing the detection risk in the Vaccine group. Discussion: The HR=2.714 of the scrutinized study is an unstable statistical artifact. The overwhelming majority of the observed association is a consequence of a fatal design flaw, not a biological risk, resulting in a severe lack of internal and external validity. Keywords: Covid-19 vaccine safety, Structural selection bias, Ponderation analysis, Hazard ratio decomposition, Internal/External validity, Detection bias 1. Introduction Observational studies utilizing national registries, such as those conducted in South Korea [1], represent a critical resource for post-marketing surveillance and vaccine safety verification. However, the reliance on pre-existing data necessitates strict adherence to established methodological standards, notably the STROBE (STrengthening the Reporting of Observational studies in Epidemiology) guidelines [2]. The primary goal is to ensure internal validity, that the observed association is real within the study context, which is a prerequisite for achieving external validity (that is generalizability to the broader population). The recent study published in [1] reported a strikingly high, unadjusted Hazard Ratio (HRGross) of 2.714 for vitiligo following COVID-19 vaccination, based on a comparison between a Vaccinated (V) cohort (mean age=56.32 years) and a Non-Vaccinated (Non-Vaccinated) cohort (mean age=45.51 years). This ≈11-year age difference immediately flagged critical concerns regarding confounding by indication and immortal time bias [3]. The sheer magnitude of the ≈11-year age difference, coupled with the cumulative incidence rates observed (2.22 vs 0.67 per 10,000), strongly suggests that the cohorts were inherently non-comparable. 3 Our analysis posits that the reported HR=2.714 is not a reflection of a robust biological signal but rather a quantitative measure of a fatal design flaw. We hypothesize that an Extreme Structural Selection and Detection Bias was introduced by defining the cohorts in a manner that artificially minimized the baseline risk in the NV group, while simultaneously maximizing the detection and prevalence risk in the V group. We present a rigorous, quantitative method, that is a stratified ponderation analysis using external South Korean national age-specific incidence data, to decompose the observed HR and isolate the true contribution of the structural bias [6-10]. The quantitative findings of the present research confirm this hypothesis: the structural age difference alone accounts for a calculated Structural Hazard Ratio (HR Structural) of 1.282. This means that the observed demographic imbalance explains only 16.43% of the reported excess risk. The majority of the association is captured by the Residual Hazard Ratio (HR Residual) of 2.117, which stands as a clear measure of the uncorrected methodological failure. This substantial residual value strongly indicates that the cohorts were not subject to a common support, leading to a profound violation of the assumption of comparability required by the Cox Proportional Hazards model utilized in the investigated study. Ultimately, the goal of this re-evaluation is to reassert the imperative for epidemiological validity in studies of vaccine safety derived from observational data. We demonstrate that sophisticated statistical adjustments cannot remedy fundamental flaws in cohort design where non-measured confounding factors, such as health-seeking behavior and surveillance frequency (i.e., Detection Bias), are unevenly distributed [7]. By quantitatively isolating and measuring the non-causal structural bias, our analysis provides a critical framework for interpreting extreme risk estimates and ensuring that public health conclusions are based on associations that are epidemiologically sound, rather than artifactua.l 2. Methods We here provide all the fundamental methods and data useful for the aim of pondering the structural bias on which we are inestigating. 4 2.1 Study Data and Baseline Characteristics We extracted the following key data from [1] to establish the basis of the structural bias as reported in the following Table 1. Table 1: Baseline Characteristics and Unadjusted Incidence Rates from [1]. Cohort Mean Age Standard Deviation (SD) Cumulative Incidence Rate (at 3 mo) (per 10,000 p-y) Non-Vaccinated (NV) 45.51 years 17.31 P(NV) = 0.67 Vaccinated (V) 56.32 years 16.55 P(V) = 2.22 2.2 Stratified Ponderation Analysis We performed a first preliminary quantitative analysis by combining the age distribution percentages P(i) of the V and NV groups of [1] with independent, established age-specific annual incidence rates IR(i) for vitiligo in South Korea, based on 2019 data as reported in [5]. Table 2: Input Data for Ponderation Analysis (Weighted IR) Age Group South Korean IR (per 10,000 p-y) % in NV Group P(i, NV) % in V Group P(i,V) < 20 y 3.4241 Not included in [1] Not included in [1] 20-29 y 1.5717 18.46% 9.92% 30-39 y 1.7813 25.49% 7.70% 40-49 y 1.9053 20.92% 10.82% 5 Age Group South Korean IR (per 10,000 p-y) % in NV Group P(i, NV) % in V Group P(i,V) 50-59 y 2.5874 14.16% 24.76% >= 60 y 3.3643 20.97% 45.79% Total — 100% 100% 2.3 Calculation of HR Structural and HR Residual The Expected Annual Incidence Rate, IR(Expected,) for each cohort, based solely on its structural age composition, can be calculated using the following Formula 1: IR(Expected) = ∑(IR(i) × P(i)). (1) Where IR(i) are the South Korean age-specific incidence rates from external data (Table 2) and P(i) are the proportional distributions of the respective cohort (V or NV) reported in the same Table 2. Applying this ponderation to the Non-Vaccinated (NV) cohort demographics, we obtain the baseline expected incidence, IR(NV, Expected) exactly as follows: IR(NV, Expected) = (1.5717 × 0.1846) + (1.7813 × 0.2549) + (1.9053 ×0.2092) + (2.5874 × 0.1416) + (3.3643 × 0.2097 ≈ 2.1611 / 10,000. Similarly, applying the ponderation to the Vaccinated (V) cohort demographics yields IR(V, Expected): IR(V, Expected) = (1.5717 × 0.0992) + (1.7813 × 0.0770) + (1.9053 × 0.1082) + (2.5874 × 0.2476) + (3.3643 × 0.4579) ≈ 2.7709 / 10,000. This allowed us to calculate the HR Structural as follows: HR Structural = 2.7709 / 2.1611 ≈ 1.2821. Finally, the HR Residual can be computed as the ratio between the HR provided in [1] (termed HR Observed) and our computed HR Structural: HR Residual = HR Observed / HR Structural = 2.714 / 1.2821 ≈ 2.1168. 6 3. Results: Following the calculation of the Expected Incidence Rates IR(Expected) based solely on the structural age compositions of the two cohorts (Section 2.3), we proceeded to quantify the true extent of the methodological failure. This involved decomposing the high, observed HR Observed =2.714 from [1] into two distinct components: the risk attributable purely to the structural age imbalance (HR Structural) and the risk stemming from all other uncorrected design flaws and selection biases (HR Residual). Since Hazard Ratios combine multiplicatively, that is HR Observed = HR Structural × HR Residual, the HR Residual thus acts as a precise metric for the degree of non-comparability that persists despite accounting for the known age difference. The breakdown of this risk is definitely presented in Table 3. Table 3: Decomposing the Observed Hazard Ratio (HR=2.714 [1]) Parameter Description Value Contribution to Excess Risk (HR−1) HR Observed Unadjusted Hazard Ratio from [1] 2.714 100% HR Structural HR due to Age Structural Bias Alone 1.2821 16.43% HR Residual HR Unexplained by Structural Age Bias 2.1168 83.57% In closing this Section, it is crucial to emphasize how our robust, age-specific ponderation analysis has shown that the structural age difference explains only 16.43% of the excess risk signaled by the authors of [1]. The overwhelming majority of the association (83.57%, resulting in an HR Residual of ≈ 2.12) is entirely attributable 7 to uncorrected methodological flaws which should be attributed to a basic failure in the construction of the cohort and theur subgroups. 4. Discussion We will summarize the key takeaways of this discussion into two primary issues to ensure they are properly highlighted, noting that at the heart of the matter lie problems of loss of comparability and resulting clinical significance. 4.1 The Collapse of Internal Validity: The Double Distortion Mechanism The persistence of the high residual HR (2.117) after robust adjustment for age structure (HR Structural = 1.282) provides a definitive quantitative proof that the cohorts have been constructed as non-comparable. The study's design of [1] suffers from a double distortion mechanism that fundamentally violates the core premise of observational epidemiology. First, we are talking about issues of an artificial baseline depression of the NV sub-group. In fact, the NV group was disproportionately composed of individuals in the 20−49 year age range, which falls into the natural lowsurveillance and post-first-peak incidence phase. Individuals in this group are less likely to seek frequent medical care. Critically, the small fraction of older individuals (>= 50 years) who chose not to be vaccinated during a major pandemic likely represents an exceptionally healthy survivor cohort or individuals with minimal interaction with the healthcare system [8]. This demographic makeup naturally suppresses both the true incidence rate and the rate of diagnosis (Detection Bias), yielding an artificially low baseline of 0.67/10,000. Second, we need to confront with an inflated incidence by detection and risk in the V subgroup. Conversely, in fact, the V group's composition of [1] (≈ 70% aged >= 50 years) guarantees maximal risk exposure, encompassing the entire second incidence peak of Vitiligo. Furthermore, the choice to vaccinate during a pandemic signifies a higher level of health consciousness and engagement with medical services. This heightened surveillance and utilization bias ensures that even subclinical cases of vitiligo or stable cases are more likely to be diagnosed and logged during the brief follow-up period, inflating the observed rate to 2.22/10,000. 8 This extreme structural separation, especially in the high-risk and high-surveillance age categories, represents a violation of the common support assumption. The Cox model emplyed in the scrutinized study, therefore, did not compare like with like, but rather measured the risk differential between an artificially clean control group and a maximally surveilled risk group. 4.2 Clinical and External Validity Implications for Vaccine Safety Verification The extreme HR Residual ≈ 2.12 cannot be interpreted as a genuine biological effect. A true biological signal of this magnitude would require a plausible mechanism that is not confounded by the age structure, a mechanism the original study could not isolate. Instead, the finding is a direct result of the design, which renders the study's conclusions not externally valid to any clinical scenario. The clinical implication is that the reported HR = 2.714 of [1] is gravely misleading for patients and clinicians. It does not reflect the incremental risk of vaccination but rather the difference in underlying health and healthcare seeking behavior between two demographically distinct groups in South Korea. This methodological failure is a serious breach of epidemiological reporting standards in the sense of the STROBE protocol and undermines the utility of national registry data for assessing vaccine safety signals when proper cohort matching is neglected. 4.3 Limitations and Future Directions We acknowledge several limitations to our ponderation analysis. First, the HR Structural calculation relies on the assumption that the external, age-specific incidence rates IR(i) derived from the general South Korean population (as reported from different perspectives in all the available literature [4-6]) accurately reflect the true baseline risk within the national health insurance service data utilized in [1]. Second, our analysis only addresses confounding introduced by structural age differences; we are unable to quantify the residual contributions of other unmeasured variables, such as socioeconomic status (SES), co-morbidities, or the precise effect of Detection Bias related to varying healthcare utilization frequency, all of which likely inflated the HR Residual. Furthermore, this methodological criticism does not exclude the possibility of a smaller, genuine biological signal (HR < 1.2821), which would be revealed only through a properly designed study utilizing tight Propensity Score Matching (PSM) and time-varying exposure analysis which was clearly not used by the authors of [1]. Nonetheless, we maintain that our present analysisi has provided a crucial quantitative framework for critically assessing the validity of 9 large epidemiological risk estimates derived from imbalanced cohorts and providing a relevant contribution towards the fidelity and verifiability of vaccine safety signals derived from observational cohort data. 5. Conclusion The association between COVID-19 vaccination and vitiligo (HR = 2.714) as repotred in [1] is an extreme statistical artifact. Our robust ponderation analysis, based on specific South Korean age-incidence rates, definitively proves that the structural age imbalance explains only a minor fraction (HR Structural ≈ 1.282) of the observed risk. The overwhelming HR Residual of 2.117 is the quantitative measure of the methodological failure caused by the Structural Selection and Detection Bias that have affected the construction of the retrospective cohort of [1]. This failure to establish genuinely comparable cohorts has compromised both the internal validity and external validity of the investigated study. The reported finding of [1] is therefore non-causal and should not be used to inform public health policy or safety communication. We urge the re-evaluation and potential reconsideration of the study's conclusions. Our findings also underscore the persistent reliance on and respect for methodological Gold Standards in clinical research [9, 10]. While innovation in epidemiological design is crucial, these established benchmarks must only be challenged or superseded by new studies featuring superior internal validity and robust correction for all known sources of bias. Author Information Marco Roccetti: Department of Computer Science and Engineering, University of Bologna, 4016 Bologna, Italy, [email protected]. ORCID: 0000-0003-1264-8595, sole and corresponding author Author Contributions MR conceived and designed the study, carried out all data collection and analysis, interpreted the quantitative results, and was the sole author responsible for writing and revising the manuscript. The author affirms full responsibility for the integrity of the data and the accuracy of the data analysis presented. Ethics approval and consent to participate This study uses publicly available, aggregated data that contains no private information. Therefore, ethical approval is not required