Subgroup Analysis in Pulmonary Hypertension-Specific Therapy Clinical Trials: A Systematic Review

Citation: Rodríguez-Ramallo, H.; Báez-Gutiérrez, N.; Otero-Candelera, R.; Martín, L.A.-k. Subgroup Analysis in Pulmonary Hypertension-Specific Therapy Clinical Trials: A Systematic Review. J. Pers. Med. 2022,12, 863. https://doi.org/10.3390/ jpm12060863 Academic Editor: Luis Almenar-Bonet Received: 21 April 2022 Accepted: 23 May 2022 Published: 25 May 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Journal of Personalized Medicine Systematic Review Subgroup Analysis in Pulmonary Hypertension-Specific Therapy Clinical Trials: A Systematic Review Héctor Rodríguez-Ramallo 1, Nerea Báez-Gutiérrez 2,* , Remedios Otero-Candelera 3 and Laila Abdel-kader Martín1 1Hospital Pharmacy Department, Virgen del Rocio University Hospital, 41004 Seville, Spain; hector[email protected] (H.R.-R.); laila.abdelkader[email protected] (L.A.-k.M.) 2Hospital Pharmacy Department, Reina Sofía University Hospital, 14004 Cordoba, Spain 3Pneumology Department, Virgen del Rocio University Hospital, 41004 Seville, Spain; r[email protected] *Correspondence: ner[email protected] Abstract: Pulmonary hypertension (PH) treatment decisions are driven by the results of randomized controlled trials (RCTs). Subgroup analyses are often performed to assess whether the intervention effect will change due to the patient’s characteristics, thus allowing for individualized decisions. This review aimed to evaluate the appropriateness and interpretation of subgroup analyses performed in PH-specific therapy RCTs published between 2000 and 2020. Claims of subgroup effects were evaluated with prespecified criteria. Overall, 30 RCTs were included. Subgroup analyses presented: a high number of subgroup analyses reported, lack of prespecification, and lack of interaction tests. The trial protocol was not available for most RCTs; significant differences were found in those articles that published the protocol. Authors reported 13 claims of subgroup effect, with 12 claims meeting four or fewer of Sun’s criteria. Even when most RCTs were generally at low risk of bias and were published in high-impact journals, the credibility and general quality of subgroup analyses and subgroup claims were low due to methodological flaws. Clinicians should be skeptical of claims of subgroup effects and interpret subgroup analyses with caution, as due to their poor quality, these analyses may not serve as guidance for personalized care. Keywords: pulmonary hypertension; subgroup analyses; randomized controlled trials; methodological limitations 1. Introduction Pulmonary hypertension (PH) is a rare disorder that may surge due to multiple clinical conditions or appear spontaneously without a clear cause [ 1 ]. Among other factors, the variety of etiologies of PH makes it an extremely complex disease; for this reason, a clinical classification was developed to group PH according to clinical presentation, findings, underlying conditions, and treatment [ 2 ]. PH is currently classified into five categories: group I pulmonary arterial hypertension, group II pulmonary hypertension due to left heart disease, group III pulmonary hypertension due to lung disease and/or hypoxemia, group IV chronic thromboembolic pulmonary hypertension, and group V pulmonary hypertension with unclear and/or multifactorial mechanisms. As PH affects older patients disproportionally and may cause rapid deterioration and an increased risk of death, it is considered a major health issue, specifically in countries with older populations [ 3 ]. The current standards of PH treatment include drugs targeting endothelin-1, nitric oxide, and prostacyclin pathways. These treatments aim to promote vasodilation and avoid vascular remodeling [ 1 ]. However, the extending knowledge of the pathophysiology of the disease allows for the discovery of new targets [ 4 , 5 ]. An innovative approach is to restore the balance between the activation of the growth-promoting activin growth differentiation factor pathway and the growth-inhibiting bone morphogenetic protein receptors, with sotatercept showing promising results in a phase II randomized J. Pers. Med. 2022,12, 863. https://doi.org/10.3390/jpm12060863 https://www.mdpi.com/journal/jpm J. Pers. Med. 2022,12, 863 2 of 14 controlled trial (RCT) [ 6 ]. The choice of treatment for PH will vary according to the group of PH hypertension being treated, as therapies usually considered appropriate may even be harmful in a certain subgroup of patients [1]. PH treatment decisions are driven by the results of RCTs. Usually, only average results are reported in RCTs, and trial participants are often recruited from heterogeneous populations. However, clinicians ideally want more specific information to assist them in applying trial results to individual patients. Researchers conducting an RCT usually perform a subgroup analysis to assess whether the effect of the intervention will change due to the patient’s baseline characteristics, such as underlying pathologies, age, sex, or severity of the disease, which may allow for individualized decisions. Based on subgroup analysis results, researchers may report claims of subgroup effects. However, subgroup claims should be interpreted with caution, as misstatements about subgroup effects may result in patients being denied beneficial treatments, or even receiving treatments that may be ineffective or harmful [7–9]. The need for standards for the interpretation of subgroup analysis is crucial for treatment decisions in medical practice. Previous evaluations of RCT subgroup analyses have consistently documented poor-quality methods that burden its credibility. These reports showed suboptimal decisions in the design of the analyses, such as a lack of prespecification in trial protocols, adequate statistical methods, and biological rationale [10–13]. In order to provide tools that assist readers of medical literature in evaluating the credibility of subgroup analyses, explicit criteria were developed [ 14 – 19 ]. Recent tools to evaluate subgroup credibility were published, such as Schandelmeier S et al., 2020 [ 14 ] and Gil-Sierra MD et al., 2020 [ 15 ]. However, as far as we are concerned, the “10 criteria for assessing the credibility of a subgroup claim” [ 19 ] is the most reliable tool to assess confidence in a subgroup analysis, as the criteria were widely tested in several disciplines [10–13]. The central purpose of this study was to evaluate the appropriateness and interpretation of subgroup analyses performed in PH-specific therapy RCTs. In order to achieve our goals, the following aspects were studied: •Description of subgroup analysis and claims of subgroup effects. •Research characteristics of subgroup analysis. •Analysis and interpretation of subgroup effects for primary outcomes. • Assessment of the credibility of subgroup claims using the “10 criteria for assessing the credibility of a subgroup claim” [19]. 2. Materials and Methods 2.1. Literature Search This systematic review aims to summarize the available data to solve the following research questions, framed in the population, intervention, comparator, outcome, and study (PICOS) design framework: population: patients with PH; intervention: PH-specific therapy; comparison: studies with a comparator will be considered; outcomes: subgroup analysis; study design: randomized clinical trials. The following groups of drugs were considered PH-specific therapy for this review: •Phosphodiesterase type 5 inhibitors. •Endothelin receptor antagonists. •Prostacyclin analogues and prostacyclin receptor agonists. •Calcium channel blockers. •Guanylate cyclase stimulators. A systematic search was conducted according to the preferred reporting items for a systematic review and meta-analysis (PRISMA) guidelines [ 20 ]. The systematic review protocol was registered with the prospective register for systematic review protocols (PROSPERO), registration number: CRD42021242265. The search was conducted between January 2000 and December 2020 using vocabulary and keywords controlled by MeSH terms in the MEDLINE database to identify RCT- assessing PH-specific therapy for PH patients. J. Pers. Med. 2022,12, 863 3 of 14 The search was performed in March 2021. The full literature search strategy is available in Supplemental File S1. The following criteria were used for the trial selection: Eligibility criteria: • We considered all published PH-specific therapy RCTs on PH hypertension adults with subgroup analysis reported. Exclusion criteria: •Articles written in languages other than English, Spanish, and French. •Post hoc analyses of previously published RCTs. •Articles that were not available. • Trials in which subgroup analysis credibility was impossible to evaluate due to missing data. 2.2. Study Screening and Selection Two investigators independently checked the titles and abstracts of the search results using predefined inclusion criteria. The full text was accessed for all titles that seemed to meet the inclusion criteria or have uncertainties. Two reviewers, HRR and NBG, assessed whether the article met the selection criteria. Any disagreements were resolved through discussion or arbitration with the third reviewer, LAM. 2.3. Data Extraction For data extraction, other sources included in the study were used (i.e., trial registration, published protocols, and online supplements). Data were extracted and entered into a structured Microsoft Excel (Redmond, WA, USA) database. Eligible RCTs were evaluated to determine whether a subgroup analysis was reported. Asubgroup factor was defined as a study variable by which the population may be categorized into different subgroups, i.e., sex, age, and the presence of a mutation. A subgroup analysis was defined as a specific analysis comparing two categories within a subgroup factor. For example, the analysis that compares the subgroups within the age factor >65 years vs. <65 years. A subgroup effect was defined as a difference in the magnitude of a treatment effect across a group of a study population [ 19 ]. For each RCT reporting subgroup analysis and subgroup claim, the following information was collected: Trial characteristics: Information on the funding source, year and journal of publication, journal impact factor, clinical classification of PH [ 2 ], updated by the European Society of Cardiology and the European Respiratory Society Guidelines [ 1 ], center (multicentric or unicentric), trial design (parallel, crossover, or factorial), trial type (superiority, noninferiority, or equivalence), allocation concealment, blinding of patients, and the number of patients randomized. The primary endpoint was categorized according to whether the results were statistically significant and the type of outcome variable (time-to-event, binary, continuous, or count). Reporting of subgroup analysis: Number of subgroup factors, type of subgroup factors (clinical factors or biomarkers), number of subgroup analyses and outcomes for subgroup analyses reported, forest plots used, whether it was a prespecified or post hoc subgroup, and the statistical method used to assess the heterogeneity of the treatment effect (descriptive only, subgroup pvalues and confidence interval or interaction test). When the trial protocol was available, the agreement on the number of subgroup factors, the number of subgroup analyses, and the prespecification of such analyses between the journal publication and the trial protocol were measured. In order to assess possible differences in the quality of subgroup analyses reporting according to PH clinical groups, the variables were described separately for trials including patients in different clinical PH groups [ 2 ]. Claims of subgroup effects : The mode of presentation (abstract or text only) of subgroup claims, number of subgroup claims, subgroup variable (primary or secondary outcome), and the number of outcomes for subgroup claims J. Pers. Med. 2022,12, 863 4 of 14 were recorded. A subgroup effect was considered to be claimed when the authors stated in the abstract or discussion that the intervention effect differed between the categories of the subgroup variable. The claims of subgroup effects were classified according to the strength of the claim into three categories: strong claim, a claim of a likely effect, or suggestion of a possible effect based on Sun et al. classification (Supplemental File S2). To evaluate the credibility of subgroup claims for primary outcomes, “the 10 criteria for assessing the credibility of a subgroup claim” were applied pairwise (Supplemental File S3). If the subgroup claim met less than half the criteria, the credibility of this claim was considered low. 2.4. Assessment of Risk of Bias The Cochrane Collaboration tool for assessing randomized trials [ 21 ] was used to evaluate the risk of bias in five dominions (randomization process, deviation from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result) and to present the results for each study across all dominions. Two independent reviewers evaluated the risk of bias. Possible disagreements between the reviewers were resolved by discussion or arbitration by a third reviewer when a consensus could not be reached. 2.5. Secondary Analyses The quality of subgroup analysis reports during four time periods (2000–2004, 2005–2009, 2010–2014, and 2015–2019) were compared. This analysis aims to assess whether the methodology reported to perform subgroup analyses has improved over time. 2.6. Data Analysis A descriptive analysis was developed. Continuous and categorical variables were presented as mean (range) and n(%), respectively. For those RCTs that stated a subgroup effect without providing an interaction test, P interaction was calculated using the Joaquin Primo calculator [ 22 ] to verify that there was indeed statistical significance. The inter-reviewer agreement for assessing the credibility of the subgroup claims was estimated by Cohen’s kappa coefficient. 3. Results The initial literature search identified 1837 studies. After the first review by title or abstract and the deletion of duplicates, 185 articles were selected for full-text review. Finally, 30 papers were included (Figure 1). The excluded articles and the reasons for their exclusion are provided in the supplementary material (Supplemental File S4). J. Pers. Med. 2022, 12, x FOR PEER REVIEW 5 of 15 Figure 1. Flowchart of the screening of randomized clinical trials included in this analysis. 3.1. Trial Characteristics The characteristics of the trials included in this study are listed in Table 1. The included publications reported data on 7765 randomized patients (median: 208; range: 52– 1156). Most studies were industry-funded (90%, n = 27). The most frequently selected journals for publication were The New England Journal of Medicine (n = 8) and Circulation (n = 4). Overall, 73% of the studies were published in high-impact journals (impact factor > 10). The most common PH type explored was type 1 (n = 20). The stated primary endpoint was statistically significant in 63% (n = 19) of trials. Table 1. Characteristics of trials included. Variable Nº Trials (n = 30) % Funding source Industry 27 90 Non-industry 2 7 Non specify 1 3 Year of publication 2000–2004 3 10 2005–2009 7 23 2010–2014 10 33 2015–2019 10 33 Journal Chest 2 7 Circulation 4 13 European Heart Journal 2 7 Journal of the American College of Cardiology 2 7 The Lancet Respiratory Medicine 2 7 The New England Journal of Medicine 8 27 Others 10 33 Journal impact factor <10 8 27 >10 22 73 Clinical PH classification Group 1 20 67 Group 2 3 10 Figure 1. Flowchart of the screening of randomized clinical trials included in this analysis. J. Pers. Med. 2022,12, 863 5 of 14 3.1. Trial Characteristics The characteristics of the trials included in this study are listed in Table 1. The included publications reported data on 7765 randomized patients (median: 208; range: 52–1156). Table 1. Characteristics of trials included. Variable Nº Trials (n= 30) % Funding source Industry 27 90 Non-industry 2 7 Non specify 1 3 Year of publication 2000–2004 3 10 2005–2009 7 23 2010–2014 10 33 2015–2019 10 33 Journal Chest 2 7 Circulation 4 13 European Heart Journal 2 7 Journal of the American College of Cardiology 2 7 The Lancet Respiratory Medicine 2 7 The New England Journal of Medicine 8 27 Others 10 33 Journal impact factor <10 8 27 >10 22 73 Clinical PH classification Group 1 20 67 Group 2 3 10 Group 3 2 7 Group 4 3 10 Any 2 7 Centre Multicentric 27 90 Unicentric 2 7 Not specified 1 3 Trial design Parallel 30 100 Superiority 30 100 Allocation concealment Yes 14 47 No 1 3 Unclear 15 50 Blinding Open label 1 3 Double-blinded 28 93 Not specified 1 3 Protocol freely available Yes 8 27 No 22 67 Nº patients randomized 1Total 7765 Median (range) 208 (52–1156) Nº arms Median (range) 2 (2–5) Type of primary endpoint 1 Time-to-event 5 17 Binary 2 7 Continuous 23 77 Trial met the primary endpoint 1Yes 19 63 No 8 27 PH: Pulmonary Hypertension. 1Extension trials were excluded from the descriptive analysis. J. Pers. Med. 2022,12, 863 6 of 14 Most studies were industry-funded (90%, n= 27). The most frequently selected journals for publication were The New England Journal of Medicine (n= 8) and Circulation (n= 4). Overall, 73% of the studies were published in high-impact journals (impact factor > 10). The most common PH type explored was type 1 (n= 20). The stated primary endpoint was statistically significant in 63% (n= 19) of trials. 3.2. Subgroup Analyses Characteristics of reported subgroup analyses are listed in Table 2. Subgroup analyses are mostly mentioned in the results and the discussion sections. Most trials, 57% (n= 17), did not clearly report the number of subgroup factors or subgroup analyses carried out. The remaining trials reported at least five subgroup factors or subgroup analyses in 37% (n= 11) and 40% (n= 12) of the trials, respectively. Subgroup analysis for more than one outcome was reported in 17% (n= 5) trials. Forest plots were used to report subgroup analysis data in 53% (n= 16) of the trials. Table 2. Characteristics of subgroup analysis reporting. Reporting of Subgroup Analysis Group 1 (n= 20) 1 Group 2 (n= 3) 1 Group 3 (n= 2) 1 Group 4 (n= 3) 1 Any (n= 2) 1 All Trials (n= 30) Mode of presentation Abstract 2 - - - 1 3 Methods 8 1 1 1 - 11 Results 17 3 2 3 2 27 Discussion 12 3 2 1 1 19 Supplementary Material 6 1 - 1 - 8 Nº subgroup factors 2–4 1 - - - 1 2 5–10 7 1 1 1 - 10 >10 1 - - - - 1 Unclear 11 2 1 2 1 17 Median (range) 6 (2–17) Nº subgroup analysis reported 2–4 - - - - 1 1 5–10 8 1 1 1 - 11 >10 1 - - - - 1 Unclear 11 2 1 2 1 17 Median (range) 7 (2–36) Nº subgroup outcomes 1 15 3 2 3 - 23 2–5 2 - - - 1 3 >5 2 - - - - 2 Unclear 2 - - - - 2 Median (range) 1 (1–12) Forest plot Yes 9 2 2 2 1 16 No 11 1 - 1 1 14 Prespecified or post hoc Prespecified 7 1 2 3 1 14 Post hoc 5 - - - - 5 Prespecified and post hoc 2 - - - - 2 Unclear 6 2 - - 1 9 Statistical method Descriptive 5 2 - 2 1 10 Subgroups P or CI 5 - 1 - - 6 Interaction test 8 1 1 1 - 11 Unclear 2 - - - 1 3 Subgroup claim Yes 9 1 - - 1 11 No 11 2 2 3 1 19 1 Patients included in the randomized controlled trials were classified according to the five pulmonary hypertension clinical groups [2]. CI: confidence interval. J. Pers. Med. 2022,12, 863 7 of 14 For 30% (n= 9) of trials, it was unclear whether subgroup analysis was pre-planned or post hoc; 47% (n= 14) of the trials were prespecified, 17% (n= 5) were post hoc, and 7% (n= 2) were prespecified and post hoc. Only 37% (n= 11) of the trials used an interaction test to assess heterogeneity of the treatment effect; 33% (n=10) reported subgroup analysis without any statistical analysis. The clinical trial protocol was available for 8 of the 30 RCTs included. Relevant differences were found for all 8 of the RCTs when comparing the trial protocol and the published manuscript: • Subgroup analyses: Six RCTs reported fewer subgroup analyses than prespecified in the trial protocol. The remaining two RCTs reported subgroup analyses that were not prespecified in the trial protocol; in both cases, these analyses were characterized as prespecified in the published manuscript. • Subgroup factors: The number of subgroup factors reported differed between the protocol and the published manuscript in seven cases: five RCTs reported fewer factors than those specified in the protocol. The remaining two added several subgroup factors that were not previously defined. • Selective reports of subgroup analyses by outcome: There were differences in the number of subgroup analyses reported for the primary outcome in seven RCTs. In addition, in four trial protocols, the authors specified that subgroup analysis would be carried out for primary and secondary endpoints; however, the published manuscript only reported the subgroup analyses for the primary endpoint on three of these RCTs. 3.3. Claims of Subgroup Effects Table 3lists the characteristics of RCTs with subgroup claims. In 11 RCTs [ 23 – 33 ], the authors claim heterogeneity of treatment effect in at least one subject subgroup. Two RCTs each made two claims of subgroup differences [ 32 , 33 ]. Of the RCTs with claims of a subgroup effect, 4 out of 11 reached the primary endpoint, 5 did not, and for the rest, a clear primary endpoint was not defined. Only three RCTs provided interaction test results to prove a subgroup difference. A total of 13 subgroup differences were claimed in 11 trials. The claims were classified as three (23%) strong claims, one (8%) claim of a likely effect, and nine (69%) suggestions of a possible effect. Table 4lists the 10 criteria to assess the credibility of subgroup claims as identified by strength. 3.4. Secondary Analyses Figure 2shows the evolution of the quality of the subgroup analyses reported over four periods of time: 2000–2004, 2005–2009, 2010–2014, and 2015–2019. J. Pers. Med. 2022, 12, x FOR PEER REVIEW 9 of 15 Subgroup effect consistent across related outcomes - - - - Compelling indirect evidence 1 (33%) 0 (0%) 5 (56%) 6 (46%) 1 Two trials claimed two subgroup claims each. 2 For those RCT that stated a subgroup effect without providing an interaction test, P interaction was calculated using the Joaquin Primo calculator [22] to verify that there was indeed statistical significance. 3.4. Secondary Analyses Figure 2 shows the evolution of the quality of the subgroup analyses reported over four periods of time: 2000–2004, 2005–2009, 2010–2014, and 2015–2019. An improvement was observed for most key methodological characteristics of PH- specific therapy RCTs over time, except for the use of subgroup variables as a stratification factor at randomization. Figure 2. Evolution of subgroup analysis reporting. 3.5. Risk of Bias The risk of bias graphs within studies and across studies are available in the supplemental material (Supplemental file S5). 3.6. Inter-Reviewer Agreement across Reviewers The inter-reviewer agreement for assessing the credibility of the subgroup claims was 0.88 (95% CI: 0.77–0.98), representing substantial to almost perfect agreement. 4. Discussion Subgroup analyses have the potential to generate investigation hypotheses, identify baseline factors that may influence treatment efficacy or toxicity, and help clinicians make clinical decisions for personalized care. However, misusing subgroup analyses may also lead to spurious findings and misleading interpretations [34–36]. The most frequent methodological limitations of subgroup analyses in RCTs have been reported extensively: multiple testing of hypotheses, inadequate statistical power, inappropriate a priori specification, and a lack of biological rationale [7,8,37–40]. This systematic review found that the subgroup analyses in RCTs of PH-specific therapy are generally of low quality, despite being published primarily in high-impact factor journals. For most clinical trials, the study protocol was not available; therefore, it was challenging for reviewers to verify critical aspects such as the prespecification of the subgroup analyses. Furthermore, only one RCT had available the trial protocol among those claiming a subgroup effect. Of those studies for which the protocol was available, the subgroup analyses reported in the manuscript lacked description and were significantly different from those planned in the protocol. Figure 2. Evolution of subgroup analysis reporting. J. Pers. Med. 2022,12, 863 8 of 14 Table 3. Characteristics of trials with claims of subgroup differences. Claims of Subgroup Difference Trials (n= 11) % Mode of presentation Abstract 4 36 Text only 7 64 Nº subgroup claims 1 9 82 2 2 18 Subgroup variable Primary endpoint 11 100 Forest plot Yes 2 18 No 9 82 Nº subgroup analysis 1–4 0 0 5–10 2 18 >10 1 9 Unclear 8 73 Median (range) 7 (7–12) Nº of outcomes for subgroup claims 1 8 73 2–5 1 9 >5 1 9 Unclear 1 9 Median (range) 1 (1–12) Statistical methods for subgroup analyses Descriptive 3 27 Subgroups P or CI 5 46 Interaction test 3 27 Prespecified/post hoc Prespecified 3 27 Post hoc 4 36 Prespecified and post hoc 1 9 Unclear 3 27 Protocol was freely available Yes 1 9 No 10 91 Table 4. Claims meeting Sun’s et al. subgroup criteria for primary outcomes. Criteria Strong Claim (n= 3) Claim of Likely Effect (n= 1) Suggestion of Effect (n= 9) All Claims (n= 13) Subgroup variable as a baseline characteristic 13 (100%) 1 (100%) 9 (100%) 13 (100%) Subgroup variable a stratification factor at randomization 0 (0%) 1 (100%) 2 (22%) 3 (23%) Subgroup hypothesis specified a priori 0 (0%) 0 (0%) 3 (33%) 3 (23%) A small number of hypothesized effects tested (</=5) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Significant interaction test (p< 0.05) 20 (0%) 0 (0%) 4 (45%) 4 (31%) Independence of interaction 1- - - - Direction of the subgroup effect correctly prespecified? 1 (33%) 0 (0%) 0 (0%) 1 (8%) Subgroup effect consistency across studies 2 (66%) 0 (0%) 6 (67%) 8 (62%) Subgroup effect consistent across related outcomes - - - - Compelling indirect evidence 1 (33%) 0 (0%) 5 (56%) 6 (46%) 1 Two trials claimed two subgroup claims each. 2 For those RCT that stated a subgroup effect without providing an interaction test, P interaction was calculated using the Joaquin Primo calculator [ 22 ] to verify that there was indeed statistical significance. An improvement was observed for most key methodological characteristics of PH- specific therapy RCTs over time, except for the use of subgroup variables as a stratification factor at randomization. J. Pers. Med. 2022,12, 863 9 of 14 3.5. Risk of Bias The risk of bias graphs within studies and across studies are available in the Supplemental Material (Supplemental File S5). 3.6. Inter-Reviewer Agreement across Reviewers The inter-reviewer agreement for assessing the credibility of the subgroup claims was 0.88 (95% CI: 0.77–0.98), representing substantial to almost perfect agreement. 4. Discussion Subgroup analyses have the potential to generate investigation hypotheses, identify baseline factors that may influence treatment efficacy or toxicity, and help clinicians make clinical decisions for personalized care. However, misusing subgroup analyses may also lead to spurious findings and misleading interpretations [ 34 – 36 ]. The most frequent methodological limitations of subgroup analyses in RCTs have been reported extensively: multiple testing of hypotheses, inadequate statistical power, inappropriate a priori specification, and a lack of biological rationale [7,8,37–40]. This systematic review found that the subgroup analyses in RCTs of PH-specific therapy are generally of low quality, despite being published primarily in high-impact factor journals. For most clinical trials, the study protocol was not available; therefore, it was challenging for reviewers to verify critical aspects such as the prespecification of the subgroup analyses. Furthermore, only one RCT had available the trial protocol among those claiming a subgroup effect. Of those studies for which the protocol was available, the subgroup analyses reported in the manuscript lacked description and were significantly different from those planned in the protocol. Other factors that stand out among the methodological errors when performing subgroup analyses were identified as follows: a high number of subgroup analyses reported, a high number of post hoc analyses, and the lack of an interaction test to confirm the existence of subgroup effects. When multiple subgroup analyses are carried out, the results obtained should be interpreted with caution since the probability of obtaining a false positive can be significantly augmented [ 8 ]. This risk may be increased, especially if the hypothesis of the subgroup analysis was not prespecified [ 8 , 10 , 36 ]. The approximate calculated risk for a false-positive result for five subgroup analyses is 25%; however, it may increase as the number of subgroup analyses rises. We identified a median of six subgroup analyses reported among the RCTs evaluated in this review. The prespecification of subgroup analysis is a frequent parameter measured in order to estimate methodological quality. For a subgroup analysis to be prespecified, it must be planned and documented before any examination of the data; this is based on the premise that a prespecified analysis usually follows a biological rationale. However, prespecification alone may not lead to solid subgroup analyses, as prespecified analyses may be based on unlikely and poorly formulated hypotheses [ 40 ]. In PH-specific therapy RCTs, 47% (14) of subgroup analyses were prespecified. In addition to the prespecification of the subgroup analyses, the correct direction of subgroup hypotheses must also be specified. For those claims in which the direction of the effect is not identified or is wrongly identified, their credibility could be reduced. A common mistaken belief among authors is to claim a subgroup difference when a statistically significant effect is found in one subgroup but not in the other. One of the essential criteria to appropriately establish a claim of a subgroup effect is performing an interaction test [ 41 ]. The p-value of an interaction test provides information about the probability that the existence of a subgroup difference is due to an accidental finding or chance rather than an actual subgroup effect. In this review, we observed that only 38% of the RCTs performed an interaction test to confirm the existence of a subgroup effect. Of the 9 claims of subgroup difference identified in this study, 44% (n= 4) were based on a significant interaction test. We found mixed results when compared with similar studies in other clinical areas. Wallach et al. identified that among a sample of articles that made at