The risks of converting post-hoc findings into primary outcomes in subsequent trials

Page 1 o 5 © Annals o T ansla ional Medicine. All igh s ese ed. Ann T ansl Med 2019;7(Suppl 8):S337 | h p://dx.doi.o g/10.21037/a m.2019.09.105 Edi o ial Commen a y The isks o con e ing pos -hoc indings in o p ima y ou comes in subsequen ials Filipe B. Rod igues1,2,3, Joaquim J. Fe ei a2,3,4 1UCL Hun ing on’s Disease Cen e, UCL Queen Squa e Ins i u e o Neu ology, Uni e si y College London, London, UK; 2Labo a o y o Clinical Pha macology and The apeu ics, Faculdade de Medicina, Uni e sidade de Lisboa, Lisbon, Po ugal; 3Ins i u o de Medicina Molecula , Lisbon, Po ugal; 4CNS—Campus Neu ológico Sénio , To es Ved as, Po ugal Co espondence o: Joaquim J. Fe ei a. Labo a o y o Clinical Pha macology and The apeu ics, Faculdade de Medicina, Uni e sidade de Lisboa, Lisbon, Po ugal. Email: [email p o ec ed]. P o enance: This is an in i ed a icle commissioned by he Academic Edi o D . Zhenxiang Zhao (Depa men o Neu ology, Henan P o incial People’s Hospi al, People’s Hospi al o Zhengzhou Uni e si y, People’s Hospi al o Henan Uni e si y, Zhengzhou, China). Commen on: Reilmann R, McGa y A, G ache ID, e al. Sa e y and e icacy o p idopidine in pa ien s wi h Hun ing on's disease (PRIDE-HD): a phase 2, andomised, placebo-con olled, mul icen e, dose- anging s udy. Lance Neu ol 2019;18:165-76. Submi ed Sep 03, 2019. Accep ed o publica ion Sep 18, 2019. doi: 10.21037/a m.2019.09.105 View his a icle a : h p://dx.doi.o g/10.21037/a m.2019.09.105 Hun ing on’s disease (HD) is a de as a ing neu odegene a i e condi ion caused by a iple epea expansion o he hun ing in gene (1). Al hough a e i is amongs he mos equen au osomal dominan causes o demen ia, equen ly a ec ing indi iduals in he mos p oduc i e decades o hei li es. Clinically, i is cha ac e ized by a classic iad o luc ua ing neu opsychia ic symp oms, and p og essi e mo emen and cogni i e diso de s, accompanied by o he symp oms such as weigh loss and sleep impai men . I is se e ely debili a ing, has a huge impac on quali y o li e and is a al, wi h a median su i al a e mo o onse o a ound a qua e o a cen u y (2). The culp i gene ic de ec was i s assigned o ch omosome 4 in 1983 (3) and sequenced in 1993 (4). Whils g ea hope had been placed on his disco e y, nea ly h ee decades la e he e is s ill no cu e o any in e en ion ha delays o s ops disease p og ession (5). Ne e heless, se e al p omising compounds a e cu en ly ac i e in he d ug de elopmen pipeline (6). Wi h ega ds o symp om managemen he he apeu ic a mamen a ium is b oad, al hough ew in e en ions a e suppo ed by high quali y e idence (7). As an example, o he hype kine ic mo emen diso de (i.e., cho ea) cha ac e is ic o HD, he e a e cu en ly wo FDA-app o ed medica ions— e abenazine and deu e abenazine— al hough uncla i y s ill exis s abou how dissimila hei e icacy and sa e y p o iles a e (8,9). HD is a p omising disease model o s udy neu odegene a ion, due o se e al cha ac e is ics, including: a p ecise pa hogenic agen (4), a na u al his o y ha comp ises a long p esymp oma ic phase (10,11) ollowed by an ex ended diseased su i al (12), a b oad symp oma ic spec um including mos o he clinical ea u es p esen on neu odegene a i e diseases (mo o , beha iou al and cogni i e) (1), well- alida ed assessmen ools (13-16), pionee ing imaging and bio luid bioma ke s (10,17), and well-o ganized esea ch and pa ien ne wo ks (18). These ac o s ha e encou aged d ug de elope s o in es in HD as shown by he p opo iona ely high numbe o clinical ials conduc ed in his a e disease popula ion. Un o una ely, he success a e o he de elopmen pipeline unde pe o ms when compa ing wi h o he diso de s (5). Gene ic in e en ions a e only now coming o age due o ecen b eak h oughs in DNA and RNA manipula ion echniques, and op imiza ion o d ug s abili y, immunogenici y and deli e y. The e is op imism ha hese new ools may help change he a e o diseases like Hun ing on’s. The ideal d ug de elopmen p og am needs o happen wi h minimal human and inancial bu den and maximal e iciency. This includes in o ma i e p eclinical da a and ea ly phase ial esul s, bu also ea ly “go/no go” decision imings and c i e ia. P idopidine is an in e es ing molecule om he pha macological pe spec i e (Figu e 1). Ac ing on he 337 Rod igues and Fe ei a. Con e ing pos -hoc indings in o p ima y ou comes © Annals o T ansla ional Medicine. All igh s ese ed. Ann T ansl Med 2019;7(Suppl 8):S337 | h p://dx.doi.o g/10.21037/a m.2019.09.105 Page 2 o 5 dopamine gic sys em—which is p ima ily in ol ed in he genesis o he disease pheno ype—i is classi ied as a modula o as i can ha e bo h an agonis ic and an agonis ic e ec s h ough a s a e-dependen e ec on dopamine ecep o s. When he e is o e ac i i y i ac s as an an agonis , unde unde ac i i y condi ions as an agonis , and has li le o no in luence unde physiological ci cums ances. In addi ion, i also seems o in e ac wi h N-Me hyl-D- aspa a e (NMDA) and sigma-1 ecep o s. Thus a p idopidine has been es ed in ou successi e andomized con olled ials (RCTs) in HD, en olling a g and o al o o e 1,000 pa icipan s (Table 1) (19,20,22,24). Un o una ely, in all o hem he p ima y ou come was no me . A i s , a small RCT es ed 50 mg o p idopidine a day agains placebo in 58 people wi h mani es HD, ec ui ed om 6 cen es in Sweden and No way. The ial ollowed pa icipan s o 4 weeks and was no able o show an e ec on i s p ima y ou come, a composi e cogni i e sco e. Ne e heless, seconda y and explo a o y analyses e ealed a nominal imp o emen in he modi ied mo o sco e (mMS), especially in a subg oup o mo e se e ely a ec ed pa icipan s. This subsco e o he Uni ied Hun ing on’s Disease Ra ing Scale (UHDRS) o al mo o sco e (TMS) comp ises i ems ela ed wi h olun a y mo o con ol (i.e., i ems 4–10 and 13–15) (19). Building up on hese esul s, wo ollow-up andomized con olled s udies ensued: he Me maiHD s udy in 32 Eu opean cen es and he HART s udy in 27 No h Ame ican cen es. Based on he assump ion ha p idopidine could ha e an e ec on olun a y mo emen con ol, bo h we e designed and powe ed o show a di e ence in he mMS and ec ui ed people wi h mani es HD and a mMS o 10 o mo e. The Me maiHD s udy ec ui ed 437 pa icipan s o in es iga e 45 and 90 mg o p idopidine daily compa ed wi h placebo o e 26 weeks. Alas, his ial did no show an e ec o i s p ima y and seconda y ou comes. None heless, he analyses we e s a is ically signi ican o he compa ison 90 mg e sus placebo in he pe -p o ocol sample (i.e., 70% o g ea e compliance wi h ea men and comple ed he s udy) o he mMS and in he in en ion- o- ea sample o he UHDRS TMS (20). The HART s udy had a simila design and es ed 3 dosages o p idopidine (20, 45 o 90 mg daily) agains placebo in 227 pa icipan s. The p ima y ou come e alua ed a 12 weeks depic ed no di e ences be ween he di e en dosages and he placebo a m, howe e seconda y analyses showed a signi ican di e ence o UHDRS TMS o he compa ison 90 mg e sus placebo (22). Las ly, he PRIDE-HD s udy ec ui ed 408 pa icipan s wi h mani es HD, a leas 25 poin s on he UHDRS TMS and 90% o less in he UHDRS independence sco e (IS), om 53 si es ac oss 12 coun ies in Eu ope, No h Ame ica and Aus alia. Follow-up was 53 weeks, and ou doses o p idopidine (45, 67.5, 90 and 112.5 mg daily) we e es ed agains placebo. The p ima y ou come was changed in he UHDRS TMS a 26 weeks and ailed o be achie ed. Explo a o y analyses o all es ed dosages showed simila esul s a 52 weeks. Explo a o y analyses e ealed an e ec on he UHDRS o al unc ional capaci y (TFC) in he 45 mg a m. Subg oup pos -hoc es s ound his e ec o be mo e e iden in pa icipan s in ea lie disease s ages (24). To summa ise, he clinical de elopmen pipeline o p idopidine began wi h a nega i e, ela i ely small and sho -las ing ial aimed a cogni ion bu wi h in e es ing indings on a olun a y mo emen s’ seconda y ou come. I was ollowed by wo well-powe ed bu also nega i e ials designed o in es iga e he e ec s on olun a y mo emen s. Bo h showed di e ences in a semi-s uc u ed neu ological exam scale a he highes es ed dosage (90 mg daily). A o h well-powe ed ial was deployed o in es iga e he e ec s on mo o signs ac oss a ange o dosages. The ial was also nega i e, and none o he dosages shaped compelling di e ences a e 6 mon hs and 1 yea on mo o signs, bu explo a o y in es iga ions disclosed an e ec on unc ional capaci y wi h low-dose p idopidine in ea ly disease. The cumula i e e idence om hese ou ials seems o suppo ha p idopidine is ela i ely sa e and well- ole a ed. The a p io i hypo hesis ha i has an e ec on he cogni i e ea u es o HD has been p o en alse. The indings on mo o e ec s lea n om he ea lies ials we e no eplica ed in a la ge buil - o -pu pose ial. We e he i s ones spu ious posi i e esul s s emming om explo a o y analyses, o we e he esul s o he la e ials jus un o una e? The in es iga o s blame an unexpec edly high esponse by he placebo g oup in he p ima y Figu e 1 P idopidine molecule. Annals o T ansla ional Medicine, Vol 7, Suppl 8 Decembe 2019 Page 3 o 5 © Annals o T ansla ional Medicine. All igh s ese ed. Ann T ansl Med 2019;7(Suppl 8):S337 | h p://dx.doi.o g/10.21037/a m.2019.09.105 Table 1 Clinical ials’ cha ac e is ics Name NCT N Design Popula ion Du a ion (weeks) Ac i e a m(s) Compa a o P ima y ou come Seconda y/ explao a o y ou comes Sponso Resul s Lundin e al. 2010 (19) N/A 58 RCT Mani es HD 4 P idopidine 50 mg/d (n=28) Placebo (n=30) Weigh ed cogni i e sco e a 4 weeks* Mo o , beha iou al, sleep, cogni ion, sa e y and ole abili y Neu oSea ch Sweden AB NS ∆ in p ima y ou come, and ∆ on mMS (baseline mMS ≥10) Me maiHD (20) NCT00665223 437 RCT Mani es HD (mMS ≥10) 26 P idopidine 45 (n=148) and 90 mg/d (n=145) Placebo (n=144) mMS a 26 weeks Mo o , beha iou al, cogni ion, sa e y and ole abili y Neu oSea ch A/S NS ∆ in p ima y ou come, and ∆ on UHDRS TMS (90 mg/d) and mMS (90 mg/d, pe -p o ocol) Squi ie i e al. 2013 (21) N/A 353 OLE Comple ion o Me maiHD 26 P idopidine 90 mg/d (n=353) N/A Sa e y and ole abili y a 52 weeks** Adhe ence Neu oSea ch A/S P idopidine is sa e and well- ole a ed HART (22) NCT00724048 227 RCT Mani es HD (mMS ≥10) 12 P idopidine 20 (n=56), 45 (n=55) and 90 mg/d (n=58) Placebo (n=58) mMS a 12 weeks Mo o , unc ion, beha iou al, cogni ion, sa e y and ole abili y Neu oSea ch A/S, Neu o- Sea ch Sweden AB NS ∆ in p ima y ou come, and ∆ on UHDRS TMS (90 mg/d) Open-HART (23) NCT01306929 118 OLE Comple ion o HART 156 P idopidine 90 mg/d (n=118) N/A Sa e y and ole abili y a 156 weeks Mo o and unc ion Te a Pha maceu ical Indus ies P idopidine is sa e and well- ole a ed PRIDE-HD (24) NCT02006472 408 RCT Mani es HD (TMS ≥25 & IS ≤90) 52 P idopidine 45 (n=81), 67.5 (n=82), 90 (n=81), and 112.5 mg/d (n=82) Placebo (n=82) TMS a 26 weeks Mo o , unc ion, beha iou al, cogni ion, quali y o li e, sa e y and ole abili y Te a Pha maceu ical Indus ies NS ∆ in p ima y ou come, ∆ on UHDRS TMS (45 mg/d) Open PRIDE-HD NCT02494778 248 OLE Comple ion o PRIDE-HD 364 P idopidine 90 mg/d (ongo- ing) N/A Sa e y and ole abili y a 364 weeks N/A P ilenia S udy e mina ed bu ye no epo ed *, based on Symbol Digi Modali ies Tes , Ve bal Fluency Ca ego ical, S oop Colo Naming, S oop Wo d Reading and S oop In e e ence Tes ; **, double blinded phase plus OLE; ∆, change. N/A, no applicable o a ailable; RCT, andomized con olled ial; HD, Hun ing on’s disease; NS, non-signi ican ; mMS, modi ied mo o sco e (UHDRS TMS i ems 4–10 and 13–15); UHDRS TMS, Uni ied Hun ing on’s Disease Ra ing Scale o al mo o sco e; OLE, open-label ex ension. Rod igues and Fe ei a. Con e ing pos -hoc indings in o p ima y ou comes © Annals o T ansla ional Medicine. All igh s ese ed. Ann T ansl Med 2019;7(Suppl 8):S337 | h p://dx.doi.o g/10.21037/a m.2019.09.105 Page 4 o 5 ou come. Indeed by he end o he s udy pe iod, his was he a m wi h he la ges e ec size. The sponso , Te a Pha maceu ical Indus ies L d., ocused hei a en ion on explo a o y and pos -hoc subg oup analyses concluding ha “p idopidine demons a es slowing o p og ession o Hun ing on disease in PRIDE-HD s udy as measu ed by To al Func ional Capaci y”. A e spa ked c i icism om he clinical and scien i ic communi y, awa e o he po en ial damages caused by such in e p e a ions, he Eu opean Hun ing on’s Disease Ne wo k oned down he sponso ’s s a emen explaining ha “ his should no be misunde s ood as a demons a ion o disease modi ica ion o o neu op o ec ion”. While he e a e mul iple possible jus i ica ions o hese esul s, one should always conside he possibili y ha p idopidine, as any o he compound in de elopmen , may no induce he hypo hesised clinical e ec . The cen al ne ous sys em he apeu ic a ea has a low success a e om i s -in- man o egis a ion o a ound 7–8% compa ing wi h he g and mean o 11%, and o he he apeu ic a eas such as ca dio ascula whe e he success a e is a ound 20% (25). In neu ology, he d ug de elopmen pipeline pe o ms app eciably poo ly in he phase III and egis a ion phases (25). In HD only 2 molecules su i ed hese phases, and o e all he success a e is e en lowe han ha o o he he apeu ic a eas (5). Many easons ha e been hypo hesized o explain such phenomenon: incomple e unde s anding o he disease physiopa hology; weak associa ion be ween he he apeu ic a ge , and he disease pa hogenesis and na u al his o y; limi ed animal models; incomple e e alua ion o p eclinical e ec s; subop imal pha macokine ic and pha macodynamic cha ac e is ics; limi a ions o cu en s udy designs (no bioma ke s, sho s udy du a ions); unen husias ic comme cial in e es s, among o he s. P idopidine is an example whe e se e al o hese ac o s came in o play, hal ing he al eady low chances o i s -in- man o egis a ion success. I is well s ablished ha only a small p opo ion o science gene a es posi i e esul s, including igo ous and well- epo clinical ials (26). While he scien i ic milieu should ewa d p og ess, indus y and esea che s a e o en mo i a ed by o he ac o s. As emp ing as i may sound o ega d hese a e - he-e en announcemen s ele an , his o y has hough us ha hypo heses and new ials gene a ed by pos -hoc posi i e esul s a e legi ima e bu may be w ong. Unde s andably and in he absence o be e app oaches, his s a egy is equen ly used ac oss medicine. P idopidine seems unlikely o be clinically help ul o people wi h HD bu we hope his s o y will each us abou he design o d ug de elopmen p og ammes and wha o a oid in he u u e in o de o deli e e icacious medicines and op imise d ug de elopmen . Acknowledgmen s None. Foo no e Con lic s o In e es : The au ho s ha e no con lic s o in e es o decla e. E hical S a emen : The au ho s a e accoun able o all aspec s o he wo k in ensu ing ha ques ions ela ed o he accu acy o in eg i y o any pa o he wo k a e app op ia ely in es iga ed and esol ed. Re e ences 1. McColgan P, Tab izi SJ. Hun ing on's disease: a clinical e iew. Eu J Neu ol 2018;25:24-34. 2. Rod igues FB, Ab eu D, Damásio J, e al. Su i al, mo ali y, causes and places o dea h in a Eu opean Hun ing on's disease p ospec i e coho . Mo Diso d Clin P ac 2017;4:737-42. 3. Gusella JF, Wexle NS, Conneally PM, e al. A polymo phic DNA ma ke gene ically linked o Hun ing on's disease. Na u e 1983;306:234-8. 4. A no el gene con aining a inucleo ide epea ha is expanded and uns able on Hun ing on's disease ch omosomes. The Hun ing on's Disease Collabo a i e Resea ch G oup. Cell 1993;72:971-83. 5. T a essa AM, Rod igues FB, Mes e TA, e al. Fi een yea s o clinical ials in Hun ing on's disease: a e y low clinical d ug de elopmen success a e. J Hun ing ons Dis 2017;6:157-63. 6. Rod igues FB, Fe ei a JJ, Wild EJ. Hun ing on's disease clinical ials co ne : June 2019. J Hun ing ons Dis 2019;8:363-71. 7. Bachoud-Lé i AC, Fe ei a J, Massa R, e al. In e na ional guidelines o he ea men o Hun ing on's disease. F on Neu ol 2019;10:710. 8. Rod igues FB, Dua e GS, Cos a J, e al. Te abenazine e sus deu e abenazine o Hun ing on's disease: wins o dis an cousins? Mo Diso d Clin P ac 2017;4:582-5. 9. Rod igues FB, Dua e GS, Cos a J, e al. Me a- esea ch me ics ma e : le e ega ding a icle "indi ec ole abili y Annals o T ansla ional Medicine, Vol 7, Suppl 8 Decembe 2019 Page 5 o 5 © Annals o T ansla ional Medicine. All igh s ese ed. Ann T ansl Med 2019;7(Suppl 8):S337 | h p://dx.doi.o g/10.21037/a m.2019.09.105 compa ison o deu e abenazine and e abenazine o Hun ing on disease". J Clin Mo Diso d 2017;4:19. 10. Tab izi SJ, Scahill RI, Owen G, e al. P edic o s o pheno ypic p og ession and disease onse in p emani es and ea ly-s age Hun ing on's disease in he TRACK-HD s udy: analysis o 36-mon h obse a ional da a. Lance Neu ol 2013;12:637-49. 11. Paulsen JS, Hayden M, S ou JC, e al. P epa ing o p e en i e clinical ials: he P edic -HD s udy. A ch Neu ol 2006;63:883-90. 12. Rod igues FB, Fe ei a JJ. O e all su i al and causes o dea h in neu odegene a ion-an o e looked and unde epo ed heme. JAMA Neu ol 2017;74:1379. 13. Mes e TA, Busse M, Da is AM, e al. Ra ing scales and pe o mance-based measu es o assessmen o unc ional abili y in Hun ing on's disease: c i ique and ecommenda ions. Mo Diso d Clin P ac 2018;5:361-72. 14. Mes e TA, Ca lozzi NE, Ho AK, e al. Quali y o li e in Hun ing on's disease: c i ique and ecommenda ions o measu es assessing pa ien heal h- ela ed quali y o li e and ca egi e quali y o li e. Mo Diso d 2018;33:742-9. 15. Mes e TA, Fo jaz MJ, Mahlknech P, e al. Ra ing scales o mo o symp oms and signs in Hun ing on's disease: c i ique and ecommenda ions. Mo Diso d Clin P ac 2018;5:111-7. 16. Mes e TA, an Duijn E, Da is AM, e al. Ra ing scales o beha io al symp oms in Hun ing on's disease: c i ique and ecommenda ions. Mo Diso d 2016;31:1466-78. 17. By ne LM, Rod igues FB, Johnson EB, e al. E alua ion o mu an hun ing in and neu o ilamen p o eins as po en ial ma ke s in Hun ing on's disease. Sci T ansl Med 2018. doi: 10.1126/sci anslmed.aa 7108. 18. Landweh meye GB, Fi ze -A as CJ, Giuliano JD, e al. Da a analy ics om en oll-HD, a global clinical esea ch pla o m o Hun ing on's disease. Mo Diso d Clin P ac 2016;4:212-24. 19. Lundin A, Die ichs E, Haghighi S, e al. E icacy and sa e y o he dopamine gic s abilize P idopidine (ACR16) in pa ien s wi h Hun ing on's disease. Clin Neu opha macol 2010;33:260-4. 20. de Yebenes JG, Landweh meye B, Squi ie i F, e al. P idopidine o he ea men o mo o unc ion in pa ien s wi h Hun ing on's disease (Me maiHD): a phase 3, andomised, double-blind, placebo-con olled ial. Lance Neu ol 2011;10:1049-57. 21. Squi ie i F, Landweh meye B, Reilmann R, e al. One- yea sa e y and ole abili y p o ile o p idopidine in pa ien s wi h Hun ing on disease. Neu ology 2013;80:1086-94. 22. Hun ing on S udy G oup HART In es iga o s. A andomized, double-blind, placebo-con olled ial o p idopidine in Hun ing on's disease. Mo Diso d 2013;28:1407-15. 23. McGa y A, Kiebu z K, Able V, e al. Sa e y and explo a o y e icacy a 36 mon hs in open-HART, an open-label ex ension s udy o p idopidine in Hun ing on's disease. J Hun ing ons Dis 2017;6:189-99. 24. Reilmann R, McGa y A, G ache ID, e al. Sa e y and e icacy o p idopidine in pa ien s wi h Hun ing on's disease (PRIDE-HD): a phase 2, andomised, placebo- con olled, mul icen e, dose- anging s udy. Lance Neu ol 2019;18:165-76. 25. Kola I, Landis J. Can he pha maceu ical indus y educe a i ion a es? Na Re D ug Disco 2004;3:711-5. 26. Muna ò MR, Nosek BA, Bishop DVM, e al. A mani es o o ep oducible science. Na u e Human Beha iou 2017;1:0021. Ci e his a icle as: Rod igues FB, Fe ei a JJ. The isks o con e ing pos -hoc indings in o p ima y ou comes in subsequen ials. Ann T ansl Med 2019;7(Suppl 8):S337. doi: 10.21037/a m.2019.09.105