Evaluation of appendicitis risk prediction models in adults with suspected appendicitis

Original article Evaluation of appendicitis risk prediction models in adults with suspected appendicitis RIFT Study Group on behalf of the West Midlands Research Collaborative* Correspondence to: Mr A. Bhangu, Academic Department of Surgery, University of Birmingham, Heritage Building, Mindelsohn Way, Birmingham B15 2TH, UK (e-mail: [email protected]) Background: Appendicitis is the most common general surgical emergency worldwide, but its diagnosis remains challenging. The aim of this study was to determine whether existing risk prediction models can reliably identify patients presenting to hospital in the UK with acute right iliac fossa (RIF) pain who are at low risk of appendicitis. Methods: A systematic search was completed to identify all existing appendicitis risk prediction models. Models were validated using UK data from an international prospective cohort study that captured consecutive patients aged 16–45 years presenting to hospital with acute RIF in March to June 2017. The main outcome was best achievable model specificity (proportion of patients who did not have appendicitis correctly classified as low risk) whilst maintaining a failure rate below 5 per cent (proportion of patients identified as low risk who actually had appendicitis). Results: Some 5345 patients across 154 UK hospitals were identified, of which two-thirds (3613 of 5345, 67⋅6 per cent) were women. Women were more than twice as likely to undergo surgery with removal of a histologically normal appendix (272 of 964, 28⋅2 per cent) than men (120 of 993, 12⋅1 per cent) (relative risk 2⋅33, 95 per cent c.i. 1⋅92 to 2⋅84; P<0⋅001). Of 15 validated risk prediction models, the Adult Appendicitis Score performed best (cut-off score 8 or less, specificity 63⋅1 per cent, failure rate 3⋅7 per cent). The Appendicitis Inflammatory Response Score performed best for men (cut-off score 2 or less, specificity 24⋅7 per cent, failure rate 2⋅4percent). Conclusion: Women in the UK had a disproportionate risk of admission without surgical intervention and had high rates of normal appendicectomy. Risk prediction models to support shared decision-making by identifying adults in the UK at low risk of appendicitis were identified. ∗Co-authors of this study are listed in Appendix S1 (supporting information). Presented to the 13th Scientific and Annual Meeting of the European Society of Coloproctology, Nice, France, September 2018, and the 2018 International Surgical Conference of the Association of Surgeons in Training, Edinburgh, UK, April 2018; published in abstract form as Colorectal Dis 2018; 20(Suppl 4): 76 Paper accepted 29 October 2019 Published online in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.11440 Introduction Acute appendicitis is the most common general surgical emergency worldwide1, but its diagnosis remains challenging, particularly in young women for whom there is a broader range of differential diagnoses than for men2. A key concern is overtreatment, in the form of normal appendicectomy (removal of a histologically normal appendix), which may be associated with increased postoperative complications, duration of hospital stay and healthcare costs compared with diagnostic laparoscopy alone3–5. There is little consensus on optimal diagnostic pathways for patients presenting with acute right iliac fossa (RIF) pain in the UK. Traditionally, concerns over the radiation exposure associated with CT have limited its routine use6. Although modern low-dose protocols have reduced radiation exposure whilst maintaining diagnostic performance7,8, CT rates are lower in the UK than in many other high-income countries, and this is associated with higher normal appendicectomy rates (NARs)9. To improve diagnosis of appendicitis, international guidelines10,11 recommend routine clinical risk scoring. Although the Appendicitis Inflammatory Response (AIRS) and Alvarado scores are recommended most frequently, there is little evidence to support their application, as they © 2019 The Authors. BJS published by John Wiley & Sons Ltd on behalf of BJS Society Ltd. BJS 2020; 107:73–86 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. 74 RIFT Study Group have been inconsistently and poorly validated12. None is in routine clinical use13. Consequently, although guidelines recommend that patients at low risk of appendicitis should not be admitted routinely to hospital13, diagnostic uncertainty may lead to patients being admitted for observation, increasing the likelihood of overtreatment and healthcare-related costs. Robustly validated risk prediction models that identify groups at low risk of appendicitis could help to standardize clinical assessment and inform patient and doctor decision-making. In turn, this may reduce hospital admissions, overtreatment and healthcare costs14. To influence wide-scale change, a prospective multicentre study including patients presenting with RIF pain was developed to assess whether existing risk prediction models can identify low-risk patients suitable for ambulatory management. This study represents the first planned analysis from the Right Iliac Fossa Treatment (RIFT) Study Group15. The aim of this study was to identify the optimal risk prediction model to identify young patients (aged 16–45 years) in the UK at low risk of acute appendicitis. Children and adults aged over 45 years follow distinct management pathways and will be addressed in subsequent preplanned analyses. Methods This study was undertaken in two parts. First, a systematic literature search was performed to identify all available risk prediction models for acute appendicitis. This is reported according to the PRISMA guidelines16. Second, a multicentre prospective observational cohort study was performed to collect accurate data for validation of these risk prediction models. This is reported according to Standards for Reporting Diagnostic Accuracy (STARD) guidelines17 for diagnostic accuracy studies. The RIFT Study15 captured data in the UK, Italy, Portugal, Republic of Ireland and Spain. Clinical risk score validation was preplanned to be performed in the UK for patients presenting with acute RIF pain. National differences in clinical pathways mean that analyses must be stratified by country. At the time of planning the analysis, the NAR was anticipated to be around 20 per cent in the UK18, but under 5 per cent in other European countries14,19. Given the low baseline NAR, there is little clinical need for risk scoring in European populations, so validation of risk scores in those patients would be unlikely to change clinical practice. Therefore, clinical risk score validation was preplanned to be performed in the UK only, although observed NARs from across all participating countries are presented for context. Future analyses are planned to explore variation in imaging rates across Europe. Identifying risk scoring models Systematic searches of MEDLINE and Web of Science were performed using the search terms [‘appendicitis’ OR ‘appendectomy’ OR ‘appendicectomy’] AND [‘score’ OR ‘model’ OR ‘scoring’ OR ‘nomogram’]. These results were supplemented by additional unstructured searches of Google Scholar. Reference lists from relevant articles were hand-searched for eligible studies, as were recent World Society of Emergency Surgery and Royal College of Surgeons (England) Emergency Surgery guidelines10,13.No date restrictions were set. The search was last updated on 25 July 2018. Studies were eligible for inclusion if they described a risk prediction model for the diagnosis of acute appendicitis that used clinical data and/or routine laboratory tests (full blood count, C-reactive protein, liver function tests). As the aim of the study was to identify model(s) that could be used at initial surgical assessment, models were excluded if they relied on radiological investigations or non-routine laboratory tests that are not typically available at the point of first surgical contact. Models that aimed to differentiate simple from complex appendicitis were also excluded. Only English-language articles that provided sufficient information to replicate their clinical risk model algorithm were included. Titles and abstracts were screened, followed by review of full texts of relevant articles. Study selection and data extraction were completed independently by two authors, with any disagreements resolved through discussion with a third author. Study dissemination Data collection was conducted according to a prespecified, published protocol15. The protocol was disseminated through surgical trainee-led research collaboratives. Any hospital providing acute general surgical services was eligible to participate. Study approval As this observational study collected routine, anonymized data with no change to clinical care pathways, lead investigators at participating UK centres registered the study locally as either clinical audit or service evaluation. In the Republic of Ireland, Italy, Portugal and Spain, local lead investigators were responsible for arranging research ethics committee or institutional approval locally, as appropriate. © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. Identifying adults at low risk of appendicitis 75 Patient selection Eligible patients were identified from participating centres during one of four prespecified 2-week study periods between 13 March 2017 and 18 June 2017. During each interval, all consecutive patients referred by a general practitioner or emergency physician to the on-call surgeon’s team with suspected appendicitis or acute RIF pain were identified at the point of admission to the surgical unit. Patients who had previously undergone either a therapeutic appendicectomy or an incidental appendicectomy as part of another procedure were excluded. Pregnant women have significantly different clinical needs to other patients and were therefore excluded. Pregnancy was identified by patients’ self-report. The possibility of pregnancy was further excluded by most women undergoing urinalysis. Patients who underwent appendicectomy for whom histological findings were not available were excluded, as it was not possible to determine the underlying diagnosis (appendicitis versus normal appendicectomy) for them. Patients who were managed without surgery for acute appendicitis following a positive CT or MRI diagnosis of appendicitis were also excluded, as the diagnosis was not confirmed histologically. Diagnosis of appendicitis A diagnosis of acute appendicitis was confirmed if within 30 days of enrolment in the study the patient had excision of the appendix with postoperative histological examination confirming acute appendicitis. Patients who underwent right hemicolectomy for presumed acute appendicitis were pooled with those who had appendicectomy. Based on the original histopathology report, patients were classified as having either simple or complex (gangrenous, perforated) appendicitis2. The NAR was calculated as patients with normal appendix histology as a proportion of all patients who had an appendicectomy. Patients with appendix pathology other than appendicitis (such as appendix tumour) were included in the denominator but not the numerator. Data collection Data were collected using standardized case report forms (CRFs) by teams of up to three investigators per 2-week period. The CRF was designed to be completed at the patient bedside. A large number of variables have been proposed to predict appendicitis. Therefore, to ensure feasibility of data collection, the CRF was designed to collect the data points required for the four most common adult risk prediction models identified in international guidelines10. Patient-level variables collected included age, sex, clinical symptoms and examination findings, urinalysis and blood test results. Data were collected on ultrasound, CT or MRI use, and whether these tests were positive for appendicitis (diagnosis of appendicitis in the formal radiology report). If the patient had surgery, the procedure, operative findings and histopathology results were recorded. Patients were followed during their initial admission, during any subsequent hospital admissions within 30 days of initial presentation, and then at 30 days after initial presentation using a combination of electronic and paper hospital records. Data integrity Multiple strategies were used to ensure accurate data collection. A supervising consultant surgeon at each hospital oversaw study conduct and was responsible for overall quality assurance of submitted data. To ensure data completeness, before locking of the online database, local lead investigators were contacted with specific details of missing data. Participating sites voluntarily identified independent data validators who had not been involved in the initial data collection. Data accuracy was determined by review of the following key data fields against the original clinical records for enrolled patients: whether the patient had undergone surgery; whether the patient had been readmitted within 30 days of index admission; and histopathological results, if applicable. The data accuracy rate was defined as the proportion of validated data fields that was recorded correctly. Where incorrect data were identified, validators were asked to amend those data points on the study database. Validation of risk prediction models and statistics The development of the statistical analysis plan is summarized in Appendix S2 (supporting information). Risk prediction models were validated if patients could be scored with the data points available. As there are distinct differential diagnoses for RIF pain in women, it was preplanned to stratify risk prediction model validation by sex. For maximum clinical impact, the ideal risk score would classify as many patients as possible as being at low risk of appendicitis (true negatives), but not at the expense of missing significant numbers of patients with appendicitis (false negatives). The clinical performance of each score was therefore evaluated in terms of failure rate and specificity. The failure rate is the false-negative rate: the proportion of patients stratified to the low-risk group who actually have appendicitis (false negatives/(true negatives +false negatives)). Before analysis, a modified Delphi exercise (Appendix S2, supporting information) amongst 24 © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. 76 RIFT Study Group experienced UK general surgeons agreed the maximum acceptable failure rate to be 5 per cent. Specificity is the proportion of patients who do not have appendicitis who were stratified to the low-risk group (true negatives/(true negatives +false positives)). The main outcome measure for evaluating each risk prediction model was the best achievable specificity whilst maintaining a failure rate of less than 5 per cent. The overall ability of the risk prediction models to discriminate between patients with and without acute appendicitis was determined by calculation of the area under the receiver operating characteristic (ROC) curve (AUC). Analyses were carried out in Stata®version 15 (Stata- Corp, College Station, Texas, USA). Handling of missing data As surgical collaborative cohort studies have been completed with very low rates of missing data, an overall incomplete data rate of under 2⋅5 per cent was anticipated. Therefore, there was no plan to impute missing data. A complete case analysis (list-wise deletion) was performed, and preplanned sensitivity analyses: missing data points scored as zero, representing what would happen in normal clinical practice and providing a scenario that would underestimate appendicitis risk; missing data were scored with maximum applicable points. Results The systematic search identified 26 risk prediction models (Fig.1). Fifteen of these could be validated with the data collected in the cohort study20–34 (Table 1 ). These risk prediction models were based on 17 clinical parameters (Table S1, supporting information). The other 11 models could not be validated owing to the data set lacking specific variables35–45 (Table S2, supporting information). Normal appendicectomy rate benchmarking across participating countries The overall NAR in patients aged 16–45 years was 20⋅0per cent (392 of 1957) in the UK and 6.2 per cent (54 of 868) in the other countries. In women aged 16–45 years, the NAR was 28⋅2 per cent (272 of 964) in the UK, 5⋅0 per cent (6 of 121) in Italy, 6 per cent (3 of 51) in Portugal, 25 per cent (15 Fig. 1 Flow diagram of study inclusion in the systematic review Search results n=1448 Excluded n=1407 Topic not relevant to appendicitis scoring n=1335 Scoring papers in foreign language n=4 Reports duplicating the same appendicitis scores n=41 Paediatric appendictis scores n=20 Scores for perforated appendicitis only n=5 Scores incorporating radiological parameters n=2 Excluded n=24 Reports duplicating the same appendicitis scores n=8 Scores for perforated appendicitis only n=1 Scores incorporating intraoperative parameters n=2 Scores incorporating radiological parameters n=4 Scores incorporating non-routine laboratory tests n=2 Algorithm for calculating score not reported n=6 Pregnancy included as risk variable n=1 Variables required to calculate score not available in RIFT data set n=11 Articles identified from hand search of reference lists n=9 Full-text review n=50 Studies reporting adult appendicitis scores n=26 Studies included in validation n=15 © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. Identifying adults at low risk of appendicitis 77 Table 1 Characteristics of risk prediction models identified in systematic review Model Year Country Derivation cohort Patients included in derivation cohort Van Way et al.34 1982 USA Retrospective single-centre 476 patients who underwent appendicectomy; included children and adults, and both sexes* Alvarado20 1986 USA Retrospective single-centre 305 patients presenting with abdominal pain; age range 4–80 years, 42% of patients were women Izbicki et al.28 1992 Germany Retrospective single-centre 536 patients who underwent appendicectomy; included both sexes* Eskelinen et al.25 1992 Finland Prospective multicentre 1333 patients presenting with abdominal pain; mean age 38 years, 52 per cent of patients were women Christian and Christian24 1992 India Prospective single-centre 58 patients presenting with abdominal pain; age range 10–56 years, 22% of patients were women Modified Alvarado29 1994 UK n.a.†n.a.† Eskelinen et al.26 1994 Finland Prospective multicentre 636 patients presenting with abdominal pain; included men only* van der Broek et al.33 2002 Holland Prospective single-centre 577 patients presenting with abdominal pain; all patients aged above 10 years, 59% of patients were women Birkhahn et al.22 2006 USA Prospective single-centre 439 patients presenting with abdominal pain; age range 3–93 years, 65% of patients were women AIRS21 2008 Sweden Prospective multicentre 316 patients presenting with abdominal pain; mean age 26 years, 54% of patients were women RIPASA score23 2010 Brunei Retrospective single-centre 312 patients who underwent appendicectomy; mean age 26 years, 42% of patients were women Ting et al.32 2010 Taiwan Retrospective single-centre 532 patients who underwent appendicectomy; 39% of patients were women* AAS31 2014 Finland Prospective single-centre 725 patients presenting with abdominal pain; age range 16–97 years, 58% of patients were women Goh27 2017 China n.a.†n.a.† Mikaere et al.30 2018 New Zealand Retrospective single-centre 885 patients presenting with abdominal pain; all patients aged over 15 years, 59% of patients were women *Overall age and/or sex information not reported. †Not applicable (n.a.) as modification of a previously published score (no derivation cohort). AIRS, Appendicitis Inflammatory Response Score; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis; AAS, Adult Appendicitis Score. of 59) in the Republic of Ireland and 10⋅1 per cent (18 of 179) in Spain. In men in the same age group, the NAR was 12⋅1 per cent (120 of 993) in the UK, 0⋅6 per cent (1 of 157) in Italy, 6 per cent (2 of 32) in Portugal, 3⋅4 per cent (7 of 208) in the Republic of Ireland and 3⋅3 per cent (2 of 61) in Spain. As anticipated, the NAR was higher in the UK than in other countries, so risk score validation proceeded in UK patients. All of the following data are for UK patients only. Data integrity A total of 5345 patients were included in the study (Fig.2), from across 154 UK hospitals. Of the 17 clinical parameters required to validate the risk prediction models, 0⋅9 per cent (794 of 90 865) were missing. Some 35⋅3 per cent (4461 of 12 647) of eligible data fields were assessed by independent data validators, finding overall data accuracy to be 98⋅3per cent (4384 of 4461). Patient characteristics and outcomes Two-thirds (3613 of 5345, 67⋅6 per cent) of patients were women (Tab le 2). Women were less likely to undergo any surgery than men (32⋅0versus 59⋅8 per cent respectively; relative risk 0⋅53, 95 per cent c.i. 0⋅50 to 0⋅57, P<0⋅001). Amongst women undergoing appendicectomy, 97⋅6per © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. 78 RIFT Study Group Fig. 2 Flow diagram of patient inclusion in the cohort study Records entered in database n=11 546 Excluded n=211 Previous appendicectomy n=14 Current pregnancy n=26 Missing age/sex n=14 Appendicectomies without histology or CT confirmation of diagnosis n=157 Excluded (European patients) n=2407 Excluded n=3583 Age <16 years n=1780 Age >45 years n=1750 Radiologically confirmed NOM n=31 CT confirmed appendicitis, but appendix histology unavailable n=22 Valid patients in RIFT data set n=11 335 UK patients n=8928 UK adults aged 16–45 years n=5345 Women 3613 Men 1732 NOM, non-operative management of appendicitis. Table 2 Patient management stratified by sex Women (n=3613) Men (n=1732) No. who had surgery 1156 (32⋅0) 1036 (59⋅8) No. of appendicectomies performed 964 of 1156 (83⋅4) 993 of 1036 (95⋅8) Confirmed appendicitis 625 (64⋅8) 841 (84⋅7) Other appendix pathology 67 (7⋅0) 32 (3⋅2) Histologically normal appendix 272 (28⋅2) 120 (12⋅1) No. not operated on 2457 (68⋅0) 696 (40⋅2) Values in parentheses are percentages. cent (941 of 964) had a laparoscopic procedure, of which 2⋅2 per cent (21 of 941) were converted to an open operation. Amongst men undergoing appendicectomy, 93⋅9per cent (932 of 993) had a laparoscopic procedure, of which 6⋅0 per cent (56 of 932) were converted to an open procedure. The NAR was higher in women than in men (28⋅2versus 12⋅1 per cent respectively; relative risk 2⋅33, 95 per cent c.i. 1⋅92 to 2⋅84, P<0⋅001). Within the 30-day follow-up, 27⋅4 per cent (1466 of 5345) of all patients presenting with acute RIF pain had undergone appendicectomy and been confirmed to have appendicitis on histological examination. Of all patients presenting with RIF pain, women were less likely to have a confirmed diagnosis of appendicitis than men (17⋅3versus 48⋅6 per cent respectively; relative risk 0⋅36, 0⋅33 to 0⋅39, P<0⋅001). Clinical use of risk prediction models Only 0⋅6 per cent of patients (32 of 5345) were recorded as having been formally risk-scored on admission by their clinical team. When performed, the Alvarado score was used most frequently (29 of 32, 91 per cent). Validation of risk prediction models Of the 15 risk prediction models, 11 showed consistently good discrimination for identifying appendicitis (AUC above 0⋅7) across both women and men (Fig.3and Tab le 3). In women, the Adult Appendicitis Score31 (AAS) achieved the highest specificity whilst maintaining a failure rate of less than 5 per cent in low-risk patients (Tab le 3). At a cut-off score of 8 or less, the AAS triaged 63⋅1 per cent of women who did not have appendicitis to the low-risk group (specificity), and amongst all women in the low-risk group 3⋅7 per cent in fact had appendicitis (failure rate). In men, the optimal model was the Appendicitis Inflammatory Response Score21 (AIRS), with a cut-off score of 2 or less associated with a specificity of 24⋅7 per cent and a failure rate of 2⋅4 per cent. Sensitivity analyses Overall, 0⋅9 per cent (294 of 32 517) of the variables required to calculate AAS were missing in women. The main complete-case analysis was based on the 95⋅0per cent (3433 of 3613) of women for whom all variables were available. In sensitivity analysis, the failure rate was found to range from 3⋅6to3⋅8 per cent, if missing variables were scored with either the maximum or minimum possible point value respectively. Specificity was found to range from 61⋅3to64⋅2 per cent respectively. In total, 1⋅0 per cent (156 of 15 588) of the variables required to calculate AIRS were missing in men. The complete-case analysis was based on the 93⋅9 per cent (1627 of 1732) of men for whom all variables were available. In sensitivity analysis, the failure rate ranged between 2⋅4and 2⋅9 per cent, and specificity from 23⋅3to26⋅4 per cent. © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. Identifying adults at low risk of appendicitis 79 Fig. 3 Receiver operating characteristic (ROC) curves for 15 appendicitis risk prediction models in women and men 0·25 0·50 1 – specificity a Women Sensitivity 0·75 1·00 0·25 0 1·00 0·75 0·50 0·25 0·50 1 – specificity b Men Sensitivity 0·75 1·00 0·25 0 1·00 0·75 0·50 Izbicki AUC 0·784 Mikaere AUC 0·803 Modified Alvarado AUC 0·780 RIPASA AUC 0·772 Ting AUC 0·700 van der Broek AUC 0·753 Van Way AUC 0·506 AAS AUC 0·832 AIRS AUC 0·808 Alvarado AUC 0·799 Birkhahn AUC 0·654 Christian AUC 0·761 Eskelinen (1992) AUC 0·751 Eskelinen (1994) AUC 0·645 Goh AUC 0·787 Izbicki AUC 0·787 Mikaere AUC 0·786 Modified Alvarado AUC 0·767 RIPASA AUC 0·775 Ting AUC 0·672 van der Broek AUC 0·742 Van Way AUC 0·515 AAS AUC 0·813 AIRS AUC 0·793 Alvarado AUC 0·779 Birkhahn AUC 0·681 Christian AUC 0·744 Eskelinen (1992) AUC 0·750 Eskelinen (1994) AUC 0·694 Goh AUC 0·768 aWomen; bmen. These receiver operating characteristic (ROC) curves are based on a complete-case analysis for all 15 risk prediction models. As a result they may be based on fewer patients than when each model was validated individually (Tabl e 3). This accounts for any minor discrepancies between the data presented here and those in Tab le 3 . AAS, Adult Appendicitis Score; AUC, area under the ROC curve; AIRS, Appendicitis Inflammatory Response Score; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis. Patients stratified to low-risk groups Of the 1856 women identified as low risk by the AAS in the complete-case analysis, 1560 (84⋅1 per cent) did not undergo surgery (Tab le 4). Of women who either did not undergo surgery or had a procedure other than appendicectomy, at 30 days the final diagnoses were non-specific abdominal pain (851 of 1627, 52⋅3 per cent), benign gynaecological pathology (430 of 1627, 26⋅4 per cent), urinary tract infection (99 of 1627, 6⋅1 per cent) and miscellaneous other (247 of 1627, 15⋅2 per cent). A full breakdown of diagnoses is provided in Table S3 (supporting information). Amongst the 12⋅3 per cent (229 of 1856) of women who underwent appendicectomy, 57⋅6 per cent (132 of 229) had a normal appendicectomy, 26⋅2 per cent (60 of 229) had simple appendicitis, 3⋅9 per cent (9 of 229) had complex appendicitis and 12⋅2 per cent (28 of 229) had other abnormal appendix pathology. Amongst the 28 women with other pathology, four were found to have carcinoid and one to have Crohn’s disease. The readmission rate for ongoing RIF pain amongst low-risk women who were not operated on in their index admission was 8⋅2per cent (130 of 1586). Of the women who had undergone appendicectomy on index admission, 6⋅8 per cent (14 of 207) were readmitted with postoperative complications. In the complete-case analysis, AIRS identified 209 men as being at low risk (Table 4). Of these, only 34 (16⋅3 per cent) had an appendicectomy and one man (0⋅5 per cent) underwent non-appendix surgery. At 30 days, the final diagnosis for most men who did not undergo appendicectomy was non-specific pain (110 of 175, 62⋅9 per cent) (Table S4, supporting information). Of men who had an appendicectomy, five were found to have appendicitis on histological examination, with no complex appendicitis identified. The NAR in low-risk men was 74 per cent (25 of 34). The readmission rate for ongoing RIF pain in low-risk men who were not operated on in their index admission was 7⋅3 per cent (13 of 178), and the readmission rate for postoperative complications among those who had an appendicectomy was 23 per cent (7 of 31). © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. 80 RIFT Study Group Table 3 Validation and identification of optimal thresholds for risk prediction models, stratified by sex Model AUC Optimal threshold Failure rate (%) Specificity (%) AAS31 Women 0⋅83 (0⋅82, 0⋅85) ≤83⋅763⋅1 Men 0⋅81 (0⋅79, 0⋅83) ≤6 5⋅020⋅4 AIRS21 Women 0⋅81 (0⋅79, 0⋅83) ≤3 3⋅551⋅6 Men 0⋅79 (0⋅77, 0⋅82) ≤22⋅424⋅7 Alvarado20 Women 0⋅80 (0⋅78, 0⋅82) ≤33⋅740⋅8 Men 0⋅78 (0⋅76, 0⋅80) ≤1 0 6⋅2 Birkhahn et al.22 Women 0⋅66 (0⋅64, 0⋅67) 1 3⋅818⋅1 Men 0⋅68 (0⋅66, 0⋅70) 1 10⋅121⋅6 Christian and Christian24 Women 0⋅76 (0⋅74, 0⋅78) 0 0⋅93⋅9 Men 0⋅75 (0⋅72, 0⋅77) 0 2⋅1 5⋅5 Eskelinen et al.25 Women 0⋅76 (0⋅74, 0⋅78) ≤45⋅3 4⋅4 9⋅6 Men 0⋅75 (0⋅73, 0⋅78) ≤39⋅41⋅48⋅4 Eskelinen et al.26 Women 0⋅65 (0⋅62, 0⋅67) ≤−5⋅83⋅66⋅4 Men 0⋅70 (0⋅67, 0⋅72) ≤−7⋅710⋅5 2⋅0 Goh27 Women 0⋅79 (0⋅77, 0⋅81) ≤2 4⋅146⋅3 Men 0⋅77 (0⋅75, 0⋅79) 0 5⋅38⋅4 Izbicki et al.28 Women 0⋅79 (0⋅77, 0⋅80) ≤13⋅321⋅2 Men 0⋅79 (0⋅77, 0⋅81) ≤1 6⋅1 3⋅6 Mikaere et al.30 Women 0⋅80 (0⋅78, 0⋅82) ≤1 4⋅957⋅9 Men 0⋅79 (0⋅76, 0⋅81) ≤14⋅321⋅7 Modified Alvarado29 Women 0⋅78 (0⋅76, 0⋅80) ≤34⋅543⋅6 Men 0⋅77 (0⋅75, 0⋅79) ≤211⋅825⋅4 RIPASA score23 Women 0⋅77 (0⋅75, 0⋅79) ≤5⋅5 4⋅744⋅2 Men 0⋅78 (0⋅76, 0⋅80) ≤403⋅8 Ting et al.32 Women 0⋅70 (0⋅68, 0⋅72) 0 19⋅846⋅9 Men 0⋅67 (0⋅65, 0⋅69) 0 4⋅852⋅5 van der Broek et al.33 Women 0⋅76 (0⋅74, 0⋅77) 0 3⋅324⋅6 Men 0⋅75 (0⋅72, 0⋅77) ≤214⋅323⋅1 Van Way et al.34 Women 0⋅51 (0⋅48, 0⋅53) 32 15⋅512⋅9 Men 0⋅52 (0⋅49, 0⋅54) 32 42⋅916⋅4 Values in parentheses are 95 per cent confidence intervals. AUC, area under the curve; AAS, Adult Appendicitis Score; AIRS, Appendicitis Inflammatory Response Score; RIPASA, Raja Isteri Pengiran Anak Saleha Appendicitis. © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd. Identifying adults at low risk of appendicitis 81 Table 4 Management and readmissions in patients scored as low risk, stratified by sex Women with AAS ≤8 (n=1856) Men with AIRS ≤2 (n=209) Patient management No. who had surgery 296 (15⋅9) 35 (16⋅7) No. of appendicectomies performed 229 of 296 (77⋅4) 34 of 35 (97) Simple appendicitis 60 (26⋅2) 5 (15) Complex appendicitis 9(3⋅9) 0(0) Other appendix pathology 28 (12⋅2) 4 (12) Histologically normal appendix 132 (57⋅6) 25 (74) No. not operated on 1569 (84⋅1) 174 (83⋅3) Readmission Ongoing RIF pain in patients not operated on in index admission 130 of 1586 (8⋅2) 13 of 178 (7⋅3) Postoperative complications following appendicectomy on index admission 14 of 207 (6⋅8) 7 of 31 (23) Values in parentheses are percentages. AAS, Adult Appendicitis Score; AIRS, Appendicitis Inflammatory Response Score; RIF, right iliac fossa. Table 5 Use of imaging and performance, stratified by sex and risk category Women Men Low risk (AAS ≤8) (n=1856) High risk (AAS >8) (n=1577) Low risk (AIRS ≤2) (n=209) High risk (AIRS >2) (n=1418) Ultrasound imaging 1291 (69⋅6) 916 (58⋅1) 59 (28⋅2) 205 (14⋅5) AUC* 0⋅63 (0⋅57, 0⋅70) 0⋅68 (0⋅65, 0⋅71) 1⋅00 (n.a.) 0⋅66 (0⋅60, 0⋅72) Sensitivity 0⋅28 0⋅38 1⋅00 0⋅37 Specificity 0⋅99 0⋅98 1⋅00 0⋅95 NPV 0⋅97 0⋅82 1⋅00 0⋅74 PPV 0⋅54 0⋅84 1⋅00 0⋅79 CT 208 (11⋅2) 316 (20⋅0) 41 (19⋅6) 341 (24⋅0) AUC* 0⋅99 (0⋅98, 1⋅00) 0⋅93 (0⋅90, 0⋅96) 1⋅00 (n.a.) 0⋅92 (0⋅90, 0⋅95) Sensitivity 1⋅00 0⋅92 1⋅00 0⋅94 Specificity 0⋅97 0⋅95 1⋅00 0⋅91 NPV 1⋅00 0⋅96 1⋅00 0⋅94 PPV 0⋅62 0⋅91 1⋅00 0⋅90 No imaging 434 (23⋅4) 456 (28⋅9) 111 (53⋅1) 910 (64⋅2) Values in parentheses are percentages unless indicate otherwise; *values in parentheses are 95 per cent confidence intervals. AAS, Adult Appendicitis Score; AIRS, Appendicitis Inflammatory Response Score; AUC, area under the receiver operating characteristic (ROC) curve; n.a., not applicable; NPV, negative predictive value; PPV, positive predictive value. Imaging Most women (2638 of 3613, 73⋅0 per cent) had either ultrasound or CT preoperative imaging. The imaging modality used most frequently was ultrasonography (2289 of 3613, 63⋅4 per cent). Low overall sensitivity (0⋅36) indicated that triage based solely on ultrasound imaging would misclassify most patients with appendicitis as being low risk. The failure rate for ultrasonography was 8⋅4 per cent. Stratifying women by AAS score, the AUC for ultrasound imaging was modest in both the low-risk (AUC 0⋅63) and high-risk (AUC 0⋅68) groups (Ta ble 5 ). Although CT was performed in only 15⋅1 per cent (547 of 3613) of women overall, this imaging modality was sensitive (0⋅92), with a low failure rate (2⋅1 per cent). Stratifying by AAS score, the AUC for CT was excellent in both the low-risk (AUC 0⋅99) and high-risk (AUC 0⋅93) groups. Overall, only 36⋅2 per cent (627 of 1732) of men underwent preoperative imaging. Ultrasound imaging (276 of 1732, 15⋅9 per cent) was performed less frequently than CT (398 of 1732, 23⋅0 per cent). The overall sensitivity of ultrasonography (0⋅38) was low, with a high failure rate (18⋅8 per cent). In men stratified as high risk by AIRS, ultrasound imaging had a poor AUC (0⋅66). In contrast, the overall sensitivity of CT (0⋅94) was high, with a failure rate of 4⋅5 per cent. In high-risk men, CT had an excellent © 2019 The Authors. BJS published by John Wiley & Sons Ltd www.bjs.co.uk BJS 2020; 107:73–86 on behalf of BJS Society Ltd.