Discrimination between Surgical and Nonsurgical Nuclear Cataracts Based on ROC Analysis

1 Discrimination between surgical and non-surgical nuclear cataracts based on ROC analysis Clemente Paz Filgueira1,2, Roberto F. Sánchez1,2, Elisa M. Colombo1,2, Meritxell Vilaseca3, Jaume Pujol3, Luis A. Issolio1,2. 1Departamento de Luminotecnia, Luz y Visión (DLLyV), Facultad de Ciencias Exactas y Tecnología, Universidad Nacional de Tucumán (UNT), Argentina. 2Instituto de Investigación en Luz, Ambiente y Visión (ILAV), CONICET-UNT, Argentina 3Center for Sensors, Instruments and Systems Development (CD6), Universitat Politècnica de Catalunya (UPC) (Terrassa - Barcelona, Spain) This work was supported by: ANPCyT PICT 2011 N° 1807 (Argentina), CIUNT 26/E410 (Tucumán, Argentina) and CONICET PIP 0308 (Argentina). CORRESPONDENCE: Clemente Paz Filgueira, Departamento de Luminotecnia, Luz y Visión, Facultad de Ciencias Exactas y Tecnología, Universidad Nacional de Tucumán, Av. Independencia 1800, San Miguel de Tucumán, Tucumán, Argentina. Tel/Fax: +54 0381 4361936; E-mail: [email protected] 2 ABSTRACT PURPOSE: The aim of this paper is to propose a quantitative methodology for determining a criterion to discriminate non-surgical nuclear cataract from surgical ones taking into account objective measures of intraocular scattering in patients with good visual acuity (>0.6). METHODS: Two groups of subjects were taken into account: a control group and a group with nuclear cataracts. At a first stage, eyes belonging to the cataract group were classified into “non-surgical” and “surgical” cataracts by ophthalmologists at their clinical settings. At a second stage a double pass instrument was also used to determine the Objective Scatter Index (OSI) at the laboratory. ROC curves were used to analyze OSI values to determine a value able to separate between non-surgical and surgical cataracts. RESULTS: We obtained statistically significant differences among the control and both nuclear cataract groups (p <0.05). ROC curves determined an OSI criterion level (of 2.1) to suggest surgery in nuclear cataracts with an area under curve of AUC=0.83, i.e. with 80% of sensitivity and 80% of specificity. CONCLUSIONS: ROC analysis allows separating both groups of nuclear cataract and we determined a value of OSI in nuclear cataract quantification for surgery. 3 INTRODUCTION A cataract produces an increase of the intraocular diffusion generating visual disturbances that can severely affect the daily performance of a person (1). Some of the early cataract signs include poor night vision, glare, diplopia, contrast loss and reduction of the perceived brightness; and all of them can appear without generating a significant decrease in the visual acuity (2). Patients with cataracts particularly complaint about bright light sources as car headlights or sunlight, which produce halos around objects, reduce visibility and can derive in temporary blindness. Because of this permanent patient’s discomfort, the practitioner needs to have a fast and clear diagnosis, confirming and objectively quantifying the intraocular scattering. Due to the fact that the unique existing treatment for cataract is surgery, the decision of when to deal with the operation is of great importance. For this purpose, ophthalmologist often determine through observation whether cataract development and visual performance reduction are advanced enough or not. Meanwhile visual performance reduction is usually carried out by means of visual acuity and contrast sensitivity tests (3)(4), biomicroscopy is mainly required to accurately determine the cataract morphology through observation. By means of the analysis of slit lamp images it is possible to determine the cataract type and to establish a grade according to the Lens Opacities Classification System (LOCS) III (5). Although the diagnosis can often be made without dilating the pupil, the dilation is often useful in cases where pupils are small or when a nuclear cataract is suspected (6) and as it is known the use of mydriatics may induce closed-angle glaucoma. Moreover, the slit lamp is also very used in cataract diagnosis since it is a direct method that does not require the active subject participation, even though it must be taken into account that results might be influenced by the skill of the physician. To 4 avoid this dependency and to assist the diagnose, Li and collaborators (7) presented studies that attempted an automatic grading of nuclear cataracts from slit lamp images and the development of computer based systems to diagnose nuclear cataracts (8). On the other hand, Scheimpflug imaging, which is a tool for imaging the anterior ocular segment, can also be used to obtain quantitative information on the geometry of the crystalline lens (9)(10). This technology has recently been shown as an objective and repeatable method for the assessment of lens density in nuclear cataracts as authors have reported a strong correlation between nuclear lens density and LOCS III (11). Nevertheless, these systems based on the direct observation of the lens only assess the backscattered component of the complete intraocular scattered light. Therefore, the cataract grade provided by such systems does not take into account the forward scattering towards the retina, which is the one really affecting the visual capacity(1). The retinal image analysis through a double-pass (DP) system (12) has recently arisen as a new tool that does not require the delivery of dilation drugs and might be suitable to assess the whole optical information of the eye, including the effect of higher-order aberrations and intraocular scattering (13)(14). From this technique an objective scattering index (OSI) proposed by Artal et al (15) can be computed. This parameter takes into account the forward-scattered light and has been validated in several studies (16)(17)(18). Other studies have suggested the usefulness of this parameter in the clinical prediction of intraocular scattering (16)(19)(20) and it has been shown to be useful for grading nuclear, cortical and posterior subcapsular cataracts (15)(21). Regarding aberrations, it has been shown that uncorrected defocus and astigmatism can strongly affect OSI (15) and to avoid this problem all measurements must be performed with the corrected refraction. When comparing DP images with those provided by spatially resolved method like Hartman-Shack wavefront sensor it can be 5 seen that in eyes with mild to high amount of scatter, wavefront sensors might overestimate image quality, whereas the DP technique produces a more accurate description of the optical quality (14). Moreover, this DP technique has been successfully used in the evaluation of the ocular optical quality in patients with keratitis (22), uveitis (23), dry-eye (24) and after refractive surgery (25) among other studies (26). As different cataract types produce different kinds of opacities that modify the backward and forward scattering balancing (1), it is important to analyze each of them independently. In this study we analyzed eyes with nuclear cataracts, which are usually associated with aging. Therefore, it is very important to precisely assess its evolution and to obtain reliable information about when surgery is recommended. We compared the medical decision of operating a nuclear cataract based on the classical clinical evaluation and the OSI measurements. The goal was to introduce ROC (Receiver Operating Characteristic) (27) analysis to determine a criterion level to differentiate between two cataract stages (non-surgical and surgical cataracts with good visual acuity). For this, we took the already mentioned advantages of an objective system for measuring intraocular scattering as well as ROC curves, which are widely used in clinical and diagnostic medicine (28)(29) and biostatistics as assistance in making decisions (30)(31). MATERIALS AND METHODS Subjects Two different groups of patients were enrolled in the study: a cataract and a control group. The cataract group consisted of 10 subjects (20 eyes) with a mean age (±SD) 6 of 68±9 years old (ranging from 50 to 85). The control group included data from 26 observers (47 eyes) with a mean age of 34±9 years old (ranging from 23 to 58). The inclusion criterion in the cataract group considered those eyes that presented some opacity grade exclusively in the nucleus of the crystalline lens in a relative wide range regarding the level of scattering with a corrected visual acuity (CVA) higher than or equal to 0.6. On the other hand, the control group consisted of eyes without opacities in the crystalline lens and CVA≥1. Patients included in the study did not present ocular diseases other than the cataract itself at the moment of the experiment and had not suffered other ocular diseases previously (such as glaucoma, retinopathies, keratoconus, etc). Patients that had been undergoing refractive surgery were also excluded. Every patient was informed of the subject of the study, and a written informed consent obtained, following the tenets of the Declaration of Helsinki. Clinical evaluation procedure The clinical evaluation of patients included a first stage performed by three ophthalmologists (WA, LP, EF) at their clinical setting. Examinations included autorefraction, subjective manifest refraction, and determination of the non-corrected visual acuity (NCVA), the CVA and the stenopeic visual acuity (SVA). Observations through slit lamp were also carried out, which helped to determine the cataract type, its location in a lens scheme and its density according to LOCS III. From these data and having a previously agreed criterion, the practitioner had to separate between Surgical Cataracts (SC) and Non-Surgical Cataracts (NSC).The second stage was carried out at the laboratory of visual optics of the Universidad Nacional de Tucumán, and included the intraocular scattering quantification using DP images and the 7 corresponding computation of the OSI (see next section for more information on this). Measurements were performed with the best refractive correction to avoid the influence of aberrations in the computation of the OSI, as discussed in the introduction. DP images acquisition and processing: Objective Scatter Index As already mentioned, the system to quantify intraocular scattering was based on DP images. They were recorded using a commercial instrument (The Optical Quality Analysis System, Visiometrics S.L., Spain) (13), whose schematic layout is shown in Figure 1. The instrument acquires a retinal image that corresponds to a point-source object (LD: laser diode; λ=780nm) by means of a CCD camera (CCD1), after reflection on the retina and double pass through the ocular media. The patient’s refraction is internally corrected by the instrument using an optometer that consists of two lenses (L3, L4) and two mirrors (M2, M3), allowing capturing the DP images at the best focus. The entrance pupil (EP) has a fixed diameter of 2 mm. The instrument also has an artificial and variable exit pupil (ExP) controlled by a diaphragm wheel, whose image is formed on the patient’s natural pupil plane. In this work we fixed the ExP at a value of 4mm. Furthermore, near-infrared light is used in the DP system because it is more comfortable for the subject and provides retinal image quality estimates that are comparable to those obtained with visible light (32). From each DP image, the OSI is calculated providing an estimation of the amount of intraocular scattering (15). This parameter is computed as the ratio between the amount of light recorded inside an annular area between 12 and 20 minutes of arc and that recorded within 1 minute of arc of the central peak. A similar methodological approach was proposed by Westheimer and Liang (33), who measured an index of diffusion strongly tending to increase with age. The choice of the angles from which 8 OSI is computed is based on the results obtained in a previous study (15), in which authors found a maximum correlation between OSI values and a standard cataract gradation (LOCS III) using this configuration in patients with different grades of nuclear cataracts. In the present study, each OSI value was calculated by averaging six DP images that were acquired sequentially. Furthermore, the presented data are the average of five OSI determinations. Statistical analysis The data was analyzed using the Minitab Statistical Software® 16.1.0. Comparisons were considered to be statistically significant for p values less than 0.05 (95% confidence interval). For testing data normality the Shapiro-Wilk test was used. Comparisons between the control group (CG), and the two cataract groups (NSC and SC) were performed by means of the Mann – Whitney non-parametric median test. ROC (27) curves were computed with MedCalc ® v.12.7.0.0 to establish a separation criterion between NSC and SC groups. The ROC analysis is based on the ROC curves (34), whose axes are sensitivity (power to identify positives) and specificity (power to identify negatives), which vary with the criterion level. The goal is to determine the optimum criterion level, i.e. the point in the curve where sensitivity and specificity are maximums. RESULTS Table 1 summarizes the sample size, age, CVA and OSI of each group considered in this study, i.e. the CG and the two cataract groups (NSC and SC). Figure 2 shows OSI data from eyes of the CG and the two cataract groups (NSC and SC) using box plots. The distribution of OSI in all the groups was asymmetric, and for this reason the 9 median was considered for comparisons between them. It can be seen that SC group sampling generally presents OSI values higher than the NSC group, which in turn are higher than those of the CG. The median corresponding to the CG was of 0.45, while it was of 1.80 and 2.68 for the NSC and SC groups. Furthermore, statistically significant differences could be established among the three groups (p<0.05). Considering that the OSI observed in the SC group was higher than in the NSC group, a specific OSI value was objectively determined to discriminate, according to medical criterion, between those patients who should not undergo surgery and those who should undergo surgery. Figure 3 shows the probability density function of the NSC and SC groups. It can be seen that the curves cross each other at an OSI value close to 2, which is a first approximation to separate NSC and SC. According to the aim of this paper, ROC curves were used in order to look for an optimum criterion level between non-surgical and surgical cataracts. Figure 4 shows the ROC curve corresponding to OSI data from NSC and SC groups. The area under curve (AUC) of the obtained ROC curve was of 0.830, which is a measure of the power of the test and represents, in this case, the probability for a randomly selected SC to have a higher OSI value than a randomly selected NSC. The optimum criterion level obtained was an OSI value of 2.1, which implies 80% of sensitivity and 80% of specificity. This criterion implies a predictive value of a positive (PV+), i.e. the probability of needing surgery with an OSI value higher than 2.1, of 80% and a predictive value of a negative (PV-), i.e. the probability of not needing surgery with an OSI value lower than 2.1, of 80%. Likewise, the analysis showed that OSI values higher than 2.1 are four times more likely to be found in SC than in another 16 FIGURES Figure 1. Schematic diagram of the DP instrument (14). Figure 2. OSI values for the three groups: CG, and NSC and SC groups. The box plots show five statistical descriptors: maximum, 3rd quartile, median, 1st quartile, and minimum. 17 Figure 3. Probability density functions of the NSC and SC groups. Figure 4. ROC curve for NSC and SC groups. As a reference, a curve with an AUC of 0.5 has been plotted (dashed line). 18 Figure 5. Interaction diagram between individual OSI values and the criterion level. Each individual value and error bar are the average and standard deviation of five measurements. 19 TABLES Table 1: Sample size, age, CVA and OSI (Objective Scattering Index) of the groups considered in this study. Groups Sample size (number of eyes) Mean age (years) CVA (mean) OSI (median) Control (C) 47 34 1.0 0.45 Non Surgical Cataract (NSC) 10 70 0.9 1.8 Surgical Cataract (SC) 10 66 0.8 2.68