Hyperspectral imaging for skin cancer and blood disorders diagnosis

Asian Journal of Physics Vol 29, Nos 10-12 (2020)1-20 Hyperspectral imaging for skin cancer and blood disorders diagnosis. Meritxell Vilaseca1, Francisco J. Burgos-Fernández1, Laura Rey-Barroso1, Mónica Roldán2,5, Susanna Gassiot3,5, Edurne Sarrate3,5, Ignacio Isola3,5 and Anna Ruiz Llobet4 1 Centre for Sensors, Instruments and Systems Development, Universitat Politècnica de Catalunya, Rambla de Sant Nebridi 11, 08222, Terrassa, Barcelona, Spain 2 Unitat de Microscòpia Confocal i Imatge Cel·lular, Servei de Medicina Genètica i Molecular, Institut Pediàtric de Malalties Rares (IPER), Hospital Sant Joan de Déu, 08950, Esplugues de Llobregat, Barcelona, Spain. 3 Laboratory of Hematology, Service of Laboratory Diagnosis / Institute of Pediatric Research, Hospital Sant Joan de Déu, Passeig Sant Joan de Déu 2, 08950, Esplugues de Llobregat, Spain 4 Service of Pediatric Hematology, Hospital Sant Joan de Déu, Passeig Sant Joan de Déu 2, 08950, Esplugues de Llobregat, Spain 5 Institut de Recerca Sant Joan de Déu, Santa Rosa 39-57, 08950, Esplugues de Llobregat, Barcelona, Spain Hyperspectral imaging is a novel technology for acquiring an image at a large number of wavelengths, thus allowing the study of spectral and spatial details of a sample under analysis. This technology has emerged as a promising imaging modality to be used as a diagnostic tool in several medical applications where spectral information is relevant. In this paper, we outline our most recent achievements in this field. Firstly, hyperspectral imaging systems developed to improve non-invasive diagnosis of skin cancer, consisting of digital silicon and InGaAs cameras and light emitting diodes, are described. Secondly, we present our last advances using hyperspectral technology together with confocal microscopy to improve the diagnosis of blood diseases, in particular, hemoglobinopathies such as thalassemia and cell membrane diseases such as hereditary spherocytosis. Finally, new insights on these topics are discussed. Keywords: hyperspectral imaging, confocal microscopy, skin cancer, hemoglobinopathies, membrane protein defects. 1. Introduction Optical imaging systems for non-invasive medical diagnosis remain crucial to assist physicians in their daily clinical practice. To date, efforts have focused on the acquisition of good-quality images through systems including monochrome or colour cameras. In fact, colour and spectral properties of biological tissue are caused by chromophores such as melanin, haemoglobin, water, beta-carotene, collagen and bilirubin. As a consequence of using colour cameras with only three spectral bands (R, G, and B), some biological structures and substances with different spectral signatures associated with specific diseases, which might differ among healthy and diseased tissue, may go undetected due to metamerism or limited spectral range. Hyperspectral imaging, or imaging spectroscopy, has recently emerged as a promising imaging modality used by authors to enhance and analyse spectral properties of the biological tissue with the ultimate goal of improving current diagnostic tools. In fact, they allow the reflected light from the tissue under analysis, which contains information on the chromophores, to be measured through more than three spectral bands of the electromagnetic spectrum range with high spatial resolution [1]. The final outcome of hyperspectral imaging systems is known as spectral cube, which contains all the spatial and spectral data from the sample under analysis. Depending on how the spectral cube is built, hyperspectral imaging devices are classified into different categories [2]. Point or line scanning systems, in which the whole spectrum of a single point or a single line is captured at once, require the scanning of the sample in one or two spatial directions. Unfortunately, these methods are not suitable for the in vivo analysis required in medical applications as spatial scanning requires complex mechanical configurations to scan the whole area of the sample. Moreover, it is very time-consuming and severe motion artifacts and pixel misregistrations are generated because the body is moving. This is why most medical systems based on hyperspectral imaging designed for in vivo measurements are of the type of area scanning or snapshot imaging systems. On the one hand, area scanning systems or staring imagers capture a two-dimensional data matrix (full resolution image) at one spectral band at a time. A complete spectral cube is then obtained by sequentially illuminating the tissue with light at different spectral bands and collecting the reflected light at the sensor; alternatively, they can sequentially filter the reflected light coming from the sample before reaching the camera sensor. These two approaches (active and passive) for collecting light give systems certain advantages and disadvantages for their application into medical analysis: active systems illumination are usually based on Light Emitting Diodes (LEDs), which are low cost, have a small size, long durability, and low energy consumption. They allow optical designs more compact than passive variants. Nevertheless, passive systems provide continuous spectral sampling as they are based on electro-optical devices, such as liquid crystal tuneable filters (LCTF). Examples of these systems are those published by Bekina et al. [3], who analysed skin lesions (including nevi and melanomas) using a LCTF-based spectral approach within the spectral range 450 nm – 950 nm in steps of 10 nm and proved that absorption was higher in carcinogenic lesions. Another example based on an active approach is that by Everdell et al. [4], who used a system based on LEDs for the spectral imaging of the eye fundus in the range from 500 nm to 620 nm. In order to make the acquisition faster, area scanning systems often used for medical purposes only include a few spectral bands. To overcome this, snapshot hyperspectral cameras have been put forward for functional mapping (e.g., for eye fundus imaging [5]) providing an even faster approach, which is intended to record both spatial and spectral information with only one exposure. Nevertheless, since the sensor area holds all the information from the sample, spatial or spectral resolution is very limited with maximum resolutions of 350 pixels × 350 pixels, which is insufficient to observe small biological structures that are relevant for clinical purposes. Additionally, they present extensive computational cost [6]. Lastly, most hyperspectral cameras reported to date for medical applications are limited to the analysis in the visible range, ultimately limiting the physician’s ability to detect, discriminate, and further investigate diseases as they develop. In fact, most of the hyperspectral imaging cameras are based on silicon sensors with spectral response in the visible and the near-infrared (900 nm – 1000 nm). However, hyperspectral, extended, near-infrared (exNIR) optical imaging is nowadays available thanks to new indium gallium arsenide (InGaAs) imaging sensors with high quantum efficiency within 900 nm – 1600 nm. This has sparked interest among biologists in this relatively unexplored spectral region also known as “the second near-infrared window” (900 nm – 1400 nm) [7]. Going further than 900 nm enables deeper in vivo optical imaging as photons at these wavelengths penetrate deeper into living tissue due to the lower absorption of chromophores [8]; this can be a hint to explore further this spectral range to improve medical diagnosis, as it may release information about how tissues are damaged due to water content and other factors that might be different in healthy and diseased samples. For instance, due to the increased absorption of water in this spectral range, spectral images can provide information about the presence of angiogenesis, a tumorous growth of blood vessels. In fact, the near-infrared region has been widely utilized in the past decade, and a number of clinical imaging applications have already been developed [9]. In this report, we present a review of the most recent achievements of our research group in the field of medical hyperspectral imaging. In particular, we investigate the possibilities of (i) active hyperspectral imaging systems based on silicon and InGaAs sensors for the detection of skin cancer and (ii) the combined use of hyperspectral imaging and confocal microscopy for the diagnosis of diseases affecting red blood cells (RBCs). 2. Detection of skin cancer lesions through visible and exNIR spectral imaging The incidence of skin cancer has increased over the past decades, being one in three cancers diagnosed as skin cancer. The World Health Organization estimates that 60,000 people die every year because of long sun exposures: 48.000 from melanoma, which is the most aggressive form, and 12.000 from other skin cancer types. Nowadays, visual inspections by the naked eye and through the dermoscope, a hand-held device that contains LEDs to provide white, polarized illumination and equipped with a magnification lens, are the first techniques used to diagnose the disease. However, they fail in the correct discrimination of lesions in an important number of cases. These inspections base the diagnosis in the so-called ABCDE rule, which outlines warning signs of the most common types of cancer: A is for asymmetry, B is for border irregularity, C is for colour, D is for the diameter, and E is for its evolution [10]. This technique produces a large number of false positives; that is, benign lesions erroneously classified as malignant ones. Therefore, the gold standard is still histological examination, which requires the surgical excision of the tumour (biopsy) and its analysis with an optical microscope. This procedure is costly, painful for the patient, and the multiple specialists involved make it a long-term process [11]. Due to the large number of affected people, efforts have been made to detect skin cancer non-invasively through optical imaging devices, in particular using hyperspectral imaging. The aim is to achieve a higher detection ratio and better prognostic evaluation of skin cancer at earlier stages when compared with current methods. An extensive review of hyperspectral systems and other optical modalities recently developed for the detection of skin cancer can be found in [12]. In this framework, in [13] we explored the use of an active hyperspectral system based on area scanning that analyses the spatial distribution of colour and spectral features to improve the detection of nevi, melanomas and basal cell carcinomas. The system consisted of a digital CCD camera and LEDs emitting at 8 different wavelengths (414, 447, 477, 524, 671, 735, 890, and 995 nm), which were chosen according tothe absorption curves of the principal chromophores of the skin in this range, i.e., melanin, oxy- and deoxyhaemoglobin. Infrared wavelengths used in pulse oximetry were considered as well. Besides, the system also included crossed polarizers to remove specular reflection from the skin surface to capture information from deeper tissue layers (Fig. 1). The system was later expanded into the exNIR region [14], with the use of an InGaAs camera and 5 more LEDs (1081, 1214, 1340, 1486, and 1613 nm), to evaluate deeper skin layers thanks to the higher penetration of photons at these wavelengths. Reflectance images of all lesions were then computed as follows: (1) where is the spectral reflectance image, and I(i, j), IN(i, j), and ID(i, j) contain the digital levels of the raw, neutral grey reference and dark current images, respectively; and k is the calibrated reflectance of the neutral grey reference, given by the manufacturer. Fig. 1. External view of the hyperspectral system for skin cancer diagnosis (a), main components (b) and reflectance images taken at different wavelengths (on the right the RGB and dermatoscopic images are also shown (c). In agreement with previous publications [3], the averaged reflectance of melanomas (or equivalently absorbance) was found to be lower (higher) than that of nevi, especially between 600 nm and 1100 nm (Fig. 2). However, there was a great variance among lesions of the same type, which made it difficult to discriminate them if only averaged values were taken into account. To overcome this, statistical descriptors based on the first-order statistics of the histogram were calculated for every segmented lesion to extract textural information, rather than only using averaged spectral traits. These were entropy (Ep), a statistical measure of randomness, energy (En), a numerical descriptor of the image uniformity having 1 as its maximum value for a constant image, and the third central moment (μ3), which accounts for the skewness of the histogram. In these studies, principal component analysis was also used to analyse the variability among data and establish a classification based on Support Vector Machines (SVM), reaching maximum sensitivity and specificity values of 91.3% and 84.6%, respectively. Fig. 2. Averaged spectral reflectance curves (±σ) of nevi and melanomas in the VIS-NIR and exNIR ranges (left) and histograms of a nevus and a melanoma in terms of spectral reflectance at 415 nm, with their respective values of mean, maximum, minimum, entropy (Ep), energy (En), and third central moment (μ3). 3. Diagnosis of red blood cells disorders with spectral confocal microscopy We have begun to apply hyperspectral technology in the field of hematologic diseases, especially those affecting RBCs. RBCs are specialized cells in charge of oxygen transportation, which contain haemoglobin that is able to bind oxygen and carbon dioxide molecules. Healthy RBCs have a biconcave shape and great deformation capacity when passing through capillaries. Due to defects in genetic information coding for RBCs, their shape and number can be altered, as well as their capacity to transport oxygen. Clinical manifestations of hemoglobinopathies range from mild to severe anaemia with multiorgan involvement [15]. Their diagnosis is currently based on RBC morphology under conventional optical microscopy and RBC indices, which consist of estimating the concentration of different types of haemoglobin in blood, which are most frequently determined by automated highperformance liquid chromatography [16]. However, these techniques are sometimes not specific enough to distinguish between mild and severe forms of the disease. Additionally, hemoglobinopathies can also be difficult to diagnose due to coexistence of different causes of anaemia, such as thalassemia, iron deficiency, etc. Therefore, complex and expensive genetic studies are often required. To avoid this, we recently explored the possibility of using hyperspectral technology together with confocal microscopy as a diagnostic tool for thalassemia [17], which is caused by mutations in the globin genes that result in changes of the globin chains that form haemoglobin [18]. In this study, RBCs from patients with different syndromes of thalassemia and iron deficiency anaemia were compared to cells of healthy subjects under a spectral confocal microscope following different image acquisition protocols. The microscope incorporates three lasers for excitation: a diode laser with an emission of 405 nm, and argon laser with peaks at 458nm, 476nm, 488nm, 496nm, 514nm, and a white laser that emits from 470 nm to 670 nm, combined with an acoustic-optic tuneable filter (AOTF). It incorporates hybrid detectors with high dynamic range and sensitivity capable of detecting very low signals (single photon counting) coming from RBCs from 400 nm to 790 nm. Specifically, we determined the emission of RBCs (autofluorescence) from 425 nm to 790 nm when excited at 405 nm (Fig. 3). Whereas all samples showed a common emission peak around 502 nm in the emission spectrum, those corresponding to patients with alpha-thalassemia presented an additional emission peak around 628 nm and 649 nm. In the case of iron-deficiency anaemia, the cells showed a prominent peak at 579 nm. On this basis, three experimental descriptors calculated from the mean emission intensities at these wavelengths were proposed to discriminate between diseased and healthy cells. Accordingly, spectral confocal microscopy was shown to be useful in the diagnosis of thalassemia as it allowed discrimination between healthy and diseased individuals and between different degrees of thalassemia. Fig. 3. Spectral confocal microscope used (a) and emission curves computed from autofluorescence images (images corresponding to 628 nm are also shown) from healthy and diseased samples (b). The sample with thalassemia shows an additional fluorescence peak at this wavelength (the RBCs with emission can be seen in the image on the right). We are currently working on the evaluation of hereditary spherocytosis by means of the same technique. In hereditary spherocytosis, there are structural defects in the membrane of RBCs caused by mutations in at least five known genes that code for membrane proteins. Their malfunction results in RBCs with abnormal spherical shapes and reduced deformability (spherocytes) [19]. To approach the diagnosis of this disease, we carried out experiments in which the cell membrane was stained with different colour dyes and immunolabels, to identify possible membrane defects expressed as differences in colour and shape under the confocal microscope. Afterwards, fluorescent 3D image stacks were collected to assess shape and fluorescence of cells. The first results of this immunostaining assay show that healthy subjects could be linked to higher values of mean fluorescence within the 510nm - 560 nm region when excited at 488 nm (Fig. 4). In this assay, proteins (band 3) in the cell membrane were specifically targeted by means of an antibody with a fluorescent probe (BRIC 200-iFluor488). On the other hand, we are working on the analysis of 3D image stacks in order to be able to identify spherocytes in each sample, as confocal microscopy allows precise and unequivocal 3D assessment of cellular shapes. In this case, the cell membrane is stained in-vivo with the CellMaskTM deep red dye, with emission in the 645 nm - 775 nm range when excited at 633 nm. Fig. 4 depicts confocal spectral image stacks of one field where some spherocytes can be observed. Ongoing work focuses on the implementation of image processing algorithms towards the automated classification and grading of samples with hereditary spherocytosis. Fig. 4. Projection sum of confocal spectral images (488 nm excitation wavelength and emission at 510 nm - 560 nm) for a healthy (left) and a diseased (right) individual RBCs labelled with BRIC 200. The pseudocolour scale is shown at the bottom right. Warm colors such as white and red represent maximum intensities, whereas cold colors like blue are representative of low intensities (a); corresponding mean fluorescence intensity plot, where a decrease in fluorescence intensity was observed in HS samples (b); Orthogonal views of the image volume of RBCs, showing some spherocytes (white arrows). The red signal corresponds to cell membrane labeling (excitation at 633nm and emission at 645 nm - 775 nm) (HS: Hereditary Spherocytosis). 4. Discussion Regarding skin cancer diagnosis, the developed hyperspectral systems have not yet superseded histological examination even they have reached similar sensitivity and specificity values to those obtained by experienced dermatologists through dermoscopy [20]. In fact, this continues to be the clinical gold standard, providing diagnostic confirmation after surgical excision of the lesion. Therefore, hyperspectral systems can be used as a supporting tool to dermoscopy but nowadays, only a biopsy can make the definitive diagnosis. In order to improve the sensitivity and specificity values achieved with hyperspectral technology, the recent technological advances in regards to InGaAs sensors (working in the exNIR range) with improved resolution might help. Additionally, the combination of hyperspectral imaging with other optical technologies such as confocal microscopy, 3D topography, optical coherence tomography, among others, are currently being explored and might be helpful, too [12, 21]. Hyperspectral imaging also looks promising as a means of identifying blood diseases because spectral information may arise from autofluorescence through confocal microscopy. This information can be very valuable when currently available techniques are not specific enough to distinguish between mild and severe forms of the diseases or because of the coexistence of different causes of anaemia. In this regard, although some attempts have already been made to characterize the morphology and spectral traits of RBCs using confocal microscopy [22], more research is still needed. This will enable to elucidate how novel optical imaging techniques can help overcoming the current medical limitations in diagnosing hemoglobinopathies and cell membrane diseases, especially combining spectral and 3D information given by hyperspectral imaging technology and confocal microscopy. The use of machine learning could also play a key role in the classification of skin cancer lesions and blood diseases of different etiology [23]. Hopefully, the use of all these technologies will allow overcoming some of the currently limitations thus improving the dermatologic and hematologic clinical practice. Acknowledgments This project has been funded by the Agencia Estatal de Investigación (AEI) (PID2020- 112527RB-I00 / AEI / 10.13039/501100011033). L R-B thanks the Ministry of Science, Innovation and Universities for the PhD (FPI) grant she has received. References 1. Lu G, Fei B, Medical hyperspectral imaging: a review, J. Biomed. Opt., 19(1) (2014), 010901. 2. Li Q, He X, Wang Y, Liu H, Xu D, Guo F, Review of spectral imaging technology in biomedical engineering: Achievements and challenges, J. Biomed. Opt., 18(10) (2013), 100901. 3. Diebele I, Kuzmina I, Lihachev A, Kapostinsh J, Derjabo A, Valeine L, Spigulis J, Clinical evaluation of melanomas and common nevi by spectral imaging, Biomed. Opt. Express, 3(3) (2012), 467-472. 4. Everdell NL, Styles IB, Calcagni A, Gibson J, Hebden J, Claridge E, Multispectral imaging of the ocular fundus using light emitting diode illumination, Rev. Sci. Instrum., 81(9) (2020), 093706. 5. Firn KA, Khoobehi B, Novel, noninvasive multispectral snapshot imaging system to measure and map the distribution of human retinal vessel and tissue hemoglobin oxygen saturation, Int. J. Ophthalmic Res., 1(2) (2015), 48-58. 6. Hagen N, Dereniak EL, Analysis of computed tomographic imaging spectrometers. I. Spatial and spectral resolution, Appl. Opt., 47(28) (2008), F85–F95. 7. Cao Q, Zhegalova NG, Wang ST, Akers WJ, Berezin MY, Multispectral imaging in the extended near-infrared window based on endogenous chromophores, J. Biomed. Opt., 18(19) (2013), 101318. 8. Mourant JR, Fuselier T, Boyer J, Johnson TM, Bigio IJ, Predictions and measurements of scattering and absorption over broad wavelength ranges in tissue phantoms, Appl. Opt., 36(4) (1997), 949–957. 9. Godoy SE, Ramirez DA, Myers SA, von Winckel G, Krishna S, Berwick M, Padilla S, Sen P, Krishna S, Dynamic infrared imaging for skin cancer screening, Infrared Phys. Technol., 70 (2015), 147–152. 10. Gadeliya Goodson A, Grossman D, Strategies for early melanoma detection: Approaches to the patient with nevi, J. Am. Acad. Dermatol., 60(5) (2019), 719–735. 11. Guy GP, Ekwueme DU, Tangka FK, Richardson LC, Melanoma treatment costs: a systematic review of the literature, Am. J. Prev. Med., 43(5) (2012), 537–545. 12. Rey-Barroso L, Peña-Gutiérrez S, Yáñez C, Burgos-Fernández FJ, Vilaseca M, Royo S, Optical technologies for the improvement of skin cancer diagnosis: a Review, Sensors (Basel), 21(1) (2021), 252. 13. Delpueyo X, Vilaseca M, Royo S, Ares M, Rey-Barroso L, Sanabria F, Puig S, Malvehy J, Pellacani J, Noguero F, Solomita G, Bosch T. Multispectral imaging system based on light-emitting diodes for the detection of melanomas and basal cell carcinomas: A pilot study, J. Biomed. Opt., 22(6) (2017), 065006. 14. Rey-Barroso L, Burgos-Fernández FJ, Delpueyo X, Ares M, Royo S, Malvehy J, Puig S, Vilaseca M, Visible and extended near-infrared multispectral imaging for skin cancer diagnosis, Sensors (Basel), 18(5) (2018), 1441. 15. Kohne E, Hemoglobinopathies: clinical manifestations, diagnosis, and treatment, Dtsch. Arztebl. Int., 108(31- 32) (2011), 532-540. 16. Desouky OS, Selim NS, El-Bakrawy EM, El-Marakby SM, Biophysical characterization of thalassemic red blood cells, Cell Biochem. Biophys., 55(1) (2009), 45–53. 17. Rey-Barroso L, Roldán M, Burgos-Fernández FJ, Gassiot S, Ruiz Llobet A, Isola I, Vilaseca M, Spectroscopic evaluation of red blood cells of thalassemia patients with confocal microscopy: a pilot study, Sensors (Basel), 20(14) (2020), 4039. 18. Devanesan S, Mohamad Saleh A, Ravikumar M, Perinbam K, Prasad S, Abbas HA-S, Palled SR, Jeyaprakash K, Masilamani V, Prasad S, A-Saeed Abbas H, Palled SR, Jeyaprakash K, Masilamani V, Fluorescence spectral classification of iron deficiency anemia and thalassemia, J. Biomed. Opt., 19(2) (2014), 027008. 19. Eber S, Lux S, Hereditary Spherocytosis - Defects in proteins that connect the membrane skeleton to the lipid bilayer. Semin. Hematol., 41(2) (2004), 118-141. 20. Langley RG, Walsh N, Sutherland AE, Propperova I, Delaney L, Morris SF, Gallant C, The diagnostic accuracy of in vivo confocal scanning laser microscopy compared to dermoscopy of benign and malignant melanocytic lesions: A prospective study., Arch. Dermatol., 215(4) (2007), 365–372. 21. Rey-Barroso L, Burgos-Fernández FJ, Ares M, Royo S, Puig S, Malvehy J, Pellacani G, Espinar D, Sicilia N, Vilaseca Ricart M, Morphological study of skin cancer lesions through a 3D scanner based on fringe projection and machine learning, Biomed. Opt. Express, 10(7) (2019), 3404-3409. 22. Khairy K, Foo J, Howard J, Shapes of red blood cells: comparison of 3D confocal images with the Bilayer- Couple model, Cell. Mol. Bioeng., 1(2-3) (2008), 173–181. 23. Magalhaes C, Mendes J, Vardasca R, The role of AI classifiers in skin cancer images, Skin Res. Technol., 25(5) (2019), 750–757. Corresponding author e mail: meritx[email protected] (M. Vilaseca)