Ci a ion: Okamo o, N.;
Rod íguez-Luna, M.R.; Benc eux, V.;
Al-Tahe , M.; Cinelli, L.; Felli, E.;
U ade, T.; Nkusi, R.; Mu e , D.;
Ma escaux, J.; e al.
Compu e -Assis ed Di e en ia ion
be ween Colon-Mesocolon and
Re ope i oneum Using
Hype spec al Imaging (HSI)
Technology. Diagnos ics 2022,12, 2225.
h ps://doi.o g/10.3390/
diagnos ics12092225
Academic Edi o : Hyung-kwon
Byeon
Recei ed: 31 Augus 2022
Accep ed: 12 Sep embe 2022
Published: 15 Sep embe 2022
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2022 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
diagnos ics
A icle
Compu e -Assis ed Di e en ia ion be ween Colon-Mesocolon
and Re ope i oneum Using Hype spec al Imaging
(HSI) Technology
Na iaki Okamo o 1,2,* , Ma ía Ri a Rod íguez-Luna 1,2 , Valen in Benc eux 2, Mahdi Al-Tahe 1,3 ,
Lo enzo Cinelli 1,4 , E ic Felli 1, Takeshi U ade 5, Richa d Nkusi 6, Didie Mu e 1,7,8,
Jacques Ma escaux 1, Alexand e Hos e le 1,6 , Toby Collins 1,6,† and Michele Diana 1,2,†
1Resea ch Ins i u e agains Diges i e Cance (IRCAD), 67091 S asbou g, F ance
2ICube Labo a o y, Pho onics Ins umen a ion o Heal h, 67081 S asbou g, F ance
3Depa men o Su ge y, Maas ich Uni e si y Medical Cen e , 6229 ER Maas ich , The Ne he lands
4Depa men o Gas oin es inal Su ge y, San Ra aele Hospi al IRCCS, 20132 Milan, I aly
5Depa men o Su ge y, Di ision o Hepa o-Bilia y-Panc ea ic Su ge y, Kobe Uni e si y G adua e School o
Medicine, Kobe 6500017, Japan
6Resea ch Ins i u e agains Diges i e Cance (IRCAD), Kigali, Rwanda
7Depa men o Diges i e and Endoc ine Su ge y, Nou el Hôpi al Ci il, Uni e si y o S asbou g,
67091 S asbou g, F ance
8IHU-S asbou g—Ins i u de Chi u gie Guidée pa L’image, 67091 S asbou g, F ance
*Co espondence: na iaki.okamo o@i cad.
† Toby Collins and Michele Diana equally con ibu ed o his wo k and sha e he posi ion o las au ho s.
Abs ac :
Comple e mesocolic excision (CME), which in ol es he adequa e esec ion o he umo -
bea ing colonic segmen wi h “en bloc” emo al o i s mesocolon along emb yological ascial planes
is associa ed wi h supe io oncological ou comes. Howe e , CME p esen s a highe complica ion
a e compa ed o non-CME esec ions due o a highe isk o ascula inju y. Hype spec al imaging
(HSI) is a con as - ee op ical imaging echnology, which acili a es he quan i a i e imaging o
physiological issue pa ame e s and he isualiza ion o ana omical s uc u es. This s udy e alua es
he accu acy o HSI combined wi h deep lea ning (DL) o di e en ia e he colon and i s mesen e ic
issue om e ope i oneal issue. In an animal s udy including 20 pig models, in aope a i e
hype spec al images o he sigmoid colon, sigmoid mesen e y, and e ope i oneum we e eco ded.
A con olu ional neu al ne wo k (CNN) was ained o dis inguish he wo issue classes using HSI
da a, alida ed wi h a lea e-one-ou c oss- alida ion p ocess. The o e all ecogni ion sensi i i y o
he issues o be p ese ed ( e ope i oneum) and he issues o be esec ed (colon and mesen e y)
was 79.0
±
21.0% and 86.0
±
16.0%, espec i ely. Au oma ic classi ica ion based on HSI and CNNs
is a p omising ool o au oma ically, non-in asi ely, and objec i ely di e en ia e he colon and i s
mesen e y om e ope i oneal issue.
Keywo ds:
hype spec al imaging; a i icial in elligence; in aope a i e na iga ion ool; op ical
imaging; deep lea ning; con olu ional neu al ne wo k; colo ec al su ge y
1. In oduc ion
Colo ec al cance (CRC) is he hi d mos p e alen cance wi h he second highes
mo ali y a e wo ldwide [
1
]. O e he pas ew decades, he managemen o CRC has
imp o ed signi ican ly due o be e sc eening policies and he in oduc ion o ex ended
adical “en bloc” su gical esec ion such as comple e mesocolic excision (CME) [
2
]. Ho-
henbe ge p oposed CME in colonic esec ion in 2009 [
3
]. CME consis s o he subs an ial
esec ion o umo -bea ing bowel segmen s wi h “en bloc” emo al o he mesocolon along
emb yological ascial planes. Al hough CME has been associa ed wi h ad e se ou comes,
such as a g ea e in aope a i e blood loss and a highe incidence o pos ope a i e su gical
Diagnos ics 2022,12, 2225. h ps://doi.o g/10.3390/diagnos ics12092225 h ps://www.mdpi.com/jou nal/diagnos ics
Diagnos ics 2022,12, 2225 2 o 14
complica ions [
4
], se e al s udies ha e shown ha CME has supe io oncological ou -
comes [
3
,
5
–
8
]. The numbe o lymph nodes ha es ed in oncological esec ion ep esen s
an independen p ognos ic ac o o su i al [
9
–
11
], and CME specimens a e gene ally
highe quali y wi h a highe numbe o lymph nodes compa ed o non-CME esec ions,
which may accoun o he lowe isk o local ecu ence [12].
A sound unde s anding o colo ec al ana omy is c ucial o he co ec pe o mance o
CME and p e en ion o su gical complica ions. Culligan e al. [
13
] p ecisely desc ibed he
usion ascia, which is loca ed be ween he mesen e y, he e ope i oneum, and a su gical
ex olia ion laye ha is o med wi hin he usion ascia. The au ho s demons a ed ha
he su gical ex olia ion laye can be delibe a ely dissec ed when mobilizing he colon and
i s mesen e y. This ex ensi e dissec ion be ween he mesen e y and he e ope i oneum
in CME is gene ally pe o med by isually assessing di e ences in he mic o ascula u e,
which is subjec i e and equi es a high le el o expe ise. In addi ion, in some cases, i
is challenging o ecognize hese sub le pa e ns wi h he naked eye o wi h whi e-ligh
lapa oscopy. These echnical de ails ep esen majo obs acles o he iden i ica ion o
adequa e clea age and dissec ion planes du ing CME.
In a sys ema ic e iew including 18,989 pa ien s compa ing CME wi h con en ional
colo ec al esec ion o colo ec al cance , CME was associa ed wi h imp o ed 3- and 5-yea
o e all su i al, imp o ed 3-yea disease- ee su i al, and dec eased local and dis an
ecu ences [
14
,
15
]. The au ho s epo ed ha CME does no inc ease he isk o pos ope a-
i e mo ali y o anas omo ic leakage. Howe e , CME was associa ed wi h an inc eased
isk o pos ope a i e complica ions such as splenic and supe io mesen e ic ein inju ies.
Addi ionally, s uc u es behind he pe i oneum ha a e e e ed o as “ e ope i oneal”,
such as he u e e and gonadal essels, could be inju ed [
16
]. U e e al inju y is a se ious
complica ion du ing in apel ic su ge y, and he majo i y o ia ogenic u e e al inju ies
can lead o se e e mo bidi y and e en mo ali y [
17
,
18
]. The in aope a i e iden i ica ion
o ana omical laye s can p o ec o gans om ia ogenic damage du ing colo ec al su ge y.
Ongoing esea ch has been de o ed o e ining in aope a i e me hods o con i m he
ana omical posi ion o hese s uc u es, especially he u e e . Technologies include nea -
in a ed luo escence imaging (NIRF) using indocyanine g een (ICG) o me hylene blue
(MB). Howe e , hey p o ide subop imal u e e al isualiza ion in compa ison o no el
luo escen dyes such as CW800-BK and ZW800-1, which a e cu en ly unde going clinical
ansla ion [19–26]. The e o e, u e e al inju y s ill ep esen s a bu den.
O e he pas ew decades, su geons ha e made use o new op ical imaging echnolo-
gies o enhance hei ision in diges i e su ge y, especially du ing colo ec al esec ions. This
is e lec ed by a la ge numbe o s udies on in aope a i e NIRF su ge y, which ha e been
used o cha ac e ize umo s [
27
], e alua e pe usion a he le el o anas omoses [
28
–
31
],
iden i y sen inel lymph nodes [
32
], and isualize eal- ime lympha ic low in aope a-
i ely [
33
,
34
]. The ad an ages o NIRF-guided su ge y include i s high sensi i i y, apid
eedback, and lack o adia ion [
34
]. Howe e , he na ow ange o op imal concen a ions
makes i di icul o imp o e adminis a ion p o ocols (concen a ion, dose, and iming). In
addi ion, he low pene a ion o he NIR-I window (wa eleng h o 700–900 nm in he elec-
omagne ic spec um) [
35
] makes i challenging o isualize deepe ana omical s uc u es
such as he u e e . Fluo escence guidance using ICG lymphadenec omy in CME equi es
endoscopic submucosal injec ion in o he p oximi y o he umo o a leas 12 h p io o
he ope a ion o p ecise in aope a i e mapping [
36
]. Subse osal dye injec ion has also
been s udied o in aope a i ely demons a e lympha ic d ainage [
37
]. Howe e , one o he
g ea es d awbacks o ICG is i s wa e -soluble na u e; as i di uses o e ime, he abili y o
accu a ely loca e a ec ed a eas is educed.
Hype spec al imaging (HSI) is a con as - ee op ical imaging echnology, which
combines a pho og aphic came a and a spec oscope [
38
]. Con a ily o exogenous lu-
o escence (e.g., ICG and MB), HSI p o ides in aope a i e and con ac less quan i a i e
imaging o in insic physiological p ope ies [
39
], including issue oxygena ion, umo iden-
i ica ion [
40
,
41
], o gan pe usion assessmen [
42
,
43
], and iden i ica ion o key ana omical
Diagnos ics 2022,12, 2225 3 o 14
s uc u es. Machine lea ning and deep lea ning, combined wi h op ical imaging, a e gain-
ing popula i y in he medical ield o suppo su gical decision making using la ge aining
da ase s. A i icial in elligence (AI) and compu e ision ha e imp o ed compu e -assis ed
diagnosis by ecognizing dysplas ic/neoplas ic polyps in image-guided su ge y h ough
he de ec ion o c i ical s eps du ing minimally in asi e p ocedu es such as endoscopic
slee e gas ec omy [
44
]. A p esen , AI algo i hms a e equen ly de eloped using s an-
da d colo images wi h h ee op ical channels o ed, blue, and g een (RGB), which ha e
signi ican limi a ions, such as a lack o quan i a i e pa ame e s. AI-based au oma ic ecog-
ni ion o colo ec al umo s is s ill in i s in ancy, and quan i a i e op ical imaging modali ies
such as HSI could help o u he ad ance AI algo i hms owa ds a highe accu acy and
p ecision. The combina ion o HSI and AI has ecen ly gene a ed p omising esul s. Using
a ou -laye neu al ne wo k, Bo is Jansen-Winkeln e al. achie ed an 86% sensi i i y and a
95% speci ici y in iden i ying colo ec al cance (CRC) [
45
].
Ba be io e al.
showed ha HSI
combined wi h CNNs could be used o au oma ically ecognize key ana omical s uc u es
such as blood essels and ne es [
46
]. Collins and Mak abi [
47
] showed ha colo ec al and
esophagogas ic cance could also be de ec ed wi h a ious machine lea ning models, and
CNNs o en p oduced he bes esul s.
The aim o his s udy was o assess he accu acy o HSI echnology in combina ion
wi h CNNs o au oma ically dis inguish colonic and mesen e ic issue (which a e o be
esec ed in CME) om e ope i oneal issue (which is o be spa ed).
2. Ma e ials and Me hods
2.1. Animals
All expe imen s we e pe o med a he Resea ch Ins i u e agains Diges i e Cance
(IRCAD, F ance). A o al o 20 adul pig models (Sus sc o a domes icus, ssp. la ge whi e,
mean weigh : 40 kg) we e included. The s udy was pa o he ELIOS p o ocol (Endoscopic
Luminescen Imaging o Oncology Su ge y), ully app o ed by he local E hical Commi ee
on Animal Expe imen a ion (ICOMETH No. 38.2016.01.085) and by he F ench Minis y
o Supe io Educa ion and Resea ch (MESR) (APAFIS#8721-2017013010316298- 2). All
animals used in he expe imen al labo a o y we e managed acco ding o F ench laws o
animal use and ca e and acco ding o he di ec i es o he Eu opean Communi y Council
(2010/63/EU) and ARRIVE guidelines [
48
]. The animals we e housed and acclima ized
o 48 h in an en iched en i onmen , espec ing ci cadian cycles o ligh /da kness, and
wi h cons an humidi y and empe a u e condi ions. They we e as ed 24 h be o e su ge y,
wi h ad libi um access o wa e , and inally seda ed (zolazepam + ile amine 10 mg/kg
IM) 30 min be o e he p ocedu e in o de o dec ease s ess. Anes hesia induc ion was
adminis e ed wi h p opo ol (3 mg/kg) injec ed in a enously (18 G IV ca he e in an ea
ein). The animals we e main ained wi h ocu onium (0.8 mg/kg) along wi h inhaled
iso lu ane 2%. A he end o he p o ocol, animals we e eu hanized wi h a le hal dose o
pen oba bi al (40 mg/kg).
2.2. Ana omical Rele ance, Su gical P ocedu e, and Hype spec al Da a Acquisi ion
Ana omical s uc u es and hei emb yological phylogeny should be unde s ood
du ing su ge y. The mesen e y and he e ope i oneum a e de i ed om he same emb y-
ological s uc u e e e ed o as mesode m, and hey a e connec ed o he abdominal ca i y.
The mesen e y is a double laye o pe i oneum, which includes e ope i oneum, connec i e
issue, blood essels, ne es, lymph nodes, and adipose issue [
49
]. The spec al p o iles o
he mesen e y and e ope i oneum di e based on se e al pa ame e s, including wa e
con en (peaking a abou 980 nm) and adipose con en (peaking a abou
740 nm
) [
50
].
In addi ion, he spec al p o ile o hemoglobin (Hb) signi ican ly di e s be ween he oxy-
gena ed and deoxygena ed s a es and s ongly con ibu es o he o e all issue spec al
p o ile [
51
]. Changes in hemoglobin concen a ion and oxygen sa u a ion due o ascula
con en can be measu ed using HSI.
Diagnos ics 2022,12, 2225 4 o 14
To expose in ape i oneal and e ope i oneal s uc u es, he abdominal ca i y was
accessed ia a midline lapa o omy, and a sel - e aining e ac o was placed o expose he
egion o in e es (ROI).
HSI images we e ob ained om he igh o le side o he sigmoid colon mesen-
e y a he le el o he in e io mesen e ic a e y, including he sigmoid colon and he
e ope i oneum. The HSI came a (TIVITA
®
, Diaspec i e Vision GmbH, Ge many) was a
push-b oom scanning de ice wi h a complemen a y me al oxide semiconduc o (CMOS)
image senso wi h a spa ial esolu ion o 640
×
476 pixels and a spec al ange om 500
o 1000 nm (5 nm spec al esolu ion inc emen s, o aling 100 bins). In he i s inc emen ,
501 wa eleng h
bands we e used om 500 o 1000 nm. In a second inc emen , 5
×
binning
in he wa eleng h dimension was pe o med, esul ing in a inal hype cube o 100 e-
quency bins, namely he wa eleng h om 500 o 995 nm. Howe e , due o binning se ings,
he wa eleng hs om 996 o 999 we e included.
The HSI came a was posi ioned 50 cm abo e he ROI and, immedia ely be o e and
du ing image cap u e, en i onmen al ligh s we e u ned o , and en ila ion was paused.
Cap u e ime ook app oxima ely 6 s, and he placemen o he came a du ing he p ocedu e
is ep esen ed in Figu e 1. In his s udy, ela i e alues we e conside ed, and no whi e
balance was pe o med du ing image acquisi ions.
The op ical sys em causes a signi ican smile on he came a senso . A signi ican
keys one e ec is no obse able i he spec ome e is p ope ly adjus ed. Wa eleng h
calib a ion is pe o med o e e y single ow o he aw senso image. Wi h his me hod,
he wa eleng h calib a ion and he smile co ec ion a e pe o med in one single s ep.
The esul is a wa eleng h image wi h he y and he calib a ed
λ
-dimension om 500 o
1000 nm. E e y single spec ome e uni is es ed o co ec imaging o he spec al lines
o he k yp on gas lamp du ing and a e p oduc ion. This p inciple was p o en wi h
di e en spec al measu emen s anda ds and subs ances whe e spec a a e well-known
(e.g., colo checke , use o di e en liquids, e c.). The calib a ed HSI da a we e used o
de elop and ain he CNN.
2.3. Deep Lea ning Model
Based on a p e ious animal s udy om Ba be io e al. compa ing he wo mos
success ul deep lea ning models, i.e., suppo ec o machines (SVMs) [52–54] and CNNs
o HSI issue segmen a ion, he CNN model achie ed an o e all highe sensi i i y o
all issue classes (89.4%) excep o he ne e class, which had a sensi i i y o 76.3% [
46
].
Consequen ly, his combined wi h deep lea ning allowed o he au oma ic disc imina ion
o six di e en issue classes (a e y, ein, adipose, muscle, skin, and ne e), suppo ing
ou decision o adop he CNN model in his s udy.
2.4. Anno a ion
Immedia ely a e each image acquisi ion, he ope a ing su geons (N.O and M.R.R.L)
used an image manipula ion so wa e (GIMP, GNU Image Manipula ion P og am, open
sou ce) o manually anno a e he RGB images associa ed wi h each HSI (Figu e 2A).
Figu e 2B
shows examples o anno a ed images isualized in g ayscale wi h o e laid
anno a ions ep esen ed as colo ed egions. Anno a ions in g een ep esen issue o be
emo ed (colon and mesocolon), and pu ple anno a ions we e used o he issue o be
p ese ed ( e ope i oneum). The o al numbe o anno a ed colon-mesocolon pixels was
357,811, wi h an a e age o 17,890
±
12,438 (SD) pe image. The o al numbe o an-
no a ed pe i oneum pixels was 81,655, wi h an a e age o 4083
±
3749 (SD) pe image.
Consequen ly, he da a had a class imbalance o 4.32:1.
Diagnos ics 2022,12, 2225 5 o 14
Diagnos ics 2022, 12, x FOR PEER REVIEW 5 o 15
Figu e 1. Se -up du ing expe imen s: The abdominal ca i y was accessed ia a midline lapa o omy
o expose he egions o in e es (i.e., sigmoid colon, sigmoid mesocolon, and e ope i oneum). The
dis al lens o he hype spec al came a was posi ioned 50 cm abo e he ROIs. Ex e nal ligh in e -
e ence was a oided du ing image acquisi ion. The TIVITA™ issue came a is composed o a ligh -
ning uni (whi e a ow), a medical ca ( ed a ow), and a Box PC (blue a ow).
Figu e 1.
Se -up du ing expe imen s: The abdominal ca i y was accessed ia a midline lapa o omy
o expose he egions o in e es (i.e., sigmoid colon, sigmoid mesocolon, and e ope i oneum).
The dis al lens o he hype spec al came a was posi ioned 50 cm abo e he ROIs. Ex e nal ligh
in e e ence was a oided du ing image acquisi ion. The TIVITA
™
issue came a is composed o a
ligh ning uni (whi e a ow), a medical ca ( ed a ow), and a Box PC (blue a ow).
Diagnos ics 2022,12, 2225 6 o 14
Diagnos ics 2022, 12, x FOR PEER REVIEW 7 o 15
Figu e 2. Example isualiza ion o HS images, anno a ions, and au oma ic issue ecogni ion esul s
p o ided by he ained CNN model. The igu e is a anged in h ee ows o images, each co e-
sponding o h ee example images om he da ase . F om le o igh , he i s image column (A,E,I)
shows he RGB images ex ac ed om HSI da a. The second column (B,F,J) shows he anno a ions
p o ided by he expe su geons. Pu ple co esponds o e ope i oneum ( issue o be p ese ed),
and g een co esponds o colon-mesocolon ( issue o be esec ed). The hi d column (C,G,K) shows
he au oma ically p edic ed issue classes wi hin he anno a ed egions p o ided by he CNN. The
ou h column (D,H,L) shows he co esponding e o maps (showing all misclassi ied pixels).
2.5. CNN Model T aining and E alua ion
2.5.1. Image P ocessing Pipeline wi h a T ained CNN
Fo each pixel o in e es in an HSI, one HSI sub- olume was ex ac ed and cen e ed
on he co esponding spa ial coo dina es o he pixel. This sub- olume was hen used o
Figu e 2.
Example isualiza ion o HS images, anno a ions, and au oma ic issue ecogni ion esul s
p o ided by he ained CNN model. The igu e is a anged in h ee ows o images, each co espond-
ing o h ee example images om he da ase . F om le o igh , he i s image column
(A,E,I) shows
he RGB images ex ac ed om HSI da a. The second column (
B
,
F
,
J
) shows he anno a ions p o ided
by he expe su geons. Pu ple co esponds o e ope i oneum ( issue o be p ese ed), and g een
co esponds o colon-mesocolon ( issue o be esec ed). The hi d column (
C
,
G
,
K
) shows he au o-
ma ically p edic ed issue classes wi hin he anno a ed egions p o ided by he CNN. The ou h
column (D,H,L) shows he co esponding e o maps (showing all misclassi ied pixels).
A CNN was hen ained o dis inguish colon and mesocolon issue om e ope i-
oneal issue using deep lea ning.
Diagnos ics 2022,12, 2225 7 o 14
2.5. CNN Model T aining and E alua ion
2.5.1. Image P ocessing Pipeline wi h a T ained CNN
Fo each pixel o in e es in an HSI, one HSI sub- olume was ex ac ed and cen e ed on
he co esponding spa ial coo dina es o he pixel. This sub- olume was hen used o ain
a CNN, which gene a ed a p edic i e sco e o he wo issue classes, and he issue class
wi h he highes sco e was associa ed wi h he pixel. This p ocess was hen epea ed o
each pixel, gene a ing a spa ial issue p edic ion map (also known as image segmen a ion).
2.5.2. CNN A chi ec u e
CNNs ha e been he dominan machine lea ning model o pa e n ecogni ion in
op ical image da a, including HSI [
55
,
56
]. A CNN lea ns ele an spa iospec al ea u es
h ough a se ies o ainable con olu ional il e s (hidden laye s). The ex ac ed ea u es a e
ed o he ollowing laye in o de o gene a e a hie a chy o spa iospec al ea u es. The
il e weigh s a e ainable pa ame e s, which a e au oma ically adjus ed du ing aining.
Deepe hidden laye s a e used o ex ac highe le el ea u es. Based on he ex ac ed
ea u es, he las laye de e mines he classi ica ion p edic ion.
The CNN a chi ec u e ha we selec ed [
57
] has been shown o wo k well on ela ed
issue ecogni ion p oblems [
46
,
47
], and i s a chi ec u e is p o ided in Table 1. The CNN
used a spa ial window o 5 by 5 pixels and had 31,532 ainable pa ame e s wi h se en
hidden laye s, including six con olu ional laye s wi h ReLU ac i a ions. I had one ully
connec ed inal laye wi h wo ou pu neu ons co esponding o wo classes; he i s class
was colon o mesocolon issue (colon-mesocolon), and he second class was e ope i oneal
issue. Class imbalance was handled in he aining loss unc ion by downweighing he
majo i y class using in e se equency median weigh ing [58].
Table 1. De ails o CNN a chi ec u e.
Laye Fil e Shape Numbe o Ou pu
Channels S ide Numbe o T ainable
Pa ame e s
Con 1 (3.3.3) 20 (1.1.1) 560
ReLU / / / /
Pool1 (3.1.1) 20 (2.1.1) 1220
Con 2 (3.3.3) 35 (1.1.1) 18,935
ReLU / / / /
Pool2 (3.1.1) 35 (2.1.1) 3710
Con 3 (3.1.1) 35 (1.1.1) 3710
ReLU / / / /
Pool3 (2.1.1) 35 (2.1.1) 2485
ReLU / / / /
FC (455.2.1) 2 / 912
To al ainable pa ame e s: 31,532; The CNN had h ee con olu ional laye s (Con 1, Con 2, and Con 3) and h ee
pooling laye s (Pool1, Pool2, and Pool3). All h ee pooling laye s we e implemen ed as 1D con olu ional il e s
( il e ing in he spec al dimension), and hey included a s ide o 2 in he spec al dimension and a s ide o 1 in
he wo spa ial dimensions. The s ide had he e ec o educing he spec al dimension by a ac o o 2 a each
pooling laye .
2.5.3. CNN T aining
HSI sub- olumes we e cen e ed on e e y anno a ed hype spec al image pixel, o m-
ing spa ial pa ches o 5 by 5 pixels, co esponding o 5 by 5 by 100 sub- olumes. The
hi d sub- olume dimension co esponded o he spec al dimension, which ep esen ed
100 spec al
wa eleng hs p oduced by he came a. In o al, he e we e 439,466 aining
samples. Because his s udy conside ed 20 images, a ixed ain/ alida e/ es spli ( yp-
ically using 70%/30%/10%) was no possible (only wo images o es ing). As a esul ,
he model’s abili y o gene alize on independen (un ouched) da a was checked wi h k-
old c oss- alida ion, and special ca e was aken o p e en da a leakage (occu ing when
a model has access o in o ma ion in he es se , which could a i icially in la e i s pe -
o mance). To main ain es independence, no in o ma ion abou he es da a was e e
Diagnos ics 2022,12, 2225 8 o 14
used o in luence he ained CNN: he CNN’s design and se ings (e.g., spa ial window
size, s uc u e, ac i a ion unc ions, e c.) and he aining pa ame e s (e.g., lea ning a e,
weigh decay, and op imiza ion algo i hm) we e all p og ammed be o e da a we e collec ed.
Consequen ly, he e was a s ong isola ion be ween model de elopmen and pe o mance
e alua ion. The CNN design was he same as used in ou p io animal s udy on HSI-based
issue ecogni ion [
46
] and ou human s udy on HSI-based colo ec al cance ecogni ion [
59
].
All aining pa ame e s we e simila o Collins e al.’s [59], and hey we e no modi ied in
his s udy’s da a.
To ain and es he CNN, he anno a ed sub- olume samples we e spli in o aining
and es se s using 5- old c oss- alida ion (CV). Impo an ly, his was pe o med so ha he
CNN was ne e ained and es ed on da a om he same animal. The 20 hype spec al
images (one pe animal) we e andomly pa i ioned in o i e se s (S1,
. . .
, S5), wi h each se
ha ing ou HSIs. Fi e CNN models we e hen ained. The i s CNN was ained using
he anno a ed HSI sub- olumes in se s S1, S2, S3, and S4, (16 images); i s pe o mance was
hen es ed on he anno a ed sub- olumes in se S5 ( ou images). Fou o he CNN models
we e ained simila ly, each using a di e en es se ha was excluded om i s aining
se . The CNNs we e ained using ba ch g adien descen wi h bina y c oss-en opy loss
unc ion and in e se median equency class balancing. T aining was un o 250 epochs
using a ba ch size o 8192, a lea ning a e o 0.01, and a weigh decay o 0.0005. The CNNs
we e implemen ed using PyTo ch 1.4.
2.6. Pe o mance Me ics and S a is ical Me hods
Pe o mance was e alua ed wi h s anda d me ics used in machine lea ning, imple-
men ed by means o Py hon sciki -lea n ( e sion 0.22.1, h ps://sciki -lea n.o g; (accessed
on 17 Augus 2021)). Sensi i i y, speci ici y, and wo o he well-es ablished pe o mance
me ics we e used, i.e., he F1 sco e and he ecei e ope a o cu e a ea unde cu e (ROC
AUC). Unlike sensi i i y and speci ici y, F1 gi es a single pe o mance sco e ( he ha monic
mean o ecall and p ecision). The ROC AUC is a complemen a y pe o mance s a is ic,
which shows he model’s abili y o ank samples. (I gi es a highe sco e o a ue class
compa ed o a alse class.) Unlike sensi i i y, speci ici y, and F1, ROC AUC is popula since
i indica es he model’s p edic i e pe o mance wi hou equi ing a decision h eshold.
3. Resul s
3.1. Pe o mance Me ics
The quan i a i e esul s a e summa ized in Table 2. The CNNs achie ed a ela-
i ely high mean sensi i i y o colon-mesocolon (86.0
±
16.0%) and e ope i oneum
(
79.0 ±21.0%
). The mean F1 sco e was 0.90
±
0.11 o colon-mesocolon and 0.65
±
0.25 o
e ope i oneum. The mean ROC AUC was 0.92
±
0.12 o colon-mesocolon and
0.92 ±0.12
o e ope i oneum, bo h o which a e conside ed ou s anding [60].
Table 2. Model pe o mance me ics.
Mean ±SD Recall
(Sensi i i y) Speci ici y F1 sco e MCC ROC AUC
Tissue o be esec ed
0.86 ±0.16 0.79 ±0.21 0.90 ±0.11 0.60 ±0.23 0.92 ±0.12
(Colon-Mesocolon)
n= 20
Tissue o be le
0.79 ±0.21 0.86 ±0.16 0.65 ±0.25 0.60 ±0.23 0.92 ±0.12
(Re ope i neum)
n= 20
Sensi i i y, speci ici y, F1 sco e, MCC, and ROC AUC. A e aged me ics compu ed using mac o-a e aging (a
image le el) wi h s anda d de ia ion; MCC: Ma hews co ela ion coe icien ; ROC AUC: Recei e Ope a o
Cu e A ea-Unde -Cu e.
3.2. Pe o mance Visualiza ion
Visually, he CNNs pe o med di e en ia ion adequa ely be ween colon-mesocolon
and e ope i oneum wi h minimal e o s. Rep esen a i e esul s a e shown in Figu e 2
Diagnos ics 2022,12, 2225 9 o 14
wi h h ee cases a anged in h ee columns. In he i s column, mos o he colon-mesocolon
issue was co ec ly ecognized (g een egions) in cong uence wi h he ela i ely high mean
colon-mesocolon sensi i i y sco e o 0.86 (Table 1). Mos o he e ope i oneum issue was
co ec ly ecognized (blue egions). Howe e , he e was some misclassi ica ion wi h he
colon-mesocolon. I was in line wi h he lowe colon-mesocolon speci ici y o 0.79 (Table 1).
When conside ing only he i s image ( op ow in Figu e 2), he colon-mesocolon sensi i i y
was 0.99, and speci ici y was 0.80, indica ing some colon-mesocolon o e -segmen a ion.
The second image (middle ow in Figu e 2) had a sensi i i y o 0.82 and speci ici y o 0.95,
indica ing some colon-mesocolon unde -segmen a ion. The hi d image (bo om ow in
Figu e 2) had a de ec ion sensi i i y o 0.95 and speci ici y o 0.95.
4. Discussion
The objec i e o his s udy is o demons a e he abili y o he combina ion o HSI wi h
CNNs ained wi h deep lea ning o disc imina e be ween issues o be esec ed (colon and
mesocolon) om issues o be p ese ed ( e ope i oneum) du ing colo ec al esec ion.
The mains ay o image-guided su ge y in ecen decades has been ocused on NIRF.
NIRF-guided su ge y has had excellen esul s in imp o ing he isualiza ion o issue wi h
blood and lympha ic low [
27
–
34
]. Howe e , he ecogni ion o dissec ion planes necessa y
o accu a e CME su ge y is di icul o achie e using NIRF. The challenge in CME is o
de ec he so-called “a ascula plane” in which only mic oscopic blood essels pass: his is
he ocus o ICG IV in usion ascula mapping [61].
Recen ly, he e has been g ea in e es in in es iga ing con as - ee op ical imaging
modali ies. HSI has become a powe ul ool o objec i ely assess he unseen based on
he in e ac ion o emi ed, abso bed, e lec ed, and sca e ed ligh wi h biochemical issue
componen s. This echnique, which is con as - ee and non-in asi e, has he po en ial o
become a aluable na iga ion ool and he u u e o in aope a i e guidance [
62
,
63
]. How-
e e , HSI da a a e e y di icul o he su geon o di ec ly in e p e . Su geons mus ha e
access o ad anced machine lea ning algo i hms o p ocess da a in o de o au oma ically
ecognize issue pa e ns based on hei spa iospec al signa u es; his p o ides su geons
wi h p ecise and ele an issue in o ma ion, which is no isible o he naked eye o using
a s anda d RGB came a.
In he cu en
in i o
non-su i al po cine model, using HSI in combina ion wi h a
CNN, we we e able o au oma ically ecognize he ana omy o he meso helial laye ha
needs o be dissec ed du ing CME, which cons i u es a c i ical key s ep du ing ad anced
colonic esec ion.
The po en ial clinical ele ance o HSI lies in he be e eplicabili y o CME—assu ing
emo al o all lymph nodes while dec easing he ecu ence a e. An HSI-guided dissec ion
could help o achie e mo e accu a e and ex ensi e dissec ion adhe ing o emb yological
planes while p e en ing damage o o he e ope i oneal o gans and also p e en ing
neoplas ic issue om being un esec ed.
Ve y p omising esul s we e ob ained in his s udy. Howe e , he e a e some limi a-
ions. The HSI came a sys em had a limi ed elec omagne ic spec al ange (500–1000 nm),
a ela i ely low spec al esolu ion (5 nm), and low spa ial esolu ion (640 by 476 pixels).
Consequen ly, he use o low- esolu ion images could e y well dec ease accu acy in he
anno a ion p ocess e en when pe o med by expe ienced hands, which may in oduce
e o s in he da ase . To minimize such e o s, wo expe ienced su geons made anno a ions
in he ope a ing oom while e i ying he dissec ion planes in aope a i ely. The main
eason o using his ype o came a was ha he spec al p o ile could be unde s ood
a molecula le el in his spec al ange (500–1000 nm) [
35
]. I also did no dis up he
su gical wo k low (app oxima ely 6 s in acquisi ion ime), and i was clinically app o ed,
making i easie o ansla e ou s udy esul s in o clinical ials and u u e esea ch in
colo ec al su ge y.
A second limi a ion lies in he small numbe o samples using po cine models, in
which he mesen e y is signi ican ly hinne as compa ed o human models, which ha e
