T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018
167
RESEARCH PERFORMANCE OF UNIVERSITY SCHOLARS
h p://dx.doi.o g/10.1590/2318-08892018000200003
1 VŠB – Technical Uni e si y o Os a a, Facul y o Me allu gy and Ma e ials Enginee ing, Depa men o Chemis y. Lis opadu 15, 70830, Os a a-Po uba,
Czech Republic. E-mail: <pe .p aus@ sb.cz>.
Recei ed on Janua y 27, 2017, inal e sion esubmi ed on July 11, 2017 and app o ed on Augus 16, 2017.
Como ci a es e a igo/How o ci e his a icle
P aus, P. S a is ical e alua ion o esea ch pe o mance o young uni e si y schola s: A case s udy. T ansin o mação, . 30, n. 2, p. 167-177, 2018. h p://
dx.doi.o g/10.1590/2318-08892018000200003
ORIGINAL ORIGINAL
h p://dx.doi.o g/10.1590/2318-08892018000200003
CC
BY
S a is ical e alua ion o esea ch pe o mance
o young uni e si y schola s: A case s udy
A aliação es a ís ica do desempenho em pesquisa de
jo ens es udan es uni e si á ios: es udo de caso
Pe PRAUS1 0000-0002-1336-4108
Abs ac
The esea ch pe o mance o a small g oup o 49 young schola s, such as doc o al s uden s, pos doc o al and junio esea che s,
wo king in di e en echnical and scien i ic ields, was e alua ed based on 11 ypes o esea ch ou pu s. The schola s wo ked
a a echnical uni e si y in he ields o Ci il Enginee ing, Ecology, Economics, In o ma ics, Ma e ials Enginee ing, Mechanical
Enginee ing, and Sa e y Enginee ing. P incipal Componen Analysis was used o s a is ically analyze he esea ch ou pu s
and i s esul s we e compa ed wi h ac o and clus e analysis. The me ics o esea ch p oduc i i y desc ibing he ypes o
esea ch ou pu s included he numbe o pape s, books and chap e s published in books, he numbe o pa en s, u ili y models
and unc ion samples, and he numbe o esea ch p ojec s conduc ed. The me ics o ci a ion impac included he numbe
o ci a ions and h-index. F om hese me ics – he a iables – he p incipal componen analysis ex ac ed 4 main p incipal
componen s. The 1s p incipal componen cha ac e ized he ci ed publica ions in high-impac jou nals indexed by he Web o
Science. The 2nd p incipal componen ep esen ed he ou pu s o applied esea ch and he 3 d and 4 h p incipal componen s
ep esen ed o he kinds o publica ions. The esul s o he p incipal componen analysis we e compa ed wi h he hie a chical
clus e ing using Wa d’s me hod. The sca e plo s o he p incipal componen analysis and he Mahalanobis dis ances we e
calcula ed om he 4 main p incipal componen sco es, which allowed us o s a is ically e alua e he esea ch pe o mance o
indi idual schola s. Using a iance analysis, no in luence o he ield o esea ch on he o e all esea ch pe o mance was ound.
Unlike he s a is ical analysis o indi idual esea ch me ics, he app oach based on he p incipal componen analysis can p o ide
a complex iew o he esea ch sys ems.
Keywo ds: Me ics esea ch. Mul i a ia e analysis. Resea ch pe o mance e alua ion. Young schola s.
Resumo
O desempenho da pesquisa de um pequeno g upo de 49 jo ens, es udan es de dou o amen o, pesquisado es júnio es e de pós-
-dou o ado, que a uam em di e en es campos écnicos e cien í icos, oi a aliado com base em 11 ipos de esul ados de pesquisa. Os
es udan es desen ol em o seu abalho de pesquisa numa uni e sidade écnica nos campos de Engenha ia Ci il, Ecologia, Economia,
In o má ica, Engenha ia de Ma e iais, Engenha ia Mecânica e Engenha ia de Segu ança. Uma a aliação es a ís ica dos esul ados
da pesquisa oi ealizada po análise de componen es p incipais, e seus esul ados o am compa ados com a análise de a o es e
ag upamen os. As mé icas da p odu i idade da pesquisa que desc e em os ipos de esul ados de pesquisa incluí am o núme o de
a igos, li os e capí ulos de li o publicados, o núme o de pa en es, modelos de u ilidade e amos as de unção e o núme o de p oje os
de pesquisa conduzidos. As mé icas de impac o da ci ação incluí am o núme o de ci ações e o índice-h. A pa i dessas mé icas –
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018 h p://dx.doi.o g/10.1590/2318-08892018000200003
P. PRAUS
168
a iá eis –, a análise de componen es p incipais ex aiu qua o p incipais componen es. O p imei o componen e p incipal ca ac e izou
as publicações ci adas em jo nais indexados pela Web o Science. O segundo componen e p incipal ep esen ou os esul ados da
pesquisa aplicada, e o e cei o e o qua o componen es p incipais ep esen a am ou os ipos de publicações. Os esul ados das análises
de componen es p incipais o am compa ados com o mé odo hie á quico de análise de clus e s de Wa d. Os g á icos de dispe são
de análises de componen es p incipais e as dis âncias de Mahalanobis, calculadas a pa i das qua o pon uações de Componen es
P incipais, pe mi i am a alia es a is icamen e o desempenho da pesquisa indi idual dos es udan es. Usando a análise de a iância,
nenhuma in luência do campo de pesquisa no desempenho ge al da pesquisa oi encon ada. Ao con á io da análise es a ís ica das
mé icas de pesquisa indi iduais, es a abo dagem baseada em análise de componen es p incipais pode o nece uma isão complexa
dos sis emas de pesquisa.
Pala as-cha e: Mé icas de pesquisa. Análise mul i a iada. A aliação do desempenho da pesquisa. Jo ens es udan es.
In oduc ion
The pe o mance o uni e si y schola s is unde e alua ion on se e al occasions, such as du ing he
selec ion o candida es o eaching and esea ch posi ions, selec ion o esea ch p ojec s o unding, e alua ion o
e ec i eness o s udy and esea ch p og ams, e c. These decisions should be made objec i ely and anspa en ly
based on a ious eliable and objec i e c i e ia.
Resea ch pe o mance can be e alua ed by he me ics o p oduc i i y and ci a ion impac (Minge s;
Leydesdo , 2015). The i s ca ego y consis s o di e en me ics, such as he numbe o pape s, books, epo s,
chap e s in books, he numbe o pa en s and u ili y models, he numbe o g an ed esea ch p ojec s, he
amoun o unds ob ained o esea ch, e c. The second ca ego y o me ics includes he numbe o ci a ions, he
h-index (Hi sch, 2005) and o he ela ed indexes (Minge s; Leydesdo , 2015). Possibili ies o measu ing esea ch
p oduc i i y is a b oad opic, which has been widely discussed in many pape s o a long ime, e.g. (Nagpaul; Roy,
2003; Ab amo; Cice o; D’Angelo, 2013; Ab amo; D´Angelo, 2014).
A echnical uni e si ies, echnical and scien i ic ields based on applied and undamen al esea ch and
hei ou pu s can ha dly be compa ed wi h each o he . Fo example, he esul s o enginee ing disciplines a e
mos ly pa en ed con a y o hose in economics o compu e science, which a e mos ly published in pape s. The
numbe o pape s and ci a ions indica e he scien i ic impac o esea che s in hei ield (Podlubny, 2005; Podlubny;
Kassayo a, 2006). The h-index is used o measu e p oduc i i y, as well as he numbe o ci a ions o esea che s,
and i can be also used o e alua e depa men s, uni e si ies, and esea ch ins i u es (Laza idis, 2010). Al hough he
h-index is o en c i icized, pa icula ly due o he dependence on age o he esea che s and ype o scien i ic ields,
i is s ill e alua ed by di e en da abases, such as he Web o Science (WoS), Scopus and Google Schola . Some
modi ica ions in he h-index ha e been de eloped (Sch eibe ; Malesios; Psa akis, 2012).
The esul s o applied esea ch a e mos ly cha ac e ized by he numbe o pa en s, u ili y models, so wa e,
p o o ypes, unc ion samples, e c. These esul s a e mos ly de eloped in coope a ion wi h indus ial pa ne s by
con ac ual esea ch and ha e a di ec impac on he de elopmen o socie y. The numbe o g an ed p ojec s is
no a ypical esea ch pe o mance me ics, bu i shows he abili y o schola s/ esea che s o se up and conduc
p ojec s on new and a ac i e esea ch opics. These p ojec s a e o en g an ed o pe sons and eams who ha e
been p oducing high-quali y esul s ha a e app ecia ed by he esea ch communi y. In gene al, he esea ch
pe o mance is a complex mul i a ia e p oblem, which mus be sol ed by app op ia e s a is ical me hods.
The P incipal Componen Analysis (PCA) is a basic mul i a ia e s a is ical me hod ha is used o educe
dimensionali y o o iginal da a desc ibed by se e al a iables o ew main signi ican componen s (Jolli e, 2002).
PCA is used o sol e many scien i ic and echnical p oblems ha a e desc ibed by many a iables. In he case
o esea ch e alua ion, PCA has al eady been applied o analyze uni e si y ankings (Dehon; McCa hine; Ve a di,
2010; Co a; Pais; Fo mosinho, 2013; Docampo; C am, 2015), and e alua e pha maceu ical i ms (Ramani, 2002),
esea ch ins i u es (O ega; López-Rome o; Fe anández, 2011), di e en esea ch eams (Rey-Rocha; Ga zón-Ga cía;
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018
169
RESEARCH PERFORMANCE OF UNIVERSITY SCHOLARS
h p://dx.doi.o g/10.1590/2318-08892018000200003
Ma ín-Sempe e, 2006), o o he scien i ic ields (Almeida; Pais; Fo mosinho, 2009). PCA was also used o desc ibe
coun y pa icipa ion in 6 h F amewo k P og amme (O ega; Aguillo, 2010). In he li e a u e, he e a e some PCA
applica ions o disc imina e di e en kinds o scien is s cha ac e ized by se e al me ics including he h-index and
g-index (Cos as; Bo dons, 2008) o analyze 39 measu es o scien i ic impac (Bollen e al., 2009), bibliome ics, and
scien i ic me ics (F ancesche , 2009; Cos as; Van Leeuwen; Bo dons, 2010; Todeschini, 2011; Sche ibe ; Malesios;
Psa akis, 2012; To es-Salinas e al., 2013), as well as he pa en ing ac i i ies o schola s (Baldini; G imaldi; Sob e o,
2007) e c.
As gi en abo e, PCA has been widely used in Scien ome ics, bu he e a e only ew applica ions o e alua e
small g oups o esea che s. The aim o his s udy was o e alua e he esea ch pe o mance o a small g oup o 49
young schola s, whose ou pu s we e p oduced in di e en echnical and scien i ic ields and in di e en a iabili y
esul ing om di e en wo k expe iences. The esea che s consis ed o doc o al s uden s, pos doc o al, and junio
esea che s wo king a a echnical uni e si y. The pe o mance o each schola was ep esen ed by 11 me ics
consis ing o se e al ou pu s o undamen al and applied esea ch and ci a ion me ics. To e alua e he o e all
esea ch pe o mance, he Mahalanobis dis ances we e used o iden i y each schola and we e calcula ed om
selec ed p incipal componen sco es and s a is ically p ocessed. This app oach is new and i allows us o compa e
schola s wo king in di e en ields.
Me hodological p ocedu es
Da a collec ion
The da a analyzed was composed o he esea ch me ics o 49 young schola s in undamen al and applied
esea ch ca ied ou a a echnical uni e si y in he Czech Republic. Thei basic esea ch ou pu s we e ep esen ed by
he numbe o pape s published in pee - e iewed jou nals wi h impac ac o s indexed in WoS (Jimp) and wi hou
impac ac o s (J e ) indexed in o he da abases, he numbe o pape s published in Con e ence P oceedings (CP),
he numbe o Books (B) and Chap e s published in books (CH), and he numbe o Resea ch P ojec s (RP) ob ained.
The quali y o hei esea ch wo k was exp essed by he h-index (HI) and he o al numbe o Ci a ions (Ci ). The
con e ence p oceedings we e indexed by WoS. The o al numbe o ci a ions and h-index we e also e alua ed using
WoS. The applied esea ch ou pu s we e cha ac e ized by he numbe o Pa en s (PT), U ili y Models (UM), and
Func ion Samples (FS).
The basic s a is ics o he esea ch me ics we e exp essed by hei o al numbe (N), mean, s anda d
de ia ion, and maximal magni udes (Table 1). The o iginal da a ma ix con ained 51 schola s, bu 2 o hem we e
conside ed as ou lie s by he box-and-whiske diag ams and excluded om u he s a is ical ea men . These 2
schola s had an excessi e numbe o ci a ions (752) and u ili y models (18), espec i ely. The 49 schola s analyzed
wo ked in 7 di e en esea ch ields: Ci il Enginee ing (n=4), Ecology (n=16), Economics (n=7), Compu e Science
(n=7), Ma e ials Enginee ing (n=10), Mechanical Enginee ing (n=3), and Sa e y Enginee ing (n=2).
Theo y
P incipal Componen Analysis
The main objec i e o PCA is o look o new la en (hidden) a iables o n samples, which a e o hogonal
(no co ela ed) o each o he . Each la en a iable – p incipal componen – is a linea combina ion o p a iables x
and i desc ibes a di e en sou ce o o al a ia ion:
1m=xi1w1m+xi2w2m+… xip wpm (1)
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018 h p://dx.doi.o g/10.1590/2318-08892018000200003
P. PRAUS
170
Whe e im is he sco e o he i- h objec in he m- h componen . The componen loadings a e he con ibu ion
measu es o a pa icula a iable o he p incipal componen s. The a iabili y o he p incipal componen s is gi en
by co esponding eigen alues λm, whe e m=1,2, …p, which a e o de ed as λ1>λ2>… λp, whe e each eigen alue is
he a iance o he co esponding m- h componen . PCA can be pe o med by he eigen alue decomposi ion o a
co ela ion (o co a iance) ma ix o by he singula alue decomposi ion o he o iginal da a ma ix.
Clus e analysis
Clus e analysis consis s o a numbe o di e en me hods ha o ganize objec s in o g oups o simila objec s.
This explo a o y me hod is used o disco e he da a s uc u e among no only obse a ions, bu also among
a iables, a anged in o dend og ams. The u ilized me hods, algo i hms, and simila i y/dissimila i y measu es a e
desc ibed elsewhe e in he li e a u e (E e i , 2001). In his s udy, common linkage (single-linkage, comple e-
linkage, a e age linkage) and he Wa d’s hie a chical clus e ing (Wa d, 1963) me hod we e used o he analysis o
esea ch pe o mance.
S a is ic compu a ions
The o iginal da a ma ix was se up and p ocessed in MS Excel. Mul i a ia e analysis and ANOVA and o he
s a is ical calcula ions we e pe o med by he so wa e packages STATGRAPHIC Plus 5.0, QC.Expe 3.3.0.6. (T iloby e)
and XLSTAT 2017 (Addinso ). Be o e he mul i a ia e analysis, he da a we e s anda dized o a oid misclassi ica ions
a ising om di e en o de s o magni ude o he a iables. Fo his pu pose, he da a we e mean cen e ed (μ) and
scaled by he s anda d de ia ions (σ) as (x-μ)/σ.
Resul s and Discussion
The esea ch pe o mance o he young schola s was cha ac e ized by he 11 me ics shown in Table 1. The
me ics we e selec ed o co e a ious disciplines, in which he undamen al and applied esea ch was pe o med.
The me ics showing publica ion ac i i ies o he schola s we e he numbe o pape s published in jou nals and
con e ence p oceedings, and he numbe o books and chap e s published in books. The applied esea ch esul s
we e cha ac e ized by he numbe o pa en s, u ili y models, and unc ion samples. The quali y o esea ch wo k was
exp essed by he h-index, he numbe o ci a ions o he o e all scien i ic p oduc ion, and he numbe o esea ch
p ojec s conduc ed. The a icles in jou nals wi h impac ac o , ci a ions, and h-index we e aken om WoS, which
has been p e e ed by he Czech esea ch e alua ion sys em (Vaněček, 2014; Good e al., 2015). The pee - e iewed
jou nals we e aken om o he da abases, such as Scopus, ERIH, and a lis o he Czech non-impac and
pee - e iewed jou nals (R&D…, 2015).
P incipal componen analysis was pe o med by compu ing he eigen alues and co esponding eigen ec o s
o he co ela ion ma ix composed o he abo emen ioned esea ch me ics. P io o PCA, he Ba le ’s Sphe ici y
es con i med ha he co ela ion ma ix was signi ican ly di e en om he iden i y one (p<0.0001). The e is no
uni e sal ule o es ima ing he numbe o PCs. The eigen alues o all PCs we e calcula ed a 2.9866, 1.9727, 1.5726,
0.9778, 0.8939, 0.7556, 0.6814, 0.62269, 0.2982, 0.1612 and 0.0773. The co esponding cumula i e a iabili ies
we e calcula ed a 27.15%, 45.09%, 59.38%, 68.27%, 76.40%, 83.27%, 89.46%, 95.12%, 97.83%, 9.30% and 100.00%,
espec i ely. Acco ding o he magni ude o eigen alue, which should be equal o o highe han 1 (Kaise , 1960), 3
main PCs explaining 59.00% o he o al da a a iance can be selec ed. Howe e , his o al a iance is low, so a sc ee
plo was used o es ima e he numbe o app op ia e PCs (Ca ell, 1966). The s eep eigen alue dec ease s opped a
he 4 h PC, he e o e, 4 main PCs p o iding 68.00% o he o al da a a iance we e selec ed: 1s PC (PC1), 2nd (PC2), 3 d
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018
171
RESEARCH PERFORMANCE OF UNIVERSITY SCHOLARS
h p://dx.doi.o g/10.1590/2318-08892018000200003
PC (PC3) and 4 h PC (PC4) explained 27.20%, 17.90%, 14.30% and 8.90% o he o al a iance, espec i ely. Thus, he
o iginal da a dimensionali y was educed om 11 o 4. Rela ionships among he esea ch me ics we e discussed
and he indi idual schola s we e isualized by means o he PC sca e plo s.
In e p e a ion o p incipal componen s
In gene al, he in e p e a ion o p incipal componen s is necessa y o unde s and he da a s uc u e. The
componen loadings (Table 2) we e conside ed in o de o us o ind ela ionships among o iginal me ics.
The i s PC was mos ly in luenced by he numbe o a icles in jou nals wi h impac ac o s, numbe o hei
ci a ions, and he au ho s’ h-index. I is no su p ising ha all hese me ics co ela ed well wi h each o he because
hey usually indica e high-quali y publica ion esul s. The highes a iabili y o hese me ics was gi en by he high
di e ences among indi idual schola s publishing in jou nals wi h impac ac o s.
The second PC was mos ly sa u a ed by he ou pu s o applied esea ch ( unc ion samples, u ili y models,
and pa en s) and he numbe o book chap e s. Unlike Jimp, he numbe o all hese ou pu s we e ela i ely low
(Table 1) and we e p oduced by se e al au ho s. The e o e, he a ia ion o PC2 was lowe . The unc ion samples
we e he p e ailing applied esea ch ou pu . Thei numbe well co ela ed wi h he numbe o u ili y models
because hey we e mos ly c ea ed by he same schola s. The numbe o pa en s and u ili y models also signi ican ly
Table 1. Basic s a is ics o schola s’ ou pu s.
Jimp J e CP B CH RP HI Ci PT UM FS
N363 653 424 87 61 188 -- 1303 10 10 32
Mean 7.41 13.3 8.65 1.78 1.24 3.84 2.41 26.6 0.20 0.20 0.65
SD 6.96 12.9 18.1 2.60 1.90 3.86 1.81 41.3 0.61 0.58 1.41
Max 33 60 121 13 10 15 9 233 3 3 6
No e: CP: Con e ence P oceedings; B: Numbe o Books; CH: Chap e published in books; RP: Resea ch P ojec s; HI: h-index; Ci : Ci a ions; PT: Pa en s; UM: U ili y
Models; FS: Func ion Samples.
Sou ce: P epa ed by au ho (2017).
Table 2. Loadings o 4 main p incipal componen s.
Me ics PC1 PC2 PC3 PC4
B 0.2556 -0.2672 0.2919 -0.1052
Ci 0.5538 -0.0259 -0.0398 0.0211
CP 0.0116 0.2188 -0.4774 -0.5561
FS 0.0434 0.4200 0.1784 0.0480
HI 0.5322 0.0541 -0.1499 0.4991
CH -0.0628 -0.4229 0.0732 0.0380
Jimp 0.5156 0.0545 -0.1910 -0.2211
J e -0.0054 -0.2616 0.3321 0.4965
PT -0.0083 0.3540 0.4283 0.2677
RP 0.2607 -0.2188 0.4184 0.0322
UPT 0.0702 0.5267 0.3470 0.2416
No e: PC: P incipal Componen .
Sou ce: P epa ed by au ho (2017).

T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018 h p://dx.doi.o g/10.1590/2318-08892018000200003
P. PRAUS
172
co ela ed wi h each o he o he same eason. PC2 also con ained a ela i ely high loading o he book chap e s,
which nega i ely co ela ed wi h he applied esea ch esul s. The au ho s who published he chap e s we e no
hose who we e p oducing pa en s and o he applied esul s.
The hi d PC was mainly in luenced by he numbe o pape s in pee - e iewed jou nals wi hou impac
ac o s, he numbe o pape s in con e ence p oceedings, books and esea ch p ojec s. The low a iabili y o he
me ics, such as J e and CP, indica ed ha a majo i y o he schola s published in hese media. Thei nega i e
co ela ion indica ed ha he au ho s we e deciding be ween bo h possibili ies. A ew books we e published by
se e al au ho s mos ly in Czech publishing houses. The e o e, his me ic showed low a ia ion and was included in
PC3. The numbe o esea ch p ojec s was co ela ed wi h he numbe o books because bo h ou pu s mos ly had
he same au ho s. The high posi i e loading o he pa en s (0.4283) canno be explained by hei ela ionships wi h
hese me ics. A p obable explana ion is ha he esea ch me ics o low magni udes and low a ia ions weakly
co ela e wi h each o he wi hou any logical eason.
The ou h PC, jus as PC3, was sa u a ed by he numbe o pape s in pee - e iewed jou nals and numbe
o pape s in con e ence p oceedings, bu he h-index also in luenced his componen . Howe e , he co ela ion
coe icien s be ween hese a iables we e oo low (up o -0.204) o come o any conclusion.
In gene al, he main PCs we e ound o cha ac e ize well he a icles ci ed in jou nals wi h impac ac o
(PC1), he esul s o applied esea ch (PC2), and o he ypes o publica ions (PC3 and PC4). The PCA esul s we e
con i med by ac o analysis, which iden i ied simila ac o loadings (no shown he e).
E alua ion o schola s in PC space
The schola s we e e alua ed acco ding o hei pe o mance using he PCA sca e plo s. The sca e plo
composed o he PC1 and PC2 sco es is shown in Figu e 1a. The high posi i e PC1 sco es indica e many well-ci ed
pape s published in high-impac jou nals, which is ypical o schola s wo king in he ields o Ma e ials Enginee ing,
Ecology, and Economics. The high posi i e PC2 sco es ep esen he high numbe o applied esea ch esul s, bu
he high nega i e PC2 sco es indica e a lo o chap e s in books. The e o e, he poin s in he 1s quad an ep esen
he schola s wi h he bes pe o mance. The schola s placed in he 4 h quad an we e good a publishing and hose
in 2nd quad an c ea ed mo e ou pu s o applied esea ch ou pu s. Mos o he schola s in he 3 d quad an c ea ed
ela i ely he low numbe s o pape s as well as applied esea ch ou pu s. Ob iously, he schola s nº 31, 41 and 42
signi ican ly di e om he o he s due o hei high numbe o a icles in high-impac jou nals and, consequen ly,
he high numbe o ci a ions and h-indexes. The schola nº 1 is di e en due o he highe numbe o pa en s, u ili y
models, and unc ion samples. The schola s nº 12 and 30 published a lo o book chap e s.
Figu e 1b shows he sca e plo o he PC1 and PC3 sco es. The poin s o numbe s 1, 31, 41 and 42 a e also
well isible. The schola nº 3 published many con ibu ions in he con e ence p oceedings. The schola s wi h high
nega i e PC3 sco es published in con e ence p oceedings. Likewise, mos o he schola s a e concen a ed in he
2nd and 3 d quad an s. The wo sca e plo s also show ha no well-de ined clus e s consis ing o he schola s om
he same o simila ields we e ound.
As discussed abo e, all he schola s can be cha ac e ized by he coo dina es in he PC space. Thei loca ions
in he sca e plo quad an s as well as hei dis ances om he coo dina e o igins a e impo an o hei e alua ion.
To imp o e esolu ion o indi idual pe sons, he Mahalanobis dis ance T o each schola was calcula ed as
ollows:
(3)




ii i
=-
C-1
-
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018
173
RESEARCH PERFORMANCE OF UNIVERSITY SCHOLARS
h p://dx.doi.o g/10.1590/2318-08892018000200003
Whe e μ is he mean o he PC sco e xi, C is he co ela ion ma ix composed o he PC sco es (Maesschalck;
Jouan-Rimbaud; Massa , 2000), and m is he numbe o p incipal componen s (m=4). S a is ical es s o he T alues
o all 49 schola s showed he logno mal dis ibu ion. The e o e, loga i hms o he dis ances T we e calcula ed o
ob ain he no mal dis ibu ion (Skewness=0.596, Ku osis=3.85) o u he s a is ical use; he no mali y was also
con i med by he D’Agos ino (p=0.196) and Kolmogo o -Smi no (p=0.133) es s. The o e all pe o mance cha
was de eloped as shown in Figu e 2.
The mean μT=0.157 and s anda d de ia ion σT=0.233 o logT we e calcula ed. The pe o mance limi μT+2σT
(=0.623) sepa a es he schola wi h excellen o weak esul s. The o e all pe o mance o he schola s be ween μT
and μT+σT (=0.390) should be be e o wo se han ha o he a e age schola s who can be ound below μ, ha
is, close o he o igin o he PC space. The excep ional schola s, numbe s 1, 3, 18, 31, 41, 42, p e iously iden i ied in
he sca e plo s a e well isible. The o e all pe o mance cha can be also used as a con ol cha o he s a is ical
e alua ion o schola s o e a long cou se o ime.
The Mahalanobis dis ances in he esea ch ields we e also es ed by one-way ANOVA. The a iance es s, such
as Coch an’s C (p=0.397), Ba le ’s (p=0.570), Ha ley’s, Le ene’s (p=0.982), K uskal-Wallis (p=0.514) and Sche é’s es s
(p=0.941-1.000) con i med ha he e we e no s a is ically signi ican di e ences be ween he s anda d de ia ions
o he T magni udes co esponding o he ields. In addi ion, he mul i a ia e ANOVA (MANOVA) was applied o he
o iginal da a. Wilks’s es (Rao’s app oxima ion) con i med ha he e was no signi ican e ec o he ields on he
esea ch ou pu s (p=0.403).
Compa ison o P incipal Componen Analysis wi h clus e analysis
The PCA esul s we e compa ed wi h hie a chical clus e ing o he o iginal 11-dimensional da a. The linkage
clus e ing me hods we e es ed, bu he bes o ganized dend og ams we e ob ained by Wa d´s me hod.
The dend og am in Figu e 3 shows wo main clus e s co esponding o he publica ion cha ac e is ics and
he applied esea ch cha ac e is ics. The le “publica ion” clus e is di ided in o wo sub-clus e s: he publica ions in
impac jou nal and he ci a ion cha ac e is ics simila o PC1 and o he publica ions simila o PC3. The composi ion
o he igh “applied esea ch clus e ” is simila o PC2. Unlike PCA, he book chap e s we e included in he
Figu e 1. Sca e plo o PC1 and PC2 sco es.
Sou ce: P epa ed by au ho (2017).
6
5
4
3
2
1
0
-1
-2
-3
1
17
33 318
36
46 25 26
19
213
47
49
41
16
37
48 32
24
20 8
214
2840
35
38
29
15
11
45
12
30
31
23 4
P1C
-2 02468
4
35
46
164
2
P 2C
42
29
A
4
2
0
-2
-4
-6
P 3C
P1C
1
18
12 15
29
11 6
17
38 5
25 37
49942
3
33 26 31 42
41
19
24
2836
43
8
4
35
32
44
7
7
21
66
6
16
22
34
4
0
8
4
8
2
2
466
B
-2 04
68
2
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018 h p://dx.doi.o g/10.1590/2318-08892018000200003
P. PRAUS
174
“publica ion” clus e ”. The easons may be due o he di e en mechanisms o he wo me hods and he p esence
o in o ma ion noise, which was emo ed by PCA.
Two main clus e s a e isible in he dend og am in Figu e 3b. The composi ion o he clus e s was compa ed
wi h he poin s dis ibu ed in quad an s o he sca e plo shown in Figu e 1a. The le clus e con ains he schola s
om he 1s quad an and a ew schola s om he 2nd and 4 h quad an s. The igh la ge clus e can be di ided in o
wo sub-clus e s combining he schola s om he 2nd, 3 d and 4 h quad an s. Wa d´s me hod was also applied o
he clus e ing o he schola s exhibi ed in he PC space, bu he composi ion o clus e s was mixed. In addi ion, he
sepa a ion o schola s in o clus e s was independen on hei pe o mance.
The dend og am o he esea ch ou pu s co esponded well wi h he esul s o he PCA. The PCA sca e
plo s we e mo e anspa en and be e o ganized han he dend og ams. The posi ion o schola s in he quad an s
can easily explain hei ac i i ies: he publica ion o a icles in high-impac jou nals connec ed wi h ci a ion impac
(PC1), he p oduc ion o applied esea ch ou pu s (PC2) and o he publica ions (PC3 and PC4). Be e esolu ion o
indi idual schola s was achie ed using hei o e all pe o mance cha .
Schola ’s e alua ion in con ex wi h esea ch me ics
The PCA esul s p o ided an insigh in o he s uc u e o he young schola s wo king in di e en esea ch
ields. I was ound ha he publica ion o pape s in high-impac and well-ci ed jou nals was he main p oblem
Figu e 2. O e all esea ch pe o mance cha o schola s e alua ed.
Sou ce: P epa ed by au ho (2017).
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
01020304050
Schola s




1
3
12
13 15
18
17
2
11
10
4
79
5616
20
21
22
23
14 19
8
24
25 26 28
27
29
30
31
34
32
37
38
40
41
42
46
44
45 47 49
48
33
36 39 43
35
T ansIn o mação, Campinas, 30(2):167-177, maio/ago., 2018
175
RESEARCH PERFORMANCE OF UNIVERSITY SCHOLARS
h p://dx.doi.o g/10.1590/2318-08892018000200003
Figu e 3. Wa d´s dend og am o esea ch ou pu s and Wa d´s dend og am o schola s.
Sou ce: P epa ed by au ho (2017).
B
RP
CH
J e
Ci
HI
Jimp
CP
FS
PT
UM
500
400
300
200
100
0
Dis ance
A
1
13
33
17
18
3
15
31
19
25
26
37
24
32
41
42
2
7
47
20
48
14
28
16
36
22
8
27
39
44
9
10
21
35
46
5
6
11
45
43
38
34
29
12
23
40
30
4
49
Dis ance
600
500
400
300
200
100
0
B
indica ed by he highes a ia ion. The schola s mus be mo i a ed o conduc high-quali y esea ch whose esul s
could be published in well-ci ed scien i ic jou nals (Viei a; Gomes, 2010). I is one o he ways o imp o e he o e all
esea ch pe o mance and he quali y o he educa ional sys em.
On he o he hand, he a icles in con e ence p oceedings we e equen esul s. The e a e wo main easons
o his. Fi s , in compa ison wi h high-impac scien i ic jou nals, he demands on he quali y o con e ence pape s
a e lowe . This e ec is e iden especially in cases o la ge in e na ional mul idisciplina y con e ences. Second, he
publica ion p ocess o con e ence p oceedings is much as e han in scien i ic jou nals. These ac o s play a ole in
apidly de eloping ields, such as enginee ing sciences and compu e science (La sen; Ins, 2010).