Array programming with NumPy

This is a sel -a chi ed e sion o an o iginal a icle. This e sion may di e om he o iginal in pagina ion and ypog aphic de ails. Au ho (s): Ti le: Yea : Ve sion: Copy igh : Righ s: Righ s u l: Please ci e he o iginal e sion: CC BY 4.0 h ps://c ea i ecommons.o g/licenses/by/4.0/ A ay p og amming wi h NumPy © Au ho s, 2020 Published e sion Ha is, Cha les R.; Millman, K. Ja od; an de Wal , S é an J.; Gomme s, Ral ; Vi anen, Pauli; Cou napeau, Da id; Wiese , E ic; Taylo , Julian; Be g, Sebas ian; Smi h, Na haniel J.; Ke n, Robe ; Picus, Ma i; Hoye , S ephan; an Ke kwijk, Ma en H.; B e , Ma hew; Haldane, Allan; del Río, Jaime Fe nández; Wiebe, Ma k; Pe e son, Pea u; Gé a d-Ma chan , Pie e; Sheppa d, Ke in; Reddy, Tyle ; Weckesse , Wa en; Abbasi, Hamee ; Gohlke, Ch is oph; Oliphan , T a is E. Ha is, C. R., Millman, K. J., an de Wal , S. J., Gomme s, R., Vi anen, P., Cou napeau, D., Wiese , E., Taylo , J., Be g, S., Smi h, N. J., Ke n, R., Picus, M., Hoye , S., an Ke kwijk, M. H., B e , M., Haldane, A., del Río, J. F., Wiebe, M., Pe e son, P., . . . Oliphan , T. E. (2020). A ay p og amming wi h NumPy. Na u e, 585(7825), 357-362. h ps://doi.o g/10.1038/s41586-020- 2649-2 2020 Na u e | Vol 585 | 17 Sep embe 2020 | 357 Re iew A ay p og amming wi h NumPy Cha les R. Ha is1, K. Ja od Millman2,3,4 ✉, S é an J. an de Wal 2,4,5 ✉, Ral Gomme s6 ✉, Pauli Vi anen7,8, Da id Cou napeau9, E ic Wiese 10, Julian Taylo 11, Sebas ian Be g4, Na haniel J. Smi h12, Robe Ke n13, Ma i Picus4, S ephan Hoye 14, Ma en H. an Ke kwijk15, Ma hew B e 2,16, Allan Haldane17, Jaime Fe nández del Río18, Ma k Wiebe19,20, Pea u Pe e son6,21,22, Pie e Gé a d-Ma chan 23,24, Ke in Sheppa d25, Tyle Reddy26, Wa en Weckesse 4, Hamee Abbasi6, Ch is oph Gohlke27 & T a is E. Oliphan 6 A ay p og amming p o ides a powe ul, compac and exp essi e syn ax o accessing, manipula ing and ope a ing on da a in ec o s, ma ices and highe -dimensional a ays. NumPy is he p ima y a ay p og amming lib a y o he Py hon language. I has an essen ial ole in esea ch analysis pipelines in ields as di e se as physics, chemis y, as onomy, geoscience, biology, psychology, ma e ials science, enginee ing, inance and economics. Fo example, in as onomy, NumPy was an impo an pa o he so wa e s ack used in he disco e y o g a i a ional wa es1 and in he i s imaging o a black hole2. He e we e iew how a ew undamen al a ay concep s lead o a simple and powe ul p og amming pa adigm o o ganizing, explo ing and analysing scien i ic da a. NumPy is he ounda ion upon which he scien i ic Py hon ecosys em is cons uc ed. I is so pe asi e ha se e al p ojec s, a ge ing audiences wi h specialized needs, ha e de eloped hei own NumPy-like in e aces and a ay objec s. Owing o i s cen al posi ion in he ecosys em, NumPy inc easingly ac s as an in e ope abili y laye be ween such a ay compu a ion lib a ies and, oge he wi h i s applica ion p og amming in e ace (API), p o ides a lexible amewo k o suppo he nex decade o scien i ic and indus ial analysis. Two Py hon a ay packages exis ed be o e NumPy. The Nume ic pack- age was de eloped in he mid-1990s and p o ided a ay objec s and a ay-awa e unc ions in Py hon. I was w i en in C and linked o s and- a d as implemen a ions o linea algeb a3,4. One o i s ea lies uses was o s ee C++ applica ions o ine ial con inemen usion esea ch a Law ence Li e mo e Na ional Labo a o y 5 . To handle la ge as onomi- cal images coming om he Hubble Space Telescope, a eimplemen a- ion o Nume ic, called Numa ay, added suppo o s uc u ed a ays, lexible indexing, memo y mapping, by e-o de a ian s, mo e e icien memo y use, lexible IEEE 754-s anda d e o -handling capabili ies, and be e ype-cas ing ules6. Al hough Numa ay was highly compa ible wi h Nume ic, he wo packages had enough di e ences ha i di ided he communi y; howe e , in 2005 NumPy eme ged as a ‘bes o bo h wo lds’ uni ica ion 7 —combining he ea u es o Numa ay wi h he small-a ay pe o mance o Nume ic and i s ich C API. Now, 15 yea s la e , NumPy unde pins almos e e y Py hon lib a y ha does scien i ic o nume ical compu a ion8–11, including SciPy12, Ma plo lib13, pandas14, sciki -lea n15 and sciki -image16. NumPy is a communi y-de eloped, open-sou ce lib a y, which p o ides a mul- idimensional Py hon a ay objec along wi h a ay-awa e unc ions ha ope a e on i . Because o i s inhe en simplici y, he NumPy a ay is he de ac o exchange o ma o a ay da a in Py hon. NumPy ope a es on in-memo y a ays using he cen al p ocessing uni (CPU). To u ilize mode n, specialized s o age and ha dwa e, he e has been a ecen p oli e a ion o Py hon a ay packages. Unlike wi h he Numa ay–Nume ic di ide, i is now much ha de o hese new lib a ies o ac u e he use communi y—gi en how much wo k is al eady buil on op o NumPy. Howe e , o p o ide he communi y wi h access o new and explo a o y echnologies, NumPy is ansi ioning in o a cen al coo dina ing mechanism ha speci ies a well de ined a ay p og amming API and dispa ches i , as app op ia e, o special- ized a ay implemen a ions. NumPy a ays The NumPy a ay is a da a s uc u e ha e icien ly s o es and accesses mul idimensional a ays 17 (also known as enso s), and enables a wide a ie y o scien i ic compu a ion. I consis s o a poin e o memo y, along wi h me ada a used o in e p e he da a s o ed he e, no ably ‘da a ype’, ‘shape’ and ‘s ides’ (Fig.1a). h ps://doi.o g/10.1038/s41586-020-2649-2 Recei ed: 21 Feb ua y 2020 Accep ed: 17 June 2020 Published online: 16 Sep embe 2020 Open access Check o upda es 1Independen esea che , Logan, UT, USA. 2B ain Imaging Cen e , Uni e si y o Cali o nia, Be keley, Be keley, CA, USA. 3Di ision o Bios a is ics, Uni e si y o Cali o nia, Be keley, Be keley, CA, USA. 4Be keley Ins i u e o Da a Science, Uni e si y o Cali o nia, Be keley, Be keley, CA, USA. 5Applied Ma hema ics, S ellenbosch Uni e si y, S ellenbosch, Sou h A ica. 6Quansigh , Aus in, TX, USA. 7Depa men o Physics, Uni e si y o Jy äskylä, Jy äskylä, Finland. 8Nanoscience Cen e , Uni e si y o Jy äskylä, Jy äskylä, Finland. 9Me ca i JP, Tokyo, Japan. 10Depa men o Enginee ing, Uni e si y o Camb idge, Camb idge, UK. 11Independen esea che , Ka ls uhe, Ge many. 12Independen esea che , Be keley, CA, USA. 13En hough , Aus in, TX, USA. 14Google Resea ch, Moun ain View, CA, USA. 15Depa men o As onomy and As ophysics, Uni e si y o To on o, To on o, On a io, Canada. 16School o Psychology, Uni e si y o Bi mingham, Edgbas on, Bi mingham, UK. 17Depa men o Physics, Temple Uni e si y, Philadelphia, PA, USA. 18Google, Zu ich, Swi ze land. 19Depa men o Physics and As onomy, The Uni e si y o B i ish Columbia, Vancou e , B i ish Columbia, Canada. 20Amazon, Sea le, WA, USA. 21Independen esea che , Saue, Es onia. 22Depa men o Mechanics and Applied Ma hema ics, Ins i u e o Cybe ne ics a Tallinn Technical Uni e si y, Tallinn, Es onia. 23Depa men o Biological and Ag icul u al Enginee ing, Uni e si y o Geo gia, A hens, GA, USA. 24F ance-IX Se ices, Pa is, F ance. 25Depa men o Economics, Uni e si y o Ox o d, Ox o d, UK. 26CCS-7, Los Alamos Na ional Labo a o y, Los Alamos, NM, USA. 27Labo a o y o Fluo escence Dynamics, Biomedical Enginee ing Depa men , Uni e si y o Cali o nia, I ine, I ine, CA, USA. ✉e-mail: millman@be keley.edu; s e an @be keley.edu; al .gomme [email protected] 358 | Na u e | Vol 585 | 17 Sep embe 2020 Re iew The da a ype desc ibes he na u e o elemen s s o ed in an a ay. An a ay has a single da a ype, and each elemen o an a ay occupies he same numbe o by es in memo y. Examples o da a ypes include eal and complex numbe s (o lowe and highe p ecision), s ings, imes amps and poin e s o Py hon objec s. The shape o an a ay de e mines he numbe o elemen s along each axis, and he numbe o axes is he dimensionali y o he a ay. Fo example, a ec o o numbe s can be s o ed as a one-dimensional a ay o shape N, whe eas colou ideos a e ou -dimensional a ays o shape (T,M,N,3). S ides a e necessa y o in e p e compu e memo y, which s o es elemen s linea ly, as mul idimensional a ays. They desc ibe he num- be o by es o mo e o wa d in memo y o jump om ow o ow, col- umn o column, and so o h. Conside , o example, a wo-dimensional a ay o loa ing-poin numbe s wi h shape (4,3), whe e each elemen occupies 8by es in memo y. To mo e be ween consecu i e columns, we need o jump o wa d 8by es in memo y, and o access he nex ow, 3×8=24by es. The s ides o ha a ay a e he e o e (24,8). NumPy can s o e a ays in ei he C o Fo an memo y o de , i e a ing i s o e ei he ows o columns. This allows ex e nal lib a ies w i en in hose languages o access NumPy a ay da a in memo y di ec ly. Use s in e ac wi h NumPy a ays using ‘indexing’ ( o access sub- a ays o indi idual elemen s), ‘ope a o s’ ( o example, +, − and × o ec o ized ope a ions and @ o ma ix mul iplica ion), as well as ‘a ay-awa e unc ions’; oge he , hese p o ide an easily eadable, exp essi e, high-le el API o a ay p og amming while NumPy deals wi h he unde lying mechanics o making ope a ions as . Indexing an a ay e u ns single elemen s, suba ays o elemen s ha sa is y a speci ic condi ion (Fig.1b). A ays can e en be indexed using o he a ays (Fig.1c). Whe e e possible, indexing ha e ie es a suba ay e u ns a ‘ iew’ on he o iginal a ay such ha da a a e sha ed be ween he wo a ays. This p o ides a powe ul way o ope a e on subse s o a ay da a while limi ing memo y usage. To complemen he a ay syn ax, NumPy includes unc ions ha pe o m ec o ized calcula ions on a ays, including a i hme ic, s a is ics and igonome y (Fig.1d). Vec o iza ion—ope a ing on en i e a ays a he han hei indi idual elemen s—is essen ial o a ay p og amming. This means ha ope a ions ha would ake many ens o lines o exp ess in languages such as C can o en be implemen ed as a single, clea Py hon exp ession. This esul s in concise code and ees use s o ocus on he de ails o hei analysis, while NumPy handles looping o e a ay elemen s nea -op imally— o example, aking s ides in o conside a ion o bes u ilize he compu e ’s as cache memo y. When pe o ming a ec o ized ope a ion (such as addi ion) on wo a ays wi h he same shape, i is clea wha should happen. Th ough ‘b oadcas ing’ NumPy allows he dimensions o di e , and p oduces esul s ha appeal o in ui ion. A i ial example is he addi ion o a scala alue o an a ay, bu b oadcas ing also gene alizes o mo e com- plex examples such as scaling each column o an a ay o gene a ing a g id o coo dina es. In b oadcas ing, one o bo h a ays a e i ually duplica ed ( ha is, wi hou copying any da a in memo y), so ha he shapes o he ope ands ma ch (Fig.1d). B oadcas ing is also applied when an a ay is indexed using a ays o indices (Fig.1c). O he a ay-awa e unc ions, such as sum, mean and maximum, pe o m elemen -by-elemen ‘ educ ions’, agg ega ing esul s ac oss one, mul iple o all axes o a single a ay. Fo example, summing an n-dimensional a ay o e d axes esul s in an a ay o dimension n−d (Fig.1 ). NumPy also includes a ay-awa e unc ions o c ea ing, eshaping, conca ena ing and padding a ays; sea ching, so ing and coun ing da a; and eading and w i ing iles. I p o ides ex ensi e suppo o gene a ing pseudo andom numbe s, includes an asso men o p ob- abili y dis ibu ions, and pe o ms accele a ed linea algeb a, using one o se e al backends such as OpenBLAS18,19 o In el MKL op imized o he CPUs a hand (see Supplemen a y Me hods o mo e de ails). Al oge he , he combina ion o a simple in-memo y a ay ep e- sen a ion, a syn ax ha closely mimics ma hema ics, and a a ie y o a ay-awa e u ili y unc ions o ms a p oduc i e and powe ully exp essi e a ay p og amming language. In [1]: impo numpy as np In [2]: x = np.a ange(12) In [3]: x = x. eshape(4, 3) In [4]: x Ou [4]: a ay([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]]) In [5]: np.mean(x, axis=0) Ou [5]: a ay([4.5, 5.5, 6.5]) In [6]: x = x - np.mean(x, axis=0) In [7]: x Ou [7]: a ay([[-4.5, -4.5, -4.5], [-1.5, -1.5, -1.5], [ 1.5, 1.5, 1.5], [ 4.5, 4.5, 4.5]]) aDa a s uc u e gExample x = 012 345 678 9 10 11 da a da a ype shape s ides 8-by e in ege (4, 3) (24, 8) 1234567 0 8 9 10 11 8 by es pe elemen 3 × 8 = 24 by es o jump one ow down b Indexing ( iew) 10 11 9 9 x[:,1:] →wi h slices 12 45 78 0 0 3 3 6 6 x[:,::2]→wi h slices wi h s eps 02 35 6 8 911 01 1 2 3 4 4 5 67 7 8 9 10 10 11 Slices a e s a :end:s ep, any o which can be le blank dVec o iza ion +→ 01 34 67 9 10 1 1 1 1 1 1 1 1 12 45 78 10 11 eB oadcas ing × 3 6 0 9 12 → 00 36 612 9 18 Reduc ion 01 34 67 9 10 2 5 8 11 3 12 21 30 sum axis 1 18 22 26 sum axis 0 66 sum axis (0,1) c Indexing (copy) 43 76 wi h a ays wi h b oadcas ing → x→ , 2 1 1 0 x , 11 22 10 10 x wi h a ays x[0,1],x[1,2] 15 →→ 0 1 1 2 , x[x > 9] wi h masks 10 11 → →5wi h scala s x[1,2] Fig. 1 | The NumPy a ay inco po a es se e al undamen al a ay concep s. a, The NumPy a ay da a s uc u e and i s associa ed me ada a ields. b, Indexing an a ay wi h slices and s eps. These ope a ions e u n a ‘ iew’ o he o iginal da a. c, Indexing an a ay wi h masks, scala coo dina es o o he a ays, so ha i e u ns a ‘copy’ o he o iginal da a. In he bo om example, an a ay is indexed wi h o he a ays; his b oadcas s he indexing a gumen s be o e pe o ming he lookup. d, Vec o iza ion e icien ly applies ope a ions o g oups o elemen s. e, B oadcas ing in he mul iplica ion o wo-dimensional a ays. , Reduc ion ope a ions ac along one o mo e axes. In his example, an a ay is summed along selec axes o p oduce a ec o , o along wo axes consecu i ely o p oduce a scala . g, Example NumPy code, illus a ing some o hese concep s. Na u e | Vol 585 | 17 Sep embe 2020 | 359 Scien i ic Py hon ecosys em Py hon is an open-sou ce, gene al-pu pose in e p e ed p og amming language well sui ed o s anda d p og amming asks such as cleaning da a, in e ac ing wi h web esou ces and pa sing ex . Adding as a ay ope a ions and linea algeb a enables scien is s o do all hei wo k wi hin a single p og amming language—one ha has he ad an age o being amously easy o lea n and each, as wi nessed by i s adop ion as a p ima y lea ning language in many uni e si ies. E en hough NumPy is no pa o Py hon’s s anda d lib a y, i ben- e i s om a good ela ionship wi h he Py hon de elope s. O e he yea s, he Py hon language has added new ea u es and special syn ax so ha NumPy would ha e a mo e succinc and easie - o- ead a ay no a ion. Howe e , because i is no pa o he s anda d lib a y, NumPy is able o dic a e i s own elease policies and de elopmen pa e ns. SciPy and Ma plo lib a e igh ly coupled wi h NumPy in e ms o his- o y, de elopmen and use. SciPy p o ides undamen al algo i hms o scien i ic compu ing, including ma hema ical, scien i ic and enginee - ing ou ines. Ma plo lib gene a es publica ion- eady igu es and isu- aliza ions. The combina ion o NumPy, SciPy and Ma plo lib, oge he wi h an ad anced in e ac i e en i onmen such as IPy hon 20 o Jupy- e 21, p o ides a solid ounda ion o a ay p og amming in Py hon. The scien i ic Py hon ecosys em (Fig.2) builds on op o his ounda ion o p o ide se e al, widely used echnique-speci ic lib a ies15,16,22, ha in u n unde lie nume ous domain-speci ic p ojec s 23–28 . NumPy, a he base o he ecosys em o a ay-awa e lib a ies, se s documen a ion s anda ds, p o ides a ay es ing in as uc u e and adds build sup- po o Fo an and o he compile s. Many esea ch g oups ha e designed la ge, complex scien i ic lib a - ies ha add applica ion-speci ic unc ionali y o he ecosys em. Fo example, he eh -imaging lib a y29, de eloped by he E en Ho izon Telescope collabo a ion o adio in e e ome y imaging, analysis and simula ion, elies on many lowe -le el componen s o he scien i ic Py hon ecosys em. In pa icula , he EHT collabo a ion used his lib a y o he i s imaging o a black hole. Wi hin eh -imaging, NumPy a ays a e used o s o e and manipula e nume ical da a a e e y s ep in he p ocessing chain: om aw da a h ough calib a ion and image econ- s uc ion. SciPy supplies ools o gene al image-p ocessing asks such as il e ing and image alignmen , and sciki -image, an image-p ocessing lib a y ha ex ends SciPy, p o ides highe -le el unc ionali y such as edge il e s and Hough ans o ms. The ‘scipy.op imize’ module pe o ms ma hema ical op imiza ion. Ne wo kX 22 , a package o com- plex ne wo k analysis, is used o e i y image compa ison consis ency. As opy 23,24 handles s anda d as onomical ile o ma s and compu es ime–coo dina e ans o ma ions. Ma plo lib is used o isualize da a and o gene a e he inal image o he black hole. The in e ac i e en i onmen c ea ed by he a ayp og amming oun- da ion and he su ounding ecosys em o ools—inside o IPy hon o Jupy e —is ideally sui ed o explo a o y da a analysis. Use s can luidly inspec , manipula e and isualize hei da a, and apidly i e a e o e ine p og amming s a emen s. These s a emen s a e hen s i ched oge he in o impe a i e o unc ional p og ams, o no ebooks con aining bo h compu a ion and na a i e. Scien i ic compu ing beyond explo a o y wo k is o en done in a ex edi o o an in eg a ed de elopmen en i- onmen (IDE) such as Spyde . This ich and p oduc i e en i onmen has made Py hon popula o scien i ic esea ch. To complemen his acili y o explo a o y wo k and apid p o o- yping, NumPy has de eloped a cul u e o using ime- es ed so wa e enginee ing p ac ices o imp o e collabo a ion and educe e o 30. This cul u e is no only adop ed by leade s in he p ojec bu also en husi- as ically augh o newcome s. The NumPy eam was ea ly o adop dis ibu ed e ision con ol and code e iew o imp o e collabo a ion can e a Chemis y Biopy hon Biology As opy As onomy simpeg Geophysics NLTK Linguis ics Quan Econ Economics SciPy Algo i hms Ma plo lib Plo s sciki -lea n Machine lea ning Ne wo kX Ne wo k analysis pandas, s a smodels S a is ics sciki -image Image p ocessing P syc h o Py kh me Q iime2 FiPy d eepc h e m li b osa P y W a e l e s S unP y Q uTiP y n ib a b e l ye ll ow b i c k mne-py h on s c iki - HEP e h - i mag i n g MDA na l ys i s i is cesium Py C h on o Founda ion A pplica ion-speci ic Domain-speci ic Technique-speci ic A ay P o ocolsNumPy API Py hon Language IPy hon / Jupy e In e ac i e en i onmen s NumPy A ays New a ay implemen a ions Fig. 2 | NumPy is he base o he scien i ic Py hon ecosys em. Essen ial lib a ies and p ojec s ha depend on NumPy’s API gain access o new a ay implemen a ions ha suppo NumPy’s a ay p o ocols (Fig.3). 360 | Na u e | Vol 585 | 17 Sep embe 2020 Re iew on code, and con inuous es ing ha uns an ex ensi e ba e y o au o- ma ed es s o e e y p oposed change o NumPy. The p ojec also has comp ehensi e, high-quali y documen a ion, in eg a ed wi h he sou ce code31–33. This cul u e o using bes p ac ices o p oducing eliable scien i ic so wa e has been adop ed by he ecosys em o lib a ies ha build on NumPy. Fo example, in a ecen awa d gi en by he Royal As onomi- cal Socie y o As opy, hey s a e: “The As opy P ojec has p o ided hund eds o junio scien is s wi h expe ience in p o essional-s anda d so wa e de elopmen p ac ices including use o e sion con ol, uni es ing, code e iew and issue acking p ocedu es. This is a i al skill se o mode n esea che s ha is o en missing om o mal uni e si y educa ion in physics o as onomy” 34 . Communi y membe s explici ly wo k o add ess his lack o o mal educa ion h ough cou ses and wo kshops35–37. The ecen apid g ow h o da a science, machine lea ning and a i- icial in elligence has u he and d ama ically boos ed he scien i ic use o Py hon. Examples o i s impo an applica ions, such as he eh -imaging lib a y, now exis in almos e e y discipline in he na u- al and social sciences. These ools ha e become he p ima y so wa e en i onmen in many ields. NumPy and i s ecosys em a e commonly augh in uni e si y cou ses, boo camps and summe schools, and a e he ocus o communi y con e ences and wo kshops wo ldwide. NumPy and i s API ha e become uly ubiqui ous. A ay p oli e a ion and in e ope abili y NumPy p o ides in-memo y, mul idimensional, homogeneously yped ( ha is, single-poin e and s ided) a ays on CPUs. I uns on machines anging om embedded de ices o he wo ld’s la ges supe compu e s, wi h pe o mance app oaching ha o compiled languages. Fo mos i s exis ence, NumPy add essed he as majo i y o a ay compu a- ion use cases. Howe e , scien i ic da ase s now ou inely exceed he memo y capac- i y o a single machine and may be s o ed on mul iple machines o in he cloud. In addi ion, he ecen need o accele a e deep-lea ning and a i icial in elligence applica ions has led o he eme gence o special- ized accele a o ha dwa e, including g aphics p ocessing uni s (GPUs), enso p ocessing uni s (TPUs) and ield-p og ammable ga e a ays (FPGAs). Owing o i s in-memo y da a model, NumPy is cu en ly unable o di ec ly u ilize such s o age and specialized ha dwa e. Howe e , bo h dis ibu ed da a and also he pa allel execu ion o GPUs, TPUs and FPGAs map well o he pa adigm o a ay p og amming: he e o e leading o a gap be ween a ailable mode n ha dwa e a chi ec u es and he ools necessa y o le e age hei compu a ional powe . The communi y’s e o s o ill his gap led o a p oli e a ion o new a ay implemen a ions. Fo example, each deep-lea ning amewo k c ea ed i s own a ays; he PyTo ch 38 , Tenso low 39 , Apache MXNe 40 and JAX a ays all ha e he capabili y o un on CPUs and GPUs in a dis ibu ed ashion, using lazy e alua ion o allow o addi ional pe - o mance op imiza ions. SciPy and PyDa a/Spa se bo h p o ide spa se a ays, which ypically con ain ew non-ze o alues and s o e only hose in memo y o e iciency. In addi ion, he e a e p ojec s ha build on NumPy a ays as da a con aine s, and ex end i s capabili ies. Dis ib- u ed a ays a e made possible ha way by Dask, and labelled a ays— e e ing o dimensions o an a ay by name a he han by index o cla i y, compa e x[:,1] e sus x.loc[:,' ime']—by xa ay41. Such lib a ies o en mimic he NumPy API, because his lowe s he ba ie o en y o newcome s and p o ides he wide communi y wi h a s able a ayp og amming in e ace. This, in u n, p e en s dis up i e schisms such as he di e gence be weenNume ic and Numa ay. Bu explo ing new ways o wo king wi h a ays is expe imen al by na u e and, in ac , se e al p omising lib a ies (such as Theano and Ca e) ha e al eady ceased de elopmen . And each ime ha a use decides o y a new echnology, hey mus change impo s a emen s and ensu e ha he new lib a y implemen s all he pa s o he NumPy API hey cu en ly use. Ideally, ope a ing on specialized a ays using NumPy unc ions o seman ics would simply wo k, so ha use s could w i e code once, and would hen bene i om swi ching be ween NumPy a ays, GPU a ays, dis ibu ed a ays and so o h as app op ia e. To suppo a ay ope a ions be ween ex e nal a ay objec s, NumPy he e o e added he capabili y o ac as a cen al coo dina ion mechanism wi h a well speci ied API (Fig.2). To acili a e his in e ope abili y, NumPy p o ides ‘p o ocols’ (o con ac s o ope a ion), ha allow o specialized a ays o be passed o NumPy unc ions (Fig.3). NumPy, in u n, dispa ches ope a ions o he o igina ing lib a y, as equi ed. O e ou hund ed o he mos popula NumPy unc ions a e suppo ed. The p o ocols a e implemen ed by widely used lib a ies such as Dask, CuPy, xa ay and PyDa a/Spa se. Thanks o hese de elopmen s, use s can now, o example, scale hei compu a ion om a single machine o dis ibu ed sys ems using Dask. The p o ocols also compose well, allowing use s o edeploy NumPy code a scale on dis ibu ed, mul i-GPU sys ems ia, o ins ance, CuPy a ays embedded in Dask a ays. Using NumPy’s high-le el API, use s can le e age highly pa allel code execu ion on mul iple sys ems wi h millions o co es, all wi h minimal code changes42. These a ay p o ocols a e now a key ea u e o NumPy, and a e expec ed o only inc ease in impo ance. The NumPy de elope s— many o whom a e au ho s o his Re iew—i e a i ely e ine and add p o ocol designs o imp o e u ili y and simpli y adop ion. Ou pu a ays Inpu a ays NumPy API np.s ack np. eshape np. anspose np.a gmin np.mean np.s d np.max np.cos np.a c an np.log np.cumsum np.di ... NumPy a ay p o ocols In [1]: impo numpy as np In [2]: impo dask.a ay as da In [3]: x = da.a ange(12) In [4]: x = np. eshape(x, (4, 3)) In [5]: x Ou [5]: dask.a ay<..., shape=(4, 3), ... > In [6]: np.mean(x, axis=0) Ou [6]: dask.a ay<..., shape=(3,), ...> In [7]: x = x - np.mean(x, axis=0) In [8]: x Ou [8]: dask.a ay<..., shape=(4, 3), ... > A ay implemen a ion NumPy Dask CuPy PyDa a/ Spa se ... ... Dask NumP y CuPy PyDa a Spa se ... Dask NumPy CuPy PyDa a Spa se Fig. 3 | NumPy’s API and a ay p o ocols expose new a ays o he ecosys em. In his example, NumPy’s ‘mean’ unc ion is called on a Dask a ay. The call succeeds by dispa ching o he app op ia e lib a y implemen a ion (in his case, Dask) and esul s in a new Dask a ay. Compa e his code o he example code in Fig.1g. Na u e | Vol 585 | 17 Sep embe 2020 | 361 Discussion NumPy combines he exp essi e powe o a ay p og amming, he pe o mance o C, and he eadabili y, usabili y and e sa ili y o Py hon in a ma u e, well es ed, well documen ed and communi y-de eloped lib a y. Lib a ies in he scien i ic Py hon ecosys em p o ide as imple- men a ions o mos impo an algo i hms. Whe e ex eme op imiza- ion is wa an ed, compiled languages can be used, such as Cy hon43, Numba44 and Py h an45; hese languages ex end Py hon and ans- pa en ly accele a e bo lenecks. Owing o NumPy’s simple memo y model, i is easy o w i e low-le el, hand-op imized code, usually in C o Fo an, o manipula e NumPy a ays and pass hem back o Py hon. Fu he mo e, using a ay p o ocols, i is possible o u ilize he ull spec um o specialized ha dwa e accele a ion wi h minimal changes o exis ing code. NumPy was ini ially de eloped by s uden s, acul y and esea che s o p o ide an ad anced, open-sou ce a ay p og amming lib a y o Py hon, which was ee o use and unencumbe ed by license se e s and so wa e p o ec ion dongles. The e was a sense o building some hing consequen ial oge he o he bene i o many o he s. Pa icipa ing in such an endea ou , wi hin a welcoming communi y o like-minded indi iduals, held a powe ul a ac ion o many ea ly con ibu o s. These use –de elope s equen ly had o w i e code om sc a ch o sol e hei own o hei colleagues’ p oblems—o en in low-le el languages ha p eceded Py hon, such as Fo an46 and C. To hem, he ad an ages o an in e ac i e, high-le el a ay lib a y we e e iden . The design o his new ool was in o med by o he powe ul in e ac i e p og amming languages o scien i ic compu ing such as Basis 47–50 , Yo ick 51 , R 52 and APL 53 , as well as comme cial languages and en i on- men s such as IDL (In e ac i e Da a Language) and MATLAB. Wha began as an a emp o add an a ay objec o Py hon became he ounda ion o a ib an ecosys em o ools. Now, a la ge amoun o scien i ic wo k depends on NumPy being co ec , as and s able. I is no longe a small communi y p ojec , bu co e scien i ic in as uc u e. The de elope cul u e has ma u ed: al hough ini ial de elopmen was highly in o mal, NumPy now has a oadmap and a p ocess o p opos- ing and discussing la ge changes. The p ojec has o mal go e nance s uc u es and is iscally sponso ed by NumFOCUS, a nonp o i ha p omo es open p ac ices in esea ch, da a and scien i ic compu ing. O e he pas ew yea s, he p ojec a ac ed i s i s unded de elop- men , sponso ed by he Moo e and Sloan Founda ions, and ecei ed an awa d as pa o he Chan Zucke be g Ini ia i e’s Essen ials o Open Sou ce So wa e p og amme. Wi h his unding, he p ojec was (and is) able o ha e sus ained ocus o e mul iple mon hs o implemen subs an ial new ea u es and imp o emen s. Tha said, he de elop- men o NumPy s ill depends hea ily on con ibu ions made by g adu- a e s uden s and esea che s in hei ee ime (see Supplemen a y Me hods o mo e de ails). NumPy is no longe me ely he ounda ional a ay lib a y unde lying he scien i ic Py hon ecosys em, bu i has become he s anda d API o enso compu a ion and a cen al coo dina ing mechanism be ween a ay ypes and echnologies in Py hon. Wo k con inues o expand on and imp o e hese in e ope abili y ea u es. O e he nex decade, NumPy de elope s will ace se e al challenges. New de ices will be de eloped, and exis ing specialized ha dwa e will e ol e o mee diminishing e u ns on Moo e’s law. The e will be mo e, and a wide a ie y o , da a science p ac i ione s, a la ge p opo ion o whom will use NumPy. The scale o scien i ic da a ga he ing will con- inue o inc ease, wi h he adop ion o de ices and ins umen s such as ligh -shee mic oscopes and he La ge Synop ic Su ey Telescope (LSST)54. New gene a ion languages, in e p e e s and compile s, such as Rus 55, Julia56 and LLVM57, will c ea e new concep s and da a s uc u es, and de e mine hei iabili y. Th ough he mechanisms desc ibed in his Re iew, NumPy is poised o emb ace such a changing landscape, and o con inue playing a leading pa in in e ac i e scien i ic compu a ion, al hough o do so will equi e sus ained unding om go e nmen , academia and indus- y. Bu , impo an ly, o NumPy o mee he needs o he nex decade o da a science, i will also need a new gene a ion o g adua e s uden s and communi y con ibu o s o d i e i o wa d. 1. Abbo , B. P. e al. Obse a ion o g a i a ional wa es om a bina y black hole me ge . Phys. Re . Le . 116, 061102 (2016). 2. Chael, A. e al. High- esolu ion linea pola ime ic imaging o he E en Ho izon Telescope. As ophys. J. 286, 11 (2016). 3. Dubois, P. F., Hinsen, K. & Hugunin, J. Nume ical Py hon. Compu . Phys. 10, 262–267 (1996). 4. Asche , D., Dubois, P. F., Hinsen, K., Hugunin, J. & Oliphan , T. E. An Open Sou ce P ojec : Nume ical Py hon (Law ence Li e mo e Na ional Labo a o y, 2001). 5. Yang, T.-Y., Fu nish, G. & Dubois, P. F. S ee ing objec -o ien ed scien i ic compu a ions. In P oc. TOOLS USA 97. In l Con . Technology o Objec O ien ed Sys ems and Languages (eds Ege,R., Singh,M.& Meye ,B.) 112–119 (IEEE, 1997). 6. G een ield, P., Mille , J. T., Hsu, J. & Whi e, R. L. numa ay: a new scien i ic a ay package o Py hon. In PyCon DC 2003 h p://ci esee x.is .psu.edu/ iewdoc/download?d oi=10.1.1.112.9899 (2003). 7. Oliphan , T. E. Guide o NumPy 1s edn (T elgol Publishing, 2006). 8. Dubois, P. F. Py hon: ba e ies included. Compu . Sci. Eng. 9, 7–9 (2007). 9. Oliphan , T. E. Py hon o scien i ic compu ing. Compu . Sci. Eng. 9, 10–20 (2007). 10. Millman, K. J. & Ai azis, M. Py hon o scien is s and enginee s. Compu . Sci. Eng. 13, 9–12 (2011). 11. Pé ez, F., G ange , B. E. & Hun e , J. D. Py hon: an ecosys em o scien i ic compu ing. Compu . Sci. Eng. 13, 13–21 (2011). Explains why he scien i ic Py hon ecosys em is a highly p oduc i e en i onmen o esea ch. 12. Vi anen, P. e al. SciPy 1.0— undamen al algo i hms o scien i ic compu ing in Py hon. Na . Me hods 17, 261–272 (2020); co ec ion 17, 352 (2020). In oduces he SciPy lib a y and includes a mo e de ailed his o y o NumPy and SciPy. 13. Hun e , J. D. Ma plo lib: a 2D g aphics en i onmen . Compu . Sci. Eng. 9, 90–95 (2007). 14. McKinney, W. Da a s uc u es o s a is ical compu ing in Py hon. In P oc. 9 h Py hon in Science Con . (eds ande Wal ,S.& Millman,K.J.) 56–61 (2010). 15. Ped egosa, F. e al. Sciki -lea n: machine lea ning in Py hon. J. Mach. Lea n. Res. 12, 2825–2830 (2011). 16. ande Wal , S. e al. sciki -image: image p ocessing in Py hon. Pee J 2, e453 (2014). 17. ande Wal , S., Colbe , S. C. & Va oquaux, G. The NumPy a ay: a s uc u e o e icien nume ical compu a ion. Compu . Sci. Eng. 13, 22–30 (2011). Discusses he NumPy a ay da a s uc u e wi h a ocus on how i enables e icien compu a ion. 18. Wang, Q., Zhang, X., Zhang, Y. & Yi, Q. AUGEM: au oma ically gene a e high pe o mance dense linea algeb a ke nels on x86 CPUs. In SC’13: P oc. In l Con . High Pe o mance Compu ing, Ne wo king, S o age and Analysis 25 (IEEE, 2013). 19. Xianyi, Z., Qian, W. & Yunquan, Z. Model-d i en le el 3 BLAS pe o mance op imiza ion on Loongson 3A p ocesso . In 2012 IEEE 18 h In l Con . Pa allel and Dis ibu ed Sys ems 684–691 (IEEE, 2012). 20. Pé ez, F. & G ange , B. E. IPy hon: a sys em o in e ac i e scien i ic compu ing. Compu . Sci. Eng. 9, 21–29 (2007). 21. Kluy e , T. e al. Jupy e No ebooks—a publishing o ma o ep oducible compu a ional wo k lows. In Posi ioning and Powe in Academic Publishing: Playe s, Agen s and Agendas (eds Loizides,F.& Schmid ,B.) 87–90 (IOS P ess, 2016). 22. Hagbe g, A. A., Schul , D. A. & Swa , P. J. Explo ing ne wo k s uc u e, dynamics, and unc ion using Ne wo kX. In P oc. 7 h Py hon in Science Con . (eds Va oquaux,G., Vaugh ,T.& Millman,K.J.) 11–15 (2008). 23. As opy Collabo a ion e al. As opy: a communi y Py hon package o as onomy. As on. As ophys. 558, A33 (2013). 24. P ice-Whelan, A. M. e al. The As opy P ojec : building an open-science p ojec and s a us o he 2.0 co e package. As on. J. 156, 123 (2018). 25. Cock, P. J. e al. Biopy hon: eely a ailable Py hon ools o compu a ional molecula biology and bioin o ma ics. Bioin o ma ics 25, 1422–1423 (2009). 26. Millman, K. J. & B e , M. Analysis o unc ional magne ic esonance imaging in Py hon. Compu . Sci. Eng. 9, 52–55 (2007). 27. The SunPy Communi y e al. SunPy—Py hon o sola physics. Compu . Sci. Disco . 8, 014009 (2015). 28. Hamman, J., Rocklin, M. & Abe na hy, R. Pangeo: a big-da a ecosys em o scalable Ea h sys em science. In EGU Gene al Assembly Con . Abs ac s 12146 (2018). 29. Chael, A. A. e al. eh im: imaging, analysis, and simula ion so wa e o adio in e e ome y. As ophysics Sou ce Code Lib a y h ps://ascl.ne /1904.004 (2019). 30. Millman, K. J. & Pé ez, F. De eloping open sou ce scien i ic p ac ice. In Implemen ing Rep oducible Resea ch (eds S odden,V., Leisch,F.& Peng,R.D.) 149–183 (CRC P ess, 2014). Desc ibes he so wa e enginee ing p ac ices emb aced by he NumPy and SciPy communi ies wi h a ocus on how hese p ac ices imp o e esea ch. 31. ande Wal , S. The SciPy Documen a ion P ojec ( echnical o e iew). In P oc. 7 h Py hon in Science Con . (SciPy 2008) (eds Va oquaux,G., Vaugh ,T.& Millman,K.J.) 27–28 (2008). 32. Ha ing on, J. The SciPy Documen a ion P ojec . In P oc. 7 h Py hon in Science Con e ence (SciPy 2008) (eds Va oquaux,G., Vaugh ,T.& Millman, K.J.) 33–35 (2008). 33. Ha ing on, J. & Goldsmi h, D. P og ess epo : NumPy and SciPy documen a ion in 2009. In P oc. 8 h Py hon in Science Con . (SciPy 2009) (eds Va oquaux,G., ande Wal ,S.& Millman,K.J.) 84–87 (2009). 34. Royal As onomical Socie y Repo o he RAS ‘A’ Awa ds Commi ee 2020: As opy P ojec : 2020 G oup Achie emen Awa d (A) h ps:// as.ac.uk/si es/de aul / iles/2020-01/ G oup%20Awa d%20-%20As opy.pd (2020). 35. Wilson, G. So wa e ca pen y: ge ing scien is s o w i e be e code by making hem mo e p oduc i e. Compu . Sci. Eng. 8, 66–69 (2006). 362 | Na u e | Vol 585 | 17 Sep embe 2020 Re iew 36. Hannay, J. E. e al. How do scien is s de elop and use scien i ic so wa e? In P oc. 2009 ICSE Wo kshop on So wa e Enginee ing o Compu a ional Science and Enginee ing 1–8 (IEEE, 2009). 37. Millman, K. J., B e , M., Ba nowski, R. & Poline, J.-B. Teaching compu a ional ep oducibili y o neu oimaging. F on . Neu osci. 12, 727 (2018). 38. Paszke, A. e al. Py o ch: an impe a i e s yle, high-pe o mance deep lea ning lib a y. In Ad ances in Neu al In o ma ion P ocessing Sys ems 32 (eds Wallach,H.e al.) 8024–8035 (Neu al In o ma ion P ocessing Sys ems, 2019). 39. Abadi, M. e al. Tenso Flow: a sys em o la ge-scale machine lea ning. In OSDI’16: P oc. 12 h USENIX Con . Ope a ing Sys ems Design and Implemen a ion (chai s Kee on, K. & Roscoe, T.) 265–283 (USENIX Associa ion, 2016). 40. Chen, T. e al. MXNe : a lexible and e icien machine lea ning lib a y o he e ogeneous dis ibu ed sys ems. P ep in a h p://www.a xi .o g/abs/1512.01274 (2015). 41. Hoye , S. & Hamman, J. xa ay: N–D labeled a ays and da ase s in Py hon. J. Open Res. So w. 5, 10 (2017). 42. En sche , P. Dis ibu ed mul i-GPU compu ing wi h Dask, CuPy and RAPIDS. In Eu oPy hon 2019 h ps://ep2019.eu opy hon.eu/media/con e ence/slides/ X8dJsD-dis ibu ed-mul i-gpu-compu ing-wi h-dask-cupy-and- apids.pd (2019). 43. Behnel, S. e al. Cy hon: he bes o bo h wo lds. Compu . Sci. Eng. 13, 31–39 (2011). 44. Lam, S. K., Pi ou, A. & Seibe , S. Numba: a LLVM-based Py hon JIT compile . In P oc. Second Wo kshop on he LLVM Compile In as uc u e in HPC, LLVM ’15 7:1–7:6 (ACM, 2015). 45. Guel on, S. e al. Py h an: enabling s a ic op imiza ion o scien i ic Py hon p og ams. Compu . Sci. Disco . 8, 014001 (2015). 46. Donga a, J., Golub, G. H., G osse, E., Mole , C. & Moo e, K. Ne lib and NA-Ne : building a scien i ic compu ing communi y. IEEE Ann. His . Compu . 30, 30–41 (2008). 47. Ba e , K. A., Chiu, Y. H., Pain e , J. F., Mo ele , Z. C. & Dubois, P. F. Basis Sys em, Pa I: Running a Basis P og am—A Tu o ial o Beginne s UCRL-MA-118543, Vol.1 (Law ence Li e mo e Na ional Labo a o y 1995). 48. Dubois, P. F. & Mo ele , Z. Basis Sys em, Pa II: Basis Language Re e ence Manual UCRL-MA-118543, Vol.2 (Law ence Li e mo e Na ional Labo a o y, 1995). 49. Chiu, Y. H. & Dubois, P. F. Basis Sys em, Pa III: EZN Use Manual UCRL-MA-118543, Vol.3 (Law ence Li e mo e Na ional Labo a o y, 1995). 50. Chiu, Y. H. & Dubois, P. F. Basis Sys em, Pa IV: EZD Use Manual UCRL-MA-118543, Vol.4 (Law ence Li e mo e Na ional Labo a o y, 1995). 51. Mun o, D. H. & Dubois, P. F. Using he Yo ick in e p e ed language. Compu . Phys. 9, 609–615 (1995). 52. Ihaka, R. & Gen leman, R. R: a language o da a analysis and g aphics. J. Compu . G aph. S a . 5, 299–314 (1996). 53. I e son, K. E. A p og amming language. In P oc. 1962 Sp ing Join Compu e Con . 345–351 (1962). 54. Jenness, T. e al. LSST da a managemen so wa e de elopmen p ac ices and ools. In P oc. SPIE 10707, So wa e and Cybe in as uc u e o As onomy V 1070709 (SPIE and In e na ional Socie y o Op ics and Pho onics, 2018). 55. Ma sakis, N. D. & Klock, F. S. The Rus language. Ada Le e s 34, 103–104 (2014). 56. Bezanson, J., Edelman, A., Ka pinski, S. & Shah, V. B. Julia: a esh app oach o nume ical compu ing. SIAM Re . 59, 65–98 (2017). 57. La ne , C. & Ad e, V. LLVM: a compila ion amewo k o li elong p og am analysis and ans o ma ion. In P oc. 2004 In l Symp. Code Gene a ion and Op imiza ion (CGO’04) 75–88 (IEEE, 2004). Acknowledgemen s We hank R.Ba nowski, P.Dubois, M.Eickenbe g, and P.G een ield, who sugges ed ex and p o ided help ul eedback on he manusc ip . K.J.M. and S.J. .d.W. we e unded in pa by he Go don and Be y Moo e Founda ion h ough g an GBMF3834 and by he Al ed P. Sloan Founda ion h ough g an 2013-10-27 o he Uni e si y o Cali o nia, Be keley. S.J. .d.W., S.B., M.P. and W.W. we e unded in pa by he Go don and Be y Moo e Founda ion h ough g an GBMF5447 and by he Al ed P. Sloan Founda ion h ough g an G-2017-9960 o he Uni e si y o Cali o nia, Be keley. Au ho con ibu ions K.J.M. and S.J. .d.W. composed he manusc ip wi h inpu om o he s. S.B., R.G., K.S., W.W., M.B. and T.R. con ibu ed ex . All au ho s con ibu ed subs an ial code, documen a ion and/o expe ise o he NumPy p ojec . All au ho s e iewed he manusc ip . Compe ing in e es s The au ho s decla e no compe ing in e es s. Addi ional in o ma ion Supplemen a y in o ma ion is a ailable o his pape a h ps://doi.o g/10.1038/s41586-020- 2649-2. Co espondence and eques s o ma e ials should be add essed o K.J.M., S.J. .W. o R.G. Pee e iew in o ma ion Na u e hanks Edoua d Duchesnay,Alan Edelman and he o he , anonymous, e iewe (s) o hei con ibu ion o he pee e iew o his wo k. Rep in s and pe missions in o ma ion is a ailable a h p://www.na u e.com/ ep in s. Publishe ’s no e Sp inge Na u e emains neu al wi h ega d o ju isdic ional claims in published maps and ins i u ional a ilia ions. Open Access This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License, which pe mi s use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Commons license, and indica e i changes we e made. The images o o he hi d pa y ma e ial in his a icle a e included in he a icle’s C ea i e Commons license, unless indica ed o he wise in a c edi line o he ma e ial. I ma e ial is no included in he a icle’s C ea i e Commons license and you in ended use is no pe mi ed by s a u o y egula ion o exceeds he pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde . To iew a copy o his license, isi h p://c ea i ecommons.o g/licenses/by/4.0/. © The Au ho (s) 2020