Gaussian processes regressors for complex proper signals in digital communications

Depósi o de in es igación de la Uni e sidad de Se illa h ps://idus.us.es/ This is an Accep ed Manusc ip o an a icle published by 2014 IEEE 8 h Senso A ay and Mul ichannel Signal P ocessing Wo kshop (SAM) on 25/06/2014, a ailable a : h ps://doi.o g/10.1109/SAM.2014.6882359 “© 2014 IEEE. Pe sonal use o his ma e ial is pe mi ed. Pe mission om IEEEmus be ob ained o all o he uses, in any cu en o u u e media, including ep in ing/ epublishing his ma e ial o ad e ising o p omo ional pu poses, c ea ing new collec i e wo ks, o esale o edis ibu ion o se e s o lis s, o euse o any copy igh ed componen o his wo k in o he Wo ks” 1 Gaussian P ocesses Reg esso s o Complex P ope Signals in Digi al Communica ions Ra ael Boloix-To osa, F. Ja ie Pay´ an-Some and Juan Jos´ e Mu illo-Fuen es Dep. Teo ´ ıa de la Se˜ nal y Comunicaciones Escuela Supe io de Ingenie a. Uni e sidad de Se illa Camino de los Descub imien os s/n, 41092, Se illa, Spain Email: [email p o ec ed] Abs ac In his pape we de elop he complex- alued e sion o he Gaussian p ocesses o eg ession (GPR) o p ope complex signals. This ool has p o ed o be use ul in he nonlinea de ec ion in digi al communica ions in eal alued models. GPRs can be cas as nonlinea MMSE whe e hype pa ame e s can be uned op imizing a ma ginal likelihood (ML). This ea u e allows o a lexible ke nel ha can easily adap ei he o a linea o nonlinea solu ion. We in oduce he complex- alued o m o he GPR, and de elop i o he p ope complex case. We also deal wi h he op imiza ion o he ML. Some expe imen s included illus a e he good pe o mance o he p oposal. I. INTRODUCTION In digi al communica ions he op imal de ec ion is usually nonlinea and un easible due o i s compu a ional complexi y and he need o non-obse able da a. Hence app oxima e solu ions a e conside ed. The MMSE c i e ia is widely used, whe e he linea o m is p e e ed i compu a ional complexi y is o be kep o a minimum. Gaussian p ocesses o eg ession (GPR) ha e been p oposed as nonlinea MMSE solu ions ha p o ide a good comp omise be ween pe o mance and he pai complexi y plus aining da a needed [1], [2]. Thei majo ad an age being he op imiza ion o i s hype pa ame e s h ough he ma ginal likelihood. These solu ions ha e been de eloped o eal alued models, assuming ha an augmen ed inpu space wi h eal and complex pa s could be used o cope wi h he complex- alued model [3]. In [4] we may ind a good desc ip ion o he digi al communica ion model deeply e ised om he complex- alued poin o iew. Besides, esul s on he ke nel design o he complex case ha e been epo ed ecen ly [5], [6]. Based on hese wo ks, we p opose a ull desc ip ion o he GPR o p ope complex- alued sys ems. We de elop he ma ginal likelihood needed in he op imiza ion o he hype pa ame e s, and i s de i a ion o op imize i . Some expe imen al esul s ha e been included o illus a e he good pe o mance o his solu ion. Ap il 12, 2015 DRAFT 2 II. SYSTEM MODEL AND DETECTION We p opose a gene al model whe e we ansmi Ksymbols e e y sample ime, b = [b (1), b (2), ..., b (K)]>, and ecei e Nsignals a anged in he column- ec o : =P· +n ,(1) whe e = [b> ,b> −1. . . b> −Ms+1]>and n is an N-dimensional column- ec o wi h addi i e whi e Gaussian noise, whe e Ms akes in o accoun he memo y o leng h o he channel esponse. Unless needed, subindex will be omi ed he ea e . As an example, in synch onous DS-CDMA he ma ix Pin (1) summa izes he e ec o he channel, he leng h- Nsp eading codes and he di e en ampli udes o each o he Kuse s, and is a KMs-dimensional ec o including he ansmi ed bi s (Ms= 2 i 2≤Mc≤K, whe e Mcis he leng h o he channel in chip pe iods) [7], [1]. Many o he communica ion sys ems can be modeled as (1) aswell. A he ecei e , we lea n he sys em model using aining da a. In wi eless communica ions sho aining sequences a e manda o y o inc ease he numbe o in o ma ion bi s. A e he aining s age, o e e y new ecei ed ec o , deno ed as ∗, a de ec o es ima es he j- h ansmi ed symbol as ˆ b∗(j) = ( ∗) o some j, whe e his index will be omi ed he ea e . Linea de ec o s wo k well when he in e -symbolic in e e ence (ISI) and he use s in e e ence a e negligible. In he gene al case, he solu ion is highly nonlinea and nonlinea de ec o s a e necessa y o ge a good es ima ion. In digi al communica ions, linea MMSE es ima o s a e ypically used, due o hei simplici y and nea ly op imal esul s in many s anda d scena ios. The linea , ( ) = >w, MMSE de ec o can be exp essed as a Wiene il e [8]: wmmse = a gmin w Ehb− >w2i= (C )−1C b,(2) whe e C and C ba e, espec i ely, he au oco ela ion o he inpu s and he c oss-co ela ion be ween he inpu s and ou pu s. This solu ion can be es ima ed om aining da a by eplacing C and C bby hei sampled e sions: wsmmse = (R·R>)−1R·b,(3) whe e he columns o Ra e he ecei ed signals a e e y symbol in he aining se . We deno e his solu ion as linea sampled MMSE (SMMSE). Howe e , in he gene al case, he MMSE is no linea . I is widely known ha he MMSE solu ion o AWGN channels is gi en by he condi ional mean o bgi en : ˆ b∗= mmse( ∗) = E[b∗| ∗],(4) which is a nonlinea unc ion o he inpu s, unless he inpu s a e Gaussian dis ibu ed, ha i is no he case. The compu a ion o he op imal solu ion is ha d o accomplish, gi en he g ea numbe o possible combina ions o ansmi ed symbols due o he numbe o poin s in he ansmi ed cons ella ions, he numbe o use s and he channel memo y. In [1], [2] he eal alued GPR is p oposed as a nonlinea ool o es ima e he MMSE esponse. Ap il 12, 2015 DRAFT 3 Howe e , o ou knowledge, he e is no o mula ion a ailable o complex- alued sys ems. In he ollowing we de elop a comple e o mula ion o he GPR and hype pa ame e s adap a ion o he complex p ope case, which is he main con ibu ion o his wo k. III. COMPLEX GAUSSIAN PROCESSES A complex Gaussian p ocess (x)is a collec ion o complex andom a iables, any ini e numbe o which ha e a join complex Gaussian dis ibu ion. The gene al pd o a complex Gaussian andom ec o z: Ω →CDis [4] p(z) = 1 πnde 1/2Rzz exp −1 2(z−µz)HR−1 zz (z−µz),(5) whe e he augmen ed ec o zis ob ained by s acking zon op o i s complex conjuga e z∗. Simila ly, he augmen ed mean ec o µzis ob ained by s acking ec o µzon op o i s complex conjuga e µ∗ z.Rzz is he augmen ed co a iance ma ix o z Rzz =  Rzz ˜ Rzz ˜ R∗ zz R∗ zz  ,(6) whe e Rzz =E(z−µz)(z−µz)His he usual co a iance ma ix and ˜ Rzz =E(z−µz)(z−µz)>is he complemen a y co a iance o pseudo-co a iance ma ix. I is impo an o no e ha bo h Rzz and ˜ Rzz a e equi ed o a comple e second-o de cha ac e iza ion o he complex Gaussian andom ec o z∼ N µz,Rzz,˜ Rzz. A complex Gaussian p ocess (x)is comple ely speci ied by i s augmen ed mean unc ion, µ(x), i s co a iance unc ion, k(xi,xj), and i s complemen a y co a iance unc ion, ˜ k(xi,xj), i.e. (x)∼ GP(µ(x), k(xi,xj),˜ k(xi,xj)). The e is an impo an special case in which he complemen a y co a iance o a complex Gaussian ec o anishes. De ini ion 3.1: [4] I he complemen a y co a iance ma ix anishes, ˜ Rzz = 0,zis called p ope , o he wise i is called imp ope . The pd (5) o a complex p ope Gaussian andom ec o z: Ω →CDsimpli ies o p(z) = 1 πnde Rzz exp −(z−µz)HR−1 zz (z−µz).(7) We can p opose now he ollowing de ini ion: De ini ion 3.2: I he complemen a y co a iance unc ion o a complex Gaussian p ocess anishes, he p ocess is called p ope , o he wise i is called imp ope . IV. COMPLEX PROPER GAUSSIAN PROCESSES FOR REGRESSION We conside he model (x) = φ(x)Hw,(8) whe e xis he complex inpu ec o o dimension Dand we speci y a p io o e he pa ame e s w: a ze o mean p ope complex Gaussian wi h co a iance ma ix Σp, i.e., w∼ N (0,Σp,0). This linea eg ession model gi es an example o a ze o mean complex Gaussian p ocess, µ(x) = φ(x)HE[w] = 0, wi h co a iance unc ion k(xi,xj) = φ(xi)HEwwHφ(xj) = φ(xi)HΣpφ(xj) = ϕ(xi)Hϕ(xj). The complemen a y co a iance unc ion Ap il 12, 2015 DRAFT 4 anishes, ˜ k(xi,xj) = φ(xi)HEww>φ∗(xj)=0, i.e., (x)is a complex p ope Gaussian p ocess: (x)∼ GP(0, k(xi,xj),0). We can speci y a co a iance unc ion, ake ninpu ec o s xi,i= 1, ..., n agg ega ed column-wise in he D×n ma ix X, and w i e ou he co esponding n×nco a iance ma ix K(X,X)elemen wise om k(xi,xj). Fo hose inpu s we gene a e a complex p ope Gaussian andom ec o ∼ N (0,K(X,X),0). In he mo e ealis ic si ua ion y= (x) + =φ(x)Hw+, (9) he obse ed alues ydi e om (x)by addi i e complex p ope Gaussian noise wi h ze o mean and a iance σ2 n, i.e., ∼ N 0, σ2 n,0. Hence, he complex andom ec o y ha esul s om he inpu s in Xis also p ope Gaussian: y∼ N (0,C,0), wi h C=K(X,X)+σ2 nI. Fu he mo e, yand a e c oss-p ope , as ˜ Ry =Ey >= E >+E >= 0. Hence, yand a e join ly p ope [4], i.e., he composi e complex andom ec o y>, >> is Gaussian p ope . I we ake now a numbe o es inpu ec o s agg ega ed column-wise in ma ix X?, he join dis ibu ion o he aining ou pu s yand he es ou pu s ?is   y ? ∼ N  0,  C K(X,X?) K(X?,X)K(X?,X?) ,0 ,(10) and he condi ional dis ibu ion o ?gi en yis ?|x?,X,y∼ N ¯ ?,co ( ?),0,(11) whe e ¯ ?=K(X?,X)C−1y,(12) co ( ?) = K(X?,X?)−K(X?,X)C−1K(X,X?).(13) In he case ha he e is only one es inpu ec o x?, ¯ ?=k> ?C−1y,(14) co ( ?) = k(x?,x?)−k> ?C−1k?,(15) whe e k(x?) = k?is he ec o o co a iances be ween he es poin and he n aining poin s. A. The ma ginal likelihood The ma ginal likelihood (o e idence) is he in eg al o he likelihood imes he p io p(y|X) = Zp(y| ,X)p( |X)d ,(16) He e, he p io is complex p ope Gaussian, |X∼ N (0,K(X,X),0). Since yand a e join ly p ope , he likelihood is he ac o ized p ope Gaussian, y| ∼ N  , σ2 nI,0. Now, using (7) in (16) he log ma ginal likelihood yields L= log p(y|X) = −yHC−1y−log de C−nlog π, (17) i.e., y|X∼ N (0,C,0). Ap il 12, 2015 DRAFT 5 B. Model selec ion o complex p ope GP eg ession In o de o u n complex Gaussian p ocesses in o powe ul p ac ical ools i is essen ial o de elop me hods ha add ess he model selec ion p oblem. I is common o use a hie a chical speci ica ion o models, wi h he pa ame e s, wa he i s le el and some hype pa ame e s θia he second le el. In ou model, he co a iance unc ion k(xi,xj)can be pa ame e ized in e ms o he hype pa ame e s θi. I is a p ope y o he ma ginal likelihood ha i au oma ically inco po a es a ade-o be ween model i and model complexi y [9]. This is he eason why he ma ginal likelihood is aluable in sol ing he model selec ion p oblem. The log ma ginal likelihood in (17) can be explici ly w i en condi ioned on he hype pa ame e s, L(θ), wi h C=C(θ) = K(X,X) + σ2 nI, whe e θis he ec o o hype pa ame e s. We can maximize (17) in o de o se hose hype pa ame e s. No ice ha he a iance o he addi i e noise, σ2 n, is also ea ed as ano he hype pa ame e and can be es ima ed om his op imiza ion. L(θ)in (17) is a eal unc ion o a complex He mi ian ma ix C(θ) ha depends on he hype pa ame e s. The chain ule can be used o seek he de i a i e o he log ma ginal likelihood wi h espec o C(θ)and hen he de i a i e o C(θ)wi h espec o he hype pa ame e s. Howe e , he Cauchy-Riemann condi ions dic a e ha noncons an eal alued unc ions ha a e de ined on complex domains a e no holomo phic, so he complex de i a i e canno be used. Ins ead, we mus seek o mal o Wi inge de i a i es [5], [10]. The chain ule o he Wi inge de i a i e o he unc ion L(θ)wi h espec o he hype pa ame e θiyields [10] ∂L ∂θi = n X k=1 n X l=1 ∂L ∂Clk ∂Clk ∂θi +∂L ∂C∗ lk ∂C∗ lk ∂θi,(18) whe e Clk is he elemen (l, k)o ma ix C. When inding he Wi inge de i a i es ∂L ∂Clk and ∂L ∂C∗ lk he elemen s Clk and C∗ lk a e ea ed as independen a iables. The e o e, ∂L ∂θi = n X k=1 n X l=1 ∂L ∂Clk ∂Clk ∂θi = T "∂L ∂C>∂C ∂θi#.(19) The de i a i e o he i s e m o he unc ion Lin (17) wi h espec o Cyields ∂ ∂C−yHC−1y=−∂ ∂CT yHC−1y =C−> yyH>C−>.(20) The de i a i e o he second e m o he unc ion Lin (17) wi h espec o Cyields ∂ ∂C(−log de C) = −C−>.(21) Subs i u ion o (20) and (21) in (19) yield ∂L ∂θi = T C−1yyHC−1−C−1∂C ∂θi.(22) Ap il 12, 2015 DRAFT 6 C. Complex co a iance unc ions We ecall he e he complex ke nel unc ion used in he Expe imen s sec ion o his s udy, he independen Gaussian ke nel [6]. This ke nel is de ined as κ(xi,xj) = κR(αi, αj) + κR(βi, βj)) +j(κR(αi, βj)−κR(βi, αj)) ,(23) whe e xl=αl+jβland κRis he well-known eal Gaussian ke nel. This ke nel is a gene ic ex ension o κRand inhe i s he in ui i e physical meaning o a measu e o simila i y o he samples ha κRhas. V. EXPERIMENTS By eplacing and bin (1) by xand y, espec i ely, in (9) we yield he complex GPR model o es ima e b. Tha es ima e can be ound as ¯ ?in (14) o each new ecei ed ec o x?= ∗. In hese expe imen s we ocus on di e en scena ios o show he good beha io o his de ec o . We conside he e QPSK modula ions. No e ha hey a e p ope complex- alued and so a e he ecei ed signals in (1). In Fig. 1 we ha e included he BER o he complex GPR wi h he ke nel in (23) o he expe imen in Sec ion V.A in [1]. In his expe imen we ha e a DS-CDMA sys em wi h K= 2 equal powe use s and signa u es [+1+1−1−1] and [+1 −1−1 + 1]. The channel is gi en by c(z) = 0.3+0.7z−1+ 0.3z−2. The ou ecei ed chips a e he wo ma ched il e s a e he inpu s, x, i.e., N= 2. The numbe o aining samples used was n= 64. In Fig. 2 he decision bounda y is included o he complex GPR wi h 64 aining samples and SNR = 16 dB, along wi h he poin s co esponding o he bi s o he eal pa o he inpu s o bo h use s. This expe imen illus a es a scena io whe e he GPR is o majo in e es , since he linea esponse ails o de ec he ansmi ed symbols. The complex GPR success ully compu es he nonlinea solu ion, e en o he sho numbe o aining samples used. In Fig. 3 we include a DS-CDMA scena io wi h K= 10 equal powe use s and signa u es wi h N= 31 chips and a channel o 10 ze o mean complex Gaussian iid aps. We depic he BER along he numbe o aining samples, n, o a SNR = 12 dB. In his expe imen , he inpu s o he de ec o a e N= 31 ecei ed chips o one symbol pe iod. Since he la ge dimension, he solu ion is nea ly linea and he linea SMMSE exhibi s a good pe o mance. The GPR wi h he nonlinea complex ke nel adap s well o his linea solu ion, e en o a sho numbe o samples. In he sho leng h scena io he egula iza ion e m lea ned is high o compensa e o he lack o in o ma ion, and he GPR pe o ms be e han he SMMSE. As he numbe o aining samples inc eases, he nonlinea pa o he ke nel lea ns he linea solu ion and he GPR is able o ollow he SMMSE. The sligh misma ch be ween bo h cu es is again due o he egula iza ion e m. VI. CONCLUSION In his pape we de elop a comple e o mula ion o he GPR and hype pa ame e s adap a ion o he complex p ope case. Complex p ope GPR can adap ei he o linea o nonlinea solu ions wi h sho aining sequences, he e o e hey a e o in e es as de ec o s in communica ion applica ions, as shown in he included expe imen s. Ap il 12, 2015 DRAFT 7 4 6 8 10 12 14 16 18 20 10−5 10−4 10−3 10−2 10−1 SNR(dB) BER MF SMMSE GPR Fig. 1. BER along he SNR in a DS-CDMA scena io wi h K= 2 use s and signa u es [+1 + 1 −1−1] and [+1 −1−1 + 1] o SMMSE (◦), he ma ched il e () and GPR () de ec o s o n= 64 aining samples. −1 0 1 −1 −0.5 0 0.5 1 <(x(1)) <(x(2)) Fig. 2. Decision bounda y o he GPR in a DS-CDMA scena io wi h K= 2 use s and signa u es [+1 + 1 −1−1] and [+1 −1−1 + 1] o n= 64 aining samples and SNR = 16 dB. ACKNOWLEDGMENT This wo k was pa ially unded by Spanish go e nmen (Minis e io de Educaci´ on y Ciencia, Consolide -Ingenio 2010 CSD2008-00010, TEC2012-38800-C03-{02}) and Eu opean Union (FEDER). Ap il 12, 2015 DRAFT 8 50 100 150 200 250 10−5 10−4 10−3 10−2 10−1 n BER MF SMMSE GPR Fig. 3. BER along n, in a DS-CDMA scena io wi h K= 10 use s and signa u es wi h N= 31 o SMMSE (◦), he ma ched il e () and GPR () de ec o s o SNR = 12 dB. REFERENCES [1] J. Mu illo-Fuen es and F. P´ e ez-C uz, “Gaussian p ocess eg esso s o mul iuse de ec ion in DS-CDMA sys ems,” IEEE T ansac ions on Communica ions, ol. 57, no. 8, pp. 2339–2347, 2009. [2] F. P´ e ez-C uz, S. Van Vae enbe gh, J. Mu illo-Fuen es, M. Laza o-G edilla, and I. San ama ia, “Gaussian p ocesses o nonlinea signal p ocessing: An o e iew o ecen ad ances,” Signal P ocessing Magazine, IEEE, ol. 30, no. 4, pp. 40–50, July 2013. [3] F. P´ e ez-C uz, J. Mu illo-Fuen es, and S. Ca o, “Nonlinea channel equaliza ion wi h Gaussian P ocesses o Reg ession,” IEEE T ansac ions on Signal P ocessing, ol. 56, no. 10, pp. 5283–5286, Oc . 2008. [4] P. J. Sch eie and L. L. Scha , S a is ical Signal P ocessing o Complex-Valued Da a. The Theo y o Imp ope and Nonci cula Signals. Camb idge, UK: Camb idge Uni e si y P ess, 2010. [5] P. Bouboulis and S. Theodo idis, “Ex ension o Wi inge ’s calculus o ep oducing ke nel Hilbe spaces and he complex ke nel LMS,” Signal P ocessing, IEEE T ansac ions on, ol. 59, no. 3, pp. 964–978, 2011. [6] F. Toba , A. Kuh, and D. Mandic, “A no el augmen ed complex alued ke nel LMS,” in Senso A ay and Mul ichannel Signal P ocessing Wo kshop (SAM), 2012 IEEE 7 h, 2012, pp. 473–476. [7] S. Chen, A. K. Samingan, and L. Hanzo, “Suppo ec o machine mul iuse ecei e o DS-CDMA signals in mul ipa h channels,” IEEE T ansac ions on Neu al Ne wo k, ol. 12, no. 3, pp. 604–611, Decembe 2001. [8] S. Ve d´ u, Mul iuse De ec ion. Camb idge Uni e si y P ess, 1998. [9] C. E. Rasmussen and C. K. I. Williams, Gaussian P ocesses o Machine Lea ning. Massachuse s Ins i u e o Technology, Camb idge, Massachuse s: MIT P ess, 2006. [10] A. Hjo ungnes and D. Gesbe , “Complex- alued ma ix di e en ia ion: Techniques and key esul s,” Signal P ocessing, IEEE T ansac ions on, ol. 55, no. 6, pp. 2740–2746, June 2007. Ap il 12, 2015 DRAFT