Depósi o de in es igación de la Uni e sidad de Se illa
h ps://idus.us.es/
This is an Accep ed Manusc ip o an a icle published by 2014 IEEE 8 h Senso
A ay and Mul ichannel Signal P ocessing Wo kshop (SAM) on 25/06/2014,
a ailable a : h ps://doi.o g/10.1109/SAM.2014.6882359
“© 2014 IEEE. Pe sonal use o his ma e ial is pe mi ed. Pe mission om IEEEmus be
ob ained o all o he uses, in any cu en o u u e media, including
ep in ing/ epublishing his ma e ial o ad e ising o p omo ional pu poses, c ea ing
new collec i e wo ks, o esale o edis ibu ion o se e s o lis s, o euse o any
copy igh ed componen o his wo k in o he Wo ks”
1
Gaussian P ocesses Reg esso s o Complex
P ope Signals in Digi al Communica ions
Ra ael Boloix-To osa, F. Ja ie Pay´
an-Some and Juan Jos´
e Mu illo-Fuen es
Dep. Teo ´
ıa de la Se˜
nal y Comunicaciones
Escuela Supe io de Ingenie a. Uni e sidad de Se illa
Camino de los Descub imien os s/n, 41092, Se illa, Spain
Email: [email p o ec ed]
Abs ac
In his pape we de elop he complex- alued e sion o he Gaussian p ocesses o eg ession (GPR) o p ope
complex signals. This ool has p o ed o be use ul in he nonlinea de ec ion in digi al communica ions in eal alued
models. GPRs can be cas as nonlinea MMSE whe e hype pa ame e s can be uned op imizing a ma ginal likelihood
(ML). This ea u e allows o a lexible ke nel ha can easily adap ei he o a linea o nonlinea solu ion. We
in oduce he complex- alued o m o he GPR, and de elop i o he p ope complex case. We also deal wi h he
op imiza ion o he ML. Some expe imen s included illus a e he good pe o mance o he p oposal.
I. INTRODUCTION
In digi al communica ions he op imal de ec ion is usually nonlinea and un easible due o i s compu a ional
complexi y and he need o non-obse able da a. Hence app oxima e solu ions a e conside ed. The MMSE c i e ia
is widely used, whe e he linea o m is p e e ed i compu a ional complexi y is o be kep o a minimum. Gaussian
p ocesses o eg ession (GPR) ha e been p oposed as nonlinea MMSE solu ions ha p o ide a good comp omise
be ween pe o mance and he pai complexi y plus aining da a needed [1], [2]. Thei majo ad an age being
he op imiza ion o i s hype pa ame e s h ough he ma ginal likelihood. These solu ions ha e been de eloped o
eal alued models, assuming ha an augmen ed inpu space wi h eal and complex pa s could be used o cope
wi h he complex- alued model [3]. In [4] we may ind a good desc ip ion o he digi al communica ion model
deeply e ised om he complex- alued poin o iew. Besides, esul s on he ke nel design o he complex case
ha e been epo ed ecen ly [5], [6]. Based on hese wo ks, we p opose a ull desc ip ion o he GPR o p ope
complex- alued sys ems. We de elop he ma ginal likelihood needed in he op imiza ion o he hype pa ame e s,
and i s de i a ion o op imize i . Some expe imen al esul s ha e been included o illus a e he good pe o mance
o his solu ion.
Ap il 12, 2015 DRAFT
2
II. SYSTEM MODEL AND DETECTION
We p opose a gene al model whe e we ansmi Ksymbols e e y sample ime, b = [b (1), b (2), ..., b (K)]>,
and ecei e Nsignals a anged in he column- ec o :
=P· +n ,(1)
whe e = [b>
,b>
−1. . . b>
−Ms+1]>and n is an N-dimensional column- ec o wi h addi i e whi e Gaussian
noise, whe e Ms akes in o accoun he memo y o leng h o he channel esponse. Unless needed, subindex will
be omi ed he ea e .
As an example, in synch onous DS-CDMA he ma ix Pin (1) summa izes he e ec o he channel, he leng h-
Nsp eading codes and he di e en ampli udes o each o he Kuse s, and is a KMs-dimensional ec o
including he ansmi ed bi s (Ms= 2 i 2≤Mc≤K, whe e Mcis he leng h o he channel in chip pe iods)
[7], [1]. Many o he communica ion sys ems can be modeled as (1) aswell.
A he ecei e , we lea n he sys em model using aining da a. In wi eless communica ions sho aining
sequences a e manda o y o inc ease he numbe o in o ma ion bi s. A e he aining s age, o e e y new ecei ed
ec o , deno ed as ∗, a de ec o es ima es he j- h ansmi ed symbol as ˆ
b∗(j) = ( ∗) o some j, whe e his
index will be omi ed he ea e . Linea de ec o s wo k well when he in e -symbolic in e e ence (ISI) and he use s
in e e ence a e negligible. In he gene al case, he solu ion is highly nonlinea and nonlinea de ec o s a e necessa y
o ge a good es ima ion.
In digi al communica ions, linea MMSE es ima o s a e ypically used, due o hei simplici y and nea ly op imal
esul s in many s anda d scena ios. The linea , ( ) = >w, MMSE de ec o can be exp essed as a Wiene il e
[8]:
wmmse = a gmin
w
Ehb− >w2i= (C )−1C b,(2)
whe e C and C ba e, espec i ely, he au oco ela ion o he inpu s and he c oss-co ela ion be ween he inpu s
and ou pu s. This solu ion can be es ima ed om aining da a by eplacing C and C bby hei sampled e sions:
wsmmse = (R·R>)−1R·b,(3)
whe e he columns o Ra e he ecei ed signals a e e y symbol in he aining se . We deno e his solu ion as
linea sampled MMSE (SMMSE).
Howe e , in he gene al case, he MMSE is no linea . I is widely known ha he MMSE solu ion o AWGN
channels is gi en by he condi ional mean o bgi en :
ˆ
b∗= mmse( ∗) = E[b∗| ∗],(4)
which is a nonlinea unc ion o he inpu s, unless he inpu s a e Gaussian dis ibu ed, ha i is no he case.
The compu a ion o he op imal solu ion is ha d o accomplish, gi en he g ea numbe o possible combina ions
o ansmi ed symbols due o he numbe o poin s in he ansmi ed cons ella ions, he numbe o use s and he
channel memo y. In [1], [2] he eal alued GPR is p oposed as a nonlinea ool o es ima e he MMSE esponse.
Ap il 12, 2015 DRAFT
3
Howe e , o ou knowledge, he e is no o mula ion a ailable o complex- alued sys ems. In he ollowing we
de elop a comple e o mula ion o he GPR and hype pa ame e s adap a ion o he complex p ope case, which is
he main con ibu ion o his wo k.
III. COMPLEX GAUSSIAN PROCESSES
A complex Gaussian p ocess (x)is a collec ion o complex andom a iables, any ini e numbe o which ha e
a join complex Gaussian dis ibu ion.
The gene al pd o a complex Gaussian andom ec o z: Ω →CDis [4]
p(z) = 1
πnde 1/2Rzz
exp −1
2(z−µz)HR−1
zz (z−µz),(5)
whe e he augmen ed ec o zis ob ained by s acking zon op o i s complex conjuga e z∗. Simila ly, he augmen ed
mean ec o µzis ob ained by s acking ec o µzon op o i s complex conjuga e µ∗
z.Rzz is he augmen ed
co a iance ma ix o z
Rzz =

Rzz ˜
Rzz
˜
R∗
zz R∗
zz 
,(6)
whe e Rzz =E(z−µz)(z−µz)His he usual co a iance ma ix and ˜
Rzz =E(z−µz)(z−µz)>is he
complemen a y co a iance o pseudo-co a iance ma ix. I is impo an o no e ha bo h Rzz and ˜
Rzz a e equi ed
o a comple e second-o de cha ac e iza ion o he complex Gaussian andom ec o z∼ N µz,Rzz,˜
Rzz.
A complex Gaussian p ocess (x)is comple ely speci ied by i s augmen ed mean unc ion, µ(x), i s co a iance
unc ion, k(xi,xj), and i s complemen a y co a iance unc ion, ˜
k(xi,xj), i.e. (x)∼ GP(µ(x), k(xi,xj),˜
k(xi,xj)).
The e is an impo an special case in which he complemen a y co a iance o a complex Gaussian ec o anishes.
De ini ion 3.1: [4] I he complemen a y co a iance ma ix anishes, ˜
Rzz = 0,zis called p ope , o he wise i
is called imp ope .
The pd (5) o a complex p ope Gaussian andom ec o z: Ω →CDsimpli ies o
p(z) = 1
πnde Rzz
exp −(z−µz)HR−1
zz (z−µz).(7)
We can p opose now he ollowing de ini ion:
De ini ion 3.2: I he complemen a y co a iance unc ion o a complex Gaussian p ocess anishes, he p ocess
is called p ope , o he wise i is called imp ope .
IV. COMPLEX PROPER GAUSSIAN PROCESSES FOR REGRESSION
We conside he model
(x) = φ(x)Hw,(8)
whe e xis he complex inpu ec o o dimension Dand we speci y a p io o e he pa ame e s w: a ze o
mean p ope complex Gaussian wi h co a iance ma ix Σp, i.e., w∼ N (0,Σp,0). This linea eg ession model
gi es an example o a ze o mean complex Gaussian p ocess, µ(x) = φ(x)HE[w] = 0, wi h co a iance unc ion
k(xi,xj) = φ(xi)HEwwHφ(xj) = φ(xi)HΣpφ(xj) = ϕ(xi)Hϕ(xj). The complemen a y co a iance unc ion
Ap il 12, 2015 DRAFT
4
anishes, ˜
k(xi,xj) = φ(xi)HEww>φ∗(xj)=0, i.e., (x)is a complex p ope Gaussian p ocess: (x)∼
GP(0, k(xi,xj),0).
We can speci y a co a iance unc ion, ake ninpu ec o s xi,i= 1, ..., n agg ega ed column-wise in he D×n
ma ix X, and w i e ou he co esponding n×nco a iance ma ix K(X,X)elemen wise om k(xi,xj). Fo
hose inpu s we gene a e a complex p ope Gaussian andom ec o ∼ N (0,K(X,X),0).
In he mo e ealis ic si ua ion
y= (x) + =φ(x)Hw+, (9)
he obse ed alues ydi e om (x)by addi i e complex p ope Gaussian noise wi h ze o mean and a iance
σ2
n, i.e., ∼ N 0, σ2
n,0. Hence, he complex andom ec o y ha esul s om he inpu s in Xis also p ope
Gaussian: y∼ N (0,C,0), wi h C=K(X,X)+σ2
nI. Fu he mo e, yand a e c oss-p ope , as ˜
Ry =Ey >=
E >+E >= 0. Hence, yand a e join ly p ope [4], i.e., he composi e complex andom ec o y>, >>
is Gaussian p ope . I we ake now a numbe o es inpu ec o s agg ega ed column-wise in ma ix X?, he join
dis ibu ion o he aining ou pu s yand he es ou pu s ?is


y
?
∼ N 
0,

C K(X,X?)
K(X?,X)K(X?,X?)
,0
,(10)
and he condi ional dis ibu ion o ?gi en yis
?|x?,X,y∼ N ¯
?,co ( ?),0,(11)
whe e
¯
?=K(X?,X)C−1y,(12)
co ( ?) = K(X?,X?)−K(X?,X)C−1K(X,X?).(13)
In he case ha he e is only one es inpu ec o x?,
¯
?=k>
?C−1y,(14)
co ( ?) = k(x?,x?)−k>
?C−1k?,(15)
whe e k(x?) = k?is he ec o o co a iances be ween he es poin and he n aining poin s.
A. The ma ginal likelihood
The ma ginal likelihood (o e idence) is he in eg al o he likelihood imes he p io
p(y|X) = Zp(y| ,X)p( |X)d ,(16)
He e, he p io is complex p ope Gaussian, |X∼ N (0,K(X,X),0). Since yand a e join ly p ope , he
likelihood is he ac o ized p ope Gaussian, y| ∼ N  , σ2
nI,0. Now, using (7) in (16) he log ma ginal likelihood
yields
L= log p(y|X) = −yHC−1y−log de C−nlog π, (17)
i.e., y|X∼ N (0,C,0).
Ap il 12, 2015 DRAFT

5
B. Model selec ion o complex p ope GP eg ession
In o de o u n complex Gaussian p ocesses in o powe ul p ac ical ools i is essen ial o de elop me hods
ha add ess he model selec ion p oblem. I is common o use a hie a chical speci ica ion o models, wi h he
pa ame e s, wa he i s le el and some hype pa ame e s θia he second le el. In ou model, he co a iance
unc ion k(xi,xj)can be pa ame e ized in e ms o he hype pa ame e s θi.
I is a p ope y o he ma ginal likelihood ha i au oma ically inco po a es a ade-o be ween model i and
model complexi y [9]. This is he eason why he ma ginal likelihood is aluable in sol ing he model selec ion
p oblem.
The log ma ginal likelihood in (17) can be explici ly w i en condi ioned on he hype pa ame e s, L(θ), wi h
C=C(θ) = K(X,X) + σ2
nI, whe e θis he ec o o hype pa ame e s. We can maximize (17) in o de o se
hose hype pa ame e s. No ice ha he a iance o he addi i e noise, σ2
n, is also ea ed as ano he hype pa ame e
and can be es ima ed om his op imiza ion.
L(θ)in (17) is a eal unc ion o a complex He mi ian ma ix C(θ) ha depends on he hype pa ame e s.
The chain ule can be used o seek he de i a i e o he log ma ginal likelihood wi h espec o C(θ)and hen
he de i a i e o C(θ)wi h espec o he hype pa ame e s. Howe e , he Cauchy-Riemann condi ions dic a e ha
noncons an eal alued unc ions ha a e de ined on complex domains a e no holomo phic, so he complex
de i a i e canno be used. Ins ead, we mus seek o mal o Wi inge de i a i es [5], [10].
The chain ule o he Wi inge de i a i e o he unc ion L(θ)wi h espec o he hype pa ame e θiyields
[10]
∂L
∂θi
=
n
X
k=1
n
X
l=1 ∂L
∂Clk
∂Clk
∂θi
+∂L
∂C∗
lk
∂C∗
lk
∂θi,(18)
whe e Clk is he elemen (l, k)o ma ix C. When inding he Wi inge de i a i es ∂L
∂Clk and ∂L
∂C∗
lk
he elemen s
Clk and C∗
lk a e ea ed as independen a iables. The e o e,
∂L
∂θi
=
n
X
k=1
n
X
l=1
∂L
∂Clk
∂Clk
∂θi
= T "∂L
∂C>∂C
∂θi#.(19)
The de i a i e o he i s e m o he unc ion Lin (17) wi h espec o Cyields
∂
∂C−yHC−1y=−∂
∂CT yHC−1y
=C−> yyH>C−>.(20)
The de i a i e o he second e m o he unc ion Lin (17) wi h espec o Cyields
∂
∂C(−log de C) = −C−>.(21)
Subs i u ion o (20) and (21) in (19) yield
∂L
∂θi
= T C−1yyHC−1−C−1∂C
∂θi.(22)
Ap il 12, 2015 DRAFT
6
C. Complex co a iance unc ions
We ecall he e he complex ke nel unc ion used in he Expe imen s sec ion o his s udy, he independen
Gaussian ke nel [6]. This ke nel is de ined as
κ(xi,xj) = κR(αi, αj) + κR(βi, βj))
+j(κR(αi, βj)−κR(βi, αj)) ,(23)
whe e xl=αl+jβland κRis he well-known eal Gaussian ke nel. This ke nel is a gene ic ex ension o κRand
inhe i s he in ui i e physical meaning o a measu e o simila i y o he samples ha κRhas.
V. EXPERIMENTS
By eplacing and bin (1) by xand y, espec i ely, in (9) we yield he complex GPR model o es ima e b.
Tha es ima e can be ound as ¯
?in (14) o each new ecei ed ec o x?= ∗. In hese expe imen s we ocus
on di e en scena ios o show he good beha io o his de ec o . We conside he e QPSK modula ions. No e ha
hey a e p ope complex- alued and so a e he ecei ed signals in (1).
In Fig. 1 we ha e included he BER o he complex GPR wi h he ke nel in (23) o he expe imen in Sec ion V.A
in [1]. In his expe imen we ha e a DS-CDMA sys em wi h K= 2 equal powe use s and signa u es [+1+1−1−1]
and [+1 −1−1 + 1]. The channel is gi en by c(z) = 0.3+0.7z−1+ 0.3z−2. The ou ecei ed chips a e he
wo ma ched il e s a e he inpu s, x, i.e., N= 2. The numbe o aining samples used was n= 64. In Fig. 2 he
decision bounda y is included o he complex GPR wi h 64 aining samples and SNR = 16 dB, along wi h he
poin s co esponding o he bi s o he eal pa o he inpu s o bo h use s. This expe imen illus a es a scena io
whe e he GPR is o majo in e es , since he linea esponse ails o de ec he ansmi ed symbols. The complex
GPR success ully compu es he nonlinea solu ion, e en o he sho numbe o aining samples used.
In Fig. 3 we include a DS-CDMA scena io wi h K= 10 equal powe use s and signa u es wi h N= 31 chips
and a channel o 10 ze o mean complex Gaussian iid aps. We depic he BER along he numbe o aining samples,
n, o a SNR = 12 dB. In his expe imen , he inpu s o he de ec o a e N= 31 ecei ed chips o one symbol
pe iod. Since he la ge dimension, he solu ion is nea ly linea and he linea SMMSE exhibi s a good pe o mance.
The GPR wi h he nonlinea complex ke nel adap s well o his linea solu ion, e en o a sho numbe o samples.
In he sho leng h scena io he egula iza ion e m lea ned is high o compensa e o he lack o in o ma ion, and
he GPR pe o ms be e han he SMMSE. As he numbe o aining samples inc eases, he nonlinea pa o he
ke nel lea ns he linea solu ion and he GPR is able o ollow he SMMSE. The sligh misma ch be ween bo h
cu es is again due o he egula iza ion e m.
VI. CONCLUSION
In his pape we de elop a comple e o mula ion o he GPR and hype pa ame e s adap a ion o he complex
p ope case. Complex p ope GPR can adap ei he o linea o nonlinea solu ions wi h sho aining sequences,
he e o e hey a e o in e es as de ec o s in communica ion applica ions, as shown in he included expe imen s.
Ap il 12, 2015 DRAFT
7
4 6 8 10 12 14 16 18 20
10−5
10−4
10−3
10−2
10−1
SNR(dB)
BER
MF
SMMSE
GPR
Fig. 1. BER along he SNR in a DS-CDMA scena io wi h K= 2 use s and signa u es [+1 + 1 −1−1] and [+1 −1−1 + 1] o SMMSE
(◦), he ma ched il e () and GPR () de ec o s o n= 64 aining samples.
−1 0 1
−1
−0.5
0
0.5
1
<(x(1))
<(x(2))
Fig. 2. Decision bounda y o he GPR in a DS-CDMA scena io wi h K= 2 use s and signa u es [+1 + 1 −1−1] and [+1 −1−1 + 1]
o n= 64 aining samples and SNR = 16 dB.
ACKNOWLEDGMENT
This wo k was pa ially unded by Spanish go e nmen (Minis e io de Educaci´
on y Ciencia, Consolide -Ingenio
2010 CSD2008-00010, TEC2012-38800-C03-{02}) and Eu opean Union (FEDER).
Ap il 12, 2015 DRAFT
8
50 100 150 200 250
10−5
10−4
10−3
10−2
10−1
n
BER
MF
SMMSE
GPR
Fig. 3. BER along n, in a DS-CDMA scena io wi h K= 10 use s and signa u es wi h N= 31 o SMMSE (◦), he ma ched il e () and
GPR () de ec o s o SNR = 12 dB.
REFERENCES
[1] J. Mu illo-Fuen es and F. P´
e ez-C uz, “Gaussian p ocess eg esso s o mul iuse de ec ion in DS-CDMA sys ems,” IEEE T ansac ions on
Communica ions, ol. 57, no. 8, pp. 2339–2347, 2009.
[2] F. P´
e ez-C uz, S. Van Vae enbe gh, J. Mu illo-Fuen es, M. Laza o-G edilla, and I. San ama ia, “Gaussian p ocesses o nonlinea signal
p ocessing: An o e iew o ecen ad ances,” Signal P ocessing Magazine, IEEE, ol. 30, no. 4, pp. 40–50, July 2013.
[3] F. P´
e ez-C uz, J. Mu illo-Fuen es, and S. Ca o, “Nonlinea channel equaliza ion wi h Gaussian P ocesses o Reg ession,” IEEE T ansac ions
on Signal P ocessing, ol. 56, no. 10, pp. 5283–5286, Oc . 2008.
[4] P. J. Sch eie and L. L. Scha , S a is ical Signal P ocessing o Complex-Valued Da a. The Theo y o Imp ope and Nonci cula Signals.
Camb idge, UK: Camb idge Uni e si y P ess, 2010.
[5] P. Bouboulis and S. Theodo idis, “Ex ension o Wi inge ’s calculus o ep oducing ke nel Hilbe spaces and he complex ke nel LMS,”
Signal P ocessing, IEEE T ansac ions on, ol. 59, no. 3, pp. 964–978, 2011.
[6] F. Toba , A. Kuh, and D. Mandic, “A no el augmen ed complex alued ke nel LMS,” in Senso A ay and Mul ichannel Signal P ocessing
Wo kshop (SAM), 2012 IEEE 7 h, 2012, pp. 473–476.
[7] S. Chen, A. K. Samingan, and L. Hanzo, “Suppo ec o machine mul iuse ecei e o DS-CDMA signals in mul ipa h channels,” IEEE
T ansac ions on Neu al Ne wo k, ol. 12, no. 3, pp. 604–611, Decembe 2001.
[8] S. Ve d´
u, Mul iuse De ec ion. Camb idge Uni e si y P ess, 1998.
[9] C. E. Rasmussen and C. K. I. Williams, Gaussian P ocesses o Machine Lea ning. Massachuse s Ins i u e o Technology, Camb idge,
Massachuse s: MIT P ess, 2006.
[10] A. Hjo ungnes and D. Gesbe , “Complex- alued ma ix di e en ia ion: Techniques and key esul s,” Signal P ocessing, IEEE T ansac ions
on, ol. 55, no. 6, pp. 2740–2746, June 2007.
Ap il 12, 2015 DRAFT