Blind Equalization in Dynamic PMD Channels Using Variational Autoencoders with LSTM

Blind Equalization in Dynamic PMD Channels Using Variational Autoencoders with LSTM Jos´ e N´ u˜ nez-Kasaneva, Boris Karanov, Alex Alvarado and Gabriele Liga 1Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands *[email protected] Abstract: We propose a new variational autoencoder-based blind equalizer for polarization demultiplexing, and assess it in a dynamic polarization channel. We demonstrate a 0.4 dB SNR gain and doubled tolerance to state-of-polarization drift compared to CMA-RDE. © 2024 The Author(s) 1. Introduction Fiber birefringence leads to simultaneous coupling and filtering effects on the transmitted polarization channels, a channel impairment referred to as polarization mode dispersion (PMD). One of the main functions of a coherent optical receiver is the polarization channel demultiplexing in the presence of PMD. PMD can be described as the combination of two phenomena: a) a local rotation of the state of polarization (SOP) of the optical field; b) a differential group delay (DGD) between the local principal states of polarization. Both these phenomena are in general time-varying, with the SOP rotations representing the main component of a dynamic PMD channel. SOP fluctuations over different timescales have been reported in the literature, spanning from days [1] to µs [2]. Coherent receivers require channel equalization to track the SOP changes and compensate for the link DGD fluctuations. Blind equalization schemes are widely used for this purpose, with the constant modulus algorithm (CMA) being the most popular equalization method [3]. For multi-modulus constellations, the radius-directed equalizer (RDE) [4] combined with CMA (CMA-RDE) achieves better convergence. Recently, new machine learning techniques were introduced for polarization demultiplexing, such as the variational autoencoder-based linear equalizer (VAE-LE) [5]. The VAE-LE uses the variational inference principle [6] to train the 2 ×2 finite impulse response (FIR) multi-input multi-output filter. The VAE-LE has been shown to outperform CMA for high-order modulation formats [5], but its performance has not been characterized in time-varying polarization channels [1,2,7]. In this paper, we propose an extension of the VAE-LE blind equalizer in [5], incorporating a long-short-term memory (LSTM) network. The goal of this LSTM network is to mitigate rapid polarization changes in dynamic optical fiber channels. Our results demonstrate that the proposed architecture significantly improves the SOP tracking, leading to an improved outage probability and increased tolerance to time-varying polarization channels compared to CMA-RDE. 2. Channel model and proposed equalizer 2.1. Channel model In this work, we utilize the dynamic PMD channel model proposed in [7], which is mathematically described by S(f,k) = N ∏ n=1 Hn(k)Bn(f),(1) Bn(f) = M ∏ m=1 Rm,nDm,n(f),(2) S(f,k)is a 2 ×2 frequency ( f) and discrete-time (k)-dependent matrix, representing the block-wise varying frequency response of the polarization channel. S(f,k)consists of Nsections, each comprising a so-called hinge, modeled by a time-dependent 2 ×2 complex unitary matrix Hn(k), and a static random birefringence section described by the frequency-dependent unitary 2 ×2 complex matrix Bn(f). The hinges model localized timevarying perturbations of the SOP along the optical link, whereas the birefringence sections describe a static frequency-selective polarization rotation. Each birefringence section in (2) consists of a concatenation of Msections each comprising a complex 2 ×2 rotation matrix Rm,nthat scatters the SOP isotropically on the Poincar´ e sphere, and a delay element Dm,n(f) = diag(exp(jπfτm,n),exp(−jπfτm,n)), where τm,nare drawn independently from a Gaussian distribution according to [7, Sec. II]. The temporal drift caused by each hinge is described by Hn(k) = J(˙ αn(k))·Hn((k−1)), where J(˙ αn(k)) = exp(−j˙ αn(k)·−→ σ), is a random Jones matrix with −→ σbeing the Pauli tensor [8, eq. (3)]. The parameter ˙ αnis a vector of Stokes innovation angles assumed to be an i.i.d. zero-mean Gaussian distribution with variance σ2 v=2π∆PT/N, where ∆Pis the polarization linewidth and Tis the selected SOP innovation period [8]. The channel model used in this work only considers unitary polarization effects, neglecting other impairments such as chromatic dispersion and fiber nonlinearity. Finally, white Gaussian noise is added at the channel output. 2.2. VLSTM Equalizer The structure of VAE-LE was recently proposed in [5] and is illustrated in Fig. 1(a). The VAE-LE utilizes the evidence lower bound (ELBO) to approximate the maximum likelihood (ML) channel estimation and equalize the received symbols Rx,y i, where xand yrepresent the dual polarization, where each polarization has the respective in-phase (I) and quadrature (Q) components within frame i(number of symbols/symbol rate). The equalizer incorporates a complex-valued 2 ×2 butterfly structure with FIR filters, whose output are the demultiplexed symbols ˆ Rx,y i. The VAE-LE uses one filter system for equalization where the LE filter weights are represented by the tensor hi est, which are updated by the ELBO function and also used for the channel estimation as presented in [5, eq. (6)]. A maximum a posteriori (MAP) soft demapper is used to translate ˆ Rx,y iinto the probabilities of the corresponding symbols ˆqx,y i. The soft demapper is followed by a MAP symbol detector producing the estimated symbol ˆ Tx,y ithat goes to the SER the complex vector Tx,y iof transmitted symbols. SER Estimation VAE-LE Linear Equalizer Rx i Ry i Soft Demapper ˆ Rx i ˆ Ry i ELBO hi est ˆ hi est To LSTM From LSTM hi est MAP Symbol Detector SER ˆ Tx i ˆ Ty i ˆqx i ˆqy i Tx iTy i (a) VAE-LE SERi−1 Estimation LSTM VAE-LE SERi Estimation Proposed block Rx i−1 Ry i−1 ˆqx i−1 ˆqy i−1 hi−1 est ˆ hi−1 est Rx i Ry i ˆqx i ˆqy i Rx i Ry i (b) Fig. 1: Structure of the VAE-LE (a), and VLSTM (b). The LSTM is a type of recurrent neural network designed for time-dependent tasks, capable of learning longterm dependencies [9], we used this property to estimate the incoming filter tap weight ˆ hi est for different frames. We assumed that within a frame, Rx,y iis correlated with Tx,y iby (1). Based on this assumption, there is no correlation between different frames. So, we use the VAE-LE structure previously described with the addition of an LSTM block (yellow block), after the time frame i−1 of the VAE-LE, as illustrated in Fig 1(b). The resulting VAE-LE+LSTM (VLSTM) scheme, addresses the PMD channels impairments at frame iby using the previously tap weight filter hi−1 est as the hidden state of the LSTM block, and also using the received symbols of the next time frame Rx,y ias the LSTM input. This generated a new ˆ hi−1 est , which serves as the new starting tap weight filter for the new frame i represented as the dashed arrow in Fig.1(a). 3. Results In this section, we compare the performance of 3 blind equalization schemes: CMA-RDE, VAE-LE, and VLSTM. The numerical simulation setup is based on (1) and (2). The model’s input is a single-channel polarizationmultiplexed signal employing a uniform 64-QAM modulation format pulse shaped using a root-raised cosine with 0.1 roll-off and an oversampling factor of 2 samples per symbol. The DGDs in the Dm,n(f)matrices were computed assuming a symbol rate of 40 GBd, and a PMD coefficient of 0.1 ps/√km. The optical fiber link consists of 10 spans of 100 km each, with hinge sections at the end of each span. Each span is then split into 1,000 birefringence segments. All equalizers were trained using frames of 15,000 symbols. The learning rates for the VAE-LE and VLSTM models were initialized at lr=4×10−3, and lr=10−3for the CMA-RDE. All equalizers learning rates were reduced by 50% every 20 frames, while the LSTM learning rate was reduced by a factor of 1,000. In this work, we focus on the polarization tracking performance of equalizers, omitting their singularity behavior, which has been covered for CMA-RDE and VAE-LE in previous studies [10]. Fig. 2(a) shows the convergence behavior of the 3 blind equalization schemes analyzed for a near-static and a dynamic scenario with ∆P1 rad/s and 103rad/s, respectively. For the CMA-RDE scheme, after 100 frames, the loss function of the scheme switches from CMA to RDE, reaching a steady state within 106 frames. We define steady state the point at which the windowed SER over the last 5 frames does not decrease by more than 20%. Moreover, the VAE-LE equalizer exhibits a faster convergence (43 frames), but also shows higher steady-state SER values compared to the CMA-RDE scheme, where the impact of the SOP drift is noticeable after reaching the steady state with a higher SER oscillation in the 050 100 150 200 10−3 10−2 10−1 100 (a) Frame index i SER 200 220 240 10−3 CMA-RDE VAE-LE [5] VLSTM ∆P=1 rad/s ∆P=103rad/s 100101102103104 10−3 10−2 (b) ∆P[rad/s] SER 104 10−2 ×2×1.3 CMA-RDE VAE-LE [5] VLSTM Fig. 2: Results for the blind equalization schemes analyzed in this work at SNR=25 dB: a) SER vs frame index over 200 frames with a windowed SER of 5 frames, and b) SER vs SOP drift speed (∆P). 21 21.522 22.523 23.5 10−3 10−2 10−1 100 (a) 0.4 dB 0.5 dB Pout =2×10−3 SNR [dB] Pout[%] CMA-RDE VAE-LE [5] VLSTM 100101102103104 10−3 10−2 10−1 100 (b) ∆P[rad/s] Pout[%] 103 10−2 ×2.4 ×2 CMA-RDE VAE-LE [5] VLSTM Fig. 3: Outage probability for the 3 different equalizers analyzed in this work, an SER threshold of 10−2and, for: a) different SNRs and a fixed ∆P=103rad/s; b) different ∆Pvalues for SNRs that gives Pout =10−3with ∆P=0 rad/s. dynamic scenario compared with the near-static one. The VLSTM scheme achieves the best performance among the 3 analyzed equalizers in both channel conditions, showing both faster convergence (46 frames), and lower steady-state SER. Fig. 2(b), shows the average SER and standard deviation for different blind equalization schemes as a function of ∆Pat a fixed SNR of 25 dB, where the average SER is obtained with the last 50 frames, and the characterization of the channel spanning from a near-static (∆P1 rad/s) to an extremely dynamic SOP drift scenario [2]∆P10,000 rad/s. For the case ∆P1 rad/s, VAE-LE exhibited the worst performance with a higher average SER value of 1.07× 10−3, while, RDE and VLSTM show comparable average SER values of 8.2×10−4and 6.8×10−4, respectively. This SER behavior changes for ∆P>103rad/s, where all the equalizer schemes reach to a breaking point. In this highly dynamic scenario, VLSTM shows a tracking performance improvement of a factor of 2 and 1.3 in ∆Pcompared to the RDE and VAE-LE, respectively. These results show that the LSTM block in Fig. 1(b) can effectively reduce the impact of rapid polarization fluctuations. In Figs. 3(a)-(b), we characterize the outage probability Pout of the 3 equalizers, which is estimated as the fraction of frames above a given SER threshold over the total number of transmitted frames. Only the frames after convergence was reached were taken into account with a total of 1,000 frames. For both scenarios, we chose an SER threshold of 10−2. Fig. 3(a) shows Pout as a function of the SNR for ∆P=103rad/s. We observe that VLSTM decreases significantly Pout for SNR≥22 dB. At Pout =2×10−3(purple line), VLSTM achieves gains of 0.4 dB and 0.5 dB SNR over RDE and VAE-LE, respectively. Fig. 3(b) illustrates Pout for different values of ∆P. To isolate the effect of ∆Pon Pout , we operated the different equalizers at different SNR values such that they exhibit the same SER=10−3in the static channel case (∆P=0). These SNR values are 22.2 dB for the VLSTM, and 22.6 dB for both VAE-LE and CMA-RDE. Fig. 3(b) shows that VLSTM can tolerate ×2 to ×2.4 higher ∆P than CMA-RDE and VAE-LE, respectively, at a Pout =2 ×10−2. 4. Conclusions We proposed a new blind equalization scheme for polarization demultiplexing and PMD compensation which combines a VAE-LE with an LSTM block. This scheme shows improved tracking performance in fast SOP drift environments with PMD compared to conventional blind equalization schemes such as CMA-RDE, and a plain VAE-based equalizer. Minor SER improvements were observed in slowly varying SOP scenarios. However, VLSTM can significantly reduce the system outage probability in fast-changing PMD channels. In such scenarios, VLSTM can tolerate up to 2.4 times higher SOP drift speeds compared to VAE-LE, and twice as fast SOP rotations compared to the CMA-RDE scheme. Furthermore, the VLSTM provides a 0.4 dB and 0.5 dB gain at fixed outage probability over CMA-RDE and VAE-LE schemes for the same ∆P, respectively. Acknowledgements: This project has received funding from the European Union’s Horizon 2024 research and innovation program under the Marie Skłodowska-Curie grant agreement No 101119983. References 1. M. Karlsson et al., “Long-term measurement of PMD and polarization drift in installed fibers,” JLT, July 2000. 2. P. Krummirich et al., “Extremely fast (microsecond timescale) polarization changes ...” in OFC, 2004, p. 3 pp. vol.2. 3. D. Godard, “Self-Recovering Equalization and Carrier Tracking ...” IEEE Trans. Comm., vol. 28, no. 11, pp. 1867–1875, 1980. 4. M. Ready et al., “Blind equalization based on radius directed adaptation,” in ICASSP, 1990, pp. 1699–1702 vol.3. 5. V. Lauinger et al., “Blind Equalization and Channel Estimation in Coherent Optical ...” JSAC, vol. 40, no. 9, pp. 2529–2539, 2022. 6. D. M. Blei et al., “Variational Inference: A Review for Statisticians,” JASA., vol. 112, no. 518, p. 859–877, Apr. 2017. 7. C. Czegledi et al., “Temporal Stochastic Channel Model for Absolute Polarization State ...” in OFC, 2017. 8. C. B. Czegledi et al., “Polarization Drift Channel Model for Coherent Fibre-Optic Systems,” Scientific Reports, vol. 6, 2 2016. 9. S. Hochreiter et al., “Long Short-Term Memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997. 10. V. Lauinger et al., “Improving the bootstrap of blind equalizers with variational autoencoders,” in OFC, 2023.