OPTIMIZATION OF TWO-STAGE CARRIER PHASE

ESTIMATION FOR PROBABILISTIC SHAPING

Ankita Jain1 * , Ahmed I Abd El-Rahman1 , John C Cartledge1
1 Department of Electrical and Computer Engineering, Queen’s University, Kingston, ON, Canada
*E-mail: ankita.jain@queensu.ca

Keywords: Reduced complexity carrier phase estimation in optical transmission systems

Abstract
The impact of probabilistic shaping (PS) on two-stage carrier phase estimation employing the blind phase search (BPS) and
constellation transformation (CT) algorithms is investigated for PS-64QAM systems. With adequate adaption of BPS to the
shaping, a single implementation of CT achieves shaping gains comparable to two-stage BPS.

1 Introduction is also evaluated using experimental data for uniform 64QAM

Probabilistic shaping (PS) is an effective approach to opti-
mize the power efficiency of signal constellations under a
given signal-to-noise ratio (SNR) constraint [1]. For discrete-
alphabet QAM constellations, the ability to tune the net bit
rate and forward error correction (FEC) coding overhead for
different transmission distances can be significantly improved
by controlling the probabilities of constellation points [2, 3].
In this regard, the Maxwell-Boltzman distribution allows near-
optimal transmission capacity relative to the Shannon limit [4].
To investigate the benefits of PS for coherent systems, data-
aided receiver digital signal processing (DSP) is often con-
sidered [2, 3]. Existing blind algorithms originally intended
for uniform constellations may be suboptimal for shaped con-
stellations. In particular, higher-order QAM formats are more
vulnerable to phase noise due to reduced Euclidean distances.
A crucial stage in the DSP is therefore the carrier phase estima-
tion (CPE). Uniform dual polarization (DP) 64QAM systems
have used CPE based on the decision-directed feedback, QPSK
partitioning, digital phase lock loop, and blind phase search
(BPS) algorithms [5–7]. BPS exhibits relatively better phase
noise tolerance at the expense of high computational complex-
ity [7]. However, it has been shown recently that single stage
BPS is adversely affected due to PS and delivers only a modest
shaping gain even with 60 test phases and block lengths up to
500 symbols [8]. Large block lengths may impede tracking of
the phase noise, particularly for low SNR values. In this case,
two-stage CPE techniques can be more efficient.
To reduce the computational effort, several two-stage feed-
forward CPE schemes have been reported which first use BPS
for coarse phase estimation followed by a second stage for
fine phase estimation [9]. Examples of fine phase estimation
algorithms include BPS, maximum likelihood (ML), and con-
stellation transformation (CT). For uniform 64QAM, CT has
been shown to reduce complexity compared to BPS and out-
perform ML [10, 11]. In this paper, we explore the efficacy of
the BPS+CT algorithm for PS 64QAM using extensive MAT-
LAB simulations with the goal of minimizing the complexity
of the BPS stage while achieving performance equivalent to
two-stage BPS (BPS+BPS). To this end, four different prob-
ability mass functions (PMFs) are employed. The algorithm
and one shaped constellation. The results, reported in terms of
the generalized mutual information (GMI) [12], indicate that
while BPS requires adaptation of the block length for different
shaping rates, a single implementation of CT can be used in
achieving shaping gains comparable to BPS [13], in terms of
both SNR and transmission reach.
2 Two-stage Carrier Phase Estimation
For a simplified linear transmission which includes only laser
phase noise, a received sample value prior to the CPE stage can
be written as,
E(n) = A(n) exp(θ(n) + φ(n)) + N (n) (1)
where A(n) and θ(n) denote the amplitude and phase modu-
lation, φ(n) and N (n) represent the combined transmitter and
receiver laser phase noise and amplified spontaneous emission
(ASE) noise, respectively, and n is the time index. CPE relies
on φ(n) being slowly varying relative to the symbol period and
hence can be assumed constant over a block of symbols with a
precribed length. For a given block of received samples, BPS
mainly comprises the following steps: (i) the sample values are
rotated by B equidistant test phase angles, (ii) ML decisions
are applied to the rotated samples in order to remove the phase
modulation, and (iii) the error in Euclidean distance is calcu-
lated and the test phase angle yielding the minimum sum of
squared errors is selected. In order to reduce the effect of ASE
noise, the distances of N1 consecutive symbols rotated by the
same test phase angle are averaged for each test phase [14].
The optimum value of block length depends on optical signal-
to-noise ratio (OSNR), and shaping rate, in the case of shaped
constellations, which is discussed later in this paper.
After coarse phase estimation, a fine estimate can be
obtained either by using a BPS stage or by transforming the
64QAM constellation into a QPSK constellation and applying
the 4th power algorithm [10]. Denoting I and Q as the real and
imaginary parts of a signal sample value, the transformation
from 64QAM to QPSK is carried out in two steps: (i) In the first
step, the 16 points in each quadrant are mapped into 4 points
using (2), such that 64QAM is mapped to 16QAM
Y1 = [I1 − sign(I1 − 2sign(I1 ))]
(2)
+ j[Q1 − sign(Q1 − 2sign(Q1 ))]

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on November 13,2023 at 16:51:15 UTC from IEEE Xplore. Restrictions apply.
Fig. 1 Histograms of uniform and PS constellations with Fig. 2 Threshold block length of first stage (BPS) vs SNR for
entropies of 6, 5.90, 5.69, 5.57 bits/2D symbol from (a) to (d). uniform and PS 64QAM.
where I1 = Real(I + jQ − p) and Q1 = Imag(I + jQ − p), p
is the median point of each quadrant and sign(.) is the sign
function. (ii) In the second step, the 16QAM constellation is
mapped to a QPSK constellation using (3)
Y2 = [I2 − 2sign(I2 − 4sign(I2 ))]
(3)
+ j[Q2 − 2sign(Q2 − 4sign(Q2 ))]
where I2 = Real(Y1 + p) and Q2 = Imag(Y1 + p). After steps
(i) and (ii), the 4th power algorithm is applied to the transformed
QPSK symbols, thus implementing fine CPE. In the 4th power
method, the symbols in a block of length N2 are averaged to
reduce the effect of ASE noise.

3 Simulation Setup and Results

The simulation setup comprised a constant composition dis-
tribution matcher (CCDM) which generated the four PMFs Fig. 3 GMI vs block length of second stage (CT (left), BPS
shown in Fig. 1 [15]. The transmission symbol rate was 28 (right)) for uniform and PS 64QAM at SNR ∼ 17 dB.
Gbaud and root-raised-cosine pulse shaping with a roll-off fac-
tor of 0.1 was used. The laser phase noise was modelled as a
Wiener process with a variance of 2πΔνT , where Δν is the
laser linewidth and T is the symbol duration. The OSNR of
the system was set by ASE noise loading. The transmitter and
LO lasers had linewidths of 100 kHz. The DSP consisted of
matched filtering, resampling, and CPE using the BPS+CT and
BPS+BPS algorithms, followed by GMI calculations.
The two stages of the BPS+CT algorithm are optimized
in order. To keep the computational effort low, only 8 test
phases were considered for the first-stage BPS for all the cases.
In optimizing the BPS stage, the block length N1 needs to
be increased for lower SNR values and as the shaping gets
stronger [8]. This is required because for shaped constellations,
symbols having lower magnitude are more probable. Conse-
quently, the application of BPS leads to impaired phase noise
estimation owing to cycle slips. Hence, in order to reduce the Fig. 4 GMI vs SNR for uniform and PS 64QAM with BPS+CT
effect of ASE noise, the averaging operation needs to be car- (left) and BPS+BPS (right) employed for CPE.
ried over a large number of consecutive symbols. Fig. 2 shows and P3 requires N1 > 160, 220, 240, respectively, for SNR lev-
the SNR dependence of the minimum block length required els < 14 dB. For high SNR values (> 18 dB), the threshold
for each constellation shaping to avoid cycle slips. This block block length drops to about 25 irrespective of the constellation
length is denoted as the threshold block length. While uniform shaping. The CT algorithm was then optimized for each value
64QAM requires N1 < 100, PS 64QAM with shapings P1, P2

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on November 13,2023 at 16:51:15 UTC from IEEE Xplore. Restrictions apply.
of SNR considering the 8 test phases and the variation of N1
corresponding to different shapings as shown in Fig. 2.
Fig. 3 (left) shows the dependence of GMI on block length
N2 for an SNR value of 17 dB with BPS+CT employed as CPE
for the different constellation shapings. Notably, the optimum
block length N2 is found to be about 40 irrespective of the
constellation shaping. Similar trends are obtained when BPS
is used as the second stage as shown in Fig. 3 (right) at the
expense of 4 extra test phases ∈ [−π/32, π/32) [14]. Hence,
N2 = 40 has been used for subsequent simulations.
Fig. 4 depicts the dependence of GMI on SNR for uni-
form and PS 64QAM with the BPS+CT algorithm (left) and
BPS+BPS algorithm (right). Stronger shaping yields the maxi-
mum GMI value at smaller values of SNR. Assuming a forward
error correction (FEC) overhead of 20%, for both BPS+CT and
BPS+BPS, P1 achieves an SNR shaping gain of ∼ 0.6 dB com- Fig. 5 Threshold block length of first-stage (BPS) vs transmis-
pared to uniform 64QAM at a GMI of 4.9 bits/2D symbol. P2 sion distance for uniform and PS 64QAM (P2).
and P3 achieve gains of ∼ 0.9 dB at GMIs of 4.69 and 4.57
bits/2D symbol respectively. However, the BPS+CT algorithm
has an advantage of reduced computational complexity.

4 Experimental Verification
To verify the simulated results in a more practical environ-
ment, a transmission experiment was conducted for uniform
64QAM and PS 64QAM corresponding to an entropy of 5.69
bits/2D symbol (P2). The experiment used a root-raised-cosine
pulse shape with a roll-off factor of 0.1. The digital waveforms
corresponding to DP uniform and PS 64QAM using CCDM
were generated and uploaded to a four-channel 65 GSa/s arbi-
trary waveform generator operating at a symbol rate of 28
Gbaud. The output RF signals were fed to DP IQ modula-
tors. The transmission link comprised a 4-span recirculating
loop with each span consisting of 75 km fiber followed by an Fig. 6 GMI vs transmission distance for uniform and PS
EDFA with noise figure 5 dB and an optical bandpass filter to 64QAM (P2) with BPS+CT and BPS+BPS.
eliminate the out-of-band noise. The received signal was ampli- BPS (over the reduced angular range). PS 64QAM achieves
fied, filtered and sampled using a 32 GHz, 80 GSa/s coherent better performance compared to uniform 64QAM for distances
receiver. Finally, the transmitted symbols were recovered using exceeding 450 km for both CPE algorithms. Considering an
offline DSP consisting of matched filtering, quadrature imbal- FEC overhead of 20%, PS 64QAM achieves an increase in the
ance compensation, clock recovery, CD compensation, adap- transmission distance of ∼ 310 km for both algorithms with
tive equalization, frequency offset compensation, and CPE. To respect to uniform 64QAM at a GMI of 4.69 bits/2D symbol.
eliminate the residual inter-symbol interference, a decision- Similar to Fig. 4, BPS+CT provides the same shaping gain at
directed least-mean-square equalizer was employed, followed lower complexity as compared to BPS+BPS which utilizes 4
by GMI calculations. [16]. extra test phases.
Fig. 5 presents the dependence of threshold block length for
the first-stage BPS algorithm on transmission distance for 8 5 Conclusion
test phases. While for uniform 64QAM, N1 is optimized at
The impact of PS on the BPS+CT and BPS+BPS algorithms
values less than 60 for distances up to 1800 km, PS 64 QAM
has been studied through numerical simulations and experi-
exhibits an increase in N1 by a factor up to ∼ 2.5 as the dis-
mental verification. It has been shown that the first-stage BPS
tance increases up to 1800 km. Furthermore, considering the
requires adaptation of the block length for different constel-
block lengths depicted in Fig. 5, the CT and second-stage BPS
lation shapings, and that the optimum block length exhibits
algorithms were optimized at N2 = 60 for both the uniform
a larger variation with SNR as the shaping becomes stronger.
and shaped constellations. Finally, in Fig. 6 the dependence of
With this first-stage accommodation, a single implementation
GMI on transmission distance for both the uniform and shaped
of the CT algorithm or second-stage BPS algorithm can be used
constellations is provided for BPS+CT and compared with
to achieve the anticipated shaping gains in terms of both SNR
BPS+BPS using the same values for N1 and N2 , 8 test phases
and transmission reach.
for the first-stage BPS, and 4 test phases for the second-stage

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on November 13,2023 at 16:51:15 UTC from IEEE Xplore. Restrictions apply.
6 References (1), pp.77-80
[14] Pfau, T., Noé, R.: ‘Phase-noise-tolerant two-stage carrier
[1] Yankov, M. P., Zibar, D., Larsen, K. J., et al.: ‘Constella- recovery concept for higher order QAM formats’, Journal
tion shaping for fiber-optic channels with QAM and high of Selected Topics in Quantum Electronics, 2010, 16, (5),
spectral efficiency’, IEEE Photonics Technology Letters, pp. 1210-1216
2014, 26, (23), pp. 2407-2410 [15] Schulte, P., Böcherer, G.: ‘Constant composition dis-
[2] Böcherer G., Schulte, P., Steiner, F.: ‘Probabilistic shap- tribution matching’, IEEE Transactions on Information
ing and forward error correction for fiber-optic communi- Theory, 2016, 62, (1), pp. 430-434
cation systems’, Journal of Lightwave Technology, 2019, [16] El-Rahman, A. I. A., Cartledge, J. C.: ‘Implications of
37, (2), pp. 230-243 distributed link polarization-dependent loss on bit wise
[3] Buchali, F., Steiner, F., Böcherer, G., et al.: ‘Rate adap- achievable information rates for probabilistically shaped
tation and reach increase by probabilistically shaped and uniform DP 64-QAM’, Journal of Lightwave Tech-
64-QAM: An experimental demonstration’, Journal of nology, 2019, 37, (4), pp. 1187-1194
Lightwave Technology, 2016, 34, (7), pp. 1599-1609
[4] Ghazisaeidi, A., de Jauregui Ruiz, I.F., Rios-Müller, R., et
al.: ‘Advanced C+L-band transoceanic transmission sys-
tems based on probabilistically shaped PDM-64QAM’,
Journal of Lightwave Technology, 2017, 35, (7), pp.
1291-1299
[5] Chen, Y., Huang, X. G.: ‘A linewidth-tolerant two-stage
CPE using a new QPSK-partitioning approach and an
enhanced maximum likelihood detection for 64-QAM
coherent optical systems’, Journal of Lightwave Technol-
ogy, 2015, 33, (18), pp. 3883-3889
[6] Gao, Y., Lau, A. P. T., Lu, C.: ‘Modulation-format-
independent carrier phase estimation for square M-QAM
systems’, IEEE Photonics Technology Letters, 2013, 25,
(11), pp. 1073-1076
[7] Zhuge, Q., Chen, C., Plant, D. V.: ‘Low computation
complexity two-stage feedforward carrier recovery algo-
rithm for M-QAM’. Proc. Optical Fiber Communication
Conf., 2011, paper OMJ5
[8] Mello, D. A. A., Barbosa, F. A., Reis, J. D.: ‘Interplay of
Probabilistic Shaping and the Blind Phase Search Algo-
rithm’, Journal of Lightwave Technology, 2018, 36, (22),
pp. 5096-5105
[9] Xiang, M., Fu, S., Deng, L., et al.: ‘Low-complexity feed-
forward carrier phase estimation for M-ary QAM based
on phase search acceleration by quadratic approximation’
Optics Express, 2015, 23, (15), pp. 19142-19153
[10] Bilal, S. M., Fludger, C. R., Curri, V., et al.: ‘Multi-
stage carrier phase estimation algorithms for phase noise
mitigation in 64-quadrature amplitude modulation opti-
cal systems’, Journal of Lightwave Technology, 2014, 32,
(17), pp. 2973-2980
[11] Su, X., Zhang, Z., Zhang, X., et al.: ‘Laser-linewidth
tolerant and low-complexity two-stage carrier phase esti-
mation for M-QAM with coherent detection’. Proc.
Asia Communications and Photonics Conf., 2013, paper
AW3F-7
[12] Alvarado, A., Fehenberger, T., Chen, B., et al.: ‘Achiev-
able information rates for fiber optics: Applications and
computations’, Journal of Lightwave Technology, 2018,
36, (2), pp. 424-439
[13] Su, X., Xi, L., Tang, X., et al.: ‘A multistage CPE scheme
based on crossed constellation transformation for M-
QAM’, IEEE Photonics Technology Letters, 2015, 27,

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on November 13,2023 at 16:51:15 UTC from IEEE Xplore. Restrictions apply.