3688 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 40, NO.

12, JUNE 15, 2022

Design of Probabilistic Shaping 4D Ultra High Order

Modulation Format With 8APSK Pilot Aided Carrier
Phase Recovery
Feng Tian , Tianze Wu , Yiqing Ji, Chuxuan Wang , Qi Zhang , Member, IEEE, Ran Gao , Member, IEEE,
Zhipei Li, Qinghua Tian , and Xiangjun Xin

Abstract—A new four-dimensional (4D) probabilistically shaped communication services, the internet traffic is growing rapidly
(PS) ultra-high order modulation format is proposed with pilot for center interconnection. Consequently, the development of
aided carrier phase recovery (PA-CPR) algorithm to improve the high-capacity optical communication is necessary. The tech-
general mutual information (GMI) performance. While the struc-
ture of the 4D-PS ultra-high order modulation format uses constel- nology of ultra-high order modulation (UHM) is used to in-
lation set partitioning (SP) based on amplitude translation, and the crease the spectral efficiency (SE) toward the Shannon limit
PA-CPR applies 8-level amplitude phase shift keying (8APSK) pi- and meet the growing demand for high transmission capacity.
lot. To demonstrate the feasibility of the scheme, the PS-1024QAM Many experiments for ultra-high order signals on single-carrier
system is designed on the simulation platform. The results show that have been demonstrated [1]–[10]. The first 1024 QAM single-
the proposed 8APSK-PA-CPR algorithm outperforms the QPSK-
PA-CPR algorithm. The gain of the optimal 8APSK-PA-CPR is carrier coherent optical transmission is demonstrated in [10],
∼0.4 dB compared with the quadrature phase-shift keying (QPSK) in which a 60 Gbit/s polarization-multiplexed signal at a SE
PA-CPR algorithm. The steep drop of blind phase search (BPS) of 13.8 bit/s/Hz was transmitted over 150 km [1]. Similarly,
with different PS factors is overcome by 8APSK-PA-CPR when probabilistically shaped 4096-QAM has been transmitted over
the signal-to-noise (SNR) ratio is lower than 24 dB. The maximum 50.9 km at a potential SE of 15.9 bits/s/Hz and a net bit rate of
improvement of GMI is 1.3 bit/symbol when the SNR is higher
than 26 dB and the complexity of the 8APSK-PA-CPR algorithm 484.4 Gb/s [9]. So far, the highest-order QAM constellations
is reduced by 99.9% compared with the BPS algorithm. The gain that have been demonstrated in optical systems are 10-GBd
obtained from the proposed 4D modulation scheme is ∼2.8 dB probabilistically shaped square 16384-QAM [10] at a net bit
under the condition of GMI = 17.7971 bit/symbol. It demonstrated rate of 223.8 Gb/s. However, as the modulation format order
that this scheme has a strong tolerance to noise, low computational increases, the minimum Euclidean distance decreases, and thus
complexity, and high coding gain. It is a feasible and flexible scheme
for PS high-order modulation format transmission systems. the ability to resist noise declines. Therefore, it is critical to
propose a novel modulation format to expand the Euclidean
Index Terms—Carrier phase recovery, coherent optical distance of constellation points and optimize the DSP algorithm
communications, four-dimensional, pilot aided, probabilistic
shaping, set partitioning.
to improve the system performance.
Therefore, the four-dimensional (4D) modulation format is
I. INTRODUCTION an advanced optical modulation format that is generated by
drawing into the check bit through modulation coding in the two
ITH the rapid development of mobile communication
W networks, live video broadcast, telemedicine, and other
quadrature-phase components and two polarizations of the opti-
cal field. It has attracted a lot of attention in recent years because
Manuscript received November 8, 2021; revised January 12, 2022 and Febru- it can increase the minimum Euclidean distance between sym-
ary 10, 2022; accepted February 13, 2022. Date of publication February 23, bols [11]–[14]. The 4D modulation format uses Ungerboeck’s
2022; date of current version June 16, 2022. This work was supported in part by
the National Key R&D program of China under Grant 2020YFB1805805 and in set partitioning strategy to generate the parity by XOR operation
part by the National Natural Science Foundation of China (NSFC) under Grants of information bits [11], such as 128-level SP quadrature ampli-
61875248, 62021005, and 62027819. (Corresponding author: Feng Tian.) tude modulation (128-SP-QAM) and 2048-SP-QAM. Moreover,
Feng Tian, Tianze Wu, Yiqing Ji, Chuxuan Wang, Qi Zhang, and Qinghua
Tian are with the School of Electronic Engineering, Beijing University probabilistic shaping (PS) is a technology that can increase the
of Posts and Telecommunications, Beijing 100876, China (e-mail: tian- mutual information of signals toward the Shannon limit and
feng@[Link]; wutianze@[Link]; jyiqing@[Link]; wangchux- provide a flexible platform for rate adaptation. PS-based multi-
uan@[Link]; zhangqi@[Link]; tianqh@[Link]).
Ran Gao and Zhipei Li are with the School of Information and Elec- dimensional signaling schemes have recently been reported [15],
tronics, Beijing Institute of Technology, Beijing 100081, China (e-mail: gao- [16] to provide higher fiber nonlinear tolerance. The PS scheme,
ran198412@[Link]; lizhipei@[Link]). however, is incompatible with the SP-QAM modulation format
Xiangjun Xin is with the School of Electronic Engineering, Beijing University
of Posts and Telecommunications, Beijing 100876, China, and also with the due to the presence of parity bits.
School of Information and Electronics, Beijing Institute of Technology, Beijing Besides, the scheme of UHM is more sensitive for channel
100081, China (e-mail: xjxin@[Link]). noise, especially for phase noise. Moreover, the reported phase
Color versions of one or more figures in this article are available at
[Link] recovery algorithms are not suitable for the 4D-PS-MQAM
Digital Object Identifier 10.1109/JLT.2022.3153321 signal. It’s necessary to develop the advanced carrier phase
0733-8724 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See [Link] for more information.

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
TIAN et al.: DESIGN OF PROBABILISTIC SHAPING 4D ULTRA HIGH ORDER MODULATION 3689

Fig. 1. The diagram of PS M-PAM signal generation based on PAS architec-

ture. Fig. 2. The diagram of PS-4D-SP-1024QAM overall mapping process.

√
recovery (CPR) algorithm to improve transmission performance. M-QAM√ into two independent M -level pulse amplitude mod-
Many CPR algorithms have been reported, such as blind search ulation ( M -PAM) signals. The PAS structure of each com-
algorithm (BPS), quadrature phase-shift keying (QPSK) parti- ponent includes a constant component distribution matcher
tioning, maximum likelihood estimation (BPS/ML), quadratic (CCDM) and a forward error correction (FEC) encoder [24].
approximation (BPS/QA) algorithm, and Viterbi-Viterbi (V-V) The n binary bits are divided into n1 and n2 parts. Firstly,
algorithm [17]–[21]. The BPS algorithm have the mutual in- n1 bits are entered√into CCDM to generate m non-uniform
formation punishment with the PS signal at low signal-to-noise positive amplitude M /2-PAM signals,√ and the information
ratio (SNR) [22]. The QPSK partitioning algorithm based on rate of the obtained positive amplitude M /2-PAM signal is
constellation-dependent is inappropriate to phase recovery for β=n1 /m bit/symbol. Then, the obtained symbols from CCDM
high-order modulation format signals. The two-stage BPS al- are reflected into binary labels, which are combined with n2
gorithm has large computational complexity after introducing bits together to enter the FEC encoder for generating parity
a two-stage configuration. The pilot-aided carrier phase re- bits. These
√ parity bits and n2 are used as the sign bits to
covery (PA-CPR) algorithm has high noise tolerance and low form a M -PAM signal following Maxwell-Boltzmann (MB)
computational complexity. However, the main drawback is the distribution [25]. The MB distribution can be expressed as:
inherent overhead (OH) which will reduce the SE of the system. 2
 2
Therefore, it is necessary to optimize the PA-CPR scheme for PX (x) = e−λx /Z(x ), Z(x ) = e−λx (1)
x ∈X
PS-4D-SP-1024QAM. √
In this paper, the scheme of PS-4D-SP-1024QAM combined where x represents the M -PAM constellation points and λ
with the 8-level amplitude phase shift keying aided pilot carrier denotes the shaping factor.
phase algorithm (8APSK-PA-CPR) is proposed. The amplitude
B. Generation of PS-4D-SP-1024QAM
translation method is used to generate the parity for the SP
modulation format. The redundancy generated from the parity The structure of PS-4D-SP-1024QAM is shown in Fig. 2,
of PS-4D-SP-1024QAM will be carried by the pilot to improve which can maintain probability distribution characteristics after
the SE. At the receiver, the information of the pilot is taken out SP encoding. The n bits enter the processing unit of DP-PAS
for signal demapping after preliminary recovery and decision and pass through the SP to perform the amplitude translation in
of pilot symbols. Different laser linewidths are used to simulate the first stage which generate m symbols and 2m binary label
the 20-GBaud PDM-PS-1024QAM and PS-4D-SP-1024QAM bits respectively. In the second stage, the mentioned 2m binary
transmission systems. The results demonstrate that the proposed label bits are used to generate 2k symbols and 4k label bits in
8APSK-PA-CPR algorithm outperforms the QPSK-PA-CPR al- the same process. Finally, the bits coming from the second stage
gorithm at average power. The gain improved by the optimal are mapped into 2k/3 PDM-8APSK symbols, which is used as
8APSK-PA-CPR is ∼0.4 dB compared with the quadrature the pilots for CPR in optical fiber communication systems.
phase-shift keying (QPSK) PA-CPR algorithm. 8APSK-PA- Fig. 3 shows the generation of SP-QAM modulation which is
CPR overcomes the steep drop in BPS with different PS factors performed by XOR coding on the information bits, cf. Fig. 3(c).
when the SNR is less than 24 dB. The maximum improvement In PS systems, the coding after shaping at the transmitter
of GMI is 1.3 bit/symbol when the SNR is higher than 26 dB causes the shaped symbol distribution to be distorted, shown in
and the complexity of the 8APSK-PA-CPR algorithm is reduced Fig. 3(a). On the other hand, XOR coding before shaping at the
by 99.9% compared with the BPS algorithm. The SNR gain transmitter, has no practical effect on decoding at the receiver,
obtained from PS-4D-SP-1024QAM with 8APSK-PA-CPR is Fig. 3(b). In this section, the amplitude translation method is
∼2.8 dB. proposed to overcome the incompatibility between SP and PS
for ultra-high order constellations.
II. PRINCIPLE OF PS-4D-SP-1024QAM Fig. 4 shows that the 1024QAM constellation is divided into
four subsets (A, B, C, and D) named subset constraint (SSC). The
A. Probabilistic Amplitude Shaping corresponding 4D-1024QAM signals are generated according to
Probabilistic amplitude shaping (PAS) is a mainstream stan- the subfamily constraints (SFC) in Table I.
dard probabilistic shaping scheme [23]. As shown in Fig. 1, it The minimum Euclidean distance of the 4D-1024QAM can be
divides the in-phase and quadrature components of the square expanded by twice. Firstly, the non-uniform amplitude set of one

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
3690 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 40, NO. 12, JUNE 15, 2022

Fig. 3. The structure of 4D-SP-1024QAM.

Fig. 5. Schematic diagram of non-uniform constellation set partitioning.

Fig. 6. Schematic of amplitude translation.

Fig. 4. The diagram of 1024QAM constellation Set-partitioning.

Amplitude translation is used to make the PS signal satisfy

TABLE I the above SFC. After the shaping, the 2D signals on two polar-
SUBFAMILY CONSTRAINTS izations corresponding to a time interval are:
Spol = Spol,I + iSpol,Q (2)
where Spol,l and Spol,Q represents the I-component and Q-
component signal on pol polarization, respectively, and pol is
the X or Y polarization.
The 2D signals on X polarization is firstly selected and the
subset of corresponding Y-polarization 2D signals are judged
according to Table I. Then 2D signals are decomposed into 1D
signals for analysis. There are two cases:
dimensional (1D) signal generated by PS is divided into a subset 1) If the 1D signal on the Y polarization mismatches the
1 2
C1D and C1D shown in Fig. 5. Then, the two 1D subsets are current SFC, the 1D signal is shifted to the adjacent signal
pairwise combined into two-dimensional (2D) subset A, B, C, points (forward translation is adopted in this paper) as the
and D. Finally, 2D subsets are combined into four 4D subfamilies red arrow shows in Fig. 6, i.e., the complementary 1D
(AD, BC, CB, DA) as shown in Fig. 5 corresponding to subset, and then the binary label bit is set to 1. In the
Table I. shifting process, there are two special points at both ends

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
TIAN et al.: DESIGN OF PROBABILISTIC SHAPING 4D ULTRA HIGH ORDER MODULATION 3691

of the constellation during the translation. If the signal is

beyond the limit of the constellation on translation, the
points at the end are translated to the head to form a ring
structure, as the blue arrow in Fig. 6. Take the in-phase
component as an example:
− 
s−→
y,I = sy,I + d, sy,I < max (C1D )
− : label → 1
s−→
y,I = min (C1D ) , sy,I = max (C1D )
(3)
−−→
where Sy,I is the translated signal for Y polarization, d rep-
resents the Euclidean distance between adjacent constellation
points, and min (·) and max (·) represent the minimum and
maximum values of the set respectively.
2) If the 1D signal matches the SFC, the translation is unnec-
essary and the label bit is 0:
−−→
Sy,I = Sy,I : label → 0 (4)
In the process of signal generation, a 4D symbol will generate
Fig. 7. NGMI comparison of signals following MB distribution and quasi-MB
two binary label bits. distribution with different λ.

C. Redundancy Processing and Decoding

In this paper, the proposed scheme can reduce the loss of L = {0, 1} generated by 4D signal follows the (0-1) distribution
transmission efficiency by saving overhead. It is assumed that with parameter 1/2. Y and L are independent of each other. When
the overhead of 4D is OH4D and the overhead generated by the label is 0, the distribution of constellation set Y  remains
pilot is OHP , so the total overhead is OHT otal =OHP + OH4D unchanged, that is:
in general. When using pilot and multi-dimensional signals,
2
the generated overhead decreases compared to using pilot and PY  (yi ) = P {Y = yi } = e−λyi /Z(y ) (5)
multi-dimensional signals separately. The saved overhead can be
∗
expressed as OHC =OHP ∩ OH4D . The total cost is OHTotal = The signal point is amplitude translated when the label is 1,
OHP + OH4D − OHC . Fig. 2 shows that 4n bits obtain 4m and the probability is:
+ 8k 4D symbols and 16k bits after two stages of PAS and 
amplitude translation. In this paper, 16k bits continue to be PY (y√M ) i = 1√
PY  (yi ) = (6)
mapped into 8k/3 symbols as a part of pilot symbols for phase PY (yi−1 ) 2 < i ≤ M
recovery, which improves the utilization of overhead. These
pilots also can be used for demodulation at the receiver The probability mass function (PMF) of Y  is:
The 4D signal demodulation is divided into two parts: one is
PY  (yi ) = P {Y  = yi }
SP demodulation and the other is PS/FEC decoding. Firstly, the
=P{Y  = yi , L = 0} + P {Y  = yi , L = 1}
pilot signal is demodulated into label bits, a hard decision (HD) 2 (7)
e−λyi /Z(y ) i = 1√
for a signal is operated. Then, the signal on the Y polarization is = 2 2
translated into the backward direction according to the label bit. e−λyi + e−λyi−1 /2Z(y ) 1 < i ≤ M
If the label bit is 1, the amplitude of the corresponding signal
Then the normalized generalized mutual information (NGMI)
component is translated in the backward direction. If the binary
performance of PS-1024QAM signals following MB distribu-
label bit is 0, there is no need for translation. When the translation
tion and quasi MB distribution with different is simulated and
of all symbols is completed, the bit metric decoding (BMD) [26]
compared.
or HD [5] of PS is performed to obtain the label bits in the first
In Fig. 7, For λ = 0.2 the 1024QAM constellation is truncated
stage, and the signal is demodulated.
obviously, and the information entropy is greatly affected by
the change of distribution. There is a about 5 dB gap between
D. Influence on Constellation Probability Distribution quasi-MB distribution and MB distribution; For λ = 0.02 the in-
The probability distribution of the signal on the Y polarization fluence of distribution change on information entropy weakens,
will inevitably produce a certain offset when the amplitude of the and the gap decreases to ∼2 dB; For λ = 0.002, the degree
signal point is translated, resulting in a constellation following a of shaping is very small, and the change of NGMI can be
quasi-MB distribution. The derivation of probability distribution negligible. Therefore, the constellation scale is large when the
of the Y polarization signal is as follows. shaping degree is small. The NGMI degradation caused by quasi
It is assumed that √the one-dimensional constellation set Y = MB distribution can be ignored, and the application in higher
{±1, ±3, ±5, . . . , ± M } of Y polarization signal after PAS shaping degree or lower order modulation format needs to be
follows the MB distribution with a parameter of λ. Label bit set reconsidered.

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
3692 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 40, NO. 12, JUNE 15, 2022

Fig. 8. The frame structure of implementing the pilot-aided CPR.

III. PS-4D-SP-1024QAM PHASE RECOVERY SCHEME Fig. 9. The implementation of pilot aided CPR scheme.
A. Phase Noise
The DSP algorithm in coherent optical communication in-
cludes dispersion compensation, clock recovery, polarization
demultiplexing, frequency offset estimation, and phase recov-
ery. Except for phase noise, all the channel impairments are
compensated in this paper. The received signal at discrete time
k can be written as:
yk = xk ejθk + nk (8)
where xk is the transmitted symbol, θk is laser phase noise, nk
is AWGN additive white Gaussian noise. The phase noise can
be modeled as a Wiener process, and is given by (9):
t

θt = wi (9)
i=−∞

where wi are independent identically distributed Gaussian ran-

dom variables with zero means and the variance [27]:
2
σw = 2π (Δf · T s) (10) Fig. 10. 8APSK-pilot constellation point.

Ts is the symbol period and Δf is the sum of the linewidths

of the Tx laser and the local oscillator.
obtain the original signal Pn ’. The phase noise of the pilot signal
B. The Principle of Pilot Aided Algorithm θest is estimated by multiplying Pn with the conjugate of the
original 8APSK pilot Pn ’. The payload signal phase noise θfinal
The digital signal processing of the pilot-aided algorithm is obtained by linear interpolation of θest in adjacent CPR blocks,
mainly depends on the known symbols. The frame structure as shown in (11):
of the signal is shown in Fig. 8. It consists of an initial pilot
sequence and a periodic payload. The inserted pilot symbols are ϕa,b = ϕa + b · (ϕa+1 − ϕa )/N (11)
used for continuous phase tracking and phase estimate. There are
where ϕa is the pilot phase noise of the ath CPR block, ϕa,b
p pilot symbols with 8APSK inserted into the N symbols payload
is the bth payload signal phase noise in the ath CPR block, N
sequence at the receiver, and the insertion ratio is p/N. The length
is the length of the PS-4D-SP-1024QAM payload in each CPR
of the CPR block can be dynamically adjusted according to the
block [28]. Finally, the original payload signal is obtained by
shaping parameters in the PS systems to estimate the optimal
subtracting the estimated phase θfinal from the received payload
phase.
signal Xn . The algorithm shows that the PA algorithm will not
Fig. 9 shows the block diagram of the PA-CPR scheme.
be affected by the modulation format type.
Generally, the pilot will not carry any meaningful information.
However, the second stage amplitude translation generates some
labels described in Ⅱ B. The labels are carried by 8APSK C. Pilot Modulation Format
pilots, which is a kind of meaningful information, because we The amplitude of the pilot signal has different effects on the
need to extract these labels at the receiver to demodulate the PA-CPR performance. The pilot signal with a small amplitude
signals. Therefore, the pilot should be preliminary recover at is more sensitive to AWGN. Moreover, the power of the payload
the receiver, which is the purpose of introducing discrete circle signal will decrease with the increasing of pilot amplitude. We
(DC) Viterbi-Viterbi. The signal An added white Gaussian noise design a novel scheme with 8APSK-PA to solve the problems.
and phase noise which includes payload signal Xn and pilot Fig. 10 shows the pilot structure with the eight different con-
signal Pn . The 8APSK pilot signal Pn is extracted from An stellation points in the inner circle (IC) and the outer circle
and it’s recovered by the DC-VV algorithm and be decided to (OC). The amplitude levels in the 1024-QAM constellation

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
TIAN et al.: DESIGN OF PROBABILISTIC SHAPING 4D ULTRA HIGH ORDER MODULATION 3693

Fig. 11. Simulation setup for the 20Gbaud PDM-PS-1024QAM and PS-4D-
SP-1024QAM systems.

are A = {±1, ±3, ±5, . . . , ±31}. The absolute value of the

amplitude level is selected as the amplitude value of the IC and
OC (AoIC and AoOC). For QPSK pilot recovery algorithm, the
accuracy of single pilot amplitude for phase recovery is reduced.
Fig. 12. The RSNR at the BER of 3.8×10−3 versus linewidth for PDM-PS-
To improve the estimation accuracy, the optimal 8APSK pilot 1024QAM system using different PA-CPR amplitude.
amplitude in IC and OC is chosen in phase recovery scheme. In
this paper, the 8APSK is selected as of the pilot for phase recov-
ery to be better matched with PS-4D-SP-1024QAM signals. The
PS-4D-SP-1024QAM is designed to meet the modulation format
redundancy requirements with the number of pilot symbols and
control the pilot insertion rate. In addition, the 8APSK pilot
scheme can improve the recovery performance of the algorithm
without changing the average transmission power of the sys-
tem compared with QPSK. Moreover, the 8APSK-PA-CPR has
lower complexity than blind phase recovery algorithm.

IV. SIMULATION AND RESULTS

Fig. 11 shows the simulation setup for 20 GBaud PDM-
PS-1024QAM and PS-4D-SP-1024QAM transmission systems.
The 1250 pilot symbols generated by random generation or
amplitude translation are evenly inserted into 33750 signal
symbols to form a block of 35000 symbols, resulting in a pilot
insertion ratio of 1/27. The light is emitted at 1550 nm by a
laser, separated by a polarization beam splitter, and modulated Fig. 13. Pilot amplitude optimization of 8APSK-PA-CPR.
by two IQ modulators with a PDM-PS-1024QAM signal or a
PS-4D-SP-1024QAM signal at the transmitter. The signal is
sent to a variable optical attenuator (VOA) and an EDFA, which a 7% FEC overhead. It is demonstrated that the maximum gain
provides the ASE noise to control the SNR. Afterward, the signal of the AoIC = 13 and AoOC = 17 8APSK-PA-CPR obtained
is transmitted across 25km G652D standard single-mode fiber. by amplitude optimization is ∼0.4 dB compared with 15 level
Other impairments in coherent optical communication systems QPSK-PA-CPR at the equal average power. The performance of
are assumed to be completely compensated, leaving only phase phase recovery algorithm is affected by the AoIC and AoOC (see
noise and AWGN to be considered. The signals are received by Fig. 12). This is because the pilot points with small amplitude
an integrated coherent receiver and then processed offline by a are more affected by AWGN than those with large amplitude.
DSP algorithm. To find the optimal AoIC and AoOC, we numerically simulate
the GMI performance of different IC and OC pilot schemes with
A. Pilot Amplitude Optimization laser linewidth of 10 kHz and 100 kHz. HD-GMI is derived as
[29]:
On the PDM-PS-1024QAM, the 8APSK-PA-CPR scheme
is compared to the QPSK-PA-CPR scheme at approximately GM IHD = H [1 − (−BERlog2 (BER)−(1 − BER)
average power. The amplitude of the pilot in two circles is swept
× log2 (1−BER))] (12)
and optimized. Fig. 12 shows the SNR variation with different
laser linewidths for QPSK-PA-CPR and 8APSK-PA-CPR with where H represents constellation entropy. As shown in Fig. 13,
different AoIC and AoOC at approximate average power. The the PS factor λ is set to 0.01 and the SNR to 31.5 dB in the
performance of the above algorithms is analyzed by calculating process of radius optimization. GMI is improved when the AoIC
the required SNR at the BER of 3.8×10−3 , which corresponds to and AoOC increase gradually. However, there will be a GMI

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
3694 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 40, NO. 12, JUNE 15, 2022

Fig. 14. GMI results versus λ with different phase recovery algorithms in PS-1024QAM systems when the SNRs are (a) 20 dB. (b) 23 dB. (c) 26 dB. (d) 29 dB.

ceiling because the effect of AWGN for the pilot format has algorithm with PS, even though increasing the window size
been minimized when it increases to a certain extent. In Fig. 12, reduces the sudden drop in GMI. The impact of different PS
a 15 level is adopted in QPSK-PA-CPR to maintain the average factors on GMI in the 8PASK-PA-CPR scheme is evaluated.
transmitted power in the system. Due to the increase in the Fig. 14 shows the GMI performance of 8PASK-PA-CPR and
average energy of the transmitted signal, the system requires BPS (N = 100, B = 128) in different PS factors λ. The SNRs
higher transmitted power to keep the same operating SNR. The are 20 dB, 23 dB, 26 dB and 29 dB respectively. For SNR
selection of CPR pilots at the average energy is more appropriate. = 20 dB (Fig. 14(a)) the sudden drop of GMI under BPS is
In Fig. 13, the AoIC = 13 and AoOC = 17 is chosen as close obvious from 0 to 0.06. 8APSK-PA-CPR does not cause a sudden
as possible to the average power to carry out the amplitude of drop in the whole measurement interval of PS factor λ. In the
the 8APSK-PA-CPR scheme to avoid the change of the overall condition of low SNR, the additive white noise is dominant and
average power. the 8APSK-PA-CPR has accurate recovery on phase noise. The
BPS algorithm is a blind estimation of AWGN and phase noise.
The performance of 8APSK-PA-CPR is inferior to that of blind
B. Compatibility Evaluation of 8APSK-PA-CPR in phase recovery algorithm when the AWGN is too large to be
Probabilistic Shaping System accurately estimated. In this situation of SNR, the performance
In PS systems, the GMI will suddenly decline at the optimal of 8APSK is greatly affected by AWGN compared with BPS,
value of λ [17] when using the BPS algorithm for phase recovery resulting in a gap of 0.5 bit/symbol. Moreover, the FEC threshold
at low SNR. The BPS algorithm has a strong dependence on of GMI corresponding to 3.8×10−3 cannot be reached. For
PS, even for long noise rejection windows. The decisions made SNR = 23 dB (Fig. 14(b)), the sudden drop caused by the
inside the BPS algorithm are affected by shaping. The impact of BPS algorithm is alleviated, and the GMI of 8APSK-PA-CPR
PS on BPS can affect the overall system performance, specially increases by ∼0.1 bit/symbol compared with that of BPS. The
at low SNRs [21]. It reflects the incompatibility of the BPS GMI performance of 8APSK-PA-CPR is better than that of

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
TIAN et al.: DESIGN OF PROBABILISTIC SHAPING 4D ULTRA HIGH ORDER MODULATION 3695

Fig. 15. GMI results versus SNR with different polarizations in PDM-PS-1024QAM and PS-4D- SP-1024QAM systems when the value of shaping parameter
is 0.02, the linewidths are (a) 10 kHz. (b) 50 kHz. (c) 100 kHz. (d) 200 kHz. .

BPS as the value of SNR increases. For example, the maximum there is a larger gap between PS-1024QAM and Shannon limit
GMI increases by 0.7 bit/symbol when SNR = 26 dB, and the [4] The Euclidean distance is expanded by multidimensional
maximum GMI increases by 1.3 bit/symbol when SNR = 29 set-partition, so the noise tolerance of PS-4D-SP-1024QAM is
dB, approaching the limit (see Fig. 14(c) and (d)). The GMI higher than that of PS-1024QAM. As shown in Fig. 15, the
with 8APSK-PA-CPR can reach the FEC threshold at this SNR, SNR gain of PS-4D-SP-1024QAM is ∼2.8 dB compared with
when the shaping parameter is greater than 0.02. It is acceptable PDM-PS-1024QAM. The gain of 2.8 dB is the theoretical gain of
to obtain significant gain at higher SNR (greater than 26 dB). modulation format. For linewidth = 10 kHz (Fig. 15(a)) the gaps
between Shannon limit with the GMI of PS-4D-SP-1024QAM
and that of PDM-PS-1024QAM is 0.5 dB and 3.5 dB respec-
C. Evaluation of PS-4D-SP-1024QAM and 8APSK-PA-CPR tively when the GMI is 8 bit/symbol. For linewidth = 200 kHz
Performance (Fig. 15(d)) the gaps rise to 1.3 dB and 4.6 dB, respectively.
In the 8APSK-PA-CPR scheme, the overhead generated by The sensitivity of PS-4D-SP-1024QAM decreases by only 0.7
the second stage is put into the pilot symbol to obtain additional dB from the linewidth 10 kHz to 200 kHz, while the sensitivity
mutual information and increase the transmission efficiency of of PDM-PS-1024QAM decreases by 1.1 dB, indicating that the
the system. Fig. 15. shows the SNR and GMI curves of PDM-PS- proposed scheme has a high phase noise tolerance and better
1024QAM and PS-4D-SP-1024QAM applied with the 8APSK- pilot phase recovery performance behavior. The penalty between
PA-CPR scheme. The PS factor is 0.02, and the linewidths are X polarization and Y polarization of PS-4D-SP-1024QAM for-
set to 10 kHz, 50 kHz, 100 kHz, and 200 kHz respectively. mat is ∼0.15 dB due to the amplitude translation taken in the
In the simulation, the HD decoder is used at the receiver and 4D format. The maximum GMI of PS-4D-SP-1024QAM can

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
3696 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 40, NO. 12, JUNE 15, 2022

TABLE II REFERENCES
COMPUTATIONAL COMPLEXITY OF DIFFERENT ALGORITHMS
[1] Y. Koizumi, K. Toyoda, M. Yoshida, and M. Nakazawa, “1024 QAM (60
Gbit/s) single-carrier coherent optical transmission over 150 km,” Opt.
Exp., vol. 20, no. 11, pp. 12508–12514, 2012, doi: 10.1364/oe.20.012508.
[2] S. Beppu, K. Kasai, M. Yoshida, and M. Nakazawa, “2048 QAM (66Gbit/s)
single-carrier coherent optical transmission over 150 km with a potential
SE of 15.3 bit/s/Hz,” Opt. Exp., vol. 23, no. 4, pp. 4960–4969, 2015,
doi: 10.1364/oe.23.004960.
[3] R. Maher, K. Croussore, M. Lauermann, R. Going, X. Xu, and J. Rahn,
“Constellation shaped 66 GBd DP-1024QAM transceiver with 400 km
transmission over standard SMF,” in Proc. Eur. Conf. Opt. Commun.,
Gothenburg, Sweden, 2017, pp. 1–3.
[4] S. Okamoto, M. Terayama, M. Yoshida, K. Kasai, T. Hirooka, and M.
be achieved at 17.7971 bit/symbol with the increasing of SNR Nakazawa, “Experimental and numerical comparison of probabilistically
for the linewidth at the FEC threshold of 3.8×10−3 . The final shaped 4096 QAM and a uniformly shaped 1024 QAM in all-Raman
net bit rate is 293.7 Gbit/s considering a 7% FEC overhead, a amplified 160 km transmission,” Opt. Exp., vol. 26, no. 3, pp. 3535–3543,
Feb. 2018.
pilot insertion ratio of 1/27 and redundant symbols from the first [5] S. L. Olsson, J. Cho, S. Chandrasekhar, X. Chen, E. C. Burrows, and P. J.
stage. Winzer, “Record-high 17.3-bit/s/Hz spectral efficiency transmission over
50 km using probabilistically shaped PDM 4096-QAM,” in Proc. Opt.
Fiber Commun. Conf., 2018, pp. 1–3.
D. Computational Complexity Analysis [6] S. L. Olsson, J. Cho, S. Chandrasekhar, X. Chen, P. J. Winzer, and S.
Makovejs, “Probabilistically shaped PDM 4096-QAM transmission over
The complexity of the DSP algorithm is important for prac- up to 200 km of fiber using standard intradyne detection,” Opt. Exp.,
tical application. The low complexity algorithm can effectively vol. 26, no. 4, pp. 4522–4530, Feb. 2018, doi: 10.1364/oe.26.004522.
[7] M. Terayama, S. Okamoto, K. Kasai, M. Yoshida, and M. Nakazawa,
reduce the system power consumption and increase the speed “4096 QAM (72 Gbit/s) single-carrier coherent optical transmission with
of hardware processing. As shown in Table II, the complexity a potential SE of 15.8 bit/s/Hz in all-Raman amplified 160 km fiber link,”
includes four aspects: real multipliers, real adders, comparison, in Proc. Opt. Fiber Commun. Conf. Expo., 2018, pp. 1–3.
[8] Y. Wang, S. Okamoto, K. Kasai, M. Yoshida, and M. Nakazawa, “Single-
and decision operation. The parameters are selected when the channel 200 Gbit/s, 10 Gsymbol/s-1024 QAM injection-locked coherent
three algorithms are based on optimum implementations. Tak- transmission over 160 km with a pilot-assisted adaptive equalizer,” Opt.
ing the linewidth of 100 kHz as an example (N = 100, B = Exp., vol. 26, no. 13, pp. 17015–17024, 2018, doi: 10.1364/oe.26.017015.
[9] X. Chen, S. Chandrasekhar, J. Cho, and P. Winzer, “Transmission of 30-
128), the DP-BPS algorithm requires 12NB+2B real multiplier, GBd polarization-multiplexed probabilistically shaped 4096-QAM over
12NB+2B real adders, 2B comparison, and 2NB decision oper- 50.9-km SSMF,” Opt. Exp., vol. 27, no. 21, pp. 29916–29923, Oct. 2019.
ation. In the 8APSK-PA-CPR scheme the DC-VV algorithm is [10] X. Chen, J. Cho, A. Adamiecki, and P. Winzer, “16384-QAM transmission
at 10 GBd over 25-km SSMF using polarization-multiplexed probabilistic
used to estimate the phase of the 8APSK pilot, which increases constellation shaping,” in Proc. 45th Eur. Conf. Opt. Commun., Dublin,
a small amount of complexity (2 Comparison) compared with Ireland, 2019, pp. 1–4.
QPSK-PA-CPR. The complexity of the 8APSK-PA-CPR algo- [11] T. A. Eriksson, M. Sjödin, P. Johannisson, P. A. Andrekson, and M. Karls-
son, “Comparison of 128-SP-QAM and PM-16QAM in long-haul WDM
rithm is also reduced by 99.99% compared with the DP-BPS transmission,” Opt. Exp., vol. 21, no. 16, pp. 19269–19279, Aug. 2013.
algorithm. [12] J. K. Fischer et al., “Generation, transmission, and detection of 4-D set par-
titioning QAM signals,” J. Lightw. Technol., vol. 33, no. 7, pp. 1445–1451,
Apr. 2015.
V. CONCLUSION [13] J. Lu et al., “Carrier phase recovery for set-partitioning QAM formats,” J.
Lightw. Technol., vol. 36, no. 18, pp. 4129–4137, Sep. 2018.
We demonstrate a scheme of probabilistic shaping 4D ultra- [14] P. J. Winzer, “High-spectral-efficiency optical modulation formats,” J.
high order modulation format with pilot aided carrier phase re- Lightw. Technol., vol. 30, no. 24, pp. 3824–3835, Dec. 2012.
[15] F. Steiner et al., “Experimental verification of rate flexibility and prob-
covery. The four-dimensional set partitioning modulation format abilistic shaping by 4D signaling,” in Proc. Opt. Fiber Commun. Conf.
based on amplitude translation in probabilistic shaping systems Exhibit., San Diego, CA, USA, 2018, pp. 1–3.
is designed to approach the Shannon limit. The 8APSK is used as [16] Q. Guo, W. Peng, C. Yan, and Y. Bai, “Multi-dimensional probabilistic
shaping for higher fibre nonlinearity tolerance,” in Proc. Eur. Conf. Opt.
the pilot format to optimize the performance of the CPR scheme. Commun., Dublin, Ireland, 2019, pp. 1–3.
It is demonstrated that 8APSK-PA-CPR shows great resilience [17] J. Hu, Z. Sun, X. xu, Y. Lu, and Y. Qiao, “Carrier phase recovery exploiting
to phase noise compared with the traditional PA-CPR and BPS recursive blind phase search algorithm for probabilistic shaping 64QAM
systems,” in Proc. 2020 Asia Commun. Photon. Conf. Int. Conf. Inf.
algorithm. The SNR gain of the 8APSK-PA-CPR scheme ob- Photon. Opt. Commun., Beijing, China, 2020, pp. 1–3.
tained by amplitude optimization is ∼0.4 dB compared with the [18] I. Fatadin, D. Ives, and S. J. Savory, “Laser linewidth tolerance for 16-
traditional QPSK-PA-CPR. The steep drop of BPS with different QAM coherent optical systems using QPSK partitioning,” IEEE Photon.
Technol. Lett., vol. 22, no. 9, pp. 631–633, May 2010.
PS factors is overcome by 8APSK-PA-CPR when the SNR ratio [19] Y. Gao et al., “Low-complexity two-stage carrier phase estimation for
is lower than 24 dB. The maximum improvement of the general- 16-QAM systems using QPSK partitioning and maximum likelihood
ized mutual information (GMI) is 1.3 bit/symbol when the SNR detection,” in Proc. Conf. Opt. Fiber Commun., Los Angeles, CA, USA,
2011, pp. 1–3.
is higher than 26 dB. The complexity of the 8APSK-PA-CPR [20] Q. Zhang and C. Shu, “Viterbi and Viterbi algorithm based phase recovery
algorithm is reduced by 99.9% compared with the BPS. We for probabilistically shaped signals,” IEEE J. Lightw. Technol., vol. 39,
demonstrated the performance of PS-4D- SP-1024QAM with no. 5, pp. 1364–1370, Mar. 2021.
[21] J. Lu, S. Fu, M. Tang, M. Xiang, P. Shum, and D. Liu, “Low-complexity
8APSK-PA-CPR under different laser linewidth. The simulation carrier phase estimation for M-ary QAM based on blind phase search
results show that the proposed scheme obtained ∼2.8 dB SNR using simplified measurement,” in Proc. Int. Conf. Opt. Commun. Netw.,
gain compared with PDM-PS-1024QAM. Hangzhou, China, 2016, pp. 1–3.

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.
TIAN et al.: DESIGN OF PROBABILISTIC SHAPING 4D ULTRA HIGH ORDER MODULATION 3697

[22] D. A. A. Mello, F. A. Barbosa, and J. D. Reis, “Interplay of probabilistic Qi Zhang (Member, IEEE) received the Ph.D. degree from the Beijing Univer-
shaping and the blind phase search algorithm,” IEEE J. Lightw. Technol., sity of Posts and Telecommunications, Beijing, China, in 2005. Her research
vol. 36, no. 22, pp. 5096–5105, Nov. 2018. interests include optical communication and satellite communication.
[23] J. Cho, X. Chen, S. Chandrasekhar, and P. Winzer, “On line rates, infor-
mation rates, and spectral efficiencies in probabilistically shaped QAM
systems,” Opt. Exp., vol. 26, no. 8, pp. 9784–9791, 2018.
[24] G. Böcherer, F. Steiner, and P. Schulte, “Bandwidth efficient and rate
matched low-density parity-check coded modulation,” IEEE Trans. Com-
mun., vol. 63, no. 12, pp. 4651–4665, Dec. 2015.
[25] R. Kschischang and S. Pasupathy, “Optimal nonuniform signaling for
Gaussian channels,” IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 913–929, Ran Gao (Member, IEEE) received the Ph.D. degree in electronic science and
May 1993. technology from the Beijing Institute of Technology, Beijing, China, in 2015. He
[26] G. Böcherer, “Probabilistic signal shaping for bit-metric decoding,” in is currently a Professor with the School of Information and Electronics, Beijing
Proc. IEEE Int. Symp. Inf. Theory, Honolulu, HI, USA, 2014, pp. 431–435. Institute of Technology. His research interests include fiber optical sensors,
[27] M. Morsy-Osman, Q. Zhuge, L. R. Chen, and D. V. Plant, “Feedfor- optical waveguide, and measurement instruments.
ward carrier recovery via pilot-aided transmission for single-carrier sys-
tems with arbitrary M-QAM constellations,” Opt. Exp., vol. 19, no. 24,
pp. 24331–24343, 2011.
[28] H. Cui, X. Xu, S. Liu, Y. Lu, and Y. Qiao, “QPSK pilot-aided carrier
phase recovery algorithm for 800-Gb/s/λ DP-256QAM transmission,” in
Proc. Asia Commun. Photon. Conf./Int. Conf. Inf. Photon. Opt. Commun.,
Beijing, China, 2020, pp. 1–3.
[29] A. Sheikh, A. Graell i Amat, and G. Liva, “Achievable information rates Zhipei Li received the B.S. degree in microelectronics from Harbin Engineering
for coded modulation with hard decision decoding for coherent fiber-optic University, Harbin, China, in 2013, and the Ph.D. degree in electronic science
systems,” J. Lightw. Technol., vol. 35, no. 23, pp. 5069–5078, Dec. 2017. and technology from the Beijing University of Posts and Telecommunications,
Beijing, China, in 2019. He was with Transmission and Access Research
Department of Huawei for one year. He is currently working on postdoctoral
research with the School of Information and Electronics, Beijing Institute of
Feng Tian received the Ph.D. degree from the Beijing University of Posts and Technology, Beijing, China. He is also engaged in research on high-speed
Telecommunications, Beijing, China, in 2013. Her research interests include fiber communication systems including high-baudrate and high-order modu-
high-speed fiber communication systems and digital signal processing. lation format signal generation, impairment compensation for high-bandwidth
optoelectronic devices, beyond 1Tb/s transmission in single channel, and other
coherent digital signal processing techniques.

Tianze Wu received the B.S. degree from Yanbian University, Yanji, China,
in 2018. He is currently working toward the Ph.D. degree with the Beijing
University of Posts and Telecommunications, Beijing, China. His research
interests include high-capacity fiber communication system, coded modulation,
and probabilistic shaping.
Qinghua Tian received the Ph.D. degree from the Beijing University of Posts
and Telecommunications, Beijing, China, in 2013. Her research interests include
optical communications and satellite communications.
Yiqing Ji received the B.S. degree from Xi’an Shiyou University, Xian, China,
in 2020. Her research interests include mutiband fiber communication systems
and digital signal processing.

Chuxuan Wang received the B.S. degree in electronic science and technology Xiangjun Xin received the Ph.D. degree from the Beijing University of Posts
in 2017 from the Beijing University of Posts and Telecommunications, Beijing, and Telecommunications, Beijing, China, in 2004. His research interests include
China, where he is currently working toward the Ph.D. degree. His current high-speed fiber communication systems, broadband optical transmission tech-
research interests include coherent optical communication system, few-mode nologies, and all optical networks.
fiber, and nonlinear compensation.

Authorized licensed use limited to: Universidade Estadual de Campinas. Downloaded on September 26,2023 at [Link] UTC from IEEE Xplore. Restrictions apply.