1

A NON-LINEAR IMPROVED CNN EQUALIZER WITH BATCH GRADIENT DECENT IN 5 G

WIRELESS-OPTICAL COMMUNICATION
Asish B Mathews,
Research Scholar, Department of ECE, Noorul Islam Centre for Higher Education,
Kanyakumari, Tamilnadu, India.
asishbmathews@gmail.com
Arun B Mathews
HSST, MHSS, Pathanamthitta
arunbmathews@gmail.com
G.Glan Devadhas,
Professor, Dept of EIE, Vimal Jyoti Engineering College, Kannur, Kerala.
glendeva@gmail.com

Abstract — A paper suggests novel nonlinear convolutional neural network (CNN) equalizer for
propagation in hybrid microwave-optical system. Batch Gradient Descent is implemented to
efficiently train the CNN equalizer. The approach suggested eliminates non-linearity and produces
improved performance. Through incorporating the GFDM network, this solution greatly increases
the storage ability of consumers and also offers a larger coverage range. The suggested CNN
equalizer is correlated with other approaches, including statistical models and estimator
techniques. Experimental finding shows that CNN produces the highest value that is marginally
better than the other equalizer with a numerical difficulty reduced to zero. The tests obtained have
shown that the new approach work much better than traditional techniques.
Keywords—Conventional neural network equalizer (CNN), Adjustable Nadaraya-Watson
Prediction (ANWP) and Modified Fractional Linear Estimator.

1. INTRODUCTION
In low-access optical communication networks with higher bandwidth access and low operating
costs, numerous applications that use big data and cloud computing play an active role. In Amplitude
Modulation Direct Detection systems, nonlinear components such as the Electro Optical (EO) modulator,
square law detector, and fibre propagation cause nonlinear distortion. The device's performance is
decreased as a result of these characteristics. The coordination mechanism has become increasingly
important for a variety of machine learning techniques [1], [2]. Optical fibre communication tasks that
are performed in a machine learning perspective include performance observation, fibre nonlinearity
reduction, carrier recovery, and recognizing modulation format [3]
[5].Fibre dispersion and nonlinear effects restrict the data rate in current optical transmission networks
[6] .
Many filters have been added, and the cyclic prefix has been surpassed [7]. A new digital signal
processing system has been developed that focuses on bandwidth-efficient mobile front haul aggregation
and deaggregation [8]. GFDM is a 5G waveform that uses sub-band filtering to minimize OoBE and
increase the waveform efficiency [9]. Established methods such as OFDM, FOFDM have few faults,
such as reduced efficiency, in coverage severe distances in highly populated areas and do not effectively
eradicate dispersion and nonlinear effects [10].
The binary data was explicitly observed in [11], but with the drawback of a long training period.
In coherent optical OFDM system, DL was successfully applied to estimate optical channels [11] and
[12], and the system was expanded to MIMO wireless systems [13].
2. PROPOSED METHOD
2.1 CNN EQUALIZER
Because distortion is nonlinear, adaptive equalization is achieved using a variety of neural
networks. On the other hand, densely connected networks require a large number of parameters, and the
complexity increases exponentially as the code length increases. In contrast to machine learning-based
equalization systems, CNN requires limited data preprocessing and has good feature extraction
capabilities. Since the ISI only exists between consecutive bits of a transmitted sequence and the effect
of nonlinear distortion is independent for each bit, a group of neurons in CNN only respond to a limited
range of inputs.
The nonlinear channel is considered in a discrete-time dispersive optical communication device.
The channel distortion is caused by the ISI, nonlinearities, and AWGN in this case.
2.1.1 Inter-symbol Interference
A finite impulse response filter is used to model the channel with ISI. [g 1 , g 2 , … . , g L ]T gives the
corresponding filter coefficients with length L. The signal with ISI is equivalent to the following channel
input convolution with the FIR filter.
v=s ⊗ g (1)
2.1.2 Nonlinear Distortion
The main sources of nonlinearities in the communication system are amplifiers and mixers,
which can be modelled by a nonlinear function h [•]. The output transmitted through the tube that
associated with ISI, nonlinearities, and AWGN is expressed as,

r i=h [ v i ] +ni (2)

2.1.3 Maximum Likelihood Equalizer
To estimate the channel condition for the ML channel equalizer, a training sequence
so ={ s01 , s 02 , … ., s0n } with n known bits is sent to the receiver side. For the channel coefficients, the ML
criterion is used.
h^ ML =arg max p (r ∨¿ s , h) ¿
0 o
(3)
After the channel coefficients have been estimated, the posterior probability of each transmitted
bit can be calculated.
p ( si=s|r , h^ ML ) ,i=1, 2 , .., N (4)
^ is obtained by feeding the output of the ML equaliser into
For information bits m, the estimate m
the channel decoder.

r
CNN Equalizer Decoder
Channel
v

Conv + ReLU
m Conv + ReLU
s
Channel
H(z) f(v)

Conv
Encoder

Figure 1 CNN System Architecture

The adopted CNN consists of several convolutional layers of ReLU, with only
convolution operations in the final layer. As channel equalization is used in computer vision, the input to
CNN is a 1-D real vector rather than regular 2-D representation. Using ReLU, 2-D convolution can be
rewritten in the form of 1-D.
C K
Y i , j=σ ( ∑ ∑ W i, c ,k , X c. k+ j +bi ) (5)
c=1 k=1

2.1.4 NEURAL NETWORK DECODER (NND)

Each neuron in a fully-connected layer is correlated with its weight and is densely
connected to the previous layer. From the CNN equalizer, NND takes a 1-D vector as a data. To compute
a single layer in NND, use the following matrix multiplication.

y=σ (Wx +b) (6)

2.2 IMPROVED ANN EQUALIZER
Artificial Neural Network plays a major role in learning the non-linear mapping function between
input and output data. The input layer is made up of M neurons, the hidden layer is made up of K
neurons, and the output layer consists of a neuron, all of which together constitute an ANN. The
nonlinear mapping characteristic present in the network is represented as the activation function of each
and every neurons. A hidden layer with the help of a neural network in which every continuous function
can be inferred is explained.
Backpropagation (BP) algorithm is used to train the ANN equalizer under the Mean Square Error criterio
nThe output formed from the fed input continued by back-propagation (BP) can easily be predicted using
forward propagation.
The gradient descent is a prevalent method for the creation of different shallow and deep neural n
etworks. It is implemented with a view to maximizing the cost feature associated with W1 and W2.The
calculation of cost function is done based on a number of preferred data which in turn divides the
gradient descent into three variant.
The Batch Gradient Descent (BGD) calculates the cost function’s gradient for the complete
training set to accomplish an update step, but the training process may not be rapid in the ANN when
large dataset are being encountered. The stochastic gradient descent (SGD) accomplishes single
parameter update for single training instance. Nevertheless, SGD achieves a high variance along with
recurrent update and that is typically the reason for cost function to swing theatrically. This typically
makes convergence speed obfuscate to the precise least.

2.3 ADJUSTABLE NADARAYA WATSON ESTIMATOR

Haar kernels based version is efficient computationally and are defined consequently to achieve
finite samples. Nonparametric approximation of regression function has been developed with the help of
Adjustable Nadaraya–Watson approach.
The Adjustable Nadaraya–Watson estimator has various abilities which include preservation of
bias and variance. Consider the process {( X 1 Y 1¿ }1 ¿ ∞t=−∞ is stationary.

m ( x ) =E ( ∅ ( Y 1 )∨X 1 =x ) (7)

Which represents the regressions function, where ∅ ( Y 1 ) is an arbitrary measurable function. The
conditional distribution can also be estimated along with regression function ( ∅ ( Y 1 )=1(Y 1 ≤ y) for
every fixed value y and conditional distribution. The polynomial function performs approximation when
th
( P+1) continuous derivative is existing in m(u) as
 m ( u) ≈ ¿ (8)

Here K(.) represents the kernal function,

K h ( . )=
( h. ) c
K .
(x )
which defines the weight function, reliant
n, t
h
on the preferred data and h=h n>0 denotes the bandwidth. The adjustable Nadaraya Watson estimator can
be defined by,

∑ K h (x−X t )∅ (Y t )
^ NW ( x )= t =1
m n
(9)
∑ K h (x− X t )
t=1

and if p =1 and c n ,t (x) =1, it is the local linear estimate

n
^ ¿ ( x )=∑ wt ∅ (Y t )
m (10)
t =1

2.4 MODIFIED FRACTIONAL LINEAR ESTIMATOR

A new generalized origin of low order optimization of linear prediction has been developed using
fractional derivatives. The improved version of order based fractional derivative formula allows versatile
characterization of definitive definitions. The exceptional types of optimal partial linear predictors are
first-order optimization and second-order optimization of linear predictors. Limited memory is sufficient
for the closed form expressions of this prophet. An estimate of the optimal sequence of fractional
derivations is accomplished by determining the inverse of predictive memory.

3. RESULTS AND DISCUSSION

The suggested process is implemented using optisystem and Matlab 2016a, with the optical signal
being sent as an input signal. No linearity is reduced by the equalizer. The feedback for the proposed
hybrid design of the GFDM based.
The findings of the experiment were obtained in the presence of ISI and maximum likelihood effects
in the contact medium. Table 1 summaries the parameters for the 16-QAM modulation scheme and the
CNN equalization used.
Table 1: Simulation Parameters
Modulation 16-QAM
Equalization CNN
Channel Quality Indicator 7
No. of transmit antenna 2
No. of receive antenna 2
 Scheduler PF,RR,Greedy

The GFDM Modulator modulates the binary data into the QAM format. Figure 1 shows the
impulse series that is provided by GFDM modulation. The information descriptions of the ith subcarrier
are up sampled using the sampling factor N.

Figure 1: GFDM Input Signal

Figure 2 shows the data being converted back to serial form and modulated with QAM.

Figure 2: QAM Waveform

 Figure 3: GFDM Signal Generation

Figure 4: Performance of throughput in GFDM

The performance analysis of throughput that focused on the values of distance and throughput are
shown in figure 4. GFDM based Enhanced Proportional Fair has better efficiency (98.5%) when
compared to other algorithms.
 performance comparison
45

40
GFDM greedy
35 GFDM rr

sum of logarithmic throughput

GFDM pf
GFDM ipf
30

25

20

15

10

0
2 4 6 8 10 12 14 16 18 20
No.of base stations

Figure 5: Scheduling algorithms comparison

Figure 5 shows GFDM system scheduling algorithm performance. GFDM based enhanced
proportional fair has better throughput when compared with other scheduling algorithm.

Adjustable Nadaraya watson estimator output

0.45
equalized signal
0.4 predicted signal-ANWE

0.35

0.3
Predicted value

0.25

0.2

0.15

0.1

0.05

0
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Iteration

Figure 6: Adjustable Nadaraya-Watson estimator

The Adjustable Nadaraya – Watson method is used for nonparametric estimation of regression fu
nction. GFDM Enhanced Proportional Equal increases the coordination and distribution of services for c
onsumers as is seen in Figure 6.
The transmission was executed over a multipath channel. Figure 7 shows the production of
analyzing the performance of a CNN model.

Figure 7: Output of the CNN Equalizer

 Figure 8: Performance of throughput using CNN
Figure 8 depicts the signal throughput achievable by all users. In GFDM system, the
scheduling algorithm throughput is analyzed using CNN equalizer. GFDM based proportional fair has
better throughput when compared to other algorithm.

Figure 9: Error rate output of various equalizers

Figure 9 shows the Bit Error Rate of different algorithms. BER vs Eb/No analyzed with various
equalizer. CNN equalizer produce better performance than other equalizer and also it shows that as
Eb/No increases then CNN equalizer decreases significantly.

Figure 10. Comparison of average capacity of GFDM system

 The average potential of a GFDM system is demonstrated for various numbers of transmitter and
receiver antennas, as shown in figure 10. If the number of transmitters grows, the system's power grows
as well.

Figure 11. CNN Equalizer performance comparison

The signal throughput efficiency for all users is shown in Figure 11. In contrast to the other two
algorithms, the CNN algorithm produces a very high throughput standard.

Table 2: Gain of estimator methods

SI.NO Method Gain in db
1 Linear estimator 12.21
2 Fractional linear 13.28
estimator
3 Modified fractional 15.43
linear estimator

linear estimator
Original signal
0.2 Linear estimator
Amplitude

-0.2
0 0.5 1 1.5 2 2.5
5
Time in samples 10
fractional linear estimator
Original signal
0.2 fractional linear estimator
Amplitude

-0.2
0 0.5 1 1.5 2 2.5
Time in samples 5
10

Figure 12: Linear and fractional linear estimator output.

The Figure 12 indicates linear estimator and fractional linear estimator outputs. The gain value
of linear estimator is 12.21db and the fractional linear estimator gain value is 13.28db. By using
fractional linear estimator, overall original signal is obtained.
 The modified fractional linear estimator output shows the better result while comparing the
linear estimator and fractional linear estimator techniques. The gain value of the modified fractional
linear estimator is 15.43db. Based on the gain value, the modified fractional linear estimator signal is
best and is depicted in Figure 13.

Modified Fractional Linear Estimator

Original signal
0.3 Modified Fractional Linear Estimator

0.25

0.2

0.15

0.1

Amplitude
0.05

-0.05

-0.1

-0.15

-0.2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Time in samples 105

Figure 13: Modified fractional linear estimator output signal

As a result, different CNN configurations are evaluated to decide the best structure. Figure 14
shows the BER results for each configuration. It can be seen that in a 4-layer CNN, more filters
guarantee better efficiency (red line). Deeper structures, on the other hand, outperform shallow structures
with the same or less parameters (blue and green line).

Figure 14: Comparison of BER with CNN equalizers over a linear channel
Figure 15 depicts proposed CNN equalizer's decision boundary. The received signal obtained
with ISI and nonlinear distortion. To retrieve the original signal, CNN establishes a nonlinear decision
boundary.
 Figure 15: Comparison of BER with various equalizers over a nonlinear channel
From figure 16, it is clear that the output of the CNN scheme is very similar to that of the DL
system, with a small improvement on high SNR. Furthermore, CNN equalizer can be used for long
codes.

Figure 16: Over a nonlinear channel, the BER output of various equalization and decoding schemes is
shown
 Figure 17: After 20 km SMF, BER output vs. obtained power for the PAR value of 0.25

Figure 18: For a 40 km SMF and a PAR of 0.25, BER output vs. obtained power

Table.3 Comparison of the computational complexity and performance improvement for different
equalization technologies
Equalization FFE/ DD-FTN ID-FFE/ CR-FFE VNLE Sparse SVM CNN
Technologies DFE ID-DFE VNLE
Computational Low Fair Low Low High Fair High High
Complexity
Performance Low Fair Fair Fair High High High High
Improvement
 Figure 19: CNN equalization and decoding efficiency are compared.
Figure 19 shows that CNN equalization technique outperform the baseline system in this
channel, with a gain of about 1 ∼ 2 dB, demonstrating the efficacy of the proposed method.

Figure 20: GFDM detection and decoding efficiency was compared.

For channel estimation, the proposed methods are compared to minimum mean square error
methods is shown in figure 20. The results show that CNN equalizer can provide a better output in signal
detection when compared to MMSE method.
4. CONCLUSION
A modern highspeed CNN equalizer is being modeled and experimentally implemented. The opti
cal signal is produced in the proposed method and CNN equalizer technique is used to the non-linearity e
ffect. In terms of BER, the system with CNN balance achieves a major change. It's due to the CNN
equalizer's composition, which includes a memory-less nonlinear square to mitigate nonlinear effects.
The CNN equalizer is found to be the excellent equalizer related to the performance of others. It
determines the favorable expectations to make use of state-of-art CNN based methods which can reduce
the nonlinear distortion defects in optical transmission system.
DISCLOSURES
We authors, hereby declare that there is no conflict of interest.
 REFERENCES

1. Jiang, Chunxiao & Lajos Hanzo. "Machine learning paradigms for next-generation wireless
networks." IEEE Wireless Communications 24, (2017): 98-105.
2. Khan & Faisal Nadeem. "Machine learning methods for optical communication systems." In Signal
Processing in Photonic Communications, pp. SpW2F-3., 2017.
3. Thrane & Jakob "Machine learning techniques for optical performance monitoring from directly
detected PDM-QAM signals." Journal of Lightwave Technology 35, no. 4 (2017): 868-875.
4. Zibar, Darko & Molly Piels, "Machine learning techniques in optical communication." Journal of
Lightwave Technology 34, no. 6 (2016): 1442-1452.
5. Khan & Faisal Nadeem "Modulation format identification in heterogeneous fiber-optic networks using
artificial neural networks." Optics express 20, no. 11 (2012): 12422-12431.
6. Essiambre & René-Jean "Capacity limits of optical fiber networks." Journal of Lightwave Technology
28, no. 4 (2010): 662-701.
7. Soldani & David"Horizon 2020 and beyond: on the 5G operating system for a true digital society."
IEEE
Vehicular Technology Magazine 10, no. 1 (2015): 32-42.
8. Shah & Syed Adeel Ali. "Adaptive beaconing approaches for vehicular ad hoc networks: A survey."
IEEE Systems Journal 12, no. 2 (2018): 1263-1277.
9. Chiti and Giovanni Rigazzi. "Communications protocol design for 5G vehicular networks." In 5G
Mobile Communications, pp. 625-649., 2017.
10. Deniz C & Uyan O G, “On the performance of LTE downlink scheduling algorithms: A case study
on
edge throughput”, Computer Standards & Interfaces, Elsevier, pp. 96-108, August 2018.
11. T. Gruber, S. Cammerer, J. Hoydis, and S. ten Brink, “On deep learning based channel decoding,” in
Annual Conference on Information Sciences and Systems (CISS). IEEE, 2017, pp. 1–6.
12. W. Xu, Z. Wu, Y.-L. Ueng, X. You, and C. Zhang, “Improved polar decoder based on deep
learning,”
in IEEE International Workshop on Signal Processing Systems (SiPS), 2017, pp. 1–6.
13. H. Ye and G. Y. Li, “Initial results on deep learning for joint channel equalization and decoding,” in
IEEE Vehicular Technology Conference (VTC-Fall), 2017, pp. 1–5.
14. V. Jain and S. Seung, “Natural image denoising with convolutional networks,” in Advances in
Neural
Information Processing Systems, 2009, pp. 769–776.
15. F. Liang, C. Shen, and F. Wu, “An iterative bp-cnn architecture for channel decoding,” IEEE Journal
of Selected Topics in Signal Processing, vol. 12, no. 1, pp. 144–159, 2018.