1.

Introduction
Wireless networks and mobile devices are subject to the same risks and vulnerabilities as traditional
wired networks. However, the risks and threats associated with wireless networks have recently taken
on a new dimension, ostensibly because the communication air medium of wireless networks is openly
exposed to intruders, who use this to launch malicious attacks such as eavesdropping, spoofing, data,
and jamming attacks to disrupt operations.

Security techniques provide enhancements in security measures to detect and react against the attacks.
However, these techniques are no longer sufficient because attackers are smart enough to evade
conventional security systems. Moreover, the new communication technologies such as 5G require
low latency and high data rate and unlimited connectivity, and then these massive potentials bring
about many security challenges.

Speeds will be multiplied many times over in 6G compared to 5G, latency will be non-observable,
connection will be ubiquitous, and crucial infrastructures will be automated utilizing the underlying
network architecture. As a result, the network security paradigm will shift even farther toward very
flexible, dynamic, and autonomous systems. With the convergence of numerous IoT devices and
services, UAVs, V2X, and smart home appliances within 6G networks, distinguishing between a
security attack and genuine traffic would be almost impossible or unmanageable without the use of
efficient security techniques, because the conventional defense techniques are often resource
inefficient or might have limited protection for such new communication technologies.

As a result, ML-based security solutions are unavoidable. Machine learning has emerged as a
promising solution and a volume of literature exists on the methodological studies of ML to resolve
the security challenge. Unfortunately, the majority of machine learning ML algorithms are lifted from
other fields (CV and NLP) that thrive in tiny, confined contexts. The adoption of such ML systems in
wireless communication systems may unintentionally expose the network to major security risks,
including unauthorized resource consumption, denial of service attacks, and the leakage of sensitive
data. ML-based new communication technologies might be attacked using both conventional and ML-
based attacks, the last one is referred to as adversarial machine learning.

Similar to the general purpose 6G vision, which builds intelligence on top of 5G networks that have
been cloudified and softwarized, the 6G security vision similarly closely integrates ML to enable
security automation. ML-based security techniques must be proactive, self-aware and self-adaptive,
and should be considered from end-to-end networks perspective especially in 6G; hence, physical
layer security PLS is the most important security domain for the future radio communication
technologies [1].

Several questions might be asked in the security context: What are the threats and risks for the future
radio communication systems? What are the applications of ML for radio communications security?
What is the influence of conventional attacks on ML? This chapter will answer these questions.
 2. Threats on future radio communications:
PHY layer attacks:

Wireless communications security refers to any steps taken to prevent unwanted access or harm to
information communicated through wireless networks, as well as to assure data integrity and
confidentiality. The majority of current security designs might be breached because airwaves are
vulnerable to spying from anybody with a radio frequency RF antenna. For example, online
transactions for e-commerce and credit card reasons might be attacked to steal user’s personal
information, and this is a critical security issue, hence such systems must be secure enough. The
security risks or attacks might be launched using networking concepts such as network injection attack
when the attacker aims to use a re-configurable networking command, radio frequency concepts….
Because this report seeks to discuss the radio communication domain, and the future radio
communications such as 6G focus more on PHY layer, hence the PLS will be discussed, and here we
can distinguish three basic attacks, eavesdropping, spoofing and jamming. To describe these attacks,
Bob, Alice and Eve model is used.

Jamming Attacker Eve's goal in jamming

attacks is to disrupt lawful
communication by injecting
jamming signals (interference)
into the channel, causing Bob to
receive badly distorted
messages. The top part of Fig.1

Eavesdropping The adversary Eve uses

eavesdropping attacks to
retrieve the message sent by
transmitter Alice by simply
listening in on the channel. The
middle part of Fig.1

Spoofing Spoofing attacks include the

adversary Eve impersonating
the identity of the legitimate
transmitter Alice such that the
receiver Bob believes the
message was sent by Alice. The
bottom part of Fig.1.

Fig. 1. Alice (Transmitter), Bob(Receiver) and Eve(Attacker) Table. 1. Description of Attacks based on Fig. 1 topology

Technical descriptions:

Jamming: These attacks are a subset of denial of service DoS attacks in which malicious nodes
interrupt legitimate communication by generating purposeful interference in networks; therefore it is
radio signal transmission that disrupts communications by reducing the Signal-to-Inference-plus-Noise
ratio SINR [2].
Eavesdropping: The act of illegitimately intercepting and receiving information transferred across
wireless communication channels, which may result in the compromised information or data. Because
the digital radio is insecure, an attacker can intercept the signal from a safe distance. Man-In-The-
Middle MITM: is a type of eavesdropping attack in which the attacker surreptitiously intercepts two
parties' conversations [3]. The attacker impersonates the identities of both parties and acquires access
to information that the two are attempting to convey to each other.

Spoofing: It’s a sort of attack in which an attacker impersonates another computer on the network
using information received through a wireless sniffer. Spoofing attacks frequently target corporate
networks and may be used to steal sensitive information or to conduct man-in-the-middle attacks on
network hosts [3].

Countermeasures in PHY layer:

Because jamming is a very severe DoS attack, it is critical to have strong detection and
countermeasures in place. Traditional defense techniques against jamming attacks include channel
hopping to fool the jammer by switching between channels/ spread spectrum techniques which aim
to reduce the jammer impact by spreading the communication on a larger bandwidth, channel surfing
which offers migration to another when a jammer comes within range and disables communication on
a specific channel and spatial retreat transports mobile nodes from the jammed location to a secure
one…. [2].

For the spoofing attacks, the most employed countermeasure is the physical layer authentication
techniques, by using channel-based physical layer security schemes which exploit the traditional
channel features such as channel state information and received signal strength to detect spoofing
attacks.

And finally for eavesdropping attacks mitigations, in the contrary of traditional techniques which aim
to hide the messages under ciphers, the physical layer techniques prevent the attacker from getting
correct messages. Various techniques are used such as beamforming, coding, PHY encryption, or
using intelligent noise approaches which add noise intentionally to limit the eavesdropper efficiency.

All the techniques used to mitigate these

different types of attacks need to be designed
in an effective way, because they need to be
adaptive to the systems environmental
dynamic properties, which needs a frequently
updated knowledge about the communication
environment, in addition to the big demand of
efficient computational tools, hence, ML
techniques can be employed to design such
defense techniques efficiently, these
techniques can be considered from different
defense philosophies, some of them are prior
attacks such as monitoring and robust models
techniques, the others are during the attack time Fig. 2. ML as Secure Solution for PHY layer Attacks.
or posterior the attack (detection techniques).
 3. Based Machine Learning Secure Approaches:
As discussed before security approaches need the machine learning intervention, ML enables
automatic learning from and adaptation to wireless communication characteristics that are difficult to
capture with hand-crafted data and models. This section highlights research initiatives that integrate
machine learning to defend physical layer wireless attacks.

The ML solutions for the security can be divided into three categories based on the philosophy of the
defense; the following Fig. 3 and Table. 2 illustrate and describe these categories.

Monitoring ML solutions at aim to monitor the

systems; one example is anomaly
detection where the technique should
detect a malicious behavior.
Detection ML techniques detect a specific risk
such as jamming attacks, or multiple
risks.
Robustness At this stage ML models should be
robust to prevent the risk.

Fig. 3. Defense Philosophy. Table. 2. ML aim at each phase.

3.1. Monitoring ML Solutions:

Anomaly detection AD:

The goal of anomaly detection is to distinguish between normal and abnormal signals. These
anomalous signals might be created by a poorly designed process, or it could be formed as a result of
an attack or hacking effort or it could be generated by interference in the wireless network, bit errors,
re-transmissions, or signal jammers [4].

Traditional AD techniques are manual, and manually building anomaly detection is limited because it
creates a system that cannot adapt (or is costly and untimely to adapt), for this ML tools are used
because they are efficient, adaptive , on time, and can handle big computations.

Support vector machines have demonstrated state-of-the-art potential in many binary classification
applications, the messages can be classified into legitimate and malicious [5].

A supervised machine learning technique (random forest) is used to detect fault symptoms and
determine the cause. For example to detect operational anomalies in a base station, they employ
referenced signal received power RSRP provided by users during a specific time period. Certain base
station defects cause a significant shift in the RSRP measurements and a recognized electromagnetic
radiation pattern surrounding the base station. To do fault analysis, [6] provides a framework that
distinguishes between normal and abnormal processes in a changing environment in order to eliminate
superfluous fault alerts.

In [7] the identification of anomalous ZigBee signals is centered on characteristics taken from the in-
phase and quadrature components, as well as network traffic statistics were discussed. Authors
assessed the effectiveness of five supervised machine learning algorithms for anomalous RF
identification (Random Forest, J48, JRip, Naive Bayes, and PART) and determined the top learner.
Moreover, DL approaches can achieve better results due to the high complexity handling of such
techniques, in [8] authors propose integration of DL-based anomaly detection service into the 3GPP
mobile cellular IoT framework. The suggested architecture incorporates autoencoder-based anomaly
detection modules at both IoT devices ADM-EDGE and mobile core networks ADM-FOG, which
balance system responsiveness and accuracy.

3.2. Detection ML solutions:

The detection concepts can be i) an extension of the previous concepts in terms of anomaly detection,
some ML algorithms offers classification of the detected anomalies. Or additional technique should be
employed to classify the anomalies, like in [6], where the framework uses an additional supervised
machine learning system to classify the detected fault. And it can be ii) a standalone technique which
address directly a specific risk.

Jamming attacks detection:

Jamming attacks aim to disrupt a wireless network, resulting in an unwanted denial of service DoS.
Several forms of jamming detection algorithms have been presented in the recent decade. The majority
of these strategies are ineffective in detecting clever jammers. As a result, effective and rapid jamming
detection systems with high accuracy based on machine learning are in high demand.

Conventional approaches manually set thresholds for the selected metrics based on empirical
observations. However, during normal operation other effects such as network congestion and
challenging wireless link conditions can exhibit a similar impact as jamming, which degrades the
detection accuracy. Furthermore, adding more metrics, which theoretically increases the accuracy,
complicates the problem of the manual threshold setting [9]. Authors in [10] address this limitation, a
machine learning (random forest)-based jamming detection technique for 802.11 networks that
weighs and combines a large collection of parameters and automatically determines optimal
thresholds, avoiding the time-consuming and error-prone human tuning process. The approach
achieves remarkably high detection rates in indoor and mobile outdoor scenarios even under
challenging link conditions.

Another example, the topology's continuous and rapid modifications, as well as its great mobility of
the communication nodes that characterize a vehicular network all contribute to make detection more
difficult. Furthermore, the successful identification of a jamming attempt may be hampered by a
variety of urban-environmental circumstances, such as interference from neighboring wireless nodes,
bad network conditions, and so on. They can all result in false-positive detection or detection failure.
The existence of a variety of jammers may aggravate the problem even further. Authors in [11] offer a
new measure to be employed as an additional feature in unsupervised learning alongside existing
metrics received from on-board communication equipment in order to make the identification of
probable RF jamming attacks more robust and efficient. The suggested metric, relative speed
fluctuations RSV, is derived from variations in the relative speed of the jammer and target vehicles
and is employed as an additional feature in the unsupervised technique of clustering (k-means
algorithm), along with other cross-layer metrics. The fundamental objective for establishing and
employing the RSV measure is to evaluate whether jamming is generated by an intentional and
malevolent jammer or by an unintended source. This difference, however, is difficult to accomplish
using just previously published measurements such as the signal to noise and interference ratio SINR,
packet delivery ratio PDR, and received signal strength and interference ratio RSSI. This distinction is
critical, especially in a complex urban context, since it allows to approach the problem more
effectively.
The accuracy of the previous techniques can be insufficient in high dimensional data, and the learning
speed can increase dramatically, in order to capture non-linear relations in the data and increase both
learning speed and accuracy, DL-based jamming detection approaches are used. For example, in [12]
the research provides a framework for detecting jamming in drone networks based on a distributed
method based on supervised machine learning techniques, including multi-layer perceptrons MLP
and Decision Trees. The approach computes the characteristics of various specified metrics, such as
throughput, PDR, and RSSI, which fluctuate during a jamming attack and may thus be utilized to
identify it. The results indicate that the MLP may be effectively extended to outperform decision
trees in jamming detection even when deployed to communication scenarios for which it has not
been specifically trained.

Spoofing attacks detection (Physical layer authentication PLA):

Here comes the physical layer authentication concept. PLA detect spoofing attacks in wireless
communication by utilizing physical-layer aspects of wireless channels such as received signal
strength RSS and channel impulse response CIR. They serve as a supplement and improvement to
existing cryptographic systems. In contrast to standard cryptography-based authentication, physical
layer authentication can fill the gap in the physical layer's security mechanism. Furthermore, because
the secure fundamental is based on physical features, physical layer authentication does not require
key distribution or maintenance, making it more appropriate for heterogeneous networks, dense
networks, low-complexity IoT devices, and other new technologies.

Conventional physical layer spoofing detection (or PLA) systems are based on common wireless
properties and are limited to classic sub-6GHz wireless communication. These systems may be
divided into two types based on the wireless properties used, i) RF/hardware-based spoofing detection
when various wireless transceivers transmit RF signals with distinct features/patterns in the analog
(unique RF signal emission patterns are observed during the transition of a transmitter from the off
state to the on state.) and modulation (I/Q offset , frequency error, phase and magnitude errors, power
spectral density PSD, and local oscillator offset) domains, in order to distinguish the small distinctions
between the analog and modulation domains’ features, a high-end signal analyzer is usually necessary
to extract these characteristics with high accuracy, hence a greater configuration cost or overhead
issues are present. ii) Channel/location-based spoofing detection, when channel state information CSI
and RSS are site specifics related to path loss and channel fading. Devices in various areas display
distinct channel/location-based feature profiles. RSS, channel frequency response CFR, CIR, and so
on are suggested features in the detection of channel/location-based spoofing. These strategies are
ineffective for distinguishing collocated devices, as the attackers may replace or be extremely close to
the victim on the network [13].

Moreover, to identify spoofer Eve in wireless networks, most conventional PLA systems employ
hypothesis testing to match radio channel information with Alice's channel record. However, the
hypothesis test threshold is not always accessible, especially in dynamic networks [14].

Physical authentication systems based on machine learning provide cost-effective, more reliable,
model-free, continuous and situation-aware device validation under unknown network settings and
unexpected dynamics [15].

For a cross technology communication CTC (which is a heterogeneous communication between

wireless devices) scenario in [16], WiFi devices may conduct spoofing attacks directly against ZigBee
devices. The authors propose a machine learning-based method for detecting spoofing attacks in
heterogeneous wireless networks using physical-layer information. They model the RSS data of
legitimate ZigBee devices to build a one-class support vector machine OSVM classifier for
detecting CTC spoofing attacks.

In [17], authors proposed a PHY-layer spoofing detector based on Q-learning and formulated a PHY-
layer authentication game. In this work, they extend the study in [18] by proposing a Dyna-Q based
spoofing detector to improve the authentication speed.

While in [19] research offers a new channel-based spoofing attack detection technique in millimeter
wave mmWave massive MIMO 5G networks based on channel virtual (beamspace) representation.
Authors propose a spoofing detection for dynamic radio environment which might be used PLA, the
framework uses a deep learning technique based on FFN, and the results show that the proposed
model has 99% accuracy.

Eavesdropping detection:

Generally, eavesdroppers work in two ways. In an active attack, the eavesdropper pretends to be a
legitimate user and misleads the base station into sending signals toward it during the channel
estimation procedure. Or passive attack which is even more challenging to detect because the passive
eavesdropper can hide itself, as a result researches focus on active eavesdropping detection and
deployed robust techniques against passive eavesdropping as will be discussed in sec 3.3.

The research [20] describes a system for transforming wireless signals into structured datasets that can
be fed into machine learning algorithms for active eavesdropping detection at the physical layer. A
wireless communication system comprised of an access point AP, K legitimate users, and an active
eavesdropper is especially studied. To detect an eavesdropper who enters the system during the
authentication process, the authors create structured datasets based on various features and then use
advanced support vector machine classifiers to those structured datasets. Numerical results show
that careful parameter-tuning is required for exceeding an eavesdropper detection probability of 95%.

In [21] k-means clustering analysis technique is used to suggest an active eavesdropping user
detection approach. This technique does not need the creation of pilot sequences or the estimation of
legitimate user channel statistics. The needed clustering information is collected by generating a
sequence with only legitimate user information and then detecting eavesdropping on the base station's
received signals. The simulation results reveal that, when compared to the standard eavesdropping
detection strategy, the machine learning-based method suggested in this study performs significantly
better.

3.3. Robust ML models Solutions:

The previous section discussed some example for detection of physical layer attacks, however, as new
radio communications such as 6G enable automated security concept, the need of robust models
against these threats is necessary. This section will discuss the ML-based mitigation techniques for
robust models to ensure the security automation against eavesdropping and jamming.

Techniques for robust ML-models against jamming:

Because Alice's messages readily become unidentifiable when overlapped with the jamming signal,
jamming attacks can severely impair communication between Alice and Bob. Robust ML models use
adaptive intelligent frequency hopping, power-based, hybrid power-frequency and other techniques.
 Intelligent frequency hopping:

Spread spectrum approaches have traditionally been used to resist jammers. Frequency hopping FH
methods attempt to avoid jammers by switching the transmission frequency on a regular basis. To
minimize the effect of a jamming signal, direct sequence spread spectrum technologies distribute
transmission over a greater bandwidth. Many studies are working on developing ML-based effective
and adaptive frequency hopping schemes to be used for robust based-ML systems; the major part of
the literature works use reinforcement learning and/or deep learning, Table. 3 shows some examples.

Scenario ML Motivation Description

technique
Cognitive radar Q-learning Random frequency Overcome the smart jammer, in which the radar does not
DQ- hopping is not the best know the specific jamming model, after using proposed
Network option since ML-based models the radar was able to learn the jammer's tactics by
techniques give the contact with the environment and take the appropriate
communication systems the action to gain a high reward by using the reinforcement
ability to learn the jammer learning algorithm. DQN outperforms Q-L in terms of
strategy unique frequency learning performance, especially when available
hopping strategy. frequencies are high [22].
Complicated DQN The adaptability of the Proposed method with priority experience replay PER
electromagnetic frequency hopping based on Pareto samples PPER-DQN is suggested to
environments algorithms is important improve the anti-jamming performance of frequency
because there is always a hopping systems. This algorithm makes intelligent
possibility to deal with judgments for bivariate FH patterns. The FH pattern's
intelligent jammers. major parameters are used to build the system model,
state-action space, and reward function. The DQN pattern
is used to increase the adaptability of the FH pattern [23].

Primary PU and DQN SU may interfere with PUs Determines whether to recommend that the SU leave an
secondary users when they (SUs) exploits area of heavy jamming and chooses a frequency hopping
SU in CR both spread spectrum and pattern to defeat smart jammers. Without knowing the
user mobility to address jamming model or the radio channel model, the SU
jamming attacks. constructs an optimum anti-jamming communication
strategy in a given dynamic game utilizing Q-learning
and a deep CNN to accelerate learning speed with a high
number of frequency channels. When compared to a Q-
learning-only benchmark system, the suggested approach
improves the signal-to-interference-plus-noise ratio and
the usability of the SU against cooperative jamming [24].
flying ad-hoc federated solve the problem of Authors developed a DQN mechanism with an
network DQN periodic frequency exploration-exploitation epsilon-greedy policy, directed
jamming. by a federated learning mechanism to obtain a frequency
hopping strategy. Results show that the proposed
algorithm has better convergence and decision accuracy
performance compared with the DQN based frequency
hopping strategy. And the performance will improve when
the number of UAVs increases [25].
Table. 3. ML-based Frequency hopping techniques for jamming attacks defense.
 Hiding technique:

The anti-jamming algorithms which solely analyze how to evade jamming attacks and overlook the
fact that jammers may gain the transmission waveform or frequency action. Although present anti-
jamming technologies can ensure short communication effects, their long-term effectiveness may
suffer when intelligent jammers are capable of learning from previous communication operations. For
this purpose [26] a hidden anti-jamming strategy is proposed, which is based on the principle of
reducing the jammer's sensing probability. To begin, the jammer's detecting probability is calculated
by measuring the correlation between the jammer's and the user's behaviors. Later, a deep
reinforcement learning framework is built with the goal of not only increasing communication
throughput but also reducing the connection between the jammer and the user's actions. Finally, a
secret anti-jamming method is provided that correlates instantaneous return with user communication
quality and the correlation between users and jammer. The results demonstrate that the suggested
approach not only prevents jamming detection but also enhances its anti-jamming performance when
compared to the algorithms which solely emphasizes jamming avoidance.

Power techniques:

ML Tech Motivation Description

CNN Jamming Authors propose a framework to cancel the jamming effect, where first the jamming is
detection and detected after that the framework works on jamming interference cancellation. They
jammer’s demonstrate that the receiving node equipped with the proposed approach can detect a
interference jammer with over 99% of accuracy and achieve a Bit Error Rate BER as low as 10−6 even
cancellation. when the jammer power is nearly two orders of magnitude (18 dB) higher than the
legitimate signal, and without requiring modifications to the link modulation. In non-
adversarial settings, the approach can have other advantages such as detecting and
mitigating collisions [27].
DL-DNN Increase signal [28] proposes a novel continuous phase modulation CPM receiver model that uses DL
recovery and approaches. In the receiver model, the DL is implemented using a hybrid deep neural
synchronization network. The results demonstrate that the proposed receiver with DL boosting is
performance resilient in the face of tone jamming, which is the worst-case situation for a CPM
under severe receiver. When compared to a receiver without DL, the model obtains a 3 - 5dB
jamming improvement in BER under single-tone jamming and a 2dB improvement under multi-
situations. tone jamming.
QL Power In a NOMA scenario which is sensitive to interference, power allocation of two
DQL allocation for independent base stations BSs versus a clever jammer is represented as a sequential game
anti-jamming in [29]. In this game, each BS decides its power allocation plan individually at initially.
approach.
The smart jammer, as the follower, chooses its ideal approach depending on the BSs'
plans. This game's solutions are generated under various conditions. Three new anti-
jamming NOMA power allocation schemes are proposed in a two-cell scenario: a) Q-
Learning based Unselfish QLU NOMA power allocation scheme, b) Deep Q-Learning
based Unselfish DQLU NOMA power allocation scheme, and c) Hot Booting Deep Q-
Learning based Unselfish HBDQLU NOMA power allocation scheme. The BSs in these
approaches do not communicate with each other. However, the research suggests that the
offered strategies would converge to the best strategy from the perspective of the entire
network with a high likelihood. The results demonstrate that the suggested methods are
convergent and outperform the Q-Learning-based Selfish QLS NOMA power allocation
technique.
Table. 4. Based-power-ML technique for jamming mitigation.
Hybrid power and frequency technique:

Existing works exclusively examine anti-jamming strategies in the frequency or power domains.
With the advancement of communication devices, an increasing number of communication devices
can alter communication frequency while also adjusting power.
In [30] the anti-jamming problem addressed in both the frequency and power domains. Authors
formulate the anti-jamming issue as a Markov decision process MDP. And they apply the DRL
method to solve the problem. The results demonstrate that the suggested approach provides good
throughput while using less power.

Techniques for robust ML-models against eavesdropping:

Some examples for detecting active eavesdroppers were shown; however a passive eavesdropper may
intercept and extract information from a sent signal without being discovered once within the
communication range. As a result, security concerns have grown in wireless communication networks,
and new security algorithms are being developed to secure lawful communications from prospective
eavesdroppers.

The most prevalent security mechanisms involve cryptographic encryption techniques and often
operate at the upper layers of wireless networks. However, the performance of encryption may be
hampered by obstacles and vulnerabilities such as computational cost, secret key distribution and
maintenance, and so on. More crucially, even when encryption is provided, it is much desired to
improve the basic security of wireless communications by limiting eavesdropper interception of
propagating signals in the first place. Consequently, the physical layer techniques are widely studied,
the following table classifies these techniques.

Information theory-based Limiting the eavesdropper’s Information locking

performance
Secrecy capacity for wiretap [32], SISO fading Artificial noise [31], PHY encryption [43]
[33], Gaussian broadcast [34], Gaussian beamforming [41] and and directional
multiple access [35], MIMO [36], MISO [37], secret transmit [42], in modulation [44].
and SIMO [38] channels. LDPC for wiretap condition of insuring QoS
channel [39]. Cooperative relying networks or SINR for the legitimate
[40]. receivers.
Table. 5. Conventional PLS security measure for eavesdropping defense

To obtain higher secrecy, lower cost and efficient algorithms machine learning is inevitable, the
following lines gives some examples of ML-based techniques for each approach.

ML-Based Information theoretic solutions:

The simplest scenario which involves both tasks of reliable transmission and secrecy is the wiretap
channel, which is a three-node network consists of a transmitter, a legitimate receiver, and an
eavesdropper [45].

The purpose of a wiretap channel system is to minimize information leakage to an eavesdropper while
increasing transmission performance to the desired or legitimate recipient. Complicated systems or
channel models make designing secrecy systems based on information theory challenging. ML
techniques may intervene to address this issue; the following table lists some examples.
ML Motivation Description
Technique
AE Keep the In [46] a framework which can fool the eavesdropper to miss-distinguish between
eavesdropper symbols was proposed. In this strategy, neural networks trained to optimize
away from encoding and decoding functions simultaneously in order to achieve reliable message
correct reading.
transmission. The transmission is kept secret by employing a modified secure loss
(Gaussian
wiretap function based on cross-entropy, which may be implemented using state-of-art
channel). machine-learning libraries. This secure loss function technique is used in a Gaussian
wiretap channel setup. By clustering learnt constellations, the neural network learns a
trade-off between reliable communication and information secrecy.
DNN Unavailable Authors in [45] offer a neural secure communications model based on dual mutual
knowledge information neural estimation MINE. This method's security constraints are built just
about using the input and output signal samples of the legitimate and eavesdropper
eavesdropper's
channels, with the added bonus that training the encoder is fully independent of the
decoder or its
output. decoder. Furthermore, because secure coding is not designed to rely on the
(Gaussian eavesdropper's decoding outcomes, the security performance is unaffected by the
wiretap eavesdropper's decoding techniques. Results show that whether the eavesdropper
channel). learns from the decoder himself or uses the legal decoder, the performance of the
approach is assured.
DNN Unavailable For efficient, reliable, and secure information transmission, a proposed DL-based
knowledge precoding learns the input covariance matrix by offline training over a wide
about the collection of input channels and their related covariance matrices. Furthermore, by
number of
devoting time to offline training, this strategy significantly decreases compute
antennas at the
eavesdropper. complexity in real-time applications. The suggested DL-based precoding approach is
(MIMO substantially quicker than previous precoding methods and achieves near-capacity
Gaussian secrecy rates. In terms of the number of antennas at the eavesdropper, DL-based
wiretap precoding is likewise more resilient than transitional precoding techniques.
channel). This novel precoding method shows promise in situations where latency and
complexity are crucial [47].
DL Optimize The model can adapt to the legitimate user's current channel and withstand
security rate eavesdropping from eavesdroppers. The model considers the statistical
with power characteristics of the channel and may learn the CSI to efficiently build secure
allocation for
beamforming to prevent eavesdropping. When the receiver demodulates the received
cooperative
relay. signals, the performance of BER approaches the Shannon limit, and also assure that
eavesdroppers cannot demodulate the transmitted signals. In addition, authors
develop a power allocation mechanism that takes into account the CSI of the
eavesdropper on the relay. The results show that the proposed model can meet the
pre-specified BER requirements. By optimizing power allocation at the relay, the
model may optimize system security rate [48].
Table. 6. Wiretap Channel Security using ML.

Limiting the eavesdropper’s performance:

Beamforming is a signal processing technique used to transmit signals effectively in intended

directions in order to achieve the maximum signal difference between the intended and unintended
receivers. Beamforming is an important approach in PLS that has received a lot of attention in the
literature. In PLS, a beamforming problem involves steering the transmitted signal towards the target
user while accounting for an interfering user attempting to decode the transmitted information [49].
The design of a security scheme for beamforming prediction is critical for next-generation wireless
networks, the beamforming can be described by an optimization problem, and as ML is a powerful
tool for optimization, hence it becomes a suitable solution to find optimal beamforming systems.

Authors in [50] propose an upgraded secure technique for a wireless communication system under
danger from an intelligent attacker capable of smart eavesdropping. The traditional secure technique
employs a Q-learning-based algorithm to achieve a Nash equilibrium NE in the context of a zero-sum
game between the transmitter and attacker, which, however, necessitates a far higher number of
antennas at the transmitter than at the attacker. To address this issue, the framework first considers a
situation in which the attacker can increase the number of antennas flexibly in order to enhance the
attack rate. Then it uses beamforming at the transmitter by adaptively adjusting the number of
antennas at the transmitter and the legitimate receiver to the number of antennas at the attacker to
suppress the eavesdropping.

Artificial noise AN: The concept of employing generated noise to improve physical layer security was
initially described in [51]. They identified AN-based transmission as a viable strategy that may be
used in PLS to ensure secure wireless communication. The approach entails purposely decreasing the
quality of the eavesdropper's channel by creating an interference signal that is used to disrupt their
eavesdropping capabilities. Generating AN depends on the transmitter’s knowledge of the
eavesdroppers’ channel state information. In a case where the eavesdropper’s CSI is unknown, the
isotropic AN is generated.

Machine learning techniques can be deployed to tackle some AN transmissions problems. For example
in [52] authors consider a secure precoding optimization problem for the AN scheme in MISO wiretap
channels, previous research has shown that the generalized AN scheme, which permits some of the
AN signal to be injected into the legitimate receiver's channel, is the best precoding method for MISO
wiretap channels. The optimality, however, is only valid under certain ideal assumptions, such as
flawless channel estimation and spatially uncorrelated channels. To overcome this restriction, the
framework offers the deep AN scheme, a revolutionary DNN-based safe precoding approach. The
deep AN method, is a secure precoding technique that uses a DNN to simultaneously design and
optimize the precoders for the information signal and the AN signal. The results show that the
proposed deep AN scheme outperforms the generalized AN scheme in a variety of real-world wireless
situations.

Covert communication: which aims to hide the existence of a communication channel from an
observing eavesdropper. Besides encryption, hiding signal transmission deeply under noise
background highly proliferates the covertness in the physical layer. Covert communication hides the
transmission of a message from an adversary while ensuring reliable information decoding at the
receiver, providing enhanced security in wireless communications [53].

For example, federated learning networks FLNs are subject to eavesdropping attacks because to the
necessity for frequent model upgrades and communications. FLNs have a significant difficulty in
balancing privacy protection with effective distributed model training. Existing defenses have
substantial computing requirements and are exclusively intended for specific FLN attacks. The Covert
Communication-based Federated Learning CCFL strategy is proposed by the authors in [54] to
overcome this gap. Based on the emerging communication security technique of covert
communication, which conceals the existence of wireless communication activities, CCFL can reduce
attackers' ability to extract useful information from the FLN training protocol, which is a fundamental
step in most existing attacks, and thus holistically improves FLN privacy. They thoroughly test CCFL
in real-world circumstances where the FL latency is tailored to meet specific security criteria. The
results show that the suggested technique is highly successful in terms of both training efficiency and
communication security.

Cooperative jamming:

A cooperative jamming scheme is a typical physical layer-based solution to broadcast artificial noise
to block eavesdropping while not degrading the receiving performance of legitimate transceivers. The
artificial noise is actively transmitted by the transceiver or a selected jammer [55].

ML can be deployed for power allocation control for example, in [56] anti-jamming power control
problem of secondary users SUs in a large-scale cooperative cognitive radio network attacked by a
smart jammer with the capability to sense the ongoing transmission power was discussed. The
interactions between cooperative SUs and a jammer are investigated with game theory. Authors
derive the Stackelberg equilibrium of the anti-jamming power control game consisting of a source
node, a relay node and a jammer and compare it with the Nash equilibrium of the game. Power
control strategies with reinforcement learning methods such as Q-learning and WoLF-PHC are
proposed for SUs without knowing network parameters (i.e., the channel gains and transmission costs
of others and so on) to achieve the optimal powers against jamming in this cooperative anti-jamming
game.

Another research [57], discussed wireless body area networks WBAN and its security difficulties,
particularly eavesdropping attacks sue to limited resources. Authors propose DRL and mobile edge
computing MEC technologies to develop a DRL-MEC-based jamming-aided anti-eavesdropping
DMEC-JAE method to resist eavesdropping attacks without the knowledge of CSI.

Information locking:

PLS encryption: is essentially a cross-layer approach, which combines the secret key generation at
the physical layer and the encryption at the application layer [58]. Unlike traditional cryptographic
approaches, PLS uses the inherent properties of wireless channels, such as noise, fading, and
interference, to improve signal reception at the legitimate receiver while degrading received signal
quality at the eavesdropper, and achieves keyless secure transmission through signal design and signal
processing.

PLS methods provide the following benefits over cryptographic approaches: i) PLS encryption
methods do not rely on encryption/decryption processes; this alleviates the complexity of distributing
and managing secret keys in large-scale heterogeneous networks. ii) PLS encryption approaches may
completely use wireless channel characteristics to realize adaptive signal design and resource
allocation, resulting in variable security-level setups and QoS assurance. iii) PLS encryption
approaches only need the completion of relatively basic signal processing algorithms, resulting in less
overhead than the encryption-based method. Based on these considerations, it is possible to conclude
that PLS encryption is a potential approach for securing future wireless communications such as IoT.

Although PLS encryption approaches research has shown productive results, developing PLS solutions
for future applications remains difficult. Next-generation communication features include low-cost,
wide-area coverage, enormous connections, and diverse services. The question of how to build PLS
techniques that are well suited to these characteristics remain unclear.

ML techniques might be bridged to design effective PLS encryptions techniques. For example,
physical layer key generation PKG generates cryptographic keys using highly correlated wireless
channel measurements, which rely on reciprocal channel properties between uplink and downlink, and
is a viable wireless security approach for the IoT. However, it is difficult to extract common
characteristics in frequency division duplexing FDD systems because uplink and downlink
transmissions operate at distinct frequency bands with non-reciprocal channel frequency responses.
Existing PKG approaches for FDD systems have several drawbacks, including significant overhead
and security issues. In [59], authors propose a key generation KG strategy that employs the feature
mapping function produced from deep learning between different frequency bands to make two users
create substantially comparable channel characteristics in FDD systems. They first demonstrate that a
FNN with a single hidden layer may approach the band feature mapping function in a particular
environment. Then, a key generation neural Network KGNet is presented for the building of reciprocal
channel features, as well as a key generation strategy based on the KGNet. The results show that the
KGNet-based key generation method performs admirably in terms of unpredictability, key generation
ratio, and key error rate.

Directional Modulation:

Directional modulation DM, as an efficient secure transmission way, offers security through its
directive property and this is suitable for line-of-propagation LoP channels such as mmWave massive
MIMO, satellite communication, unmanned aerial vehicle UAV, and smart transportation. A natural
combination of DM with mmWave has the potential to provide a high-data-rate secure transmission
technique for future wireless networks, particularly femtocells and picocells with just a line-of-sight
LoS link. The DM transmitter transmits confidential messages CMs to the desired user in such a way
that the private messages cannot be intercepted by an eavesdropper; the DM technique uses the
combination of the beamforming and the AN. The DM transmitter transmits the CMs to the desired
direction and interferes with the eavesdroppers using the directional property created by antenna-
array-based beamforming and AN projection. The CMs may be safely and successfully conveyed to
the target destination due to the cooperative operation of AN projecting and precoding, and AN
severely corrupts the eavesdroppers [60].

For secure DM, the DM transmitter must have high-resolution direction of arrival DOA measurement
and high-precision DOA error density estimate, thus improve the security performance since expected
information and AN can be accurately transmitted to the desired direction and eavesdropping
direction, respectively. How to achieve a high-precision DOA measurement? ML is a new and
advanced method to address such a problem.

For example in [61]authors present a Bayesian learning based method to improve the DOA
measurement precision at DM transmitter with fully-digital structure, their idea can be readily
extended to the hybrid analog and digital structure. Another example, [60] proposes statistical
learning-based DOA measurement method to make a substantial secrecy rate SR performance gain
compared to single-snapshot measurement without machine learning for a given null-space projection
beamforming scheme.

ML-Based robust models in the perspective of spoofing attacks is discussed in the previous
section, PLA offers detection and thus mitigation.

4. Intelligent reflection surface and ML to enhance the security:

Employing a large number of active antennas and relays in PLS systems incurs an excessive hardware
cost and the system complexity. Moreover, cooperative jamming and transmitting AN require extra
transmit power for security guarantees, to tackle this issues IRS has recently been seen as a potential
 new technique for next-generation wireless communications security. However, the contributions are
obtained under the assumption that perfect CSI is available at the Alice, additionally they use
traditional optimization approaches which are inefficient for large-scale systems. To obtain perfect
CSI and efficient techniques, it is necessary to consider robust based-ML security performance of the
system [62], [63]. The following table shows some literature works for Based-ML IRS-aided PLS.

Technique description
The RL WoLF-PHC- In [64] by modifying the surface reflecting components at the IRS, the IRS can improve
based joint power anti-jamming communication performance and decrease jamming interference.
allocation and reflecting Due to the dynamic and uncertain nature of the jamming model and jamming behavior, a
beamforming optima- win or learn fast policy hill-climbing WoLF-PHC learning technique is presented to
zation approach for the simultaneously optimize the anti-jamming power allocation and reflecting
IRS-assisted comm- beamforming strategy without knowledge of the jamming model. When compared to
unication system against existing solutions, results show that the proposed anti-jamming-based-learning strategy can
smart jamming. efficiently enhance both the IRS-assisted system rate and transmission protection level.
DL-DNN based In the work [65], DL is applied to design the optimal reflection coefficients of the IRS
reflection coefficients elements for the proposed wiretapping scenario. The results show that the proposed model
tuning optimization for does not offer higher SR comparing to the conventional optimization methods, but it is
eavesdropping. cost-effective
Table. 7. ML for IRS-assisted communications to improve PLS

5. Conclusion
Through this chapter, threats on future radio communications: jamming, eavesdropping, and spoofing
with existed countermeasures were discussed. After that, the machine learning applications to mitigate
these threats were introduced with a comparison with the conventional defense techniques; the study
was divided into three defense philosophies: monitoring and robust systems which are pre-attack
defense approaches, and detection which is during or after the attack scenario. The eavesdropping
robust model techniques were the main part because the passive eavesdropper creates a big challenge
for physical layer security. It is shown with some examples from the literature that the ML-based
techniques offer more efficiency in terms of secrecy rate, adaptability, lower computational efforts…
The last part was devoted for the ML bridging to the intelligent reflected surface to enhance the
optimization techniques and decrease the cost of the system. However, ML techniques might be used
as an adversarial option to attack both conventional and ML-based communication systems; this will
be discussed in the next chapter.