Enhancing IoT Communication Security: Analysis and

Mitigation of Vulnerabilities in MQTT, CoAP, and

XMPP Protocols

Zahid Mohammeda , Abdullah Shahwana , Ali Alazawia , Wael Elmedanya

a College of Information Technology, University of Bahrain

Abstract

As the number of Internet of Things (IoT) devices increases, securing IoT com-
munication protocols becomes critical. Due to the resource constraints of IoT
networks, these protocols are particularly vulnerable to cyberattacks. Tradi-
tional security measures often fail to address the unique challenges posed by
IoT communication, highlighting the need for specialized solutions. This re-
search evaluates security vulnerabilities in key IoT communication protocols:
MQTT, CoAP, and XMPP by identifying their strengths and weaknesses in
handling various attack scenarios. A practical comparison is made for MQTT,
examining the impact of using Transport Layer Security (TLS) on its security,
while for XMPP, a theoretical comparison for using JSON Web Token (JWT)
authentication is conducted. Additionally, the study explores the use of JWT
in combination with a complementary nonce-based solution to enhance security
and protect against inadequately addressed attacks in CoAP. The findings offer
valuable insights that contribute to the development of more secure implemen-
tations for the three IoT communication protocols.
Keywords: Internet of Things (IoT), Security vulnerabilities, Mitigation
strategies, MQTT , CoAP , XMPP , Encryption, Authentication, Lightweight
protocols

Preprint submitted to Elsevier January 3, 2025

1. Introduction

The rapid growth of the Internet of Things (IoT) has transformed indus-
tries by improving efficiency, convenience, and the ability to collect real-time
data. IoT systems are now widely used in critical areas such as healthcare,
transportation, and home automation, providing seamless connectivity and au-
tomation. However, as the number of connected devices continues to increase,
securing these systems has become a major challenge. For instance, the 2023
Mirai Botnet attack exploited vulnerabilities in IoT devices, disrupting services
worldwide and exposing the risks of weak security in IoT systems. Therefore,
IoT devices are inherently vulnerable due to their distributed nature, limited
computing resources, and diverse communication protocols.
The growing interconnectedness of devices amplifies these risks, leading to
threats such as unauthorized access, data breaches, and service disruptions. As
IoT systems become more integral to both personal and industrial applications,
the need for stronger, more effective security measures is critical. Without these
measures, organizations and users relying on these ecosystems remain exposed
to escalating security threats.

1.1. Research Problem

Many widely used IoT communication protocols, such as MQTT, CoAP,
and XMPP, prioritize efficiency over security, resulting in vulnerabilities such
as weak encryption, poor authentication, and exposure to various attack vec-
tors. These protocols are also susceptible to issues such as request delay attacks,
selective blocking, and improper handling of fragmented messages. Addition-
ally, the lack of standardization across IoT devices complicates interoperability,
exposing systems to further risks, including data breaches, unauthorized ac-
cess, and infrastructure disruptions. The implications of these vulnerabilities
are far-reaching, resulting in financial losses, compromised privacy, and threats
to critical infrastructure. Addressing these issues through a unified and scalable
security framework is essential to ensure the reliability, safety, and privacy of
IoT systems.

2
1.2. Research Objectives

This study focuses on the security challenges in the MQTT, CoAP, and
XMPP protocols. These protocols are widely used for communication between
IoT devices but suffer from a range of vulnerabilities. The primary objective is to
evaluate the security vulnerabilities inherent in MQTT, CoAP, and XMPP, in-
cluding issues such as insufficient encryption, weak authentication mechanisms,
and susceptibility to attacks such as request delays and selective blocking. It
also aims to analyze the trade-offs between security and performance in MQTT,
highlighting the impact of TLS encryption on latency and proposing lightweight
encryption at the application layer to reduce communication overhead. Further-
more, the study introduces a JWT-based authentication solution for CoAP, de-
signed to mitigate request delay attacks while enhancing device authentication,
scalability, and efficiency across IoT networks.

1.3. Research Significance

The existing research on IoT security has made considerable progress; how-
ever, many solutions remain fragmented and fail to comprehensively address
the vulnerabilities across MQTT, CoAP, and XMPP protocols. This study
contributes to ongoing efforts to enhance IoT security by providing a detailed
evaluation of vulnerabilities and mitigation strategies for MQTT, CoAP, and
XMPP. Practical insights into the trade-offs between security and performance
in IoT protocols are also offered, with a specific focus on MQTT latency analysis.
Additionally, a scalable and lightweight solution using JWT-based authentica-
tion for CoAP is proposed, ensuring improved security without compromising
performance. The findings of this study have far-reaching implications for im-
proving the reliability, safety, and privacy of IoT systems in both personal and
industrial applications.

1.4. Paper Structure

The rest of this paper is organized as follows. Section 2 presents a detailed

literature review, including an analysis of the vulnerabilities and mitigation

3
strategies for MQTT, CoAP, and XMPP. Section 3 synthesizes the findings,
identifies key gaps in existing research, and outlines the challenges in achieving
comprehensive IoT security. Section 4 discusses the methodology, including the
experimental setup and analysis techniques employed in the study. Section 5
reports the results and examines their implications for IoT security. Finally,
Section 6 concludes the paper by summarizing the findings and proposing di-
rections for future research.

2. Literature Review

The literature review examines existing research on the security vulnera-

bilities and mitigation strategies of various communication protocols, including
MQTT, CoAP, and XMPP, particularly in the context of IoT systems. It high-
lights the common security challenges faced by these protocols, such as delay
request attacks, path traversal vulnerabilities, and potential weaknesses in au-
thentication and encryption mechanisms. Additionally, the review explores pro-
posed solutions and mitigation strategies, such as the use of TLS and DTLS for
encryption, improved authentication protocols, and anomaly detection systems.

2.1. MQTT (Message Queuing Telemetry Transport)

According to [1], MQTT is a machine-to-machine (M2M) communication

protocol introduced in 1999. It is a lightweight publish-subscribe messaging
protocol designed for constrained networks with low bandwidth and high la-
tency. This protocol operates on a publish-subscribe model, allowing devices
(clients) to send messages to a broker (server), which subsequently distributes
these messages to other clients [2]. Each message is published to a specific ad-
dress known as a topic that contains certain values or commands [3]. Clients can
subscribe to multiple topics, allowing them to receive every message published
to each topic. MQTT operates as a binary protocol with a fixed 2-byte header
and message payloads that can reach a maximum size of 256 MB [4]. Figure 1
shows a simple MQTT network architecture. In this architecture, the publisher

4
sends data to a broker, which then filters and forwards the data to subscribers.
Subscribers receive data only for the topics they have subscribed to [5].

Figure 1: Basic MQTT architecture showing data flow from the publisher to
subscribers via a broker [27].

2.1.1. Security Vulnerabilities in MQTT

The MQTT protocol is equipped with several security features; however,
data encryption and entity authentication are not configured by default [4]. Ac-
cording to [6], this leads to a significant vulnerability, as many MQTT brokers
do not require passwords by default, allowing unauthorized individuals to ac-
cess and view published data. Due to these vulnerable default configurations,
the protocol can be easily exploited by attackers. Any device, including an at-
tacker’s device, can publish or subscribe to data from the broker, which enables
denial-of-service attacks to be performed on the MQTT broker [7]. Through
these vulnerabilities, communication can be disrupted, sensitive information
can be monitored, or control of devices within IoT networks can be controlled
by attackers.
Additionally, the lack of encryption leaves data transmitted over MQTT
vulnerable to interception, which raises confidentiality concerns. As highlighted
by [7], unencrypted data can be easily accessed by attackers with basic net-

5
working skills, regardless of whether authentication mechanisms are used by
the broker. To secure the entire MQTT communication channel, TLS or SSL
certificates should be used by MQTT brokers [8]. However, developing such
security mechanisms is difficult for resource-constrained devices. For instance,
while communications may be secured using the TLS protocol, the overhead it
generates is considered too much for resource-constrained devices [9].
Moreover, the lack of default authentication is another security concern in
MQTT, as the protocol was designed to be lightweight and simple for low band-
width and high latency environments. This design minimizes the overhead of
transmitted data; however, it also indicates that MQTT was not intended to
support complex security mechanisms by default, as implementing such mea-
sures would increase overhead and complexity within the protocol. While au-
thentication can be performed using the connection package, sending credentials
in plain text form remains a vulnerability. If network traffic is intercepted, sensi-
tive information can be easily accessed, leaving the system vulnerable to further
attacks [5].
Figure 2 shows an access control vulnerability, where an attacker can sub-
scribe to all topics and receive every message being published without being
authenticated.

6
 Figure 2: Access control vulnerability enables an attacker to subscribe to all
topics and access published messages without authentication [10].

2.1.2. Mitigation Strategies for MQTT Security

To address vulnerabilities within the MQTT protocol, various security mea-
sures have been proposed. These measures aim to enhance communication se-
curity, improve authentication, and prevent unauthorized access.
One effective approach recommended by IBM involves using client certifi-
cates to verify device identities before connecting to the MQTT broker. This
verification ensures that only trusted devices can access the network. However,
the implementation of a public key infrastructure (PKI) is required, which may
complicate the setup for smaller IoT environments.
Additionally, the use of MQTTS, a secure version of MQTT that utilizes
TLS encryption, provides another layer of protection. According to [6], this
version authenticates both servers and clients through certificate exchanges,
thereby preventing unauthorized access and enhancing data security within IoT
systems.
In conjunction with MQTTS, a secure version of MQTT (SMQTT) is pre-

7
sented by [11]. This version introduces a new control packet called Spublish
for publishing encrypted messages. This method employs Cipher-text Policy
and Key Policy Attribute-Based Encryption (ABE), enhanced by lightweight
Elliptic Curve Cryptography, making it suitable for IoT applications.
Furthermore, a modified OAuth model was proposed by [12] to create an ar-
chitecture tailored for MQTT. This model assigns unique Application IDs and
shared virtual spaces to applications while allowing devices to utilize Device
IDs and Device Secrets for access. Users are authenticated using their Applica-
tion ID and Secret, while devices obtain permission from the application owner
through an Access Token.
As a further enhancement, an open-source publish/subscribe system devel-
oped by [13] extends MQTT’s capabilities by integrating a key management
framework and policy enforcement. This system regulates the flow of informa-
tion in MQTT, ensuring secure messaging through effective policy enforcement.

2.2. CoAP (Constrained Application Protocol)

Constrained Application Protocol (CoAP) is an application-layer protocol

developed and introduced by the IETF CoRE Working Group in 2014 for use in
resource-constrained devices and environments, typically under an IoT context
[]. CoAP follows a RESTful architecture, treating networked objects as re-
sources, each identified by a unique Universal Resource Identifier (URI). These
URIs enable CoAP to interact with resources on the network efficiently. De-
signed to operate over low-power networks, CoAP excels at minimizing message
overhead and reducing the need for fragmentation during transmission [14].
One of CoAP’s strengths lies in its compatibility with existing web protocols.
CoAP can communicate with HTTP using proxy components, allowing HTTP
clients to interact with CoAP servers and vice versa. This feature enables better
integration with web-based systems while maintaining the efficiency required for
resource-constrained environments [2].
CoAP is compared a lot to HTTP due to their similarities. such as HTTP,
CoAP operates on a client-server model, allowing clients to interact with re-

8
sources using methods such as GET, POST, PUT, and DELETE [14]. How-
ever, unlike HTTP, which relies on TCP, CoAP is built to operate over UDP
by default, offering lower overhead and making it more suitable for constrained
networks because UDP allows for faster transmission of data with reduced la-
tency.

2.2.1. Security Vulnerabilities in CoAP

As outlined in the CoAP RFC protocol specifications, ”CoAP itself does
not provide protocol primitives for authentication or authorization; where this
is required, it can either be provided by communication security (i.e., IPsec or
DTLS) or by object security (within the payload)” [15]. To address this lack
of native security features in UDP, CoAP typically uses Datagram Transport
Layer Security (DTLS) [2]. DTLS provides encryption, integrity, and authen-
tication, ensuring that data exchanged between devices remains secure in IoT
environments. However, DTLS introduces certain logical security holes that can
enable various types of attacks [16].
It is essential to distinguish between the types of attacks being discussed in
the upcoming sections. Sections 3.2.1 and 3.2.2 focus on attacks that arise from
inadequate security in data transmission, particularly highlighting how man-in-
the-middle attackers can exploit these weaknesses to disrupt and manipulate
communication to perform unauthorized actions. These attacks, namely the
selective blocking attack and the request delay attack, emphasize the risks as-
sociated with CoAP’s reliance on protocols such as DTLS for securing messages
during transmission. In contrast, section 3.2.3 shifts the focus to vulnerabilities
inherent in the CoAP protocol’s design, specifically the necessity for additional
security measures such as proper input sanitization to prevent issues such as
path traversal. As seen from the literature, this distinction in the nature of
the security vulnerabilities in the CoAP protocol should be carefully considered
as it includes both transmission-related risks that can lead to loss of integrity
and availability and design-related flaws that could violate the confidentiality
of data.

9
A. Selective Blocking Attack. The selective blocking attack involves a man-in-
the-middle attacker who can block the delivery of specifically chosen requests
or responses while allowing others to pass through [16]. This vulnerability is
particularly critical for actuator deployments, where the implications of blocked
communication can have serious consequences.
The encryption in DTLS complicates the selective blocking of messages; how-
ever, it does not make it impossible. Proxies, when using DTLS and TLS, can
access the entire CoAP message, which means that man-in-the-middle attackers
have access to the IP addresses, ports, and message lengths. This information
can be sufficient to infer the server, resource, and command. Figure 3 outlines
the sequence of a typical selective blocking attack where the attacker intercepts
and drops a client’s message intended for the server.

Figure 3: Example sequence of a request blocking attack. The ’X’ symbol

represents the attacker intercepting and blocking the delivery of the message
with the payload “Lock” to the server[16].

The impact of this attack is significant. As illustrated in Figure 3, if an

actuator such as a door lock is targeted, the server may remain unaware of the
client’s request to lock the door because the attacker is blocking the client’s
requests. In such scenarios, the client may confuse the attack with connectivity
issues or server downtime.

B. Request Delay Attack. The request delay attack enables a man-in-the-middle

attacker to not just block packets but also introduce delays in the delivery of
specific packets, whether requests or responses [16]. This vulnerability becomes
critical when CoAP is used over unreliable and unordered transports such as

10
UDP with DTLS, where out-of-order delivery is permitted. The sequence of a
request delay attack is shown in Figure 4.

Figure 4: Example sequence of a delay request attack. The ’@’ symbol

represents a scenario where the attacker intercepts and stores the message
temporarily, then forwards it at a later time [16].

Figure 4 highlights a scenario where the attacker intercepts and stores the
client’s message to unlock a door, then forwards it at a later time after the client
and server have stopped communication. If a request to unlock a door is delayed
and later fulfilled, the door may unexpectedly unlock after the client has already
left the facility, resulting in unauthorized access. This occurs when the attacker
releases the delayed request to unlock the door from the replay window, as shown
by the second ”@” in Figure 4. This is because the default replay windows in
DTLS can be manipulated, allowing attackers to replay delayed requests of their
choice [16].

C. Path Traversal Vulnerability. Another critical vulnerability in CoAP is path

traversal, where certain implementations improperly parse or ignore URI compo-
nents such as double dots (“..”). This vulnerability allows an attacker to exploit
the URI parsing rules, potentially breaking out of the CoAP root and accessing
or manipulating unintended files on the network server via path traversal [17].

11
According to the CoAP standard:

1. The Uri-Path and Uri-Query Options can contain any character sequence,
with no percent-encoding performed.
2. The value of a Uri-Path Option MUST NOT be “.” or “..”.

This standard leaves implementations vulnerable if they do not correctly handle

such URI values. For example, an attacker could exploit a vulnerable CoAP
server by sending the following request:
GET coap://coap server/foo/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd
If the server fails to adequately restrict access based on the URI, it may
inadvertently provide access to sensitive files, and thus compromise the system’s
security [17].

2.2.2. Mitigation Strategies for CoAP Security

MQTT (Message Queuing Telemetry Transport) and CoAP (Constrained
Application Protocol) are two popular messaging protocols designed specifically
for constrained environments in the Internet of Things (IoT). MQTT, designed
for lightweight communication, is a publish-subscribe messaging protocol that
allows clients to subscribe to a broker, reducing direct communication overhead.
This makes it suitable for applications that require minimal bandwidth and low
power consumption, as is common in IoT. MQTT operates over TCP and is
typically used in situations where data needs to be transferred efficiently and
reliably.
On the other hand, CoAP is a RESTful protocol designed for simple devices
and networks. It operates over UDP and is often seen as a more efficient choice in
environments where low latency and reduced overhead are crucial, but reliability
can be sacrificed. CoAP’s ability to handle multicast and support for block-
wise transfers make it especially well-suited for IoT applications where resource-
constrained devices communicate over lossy networks.
Despite these advantages, both MQTT and CoAP have security vulnerabil-
ities that can be exploited if not properly secured. Common attacks include

12
message interception, data manipulation, and denial-of-service (DoS) attacks,
among others. Several mitigation techniques are recommended, including the
use of Transport Layer Security (TLS) for MQTT to provide encryption and
authentication, and Datagram Transport Layer Security (DTLS) for CoAP to
secure the communication channel. Additionally, securing the devices, employ-
ing proper authentication, and using anomaly detection systems are essential to
prevent potential breaches in IoT systems.
In the realm of cybersecurity for IoT, various researchers have explored how
these protocols can be fortified against attacks. For instance, various studies
have proposed solutions such as integrating machine learning techniques for
detecting anomalies in MQTT-based systems [18]. Meanwhile, others have fo-
cused on improving the security of the protocols themselves through the use of
stronger encryption standards and better message integrity checks. The research
indicates that, while both MQTT and CoAP are well-suited for IoT communi-
cation, they must be used with caution and proper security measures to avoid
compromising the integrity and privacy of IoT systems.

2.3. XMPP (Extensible Messaging and Presence Protocol)

The Extensible Messaging and Presence Protocol (XMPP) is an open XML-

based technology designed for real-time communication and is widely used in
various applications, including instant messaging, presence, and collaboration
[19] [20]. XMPP’s application-layer protocol [21] enables seamless communi-
cation between clients and servers, supporting features such as voice and video
calls, file transfers, and cross-platform collaboration. Moreover, it can be used as
a massage interaction between two points, enabling status checking [21]. How-
ever, [22] believe that IoT implementations using XMPP are typically deployed
in a decentralized client-server architecture, where communication follows the
client-server and server-server streams (see Fig. 5). As a result, two clients
cannot communicate directly without an intermediary, such as a server with a
certain level of trust.
Within all different features, this made XMPP protocol able to meet the

13
 Figure 5: XMPP protocol able different types of devices to communicate
together.

growing demand for short messaging services in an increasingly connected in-

formation society [21]. Its foundation in XML allows for smooth integration be-
tween different messaging systems, both instant and non-instant. Many promi-
nent platforms, such as WhatsApp, utilize XMPP for backend communication
[23]. In more technical details, XMPP uses a unique identifier for each device
[24], they are usually called Jabber IDs (JID), and they contain three parts (see
Fig. 6).

1. Local-Part: the username

2. Domain-Part: it is mandatory, contains the resolvable DNS name of the
entity [21]
3. Resource

Figure 6: . Example of Jabber ID of XMPP.

14
 Moreover, The protocol works by exchanging messages through XML stan-
zas (see Fig. 7), which consist of three main components: message, presence,
and info/query. This structure allows for real-time data retrieval using a push
method

Figure 7: XMPP Stanza Structure.

Security Vulnerabilities in XMPP The Extensible Messaging and Presence

Protocol (XMPP) attempted to enhance its security by incorporating Simple
Authentication and Security Layer (SASL) and Transport Layer Security (TLS)
protocols. Despite these improvements, XMPP continues to face notable se-
curity challenges. One such issue stems from SASL, which supports various
authentication methods, some of which are inherently weak. As highlighted in
[25], it is possible for a client to select a weak mechanism, such as using Base64
encoding to obfuscate confidential information. This approach fails to provide
adequate confidentiality and leaves sensitive information vulnerable to exposure.
Another security vulnerability arises from the STARTTLS extension of TLS.
While TLS requires a complete handshake before initiating communication, re-
search in [25] demonstrates that XMPP is susceptible to crafted message attacks
during a session. Attackers can exploit XMPP’s ”shotgun parser,” altering key
elements such as the “to” or “from” addresses exchanged in the initial stream
headers. Additionally, because TLS is built on TCP, message fragmentation
is possible. If XMPP messages are fragmented, there is a high likelihood that
some parts of the communication streams may not be fully protected by TLS
due to incomplete or improper encryption across fragmented streams [25].

15
 Moreover, as noted in [26], XMPP implementations have frequently encoun-
tered issues such as inadequate control over memory operations, improper cer-
tificate validation, and the presence of hard-coded accounts. These vulnerabil-
ities are well-documented in cases such as CVE-2019-1845, CVE-2019-12855,
CVE-2014-3451, CVE-2018-15720, and CVE-2016-1307.
Another significant vulnerability in XMPP is its susceptibility to Denial of
Service (DoS) attacks. As demonstrated in [25], DoS attacks can severely de-
grade network performance or even bring the network down entirely. (see Fig.8)
and (see Fig.9) provide a comparison of network throughput during normal op-
erations versus during an active DoS attack, illustrating the detrimental impact.

Figure 8: While Dos Attack, network throughput increased.

Figure 9: XMPP Stanza Structure.

2.3.1. Mitigation Strategies for XMPP Security

While XMPP does have a few security vulnerabilities, many of them can
be mitigated. Both [25] and [26] suggest using strong end-to-end encryption to
improve the confidentiality and integrity of the protocol.
Instead, for example, of relying on weak mechanisms such as Base64 en-
coding, which can leak sensitive information such as passwords, the paper [25]

16
suggests stronger authentication protocols such as SCRAM-SHA-1 or SCRAM
SHA-1-PLUS. These widely used protocols, employed by most platforms, includ-
ing MongoDB, use SHA-1 hashing functions to protect communications. Fur-
thermore, [26] performed an implementation of SHA-256 and AES encryption
on the protocol; the authors firmly believe that their project offers a workable
solution for secure instant messaging which could be ported into a wide variety
of applications and scenarios. In addition, various mitigations have been pro-
posed by XMPP’s XEP (XMPP Extension Protocol) [19] series to reduce the
impact of security threats. For example, XEP-0205 describes strategies for mit-
igating Denial of Service (DoS) attacks, and XEP-0178 covers proper certificate
management regarding SASL authentication.

2.4. Synthesis And Gaps

The literature on IoT security protocols XMPP, MQTT, and CoAP provided
valuable insights into their strengths and vulnerabilities. XMPP, commonly
used for real-time communication, integrated TLS and SASL for security but
remained susceptible to Denial of Service (DoS) attacks, message fragmenta-
tion, and weak authentication. These weaknesses in the authentication process
left it vulnerable to unauthorized access and crafted message attacks. MQTT,
designed for lightweight machine-to-machine communication, faced similar chal-
lenges. The absence of default encryption and authentication mechanisms made
it prone to data interception and unauthorized access. CoAP, optimized for
constrained environments, also had vulnerabilities, including selective blocking,
request delays, and path traversal attacks.
Although existing solutions, such as TLS/DTLS encryption, SCRAM-SHA-1
for authentication in XMPP, MQTTS for MQTT, and DTLS for CoAP, provided
some level of protection, significant gaps persisted. One major gap identified
was the inconsistency in security measures across these protocols, particularly in
the areas of authentication and authorization. Each protocol had its own inde-
pendent security implementation, creating vulnerabilities when they were used
together in an IoT ecosystem. This inconsistency posed a challenge, especially

17
in systems with diverse devices and communication protocols.
This research aimed to address these gaps by proposing targeted security en-
hancements for each protocol. For XMPP, the research’s proposed solution was
based on using JWT to strengthen the authentication process. In the case of
CoAP, the research addressed the critical vulnerability of request delay attacks
in resource-constrained environments by implementing a nonce mechanism. The
server generated a nonce based on the current timestamp and compared it with
the nonce sent by the client. If the nonces matched, the request was accepted
as fresh; otherwise, it was rejected, effectively mitigating replayed or delayed re-
quests. JWT was also employed with the nonce to ensure client authentication,
ensuring only authorized devices could access server resources.
For MQTT, the research included a simulation comparing the performance of
MQTT with and without TLS1.2, providing insights into the impact of TLS on
security and communication performance. By addressing these concerns, the re-
search provided targeted, protocol-specific security enhancements, contributing
to a more secure, reliable, and efficient implementation of IoT communication
protocols.

3. Methodology

This study employs a systematic approach to analyze the vulnerabilities of

MQTT, CoAP, and XMPP protocols, evaluate their performance, and propose
practical security enhancements in the context of IoT systems. The methodology
consists of several key phases detailed below.

3.1. Analysis

The MQTT vulnerability analysis focused on examining default configura-

tions, such as the absence of encryption and authentication. By capturing and
analyzing MQTT traffic, the study exposed the transmission of sensitive data,
including client credentials, in plaintext. Logs from the Mosquitto broker were
reviewed to assess authentication failures and unauthorized access attempts,

18
identifying potential weaknesses in the protocol’s default setup. Additionally,
the performance trade-offs between security and latency were analyzed by sim-
ulating client-broker interactions with and without TLS encryption. Metrics
such as latency and resource usage were measured to evaluate the impact of
encryption on communication delays.
The CoAP analysis focused on three specific attack scenarios based on the
literature review and experimental findings. The first attack analyzed was the
selective blocking attack, in which an attacker disrupts communication by se-
lectively dropping CoAP messages. Hypothetical scenarios from prior literature
were used to demonstrate how such selective disruptions could delay or interrupt
critical IoT operations.
A second attack, the path traversal attack, involved analyzing how malicious
actors could craft requests with manipulated paths, potentially bypassing access
controls to retrieve or manipulate sensitive information from the CoAP server.
The analysis identified CoAP’s insufficient input validation by default in some
configurations as a key factor enabling this attack.
This scenario was also analyzed through a literature review and an example,
concluding with the importance of improved user input sanitization mechanisms
for CoAP endpoints. The third attack, the request delay attack, was analyzed
through simulation and practical validation.
A default deployment of the CoAP server was implemented, revealing that
the protocol does not inherently protect against replay or delayed attacks. The
lack of mechanisms to verify the freshness of incoming requests made the server
vulnerable to these attacks. This evaluation concluded that the request delay
attack posed a critical threat that required a targeted solution.
The analysis of XMPP focused on comparing the performance characteristics
of two authentication protocols: JSON Web Tokens with RSA Signature 256
(JWT RS256) and the Salted Challenge Response Authentication Mechanism
(SCRAM-SHA-1). By examining the cumulative performance of these protocols
over a large number of iterations, the study aimed to identify any significant
differences in their efficiency.

19
3.2. Tools and Environment Setup

The tools and environment setup included using the Mosquitto MQTT bro-
ker and Paho MQTT client libraries to simulate MQTT client-broker inter-
actions. The experimental setup was implemented in a simulated IoT environ-
ment using virtual machines, reflecting constrained IoT network conditions. For
CoAP, the ”aiocoap” Python library was used to simulate server-client commu-
nications, along with the ”jwt” library for JWT generation and validation.
For XMPP, To conduct the performance evaluation, the research leveraged
various tools and libraries relevant to the XMPP ecosystem. This included the
use of XMPP server and client software, such as the Openfire XMPP server and
the Smack XMPP client API, to simulate the XMPP communication environ-
ment. Additionally, the study incorporated the ”jwt” library to facilitate the
generation and validation of JWT tokens as part of the analysis.

3.3. Proposed Solution Development

For MQTT, a fuzzy logic-based detection system was designed to enhance

the MQTT protocol’s resilience against denial-of-service attacks. Key communi-
cation parameters, such as request frequency, response time, and connection du-
ration, were incorporated into the model. The fuzzy logic system was developed
theoretically with lightweight requirements, suitable for resource-constrained
IoT devices. A fuzzy logic-based detection system will be proposed for miti-
gating DoS attacks on MQTT, while a JWT with nonce solution will address
request delay attacks in CoAP. The section will conclude by comparing the
effectiveness of existing and proposed solutions.
One such solution to improve detection capabilities is the fuzzy logic-based
detection system for DoS attacks. This system focuses on analyzing device in-
teractions and recognizing abnormal patterns, which is crucial in detecting DoS
attacks. It evaluates key parameters such as request frequency, response times,
and connection durations to identify unusual patterns that may indicate a DoS
attack, even when these patterns do not strictly violate predefined thresholds.

20
 One such innovative solution to improve detection capabilities is the fuzzy
logic-based detection system for DoS attacks. This system focuses on analyz-
ing device interactions and recognizing abnormal patterns, which is crucial in
detecting DoS attacks. It evaluates key parameters such as request frequency,
response times, and connection durations to identify unusual patterns that may
indicate a DoS attack, even when these patterns do not strictly violate prede-
fined thresholds. Table 1 provides a detailed breakdown of the system’s flow,
including input collection, fuzzification, fuzzy logic rules, inference, defuzzifica-
tion, and actionable responses, while Figure 15 visually represents the system
architecture. Together, they demonstrate how the collected inputs are processed
through fuzzy logic to generate actionable outputs for mitigating potential DoS
attacks.

Figure 10: Fuzzy Logic-based Detection System Architecture for DoS Attack.

21
 Table 1: Proposed Fuzzy Logic-based DoS Detection System Flow

Stage Description Example Parameters/Rules

Input Col- The system collects and mon-
lection itors key MQTT communica- – Number of requests per sec-

tion parameters to identify ab- ond

normal patterns. – Response time from the bro-

ker

– Duration of connections

– Packet sizes

Fuzzy Logic These inputs are evaluated us-

Rules ing fuzzy logic rules, consider- – “If the number of requests

ing varying degrees of abnor- per second is high AND

mality. the response time is slow,

THEN the system is likely
under attack.”

– “If the connection duration

is unusually long, THEN it
may indicate an attack.”

Fuzzification Raw data (e.g., high num-

& Inference ber of requests) is converted – Requests: High = 0.9,

into fuzzy values (e.g., “high”, Medium = 0.5, Low = 0.1

“low”) to determine attack – Response Time: Slow = 0.8,

likelihood. Moderate = 0.5, Fast = 0.2

Defuzzification The fuzzy output is converted

& Response into actionable decisions (e.g., – If attack likelihood is

triggering alerts or applying greater than 0.7, trigger an

countermeasures). alert

– Apply rate-limiting for high-

frequency requests or slow
response times to mitigate

22 attacks.
 When evaluating these mitigation approaches, it becomes clear that no sin-
gle solution can fully secure MQTT across all use cases. The effectiveness of any
security strategy depends on the specific requirements and constraints of the IoT
environment in which MQTT operates. For instance, resource-constrained sys-
tems, such as smart home devices, may benefit from lightweight cryptographic
algorithms such as AES or ECC, which provide a balance between security and
performance. On the other hand, industrial IoT deployments, which prioritize
strong data protection and can accommodate higher computational overhead,
are better suited for robust encryption methods such as TLS.
This highlights the importance of selecting security protocols based on the
context of deployment, carefully balancing security, latency, and resource uti-
lization to meet the system’s operational goals. For CoAP, the request delay
attack was addressed by implementing a JWT and nonce-based mechanism. A
CoAP server was developed using a secret key for signing JWT tokens and a
secret message for generating nonces. Nonce generation involved combining the
current timestamp with the secret message and generating a SHA-256 hash to
produce a unique identifier. The server validated incoming JWT tokens and en-
sured the format and freshness of nonces while tracking used nonces to prevent
replay attacks. On the client side, JWT tokens were created by encoding user
information and permissions with the secret key, and nonces were generated us-
ing the current timestamp and the shared secret message. These elements were
packaged into the POST request payload for secure communication. Simulation
and validation involved running the CoAP server and client, ensuring unique
nonces for each request, and rejecting repeated or old nonces. Additional testing
confirmed that previously used nonces, even after server restarts, were rejected,
validating the effectiveness of the solution.
Based on the findings from the performance comparison, the research did
not propose any specific solutions or enhancements to the XMPP protocol. The
focus remained on the evaluation of the existing authentication mechanisms,
specifically JWT RS256 and SCRAM-SHA-1, and their relative performance
characteristics within the XMPP context. No further development or imple-

23
mentation of new solutions was undertaken as part of this study.

4. Result and Discussion

This section will analyze the security vulnerabilities of MQTT, CoAP, and
XMPP, focusing on issues such as the lack of default encryption and authentica-
tion in MQTT, which leads to eavesdropping and potential DoS attacks. It will
review and compare existing mitigation strategies from the literature through
supported analysis and simulations.

4.1. MQTT

The findings from the evaluation of MQTT in the context of IoT security vul-
nerabilities confirm that while the protocol is widely adopted for its lightweight
design and efficiency in constrained environments, it presents significant se-
curity vulnerabilities that could jeopardize IoT applications if not adequately
addressed.
One of the critical issues observed in the MQTT protocol is its default lack
of encryption and authentication mechanisms, which leaves the system highly
vulnerable to eavesdropping and unauthorized access. This security gap is par-
ticularly concerning in IoT networks, where data integrity and privacy are es-
sential. Without encryption, sensitive data transmitted across the network can
be intercepted by attackers, who can steal or manipulate the information to
cause damage or disrupt interconnected devices.
As shown in Figure 11, the Wireshark capture of an MQTT connection
packet reveals that sensitive information, such as client ID, username, and pass-
word, is transmitted in plaintext. In this example, both the client ID and
credentials are visible, making it easy for attackers to capture and misuse these
details.
The lack of authentication further exacerbates the problem, as devices can
communicate with the broker without any validation of their identity. As
demonstrated in the logs during testing, the Mosquitto broker allows clients

24
Figure 11: Results of listening and capturing the MQTT connection command
packets.

to connect without proper authentication. In the Home Assistant API logs,

attempts to authenticate with both the FAKE and REAL usernames failed due
to incorrect passwords. Despite these failed attempts, the system didn’t block
further connections, and the broker still allowed MQTT clients to connect with-
out any issue, even without valid credentials. For instance, the following logs
show the results of the connection attempts, as seen in Figure 12:

Figure 12: Home Assistant and Mosquitto logs showing failed authentication
attempts and the lack of connection blocking.

These entries show failed authentication attempts for both the FAKE and
REAL usernames, but they do not prevent further connections. In the MQTT
log, the client mqtt-explorer-58f6fbef connected using a username (USERNAME),
but quickly disconnected. Another client (mqtt-explorer-75402a08) connected
without a username and also disconnected shortly after. These connections
occurred without any authentication checks or security measures in place to
prevent unauthorized access. Here are the corresponding logs from the MQTT

25
broker, as shown in Figure 13:

Figure 13: MQTT broker logs illustrating client connections and

disconnections without authentication.

Through analysis, it became evident that while vulnerabilities are inherent

in MQTT’s design, various mitigation strategies can effectively reduce the asso-
ciated risks. One notable approach is the adoption of MQTTS, which introduces
TLS encryption to secure data transmission between clients and brokers. By
ensuring encrypted communication, TLS significantly enhances data confiden-
tiality and mitigates the risk of interception or tampering.
However, the implementation of TLS introduces a clear trade-off between se-
curity and performance, particularly in resource-constrained IoT environments.
This trade-off is evident in Figure 14, which compares MQTT communication
with and without TLS. While TLS strengthens security, it imposes additional
computational overhead, leading to higher latency. The results clearly show
that enabling TLS results in consistently increased latency compared to unse-
cured communication, primarily due to the processing required for encryption
and decryption.
The increase in latency poses a significant challenge for IoT systems that
require real-time performance, such as sensors or actuators operating within
resource-constrained networks. The additional delay introduced by TLS can
adversely affect system responsiveness, thereby impacting the efficiency of time-
critical applications.
As highlighted earlier, securing MQTT communication involves a range of
strategies, each with its own trade-offs between security and operational ef-

26
Figure 14: Comparison of MQTT Latency With and Without TLS Encryption.

ficiency. One promising approach to address unauthorized access is the use

of client certificates or adopting the OAuth model for MQTT authentication.
These methods ensure that only authorized devices can interact with the MQTT
broker, reducing the risk of malicious data injection. However, the increased se-
curity they provide comes at the cost of administrative complexity. Setting up
a Public Key Infrastructure (PKI) or managing unique access credentials can
be challenging in large-scale IoT deployments, where devices are numerous and
potentially dispersed.
An alternative to traditional PKI is Lightweight Public Key Infrastructure
(LPKI), which offers a more resource-efficient solution. LPKI provides similar
security benefits to PKI but is optimized for constrained environments, making
it ideal for IoT systems where resources such as processing power and memory
are limited. LPKI reduces the overhead of key management, allowing for secure
authentication without the heavy resource demands associated with traditional
PKI systems.
Figure [15] illustrates the difference in efficiency by comparing the perfor-
mance of PKI and LPKI in terms of latency, energy consumption, and scal-

27
ability. As shown, LPKI significantly reduces both latency and energy usage
compared to PKI. This makes LPKI a more efficient option for IoT systems,
where minimizing delays and conserving energy is critical. The graph clearly
demonstrates how LPKI can offer secure authentication while addressing the
resource limitations typical of IoT environments.
Thus, organizations must carefully weigh the benefits of enhanced security
against the added operational burden, especially in constrained environments.

Figure 15: A Hypothetical comparison of PKI and LPKI efficiency in terms of

latency, energy consumption, and scalability.

In addition to enhancing authentication mechanisms, integrating anomaly

detection systems into MQTT brokers offers a proactive defense against mali-
cious activities, such as Denial-of-Service (DoS) attacks. These systems con-
tinuously monitor traffic patterns and identify abnormal behaviors to detect
potential threats early, minimizing the risk of disruption.

4.2. CoAP

It is evident from the literature review that CoAP’s strength lies in its
lightweight design and compatibility with HTTP. However, as highlighted, CoAP
has three main high-risk vulnerabilities: Selective Blocking Attacks, Request
Delay Attacks, and Path Traversal Vulnerabilities. As a quick recap:

28
 – Selective Blocking Attacks: These exploit CoAP’s reliance on un-
encrypted communication, allowing attackers to block specific requests
or responses. This can disrupt actuator operations and compromise the
availability and integrity of IoT systems.

– Request Delay Attacks: These involve introducing delays into commu-

nication, which can cause critical actions to be delayed or misinterpreted,
especially in actuator-based systems. This vulnerability poses significant
risks in environments where timely responses are critical for security and
functionality.

– Path Traversal Vulnerabilities: These occur when CoAP servers im-

properly handle URI components, enabling attackers to access sensitive
files on the server by exploiting weaknesses in URI parsing. This can
result in unauthorized access and potential system compromise.

The solutions previously proposed in the mitigation section for Selective

Blocking Attacks and Path Traversal Vulnerabilities were clear-cut, involving
the use of confirmable messages to prevent blocking attacks and proper URI
validation to mitigate path traversal. However, the solution for Request Delay
Attacks, as indicated in the literature review, remained unclear and vague.
The review recommended general approaches, such as employing a timestamp
mechanism or the Echo option specified in RFC9175. Given the significant
impact of request delay attacks on time-sensitive IoT applications, the following
section focuses on proposing a detailed and straightforward solution developed
using JWT (JSON Web Tokens) and nonce codes to effectively address this
vulnerability.

4.2.1. JWT and Nonce-based Solution for Request Delay Attacks

To mitigate the request delay attack, a JWT-based solution combined with
nonces has been implemented to ensure both authorization and freshness of re-
quests. JWT verifies the sender’s authorization, ensuring that only valid entities
can initiate requests.

29
 The nonce, a unique, one-time-use value, ensures the freshness of the message
and prevents replay attacks. Even if an attacker intercepts a message with a
valid nonce that has not been used or recorded by the server, the request will
still fail. This is because the implementation ensures that the server validates
the freshness of the nonce by reversing its hash, extracting the timestamp, and
comparing it to the current timestamp to verify that the request is recent and
legitimate.
Before the solution is implemented, a delay in request transmission could
have allowed an attacker to intercept and later replay a valid request, such as
unlocking a door, after a significant delay. This could bypass security mecha-
nisms, resulting in unauthorized access or other security issues.
The DTLS (Datagram Transport Layer Security) replay window could have
been manipulated, enabling delayed requests to be executed at a later time. To
demonstrate the attack, the scenario will be replicated using a CoAP server and
client built in Python, utilizing the aiocoap library.
In the example illustrated in Figure 17, a CoAP client sends an ’Unlock
Door’ message to a CoAP server. The sequence of a successful communication
is as follows:

Figure 16: CoAP client sends an “Unlock Door” message to a CoAP server.

The client message is sent to the server, which successfully receives it after
validating the JWT token. The client request was captured in Wireshark, as
shown in the figure below:

30
 Figure 17: Capturing the client request in Wireshark.

In Figure 18, the portion highlighted in Yellow is the JWT token and the
portion highlighted in green is the message. In a request delay attack, an at-
tacker would set up a specialized CoAP proxy to intercept and capture this
request. For demonstration purposes, let’s assume the attacker intercepts the
message shown above to conduct a delayed replay. By sending the same request
again with the unchanged JWT token, the attacker would successfully transmit
the ”Unlock Door” message to the server, regardless of the time it was originally
sent.
To illustrate this attack, the JWT token value from the message is assigned
to a fixed variable, and the message is sent to the server, as shown in the code
snippet in Figure 19:

Figure 18: Sending the message to the server.

After the client runs and sends the message to the server, the request is

31
accepted, and the message is successfully received by the server, as demonstrated
in Figure 20. This illustrates a request delay attack in action.

Figure 19: Request delay attack occurs in the server.

A nonce mechanism has been introduced alongside the JWT to address this
vulnerability. When the server receives a request, it verifies the nonce to ensure
that it has not been reused. Delayed requests with expired or reused nonces
are automatically rejected, preventing attackers from exploiting delayed request
delivery.
The nonce is a SHA-256 hash created by combining a secret message with
the current timestamp string. Upon receiving a message, the server generates a
hash using the same secret message and the current timestamp. If the nonce sent
in the client request matches the server-generated hash, the request is deemed
fresh and accepted. If the nonces do not match, the request is rejected, and
an appropriate error response is sent, indicating that the nonce format is either
incorrect or that the request is delayed or reused. This mechanism ensures that
even if an attacker intercepts a valid nonce from a legitimate client request (one
that the server has not yet marked as used), the request will fail when replayed
because the server will generate a nonce hash based on the current timestamp,
which will not match the intercepted nonce.
To demonstrate this mechanism, consider the server-client communication
shown in Figure 21. The client sends the ’Unlock Door’ message with a nonce
attached. Upon receiving the request, the server verifies the nonce and prints
its value for visualization purposes.
This is how the request looks like when captured in Wireshark:
As shown in Figure 22, the yellow and green portions are the JWT token
and message values respectively. The new blue portion is the value of the nonce.
If an attacker now captures this request before it reaches the server and delays
its transmission to a later time, the request will fail.

32
 Figure 20: The client sends the ”Unlock Door” message, this time with a
nonce attached.

Figure 21: Demonstration of the delayed request failure.

The server was restarted in this demonstration, and the client code was mod-
ified to send a valid nonce (underlined in red) in Figure 23 that was generated
in the previous simulation. However, as shown in the result, the server rejected
the request, returning the following error message:

– Response Code: 4.00 Bad Request

– Response Payload: Invalid nonce format or expired nonce

This confirms that the request delay attack has been successfully mitigated, as
the nonce, though valid in format, was not accepted because it was reused and
had expired. The server correctly identified the expired or reused nonce and
responded with an appropriate error message.

33
 Figure 22: Demonstration of the delayed request failure.

In the Methodology section, the implementation details of the JWT and

nonce solution will be thoroughly discussed. This will provide developers with
a clear, step-by-step guide to replicate the solution in their own systems.

4.3. XMPP
The comparative analysis of authentication mechanisms reveals significant
architectural and performance distinctions between the Salted Challenge Re-
sponse Authentication Mechanism (SCRAM) and JSON Web Tokens with RSA
Signature 256 (JWT RS256). This investigation demonstrates that while both
protocols serve authentication purposes, JWT RS256 offers substantive advan-
tages in contemporary distributed computing environments.

1. Architectural Scalability
The fundamental architectural differences between SCRAM and JWT
RS256 profoundly impact system scalability. SCRAM’s connection-bound
model necessitates persistent server-side session management, creating in-
herent scalability limitations. In contrast, JWT RS256 implements a
stateless authentication paradigm that inherently supports horizontal scal-
ing. This architectural approach is particularly critical in microservice and
cloud-native architectures, where dynamic resource allocation and mini-
mal server-side state maintenance are paramount.

34
 The stateless nature of JWT RS256 enables more efficient resource uti-
lization. By eliminating the need for extensive server-side session storage,
organizations can significantly reduce computational overhead and infras-
tructure complexity. This approach is especially beneficial in high-traffic
distributed systems where authentication mechanism efficiency directly
correlates with overall system performance
2. Cryptographic Considerations
Cryptographic architecture represents another critical point of differen-
tiation. SCRAM’s password-based authentication mechanism relies on
shared secrets, introducing inherent security vulnerabilities. JWT RS256,
utilizing asymmetric cryptography through public-key infrastructure (PKI),
offers a more robust security framework. The decentralized token verifi-
cation mechanism effectively mitigates risks associated with centralized
authentication points.
The embedded claims functionality in JWT RS256 provides enhanced se-
curity granularity. Unlike traditional authentication mechanisms, this ap-
proach allows for:
- Dynamic access control - Contextual authentication - Embedded meta-
data verification

Table 2: Characteristic Comparison: SCRAM vs. JWT RS256.

Characteristic SCRAM JWT RS256

Cryptographic Model Symmetric Key Asymmetric Key
Shared Secrets Yes No
Key Management Complexity Linear Non-linear
Scalability Limited Infinite

3. Performance Implications
As evidenced by the performance graph, JSON Web Tokens with RSA
Signature 256 (JWT RS256) demonstrate clear advantages over the Salted
Challenge Response Authentication Mechanism (SCRAM-SHA-1) in terms

35
Figure 23: JWT RS256 outperforms SCRAM-SHA-1 in execution time as
iterations increase, highlighting its efficiency for large-scale applications.

of cumulative performance over a large number of iterations.

The graph shows that JWT RS256 exhibits significantly lower cumulative
time compared to SCRAM-SHA-1 as the number of iterations increases.
After 10,000 iterations, the cumulative time for JWT RS256 is 9,566.23
milliseconds, while SCRAM-SHA-1 requires 62,792.88 milliseconds, over 6
times longer.
This data suggests that JWT RS256 offers superior performance charac-
teristics, particularly in high-volume, distributed computing environments
that require efficient and scalable authentication mechanisms. The stark
contrast in cumulative performance highlights the potential advantages of
JWT RS256 in terms of reduced latency, infrastructure cost savings, and
overall system responsiveness.
These performance implications should be carefully considered when se-
lecting an authentication protocol for modern, large-scale distributed ap-
plications. The significant performance advantage of JWT RS256 demon-
strated by this evidence is a crucial factor in determining the most appro-
priate authentication solution for such systems.

36
Table 3: Performance Benchmark Comparison between SCRAM-SHA-1 and
JWT RS256.

Metric SCRAM-SHA-1 JWT RS256

Token Generation Time (ms) 5-10 1-3
Verification Latency (ms) 50-100 10-20
Network Overhead Moderate Minimal

4. Interoperability and Standards Compliance

JWT’s alignment with established standards (RFC 7519) provides signifi-
cant advantages in cross-platform compatibility. This standardization fa-
cilitates integration with modern authentication protocols such as OAuth
2.0 and OpenID Connect. In contrast, SCRAM demonstrates more limited
interoperability across diverse technological environments.

Table 4: Interoperability and Standards Compliance Comparison between

SCRAM and JWT RS256.

Characteristic SCRAM JWT RS256

Standards Compliance Limited RFC 7519
OAuth 2.0 Integration Partial Native
OpenID Connect Integration Limited Native
Cross-Platform Support Moderate Extensive

37
Table 5: Comparison of IoT Communication Protocols: MQTT, CoAP, and
XMPP in terms of security, performance and scalability.

Feature MQTT CoAP XMPP

Protocol Publish/subscribe, Request/response, Real-time messaging
Type message broker-based, client-server model op- protocol, client-server
lightweight for IoT timized for low-power, and server-server com-
low-bandwidth devices munication
Performance High throughput and Performs well in con- Lower performance
low latency, optimized strained environments due to XML overhead
for real-time telemetry and lossy networks, us- and parsing require-
with lightweight head- ing minimal resources. ments. More suitable
ers and efficient binary UDP’s unreliability for robust, feature-rich
encoding. Ideal for may hinder perfor- applications than for
large IoT systems where mance in scenarios resource-constrained
quick message delivery requiring guaranteed environments or high
is critical. delivery. throughput.
Scalability Supports thousands Effective for small to Moderately scalable
of nodes efficiently medium-scale deploy- with decentralized
through hierarchical ments in constrained architecture, but re-
topic structures and environments. Scal- liance on XML and
optimized distribution. ability is limited for intermediaries limits its
Excellent for large-scale extensive networks due suitability for highly
IoT ecosystems. to UDP’s potential con- dynamic or large-scale
gestion and reliability IoT networks.
issues.
Security Supports TLS for secure Uses DTLS for secu- Incorporates SASL for
connections, lack native rity, vulnerable to weak authentication, TLS
encryption for payloads cryptographic methods for encryption, but has
known vulnerabilities
Architecture Broker-based architec- Client-server, designed Decentralized client-
ture, clients do not com- for constrained environ- server and server-server
municate directly ments and direct device communication streams
communication
Message Topic-based pub- Request-response with XML stanzas: mes-
38
Structure lish/subscribe mech- methods such as GET, sage, presence, and
anism for message POST, PUT, DELETE info/query
routing
5. Conclusion and Future Work

The analysis of Internet of Things (IoT) protocols shows the need to bal-
ance security and performance in environments with limited resources. This
section summarizes the key findings and outlines recommendations for future
improvements to enhance the safety and efficiency of IoT systems.

5.1. Conclusion

This study examined several Internet of Things (IoT) protocols, including

MQTT, CoAP, and XMPP, focusing on their features, weaknesses, and possible
solutions. Security improvements, such as TLS encryption, were found to make
protocols safer but added delays, highlighting the balance between security,
performance, and resource efficiency. This issue remains a key challenge in IoT
environments.
Major weaknesses were identified in the MQTT protocol, especially the lack
of default encryption and authentication. To address these issues, a fuzzy logic-
based system for detecting DoS attacks was proposed, aiming to improve secu-
rity without significantly affecting performance. For CoAP, some vulnerabilities
were addressed, but issues such as request delay attacks continue. In XMPP,
JWT authentication proved more effective than SCRAM due to its stateless na-
ture and low delay, making it ideal for large-scale, performance-sensitive setups.
These findings stress the need to match security solutions with the specific
needs of IoT applications. Systems prioritizing data integrity and confidentiality
should adopt strong security measures, even if they reduce performance slightly.
On the other hand, systems with limited resources can benefit from lighter
security strategies that balance safety and efficiency.

5.2. Future Work

Future work on MQTT will focus on creating the proposed fuzzy logic-based
system for detecting DoS attacks. This includes building a prototype designed
for resource-limited IoT environments and testing its performance in real-world
settings.
 To address the delays caused by TLS, lightweight encryption methods will
be studied to improve performance while maintaining security. More analysis of
MQTT weaknesses is needed, especially regarding weak authentication methods
and unauthorized access. Adding automated tools for monitoring traffic and
spotting unusual activity is recommended to better detect threats.
Lightweight authentication methods, such as SMQTT or OAuth, will be
explored along with testing MQTT’s performance across different IoT devices
to find the best balance of security and efficiency.
For CoAP, the proposed nonce mechanism—where the server creates a hash
using a secret message and the current time—ensures only recent requests are
accepted. However, the use of SHA-256 for creating hashes may be too de-
manding for devices with limited resources. Future work should look into using
simpler hashing methods to improve efficiency in these cases.
Further research on XMPP should analyze the long-term effects of using
JWT RS256, focusing on improving how tokens are managed and solving issues
with token cancellation in stateless systems.
Studies on quantum-safe cryptographic methods for JWT and comparisons
of signing methods such EdDSA and ECDSA are needed. Performance un-
der different network conditions and workloads, especially in edge computing,
requires thorough analysis. Additionally, standardized security practices for
token-based authentication in new systems such as IoT networks and serverless
platforms should be developed. Automated token management systems that
adjust security settings based on real-time threats and system metrics also need
further study.

40
References

[1] N. Naik, Choice of effective messaging protocols for iot systems: Mqtt,
coap, amqp and http, in: Proc. IEEE Int. Syst. Eng. Symp. (ISSE), Vienna,
Austria, 2017, p. 2. doi:10.1109/SysEng.2017.8088251.

[2] T. A. Alghamdi, A. Lasebae, M. Aiash, Security analysis of the constrained

application protocol in the internet of things, in: Second International Con-
ference on Future Generation Communication Technologies (FGCT 2013),
London, UK, 2013, pp. pp. 163–168. doi:10.1109/FGCT.2013.6767217.

[3] Z. Laaroussi, O. Novo, A performance analysis of the security communica-

tion in coap and mqtt, in: 2021 IEEE 18th Annual Consumer Communica-
tions & Networking Conference (CCNC), 2021. doi:10.1109/ccnc49032.
2021.9369565.

[4] D. Thangavel, X. Ma, A. Valera, H.-X. Tan, C. K.-Y. Tan, Performance

evaluation of mqtt and coap via a common middleware, in: Proc. 2014
IEEE 9th Int. Conf. Intell. Sens. Sens. Netw. Inf. Process. (ISSNIP), Sin-
gapore, 2014, pp. 1–6. doi:10.1109/ISSNIP.2014.6827678.

[5] M. M. Simsek, E. Atilgan, Attacks on availability of iot middleware pro-

tocols: A case study on mqtt, Vol. 4, 2023, pp. 20–21. doi:10.53608/
estudambilisim.1297052.

[6] E. Altulaihan, M. A. Almaiah, A. Aljughaiman, Cybersecurity threats,

countermeasures and mitigation techniques on the iot: Future research
directions, Vol. 11, 2022, pp. 6–7. doi:10.3390/electronics11203330.

[7] D. S. Ugalde, Security analysis for mqtt in internet of things, 2018, pp.
14–17.

[8] F. Raschbichler, Mitigate iot attacks with key mqtt security principles,
2021, [Online]. Available: https://www.hivemq.com/blog/mqtt-security-
principles-mitigate-iot-attacks/. Accessed: Oct. 7, 2024.

41
 [9] D. Dinculeană, X. Cheng, Vulnerabilities and limitations of mqtt protocol
used between iot devices, Vol. 9, 2019, p. 2. doi:10.3390/app9050848.

[10] G. Nebbione, M. C. Calzarossa, Security of iot application layer protocols:

Challenges and findings, Vol. 12, 2020, p. 55. doi:10.3390/fi12030055.

[11] M. Singh, M. A. Rajan, V. L. Shivraj, P. Balamuralidhar, Secure mqtt for

internet of things (iot), in: Proc. 2015 5th Int. Conf. Commun. Syst. Netw.
Technol., Gwalior, India, 2015, pp. 746–751. doi:10.1109/CSNT.2015.16.

[12] A. Niruntasukrat, et al., Authorization mechanism for mqtt-based internet

of things, in: Proc. 2016 IEEE Int. Conf. Commun. Workshops (ICC),
Kuala Lumpur, Malaysia, 2016, pp. 290–295. doi:10.1109/ICCW.2016.
7503802.

[13] A. Rizzardi, S. Sicari, D. Miorandi, A. Coen-Porisini, Aups: An open source

authenticated publish/subscribe system for the internet of things, Vol. 62,
2016, pp. 29–41. doi:10.1016/j.is.2016.05.004.

[14] A. A. Ali, Constrained application protocol (coap) for the iot, 2018, [On-
line]. Available: https://doi.org/10.13140/RG.2.2.33265.17766.

[15] Z. Shelby, K. Hartke, C. Bormann, The constrained application protocol

(coap), 2014, accessed: Oct. 10, 2024.
URL https://www.rfc-editor.org/rfc/rfc7252.html

[16] J. P. Mattsson, J. Fornehed, G. Selander, F. Palombini, C. Amsüss,

Attacks on the constrained application protocol (coap), 2024,
https://www.ietf.org/archive/id/draft-ietf-core-attacks-on-coap-04.html.
Accessed: Oct. 10, 2024.

[17] Aseem, Iot security – part 11 (introduction to coap protocol and secu-
rity), in: Payatu, 2024, https://payatu.com/masterclass/iot-security-part-
11-introduction-to-coap-protocol-and-security/. Accessed: Oct. 10, 2024.

42
[18] Z. Gao, J. Cao, W. Wang, H. Zhang, Z. Xu, Online-semisupervised neural
anomaly detector to identify mqtt-based attacks in real time, Vol. 2021,
2021, pp. 2–3. doi:10.1155/2021/4587862.

[19] XMPP, About xmpp, n.d.

URL https://xmpp.org/about/

[20] Wallarm, What is the extensible messaging and presence protocol (xmpp)?,
n.d.
URL https://www.wallarm.com/what/extensible-messaging-presence-protocol

[21] B. N. Iduh, Whatsapp network group chat analysis using python pro-
gramming, Vol. 12, 2023, pp. 71–78. doi:https://doi.org/10.1080/
23742917.2023.2228053.
URL https://www.tandfonline.com/doi/full/10.1080/23742917.
2023.2228053

[22] S. B. Anwar, M. J. S. Lizarondo, Security in the internet of things: A

survey on application layer protocols, in: IEEE Conference Publication,
2017.
URL https://ieeexplore.ieee.org/document/7968629

[23] B. N. Iduh, Whatsapp network group chat analysis using python program-
ming, 2020.

[24] ResearchGate, An example of extensible messaging and presence protocol

(jabber id, xmpp jid), n.d.
URL https://www.researchgate.net/figure/
An-example-of-Extensible-Messaging-and-Presence-Protocol-Jabber-ID-XMPP-JID_
tbl1_273687096

[25] M. I. Malik, I. N. McAteer, P. Hannay, N. F. Syed, B. Zubair, Xmpp

architecture and security challenges in an iot ecosystem, in: Proceedings of
the 16th Australian Information Security Management Conference, Edith
Cowan University, Perth, Australia, 2018, pp. 62–73.

43
[26] V. C.P, A. A. Chikkamannur, Iot future in edge computing, Vol. 3, IJAERS,
India, 2016, pp. 29–33, iSSN: 2349-6495 (P), 2456-1908 (O). doi:https:
//dx.doi.org/10.22161/ijaers/3.12.29.

[27] Z. Gao, J. Cao, W. Wang, H. Zhang, Z. Xu, Online-semisupervised neural

anomaly detector to identify mqtt-based attacks in real time, Vol. 2021,
2021, p. 3. doi:10.1155/2021/4587862.

44