Professional Documents
Culture Documents
Abstract—The IoT protocols used in the application layer, for statistical purposes and traffic is mostly from the sensor
namely the Constraint Application Protocol (CoAP) and Message to the cloud, it may be acceptable to lose a report here and
Queue Telemetry Transport (MQTT) have dependencies to the there. Even in such scenarios, however, reliable transport may
transport layer. The choice of transport, Transmission Control
Protocol (TCP) or the User Datagram Protocol (UDP), on the be required on occasion when firmware updates or security
other hand, has an impact on the Internet of Things (IoT) patches are done on devices.
application level performance, especially over a wireless medium. The physical layer and the medium access control, i.e., L1
The motivation of this work is to look at the impact of the protocol and L2, may differ greatly depending what kind of access
stack on performance over two different wireless medium real- technology is used. In the end to end path of an IoT application
izations, namely Bluetooth Low Energy and Wi-Fi. The use case
studied is infrequent small reports sent from the sensor device data flow, there may be fast fiber links, Ethernet cabling, as
to a central cloud storage over a last mile radio access link. We well as radio access links over a cellular network, and finally
find that while CoAP/UDP based transport performs consistently the last mile over a short range radio access technology in
better both in terms of latency and power consumption over both the capillary domain such as Bluetooth Low Energy (BLE) or
links, MQTT/TCP may also work when the use requirements Wi-Fi. In this paper, we focus on the aforementioned capillary
allow for longer latency providing better reliability. All in all, the
full connectivity stack needs to be considered when designing an domain technologies.
IoT deployment.
II. P REVIOUS WORK
Keywords-IoT, CoAP, MQTT, TCP, UDP, BLE, 802.11ah.
Previous work on comparing CoAP and MQTT has been
done in the form of small scale trials over an unnamed
I. I NTRODUCTION
radio technology [2] and by analytical means [3]. In [2], a
The idea of IoT is to use the Internet Protocol, mainly IPv6, healthcare use case was realized with prototype hardware. The
to communicate between connected devices and the cloud. A performance of the two protocols was then measured when a
plethora of standards addressing the IoT on different levels of set of medical sensor data was transferred over an unspecified
the protocol stack are available today and may be combined wireless link. It was found that unpredictable packet loss is
in different ways. handled poorly by CoAP but better by MQTT due to use
The IoT umbrella covers a large number of use cases. Here of TCP. In [3], the performance of the two protocols was
we focus on use cases where small data transmissions are studied over very lossy links. It was found that by tuning
expected from and to a large number of sensors and actuators. CoAP parameters, the performance can be greatly improved
Such use cases are for example connected lighting, connected and CoAP often performs better than MQTT due to the lower
home sensors and actuators, smart metering, smart agriculture, overhead, although some level of packet loss needs to be
etc. accepted when UDP is used. Our contribution to the discussion
Most of the popular IoT applications today use either the is to study the performance over realistically modeled actual
Internet Engineering Task Force (IETF) standardized Con- PHY and MAC protocols and see how the combination of the
straint Application Protocol (CoAP) or the OASIS specified protocols all the way from the application layer down to the
Message Queue Telemetry Transport (MQTT) as the data physical layer impacts the application performance.
transfer of choice. Other examples of IoT application layer Both BLE and Wi-Fi are traditionally deployed in a star
transfer protocols are the Hyper Text Transfer Protocol, Data or point to point topology and designed to handle two-way
Distribution Service and Advanced Message Queuing Protocol communication. Also, both technologies have recently evolved
[1]. for longer range. Thus, they provide an interesting comparison
The requirements of the use case set boundary conditions [4], [5]. IEEE 802.15.4 systems are typically designed mainly
to the reliability and latency of the communication link. If for one-way data streams and to function in a mesh topology.
the use case involves collecting large amounts of sensor data While it is true that IEEE 802.15.4 systems may also be
978-1-5090-5873-0/17/$31.00
c 2018 IEEE
simulator with detailed models of application, IP, transport, TBTT interval for beacon transmission is modelled as 100
network and link layers for both BLE and Wi-Fi. This in- ms in the current simulation. The channel bandwidth for
ternal simulator is similar in structure and capabilities to the the Wi-Fi transmissions is assumed to be 2 MHz with the
commonly used NS3 simulator. carrier frequency of 864 MHz. The antenna is assumed to be
Sensor reports were randomly generated by means of a omnidirectional.
Poisson process. For TCP, the session is terminated after each
sensor report was received in the other end of the connection, C. Simulation scenario
however, the termination procedure of a TCP connection has We simulate a scenario in which wireless sensors are
not been explicitly modelled. A simplified physical layer connected to the central device or gateway in a star topology.
was used for both technologies, with a detailed model of We assume that all the devices are within communication
the MAC layer, as well as the IP, TCP, and UDP layers. range from the central device.
The CoAP and MQTT layers were simplified with a simple The sensors communicate with a cloud server in the Internet
payload size model on top of the transport. More details on and we assume Internet delays to be 40 ms one way. This value
the technology specific models are discussed in sub-sections. is chosen to reflect a server that is deployed reasonably close
IP header compression was not modelled in the simulation. to the sensor.
Varying the payload is used instead. In the scenario devices generate sensor reports with packet
Buffer limitations of the devices have also not been mod- sizes of 25, 150 and 400 bytes as transport layer payload.
elled. The devices are also assumed to be in close vicinity of These values are chosen to show the impact of increasing
one another so that no, or at least very few packets, should be payload and to show that, e.g., header compression would
lost over the air, except for those lost due to collisions. have very little impact. We assume the transfer layer protocol
headers for CoAP and MQTT respectively to be included in
A. BLE modeling this payload as well any needed object representation overhead
The simulator contains detailed models of all parts relevant from, e.g., Light Weight Machine to Machine and security. The
to the evaluation according to revision 4.2 of the Bluetooth average report intensity is 5 packets per second. This means
specification. that in the network 5 packets are generated in total in a second.
In these evaluations the connection-oriented mode is utilized
as two-way communication is necessary, at least for the TCP VI. P ERFORMANCE E VALUATION
parts. A new BLE connection will be established for each new Simulations of CoAP with UDP and MQTT with TCP trans-
data transfer. This means that for a typical transaction, the fer layer over both BLE and IEEE 802.11ah were performed to
sensor will wake up upon generation of data to transmit, try study the impact of the transport layer protocol over different
to establish a new connection to a central device and transmit MAC layers. The main aspects studied are the extra delay
the data within the connection. and the extra power consumption caused by TCP usage when
Connection events are configured to happen every 7.5 ms compared to using UDP. We focus on simple sensor report
thus giving the sleeping cycle for a BLE connection. The traffic in the uplink direction. When using CoAP, this type
connection is initiated by the sensor who enters the connection of traffic can be sent non-confirmed, while with the MQTT
as slave. Each connection event is initiated by the master, stack TCP will provide reliability. We simulate also the CoAP
which in this case is the central device. confirmed case to show it has very little if no impact. By
utilizing two different link layers, we also gain understanding
B. IEEE 802.11ah modeling on how different sleeping mechanisms on the MAC layer
The simulator contains detailed models of all parts relevant impact the performance.
to the evaluation according to draft 8.0 of the IEEE 802.11ah
specification. A. Packet delay evaluation
Stations (STAs) are assumed to be in power save mode, The packet delays on the application layer from the time
and may wake up whenever there is uplink data to transmit. instant of packet creation and packet reception from the TCP
In power saving mode, DL data transmission from the Access and UDP layers in the receiver end are shown in Figure 3 for
Point (AP) to STA works such that the AP generates beacon both BLE and 802.11ah in cumulative distribution function
frames at periodic times given by the Target Beacon Transmis- format (CDF).
sion Time (TBTT). A Traffic Indication Map (TIM) frame in As can be seen in the BLE results, the difference between
the beacon is used to indicate the availability of the downlink CoAP (UDP) and MQTT (TCP) over the BLE connection ba-
data. If the beacon indicates availability of downlink data for sically corresponds to the three-way handshake that increases
a particular STA, then this STA transmits a Power-Save poll the number of messages that need to be exchanged. With the
(PS-poll) to the AP to inform the AP it is available in DL Internet delay of 40 ms one-way, the difference to UDP is at
to retrieve the data. In return, the AP transmits the DL data least 80 ms. The connection event interval used is 7.5 ms,
to the corresponding STA. Also, as STAs are in power-save adding an average delay of 2.75 ms to each DL oriented
mode, a STA may go to sleep, if the TIM element within the message. The spread in the delays for the packets that do not
beacon frame does not indicate that DL data is available. make it across at the first attempt depends on the BLE central
2018 Global Internet of Things Summit (GIoTS)
Packet delays for sensor reports over BLE, 5 reports per second Packet delays for sensor over 802.11AH, 5 reports per second
100 100
90 90
80 80
70 70
60 60
CDF [%]
CDF [%]
50 50
25 B, MQTT 25 B, MQTT
40 25 B, CoAP 40 25 B, CoAP
25 B, CoAP + ACK 25 B, CoAP + ACK
30 150 B, MQTT 30 150 B, MQTT
150 B, CoAP 150 B, CoAP
20 150 B, CoAP + ACK 20 150 B, CoAP + ACK
400 B, MQTT 400 B, MQTT
10 400 B, CoAP 10 400 B, CoAP
400 B, CoAP + ACK 400 B, CoAP + ACK
0 0
10-2 10-1 10-1
Packet delay [s] Packet delay [s]
(a) (b)
Fig. 3. Packet delays for BLE and 802.11ah for varying packet sizes and a fixed arrival rate of 2 packets per second.
device being busy serving another user. In the TCP case, this device. However, the sensor devices becomes unavailable to
delay may in turn cause TCP timeouts introducing further the network, whenever a connection is not open. Thus, this
delays. It should be noted that the BLE central device that model does not work well when the packet route has long
is modelled only supports one concurrent connection, which latency on the end to end path or when actuation is performed,
is significantly less than for a typical device. These results unless the connection would be opened periodically to poll for
may thus be somewhat pessimistic. DL data in the other end of the radio connection. In case such
The Wi-Fi results look rather different. Due to the 100 periodic polling would be performed, a similar degradation in
ms TIM interval, the first DL oriented transmission will take power consumption should be expected as is seen in the case
on average 50 ms longer than an uplink transmission with a of 802.11ah as the polling would introduce a function similar
uniform distribution over the 100 ms TIM period. The delay to the beacon reception in Wi-Fi.
spread from 150 ms to 250 ms can be clearly seen in the Adding header compression to the evaluation would have a
graph. very small impact on the performance due to reduced number
of bits transmitted over the air, as can be seen from the
B. Power consumption performance
difference between the different payloads.
Power consumption is affected by the choice of transfer and
thus transport layer protocol. The results are consistent and VII. C ONCLUSIONS AND FUTURE WORK
reflect the time needed to perform the actual data transfer,
which in case of MQTT (TCP) requires more messaging over In this paper we studied the performance of sensor reporting
the air (Figure 4). over CoAP and MQTT with UDP and TCP transport protocols
The larger difference between ah and BLE comes from the respectively running on top of two different radio interfaces,
beacon model used in Wi-Fi. There is no periodic wake up to namely BLE and IEEE 802.11ah. Unlike UDP, TCP includes
check for DL data in BLE, whereas in Wi-Fi this is inbuilt a handshake procedure to establish a session. This leads to
to the MAC protocol. The ah results are thus affected by the additional exchange of packets and therefore affects the delay
periodic wake up to listen the beacon every 100 ms. For a of packets and battery lifetime of IoT devices. Furthermore,
fairer comparison in the UDP case, the sensor devices could TCP acknowledgements used to ensure reliability need to be
be configured to non-TIM devices meaning that they would later received in the transmitter to avoid retransmissions.
not need to wake up to listen to the beacon. This would yield Delays are comparable for UDP between BLE and
a more similar power consumption to the BLE case. For TCP 802.11ah. They are dictated mainly by the time it takes to
use, however, TIM operation is required, as the device would transfer data over the air. When TCP is used, however, the
otherwise not be capable of two-way communication. TIM model in 802.11ah brings a built in minimum delay cor-
responding to the TIM configuration to the communication. In
C. Discussion these results, the delay spread further reflects the chosen TIM
In the evaluated scenario, BLE and 802.11ah have compa- interval of 100 ms, a typical interval configured for beacons in
rable performance. However, all disadvantages are not clear Wi-Fi networks today. However, to optimize for device battery
from the simulation results. When the end device acts as a lifetime in IoT deployments, longer TIM intervals may be
master in the BLE connection, we optimize sleep for the sensor configured.
2018 Global Internet of Things Summit (GIoTS)
0.8
15
0.6
10
0.4
5
0.2
0 0
P
K
TT
TT
P
K
TT
TT
TT
TT
oA
AC
AC
oA
AC
AC
oA
oA
oA
oA
AC
AC
Q
Q
Q
Q
C
C
C
C
M
M
M
M
+
+
+
+
B,
B,
B,
B,
B,
B,
B,
B,
B,
B,
B,
B,
P
P
P
P
oA
oA
oA
oA
oA
oA
25
25
0
0
25
25
0
0
15
40
15
40
15
40
15
40
C
C
C
C
B,
B,
B,
B,
B,
B,
0
0
25
25
15
40
15
40
(a) (b)
Fig. 4. Battery lifetime for BLE and 802.11ah for packets being generated with an inter-arrival time of 1 and 10 minutes for different packet sizes and
transport protocols. Please note different y-axis scales showing battery lifetime in years.
Regarding the battery lifetime, TCP uses more energy due [10] T. Tirronen, A. Larmo, J. Sachs, B. Lindoff, and N. Wiberg, “Machine-
to more packets sent, than UDP. The proportion between TCP to-machine communication with long-term evolution with reduced de-
vice energy consumption,” Transactions on Emerging Telecommunica-
and UDP remains for BLE and 802.11ah, but BLE device’s tions Technologies, vol. 24, no. 4, pp. 413–426, 2013.
battery lifetime is 17 times longer compared to 802.11ah. This [11] J. Sachs, N. Beijar, P. Elmdahl, J. Melen, F. Militano, and P. Salmela,
is due to the recurring wake-up to listen to the Wi-Fi beacon “Capillary networks–a smart way to get things connected,” Ericsson
Review, September, vol. 9, 2014.
every 100 ms even when there is no traffic to be received. [12] “Bluetooth Core Specification 4.2,” Bluetooth SIG, Standard, 2014.
The study confirms, the previous results where CoAP out- [13] “P802.11ah, Draft 10.0, Wireless LAN Medium Access Control (MAC)
performs TCP in latency. The study also shows differences in and Physical Layer (PHY) Specifications,” IEEE, Draft, 2016.
the different radios. While Bluetooth air interface is faster in
general, the Wi-Fi air interface provides lower latency when
the network is loaded.
R EFERENCES
[1] A. Al-Fuqaha, A. Khreishah, M. Guizani, A. Rayes, and M. Moham-
madi, “Toward better horizontal integration among IoT services,” IEEE
Communications Magazine, vol. 53, no. 9, pp. 72–79, 2015.
[2] Y. Chen and T. Kunz, “Performance evaluation of IoT protocols under
a constrained wireless access network,” in Selected Topics in Mobile
& Wireless Networking (MoWNeT), 2016 International Conference on.
IEEE, 2016, pp. 1–7.
[3] M. Collina, M. Bartolucci, A. Vanelli-Coralli, and G. E. Corazza,
“Internet of Things application layer protocol analysis over error and
delay prone links,” in Advanced Satellite Multimedia Systems Conference
and the 13th Signal Processing for Space Communications Workshop
(ASMS/SPSC), 2014 7th. IEEE, 2014, pp. 398–404.
[4] P. D. Marco, R. Chirikov, P. Amin, and F. Militano, “Coverage analysis
of Bluetooth Low Energy and IEEE 802.11ah for office scenario,” in
Personal, Indoor, and Mobile Radio Communications (PIMRC), 2015
IEEE 26th Annual International Symposium on, Aug 2015, pp. 2283–
2287.
[5] F. D. Carpio, P. D. Marco, P. Skillermark, R. Chirikov, K. Lagergren, and
P. Amin, “Comparison of 802.11ah and BLE for a home automation use
case,” in Personal, Indoor, and Mobile Radio Communications (PIMRC),
2016 IEEE 27th Annual International Symposium on, Sep 2016.
[6] Z. Shelby, K. Hartke, and C. Bormann, “The constraint application
protocol (CoAP),” Internet Requests for Comments, RFC 7252, 2014.
[7] C. Bormann, S. Lemay, H. Tschofenig, K. Hartke, B. Silverajan, and
B. Raymor, “CoAP (Constrained Application Protocol) over TCP, TLS,
and WebSockets,” IETF draft, December 2017.
[8] “MQTT, version 3.1.1,” OASIS Standard, October 2014.
[9] Y. Cheng, J. Chu, S. Radhakrishnan, and A. Jain, “TCP fast open,”
Internet Requests for Comments, RFC 7413, 2014.