MQTT Version 5.0 - What's New?
By Tom Carpenter On 03/26/2020
The MQ Telemetry Transport (MQTT) protocol was updated with a new version on March 7, 2019. This version replaces or supercedes version 3.1.1 from October 29, 2014. In this brief post, I'll explain some of the new features introduced in the latest version.
First, some clarification. MQTT 3.1 (2010) was coded as version 3 in the MQTT Connect control packet, which is used to connect to the MQTT server. When MQTT 3.1.1 was released in 2014, it was coded as version 4. To remove confusion, the standards group chose to use 5 as both the Connect control packet version number and the standard version number for the latest version.
With that clarification provided, what is MQTT? First, some debate exists wether the acronym stands for MQ Telemetry Transport or Message Queuing Telemetry Transport. The debate is over whether MQ stands for something or not. Some suggest it is just a remnant of an old IBM protocol and stands for nothing. Others suggest that standard version 3.1.1 stated, in the Status section at the beginning of the document, that, "This document was last revised or approved by the OASIS Message Queueing Telemetry Transport (MQTT) TC on the above date." and, therefore, it stands for Message Queueing. While I tend to agree with the latter (being a stickler for terminology born in a standard document), the good news is that it changes nothing about how the protocol works.
MQTT is a protocol that is heavily used in IoT implementations. It uses a pub/sub model which means that some devices publish to the MQTT server and others subscribe to it. Devices can be both publishers and subscribers at the same time.
The client is the program or device that connects to the server and publishes application messages to the server for other clients to receive or the client subscribes to application messages that other programs or devices have published. The server is the intermediary device or program between the publishers and subscribers and provides a place for application messages to be sent as a single destiation, which can then be forewarded to any number of subscribers. The application message is the data carried through the MQTT protocol and system for delivery and it includes quality of service information, the topic name for the message, and the payload of the message. MQTT uses topic names to identify messages so that subscribers can request messages from that topic.
The version 5.0 technical committee (TC) was created in 2016 to take the existing 3.1.1 version and enhance it with the following goals in mind (source: https://www.oasis-open.org/committees/mqtt/charter.php):
- Enhancements for Scalability and Large Scale Systems including the ability to communicate functional levels, define optional functions, and control resource usage.
- Improved Error Reporting allowing error indications to be returned by both MQTT client and MQTT server with the definition of additional return values.
- Formalize commonly observed patterns including, but not limited to capability discovery, request-response, correlation.
- Extensibility Mechanisms enabling the addition of data fields on packets, including application-extensible data whose interpretation is not specified by MQTT.
- Performance Improvements and additional support for Resource Constrained MQTT clients.
A primary goal was to maintain compatibility with MQTT 3.1.1 programs and devices. Therefore, many enhancements in MQTT version 5 are optional. For example, response codes were added to many messages to "improve Error Reporting" and these response codes are optional.
One new feature aimed at enhancing scalability and improving performance for constrained clients is shared subscriptions. A shared subscription works different from a normal subscription. In a normal subscription, each subscriber receives a copy of the message from the MQTT server. So, if there are three subscribers, three messages will be sent. In a shared subscription, if the same three subscribers subscribe, only one of them will receive each message. Typically, the MQTT server uses a round-robin method so that each subscriber in the shared subscription gets an equal number of messages overall. For example, when three messages are received by the MQTT server from a publisher for a shared subscription, the first message goes to the first subscriber, the second to the second, and the third to the third. The implementation can then use whatever methods required with the received data, such as aggregation of the messages or reading of the messages from all subscribers using some extra layer of communication. Such a model helps to provide fault tolerance and ensure that subscribers are not overloaded with messages in a high volume scenario.
Another addition aimed at improving performance and scalability helps to reduce the number of queued messages for a subscriber that is currently disconnected. This is the Message Expiry interval, which is a four byte integer that represents the number of seconds the message should be queud for subscribers before abandoning delivery. Additionally, when the message is sent to the subscriber, the Message Expiry interval will be set to the receive value minus the time that the message has been waiting on the server. This extra information could be used to analyze latency in delivery and possibly detect some problems with subscriber connectivity or communications. When delivery of all messages is not critical, using Message Expiry intervals can improve scalability of an MQTT system.
Finally, MQTT 3.1.1 supported only three data representations: bits, integer data values (which were 2 bytes), and UTF-8 encoded strings. MQTT 5 extends this, while continuing to support the three data representations in 3.1.1. The added data representations are 4 byte integers, variable byte integers (which use up to 4 bytes with the first 7 bits in each byte representing data and the last bit indicating whether more bytes follow, such that the value 10010111 indicates that another byte follows, but the value 10010110 indicates that another byte does not), binary data, and UTF-8 string pairs. The last one, UTF-8 string pairs, is basically two UTF-8 encoded strings allowing for name-value pairs, like those often seen in JSON. The first UTF-8 encoded string is the name and the second is the value.
More enhancements were made in MQTT 5, after all the new standard is 137 pages and version 3.1.1 was only 81 pages, so you can imagine that other enhancements were introduced. However, those mentioned here are the primary enhancements that continue to move MQTT forward as the most common protocol for IoT data transmission.
Tagged with: mqtt, iot
Blog Disclaimer: The opinions expressed within these blog posts are solely the author’s and do not reflect the opinions and beliefs of the Certitrek, CWNP or its affiliates.