MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol.

MQTT was developed by Andy Stanford-Clark (IBM) and Arlen Nipper (Eurotech; now Cirrus Link) in 1999 for the monitoring of an oil pipeline through the desert. The goals were to have a protocol, designed as bandwidth-efficient and using little battery power, because the devices were connected via satellite link and this was extremely expensive at that time.

The protocol uses a publish/subscribe architecture in contrast to HTTP with its request/response paradigm. Publish/Subscribe is event-driven and enables messages to be pushed to clients. The central communication point is the MQTT broker, it is in charge of dispatching all messages between the senders and the rightful receivers. Key elements of the communication are:

• Each client that publishes a message to the broker, includes a topic into the message.
• The topic is the routing information for the broker.
• Each client that wants to receive messages subscribes to a certain topic and the broker delivers all messages with the matching topic to the client.

The difference to HTTP is that a client doesn’t have to pull the information it needs, but the broker pushes the information to the client, in the case there is something new. Therefore each MQTT client has a permanently open TCP connection to the broker. If this connection is interrupted by any circumstances, the MQTT broker can buffer all messages and send them to the client when it is back online.

As mentioned before the central concept in MQTT, to dispatch messages, is the topics. A topic is a simple string that can have more hierarchy levels, which are separated by a slash.

A sample topic for sending temperature data of the living room could be house/living-room/temperature. On one hand the client can subscribe to the exact topic or on the other hand it can use a wildcard. The subscription to house/+/temperature would result in all message send to the previously mention topic house/living-room/temperature as well as any topic with an arbitrary value in the place of living room, for example house/kitchen/temperature.

The plus sign is a single level wild card and only allows arbitrary values for one hierarchy. If you need to subscribe to more than one level, for example to the entire subtree, there is also a multilevel wildcard (#). It allows to subscribe to all underlying hierarchy levels. For example house/# is subscribing to all topics beginning with house.

Messages in MQTT are published on topics. There is no need to configure a topic, publishing on it is enough. Topics are treated as a hierarchy, using a slash (/) as a separator. This allows sensible arrangement of common themes to be created, much in the same way as a filesystem. For example, multiple computers may all publish their hard drive temperature information on the following topic, with their own computer and hard drive name being replaced as appropriate:

Clients can receive messages by creating subscriptions. A subscription may be to an explicit topic, in which case only messages to that topic will be received, or it may include wildcards. Two wildcards are available, + or #. The + can be used as a wildcard for a single level of hierarchy. It could be used with the topic above to get information on all computers and hard drives as follows:

MQTT defines three levels of Quality of Service (QoS). The QoS defines how hard the broker/client will try to ensure that a message is received. Messages may be sent at any QoS level, and clients may attempt to subscribe to topics at any QoS level. This means that the client chooses the maximum QoS it will receive. For example, if a message is published at QoS 2 and a client is subscribed with QoS 0, the message will be delivered to that client with QoS 0. If a second client is also subscribed to the same topic, but with QoS 2, then it will receive the same message but with QoS 2. For a second example, if a client is subscribed with QoS 2 and a message is published on QoS 0, the client will receive it on QoS 0.

Higher levels of QoS are more reliable, but involve higher latency and have higher bandwidth requirements.

• 0: The broker/client will deliver the message once, with no confirmation.
• 1: The broker/client will deliver the message at least once, with confirmation required.
• 2: The broker/client will deliver the message exactly once by using a four step handshake.

After a MQTT client is connected to a broker, it can publish messages. MQTT has a topic-based filtering of the messages on the broker, so each message must contain a topic, which will be used by the broker to forward the message to interested clients. Each message typically has a payload which contains the actual data to transmit in byte format. MQTT is data-agnostic and it totally depends on the use case how the payload is structured. It’s completely up to the sender if it wants to send binary data, textual data or even full-fledged XML or JSON.

A MQTT publish message also has some more attributes, which we’re going discuss in detail:

• Topic Name
  A simple string, which is hierarchically structured with forward slashes as delimiters.
• QoS
  A Quality of Service Level (QoS) for this message. The level (0,1 or 2) determines the guarantee of a message reaching the other end (client or broker).
• Retained flag
  This flag determines if the message will be saved by the broker for the specified topic as last known good value. New clients that subscribe to that topic will receive the last retained message on that topic instantly after subscribing. More on retained messages and best practices in one of the next posts..
• Payload
  This is the actual content of the message. MQTT is totally data-agnostic, it’s possible to send images, texts in any encoding, encrypted data and virtually every data in binary.
• Packet Identifier
  The packet identifier is a unique identifier between client and broker to identify a message in a message flow. This is only relevant for QoS greater than zero. Setting this MQTT internal identifier is the responsibility of the client library and/or the broker.
• DUP flag
  The duplicate flag indicates, that this message is a duplicate and is resent because the other end didn’t acknowledge the original message. This is only relevant for QoS greater than 0.

Publishing messages doesn’t make sense if no one ever receives the message, or, in other words, if there are no clients subscribing to any topic. A client needs to send a SUBSCRIBE message to the MQTT broker in order to receive relevant messages. A subscribe message is pretty simple, it just contains a unique packet identifier and a list of subscriptions.

When publishing MQTT messages, a publishing client has no guarantee that a message is actually received by a subscribing client. It can only make sure its message gets delivered safely to the broker. The same is true for a subscribing client. If a client is connecting and subscribing to topics it is interested in, there is no guarantee when the subscriber will get the first message, because this totally depends on a publisher on that topic.

It can take a few seconds, minutes or hours until the publisher sends a new message on that topic. Until then, the subscribing client is totally in the dark, about the current status. This is were retained messages come into play.

When a subscriber connects to a broker it will only receive messages published after subscription. So it will not get any information till the publisher sends a message.

A retained message is a normal MQTT message with the retained flag set to true. The broker will store the last retained message and the corresponding QoS for that topic Each client that subscribes to a topic pattern, which matches the topic of the retained message, will receive the message immediately after subscribing. For each topic only one retained message will be stored by the broker.

In other words a retained message on a topic is the last known good value, because it doesn’t have to be the last value, but it certainly is the last message with the retained flag set to true.

Sending a retained message from the perspective of a developer is quite simple and straight-forward. You just need to set the retained flag of a MQTT publish message to true

There is also a very simple way for deleting a retained message on a topic: Just send a retained message with a zero byte payload on that topic where the previous retained message should be deleted. The broker deletes the retained message and all new subscribers won’t get a retained message for that topic anymore. Often deleting is not necessary, because each new retained message will overwrite the last one.

In order to make the subsequent code more understandable, we will use the transferring of sensor data from a temperature and brightness sensor to a control center over the internet as an example. The sensors will be connected to a Raspberry Pi, which acts as gateway to the MQTT broker, which resides in the cloud. On the other side is a second device, the control center, that also has an MQTT client and receives the data. Additionally we will implement a notification, which alerts the control center if the sensor is disconnected.

Now it's time to write a simple sample. Few pre-requisites should be available:

• A cloud MQTT broker like Cloud MQTT
• A Raspberry Pi acting as MQTT publisher
• A laptop acting as MQTT suscriber

We will link a cloud MQTT and 2 computers with the standard mosquitto command line tools, mosquitto_sub and mosquitto_pub.

Configure a test account on CloudMQTT

Intall the mosquitto clients

$ sudo apt-get install mosquitto-clients

Now, suscribe to the MQTT topic. The client will keep waiting for messages.

$ mosquitto_sub -d -u ttvqonoe -P hT4DdZNIy_JO -h m21.cloudmqtt.com -p 15911 -t "test"

Intall the mosquitto clients

$ sudo apt-get install mosquitto-clients

Now, send a message.

$ mosquitto_pub -d -u ttvqonoe -P hT4DdZNIy_JO -h m21.cloudmqtt.com -p 15911 -t "test" -m "hello"

You should see the "hello" message on Laptop client console.

Now, we will send the temperature with a fixed interval with a simple Shell using a pipe.

#!/bin/bash
while true 
do 
vcgencmd measure_temp |  mosquitto_pub -d -u ttvqonoe -P hT4DdZNIy_JO -h m21.cloudmqtt.com -p 15911 -t "test" -l
sleep 1
done