MQTT

This guide shows how to enable MQTT (3.1.1) communication support for a Murano Product. Once the MQTT protocol is enabled, the example code provided shows how a simulated device connects and activates with the Murano MQTT endpoint.

Overview

Exosite’s MQTT offering supports bi-directional device communication with the Murano Platform using the MQTT protocol. Devices can provision with a Murano Product, publish data to its resources, and receive updates about changes made to its resources.

Murano MQTT communication support translates MQTT communication into native Murano commands, allowing MQTT clients to connect and communicate with the Murano Platform. While the translation layer provides most features of MQTT 3.1.1 there are some special considerations to make for Murano integration.

Communication between the device and the Murano MQTT endpoint is secured by TLS and made available by default on port 8883 or on port 443 as an option. Note that Exosite uses SNI. MQTT client libraries are required to support SNI.

Anonymous access to the $provision topic is provided only to facilitate activating an MQTT device. The processing of activation supplies the device with the credentials (i.e., MQTT password) it needs to authenticate future sessions with Murano.

Murano will use device authentication (token, certificate etc.) to identify a device and its state. It's recommended to leave MQTT ClientId empty because a missmatch with Murano device authentication will make the device unable to connect.

Access control limits an activated device to publishing/subscribing only to the that device’s resources. Anonymous clients, by contrast, can only publish to the $provision endpoint and can only subscribe to that endpoint’s activation reply topic, which is unique to each activation request. When connecting a device to a Murano Product for the first time, it must anonymously subscribe to the $provision topic in order to receive its MQTT password via the subscription reply. The password the device receives in the reply represents the "activation" of the client and is what enables successive future connections.

In Murano, device resource subsciptions are managed centrally. This means an MQTT device is not allowed to use the subscribe command; instead, it will automatically receive updates for resources according to the product configuration.

MQTT Support
V3.1.1 Yes
TLS Yes*
Publish Yes
Subscribe Yes**
Qos 0 Yes
Qos 1 Yes
Qos 2 No

* Required

** Central management, devices are not allowed to ask for topic subscriptions

For information about the MQTT protocol, see http://mqtt.org/.

Getting Started

MQTT client libraries are readily available. Exosite requires that the library supports TLS and requires that the TLS support is modern enough that it includes Server Name Indication (SNI). Please contact Exosite support if a preferred MQTT client library fails either criteria.

This tutorial will use Python and Eclipse Paho™ MQTT Python Client to connect to Murano. For an example in C using Eclipse Paho™ MQTT C Client, take a look at https://github.com/exosite-garage/mqtt_example.

Requirements

Hardware Setup

A development computer or laptop.

Software Setup

To complete this guide, download and install the following on the development machine:

You must have a Murano account and have created a Product within it. For more information on how to create and account and a Product in the account, visit the Create a Product article for more information.

Setup A Sandbox

This guide will require files to be created on the development machine. Throughout this guide it will be assumed that all commands and files will be run from the following directory:

mkdir ~/murano-mqtt-client
cd ~/murano-mqtt-client

Configure MuranoCLI

In order to proceed, the MuranoCLI tool needs to be initialized and configured to the correct Product.

The following command is interactive. It will prompt you for a username and password as well as the desired Murano Business. It also creates some local default assets for things like hosted applications (which this guide does not use) if they don't exist already.

cd ~/murano-mqtt-client
murano init

You can verify that the correct Product ID is selected by navigating to the Product in a web browser and comparing the Product ID in the web UI with the output of the following command:

murano config product.id

Enable MQTT for Your Product

Enable the MQTT protocol on your Murano Product.

murano setting write Gateway.protocol name mqtt

Verify the current setting is "mqtt":

> murano setting read Gateway.protocol
{:name=>"mqtt", :devmode=>false, :port=>443}

Alternatively you can set it in the web UI:

image alt text

Create a Resource

Edit the file specs/resources.yaml in the sandbox directory (i.e., ~/murano-mqtt-client) to the following:

# ~/murano-mqtt-client/specs/resources.yaml
---
  temp:
      format: number

Now use MuranoCLI to create the resource specified in the above spec file:

murano syncup

Activate Your Device

The default Product settings are such that devices are allowed to register their own identities. In some provisioning models it is required or advantageous to have a list of pre-authorized device identities registered with Murano. This is also known as whitelisting. At this point, make sure that the default setting is applied and saved. Navigate to the SETTINGS tab in your Product web UI and verify that "Allow devices to register their own identity" is selected.

Another way to set or check this is to use Murano CLI:

murano setting write Gateway.provisioning presenter_identity --bool yes
  1. Save the following certificate to a file called ~/murano-mqtt-client/Murano_Root_CA.cer:

    -----BEGIN CERTIFICATE-----
    MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh
    MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
    d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
    QTAeFw0wNjExMTAwMDAwMDBaFw0zMTExMTAwMDAwMDBaMGExCzAJBgNVBAYTAlVT
    MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j
    b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IENBMIIBIjANBgkqhkiG
    9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jvhEXLeqKTTo1eqUKKPC3eQyaKl7hLOllsB
    CSDMAZOnTjC3U/dDxGkAV53ijSLdhwZAAIEJzs4bg7/fzTtxRuLWZscFs3YnFo97
    nh6Vfe63SKMI2tavegw5BmV/Sl0fvBf4q77uKNd0f3p4mVmFaG5cIzJLv07A6Fpt
    43C/dxC//AH2hdmoRBBYMql1GNXRor5H4idq9Joz+EkIYIvUX7Q6hL+hqkpMfT7P
    T19sdl6gSzeRntwi5m3OFBqOasv+zbMUZBfHWymeMr/y7vrTC0LUq7dBMtoM1O/4
    gdW7jVg/tRvoSSiicNoxBN33shbyTApOB6jtSj1etX+jkMOvJwIDAQABo2MwYTAO
    BgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUA95QNVbR
    TLtm8KPiGxvDl7I90VUwHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUw
    DQYJKoZIhvcNAQEFBQADggEBAMucN6pIExIK+t1EnE9SsPTfrgT1eXkIoyQY/Esr
    hMAtudXH/vTBH1jLuG2cenTnmCmrEbXjcKChzUyImZOMkXDiqw8cvpOp/2PV5Adg
    06O/nVsJ8dWO41P0jmP6P6fbtGbfYmbW0W5BjfIttep3Sp+dWOIrWcBAI+0tKIJF
    PnlUkiaY4IBIqDfv8NZ5YBberOgOzW6sRBc4L0na4UU+Krk2U886UAb3LujEV0ls
    YSEY1QSteDwsOoBrp+uvFRTp2InBuThs4pFsiv9kuXclVzDAGySj4dzp30d8tbQk
    CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4=
    -----END CERTIFICATE-----
    
  2. Save the following code into a file called ~/murano-mqtt-client/activate.py:

from paho.mqtt import client as mqtt
import ssl
import logging

# logging.basicConfig(level=logging.DEBUG)

pid = input("Product ID? ")
did = input("Device ID? ")
host = pid + ".m2.exosite.io"
open("product_id.txt", "w").write(pid)
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    provision_str = "$provision/" + did
    client.publish(provision_str, None, qos=0)

def on_message(client, userdata, msg):
    print("Activation succeeded!")
    token = msg.payload.decode()
    print("Token: ", token)
    open('token.txt', "w").write(token)
    client.disconnect()

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Disconnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
logger = logging.getLogger(__name__)
client.enable_logger(logger)

# see https://github.com/eclipse/paho.mqtt.python/blob/1.1/README.rst#tls_set
client.tls_set(
    ca_certs=cert,
    cert_reqs=ssl.CERT_REQUIRED
)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Now execute the above script by running the following:

python3 activate.py
  1. Provide the Product ID and a device identity of your choice when the activate.py script prompts you for them.
$ python activate.py
Product ID? x2lmj5npsktbuik9
Device ID? 12345

Activation succeeded!
Token: b7b34f55e948b94841820ea50868a2490632d78f

A successful result, as shown above, activates the device, prints the credential, and saves it to a file called ~/murano-mqtt-client/token.txt for subsequent sessions. Notice that in the Product UI the device has been "activated". Save the device ID for later usage (e.g., 12345

NOTE: The client connected anonymously and then provisioned itself using the provided device identity (e.g., 12345) as the MQTT client ID.

Publish Data

Next, use the returned credentials to reconnect and publish data to the new device in the Murano Product. The data will be published to the temp resource defined a few steps above.

Save the following code into the file ~/murano-mqtt-client/publish.py:

from paho.mqtt import client as mqtt
import ssl
import logging

# logging.basicConfig(level=logging.DEBUG)

pid = open("product_id.txt").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    res = "temp"
    resource = "$resource/" + res
    print("Publishing to Resource: ", res)
    client.publish(resource, input("Value? "), qos=0)
    print("Done. Disconnecting...")
    client.disconnect()

def on_message(client, userdata, msg):
 print("Value set, previous was: ", msg.payload.decode())

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Disconnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
logger = logging.getLogger(__name__)
client.enable_logger(logger)

# see https://github.com/eclipse/paho.mqtt.python/blob/1.1/README.rst#tls_set
client.tls_set(
    ca_certs=cert,
    cert_reqs=ssl.CERT_REQUIRED
)

# read auth token from file
token = open("./token.txt").read()
print("Using Token: ", token)
client.username_pw_set("", token)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Next, execute the script with the following command:

python3 publish.py

The script prompts the user for the data to send. The device’s resources are represented as topics "$resource/" (e.g., $resource/temp”).

Below is some example output of the script prompting the user for data and then publishing:

$ python3 publish.py
Using Product ID: d23kegyeoxb280000
Using Token: XzE3KU2Zhs9ZDl0cSz0Lf8Xp5Ez7rR0cUa1rO4qE
Publishing to Resource: temp
Value? 23
Done. Disconnecting...

The device’s temp resource value will reflect the value published by the script.

You can check the value by either using the Murano web UI or reading the state from the command line using Murano CLI.

An example of checking the value using Murano CLI is provided below:

$ murano device read 12345 temp
+-------+----------+-----+------------------+
| Alias | Reported | Set | Timestamp        |
+-------+----------+-----+------------------+
| temp  | 72       | 72  | 1505394662994644 |
+-------+----------+-----+------------------+

Example of checking using the web UI:

image alt text

Receiving data from the server

Copy the following code to a file called ~/murano-mqtt-client/subscribe.py:

from paho.mqtt import client as mqtt
import ssl
import logging

# logging.basicConfig(level=logging.DEBUG)

pid = open("product_id.txt", "r").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_message(client, userdata, msg):
 print("Received: ", msg.payload.decode())

def on_connect(client, userdata, flags, rc):
  print("Waiting for messages")

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Disconnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
logger = logging.getLogger(__name__)
client.enable_logger(logger)

# see https://github.com/eclipse/paho.mqtt.python/blob/1.1/README.rst#tls_set
client.tls_set(
    ca_certs=cert,
    cert_reqs=ssl.CERT_REQUIRED
)

token = open("token.txt", "r").read()
print("Using Token: ", token)
client.username_pw_set("", token)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Executing the script (command provided below) will subscribe to all changes to the Murano device with the token recieved with the activate.py script. The subscribe.py script will print all messages sent to the device.

python3 subscribe.py

Now, either using Murano Product web UI or by using the Murano CLI, change the state of the temp resource:

murano device write 12345 temp=68

Or using the web UI: if you directly enter values in the browser, the device will also receive those changes as messages:

image alt text

After running the command, above, in a separate terminal, the subscribe.py script should print something like the following:

$ python3 subscribe.py
Using Product ID:  n110e3xbifmfk0000
Using Token:  eY4f3tyrE4Jqe3HsLGnPYf7cACKZlb0uvKVatFxX
Waiting for messages
Received:  68

Summary

This guide showed how to configure a Murano Product to use the MQTT internet protocol to provision and activate a simulated device with a Python script, as well as publishing to device resources and subscribing to changes to device resources.

The following sections cover some related topics and how one might alter the scripts and Product settings for various other applications.

Password Authentication

You need to change the provisioning/auth_type to password first. You can use the previous examples; you need to do minor changes.

Activation

Read/publish

Certificate authentication

Activation

from paho.mqtt import client as mqtt
import os
import ssl
import logging

# logging.basicConfig(level=logging.DEBUG)

pid = input("Product ID? ")
host = pid + ".m2.exosite.io"
open("product_id.txt", "w").write(pid)
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    print("Activation succeeded!")
    client.disconnect()

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Disconnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
logger = logging.getLogger(__name__)
client.enable_logger(logger)

certfile = "./certs/" + "cert.pem"
keyfile  = "./certs/" + "key.pem"
print("Current dir: " + os.getcwd() + " Certificate: " + certfile + ", Keyfile: " + keyfile)
client.tls_set(
    ca_certs=cert,
    certfile=certfile,
    keyfile=keyfile,
    cert_reqs=ssl.CERT_REQUIRED
)

client.on_connect = on_connect
client.on_disconnect = on_disconnect

client.connect(host, 443)
client.loop_forever()

Now execute the above script by running the following:

python3 certificate_activate.py
  1. Provide the Product ID of your choice when the certificate_activate.py script prompts you for them. The device ID does not need to be set; it will be set to the common name in the certificate automatically
$ python3 certificate_activate.py
Product ID? x2lmj5npsktbuik9

Activation succeeded!

After the success message nothing need to be done you only need to use the same cert/key for any further communication.

Read/publish

from paho.mqtt import client as mqtt
import os
import ssl
import logging

# logging.basicConfig(level=logging.DEBUG)

pid = open("product_id.txt").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    resource = "$resource/" + input("Resource ID? ")
    client.publish(resource, input("Value? "), qos=0)
    client.disconnect()

def on_message(client, userdata, msg):
 print("Value set, previous was: ", msg.payload.decode())

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("DisConnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
logger = logging.getLogger(__name__)
client.enable_logger(logger)

certfile = "./certs/" + "cert.pem"
keyfile  = "./certs/" + "key.pem"
print("Current dir: " + os.getcwd() + " Certificate: " + certfile + ", Keyfile: " + keyfile)
client.tls_set(
    ca_certs=cert,
    certfile=certfile,
    keyfile=keyfile,
    cert_reqs=ssl.CERT_REQUIRED
)

client.on_connect = on_connect
client.on_disconnect = on_disconnect

client.connect(host, 443)
client.loop_forever()

Next, execute the script with the following command:

python3 certificate_publish.py

The script prompts the user for the data to send. The device’s resources are represented as topics "$resource/" (e.g., $resource/temp”).

Below is some example output of the script prompting the user for data and then publishing:

$ python3 certificate_publish.py
Using Product ID: d23kegyeoxb280000
Publishing to Resource: temp
Value? 23
Done. Disconnecting...

The device’s temp resource value will reflect the value published by the script.