MQTT

This guide shows how to enable MQTT for a Murano Product. Once the MQTT protocol is enabled, the example code provided shows how a simulated device connects and activates with the Murano MQTT endpoint.

Requirements

Hardware Setup

A development computer or laptop.

Software Setup

To complete this guide, download and install the following on the development machine:

You must have a Murano account and have created a Product within it. For more information on how to create and account and a Product in the account, visit the Create a Product article for more information.

Overview

Exosite’s MQTT offering supports bi-directional device communication with the Murano Platform using the MQTT protocol. Devices can provision with a Murano Product, publish data to its resources, and subscribe to changes made to its resources.

Communication between the device and the Murano MQTT endpoint is secured by TLS and made available on port 8883. Note that Exosite uses SNI. MQTT client libraries are required to support SNI.

Anonymous access to the $provision topic is provided only to facilitate activating an MQTT device. The processing of activation supplies the device with the credentials (i.e., MQTT password) it needs to authenticate future sessions with Murano.

Access control limits an activated device to publishing/subscribing only to the that device’s resources. Anonymous clients, by contrast, can only publish to the $provision endpoint and can only subscribe to that endpoint’s activation reply topic, which is unique to each activation request. When connecting a device to a Murano Product for the first time, it must anonymously subscribe to the $provision topic in order to receive its MQTT password via the subscription reply. The password the device receives in the reply represents the "activation" of the client and is what enables successive future connections.

For information about the MQTT protocol, see http://mqtt.org/.

Getting Started

MQTT client libraries are readily available. Exosite requires that the library supports TLS and requires that the TLS support is modern enough that it includes Server Name Indication (SNI). Please contact Exosite support if a preferred MQTT client library fails either criteria. A patched version of Eclipse Paho™ MQTT Python Client is used in the examples below.

Setup A Sandbox

This guide will require files to be created on the development machine. Throughout this guide it will be assumed that all commands and files will be run from the following directory:

mkdir ~/murano-mqtt-client
cd ~/murano-mqtt-client

Configure MuranoCLI

In order to proceed, the MuranoCLI tool needs to be initialized and configured to the correct Product.

The following command is interactive. It will prompt you for a username and password as well as the desired Murano Business. It also creates some local default assets for things like hosted applications (which this guide does not use) if they don't exist already.

cd ~/murano-mqtt-client
murano init

You can verify that the correct Product ID is selected by navigating to the Product in a web browser and comparing the Product ID in the web UI with the output of the following command:

murano config product.id

Enable MQTT for Your Product

Enable the MQTT protocol on your Murano Product.

murano setting write Gateway.protocol name mqtt

Verify the current setting is "mqtt":

> murano setting read Gateway.protocol
{:name=>"mqtt", :devmode=>false, :port=>443}

Alternatively you can set it in the web UI:

image alt text

Create a Resource

Edit the file specs/resources.yaml in the sandbox directory (i.e., ~/murano-mqtt-client) to the following:

# ~/murano-mqtt-client/specs/resources.yaml
---
  temp:
      format: number

Now use MuranoCLI to create the resource specified in the above spec file:

murano syncup

Activate Your Device

The default Product settings are such that devices are allowed to register their own identities. In some provisioning models it is required or advantageous to have a list of pre-authorized device identities registered with Murano. This is also known as whitelisting. At this point, make sure that the default setting is applied and saved. Navigate to the SETTINGS tab in your Product web UI and verify that "Allow devices to register their own identity" is selected.

Another way to set or check this is to use Murano CLI

murano setting write Gateway.provisioning presenter_identity --bool yes
  1. Save the following certificate to a file called ~/murano-mqtt-client/Murano_Root_CA.cer:

    -----BEGIN CERTIFICATE-----
    MIIGIjCCBAqgAwIBAgIJAKfLy05oNigQMA0GCSqGSIb3DQEBCwUAMIGdMQswCQYD
    VQQGEwJVUzESMBAGA1UECAwJTWlubmVzb3RhMRQwEgYDVQQHDAtNaW5uZWFwb2xp
    czEQMA4GA1UECgwHRXhvc2l0ZTEMMAoGA1UECwwDT3BzMSMwIQYDVQQDDBpFeG9z
    aXRlIEludGVybmFsIENBIChwcm9kKTEfMB0GCSqGSIb3DQEJARYQcm9vdEBleG9z
    aXRlLmNvbTAeFw0xNjExMjIxNzM2MDBaFw0yNjExMjAxNzM2MDBaMIGdMQswCQYD
    VQQGEwJVUzESMBAGA1UECAwJTWlubmVzb3RhMRQwEgYDVQQHDAtNaW5uZWFwb2xp
    czEQMA4GA1UECgwHRXhvc2l0ZTEMMAoGA1UECwwDT3BzMSMwIQYDVQQDDBpFeG9z
    aXRlIEludGVybmFsIENBIChwcm9kKTEfMB0GCSqGSIb3DQEJARYQcm9vdEBleG9z
    aXRlLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANnRFk1pXkXG
    rMyRDbMLm6cY3qYsVuxxs0EafV+7vEBwsZezzAZt/FRD+zJJWTlc3KjOoQ+ZgJ7W
    z062GZ4QvgWun93QL7RTqpCq3pFbDGz1bx7u+EfIMG8uz9lrE1Ja4s3+eXKnPAjF
    SZFfk0pefq/i6ra3gJrO1mNYX3KLJf3f1AGTl46d2+siRRp1nTEPgY+zAjdtgA+2
    zcA2fJ5un3D4o52aIQ3Pl407pJTMZs/GY4307WT8FWcIdD7dA5sjvBjUrASaLbTI
    bd7JKGfJ+u9mbdQKjU+SriiPC2JvKAtp8BHZa4y0jy12hoFgxZnOBUgDvfMQj1H0
    bSjHsBvOd+pJY/1orczb5IROFPwBx1AMjP8QR4lqYhc5jPkVdJwGRHT+S6INK96/
    fWr/SGdtI0Km7T4689AzwOY5b4VdRpydsS2e83ml23/dmAJHI+wVIUk+UQTCT+Ot
    XC0uAOMK4y0JmUlpdQZY1A/sJbwsrmnVl0kndmP0llL9m7K7/4U2WqllY9yUux2o
    YaGvMC1ubJEWUsy/MdYE3MeJeiuovzQeegI8EXN5iMGBoD48eCK+Y5H82xPuJorK
    0QBMzelYsPHw4N2294EVATiQLC5rrTk4OnZffh+GeKw9GVfilmEJ9sd7Vah0KW6+
    sIA219FCeA9WDGSxujEpan7NA7poni9dAgMBAAGjYzBhMB0GA1UdDgQWBBQyAJXD
    WsHsIJ0jbybnXx5+OeGTfDAfBgNVHSMEGDAWgBQyAJXDWsHsIJ0jbybnXx5+OeGT
    fDAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjANBgkqhkiG9w0BAQsF
    AAOCAgEArxICBDIgxUWKBzpVOWVXCLQJQZBWcmTEq7uEb12xzFo0fmO0X4E7G+tb
    q/Teh0Q4DpHkgvaDDiY8laekz7Xw/P6SUfDQ53NaFZldEPgW6s/+Zufqn6XyfGpP
    6BqdnrvEO2MiRYDvHBUip75nUfIAsiPZ6IAJSyJJOtSl9BnTJFHDqrdKAHYZLi2q
    neroGE3SMjTNdWyG+vAEOYwTnDG5TSzbyDnKx7tFwOndAcPR+DBTrMLBNO/a9e3U
    BI0IuFBS9tfOK5vOy+AWS4cLEPP8ESuwVAk/JJ34JBN0GRiT8tGVOMikxyOp/Tzw
    CeHe8qcDcK6tcP0TASI40I+QEnE2ap5eesBW6a8ueB7IF/ioDeA6ax7aWWaoYSmy
    rF+0SyAv50qswBseMqz/YdF5mELrnv9pMtrv6nlP2F9v2qmkjNr8cWm1HXpuWXbQ
    NeXUv3Yagr8/gYmGIGWTZpd7i3QB5VKTj2JdW06IJV95Ln0sdnDyuGqYRALjHAcM
    iCpULSVqZ0olY5n2H8GKniVUjZNUxzDt606khwNp/BdzTMrQMpn8aP/I7EaNOxCo
    ja63eTgabEpo2frqhvSi65UZ2GIAJlE85kN3+mX7Ulue28obk0y8BzXpUbVG2g8v
    zyyXrYmY95gijkrSUGi3oMP5e2ZF5aupmG1psNRnR89r0qpD5Zs=
    -----END CERTIFICATE-----
    
  2. Save the following code into a file called ~/murano-mqtt-client/activate.py:

from paho.mqtt import client as mqtt
import ssl

pid = input("Product ID? ")
host = pid + ".m2.exosite.io"
open("product_id.txt", "w").write(pid)
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    provision_str = "$provision/" + input("Device ID? ")
    client.publish(provision_str, None, qos=0)

def on_message(client, userdata, msg):
    print("Activation succeeded!")
    token = msg.payload.decode()
    print("Token: ", token)
    open('token.txt', "w").write(token)
    client.disconnect()

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("DisConnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
client.tls_set(
    ca_certs=cert,
    server_hostname=host,
    cert_reqs=ssl.CERT_NONE
)
client.tls_insecure_set(True)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Now execute the above script by running the following:

python activate.py
  1. Provide the Product ID and a device identity of your choice when the activate.py script prompts you for them.
$ python activate.py
Product ID? x2lmj5npsktbuik9
Device ID? 12345

Activation succeeded!
Token: b7b34f55e948b94841820ea50868a2490632d78f

A successful result, as shown above, activates the device, prints the credential, and saves it to a file called ~/murano-mqtt-client/token.txt for subsequent sessions. Notice that in the Product UI the device has been "activated". Save the device ID for later usage (e.g., 12345

NOTE: The client connected anonymously and then provisioned itself using the provided device identity (e.g., 12345) as the MQTT client ID.

Publish Data

Next, use the returned credentials to reconnect and publish data to the new device in the Murano Product. The data will be published to the temp resource defined a few steps above.

Save the following code into the file ~/murano-mqtt-client/publish.py:

from paho.mqtt import client as mqtt
import ssl

pid = open("product_id.txt").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    res = "temp"
    resource = "$resource/" + res
    print("Publishing to Resource: ", res)
    client.publish(resource, input("Value? "), qos=0)
    print("Done. Disconnecting...")
    client.disconnect()

def on_message(client, userdata, msg):
 print("Value set, previous was: ", msg.payload.decode())

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Disconnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
client.tls_set(
    ca_certs=cert,
    server_hostname=host,
    cert_reqs=ssl.CERT_NONE
)
client.tls_insecure_set(True)
token = open("./token.txt").read()
print("Using Token: ", token)
client.username_pw_set("", token)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Next, execute the script with the following command:

python publish.py

The script prompts the user for the data to send. The device’s resources are represented as topics "$resource/" (e.g., $resource/temp”).

Below is some example output of the script prompting the user for data and then publishing:

$ python publish.py
Using Product ID: d23kegyeoxb280000
Using Token: XzE3KU2Zhs9ZDl0cSz0Lf8Xp5Ez7rR0cUa1rO4qE
Publishing to Resource: temp
Value? 23
Done. Disconnecting...

The device’s temp resource value will reflect the value published by the script.

You can check the value by either using the Murano web UI or reading the state from the command line using Murano CLI.

An example of checking the value using Murano CLI is provided below:

$ murano device read 12345 temp
+-------+----------+-----+------------------+
| Alias | Reported | Set | Timestamp        |
+-------+----------+-----+------------------+
| temp  | 72       | 72  | 1505394662994644 |
+-------+----------+-----+------------------+

Example of checking using the web UI:

image alt text

Receiving data from the server

Copy the following code to a file called ~/murano-mqtt-client/subscribe.py:

from paho.mqtt import client as mqtt
import ssl

pid = open("product_id.txt", "r").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_message(client, userdata, msg):
 print("Received: ", msg.payload.decode())

def on_connect(client, userdata, flags, rc):
  print("Waiting for messages")

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("DisConnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
client.tls_set(
    ca_certs=cert,
    server_hostname=host,
    cert_reqs=ssl.CERT_NONE
)
client.tls_insecure_set(True)
token = open("token.txt", "r").read()
print("Using Token: ", token)
client.username_pw_set("", token)

client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message

client.connect(host, 443)
client.loop_forever()

Executing the script (command provided below) will subscribe to all changes to the Murano device with the token recieved with the activate.py script. The subscribe.py script will print all messages sent to the device.

python subscribe.py

Now, either using Murano Product web UI or by using the Murano CLI, change the state of the temp resource:

murano device write 12345 temp=68

Or using the web UI: if you directly enter values in the browser, the device will also receive those changes as messages:

image alt text

After running the command, above, in a separate terminal, the subscribe.py script should print something like the following:

$ python subscribe.py
Using Product ID:  n110e3xbifmfk0000
Using Token:  eY4f3tyrE4Jqe3HsLGnPYf7cACKZlb0uvKVatFxX
Waiting for messages
Received:
Received:  {"timestamp":1505396242184000,"value":68}

Summary

This guide showed how to configure a Murano Product to use the MQTT internet protocol to provision and activate a simulated device with a Python script, as well as publishing to device resources and subscribing to changes to device resources.

The following sections cover some related topics and how one might alter the scripts and Product settings for various other applications.

Password Authentication

You need to change the provisioning/auth_type to password first. You can use the previous examples; you need to do minor changes.

Activation

Read/publish

Certificate authentication

Activation

from paho.mqtt import client as mqtt
import os
import ssl

pid = input("Product ID? ")
host = pid + ".m2.exosite.io"
open("product_id.txt", "w").write(pid)
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    print("Activation succeeded!")
    client.disconnect()

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("DisConnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
certfile = "./certs/" + "cert.pem"
keyfile  = "./certs/" + "key.pem"
print("Current dir: " + os.getcwd() + " Certificate: " + certfile + ", Keyfile: " + keyfile)
client.tls_set(
    ca_certs=cert,
    server_hostname=host,
    certfile=certfile,
    keyfile=keyfile,
    cert_reqs=ssl.CERT_NONE
)
client.tls_insecure_set(True)

client.on_connect = on_connect
client.on_disconnect = on_disconnect

client.connect(host, 443)
client.loop_forever()

Now execute the above script by running the following:

python activate.py
  1. Provide the Product ID of your choice when the activate.py script prompts you for them. The device ID does not need to be set; it will be set to the common name in the certificate automatically
$ python activate.py
Product ID? x2lmj5npsktbuik9

Activation succeeded!

After the success message nothing need to be done you only need to use the same cert/key for any further communication.

Read/publish

from paho.mqtt import client as mqtt
import os
import ssl

pid = open("product_id.txt").read()
print("Using Product ID: ", pid)
host = pid + ".m2.exosite.io"
cert = "./Murano_Root_CA.cer"

def on_connect(client, userdata, flags, rc):
    resource = "$resource/" + input("Resource ID? ")
    client.publish(resource, input("Value? "), qos=0)
    client.disconnect()

def on_message(client, userdata, msg):
 print("Value set, previous was: ", msg.payload.decode())

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("DisConnected with error", rc)
        exit()

client = mqtt.Client(client_id="")
certfile = "./certs/" + "cert.pem"
keyfile  = "./certs/" + "key.pem"
print("Current dir: " + os.getcwd() + " Certificate: " + certfile + ", Keyfile: " + keyfile)
client.tls_set(
    ca_certs=cert,
    server_hostname=host,
    certfile=certfile,
    keyfile=keyfile,
    cert_reqs=ssl.CERT_NONE
)
client.tls_insecure_set(True)

client.on_connect = on_connect
client.on_disconnect = on_disconnect

client.connect(host, 443)
client.loop_forever()

Next, execute the script with the following command:

python publish.py

The script prompts the user for the data to send. The device’s resources are represented as topics "$resource/" (e.g., $resource/temp”).

Below is some example output of the script prompting the user for data and then publishing:

$ python publish.py
Using Product ID: d23kegyeoxb280000
Publishing to Resource: temp
Value? 23
Done. Disconnecting...

The device’s temp resource value will reflect the value published by the script.