UA Part 14: PubSub - 7.3.5 MQTT

MQTT is an open standard application layer protocol for Message Oriented Middleware. MQTT is often used with a Broker that relays messages between applications that cannot communicate directly.

Publishers send MQTT messages to MQTT brokers. Subscribers subscribe to MQTT brokers for messages. A Broker may persist messages so they can be delivered even if the Subscriber is not online. Brokers may also allow messages to be sent to multiple Subscribers.

The MQTT protocol defines a binary protocol used to send and receive messages from and to topics. The body is an opaque binary blob that can contain any data serialized using an encoding chosen by the application.

There are currently two versions of the MQTT protocol in use, version 3.1.1 and version 5.0. Version 5.0 expands on version 3.1.1 by adding support for connection and message properties. This enables advanced routing scenarios at the broker level in particular when using encrypted payloads.

This document defines two possible encodings for the message body: the binary encoded DataSetMessage defined in 7.2.3 and a JSON encoded DataSetMessage defined in 7.2.4.6.9.

MQTT version 3.1.1 does not provide a mechanism for specifying the encoding of the MQTT message which means the Subscribers need to be configured in advance with knowledge of the expected encoding. As a consequence, Publishers should only publish NetworkMessages using a single encoding to a unique MQTT topic name.

MQTT version 5.0 adds the encoding and the message type information to the message and connection header and therefore allows Subscribers to detect the encoding and the message mapping. No additional information is added to the meta data messages.

MQTT Publisher and Subscriber transport profiles for full and minimal support are defined in OPC 10000-7.

Message security is primarily provided by a TLS connection between the Publisher or Subscriber and the MQTT server; however, this requires that the MQTT server be trusted. For that reason, it may be necessary to provide end-to-end message security. Applications that require end-to-end message security with MQTT need to use the UADP NetworkMessages and binary message encoding defined in 7.2.3. JSON encoded message bodies need to rely on the security mechanisms provided by MQTT and the MQTT broker.

7.3.5.2 Address

The syntax of the MQTT transporting protocol URL used in the Address parameter defined in 6.2.7.3 has the following form:

mqtts://<domain name>[:<port>][/<path>]

The protocol prefix mqtts provides transport security. The default port is 8883.

mqtt://<domain name>[:<port>][/<path>]

The protocol prefix without transport security is mqtt. The default port is 1883.

wss://<domain name>[:<port>][/<path>]

The protocol prefix for MQTT over secure Web Sockets is wss. The default port is 443.

7.3.5.3 Authentication

MQTT supports the use of Username/Password authentication in the initial CONNECT packet. Aside from password credentials, implementations can use this mechanism to pass any form of secret, such as an authentication token. However, if CONNECT authentication is used, the connection should be secured.

MQTT version 5.0 also supports enhanced authentication, whereby clients can specify the desired SASL authentication method during initial CONNECT and finish the secret exchange with the broker using subsequent AUTH packets, or reauthenticate on an existing connection.

Authentication shall be performed according to the configured AuthenticationProfileUri of the PubSubConnection, DataSetWriterGroup, DataSetWriter or DataSetReader entities.

If no authentication information is provided in the form of ResourceUri and AuthenticationProfileUri, SASL Anonymous is implied.

If the authentication profile specifies SASL PLAIN authentication, a separate connection for each authentication setting is required.

7.3.5.4 Connection properties

The MQTT transport mapping for version 3.1.1 only supports the connection property ClientID using a KeyValuePair. Any other configured setting in the connection properties shall be silently discarded.

If the ClientID is not configured, the PublisherId is used as ClientID. If the PublisherId has a UInteger DataType, the UInteger value is converted to a String for the ClientID.

MQTT version 5.0 allows Publishers and Subscribers to provide MQTT connection properties as part of opening the connection.

The connection properties apply to any connection created as part of the PubSubConnection, or subordinate configuration entities, such as the WriterGroup and the DataSetWriter.

The properties are defined through the KeyValuePair array in the ConnectionProperties. The NamespaceIndex of the QualifiedName in the KeyValuePair shall be 0.

Table 201 formally defines the ConnectionProperties used for MQTT connection configuration.

Table 201 – MQTT ConnectionProperties

Key	DataType	Description
0:MqttVersion	String	Defines the MQTT version to use for the MQTT connection. Possible values are “3.1.1”, “5.0” and “BestAvailable”. The default value is BestAvailable.
0:MqttTopicPrefix	String	The <Prefix> part of the Topic convention defined in 7.3.5.7.1. The default value is “opcua”

For MQTT properties, the Name of the QualifiedName is constructed from a prefix “connection” followed by a hyphen and the MQTT property name with the following syntax.

Name = connection-<MQTT property name>

Table 202 defines the MQTT standard connection properties.

Table 202 – OPC UA MQTT standard connection property configuration

MQTT property name	OPC UA DataTypes	MQTT data types
ClientID	String	UTF-8 Encoded String
Receive Maximum	UInt16	Two Byte Integer
Maximum Packet Size	UInt32	Four Byte Integer
Session Expiry Interval	UInt32	Four Byte Integer
Topic Alias Maximum	UInt16	Two Byte Integer
Request Response Information	Boolean	Byte
Request Problem Information	Boolean	Byte

Any name not in the Table 202 is assumed to be a MQTT User Property.

7.3.5.5 RequestedDeliveryGuarantee

The BrokerTransportQualityOfService values map to MQTT publish and subscribe QoS settings as follows:

AtMostOnce and BestEffort is mapped to MQTT QoS 0.
AtLeastOnce is mapped to MQTT QoS 1.
ExactlyOnce is mapped to MQTT QoS 2.

7.3.5.6 Transport Limits and Keep Alive

If the KeepAliveTime is set on a WriterGroup, a value slightly higher than the configured value of the group in seconds should be set as MQTT Keep Alive ensuring that the connection is disconnected if the keep alive message was not sent by any writer in the specified time. If multiple WriterGroups are configured, the group with the highest KeepAliveTime setting is used for the calculation.

The implementation chooses packet and message size limits depending on the capabilities of the OS or of the capabilities of the device the application is running on. They can be made configurable through configuration model extensions or by other means.

7.3.5.7 Topics

7.3.5.7.1 General

MQTT messages are sent to Topics which provide context and other information about the message. Topics are hierarchical paths that allow Subscribers to use wildcards to select multiple Topics. Therefore, Topics are most useful when they follow a predictable pattern. This clause defines the Topic conventions for use with the MQTT mapping. All Publishers shall be able to support these conventions.

A Topic has the following general pattern:

<Prefix>/<Encoding>/<MqttMessageType>/<PublisherId>/[<WriterGroup>[/<DataSetWriter>]]

Where Topic levels are:

<Prefix> is a system defined prefix that provides a scope for the PublisherIds;The default value is ‘opcua’.
<Encoding> specifies the encoding of messages sent to the Topic (‘json’, ‘uadp’, etc);
<MqttMessageType> specifies the content of the messages published to the Topic as defined in 7.3.5.7.2;
<PublisherId> uniquely identifies a Publisher within the scope of the prefix;
<WriterGroup> is the name of a WriterGroup within the Publisher;
<DataSetWriter> is the name of a DataSetWriter within the WriterGroup;

The <Prefix> should be one Topic level, however, system designers may break <Prefix> into multiple Topic levels. Note that using multiple Topic levels prevents Subscribers from automatically discovering Publishers in the system unless they are preconfigured with the <Prefix> used for the system.

The possible values for the <Encoding> are ‘json’ or ‘uadp’ based on the message mappings defined in 7.2.3 and 7.2.4.6.9.

When Publishers are configured to publish to an MQTT Broker they shall have PublisherId assigned that can be used as Topic level.

The Topic levels that appear after the <PublisherId> depend on the <MqttMessageType>.

When these Topic conventions are used the wildcards in Table 203 may be used:

Table 203 – Examples of MQTT Wildcards

Wildcard	Notes
opcua/json/status/#	Subscribes to the status of all Publishers in the “opcua” scope. This allows the Subscriber to detect when Publishers come online or disappear.
opcua/json/metadata/#	Subscribes to the metadata of all Publishers in the “opcua” scope. This allows the Subscriber to detect changes to metadata.
opcua/json/data/device-one/#	Subscribes to all data produced by Publisher “device-one” in the “opcua” scope.
opcua/json/data/+/+/diagnostic	Subscribes to all Publishers that offer a “diagnostic” writer.

The MQTT Topic syntax places restrictions on what characters may be used in a Topic level. Specifically, Topic levels are any UTF-8 string that:

Does not start with a $
Does not include /, + or #
Does not include non-printable characters or whitespace other than the space character (U+0020).

7.3.5.7.2 MessageType mapping

<MqttMessageType> Topic levels exist for each MessageType defined in 7.2.2. Additional information like requirements for RETAIN for each Topic level is provided in Table 204. The handling of RETAIN messages is defined in 7.3.5.8.

The requirements for topic access permissions are defined in Table 205.

Table 204 – MQTT Topic level MessageType mapping

MessageType	MqttMessageType	RETAIN	Required	Specification Reference
DataSetMessage	data	False	Yes	Defined in 7.3.5.7.3. A system specific Topic may be used instead The RETAIN false is the default setting.
DataSetMetaData	metadata	True	Yes	Defined in 7.3.5.7.4. A system specific Topic may be used instead.
ApplicationDescription	application	True	No	Defined in 7.3.5.7.5.
ServerEndpoints	endpoints	True	No	Defined in 7.3.5.7.6.
Status	status	True	Yes	Defined in 7.3.5.7.7.
PubSubConnection	connection	True	Yes	Defined in 7.3.5.7.8.
ActionRequest	action-request	False	Yes	Defined in 7.3.5.7.9. A system specific Topic may be used instead
ActionResponse	action-response	False	No	Defined in 7.3.5.7.10. The ActionResponse topic can be specified by the Requestor.
ActionMetaData	action-metadata	True	Yes	Defined in 7.3.5.7.12. A system specific Topic may be used instead
ActionResponder	action-responder	True	Yes	Defined in 7.3.5.7.11.

Table 205 – MQTT Topic level access permissions

MqttMessageType	Publisher	Subscriber	Description
data	Write	Read	Variables and Events from an OPC UA applications acting as Publisher have RolePermissions. Such RolePermissions have no affect after DataSetMessages are sent to the MQTT broker. It is therefore recommended to synchronize Roles used to configure read permissions to the topics with the Roles required to access the information in the Publisher OPC UA application.
metadata	Write	Read
application	Write	Read	The information published with this message type is similar to discovery information provided with OPC UA Client Server discovery. This information is normally not restricted for read access.
endpoints	Write	Read
status	Write	Read
connection	Write	Read
action-request	Read	Write	Publisher is the Responder and Subscriber is the Requestor. The topic with message type action-request is defined by the Responder with its PublisherId but the Requestors must have write permission to the topic. Only the Responder should be able to read from the topic.
action-response	Write	Read	Publisher is the Responder and Subscriber is the Requestor. If the Responder specifies the response topic it must be ensured that the Responder has Write access to this topic. The Requestor should either use unique random correlation data or should use a private response topic where only the Requestor is able to read from.
action-metadata	Write	Read
action-responder	Write	Read

7.3.5.7.3 data Topic level

The data Topic has the form:

<Prefix>/<Encoding>/data/<PublisherId>/<WriterGroup>[/<DataSetWriter>]

The <PublisherId> Topic level is the PublisherId for the application sending the messages.

The <WriterGroup> Topic level is the name of a WriterGroup within the Publisher.

The <DataSetWriter> Topic level is the name of a DataSetWriter within the WriterGroup. If no QueueName is specified at the DataSetWriter level then the QueueName in WriterGroup TransportSettings is used and the DataSetWriter name is not part of the Topic.

The data Topic level is the default if the system owner does not have their own Topic tree. The Topic actually used is specified in the Connection Message as QueueName in the DataSetWriter or WriterGroup TransportSettings.

The messages are instances of the DataSetMessage MessageType (see 7.2.2).

The corresponding PubSubConnection Messag e is sent to the Topic with the same <Prefix>, <Encoding> and <PublisherId>. The PubSubConnection specifies the WriterGroups and DataSetWriters for a Publisher.

7.3.5.7.4 metadata Topic level

The metadata Topic has the form:

<Prefix>/<Encoding>/metadata/<PublisherId>/<WriterGroup>/<DataSetWriter>

The <PublisherId> Topic level is the PublisherId for the application sending the messages.

The <WriterGroup> Topic level is the name of a WriterGroup within the Publisher.

The <DataSetWriter> Topic level the name of a DataSetWriter within the WriterGroup.

The metadata Topic is the default if the system owner does not have their own Topic tree. The Topic actually used is specified in the Connection Message as MetaDataQueueName in the DataSetWriter TransportSettings.

The messages are instances of the DataSetMetaData MessageType (see 7.2.2).

7.3.5.7.5 application Topic level

The application Topic has the form:

<Prefix>/<Encoding>/application/<PublisherId>

The <PublisherId> Topic level is the PublisherId for the application sending the messages.

The messages are instances of the ApplicationDescription MessageType (see 7.2.2).

7.3.5.7.6 endpoints Topic level

The endpoints Topic has the form:

<Prefix>/<Encoding>/endpoints/<PublisherId>

The <PublisherId> Topic level is the PublisherId for the application sending the messages.

The messages are instances of the ServerEndpoints MessageType (see 7.2.2).

7.3.5.7.7 status Topic level

The status Topic has the form:

<Prefix>/<Encoding>/status/<PublisherId>

The <PublisherId> Topic level is the PublisherId for the application sending the messages.

The messages are instances of the Status MessageType (see 7.2.2).

A Publisher that is exclusively using a PublisherId shall register a Status message as an MQTT Will message when it creates the connection to the MQTT Broker. This message is sent automatically if the Publisher loses its connection with the MQTT Broker. The IsCyclic shall be FALSE in this case. The PubSubState value of the Will message shall be Error.

If a single MQTT client connection has multiple PubSubConnections (like for different encodings), not more than one PubSubConnection can register a Status message as MQTT Will message. All other PubSubConnections shall use cyclic Status messages.

7.3.5.7.8 connection Topic level

The connection Topic has the form:

<Prefix>/<Encoding>/connection/<PublisherId>

The <PublisherId> Topic level is the PublisherId for the application sending the messages. This value shall be the same as the PublisherId in PubSubConnection provided in the message.

The PublisherId in the PubSubConnection uniquely identifies the Publisher within the scope defined by the <Prefix>.

The TransportProfileUri in the PubSubConnection specifies the <Encoding> used for all messages for the combination of <Encoding>/<PublisherId>. Table 206 specifies the mapping between a TransportProfileUri and the encoding.

Table 206 – TransportProfileUri encodings

URI	Encoding
http://opcfoundation.org/UA-Profile/Transport/pubsub-mqtt-json	json
http://opcfoundation.org/UA-Profile/Transport/pubsub-mqtt-uadp	uadp

The messages are instances of the PubSubConnection MessageType (see 7.2.2).

7.3.5.7.9 action-request Topic level

The action-request Topic has the form:

<Prefix>/<Encoding>/action-request/<Responder>/<WriterGroup>

The <Responder> Topic level is the PublisherId for the Responder of the Actions.

The <WriterGroup> Topic level is the name of a WriterGroup.

The action-request Topic level is the default if the system owner does not have their own Topic tree. The Topic actually used is specified in the Responder message as QueueName in the WriterGroup TransportSettings.

The messages are instances of the ActionRequest MessageType (see 7.2.2).

The corresponding Responder message is sent to the Topic with the same <Prefix>, <Encoding> and <Responder>. The PubSubConnection specifies the WriterGroups and DataSetWriters for a Responder.

7.3.5.7.10 action-response Topic level

The action-response Topic has the form:

<Prefix>/<Encoding>/action-response/<Responder>/<WriterGroup>

The <Responder> Topic level is the PublisherId for the Responder of the Actions.

The <WriterGroup> Topic level is the name of a WriterGroup.

The action-response Topic level is the default if the ResponseAddress is not provided in the ActionRequest.

The messages are instances of the ActionResponse MessageType (see 7.2.2).

7.3.5.7.11 action-responder Topic level

The action-responder Topic has the form:

<Prefix>/<Encoding>/action-responder/<Responder>

The <Responder> Topic level is the PublisherId for the Responder of the Actions. This value shall be the same as the PublisherId in PubSubConnection provided in the message.

The PublisherId in the PubSubConnection uniquely identifies the Publisher within the scope defined by the <Prefix>. If it is a String, it should be as short as possible since long Topic names degrade performance.

The messages are instances of the PubSubConnection MessageType (see 7.2.2).

7.3.5.7.12 action-metadata Topic level

The action-metadata Topic has the form:

<Prefix>/<Encoding>/action-metadata/<Responder>/<WriterGroup>/<DataSetWriter>

The <Responder> Topic level is the PublisherId for the Responder of the Action.

The <Group> Topic level is the name of a WriterGroup.

The <Writer> Topic level the name of a DataSetWriter.

The action-metadata Topic is the default if the system owner does not have their own Topic tree. The Topic actually used is specified in the Responder message as MetaDataQueueName in the DataSetWriter TransportSettings.

The messages are instances of the ActionMetaData MessageType (see 7.2.2).

7.3.5.8 Message header

The default setting for the MQTT RETAIN flag are defined in Table 204. Table 208 defines an option to change the RETAIN flag setting for DataSetMessages.

A Publisher shall send all RETAIN discovery messages at start up of the Publisher. A Publisher shall update affected RETAIN topics if the Publisher configuration changes. A Publisher shall clear RETAIN topics if the discovery element is deleted from the Publisher configuration like reset the metadata topic if the related DataSetWriter is removed. A Publisher shall subscribe to its own discovery message topics at start-up and clear all topics that do not match the current Publisher configuration.

Publishers using MQTT version 3.1.1 shall clear RETAIN topics when they shut down.

Publishers using MQTT version 5.0 shall set the Message Expiry Interval on RETAIN topics and shall send a new RETAIN message before the interval expires.

The MQTT version 3.1.1 protocol does not support message headers. Any promoted field or additional fields defined on the WriterGroup or DataSetWriter other than RETAIN are not sent as MQTT message properties.

MQTT version 5.0 defines a number of standard message properties. These include properties explicitly defined in the MQTT specification, as well as the MQTT User Property which is a key-value pair of UTF-8 strings. The MQTT User Property is intended to provide a means of transferring application layer name-value tags whose meaning and interpretation are known only by the application programs responsible for sending and receiving them. They are used here to specify PubSub properties not directly supported by the MQTT protocol.

Table 207 describes how these properties shall be populated when a MQTT version 5.0 message is constructed.

Table 207 – OPC UA MQTT message properties

MQTT property name	MQTT property type	MQTT property value
UAMessageType	User Property	Valid values are “ua-<MqttMessageType>” where the MqttMessageTypes are defined in 7.3.5.7.2.
Content Type	Standard	The MIME type for the message body. The MIME types are specified in the message body subsections 7.3.5.9.1 and 7.3.5.9.2.

The MQTT message header shall include additional fields defined on the PubSubConnection, WriterGroup or DataSetWriter through the KeyValuePair array in the WriterGroupProperties and DataSetWriterProperties. The NamespaceIndex of the QualifiedName in the KeyValuePair shall be 0. The Name of the QualifiedName is constructed from a message prefix and the MQTT property name with the following syntax.

Name = <MqttMessageType>-<MQTT property name>

The Name of the key in the KeyValuePair shall have a prefix “message” followed by a hyphen and the MQTT property name.

Table 208 defines the MQTT standard message properties.

Table 208 – OPC UA MQTT standard message property configuration

MQTT property name	OPC UA DataTypes	MQTT data types	Description
RETAIN	Boolean	RETAIN bit in the header	RETAIN configuration for DataSetMessages.
Message Expiry Interval	UInt32	Four Byte Integer	Not available as message property for MQTT 3.1.1.

Any name not in the Table 208 is assumed to be a MQTT User Property.

When a field is added to the header as a MQTT User Property the value is encoded as UTF-8 encoded String. If the value is not a String, then it is encoded using the VerboseEncoding OPC UA JSON Data Encoding rules in OPC 10000-6. Promoted fields can only be sent for fields which are assumed to be a MQTT User Property and if the NetworkMessage contains only one DataSetMessage. The MQTT message header shall include additional promoted fields of the DataSet as a list of MQTT User Property name-value pairs. DataSet fields with the PromotedField flag set in the FieldMetaData fieldFlags are copied into the MQTT header. The FieldMetaData Structure is defined in 6.2.3.2.4. For a UADP message mapping the promoted fields are also included in the UADP NetworkMessage. Promoted fields shall always be included in the header even if the DataSetMessage body is a delta frame and the DataSet field is not included in the delta frame. In this case the last known value is sent in the header.

7.3.5.9 Message body

7.3.5.9.1 JSON message mapping

A JSON body is encoded as defined for the JSON message mapping defined in 7.2.4.6.9.

When sending a MQTT Version 5.0 message the MQTT ContentType property shall be set to application/json when sending uncompressed JSON messages.

JSON messages can become quite large. In order to save bandwidth and to reduce message size, on MQTT Version 5.0 the MQTT Content Type property allows to select a compression type as encoding for a JSON message.

When sending a gzip (RFC 1952) compressed JSON message on MQTT Version 5.0 the MQTT ContentType property shall be set to application/json+gzip. If a Subscriber receives messages without MQTT ContentType from MQTT Version 3.1.1 Publishers it may require manual configuration.

7.3.5.9.2 UADP message mapping

A UADP body is encoded as defined for the UADP message mapping defined in 7.2.3.

It is expected that the software used to receive UADP NetworkMessage can process the body without needing to know how it was transported.

If the encoded MQTT message size exceeds the Broker limits, it is broken into multiple chunks as described in 7.2.4.4.4.

When sending such message over MQTT Version 5.0 the ContentType property shall be set to application/opcua+uadp.