Many of the articles written about Lucid, try and explain the protocol and its uses from the users’ point of view. This is one of the first articles to take a developer-centred perspective. In this article we will look at subjects surrounding the development of a new product, or an extension of an existing product, so that it contains Lucid. We’ll also look at how Lucid intrinsically supports easy and efficient incorporation of itself into a vendor’s product.
Lucid is a protocol based on MQTT (Message Queue Telemetry Transport) which permits devices to send information about their state and configuration, and to receive instructions. The protocol bridges a gap between Operational Technology (OT) and IoT technology. For more information about Lucid please see our high level description of the protocol or our list of introductory articles.
In this article we give examples of tools or stacks that are available to support the implementation of Lucid. Please note that these are just examples and not recommendations. Implementors are free to use whichever tools they wish.
Main aims of Lucid
The main aims of developing the Lucid protocol were:
- To provide a simple protocol which would permit mass deployment of IoT devices within an OT Network.
- To make the protocol inexpensive to implement.
- To utilise common and freely available standards.
- To use inexpensive and popularly used tooling.
- To ensure good support for testing.
The following sections will break down how these aims were met by the protocol and what this means to developers.
Common underlying standards
Lucid is based on two very common, publically available standards: MQTT and JSON (JavaScript Object Notation). MQTT is used as the transport mechanism for the messages sent by the protocol and JSON is used to encode the messages sent over MQTT. The protocol also uses JSON Schema to help formalise how the JSON objects are encoded. The ubiquitous nature of these standards means that they have good resources available online to support them.
Free and commonly available standards implementations
There are several implementations of MQTT, JSON and JSON Schema available on the internet. These implementations often cost nothing to download and use and are popular with developers. For example, with MQTT there is Eclipse Paho, for JSON there are implementations like cJSON and json-c. These examples are in C which is a popular language for embedded or IoT systems; however, code is available in other languages such as C#, C++, GoLang, Java, Javascript, Python and Rust. See the Paho MQTT Download page for an indication of how broad language implementation has been.
In some cases, the platform on which the product is based may have its own development kits or use a standard operating system and often these will have JSON and MQTT support “baked in”. For example, the Zephyr RTOS which is a popular choice for some platforms includes both an MQTT library and support for JSON.
As well as the implementations of the standards that can be used in a product there is a rich ecosystem of tools surrounding these which assist in the use and testing of the standards. So, for instance, as a few examples:
- Free, good quality learning material such as MQTT Essentials from HiveMQ.
- Open source MQTT brokers if you want to install and run your own, like Mosquitto.
- Free-to-use online MQTT brokers such as Mosquitto Test Broker or HiveMQ Test Broker.
- MQTT graphical client tools such as MQTT Explorer.
- Tools to help working with JSON such as JSON Utils.
- Implementations and tools relating to JSON Schema are detailed on the JSON Schema website Implementations page, like JSON Schema Validator.
There are others available. Simple searches on the web will produce many examples that can be used.
This number and quality of resources is only available on the internet because of the popularity of MQTT and JSON. This in turn means there are many coders out there with good experience in these and so finding contractors or staff to help on the projects should be straightforward.
Full protocol specification to be freely available
Beyond the resources for things such as MQTT, JSON and JSON Schema the only extra resource required is the Lucid protocol specification. This fully describes how the standards are used to provide the full protocol.
Over the coming months WITS will be releasing the protocol specification on a new website for anyone to view and use. Currently the protocol specification is fully available to any member of WITS. If you are interested in learning more on this, please contact WITS through this webpage.
Alongside the protocol specification WITS will also be releasing some supporting JSON and a JSON Schema which will make working with the protocol easier. This will be made available through GitHub once the protocol specification is openly published on the Lucid website.
Protocol feature selection allowing easy MVP development
Protocol clients can specify what feature of the protocol they support through a Device Profile. So, an initial device might be designed which supports very little of the functionality of Lucid as a Minimum Viable Product (MVP). This allows for a quicker entry to the marketplace but leaves open the option of extending the device’s capabilities at a later date.
Common test facilities to be available within WITS
To ensure that testing of devices can easily be performed by users or vendors, the Lucid development community is proposing the generation of a common set of tests which could be run against Field Devices (FDs) or Supervisory Applications (SAs). This would be a communally-developed piece of software starting with a few tests at first and growing as members extend it to cater for more cases. It is expected that this will be available only to WITS members at first. If you are interested in this then please consider joining WITS.
Supportive development community within WITS
The entire development of Lucid flowed from ideas generated during the adoption of WITS-DNP3; see this article for more information on that. The development was undertaken by a set of volunteers from the WITS community. WITS are keen to promote Lucid and hence it is expected there will be good community support to assist with development.
WITS-sponsored Plugfests at which products can be tested
During the development of Lucid, volunteers met face-to-face to test the protocol during WITS-DNP3 “plugfests”. WITS-DNP3 plugfests are events where all Field Device and Master Station vendors meet in person to test that their products interoperate using the WITS-DNP3 standard. These offer an excellent opportunity to meet the developers from other companies and quickly resolve issues with the protocol or one’s products.
We now expect to extend all future plugfests to permit testing of both WITS-DNP3 and Lucid. Being a member of WITS will mean you are invited to the next Plugfest (which at the time of writing is in June 2024). It will provide an opportunity to see what products are available, ask questions on the protocol and try your products out, not to mention meeting and building relationships with other vendors and users in the Lucid space.
Summary
This article has laid out the aspects of the Lucid Protocol which should make the protocol efficient and easy to implement. These include:
- Development tools and products for the standards which underpin Lucid are freely and commonly available.
- Staff with skills in the underlying standards will be easier to come by than staff with skills in some of the more esoteric standards that exist within the IoT / OT space.
- The protocol specification and supporting material will be freely available when published. No need to join WITS if you do not wish to.
- There is a strong supportive community within WITS.
Good luck on your journey with Lucid.
Dave Howarth (Northumbrian Water) and Mark Davison (Terzo Digital)
February 2024
Please see our Lucid reading page for a collated list of other articles on Lucid.