Lucid is a protocol based on MQTT which permits devices to send information about their state and configuration, and to receive instructions. The protocol bridges a gap between Operational Technology (OT) and IoT technology. For more information about Lucid please see our high level description of the protocol or our list of introductory articles.
This article outlines some of the different ways in which the Lucid protocol can be used by an organisation. Lucid’s use of MQTT to transport messages has opened up new deployment options from what was available in WITS-DNP3 (a related protocol) and standard telemetry. The article presents several different deployment scenarios. Each scenario is accompanied by a small discussion of why that scenario may be useful. The aim of the article is to encourage new thoughts in relation to how the Lucid protocol can be used by both users and vendors within OT.
Introduction
The article may seem long, but it is mostly architectural diagrams, and in fact, mostly the same diagram with different parts highlighted to show the different architectural options. This section introduces this main diagram and describe its parts. It also shows the standard WITS-DNP3 scenario, so that users familiar with that, may easily appreciate the differences that are possible with Lucid.
Figure 1 – The base architectural diagram that is used throughout this article.
The base diagram shows all the elements used in various scenarios to describe some of the ways the Lucid protocol can be used. Where some scenarios do not use parts of the diagram, they will be greyed out. Lines coloured red show Lucid protocol data flows, those in black are non-Lucid flows. The bottom of the diagram covers the Field Devices (FDs). Moving up the diagram we move through progressive levels of communications or storage servers eventually reaching the user’s data lake or data warehouse at the top level. Note that the term data lake/warehouse is very loosely used in this sense to mean any data storing application, so for example it could be an OT historian or just a standard relational database. The Supervisory Application (SA) is the term used within Lucid for what might traditionally in OT be called the Master Station or Data Gatherer. The diagram is broken into user owned and managed equipment on the left coloured in green and third party owned and managed equipment on the right coloured in grey.
0 “Standard / Traditional” Telemetry
Figure 2 – The standard WITS-DNP3 process
Both in the WITS-DNP3 case or in the case of many other telemetry protocols the Field Device (FD) communicates directly with the Supervisory Application (SA). The SA passes data it collects on to the user’s data lake/warehouse or long-term storage. An analysis process can extract data from the data lake/warehouse and use it to add additional value to the collected data. The layers above the SA could also action a change and push that back to the FD via the SA.
Scenario 1 – Standard Lucid
Figure 3 – The normal Lucid scenario
In the originally envisaged implementation of Lucid the user may deploy a Lucid FD which may communicate intermittently with an MQTT broker within the user’s estate (either on alarm or on a scheduled basis). The user’s SA will likely remain permanently connected to the MQTT broker, subscribed to listen to information being sent from the Lucid FD. The SA could send relevant information on to the data lake/warehouse, although optionally this could be directly read from the MQTT broker by the data lake/warehouse if it supports Lucid. Likewise, the analysis tool could get data from the SA or directly from the MQTT broker and send instructions to the SA for sending to the FD or straight to the MQTT broker.
An important part of Lucid and something which marks it apart from other similar protocols is that devices send both their capabilities and their current configuration to the SA allowing the SA to understand how the device is configured and if necessary, create an entry for the device including all configuration, within its own database. If the Lucid device has the capability it may also accept changes to configuration through the MQTT server from the SA. This would allow, for example, alarm levels to be changed remotely.
Scenario 2 – 3rd Party Lucid FD
Figure 4 – Using a 3rd party Lucid FD
This case is almost exactly like the previous case except that the FD is not owned directly by the user. Instead, it remains owned by the 3rd party. Maybe the device is rented to the user, or maybe the device is owned by another organisation who are just making the data available to the user.
Because Lucid devices publish to an MQTT broker and because of deliberate design in the protocol, Lucid can support multiple SAs. So one could easily imagine extending this diagram to show more than one SA, perhaps one within the 3rd parties estate. This is what some of the following scenarios will build up to.
Scenario 3 – 3rd Party Lucid FD and broker
Figure 5 – Using a 3rd party Lucid FD with a third party MQTT broker
In this case a 3rd party Lucid FD communicates initially with an MQTT broker owned by the 3rd party. This would allow the 3rd party to deploy their own SA to monitor and control their FD. Meanwhile the Lucid messages could also be made available directly to an SA within the user’s domain or via an MQTT broker within the user’s domain to the user’s SA. The messages permitted out of the 3rd party’s broker could be filtered to only permit certain topics to be utilised.
One scenario envisaged is where a 3rd party offers an “OT data service” to a user – the user pays for a service to receive operational data, and it’s the 3rd party’s job to ensure the data arrives there against an agreed service level. Lucid FDs would be deployed and connected to the 3rd party MQTT broker and hence the 3rd third party’s SA. The 3rd party could use their SA to monitor things like battery life and communications, allowing them to service the devices as appropriate in order to keep the estate functioning to meet their SLA. The user could be given access to the sensor data being produced by the FD and use that to make decisions about the processes being monitored by the device. This might include giving the user the ability to set alarm limits, change alarm profiles or directly drive process points on the unit.
One might argue that this has been available in the past, and to a degree it has. Many different 3rd party systems make their data available to users, but often through proprietary interfaces which require the user to integrate these somehow into their systems. Once the user has more than a few of these different systems the burden of managing multiple interfaces can become onerous. Lucid provides the following advantages:
- Easily manage data integration into the user’s own systems.
- The ability to show data on the user’s normal applications rather than forcing the users to utilise yet another unique interface to control yet another adopted system.
- The ability to share more than a normal data feed would allow. For example, configuration, such as alarm levels, could be controlled.
- Because the interface is a known quantity (Lucid) the user is more easily able to consider sharing or transferring the service to another provider.
Scenario 4 – 3rd Party Lucid FD and broker with analysis
Figure 6 – Using a 3rd party Lucid FD and MQTT broker with analysis
Although similar to the previous example, this scenario adds analysis capability to the 3rd party’s offering. This might allow them to perform analysis and possibly even control (although this may have security implications) on the FDs independently of the user, whilst still making that data easily available to the user.
An example where this may be appropriate would be in 3rd party pipeline acoustic monitoring systems. These systems could include acoustic sensors as Lucid devices. The 3rd party’s SA or analysis software could implement proprietary algorithms for combining the acoustic data to make pipeline health information available to the user. This combined information could also be provided through Lucid.
The user, meanwhile, maintains visibility of what is happening and of alarms that the system may be generating. This visibility can be provided through their own management and control systems which the user’s staff are used to.
Scenario 5 – User’s Lucid FD publishing to a 3rd party broker
Figure 7 – A user Lucid FD publishing to both user and 3rd party SAs
This scenario turns around the previous scenario by placing FD ownership with the user, but allowing data from the FD to be shared with a third party so that they could, for instance, monitor and maintain the device on behalf of the user. Alternatively this could be a party such as a regulator who is interested in the data for reporting and monitoring purposes but is not interested in managing the device. Once again, topic filtering between the MQTT brokers could be used to control which topics a third party has access to.
Scenario 6 – 3rd Party Lucid FD via SA and broker
Figure 8 – 3rd party FD emulated as a Lucid FD via the 3rd party SA
This scenario permits all the advantages of the previous scenarios in which a 3rd party was managing and controlling aspects of the FD estate for the user, but allows the 3rd party to continue using whatever its own proprietary protocol was. The 3rd party vendor doesn’t need to re-engineer the interface to its field devices. Instead, it could develop a Lucid interface on its SA which is capable of publishing the field device data. The user would be unaware that the devices were being monitored and controlled by a proprietary protocol; instead, the 3rd party’s system would appear as if it were a large number of Lucid FDs all communicating through the client’s broker.
One disadvantage of this system for users is that they are locked to the specific 3rd party’s system. However, they do get the other advantages (mentioned above) of easier interface integration and use of their own systems to monitor the 3rd party system. For a 3rd party vendor, the big advantage is that they can develop a relatively small extension to their system to enable this level of integration with the client’s system and can leave all of their FDs untouched.
Those developing the Lucid standard expect the first projects to use Lucid will be based on this scenario, where an old system-to-system interface, currently implemented by a number of users, will be replaced with a Lucid interface. The projects are attractive to users as they can deliver the same outcomes without any change to the underlying estate of FDs (all of which will remain using their proprietary protocols).
Scenario 7 – 3rd party SA direct to user’s data store
Figure 9 – 3rd party SA direct to a user’s data store
In some cases, a user’s OT system may not have Lucid capabilities but may be planning to add Lucid in the future. Meanwhile, they may be receiving data from a 3rd party’s system through some pre-existing non-Lucid interface, as shown in the rightmost part of Figure 9.
The 3rd party could then develop a Lucid SA and FDs, possibly because of work with other users. The users may then have the option of starting to use Lucid FDs whilst still receiving data across the pre-existing interface they are currently using.
Ultimately, when the user’s Lucid SA is ready it can start subscribing to the 3rd party’s broker, leaving the FDs untouched. This gives an advantage to the vendor in that they can support users who wish to start using Lucid FDs before their SA is Lucid-capable. It also permits the user to transition to a Lucid system more easily, without any loss of data or a requirement to reconfigure all devices.
Summary
The seven different scenarios presented above have shown the different types of deployment that are now possible using the flexibility of the Lucid protocol, with its use of MQTT brokers. The aim of the article has been to encourage users and vendors to think about and seek new ways of using their devices and systems with the Lucid protocol to meet different technical and business needs.
Some of the main take-away points from the above scenarios are:
- It is possible to have multiple SAs.
- SAs do not need to be the conventional Master Stations of WITS-DNP3. SAs could be:
- Built into a 3rd party vendor’s own systems.
- Built as add-ons by the 3rd party vendor to act as a bridge between their own proprietary system and an end user’s system.
- Add-ons to standard OT systems (for example OSI Pi or Ignition).
- FDs could continue to implement proprietary protocols, communicating with an interim server which then appears to another broker as if it were multiple field devices. This approach provides a bridge between a 3rd party vendor’s own system and a user’s system which expects to ingest data in Lucid format.
- The protocol allows for different business models compared to those provided by conventional telemetry. For instance, 3rd parties managing and running an estate of field devices whilst sensor data is presented to the user and can be controlled by the user.
If any of this has given you food for thought or you would like to talk through some of the ideas, please consider joining the WITS-PSA or contacting people within the Lucid community to discuss further.
Dave Howarth (Northumbrian Water) and Mark Davison (Terzo Digital)
February 2024
Please see our Lucid reading page for a collated list of other articles on Lucid.