Making an IoT Developer’s Life Easier With Eclipse IoT Packages

As an IoT developer, one is often tasked with putting together a solution that includes one or more open source components. I remember, even as far back as 2014, using components like Eclipse Mosquitto MQTT broker and Eclipse Paho MQTT client for a pilot project with IoT Gateway at Intel. Fast forward a few years at Red Hat, where I used components like Eclipse Kura and Eclipse Kapua for a European industrial automation project. Without realizing it then, I was using these components from Eclipse IoT open source projects.

Image courtesy of Eclipse Foundation

There are hundreds of available open source projects providing IoT capabilities. Just Eclipse IoT workgroup itself hosts several dozen open source projects that developers can use for their IoT projects. The first challenge a developer faces, therefore, is more about choosing the right component for their project from a plethora of choices. Kai Hudalla has a nice article on the challenges of selecting the right project.

Once the appropriate projects are selected, the challenge becomes integrating the various projects into an end-to-end solution. My colleague Jens Reimann has written about these challenges in this article. To alleviate these developer challenges, the Eclipse IoT Packages project bundles a selection of these projects so they can be used as ready-to-use implementations by the developer. To learn more about the Eclipse IoT Packages project, check out this article.

In this blog series, we’ll be looking at the Eclipse IoT Packages by exploring the Telemetry end-to-end package. As the name suggests, this package provides a complete end-to-end solution from acquiring sensor data at the edge, sending it to the centralized site where it’s stored and visualized. This article will describe the architecture and functionality of this package, and the next article will walk through the hands-on experience deploying this package.

Telemetry End-to-End Package Architecture

The following open source projects are included in this package:

• Drogue Device: Framework for developing applications that run on tiny edge devices
• Eclipse Kura: IoT gateway for bridging the connectivity from the edge devices to the Cloud
• Drogue Cloud: Connectivity layer to send telemetry data from edge devices and control commands to edge devices
• Eclipse Ditto: Digital twin capability to mirror physical devices in their digital representation
• Apache Kafka: Event streaming platform
• Eclipse Streamsheets: Visualize the data from event streams

Drogue Device

Edge devices are the sources of IoT data. SCADA systems connect to devices and other sensors/actuators generating the data. These devices are small form-factor, based on 32-bit microcontrollers, may not be TCP/IP capable, and don’t need a full-fledged operating system. Drogue Device is the firmware framework suitable for such devices as Raspberry Pi Pico. Drogue Device is written in Rust and uses an actor-based programming model to create applications.

Learn more about Drogue Device.

Eclipse Kura

The IoT environment for an enterprise consists of brownfield devices with legacy interfaces and greenfield devices with modern interfaces like TCP/IP. In addition, there are industry-specific protocols in use for each given industry. Before the IoT data can be processed, it needs to be accessed from these field devices. This mandates the need for an IoT Gateway. Eclipse Kura enables API access to hardware interfaces (serial port, GPIO, I2C) and field protocols (Modbus, MQTT, OPC-UA) so the data can be acquired from devices and sent to a cloud platform for further processing. 

Learn more about Eclipse Kura.

Drogue Cloud

An IoT platform is needed on the cloud side that can ingest data from IoT devices and Gateways over various protocols and send control commands back to IoT devices. This platform should be able to run on any chosen cloud service provider or on-premise. Drogue IoT Cloud is the IoT platform being used for this solution. 

Learn more about Drogue Cloud.

Eclipse Ditto

There are several advantages to providing an abstraction layer for an IoT solution to interact with the physical IoT devices. Applications needing to interact with an IoT device can instead communicate with its abstraction called a digital twin. This allows applications to interact with the digital twin asynchronously even while the physical device is offline. A digital twin can be used to bring the physical device to its desired state and publish state changes. Eclipse Ditto provides the digital twin capability. 

Learn more about Eclipse Ditto.

Apache Kafka

The real-time data from IoT devices need to be routed to applications needing this data. A message broker that can consume the event stream from devices and then route it to applications that can further process this data. Apache Kafka is a high-performance event streaming platform for data pipelines, streaming analytics, and data integration. 

Learn more about Apache Kafka.

Eclipse Streamsheets

In order to gain insights, the IoT data needs to be visualized first. There are many available visualization tools, but they require prior programming knowledge and expertise to deal with the event stream. Eclipse Streamsheets allows users to create a visualization or process event stream; for example, protocol conversion using their existing spreadsheet skills. 

Learn more about Eclipse Streamsheets.

Making It All Work Together

The Telemetry end-to-end package shows the complete workflow for a typical enterprise IoT solution: telemetry data from the edge is acquired and sent to the Cloud for further analysis, storage, and visualization.

Drogue Devices at the edge location monitor environmental conditions such as temperature, vibration, pressure, and location. This data is then transmitted over a low-power wireless protocol such as Bluetooth to the IoT Gateway.

IoT Gateway running Eclipse Kura receives this telemetry data over a local Bluetooth network and sends it to the Cloud over TCP/IP. The cloud connector service in Eclipse Kura is responsible for sending the telemetry data to MQTT end-point in the Cloud. It’s also responsible for re-connection and data buffering if the network connection is disrupted.

Drogue Cloud as an IoT platform serves as the MQTT end-point for the Kura cloud connector. It authenticates the connection and waits for the message to be published. Once a message is received, it is then forwarded to the relevant Apache Kafka topic over Kafka protocol.

Apache Kafka instance receives the messages from Drogue Cloud in Kafka protocol and publishes them to Eclipse Ditto.

Eclipse Ditto is responsible for receiving the message from Kafka, normalizing the data structure, and sending changes to Streamsheets. When a new event is received from Kafka, the message is decoded, sensor information extracted, and translated into a Ditto protocol message. As a digital twin, Ditto normalizes the data structure of messages received from Drogue Cloud. Ditto also updates the state of the twin and sends out any change event that was created in the process to Streamsheets.

Streamsheets receives the change events using its internal MQTT broker, updates the spreadsheet, and refreshes the visualization to reflect the current state of the device.

The above solution only shows a subset of the capabilities of these components. Also, a real-world deployment may use a different selection of components depending on their needs. Once again, my colleague discusses some of these alternatives here. 

Coming up next…

In the next article, we’ll walk through the step-by-step process to deploy the Telemetry end-to-end package.