|
@@ -5,7 +5,7 @@
|
|
|
In EdgeX Geneva, [LF Edge eKuiper - an SQL based rule engine](https://github.com/lf-edge/ekuiper) is integrated with EdgeX. Before diving into this tutorial, let's spend a little time on learning basic knowledge of eKuiper. eKuiper is an edge lightweight IoT data analytics / streaming software implemented by Golang, and it can be run at all kinds of resource constrained edge devices. eKuiper rules are based on `Source`, `SQL` and `Sink`.
|
|
|
|
|
|
- Source: The data source of streaming data, such as data from MQTT broker. In EdgeX scenario, the data source is EdgeX message bus, which could be ZeroMQ or MQTT broker.
|
|
|
-- SQL: SQL is where you specify the business logic of streaming data processing. eKuiper provides SQL-like statements to allow you to extract, filter & transform data.
|
|
|
+- SQL: SQL is where you specify the business logic of streaming data processing. eKuiper provides SQL-like statements to allow you to extract, filter & transform data.
|
|
|
- Sink: Sink is used for sending analysis result to a specified target. For example, send analysis result to another MQTT broker, or an HTTP rest address.
|
|
|
|
|
|

|
|
@@ -22,7 +22,7 @@ The tutorial demonstrates how to use eKuiper to process the data from EdgeX mess
|
|
|
|
|
|
## eKuiper EdgeX integration
|
|
|
|
|
|
-EdgeX uses [message bus](https://github.com/edgexfoundry/go-mod-messaging) to exchange information between different micro services. It contains the abstract message bus interface and implementations for ZeroMQ & MQTT. The integration work for eKuiper & EdgeX includes following 3 parts.
|
|
|
+EdgeX uses [message bus](https://github.com/edgexfoundry/go-mod-messaging) to exchange information between different micro services. It contains the abstract message bus interface and implementations for ZeroMQ & MQTT. The integration work for eKuiper & EdgeX includes following 3 parts.
|
|
|
|
|
|
- An EdgeX message bus source is extended to support consuming data from EdgeX message bus.
|
|
|
|
|
@@ -34,7 +34,7 @@ EdgeX uses [message bus](https://github.com/edgexfoundry/go-mod-messaging) to ex
|
|
|
|
|
|
However, data type definitions are already specified in the EdgeX events/readings and to improve the using experience, user are NOT necessary to specify data types when creating stream. For any data sending from message bus, it will be converted into [corresponding data types](../guide/sources/builtin/edgex.md).
|
|
|
|
|
|
-- An EdgeX message bus sink is extended to support send analysis result back to EdgeX Message Bus. User can also choose to send analysis result to RestAPI, eKuiper already supported it.
|
|
|
+- An EdgeX message bus sink is extended to support send analysis result back to EdgeX Message Bus. User can also choose to send analysis result to RestAPI, eKuiper already supported it.
|
|
|
|
|
|

|
|
|
|
|
@@ -51,7 +51,7 @@ In out tutorial, we will use [Random Integer Device Service](https://github.com/
|
|
|
|
|
|
### Run EdgeX Docker instances
|
|
|
|
|
|
-Go to [EdgeX-compose project](https://github.com/edgexfoundry/edgex-compose), and download related Docker compose file for Ireland release, then bring up EdgeX Docker instances.
|
|
|
+Go to [EdgeX-compose project](https://github.com/edgexfoundry/edgex-compose), and download related Docker compose file for Ireland release, then bring up EdgeX Docker instances.
|
|
|
|
|
|
```shell
|
|
|
$ docker-compose -f ./docker-compose-no-secty.yml up -d --build
|
|
@@ -81,9 +81,11 @@ d4b236a7b561 redis:6.2.4-alpine "
|
|
|
|
|
|
When eKuiper gets data from messageBus and send back the processed result, user needs to specify the connection info separately when creating the source and sink.
|
|
|
Since `eKuiper 1.4.0` and `EdgeX Jakarta`, there is a new feature that user can specify the connection info in a fixed place and then source and sink can make a reference to it.
|
|
|
-* `redis` messageBus: this is especially useful when EdgeX use `secure` mode, in which case the client credentials will be injected into that share place automatically when services bootstrap.
|
|
|
+
|
|
|
+- `redis` messageBus: this is especially useful when EdgeX use `secure` mode, in which case the client credentials will be injected into that share place automatically when services bootstrap.
|
|
|
In order to use this feature, users need do some modifications on the target `docker-compose` file's `rulesengine` service part
|
|
|
-add these in `environment` part and make sure the image is `1.4.0` or later.
|
|
|
+add these in `environment` part and make sure the image is `1.4.0` or later.
|
|
|
+
|
|
|
```yaml
|
|
|
environment:
|
|
|
CONNECTION__EDGEX__REDISMSGBUS__PORT: 6379
|
|
@@ -93,8 +95,9 @@ add these in `environment` part and make sure the image is `1.4.0` or later.
|
|
|
EDGEX__DEFAULT__CONNECTIONSELECTOR: edgex.redisMsgBus
|
|
|
```
|
|
|
|
|
|
-* `mqtt/zeromq` messageBus: adjust the parameters accordingly and specify the client credentials if have.
|
|
|
- There is a `mqtt` message bus example, make sure the connection info exists in `etc/connections/connection.yaml`, for [more info](../guide/sources/builtin/edgex.md#connectionselector) please check this.
|
|
|
+- `mqtt/zeromq` messageBus: adjust the parameters accordingly and specify the client credentials if have.
|
|
|
+ There is a `mqtt` message bus example, make sure the connection info exists in `etc/connections/connection.yaml`, for [more info](../guide/sources/builtin/edgex.md#connectionselector) please check this.
|
|
|
+
|
|
|
```yaml
|
|
|
environment:
|
|
|
CONNECTION__EDGEX__MQTTMSGBUS__PORT: 1883
|
|
@@ -105,12 +108,14 @@ add these in `environment` part and make sure the image is `1.4.0` or later.
|
|
|
CONNECTION__EDGEX__MQTTMSGBUS__OPTIONAL__PASSWORD: password
|
|
|
EDGEX__DEFAULT__CONNECTIONSELECTOR: edgex.mqttMsgBus
|
|
|
```
|
|
|
+
|
|
|
After these modifications and eKuiper starts up, please read [this](../guide/sinks/builtin/edgex.md#connection-reuse-publish-example) to learn how to refer to the connection info
|
|
|
|
|
|
#### Use Redis as KV storage
|
|
|
|
|
|
Since `1.4.0`, eKuiper supports redis to store the KV metadata, user can make some modifications on the target `docker-compose` file's `rulesengine` service part to apply this change.
|
|
|
Users can add these in `environment` part and make sure the image is `1.4.0` or later.
|
|
|
+
|
|
|
```yaml
|
|
|
environment:
|
|
|
KUIPER__STORE__TYPE: redis
|
|
@@ -118,6 +123,7 @@ Users can add these in `environment` part and make sure the image is `1.4.0` or
|
|
|
KUIPER__STORE__REDIS__PORT: 6379
|
|
|
KUIPER__STORE__REDIS__PASSWORD: ""
|
|
|
```
|
|
|
+
|
|
|
*Note*: This feature only works when redis in `no-secty` mode
|
|
|
|
|
|
#### Run with native
|
|
@@ -187,11 +193,11 @@ For more detailed information of configuration file, please refer to [this doc](
|
|
|
|
|
|
### Create a rule
|
|
|
|
|
|
-Let's create a rule that send result data to an MQTT broker, for detailed information of MQTT sink, please refer to [this link](../guide/sinks/builtin/mqtt.md). Similar to create a stream, you can also choose REST or CLI to manage rules.
|
|
|
+Let's create a rule that send result data to an MQTT broker, for detailed information of MQTT sink, please refer to [this link](../guide/sinks/builtin/mqtt.md). Similar to create a stream, you can also choose REST or CLI to manage rules.
|
|
|
|
|
|
-So the below rule will get all of values from `event` topic. The sink result will
|
|
|
+So the below rule will get all of values from `event` topic. The sink result will
|
|
|
|
|
|
-- Published to topic `result` of public MQTT broker `broker.emqx.io`.
|
|
|
+- Published to topic `result` of public MQTT broker `broker.emqx.io`.
|
|
|
- Print to log file.
|
|
|
|
|
|
#### Option 1: Use Rest API
|
|
@@ -254,10 +260,10 @@ Rule rule1 was created successfully, please use 'cli getstatus rule rule1' comma
|
|
|
If you want to send analysis result to another sink, please refer to [other sinks](../guide/sinks/overview.md)
|
|
|
that supported in eKuiper.
|
|
|
|
|
|
-Now you can also take a look at the log file under `log/stream.log`, or through command `docker logs edgex-kuiper `
|
|
|
+Now you can also take a look at the log file under `log/stream.log`, or through command `docker logs edgex-kuiper`
|
|
|
to see detailed info of rule.
|
|
|
|
|
|
-```
|
|
|
+```text
|
|
|
time="2021-07-08 01:03:08" level=info msg="Serving kuiper (version - 1.2.1) on port 20498, and restful api on http://0.0.0.0:59720. \n" file="server/server.go:144"
|
|
|
Serving kuiper (version - 1.2.1) on port 20498, and restful api on http://0.0.0.0:59720.
|
|
|
time="2021-07-08 01:08:14" level=info msg="Successfully subscribed to edgex messagebus topic rules-events." file="extensions/edgex_source.go:111" rule=rule1
|
|
@@ -333,13 +339,13 @@ Connecting to 127.0.0.1:20498...
|
|
|
|
|
|
In this tutorial, we introduce a very simple use of EdgeX eKuiper rule engine. If having any issues regarding to use of eKuiper rule engine, you can open issues in EdgeX or eKuiper Github respository.
|
|
|
|
|
|
-### More Excecise
|
|
|
+### More Excecise
|
|
|
|
|
|
Current rule does not filter any data that are sent to eKuiper, so how to filter data? Please [drop rule](../api/cli/rules.md) and change the SQL in previous rule accordingly. After update the rule file, and then deploy the rule again. Please monitor the `result` topic of MQTT broker, and please verify see if the rule works or not.
|
|
|
|
|
|
#### Extended Reading
|
|
|
|
|
|
-- Starting from eKuiper 0.9.1 version, [a visualized web UI](../operation/manager-ui/overview.md) is released with a separated Docker image. You can manage the streams, rules and plugins through web page.
|
|
|
+- Starting from eKuiper 0.9.1 version, [a visualized web UI](../operation/manager-ui/overview.md) is released with a separated Docker image. You can manage the streams, rules and plugins through web page.
|
|
|
- Read [EdgeX source](../guide/sources/builtin/edgex.md) for more detailed information of configurations and data type conversion.
|
|
|
- [How to use meta function to extract additional data from EdgeX message bus?](edgex_meta.md) There are some other information are sent along with device service, such as event created time, event id etc. If you want to use such metadata information in your SQL statements, please refer to this doc.
|
|
|
- [Use Golang template to customize analaysis result in eKuiper](../guide/sinks/data_template.md) Before the analysis result is sent to different sinks, the data template can be used to make more processing. You can refer to this doc for more scenarios of using data templates.
|
|
@@ -350,4 +356,3 @@ Current rule does not filter any data that are sent to eKuiper, so how to filter
|
|
|
|
|
|
- [eKuiper Github code repository](https://github.com/lf-edge/ekuiper/)
|
|
|
- [eKuiper reference guide](../guide/streams/overview.md)
|
|
|
-
|