ekuiper-new/docs/en_US/installation.md

Installation

eKuiper provides docker image, binary packages and helm chart to install.

Running eKuiper in Docker

Please make sure docker has installed before running.

1. Get docker image.

   docker pull lfedge/ekuiper:x.x.x

1. Start docker container.

   docker run -p 9081:9081 -d --name kuiper -e MQTT_SOURCE__DEFAULT__SERVER=tcp://broker.emqx.io:1883 lfedge/ekuiper:xxx

In this example, we specify the default MQTT broker via environment variable to broker.emqx.io, which is a public MQTT test server hosted by EMQ.

For more configuration and docker image tags, please check lfedge/ekuiper in docker hub.

Running eKuiper with management console

eKuiper manager is a free eKuiper management web console which is provided as a docker image. We can use docker compose to run both eKuiper and eKuiper manager at once to ease the usage.

Please make sure docker compose has installed before running.

1. Create docker-compose.yaml file.

   version: '3.4'

   services:
       manager:
          image: emqx/ekuiper-manager:x.x.x
          container_name: ekuiper-manager
          ports:
          - "9082:9082"
          restart: unless-stopped
          environment:
            # setting default eKuiper service, works since 1.8.0
            DEFAULT_EKUIPER_ENDPOINT: "http://ekuiper:9081"
       ekuiper:
          image: lfedge/ekuiper:x.x.x
          ports:
            - "9081:9081"
            - "127.0.0.1:20498:20498"
          container_name: ekuiper
          hostname: ekuiper
          restart: unless-stopped
          user: root
          volumes:
            - /tmp/data:/kuiper/data
            - /tmp/log:/kuiper/log
          environment:
            MQTT_SOURCE__DEFAULT__SERVER: "tcp://broker.emqx.io:1883"
            KUIPER__BASIC__CONSOLELOG: "true"
            KUIPER__BASIC__IGNORECASE: "false"
     ```

2. Start docker-compose cluster.

   ```shell
   $ docker-compose -p my_ekuiper up -d

1. Check docker images running status, make sure two containers are started.

   $ docker ps
   CONTAINER ID   IMAGE                         COMMAND                  CREATED              STATUS                  PORTS                                                NAMES
   e2dbcd4c1f92   lfedge/ekuiper:latest          "/usr/bin/docker-ent…"   7 seconds ago        Up Less than a second   0.0.0.0:9081->9081/tcp, 127.0.0.1:20498->20498/tcp   ekuiper
   fa7c33b3e114   emqx/ekuiper-manager:latest   "/usr/bin/docker-ent…"   About a minute ago   Up 59 seconds           0.0.0.0:9082->9082/tcp                               manager

Please check use of eKuiper management console to set up and configure the eKuiper manager.

Install From Zip

eKuiper binary packages are released on below operating systems with AMD64, ARM and ARM64 support:

• CentOS 7 (EL7)
• CentOS 8 (EL8)
• Raspbian 10
• Debian 9
• Debian 10
• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS

For other operating systems such as Windows, users can compile from source code manually.

1. Download eKuiper zip or tar for your CPU architecture from ekuiper.org or Github.

2. Unzip the installation file:

   unzip kuiper-x.x.x-linux-amd64.zip
   

3. Start eKuiper.

   $ bin/kuiperd
   

4. Remove eKuiper. Simply delete the eKuiper directory.

After installation, all the files are inside the unzipped directory. Please check installed directory structure for detail.

Install from package

1. Download eKuiper package for your CPU architecture from ekuiper.org or Github.

2. Install eKuiper.

   • DEB package:

    # for debian/ubuntu
    $ sudo apt install ./kuiper-x.x.x-linux-amd64.deb
   

   • RPM package:

    # for CentOS
    $ sudo rpm -ivh kuiper-x.x.x-linux-amd64.rpm
   

3. Start eKuiper.

   • quick start

    $ sudo kuiperd
   

   • systemctl

    sudo systemctl start kuiper
   

4. Remove eKuiper.

   • DEB:

    sudo apt remove --purge kuiper
   

   • RPM:

    sudo yum remove kuiper
   

When installing by package, the eKuiper folders are not in the same directory. The installation structure is as below:

/usr/lib/kuiper/bin
  kuiperd
  kuiper
/etc/kuiper
  ...
/var/lib/kuiper/data
  ...
/var/lib/kuiper/plugins
  ...
/var/log/kuiper
   ...

Install via Helm (K8S、K3S)

1. Add helm repository.

    $ helm repo add emqx https://repos.emqx.io/charts
    $ helm repo update

1. Query eKuiper.

    $ helm search repo emqx
    NAME         CHART VERSION APP VERSION DESCRIPTION
    emqx/emqx    v4.0.0        v4.0.0      A Helm chart for EMQX
    emqx/emqx-ee v4.0.0        v4.0.0      A Helm chart for EMQX
    emqx/ekuiper  0.1.1         0.1.1       A lightweight IoT edge analytic software

1. Start eKuiper.

    $ helm install my-ekuiper emqx/ekuiper

1. View eKuiper status.

   $ kubectl get pods
   NAME         READY  STATUS    RESTARTS  AGE
   my-ekuiper-0 1/1    Running   0         56s

Compile from source code

1. Get the source code.

   $ git clone https://github.com/lf-edge/ekuiper.git

1. Compile.

   $ make

1. Start eKuiper.

   $ cd _build/kuiper-x.x.x-linux-amd64/
   $ bin/kuiperd

Compile Packages and Docker Images

• Packages: $ make pkg
  • Packages files that support EdgeX: $ make pkg_with_edgex
• Docker images: $ make docker

Notice that: Docker images support EdgeX by default

Cross-compile binaries

Go supports cross-compiling binaries for multiple platforms which applies to eKuiper as well. Because eKuiper depends on sqlite, CGO_ENABLED must be set to 1 which requires to install and specify the gcc of the target system.

• Install the GNU toolchain/gcc of the target system.
• Modify the Makefile to specify GOOS, GOARCH and CC and then build.

For example, to cross build ARM64 binaries in AMD64 ubuntu/debian machine, do these steps:

1. Install the GNU toolchain/gcc of the target system ARM64

     apt-get install gcc-aarch64-linux-gnu
   

2. Update the Makefile in the build command. Examples:

     GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc go build -trimpath -ldflags="-s -w -X main.Version=$(VERSION) -X main.LoadFileType=relative" -o kuiperd cmd/kuiperd/main.go
   

3. Run make

Compile with Selected Features

eKuiper allows tailoring the binary in compilation to get a customized feature set. As written by go, it also allows cross compilation. Except core runtime and REST api, there are some features that are allowed to be enabled or disabled during compilation by go build constraints. Uses can customize the built binary to include only the desired features to reduce the binary size according to the limit of the target environment.

Feature	Build Tag	Description
Core	core	The core of eKuiper. It contains the processor and REST API for stream/table/rule, the configuration processing, the SQL parser, the rule runtime etc.
CLI	rpc	The cli server
EdgeX Foundry integration	edgex	The built-in edgeX source, sink and connection
Native plugin	plugin	The native plugin runtime, REST API, CLI API etc.
Portable plugin	portable	The portable plugin runtime, REST API, CLI API etc.
External service	service	The external service runtime, REST API, CLI API etc.
Msgpack-rpc External service	msgpack	Support msgpack-rpc protocol in external service
UI Meta API	ui	The REST API of the metadata which is usually consumed by the ui
Prometheus Metrics	prometheus	Support to send metrics to prometheus
Extended template functions	template	Support additional data template function from sprig besides default go text/template functions
Codecs with schema	schema	Support schema registry and codecs with schema such as protobuf

In makefile, we already provide three feature sets: standard, edgeX and core. The standard feature set include all features in the list except edgeX; edgeX feature set include all features; And the core feature set is the minimal which only has core feature. Build these feature sets with default makefile:

# standard
make
# EdgeX
make build_with_edgex
# core
make build_core

Feature selection is useful in a limited resource target which is unlikely to run as docker container. So we only provide standard feature set in the docker images.

And users need to build from source to customize the feature sets. To build with the desired features:

go build --tags "<FEATURE>"

For example, to build with core and native plugin support:

go build --tags "core plugin"

Recommend updating the build command in the Makefile with tags and building from make.

Installation structure

Below is the directory structure after installation.

bin
  kuiperd
  kuiper
etc
  ...
data
  ...
plugins
  ...
log
  ...

bin

The bin directory includes all of executable files. Such as the kuiper server kuiperd and the cli client kuiper.

etc

The etc directory contains the default configuration files of eKuiper. Such as the global configuration file kuiper.yaml and all the source configuration files such as mqtt_source.yaml.

data

This folder saves the persisted definitions of streams and rules. It also contains any user defined configurations.

plugins

eKuiper allows users to develop your own plugins, and put these plugins into this folder. See extension for more info for how to extend the eKuiper.

log

All the log files are under this folder. The default log file name is stream.log.