Browse Source

doc(sink): file sink new properties

Signed-off-by: Jiyong Huang <huangjy@emqx.io>
Jiyong Huang 2 năm trước cách đây
mục cha
commit
f99be8dd66

+ 3 - 1
docs/en_US/guide/sinks/builtin/edgex.md

@@ -43,6 +43,8 @@ Below optional configurations are supported, please check MQTT specification for
 Notice that, the edgex action can support data templates to vary the result format, but the result of the data template must be in the object form of a JSON string, e.g. `"{\"key\":\"{{.key}}\"}"`. JSON strings in the form of arrays or non-JSON strings are not supported.
 :::
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Send to various targets
 
 By setting the combination of the properties, we can send the result to various EdgeX message bus settings.
@@ -217,7 +219,7 @@ In this case, the original metadata value (such as `id, profileName, deviceName,
 ```
 Please notice that, 
 - The device name of `Event` structure is changed to `kuiper` and the profile name is changed to `kuiperProfile`.
-- All of metadata for `Events and Readings` structure will be updated with new value. `Origin` field is updated to another value generated by eKuiper (here is `0``).
+- All metadata for `Events and Readings` structure will be updated with new value. `Origin` field is updated to another value generated by eKuiper (here is `0``).
 
 ### Publish result to a new EdgeX message bus keeping original metadata
 

+ 22 - 7
docs/en_US/guide/sinks/builtin/file.md

@@ -1,17 +1,30 @@
 # File Sink
 
-The sink is used for saving analysis result into a specified file.
+The sink saves the analysis result to a specified file. The [file source](../../sources/builtin/file.md) is the opposite connector that can read the file sink’s output.
 
 ## Properties
 
-| Property name | Optional | Description                                                                                                                                     |
-|---------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------|
-| path          | false    | The file path for saving the result, such as `/tmp/result.txt`                                                                                  |
-| interval      | true     | The time interval (ms) for writing the analysis result. The default value is 1000, which means write the analysis result with every one second. |
+| Property name | Optional | Description                                                                                                                                                    |
+|---------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| path          | false    | The file path for saving the result, such as `/tmp/result.txt`                                                                                                 |
+| interval      | true     | The time interval (ms) for flushing the analysis result into the file. The default value is 1000, which means write the analysis result with every one second. |
+| fileType      | true     | The type of the file, could be json, csv or lines. Default value is lines. Please check [file types](#file-types) for detail.                                  |
+| hasHeader     | true     | Whether to produce the header line. Currently, it is only effective for csv file type.                                                                         |
+| delimiter     | true     | The delimiter character, usually apply for the header. Only effective for csv file type.                                                                       |
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+Among them, the `format` property is used to define the format of the data in the file. Some file types can only work with specific format. Please check [file types](#file-types) for detail.
+
+### File Types
+
+The file sink can write data into different file types, such as:
+
+- lines: This is the default type. It writes line-separated files that can be decoded by the format parameter in the stream definition. For example, to write line-separated JSON strings, set the file type to lines and the format to json.
+- json: This type writes standard JSON array format files. For an example, see [here](https://github.com/lf-edge/ekuiper/tree/master/internal/topo/source/test/test.json). To use this file type, set the format to json.
+- csv: This type writes comma-separated csv files. You can also use custom separators. To use this file type, set the format to delimited.
 
 ## Sample usage
 
-Below is a sample for selecting temperature great than 50 degree, and save the result into file `/tmp/result.txt` with every 5 seconds.
+Below is a sample for selecting temperature greater than 50 degree, and save the result into file `/tmp/result.txt` with every 5 seconds.
 
 ```json
 {
@@ -20,7 +33,9 @@ Below is a sample for selecting temperature great than 50 degree, and save the r
     {
       "file": {
         "path": "/tmp/result.txt",
-        "interval": 5000
+        "interval": 5000,
+        "fileType": "lines",
+        "format": "json"
       }
     }
   ]

+ 1 - 1
docs/en_US/guide/sinks/builtin/log.md

@@ -2,5 +2,5 @@
 
 The action is used for print output message into log file, the log file is at `$eKuiper_install/log/stream.log` by default.
 
-No properties can be specified for the action.
+The common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
 

+ 3 - 1
docs/en_US/guide/sinks/builtin/mqtt.md

@@ -16,9 +16,11 @@ The action is used for publish output message into an MQTT server.
 | rootCaPath         | true     | The location of root ca path. It can be an absolute path, or a relative path, which is similar to use of certificationPath.                                                                                                                                                                                                                               |
 | insecureSkipVerify | true     | If InsecureSkipVerify is `true`, TLS accepts any certificate presented by the server and any host name in that certificate.  In this mode, TLS is susceptible to man-in-the-middle attacks. The default value is `false`. The configuration item can only be used with TLS connections.                                                                   |
 | retained           | true     | If retained is `true`,The broker stores the last retained message and the corresponding QoS for that topic.The default value is `false`.                                                                                                                                                                                                                  |
-| compression        | true     | Compress the payload with the specified compression method. Support `zlib`, `gzip`, `flate` method now.                                                                                                                                                                                                                                                               |
+| compression        | true     | Compress the payload with the specified compression method. Support `zlib`, `gzip`, `flate` method now.                                                                                                                                                                                                                                                   |
 | connectionSelector | true     | reuse the connection to mqtt broker. [more info](../../sources/builtin/mqtt.md#connectionselector)                                                                                                                                                                                                                                                        | 
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 Below is sample configuration for connecting to Azure IoT Hub by using SAS authentication.
 ```json
     {

+ 2 - 0
docs/en_US/guide/sinks/builtin/neuron.md

@@ -9,6 +9,8 @@ The action is used to publish result to the local neuron instance. Notice that,
 | tags          | true     | The field names to be sent to neuron as a tag. If not specified, all result fields will be sent.                                        |
 | raw           | true     | Default to false. Whether to convert the data to neuron format by this sink or just publish the json or data template converted result. |
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Examples
 
 Assume the sink receive result map like:

+ 0 - 13
docs/en_US/guide/sinks/builtin/redis.md

@@ -2,19 +2,6 @@
 
 The sink will publish the result into redis.
 
-## Compile & deploy plugin
-The plugin source code put in the extensions directory, but need build in the ekuiper root path
-```shell
-# cd $eKuiper_src
-# go build -trimpath --buildmode=plugin -o plugins/sinks/Redis.so extensions/sinks/redis/redis.go
-# zip redis.zip plugins/sinks/Redis.so
-# cp redis.zip /root/tomcat_path/webapps/ROOT/
-# bin/kuiper create plugin sink redis -f /tmp/redisPlugin.txt
-# bin/kuiper create rule redis -f /tmp/redisRule.txt
-```
-
-Restart the eKuiper server to activate the plugin.
-
 ## Properties
 
 | Property name | Optional | Description                                                                                                                                                                                          |

Những thai đổi đã bị hủy bỏ vì nó quá lớn
+ 2 - 0
docs/en_US/guide/sinks/builtin/rest.md


+ 2 - 0
docs/en_US/guide/sinks/plugin/image.md

@@ -21,6 +21,8 @@ Restart the eKuiper server to activate the plugin.
 | maxAge         | True     | Maximum file storage time (hours). The default value is 72, which means that the picture can be stored for up to 3 days.                                                        |
 | maxCount       | True     | The maximum number of stored pictures. The default value is 1000. The earlier pictures will be deleted. The relationship with `maxAge` is OR.                                   |
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Usage example
 
 The following example demonstrates how to receive pictures and save them to the folder `/tmp`. When the number of pictures exceeds 1000, the earlier pictures will be deleted. When the pictures are saved for more than 72 hours, the timeout pictures will be deleted.

+ 3 - 0
docs/en_US/guide/sinks/plugin/influx.md

@@ -32,6 +32,9 @@ Restart the eKuiper server to activate the plugin.
 | tagkey        | true     | The tag key of the InfluxDB                       |
 | tagvalue      | true     | The tag value of the InfluxDB                     |
 | fields        | true     | The column of the InfluxDB,split with ","         |
+
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Sample usage
 
 Below is a sample for selecting temperature great than 50 degree, and some profiles only for your reference.

+ 3 - 0
docs/en_US/guide/sinks/plugin/influx2.md

@@ -63,6 +63,9 @@ Restart the eKuiper server to activate the plugin.
 | tagKey        | true     | The tag key of the InfluxDB                       |
 | tagValue      | true     | The tag value of the InfluxDB                     |
 | fields        | true     | The column of the InfluxDB,split with ","         |
+
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Sample usage
 
 Below is a sample for selecting temperature great than 50 degree, and some profiles only for your reference.

+ 2 - 0
docs/en_US/guide/sinks/plugin/sql.md

@@ -35,6 +35,8 @@ for example, if he only wants mysql, then he can build with build tag `mysql`.
 | tableDataField | true     | Write the nested values of the tableDataField into database.                                                                                                  |
 | rowkindField   | true     | Specify which field represents the action like insert or update. If not specified, all rows are default to insert.                                            |
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Sample usage
 
 Below is a sample for using sql to get the target data and set to mysql database 

+ 2 - 0
docs/en_US/guide/sinks/plugin/tdengine.md

@@ -28,6 +28,8 @@ As the tdengine database requires a timestamp field in the table, the user must
 | sTable         | String   | true     | The super table to be use, could be a [dynamic property](../../sinks/overview.md#dynamic-properties).                                                            |
 | tableDataField | String   | true     | Write the nested values of the tableDataField into database.                                                                                                     |
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Operation example
 
 ### To create a database or table, refer to the following documents:

+ 2 - 0
docs/en_US/guide/sinks/plugin/zmq.md

@@ -19,6 +19,8 @@ Restart the eKuiper server to activate the plugin.
 | server        | false    | The url of the Zero Mq server |
 | topic         | true     | The topic to publish to       |
 
+Other common sink properties are supported. Please refer to the [sink common properties](../overview.md#common-properties) for more information.
+
 ## Sample usage
 
 Below is a sample for selecting temperature great than 50 degree, and publish the result into Zero Mq topic "temp".

+ 3 - 1
docs/zh_CN/guide/sinks/builtin/edgex.md

@@ -39,9 +39,11 @@
   - KeyPEMBlock
   - SkipCertVerify
 
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ::: v-pre
 EdgeX 动作可支持数据模板对结果格式进行变化,但是数据模板的结果必须为 JSON 字符串的 object 形式,例如 `"{\"key\":\"{{.key}}\"}"`。数组形式的 JSON 字符串或者非 JSON 字符串都不支持。
-:::
+::
 
 ## 发送到各种目标
 

+ 21 - 6
docs/zh_CN/guide/sinks/builtin/file.md

@@ -1,13 +1,26 @@
 # 文件目标(Sink)
 
-目标(Sink)用于将分析结果保存到指定文件中
+该 sink 将分析结果保存到指定文件中。[文件源](../../sources/builtin/file.md)是反向的连接器可以读取文件 sink 的输出
 
 ## 属性
 
-| 属性名称     | 是否可选 | 说明                                          |
-|----------|------|---------------------------------------------|
-| path     | 否    | 保存结果的文件路径,例如  `/tmp/result.txt`             |
-| interval | 是    | 写入分析结果的时间间隔(毫秒)。 默认值为1000,这表示每隔一秒钟写入一次分析结果。 |
+| 属性名称      | 是否可选 | 说明                                                            |
+|-----------|------|---------------------------------------------------------------|
+| path      | 否    | 保存结果的文件路径,例如  `/tmp/result.txt`                               |
+| interval  | 是    | 写入分析结果的时间间隔(毫秒)。 默认值为1000,这表示每隔一秒钟写入一次分析结果。                   |
+| fileType  | 是    | 文件类型,支持 json, csv 或者 lines,其中默认值为 lines。更多信息请参考[文件类型](#文件类型)。 |
+| hasHeader | 是    | 指定是否生成文件头。当前仅在文件类型为 csv 时生效。                                  |
+| delimiter | 是    | 指定分隔符,通常用于文件头的生成。当前仅当文件类型为 csv 时生效。                           |
+
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。其中,`format` 属性用于定义文件中数据的格式。某些文件类型只能与特定格式一起使用,详情请参阅[文件类型](#文件类型)。
+
+### 文件类型
+
+文件 sink 可以将数据写入不同类型的文件,例如:
+
+- lines:这是默认类型。它写入由流定义中的格式参数解码的行分隔文件。例如,要写入行分隔的 JSON 字符串,请将文件类型设置为 lines,格式设置为 json。
+- json:此类型写入标准 JSON 数组格式文件。有关示例,请参见[此处](https://github.com/lf-edge/ekuiper/tree/master/internal/topo/source/test/test.json)。要使用此文件类型,请将格式设置为 json。
+- csv:此类型写入逗号分隔的 csv 文件。您也可以使用自定义分隔符。要使用此文件类型,请将格式设置为 delimited。"
 
 ## 使用示例
 
@@ -20,7 +33,9 @@
     {
       "file": {
         "path": "/tmp/result.txt",
-        "interval": 5000
+        "interval": 5000,
+        "fileType": "lines",
+        "format": "json"
       }
     }
   ]

+ 1 - 1
docs/zh_CN/guide/sinks/builtin/log.md

@@ -2,5 +2,5 @@
 
 该操作用于将输出消息打印到日志文件中,默认情况下,该日志文件位于 `$ ekuiper_install/log/stream.log` 中。
 
-无法为操作指定属性
+支持通用的 sink 属性,请参阅[公共属性](../overview.md#公共属性)
 

+ 3 - 1
docs/zh_CN/guide/sinks/builtin/mqtt.md

@@ -16,9 +16,11 @@
 | rootCaPath         | 是    | 根证书路径,用以验证服务器证书。可以为绝对路径,也可以为相对路径,相对路径的用法与 `certificationPath` 类似。                                                                                                                         |
 | insecureSkipVerify | 是    | 如果 InsecureSkipVerify 设置为 `true`, TLS接受服务器提供的任何证书以及该证书中的任何主机名。 在这种模式下,TLS容易受到中间人攻击。默认值为`false`。配置项只能用于TLS连接。                                                                              |
 | retained           | 是    | 如果 retained 设置为 `true`,Broker会存储每个Topic的最后一条保留消息及其Qos。默认值是 `false`                                                                                                                        |
-| compression        | true | 使用指定的压缩方法压缩 Payload。当前支持 zlib, gzip, flate  算法。                                                                                                                                                        |
+| compression        | true | 使用指定的压缩方法压缩 Payload。当前支持 zlib, gzip, flate  算法。                                                                                                                                           |
 | connectionSelector | 是    | 重用到 MQTT Broker 的连接,详细信息,[请参考](../../sources/builtin/mqtt.md#connectionselector)                                                                                                          |
 
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 以下为使用 SAS 连接到 Azure IoT Hub 的样例。
 ```json
     {

+ 2 - 0
docs/zh_CN/guide/sinks/builtin/neuron.md

@@ -9,6 +9,8 @@
 | tags      | 是    | 发送到 neuron 的标签名列表。如果未设置,则结果中的所有列都会作为标签发送。                            |
 | raw       | 是    | 默认为 false。是否使用原始字符串格式(json或者经过数据模板转换的字符串)。若为否,则会自动将结果转换为 neuron 的格式。 |
 
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ## 示例
 
 假设接收到的结果如下所示:

+ 1 - 0
docs/zh_CN/guide/sinks/builtin/redis.md

@@ -27,6 +27,7 @@ redis 源代码在 extensions 目录中,但是需要在 eKuiper 根目录编
 | dataType     | 是    | Redis 数据的类型, 默认是 string, 注意修改类型之后,需在redis中删除原有 key,否则修改无效。目前只支持 "list" 和 "string"                                                                                                                                                        |
 | expiration   | 是    | 超时时间                                                                                                                                                                                                                                     |
 | rowkindField | 是    | 指定哪个字段表示操作,例如插入或更新。如果不指定,默认所有的数据都是插入操作                                                                                                                                                                                                   |
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
 
 ## 示例用法
 

Những thai đổi đã bị hủy bỏ vì nó quá lớn
+ 1 - 0
docs/zh_CN/guide/sinks/builtin/rest.md


+ 3 - 0
docs/zh_CN/guide/sinks/plugin/influx.md

@@ -31,6 +31,9 @@
 | tagkey       | 是    | InfluxDB的标签键       |
 | tagvalue     | 是    | InfluxDB的标签值       |
 | fields       | 是    | InfluxDB的列名,用","隔开 |
+
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ## 示例用法
 
 下面是选择温度大于50度的样本规则,和一些配置文件仅供参考。

+ 3 - 0
docs/zh_CN/guide/sinks/plugin/influx2.md

@@ -61,6 +61,9 @@ $(PLUGINS_CUSTOM):
 | tagKey      | 是    | InfluxDB的标签键       |
 | tagValue    | 是    | InfluxDB的标签值       |
 | fields      | 是    | InfluxDB的列名,用","隔开 |
+
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ## 示例用法
 
 下面是选择温度大于50度的样本规则,和一些配置文件仅供参考。

+ 2 - 0
docs/zh_CN/guide/sinks/plugin/sql.md

@@ -32,6 +32,8 @@
 | tableDataField | true     | 将 tableDataField 的嵌套值写入数据库。                 |
 | rowkindField   | true     | 指定哪个字段表示操作,例如插入或更新。如果不指定,默认所有的数据都是插入操作      |
 
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ## 使用样例
 
 下面是一个获取目标数据并写入 mysql 数据库的示例

+ 15 - 13
docs/zh_CN/guide/sinks/plugin/tdengine.md

@@ -14,19 +14,21 @@ go build -trimpath --buildmode=plugin -o plugins/sinks/Tdengine@v1.0.0.so extens
 由于 tdengine 数据库要求表中必须有时间戳字段,所以用户必须告知数据表的时间戳字段名称(必填tsFieldName)。用户可以选择是否提供时间戳数据,若不提供(provideTs=false),时间戳字段的内容由 tdengine 数据库自动生成。
 
 | 名称             | 类型       | 是否必填 | 释义                                                                                                    |
-|----------------|----------|--|-------------------------------------------------------------------------------------------------------|
-| host           | string   | 否 | 数据库域名,其值必须为域名,即 [FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html),不能为 IP 地址。其默认值为 localhost。 |
-| port           | int      | 是 | 数据库端口                                                                                                 |
-| user           | string   | 否 | 用户名,默认值为 `root` 。                                                                                     |
-| password       | string   | 否 | 密码,默认值为 `taosdata` 。                                                                                  |
-| database       | string   | 是 | 数据库名                                                                                                  |
-| table          | string   | 是 | 表名,可设置[动态属性](../overview.md#动态属性)。                                                                 |
-| fields         | []string | 否 | 将要插入的表字段集合。sink 收到的数据和数据库表中均有该字段。若为设置,则所有结果字段写入数据库。                                                   |
-| provideTs      | Bool     | 否 | 用户是否提供时间戳字段,默认为否。                                                                                     |
-| tsFieldName    | String   | 是 | 时间戳字段名称                                                                                               |
-| sTable         | String   | 否 | 使用的超级表,可设置[动态属性](../overview.md#动态属性)。                                                             |
-| tagFields      | []String | 否 | 结果中作为标签的字段。若设置 sTable 属性,则该属性必填。                                                                      |
-| tableDataField | String   | 否 | 将 tableDataField 的嵌套值写入数据库。                                          |
+|----------------|----------|------|-------------------------------------------------------------------------------------------------------|
+| host           | string   | 否    | 数据库域名,其值必须为域名,即 [FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html),不能为 IP 地址。其默认值为 localhost。 |
+| port           | int      | 是    | 数据库端口                                                                                                 |
+| user           | string   | 否    | 用户名,默认值为 `root` 。                                                                                     |
+| password       | string   | 否    | 密码,默认值为 `taosdata` 。                                                                                  |
+| database       | string   | 是    | 数据库名                                                                                                  |
+| table          | string   | 是    | 表名,可设置[动态属性](../overview.md#动态属性)。                                                                    |
+| fields         | []string | 否    | 将要插入的表字段集合。sink 收到的数据和数据库表中均有该字段。若为设置,则所有结果字段写入数据库。                                                   |
+| provideTs      | Bool     | 否    | 用户是否提供时间戳字段,默认为否。                                                                                     |
+| tsFieldName    | String   | 是    | 时间戳字段名称                                                                                               |
+| sTable         | String   | 否    | 使用的超级表,可设置[动态属性](../overview.md#动态属性)。                                                                |
+| tagFields      | []String | 否    | 结果中作为标签的字段。若设置 sTable 属性,则该属性必填。                                                                      |
+| tableDataField | String   | 否    | 将 tableDataField 的嵌套值写入数据库。                                                                           |
+
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
 
 ## 操作示例
 

+ 2 - 0
docs/zh_CN/guide/sinks/plugin/zmq.md

@@ -19,6 +19,8 @@
 | server | false    | Zero Mq 服务器的 URL        |
 | topic  | true     | The topic to publish to |
 
+其他通用的 sink 属性也支持,请参阅[公共属性](../overview.md#公共属性)。
+
 ## 使用示例
 
 下面是一个选择温度大于50度的示例,并将结果发布到 Zero Mq 主题 "temp"。