Sfoglia il codice sorgente

doc(table): support table for batch static data as data source

ngjaying 4 anni fa
parent
commit
2f85fa3029
3 ha cambiato i file con 234 aggiunte e 0 eliminazioni
  1. 91 0
      docs/en_US/cli/tables.md
  2. 91 0
      docs/en_US/restapi/tables.md
  3. 52 0
      docs/en_US/sqls/tables.md

+ 91 - 0
docs/en_US/cli/tables.md

@@ -0,0 +1,91 @@
+# Tables management
+
+The Kuiper table command line tools allows you to manage the tables, such as create, describe, show and drop table definitions.
+
+## create a table
+
+The command is used for creating a table. For more detailed information of table definition, please refer to [tables](../sqls/tables.md).
+
+```shell
+create table $table_name $table_def | create table -f $table_def_file
+```
+
+- Specify the table definition in command line.
+
+Sample:
+
+```shell
+# bin/kuiper create table my_table '(id bigint, name string, score float) WITH ( datasource = "lookup.json", FORMAT = "json", KEY = "id");'
+table my_table created
+```
+
+The command create a table named ``my_table``. 
+
+- Specify the table definition in file. If the table is complex, or the table is already wrote in text files with well organized formats, you can just specify the table definition through ``-f`` option.
+
+Sample:
+
+```shell
+# bin/kuiper create table -f /tmp/my_table.txt
+table my_table created
+```
+
+Below is the contents of ``my_table.txt``.
+
+```
+my_table(id bigint, name string, score float)
+    WITH ( datasource = "lookup.json", FORMAT = "json", KEY = "id");
+```
+
+## show tables
+
+The command is used for displaying all of tables defined in the server.
+
+```shell
+show tables
+```
+
+Sample:
+
+```shell
+# bin/kuiper show tables
+my_table
+```
+
+## describe a table
+
+The command is used for print the detailed definition of table.
+
+```shell
+describe table $table_name
+```
+
+Sample:
+
+```shell
+# bin/kuiper describe table my_table
+Fields
+--------------------------------------------------------------------------------
+id	bigint
+name	string
+score	float
+
+FORMAT: json
+KEY: id
+DATASOURCE: lookup.json
+```
+
+## drop a table
+
+The command is used for drop the table definition.
+
+```shell
+drop table $table_name
+```
+
+Sample:
+
+```shell
+# bin/kuiper drop table my_table
+table my_table dropped
+```

+ 91 - 0
docs/en_US/restapi/tables.md

@@ -0,0 +1,91 @@
+# Tables management
+
+The Kuiper REST api for tables allows you to manage the tables, such as create, describe, show and drop table definitions.
+
+## create a table
+
+The API is used for creating a table. For more detailed information of table definition, please refer to [tables](../sqls/tables.md).
+
+```shell
+POST http://localhost:9081/tables
+```
+Request sample, the request is a json string with `sql` field.
+
+```json
+{"sql":"create table my_table (id bigint, name string, score float) WITH ( datasource = \"lookup.json\", FORMAT = \"json\", KEY = \"id\")"}
+```
+
+This API can run any table sql statements, not only table creation.
+
+## show tables
+
+The API is used for displaying all of tables defined in the server.
+
+```shell
+GET http://localhost:9081/tables
+```
+
+Response Sample:
+
+```json
+["mytable"]
+```
+
+## describe a table
+
+The API is used for print the detailed definition of table.
+
+```shell
+GET http://localhost:9081/tables/{id}}
+```
+
+Response Sample:
+
+```shell
+{
+  "Name": "demo",
+  "StreamFields": [
+    {
+      "Name": "temperature",
+      "FieldType": {
+        "Type": 2
+      }
+    },
+    {
+      "Name": "ts",
+      "FieldType": {
+        "Type": 1
+      }
+    }
+  ],
+  "Options": {
+    "DATASOURCE": "lookup.json",
+    "FORMAT": "JSON"
+  }
+}
+```
+
+## update a table
+
+The API is used for update the table definition.
+
+```shell
+PUT http://localhost:9081/tables/{id}
+```
+
+Path parameter `id` is the id or name of the old table.
+
+Request sample, the request is a json string with `sql` field.
+
+```json
+{"sql":"create table my_table (id bigint, name string, score float) WITH ( datasource = \"topic/temperature\", FORMAT = \"json\", KEY = \"id\")"}
+```
+
+## drop a table
+
+The API is used for drop the table definition.
+
+```shell
+DELETE http://localhost:9081/tables/{id}
+```
+

+ 52 - 0
docs/en_US/sqls/tables.md

@@ -0,0 +1,52 @@
+# Table specs
+
+Kuiper streams are infinite.  **Table** is provided to read data from a finite source like a file or a normal database table as a batch. The batch source is supposed to be small because it will reside in the memory. The typical scenario to use table is to treat it as a static lookup dictionary to join with the stream.
+
+## Syntax
+
+Table supports almost the same syntax as streams. To create a table, run the below SQL:
+
+```sql
+CREATE TABLE   
+    table_name   
+    ( column_name <data_type> [ ,...n ] )
+    WITH ( property_name = expression [, ...] );
+```
+
+Table supports the same [data types](./streams.md#data-types) as stream. Compared to stream, it has the following limitations:
+
+1. Currently, the only and default supported type is "file", and the source plugin is not supported.
+2. Format "binary" is not supported.
+
+## File type
+
+Currently, the only supported type for table is file. To create a table that will read lookup.json file is like:
+
+```sql
+CREATE TABLE table1 (
+    name STRING,
+    size BIGINT,
+    id BIGINT
+) WITH (DATASOURCE="lookup.json", FORMAT="json");
+```
+The configure file for the file source is in */etc/sources/file.yaml* in which the path to the file can be specified.
+
+```yaml
+default:
+  fileType: json
+  # The directory of the file relative to kuiper root or an absolute path.
+  # Do not include the file name here. The file name should be defined in the stream data source
+  path: data
+```
+
+With this yaml file, the table will refer to the file *${kuiper}/data/lookup.json* and read it in json format.
+
+## Lookup table
+
+A typical usage for table is as a lookup table. Sample SQL will be like:
+
+```sql
+SELECT * FROM demo INNER JOIN table1 on demo.ts = table1.id
+```
+
+Only when joining with a table, the join statement can be run without a window.