Skip to content

Commit

Permalink
[FLINK-29023][docs][table] Update documentation of JAR statement page
Browse files Browse the repository at this point in the history
This closes apache#20630
  • Loading branch information
lsyldliu authored and wuchong committed Sep 19, 2022
1 parent be3c35c commit f6520ec
Show file tree
Hide file tree
Showing 7 changed files with 57 additions and 28 deletions.
2 changes: 2 additions & 0 deletions docs/content.zh/docs/dev/table/catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ Catalog 是可扩展的,用户可以通过实现 `Catalog` 接口来开发自
`CatalogFactory` 定义了一组属性,用于 SQL CLI 启动时配置 Catalog。
这组属性集将传递给发现服务,在该服务中,服务会尝试将属性关联到 `CatalogFactory` 并初始化相应的 Catalog 实例。

注意:从 FLINK-15635 开始,table 模块引入了一个用户类加载器,以统一管理所有的用户 jar 包,例如通过 `ADD JAR` 语句添加的 jar。在自定义 Catalog 中,如果需要明确加载某些类,请使用用户类加载器而不是线程上下文类加载器,否则可能会出现 ClassNotFoundException 异常。用户可以通过 `CatalogFactory.Context` 获得用户类加载器。

## 如何创建 Flink 表并将其注册到 Catalog

### 使用 SQL DDL
Expand Down
2 changes: 2 additions & 0 deletions docs/content.zh/docs/dev/table/sourcesSinks.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Flink 为 Kafka、Hive 和不同的文件系统提供了预定义的连接器。

本页重点介绍如何开发自定义的用户定义连接器。

注意:从 FLINK-15635 开始,table 模块引入了一个用户类加载器,以统一管理所有的用户 jar 包,例如通过 `ADD JAR` 语句添加的 jar。在用户自定义连接器中,如果需要明确加载某些类,请使用用户类加载器而不是线程上下文类加载器,否则可能会出现 ClassNotFoundException 异常。用户可以通过 `DynamicTableFactory.Context` 获得用户类加载器。

概述
--------

Expand Down
2 changes: 1 addition & 1 deletion docs/content.zh/docs/dev/table/sql/create.md
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,6 @@ Language tag 用于指定 Flink runtime 如何执行这个函数。目前,只

**USING**

指定包含该函数的实现及其依赖的 jar 资源列表。该 jar 应该位于 Flink 当前支持的本地或远程[文件系统]{{< ref "docs/deployment/filesystems/overview" >}}中,比如 hdfs/s3/oss。
指定包含该函数的实现及其依赖的 jar 资源列表。该 jar 应该位于 Flink 当前支持的本地或远程[文件系统]({{< ref "docs/deployment/filesystems/overview" >}}) 中,比如 hdfs/s3/oss。

<span class="label label-danger">注意</span> 目前只有 JAVA、SCALA 语言支持 USING 子句。
39 changes: 25 additions & 14 deletions docs/content.zh/docs/dev/table/sql/jar.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,8 @@ JAR 语句用于将用户 jar 添加到 classpath、或将用户 jar 从 classpa

目前 Flink SQL 支持以下 JAR 语句:
- ADD JAR
- REMOVE JAR
- SHOW JARS

<span class="label label-danger">注意</span> JAR 语句仅适用于 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}})。
- REMOVE JAR

<a name="run-a-jar-statement"></a>

Expand All @@ -53,10 +51,18 @@ JAR 语句用于将用户 jar 添加到 classpath、或将用户 jar 从 classpa
{{< tab "SQL CLI" >}}
```sql
Flink SQL> ADD JAR '/path/hello.jar';
[INFO] The specified jar is added into session classloader.
[INFO] Execute statement succeed.

Flink SQL> ADD JAR 'hdfs:///udf/common-udf.jar';
[INFO] Execute statement succeed.

Flink SQL> SHOW JARS;
/path/hello.jar
+----------------------------+
| jars |
+----------------------------+
| /path/hello.jar |
| hdfs:///udf/common-udf.jar |
+----------------------------+

Flink SQL> REMOVE JAR '/path/hello.jar';
[INFO] The specified jar is removed from session classloader.
Expand All @@ -72,26 +78,31 @@ Flink SQL> REMOVE JAR '/path/hello.jar';
ADD JAR '<path_to_filename>.jar'
```

目前只支持将本地 jar 添加到会话类类加载器(session classloader)中
添加一个 JAR 文件到资源列表中,该 jar 应该位于 Flink 当前支持的本地或远程[文件系统]({{< ref "docs/deployment/filesystems/overview" >}}) 中。添加的 JAR 文件可以使用 [`SHOW JARS`](#show-jars) 语句列出

<a name="remove-jar"></a>
### Limitation
目前请不要通过 `ADD JAR` 语句来使用 Hive 连接器,正确行为参考 [Flink-Hive 集成]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies).

## REMOVE JAR
<a name="show-jars"></a>

## SHOW JARS

```sql
REMOVE JAR '<path_to_filename>.jar'
SHOW JARS
```

目前只支持删除 [`ADD JAR`](#add-jar) 语句添加的 jar。
展示所有通过 [`ADD JAR`](#add-jar) 语句添加的 jar。

<a name="show-jars"></a>
<a name="remove-jar"></a>

## SHOW JARS
## REMOVE JAR

```sql
SHOW JARS
REMOVE JAR '<path_to_filename>.jar'
```

展示会话类类加载器(session classloader)中所有基于 [`ADD JAR`](#add-jar) 语句添加的 jar。
删除由 [`ADD JAR`](#add-jar) 语句添加的指定 jar。

<span class="label label-danger">注意</span> REMOVE JAR 语句仅适用于 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}})。

{{< top >}}
2 changes: 2 additions & 0 deletions docs/content/docs/dev/table/catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ The factory is discovered using Java's Service Provider Interfaces (SPI).
Classes that implement this interface can be added to `META_INF/services/org.apache.flink.table.factories.Factory` in JAR files.
The provided factory identifier will be used for matching against the required `type` property in a SQL `CREATE CATALOG` DDL statement.

Note: Since FLINK-15635, a user classloader is introduced to the table module to manage all user jars uniformly such as the jar added by `ADD JAR` statement. In custom catalogs, if some classes need to be loaded explicitly, please use the user classloader instead of thread context classloader. Otherwise, ClassNotFoundException maybe occur. Users can get the user classloader through `CatalogFactory.Context`.

## How to Create and Register Flink Tables to Catalog

### Using SQL DDL
Expand Down
2 changes: 2 additions & 0 deletions docs/content/docs/dev/table/sourcesSinks.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ for more information about built-in table sources and sinks.

This page focuses on how to develop a custom, user-defined connector.

Note: Since FLINK-15635, a user classloader is introduced to the table module to manage all user jars uniformly such as the jar added by `ADD JAR` statement. In user-defined connector, if some classes need to be loaded explicitly, please use the user classloader instead of thread context classloader. Otherwise, ClassNotFoundException maybe occur. Users can get the user classloader through `DynamicTableFactory.Context`.

Overview
--------

Expand Down
36 changes: 23 additions & 13 deletions docs/content/docs/dev/table/sql/jar.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,8 @@ or show added jars in the classpath in the runtime.

Flink SQL supports the following JAR statements for now:
- ADD JAR
- REMOVE JAR
- SHOW JARS

<span class="label label-danger">Attention</span> JAR statements only work in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}).

- REMOVE JAR

## Run a JAR statement

Expand All @@ -51,10 +48,18 @@ The following examples show how to run `JAR` statements in [SQL CLI]({{< ref "do
{{< tab "SQL CLI" >}}
```sql
Flink SQL> ADD JAR '/path/hello.jar';
[INFO] The specified jar is added into session classloader.
[INFO] Execute statement succeed.

Flink SQL> ADD JAR 'hdfs:///udf/common-udf.jar';
[INFO] Execute statement succeed.

Flink SQL> SHOW JARS;
/path/hello.jar
+----------------------------+
| jars |
+----------------------------+
| /path/hello.jar |
| hdfs:///udf/common-udf.jar |
+----------------------------+

Flink SQL> REMOVE JAR '/path/hello.jar';
[INFO] The specified jar is removed from session classloader.
Expand All @@ -68,22 +73,27 @@ Flink SQL> REMOVE JAR '/path/hello.jar';
ADD JAR '<path_to_filename>.jar'
```

Currently it only supports to add the local jar into the session classloader.
Add a JAR file to the list of resources, it supports adding the jar locates in a local or remote [file system]({{< ref "docs/deployment/filesystems/overview" >}}). The added JAR file can be listed using [`SHOW JARS`](#show-jars) statements.

## REMOVE JAR
### Limitation
Please don't use the hive connector by `ADD JAR` statement currently, the correct behavior refers to [Flink-Hive integration]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies).

## SHOW JARS

```sql
REMOVE JAR '<path_to_filename>.jar'
SHOW JARS
```

Currently it only supports to remove the jar that is added by the [`ADD JAR`](#add-jar) statements.
Show all added jars which are added by [`ADD JAR`](#add-jar) statements.

## SHOW JARS
## REMOVE JAR

```sql
SHOW JARS
REMOVE JAR '<path_to_filename>.jar'
```

Show all added jars in the session classloader which are added by [`ADD JAR`](#add-jar) statements.
Remove the specified jar that is added by the [`ADD JAR`](#add-jar) statements.

<span class="label label-danger">Attention</span> REMOVE JAR statement only work in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}).

{{< top >}}

0 comments on commit f6520ec

Please sign in to comment.