Skip to content

Commit

Permalink
[FLINK-17639] Document which FileSystems are supported by the Streami…
Browse files Browse the repository at this point in the history
…ngFileSink

This closes apache#12737.
  • Loading branch information
guoweiM authored and kl0u committed Jun 24, 2020
1 parent 44ea896 commit 00863a2
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/dev/connectors/streamfile_sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -733,6 +733,9 @@ Given this, when trying to restore from an old checkpoint/savepoint which assume
by subsequent successful checkpoints, Flink will refuse to resume and it will throw an exception as it cannot locate the
in-progress file.

<span class="label label-danger">Important Note 4</span>: Currently, the `StreamingFileSink` only supports three filesystems:
HDFS, S3, and Local. Flink will throw an exception when using an unsupported filesystem at runtime.

### S3-specific

<span class="label label-danger">Important Note 1</span>: For S3, the `StreamingFileSink`
Expand Down
2 changes: 2 additions & 0 deletions docs/dev/connectors/streamfile_sink.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,8 @@ Hadoop 2.7 之前的版本不支持这个方法,因此 Flink 会报异常。
<span class="label label-danger">重要提示 3</span>: Flink 以及 `StreamingFileSink` 不会覆盖已经提交的数据。因此如果尝试从一个包含 in-progress 文件的旧 checkpoint/savepoint 恢复,
且这些 in-progress 文件会被接下来的成功 checkpoint 提交,Flink 会因为无法找到 in-progress 文件而抛异常,从而恢复失败。

<span class="label label-danger">重要提示 4</span>: 目前 `StreamingFileSink` 只支持三种文件系统: HDFS、S3和Local。如果配置了不支持的文件系统,在执行的时候 Flink 会抛出异常。

### S3 特有的注意事项

<span class="label label-danger">重要提示 1</span>: 对于 S3,`StreamingFileSink` 只支持基于 [Hadoop](https://hadoop.apache.org/)
Expand Down

0 comments on commit 00863a2

Please sign in to comment.