Skip to content

Commit

Permalink
[FLINK-14788][documentation] Generated documentation for ExecutionChe…
Browse files Browse the repository at this point in the history
…ckpointingOptions

This closes apache#10222
  • Loading branch information
dawidwys committed Nov 22, 2019
1 parent d3dbc96 commit 65a472f
Show file tree
Hide file tree
Showing 5 changed files with 69 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
<table class="table table-bordered">
<thead>
<tr>
<th class="text-left" style="width: 20%">Key</th>
<th class="text-left" style="width: 15%">Default</th>
<th class="text-left" style="width: 10%">Type</th>
<th class="text-left" style="width: 55%">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><h5>execution.checkpointing.externalized-checkpoint-retention</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td><p>Enum</p>Possible values: [DELETE_ON_CANCELLATION, RETAIN_ON_CANCELLATION]</td>
<td>Externalized checkpoints write their meta data out to persistent storage and are not automatically cleaned up when the owning job fails or is suspended (terminating with job status <span markdown="span">`JobStatus#FAILED`</span> or <span markdown="span">`JobStatus#SUSPENDED`</span>. In this case, you have to manually clean up the checkpoint state, both the meta data and actual program state.<br /><br />The mode defines how an externalized checkpoint should be cleaned up on job cancellation. If you choose to retain externalized checkpoints on cancellation you have to handle checkpoint clean up manually when you cancel the job as well (terminating with job status <span markdown="span">`JobStatus#CANCELED`</span>).<br /><br />The target directory for externalized checkpoints is configured via <span markdown="span">`state.checkpoints.dir`</span>.</td>
</tr>
<tr>
<td><h5>execution.checkpointing.interval</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>Duration</td>
<td>Gets the interval in which checkpoints are periodically scheduled.<br /><br />This setting defines the base interval. Checkpoint triggering may be delayed by the settings <span markdown="span">`execution.checkpointing.max-concurrent-checkpoints`</span> and <span markdown="span">`execution.checkpointing.min-pause`</span></td>
</tr>
<tr>
<td><h5>execution.checkpointing.max-concurrent-checkpoints</h5></td>
<td style="word-wrap: break-word;">1</td>
<td>Integer</td>
<td>The maximum number of checkpoint attempts that may be in progress at the same time. If this value is n, then no checkpoints will be triggered while n checkpoint attempts are currently in flight. For the next checkpoint to be triggered, one checkpoint attempt would need to finish or expire.</td>
</tr>
<tr>
<td><h5>execution.checkpointing.min-pause</h5></td>
<td style="word-wrap: break-word;">0ms</td>
<td>Duration</td>
<td>The minimal pause between checkpointing attempts. This setting defines how soon thecheckpoint coordinator may trigger another checkpoint after it becomes possible to triggeranother checkpoint with respect to the maximum number of concurrent checkpoints(see <span markdown="span">`execution.checkpointing.max-concurrent-checkpoints`</span>).<br /><br />If the maximum number of concurrent checkpoints is set to one, this setting makes effectively sure that a minimum amount of time passes where no checkpoint is in progress at all.</td>
</tr>
<tr>
<td><h5>execution.checkpointing.mode</h5></td>
<td style="word-wrap: break-word;">EXACTLY_ONCE</td>
<td><p>Enum</p>Possible values: [EXACTLY_ONCE, AT_LEAST_ONCE]</td>
<td>The checkpointing mode (exactly-once vs. at-least-once).</td>
</tr>
<tr>
<td><h5>execution.checkpointing.prefer-checkpoint-for-recovery</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
<td>If enabled, a job recovery should fallback to checkpoint when there is a more recent savepoint.</td>
</tr>
<tr>
<td><h5>execution.checkpointing.timeout</h5></td>
<td style="word-wrap: break-word;">600000ms</td>
<td>Duration</td>
<td>The maximum time that a checkpoint may take before being discarded.</td>
</tr>
<tr>
<td><h5>execution.checkpointing.tolerable-failed-checkpoints</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>Integer</td>
<td>The tolerable checkpoint failure number. If set to 0, that meanswe do not tolerance any checkpoint failure.</td>
</tr>
</tbody>
</table>
1 change: 1 addition & 0 deletions docs/ops/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ The configuration keys in this section are independent of the used resource mana
### Checkpointing

{% include generated/checkpointing_configuration.html %}
{% include generated/execution_checkpointing_configuration.html %}

### RocksDB State Backend

Expand Down
1 change: 1 addition & 0 deletions docs/ops/config.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ The configuration keys in this section are independent of the used resource mana
### Checkpointing

{% include generated/checkpointing_configuration.html %}
{% include generated/execution_checkpointing_configuration.html %}

### RocksDB State Backend

Expand Down
5 changes: 5 additions & 0 deletions flink-docs/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ under the License.
<artifactId>flink-java</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime_${scala.binary.version}</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ public class ConfigOptionsDocGenerator {
new OptionsClassLocation("flink-core", "org.apache.flink.configuration"),
new OptionsClassLocation("flink-runtime", "org.apache.flink.runtime.shuffle"),
new OptionsClassLocation("flink-runtime", "org.apache.flink.runtime.jobgraph"),
new OptionsClassLocation("flink-streaming-java", "org.apache.flink.streaming.api.environment"),
new OptionsClassLocation("flink-yarn", "org.apache.flink.yarn.configuration"),
new OptionsClassLocation("flink-mesos", "org.apache.flink.mesos.configuration"),
new OptionsClassLocation("flink-mesos", "org.apache.flink.mesos.runtime.clusterframework"),
Expand All @@ -76,6 +77,7 @@ public class ConfigOptionsDocGenerator {
"org.apache.flink.configuration.ReadableConfig",
"org.apache.flink.configuration.WritableConfig",
"org.apache.flink.configuration.ConfigOptions",
"org.apache.flink.streaming.api.environment.CheckpointConfig",
"org.apache.flink.contrib.streaming.state.PredefinedOptions"));

static final String DEFAULT_PATH_PREFIX = "src/main/java";
Expand Down

0 comments on commit 65a472f

Please sign in to comment.