Skip to content

Latest commit

 

History

History
81 lines (52 loc) · 6.05 KB

changefeed-for.md

File metadata and controls

81 lines (52 loc) · 6.05 KB
title summary toc docs_area
EXPERIMENTAL CHANGEFEED FOR
which streams row-level changes to the client indefinitely until the underlying connection is closed or the changefeed is canceled.
true
reference.sql

{{site.data.alerts.callout_info}} EXPERIMENTAL CHANGEFEED FOR is the core implementation of changefeeds. For the Enterprise-only version, see CREATE CHANGEFEED. {{site.data.alerts.end}}

The EXPERIMENTAL CHANGEFEED FOR statement creates a new core changefeed, which streams row-level changes to the client indefinitely until the underlying connection is closed or the changefeed is canceled. A core changefeed can watch one table or multiple tables in a comma-separated list.

For more information, see Stream Data Out of CockroachDB Using Changefeeds.

{% include feature-phases/preview.md %}

Required privileges

Changefeeds can only be created by superusers, i.e., members of the admin role. The admin role exists by default with root as the member.

Considerations

  • Because core changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for core changefeeds. Core changefeeds also have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a CANCEL QUERY statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default.

    This cancellation behavior (i.e., close the underlying connection to cancel the changefeed) also extends to client driver usage; in particular, when a client driver calls Rows.Close() after encountering errors for a stream of rows. The pgwire protocol requires that the rows be consumed before the connection is again usable, but in the case of a core changefeed, the rows are never consumed. It is therefore critical that you close the connection, otherwise the application will be blocked forever on Rows.Close().

  • In most cases, each version of a row will be emitted once. However, some infrequent conditions (e.g., node failures, network partitions) will cause them to be repeated. This gives our changefeeds an at-least-once delivery guarantee. For more information, see Change Data Capture - Ordering Guarantees.

Synopsis

> EXPERIMENTAL CHANGEFEED FOR table_name [ WITH (option [= value] [, ...]) ];

Parameters

Parameter Description
table_name The name of the table (or tables in a comma separated list) to create a changefeed for.
option / value For a list of available options and their values, see Options below.

Options

Option Value Description
updated N/A Include updated timestamps with each row.
resolved INTERVAL Emits resolved timestamp events for the changefeed. Resolved timestamp events do not emit until all ranges in the changefeed have progressed to a specific point in time.

Set an optional minimal duration between emitting resolved timestamps. Example: resolved='10s'. This option will only emit a resolved timestamp event if the timestamp has advanced and at least the optional duration has elapsed. If unspecified, all resolved timestamps are emitted as the high-water mark advances.
envelope key_only / row Use key_only to emit only the key and no value, which is faster if you only want to know when the key changes.

Default: envelope=row
cursor Timestamp Emits any changes after the given timestamp, but does not output the current state of the table first. If cursor is not specified, the changefeed starts by doing a consistent scan of all the watched rows and emits the current value, then moves to emitting any changes that happen after the scan.

cursor can be used to start a new changefeed where a previous changefeed ended.

Example: CURSOR=1536242855577149065.0000000000
mvcc_timestamp N/A New in v21.2: Include the MVCC timestamp for each emitted row in a changefeed. With the mvcc_timestamp option, each emitted row will always contain its MVCC timestamp, even during the changefeed's initial backfill.
format json / avro Format of the emitted record. Currently, support for Avro is limited.

Default: format=json.
confluent_schema_registry Schema Registry address The Schema Registry address is required to use avro.

Avro limitations

Below are clarifications for particular SQL types and values for Avro changefeeds:

{% include {{ page.version.version }}/cdc/avro-limitations.md %}

Examples

Create a changefeed

{% include {{ page.version.version }}/cdc/create-core-changefeed.md %}

Create a changefeed with Avro

{% include {{ page.version.version }}/cdc/create-core-changefeed-avro.md %}

See also