Skip to content

Commit c3ac953

Browse files
dailaidailai
authored
[Improve][CDC] Bump the version of debezium to 1.9.8.Final (apache#6740)
1 parent 1d27375 commit c3ac953

File tree

73 files changed

+3354
-5834
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+3354
-5834
lines changed

‎docs/en/connector-v2/source/MySQL-CDC.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ When an initial consistent snapshot is made for large databases, your establishe
146146
- `interactive_timeout`: The number of seconds the server waits for activity on an interactive connection before closing it. See [MySQL’s documentation](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_interactive_timeout) for more details.
147147
- `wait_timeout`: The number of seconds the server waits for activity on a non-interactive connection before closing it. See [MySQL’s documentation](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_wait_timeout) for more details.
148148

149-
*For more database settings see [Debezium MySQL Connector](https://github.com/debezium/debezium/blob/1.6/documentation/modules/ROOT/pages/connectors/mysql.adoc#set-up)*
149+
*For more database settings see [Debezium MySQL Connector](https://debezium.io/documentation/reference/1.9/connectors/mysql.html#setting-up-mysql)*
150150

151151
## Data Type Mapping
152152

@@ -196,7 +196,7 @@ When an initial consistent snapshot is made for large databases, your establishe
196196
| inverse-sampling.rate | Integer | No | 1000 | The inverse of the sampling rate used in the sample sharding strategy. For example, if this value is set to 1000, it means a 1/1000 sampling rate is applied during the sampling process. This option provides flexibility in controlling the granularity of the sampling, thus affecting the final number of shards. It's especially useful when dealing with very large datasets where a lower sampling rate is preferred. The default value is 1000. |
197197
| exactly_once | Boolean | No | false | Enable exactly once semantic. |
198198
| format | Enum | No | DEFAULT | Optional output format for MySQL CDC, valid enumerations are `DEFAULT`、`COMPATIBLE_DEBEZIUM_JSON`. |
199-
| debezium | Config | No | - | Pass-through [Debezium's properties](https://github.com/debezium/debezium/blob/1.6/documentation/modules/ROOT/pages/connectors/mysql.adoc#connector-properties) to Debezium Embedded Engine which is used to capture data changes from MySQL server. |
199+
| debezium | Config | No | - | Pass-through [Debezium's properties](https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-connector-properties) to Debezium Embedded Engine which is used to capture data changes from MySQL server. |
200200
| common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
201201

202202
## Task Example

‎docs/en/connector-v2/source/Oracle-CDC.md

+6-1
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,11 @@
2121
The Oracle CDC connector allows for reading snapshot data and incremental data from Oracle database. This document
2222
describes how to set up the Oracle CDC connector to run SQL queries against Oracle databases.
2323

24+
## Notice
25+
26+
The Debezium Oracle connector does not rely on the continuous mining option. The connector is responsible for detecting log switches and adjusting the logs that are mined automatically, which the continuous mining option did for you automatically.
27+
So, you can not set this property named `log.mining.continuous.mine` in the debezium.
28+
2429
## Supported DataSource Info
2530

2631
| Datasource | Supported versions | Driver | Url | Maven |
@@ -240,7 +245,7 @@ exit;
240245
| inverse-sampling.rate | Integer | No | 1000 | The inverse of the sampling rate used in the sample sharding strategy. For example, if this value is set to 1000, it means a 1/1000 sampling rate is applied during the sampling process. This option provides flexibility in controlling the granularity of the sampling, thus affecting the final number of shards. It's especially useful when dealing with very large datasets where a lower sampling rate is preferred. The default value is 1000. |
241246
| exactly_once | Boolean | No | false | Enable exactly once semantic. |
242247
| format | Enum | No | DEFAULT | Optional output format for Oracle CDC, valid enumerations are `DEFAULT`、`COMPATIBLE_DEBEZIUM_JSON`. |
243-
| debezium | Config | No | - | Pass-through [Debezium's properties](https://github.com/debezium/debezium/blob/1.6/documentation/modules/ROOT/pages/connectors/oracle.adoc#connector-properties) to Debezium Embedded Engine which is used to capture data changes from Oracle server. |
248+
| debezium | Config | No | - | Pass-through [Debezium's properties](https://debezium.io/documentation/reference/1.9/connectors/oracle.html#oracle-connector-properties) to Debezium Embedded Engine which is used to capture data changes from Oracle server. |
244249
| common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
245250

246251
## Task Example

‎docs/en/connector-v2/source/Postgre-CDC.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ ALTER TABLE your_table_name REPLICA IDENTITY FULL;
109109
| inverse-sampling.rate | Integer | No | 1000 | The inverse of the sampling rate used in the sample sharding strategy. For example, if this value is set to 1000, it means a 1/1000 sampling rate is applied during the sampling process. This option provides flexibility in controlling the granularity of the sampling, thus affecting the final number of shards. It's especially useful when dealing with very large datasets where a lower sampling rate is preferred. The default value is 1000. |
110110
| exactly_once | Boolean | No | false | Enable exactly once semantic. |
111111
| format | Enum | No | DEFAULT | Optional output format for Postgre CDC, valid enumerations are `DEFAULT`、`COMPATIBLE_DEBEZIUM_JSON`. |
112-
| debezium | Config | No | - | Pass-through [Debezium's properties](https://github.com/debezium/debezium/blob/1.6/documentation/modules/ROOT/pages/connectors/postgresql.adoc#connector-configuration-properties) to Debezium Embedded Engine which is used to capture data changes from Postgre server. |
112+
| debezium | Config | No | - | Pass-through [Debezium's properties](https://debezium.io/documentation/reference/1.9/connectors/postgresql.html#postgresql-connector-properties) to Debezium Embedded Engine which is used to capture data changes from Postgre server. |
113113
| common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
114114

115115
## Task Example

‎docs/en/connector-v2/source/SqlServer-CDC.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ describes how to setup the Sql Server CDC connector to run SQL queries against S
9191
| sample-sharding.threshold | int | No | 1000 | This configuration specifies the threshold of estimated shard count to trigger the sample sharding strategy. When the distribution factor is outside the bounds specified by `chunk-key.even-distribution.factor.upper-bound` and `chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count (calculated as approximate row count / chunk size) exceeds this threshold, the sample sharding strategy will be used. This can help to handle large datasets more efficiently. The default value is 1000 shards. |
9292
| inverse-sampling.rate | int | No | 1000 | The inverse of the sampling rate used in the sample sharding strategy. For example, if this value is set to 1000, it means a 1/1000 sampling rate is applied during the sampling process. This option provides flexibility in controlling the granularity of the sampling, thus affecting the final number of shards. It's especially useful when dealing with very large datasets where a lower sampling rate is preferred. The default value is 1000. |
9393
| exactly_once | Boolean | No | false | Enable exactly once semantic. |
94-
| debezium.* | config | No | - | Pass-through Debezium's properties to Debezium Embedded Engine which is used to capture data changes from SqlServer server.<br/>See more about<br/>the [Debezium's SqlServer Connector properties](https://github.com/debezium/debezium/blob/1.6/documentation/modules/ROOT/pages/connectors/sqlserver.adoc#connector-properties) |
94+
| debezium.* | config | No | - | Pass-through Debezium's properties to Debezium Embedded Engine which is used to capture data changes from SqlServer server.<br/>See more about<br/>the [Debezium's SqlServer Connector properties](https://debezium.io/documentation/reference/1.9/connectors/sqlserver.html#sqlserver-connector-properties) |
9595
| format | Enum | No | DEFAULT | Optional output format for SqlServer CDC, valid enumerations are "DEFAULT"、"COMPATIBLE_DEBEZIUM_JSON". |
9696
| common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details. |
9797

‎seatunnel-connectors-v2/connector-cdc/connector-cdc-base/pom.xml

+5
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,11 @@
110110
<groupId>org.apache.seatunnel</groupId>
111111
<artifactId>seatunnel-format-compatible-debezium-json</artifactId>
112112
</dependency>
113+
<dependency>
114+
<groupId>org.apache.commons</groupId>
115+
<artifactId>commons-lang3</artifactId>
116+
<version>${commons-lang3.version}</version>
117+
</dependency>
113118

114119
<dependency>
115120
<groupId>junit</groupId>

0 commit comments

Comments
 (0)