Debezium is a distributed platform that turns your existing databases into event streams, so applications can quickly react to each row-level change in the databases.
Debezium can be deployed either as connector instances in a Kafka Connect cluster, or as a standalone application - Debezium Server. Debezium Server is a Quarkus-based high-performance application that streams data from database to a one of supported sinks or a user developed sink.
Debezium Server supports multiple converters to provide different output message formats.
The image requires as a dependency source and sink systems to read data from and write output messages to.
The application itself can be configured either via environment variables or via appliaction.properties injected into the container via a volume.
Starting an instance of Debezium Server using this container image is simple:
$ docker run -it --name debezium -p 8080:8080 -v $PWD/conf:/debezium/conf -v $PWD/data:/debezium/data debezium/server
If you want to try the image yourself then please follow the steps to establish the necessary environment.
Start PostgreSQL source database:
$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres debezium/example-postgres
Start Apache Pulsar sink:
$ docker run -d --name pulsar -p 6650:6650 -p 7080:8080 apachepulsar/pulsar:2.5.2 bin/pulsar standalone
Wait for Pulsar sink to start:
$ docker logs -f pulsar
Prepare Debezium Server deployment:
$ mkdir {data,conf}; chmod 777 {data,conf}
$ cat <<-EOF > conf/application.properties
debezium.sink.type=pulsar
debezium.sink.pulsar.client.serviceUrl=pulsar://pulsar:6650
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.offset.flush.interval.ms=0
debezium.source.database.hostname=postgres
debezium.source.database.port=5432
debezium.source.database.user=postgres
debezium.source.database.password=postgres
debezium.source.database.dbname=postgres
debezium.source.database.server.name=tutorial
debezium.source.schema.whitelist=inventory
debezium.source.plugin.name=pgoutput
EOF
Note that the configuration file can be replaced with environment variables where every property translates to uppercase and dots are replaced with underscore, e.g. debezium.sink.type becomes DEBEZIUM_SINK_TYPE.
Start the Debezium Server:
$ docker run -it --name debezium -p 8080:8080 -v $PWD/conf:/debezium/conf -v $PWD/data:/debezium/data --link postgres --link pulsar debezium/server
The Debezium Server image uses several environment variables to configure JVM and source/sink when running this image.
This environment variable is passed to command line when java command is invoked.
It could be used to tune memory settings etc.
This environment variable is used in the same way as JAVA_OPTS and servers only for logical separation of Debezium Server specific settings.
All configuration options that could be present in application.properties can be either added or overridden via environment variables.
This is enabled by using MicroProfile Config in Debezium Server.
Containers created using this image will expose port 8080, which is the standard port to access MicroProfile Health endpoint.
The container image exposes two volumes:
In this volume the configuration files (mostly application.properties) are located.
In this volume the data files (mostly file offset storage) are located.