Skip to content
/ cdc_deltaLake Public

Docker compose and Google Colab demo to build a CDC with Delta Lake

15 stars 20 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

masfworld/cdc_deltaLake

BranchesTags

Repository files navigation

PostgreSQL (Debezium) - Kafka - Spark Delta Lake

Description

This project is a demo for testing a CDC (Change Data Capture). All infrastructure is built using docker.

alt text

Features

  • PostgreSql as Legacy database
  • Debezium as Change Data Capture
  • Kafka to ingest data from Debezium
  • KSQLDB to transform Avro messages into JSON
  • Spark Delta Lake to manage events from legacy database

About

Docker compose and Google Colab demo to build a CDC with Delta Lake

Resources

Readme
Activity

Stars

15 stars

Watchers

1 watching

Forks

20 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages