Data Engineering examples for Airflow, Prefect, and Mage.ai; dbt for BigQuery, Redshift, ClickHouse, PostgreSQL; Spark/PySpark for Batch processing; and Kafka for Stream processing
airflow kafka spark pyspark prefect airflow-dags ksqldb typer-cli dbt-bigquery dbt-postgres mageai dbt-clickhouse dbt-redshift
-
Updated
Dec 11, 2024 - Python