The data-validation toolkit for enhanced dbt (data build tool) PR review

PySpark Hive helper methods

Basic Spark utilities

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data P…

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

The dbt-spark-livy adapter allows you to use dbt along with Apache Spark, by connecting via Apache Livy

Spark ClickHouse Connector build on DataSourceV2 API

Apache Iceberg

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team co…

The best way to write secure and reliable applications. Write nothing; deploy nowhere.

Lets Airflow DAGs run Spark jobs via Livy: sessions and/or batches.

Example project with Databricks jobs and configuration management via jsonnet

Logminer SQL Parser is a performance trimmed SQL parser to read SQL produced by Oracle Logminer

CDC Kafka Connect source for Oracle Databases leveraging Oracle Logminer

Apache Phoenix