- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 87
Home
Kiba is a lightweight Ruby ETL (Extract - Transform - Load) framework. It makes it easy to write reliable, concise, well-tested & maintainable data-processing code in Ruby.
Kiba development & maintenance is funded by LoGeek, a French data-oriented consultancy.
Read for more info:
- Core Concepts
- How to define ETL jobs with Kiba
- How to run ETL jobs with Kiba
- Implementing ETL sources
- Implementing ETL transforms
- Implementing ETL destinations
- Implementing pre and post-processors
- Implementation Guidelines
- How to extend the Kiba DSL
- Can Kiba handle multiple sources and destinations?
- Ruby 2.7+ keyword arguments compatibility note
- Article: how to sort rows with Kiba
Kiba Common is an OSS companion gem to Kiba, bringing commonly used features to Kiba:
- Enumerable Source (turn any enumerable into a stream of rows)
- Enumerable Exploder (explodes an enumerable row into a sequence of output rows)
- Source Transform Adapter (dynamical sources using input rows as instantiation parameters)
- CSV Source
- CSV Destination
- Lambda Destination
- How to add logging to Kiba?
- How to pretty-color-print rows in Kiba?
Kiba Pro adds powerful vendor-backed extensions for Kiba ETL. Purchasing a Kiba Pro subscription (via https://www.kiba-etl.org/kiba-pro) also supports the continued maintenance & development of Kiba Open-Source.
- SQL Source
- SQL Bulk Lookup (handles relationship references, many rows at once)
- SQL Upsert Destination
- SQL Bulk Insert/Upsert Destination
- File Lock (avoids concurrent ETL jobs)
- Parallel Transform (multithreaded acceleration of e.g. HTTP queries)
- Commercial FAQ (license & pricing)
- Purchase
- Changelog
- How to install Kiba Pro
- Kiba Blueprints
Home | Core Concepts | Defining jobs | Running jobs | Writing sources | Writing transforms | Writing destinations | Implementation Guidelines | Kiba Pro
This wiki is tracked by git and publicly editable. You are welcome to fix errors and typos. Any defacing or vandalism of content will result in your changes being reverted and you being blocked.