This is a prototype and a proof of concept project for now implemented in python.
The goal of the nakadi project (ნაკადი means stream in Georgian language) is to build an event bus infrastructure to:
- enable convenient development of event-driven applications
- securely and efficiently publish and consume events as easy as possible
- abstract event exchange by a stanartized RESTful API
Some additional technical requirements that we wanted to cover by this architecture:
- event ordering guarantees
- fast (near real-time) event processing
- scalable and highly available architecture
- STUPS compatible
Additional topics, that we plan to cover in the near future are:
- discoverability of the resource structures flowing into the event bus
- centralized discovery service, that will use these capabilities to collect resource schema information for easy lookup by developers
NOTE: it is not really clear if the resource schema discoverability service should be part of
nakadievent bus
- REST abstraction over Kafka-like queues
- support of event filtering per Subscription
- streaming/batching of events to/from the clients
- creation of topics
- low-level interface
- manual client side partition management is needed
- no support of commits
- high-level interface
- automatic redistribution of partitions between consuming clients
- commits should be issued to move server-side cursors
To run the project locally
Simple Nakadi startup:
gradle run
Full development pipeline:
build -> ut/it tests (depends on access to a kafka backend) -> docker (builds docker image) -> api-tests (runs tests against the docker image)