Dione

Dione - an indexing Library for data on HDFS.

The main offering is APIs for building an index for data on HDFS and querying the index in both:

• Random Access Fetches - get(key) with near-realtime latency (in Hadoop terms), and low throughput.
• Batch Processing of Specific Rows - using Spark as a distributed processing engine, retrive subset of the data (0.1% to 100% of key space) much faster than Spark/Hive joins.

Main Features

• Data and index are available for batch processing.
• Use the same technology stack for the index and for the data.
• No data duplication.
• Support multiple indices for the same data.
• No need to be the data owner.

Quick Start

See our Quick Start guide.

Detailed Documentation

Please refer to our Dione documentation.

Reporting an issue

Please open issues in the GitHub issues.

License

This project is licensed under the Apache 2 License.