NetflixGraph is a compact in-memory data structure used to represent directed graph data. You can use NetflixGraph to vastly reduce the size of your application’s memory footprint, potentially by an order of magnitude or more. If your application is I/O bound, you may be able to remove that bottleneck by holding your entire dataset in RAM. This may be possible with NetflixGraph; you’ll likely be very surprised by how little memory is actually required to represent your data.
NetflixGraph provides an API to translate your data into a graph format, compress that data in memory, then serialize the compressed in-memory representation of the data so that it may be easily transported across your infrastructure.
NetflixGraph is currently in use at Netflix. Issues generally are fixed as quickly as possible and releases done frequently.
The NetflixGraph binaries are published to Maven Central.
| GroupID/Org | ArtifactID/Name |
|---|---|
| com.netflix.nfgraph | netflix-graph |
A quick overview can be found either on the wiki or on the Netflix Tech Blog.
Since the blog article was published, a number of improvements have been made:
- contains() operations on connection sets are no longer necessarily O(n). Some memory-efficiency can be sacrificed to obtain O(1) performance.
- The library will automatically switch from the default encoding to bit set encoding if it is more memory efficient.
- Interfaces have been added to efficiently map Objects to ordinals.
Detailed documentation of NetflixGraph's features and usage can be found on the wiki.
Javadocs are available here.
NetflixGraph is built via Gradle (www.gradle.org). To build from the command line:
./gradlew build
Support can be obtained through the NetflixGraph google group