forked from facebook/rocksdb
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Blog post for write_unprepared (facebook#5711)
Summary: Introducing write_unprepared feature. Pull Request resolved: facebook#5711 Differential Revision: D16838307 Pulled By: maysamyabandeh fbshipit-source-id: d9a4daf63dd0f855bea49c14ce84e6299f1401c7
- Loading branch information
1 parent
d61d450
commit 6ec2bf3
Showing
1 changed file
with
56 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
--- | ||
title: Higher write throughput with `unordered_write` feature | ||
layout: post | ||
author: maysamyabandeh | ||
category: blog | ||
--- | ||
|
||
Since RocksDB 6.3, The `unordered_write=`true option together with WritePrepared transactions offers 34-42% higher write throughput compared to vanilla RocksDB. If the application can handle more relaxed ordering guarantees, the gain in throughput would increase to 63-131%. | ||
|
||
### Background | ||
|
||
Currently RocksDB API delivers the following powerful guarantees: | ||
- Atomic reads: Either all of a write batch is visible to reads or none of it. | ||
- Read-your-own writes: When a write thread returns to the user, a subsequent read by the same thread will be able to see its own writes. | ||
- Immutable Snapshots: The reads visible to the snapshot are immutable in the sense that it will not be affected by any in-flight or future writes. | ||
|
||
### `unordered_write` | ||
|
||
The `unordered_write` feature, when turned on, relaxes the default guarantees of RocksDB. While it still gives read-your-own-write property, neither atomic reads nor the immutable snapshot properties are provided any longer. However, RocksDB users could still get read-your-own-write and immutable snapshots when using this feature in conjunction with TransactionDB configured with WritePrepared transactions and `two_write_queues`. You can read [here](https://github.com/facebook/rocksdb/wiki/unordered_write) to learn about the design of `unordered_write` and [here](https://github.com/facebook/rocksdb/wiki/WritePrepared-Transactions) to learn more about WritePrepared transactions. | ||
|
||
### How to use it? | ||
|
||
To get the same guarantees as vanilla RocksdB: | ||
|
||
DBOptions db_options; | ||
db_options.unordered_write = true; | ||
db_options.two_write_queues = true; | ||
DB* db; | ||
{ | ||
TransactionDBOptions txn_db_options; | ||
txn_db_options.write_policy = TxnDBWritePolicy::WRITE_PREPARED; | ||
txn_db_options.skip_concurrency_control = true; | ||
TransactionDB* txn_db; | ||
TransactionDB::Open(options, txn_db_options, kDBPath, &txn_db); | ||
db = txn_db; | ||
} | ||
db->Write(...); | ||
|
||
To get relaxed guarantees: | ||
|
||
DBOptions db_options; | ||
db_options.unordered_write = true; | ||
DB* db; | ||
DB::Open(db_options, kDBPath, &db); | ||
db->Write(...); | ||
|
||
# Benchmarks | ||
|
||
TEST_TMPDIR=/dev/shm/ ~/db_bench --benchmarks=fillrandom --threads=32 --num=10000000 -max_write_buffer_number=16 --max_background_jobs=64 --batch_size=8 --writes=3000000 -level0_file_num_compaction_trigger=99999 --level0_slowdown_writes_trigger=99999 --level0_stop_writes_trigger=99999 -enable_pipelined_write=false -disable_auto_compactions --transaction_db=true --unordered_write=1 --disable_wal=0 | ||
|
||
Throughput with `unordered_write`=true and using WritePrepared transaction: | ||
- WAL: +42% | ||
- No-WAL: +34% | ||
Throughput with `unordered_write`=true | ||
- WAL: +63% | ||
- NoWAL: +131% |