Skip to content

Ruby wrapper for hiredis with multithreading support via Hiredis::ThreadSafeConnection

License

Notifications You must be signed in to change notification settings

seosgithub/threadsafe-hiredis-rb

 
 

Repository files navigation

threadsafe-hiredis-rb

There are issues with this library and certain stacks. Current advice is to use redis-rb for the time being.

This is a fork of the original hiredis-rb that includes threadsafe support. All credit for hiredis-rb is attributed to Pieter Noordhius (https://github.com/redis/hiredis-rb).

Do not include hiredis with this gem, this is not a plugin, this is a full backwards compatible fork that only has an additional class for thread safe connections (''Hiredis::ThreadSafeConnection'') and a special driver for redisrb. The include statement remains 'redis', not 'threadsafe-hiredis'.

Ruby extension that wraps hiredis. Both the synchronous connection API and a separate protocol reader are supported. It is primarily intended to speed up parsing multi bulk replies. Supports multi-threaded environments with a thread pool

Install

Install with Rubygems:

gem install threadsafe-hiredis

Usage

ThreadsafeHiredis can be used as standalone library, or be used together with redis-rb. The latter adds in support for hiredis in 2.2.

redis-rb

To use threadsafe-hiredis from redis-rb, it needs to be available in Ruby's load path. Using Bundler, this comes down to adding the following lines:

gem "threadsafe-hiredis"
gem "redis", ">= 2.2.0"

To use hiredis with redis-rb, you need to require `hiredis/thread_safe_connection_redisrb_driver' before creating a new redisrb client. This makes sure that hiredis will be used instead of the pure Ruby connection code and protocol parser. You can use Redis normally, as you would with the pure Ruby version.

Standalone: Connection

A connection to Redis can be opened by creating an instance of Hiredis::ThreadSafeConnection and calling #connect:

conn = Hiredis::ThreadSafeConnection.new
conn.connect("127.0.0.1", 6379)

Commands can be written to Redis by calling #write with an array of arguments. You can call write more than once, resulting in a pipeline of commands.

conn.write ["SET", "speed", "awesome"]
conn.write ["GET", "speed"]

After commands are written, use #read to receive the subsequent replies. Make sure not to call #read more than you have replies to read, or the connection will block indefinitely. You can use this feature to implement a subscriber (for Redis Pub/Sub).

>> conn.read
=> "OK"

>> conn.read
=> "awesome"

When the connection was closed by the server, an error of the type Hiredis::Connection::EOFError will be raised. For all I/O related errors, the Ruby built-in Errno::XYZ errors will be raised. All other errors (such as a protocol error) result in a RuntimeError.

Connections can be created lazily, but this can be prevented by persisting your thread pool and passing the optional argument standby_pool_size to Hiredis::ThreadSafeConnection. The example below would create 16 connections on initializations and then pass those connections to threads when they first request a write or read. Note that if you do not persist your threads, this pool will quickly run out and Hiredis::ThreadSafeConnection will be forced to dynamically connect.

conn = Hiredis::ThreadSafeConnection standby_pool_size: 16
conn.connect("127.0.0.1", 6379)

Standalone: Reply parser

Only using hiredis for the reply parser can be very useful in scenarios where the I/O is already handled by another component (such as EventMachine).

You can skip loading everything and just load Hiredis::Reader by requiring hiredis/reader.

Use #feed on an instance of Hiredis::Reader to feed the stream parser with new data. Use #read to get the parsed replies one by one:

>> reader = Hiredis::Reader.new
>> reader.feed("*2\r\n$7\r\nawesome\r\n$5\r\narray\r\n")
>> reader.gets
=> ["awesome", "array"]

Benchmarks

These numbers were generated by running benchmark/throughput.rb against ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]. The benchmark compares redis-rb with the Ruby connection and protocol code against redis-rb with hiredis to handle I/O and reply parsing.

For simple line or bulk replies, the throughput improvement is insignificant, while the larger multi bulk responses will have a noticeable higher throughput.

                                                        user     system      total        real
redis-rb:  1x SET pipeline, 10000 times             0.260000   0.140000   0.400000 (  0.726216)
 hiredis:  1x SET pipeline, 10000 times             0.170000   0.130000   0.300000 (  0.602332)
redis-rb: 10x SET pipeline, 10000 times             1.550000   0.660000   2.210000 (  2.231505)
 hiredis: 10x SET pipeline, 10000 times             0.400000   0.140000   0.540000 (  0.995266)

redis-rb:  1x GET pipeline, 10000 times             0.270000   0.150000   0.420000 (  0.730322)
 hiredis:  1x GET pipeline, 10000 times             0.170000   0.130000   0.300000 (  0.604060)
redis-rb: 10x GET pipeline, 10000 times             1.480000   0.640000   2.120000 (  2.122215)
 hiredis: 10x GET pipeline, 10000 times             0.380000   0.130000   0.510000 (  0.925731)

redis-rb:  1x LPUSH pipeline, 10000 times           0.260000   0.150000   0.410000 (  0.725292)
 hiredis:  1x LPUSH pipeline, 10000 times           0.160000   0.130000   0.290000 (  0.592466)
redis-rb: 10x LPUSH pipeline, 10000 times           1.540000   0.660000   2.200000 (  2.202709)
 hiredis: 10x LPUSH pipeline, 10000 times           0.360000   0.130000   0.490000 (  0.940361)

redis-rb:  1x LRANGE(100) pipeline, 1000 times      0.240000   0.020000   0.260000 (  0.307504)
 hiredis:  1x LRANGE(100) pipeline, 1000 times      0.050000   0.020000   0.070000 (  0.100293)

redis-rb:  1x LRANGE(1000) pipeline, 1000 times     2.100000   0.030000   2.130000 (  2.353551)
 hiredis:  1x LRANGE(1000) pipeline, 1000 times     0.320000   0.030000   0.350000 (  0.472789)

License

This code is released under the BSD license, after the license of hiredis-rb.

About

Ruby wrapper for hiredis with multithreading support via Hiredis::ThreadSafeConnection

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Ruby 66.0%
  • C 34.0%