Skip to content

Commit

Permalink
Merge pull request redis#1074 from NodeRedis/v.2.6.0-proposal
Browse files Browse the repository at this point in the history 
v.2.6.0-proposal
  • Loading branch information
@BridgeAR
BridgeAR committed Jun 1, 2016
2 parents bfdbf16 + a41cfa9 commit 79fc400
Show file tree
Hide file tree
Showing 25 changed files with 706 additions and 508 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ node_js:
- "0.10"
- "0.12"
- "4"
- "5"
- "6"
after_success: npm run coveralls
116 changes: 68 additions & 48 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,8 @@ __Tip:__ If the Redis server runs on the same machine as the client consider usi
| port | 6379 | Port of the Redis server |
| path | null | The UNIX socket string of the Redis server |
| url | null | The URL of the Redis server. Format: `[redis:]//[[user][:password@]][host][:port][/db-number][?db=db-number[&password=bar[&option=value]]]` (More info avaliable at [IANA](http://www.iana.org/assignments/uri-schemes/prov/redis)). |
| parser | hiredis | If hiredis is not installed, automatic fallback to the built-in javascript parser |
| string_numbers | null | Set to `true`, `node_redis` will return Redis number values as Strings instead of javascript Numbers. Useful if you need to handle big numbers (above `Number.MAX_SAFE_INTEGER === 2^53`). Hiredis is incapable of this behavior, so setting this option to `true` will result in the built-in javascript parser being used no matter the value of the `parser` option. |
| parser | javascript | __Deprecated__ Use either the built-in JS parser [`javascript`]() or the native [`hiredis`]() parser. __Note__ `node_redis` < 2.6 uses hiredis as default if installed. This changed in v.2.6.0. |
| string_numbers | null | Set to `true`, `node_redis` will return Redis number values as Strings instead of javascript Numbers. Useful if you need to handle big numbers (above `Number.MAX_SAFE_INTEGER === 2^53`). Hiredis is incapable of this behavior, so setting this option to `true` will result in the built-in javascript parser being used no matter the value of the `parser` option. |
| return_buffers | false | If set to `true`, then all replies will be sent to callbacks as Buffers instead of Strings. |
| detect_buffers | false | If set to `true`, then replies will be sent to callbacks as Buffers. This option lets you switch between Buffers and Strings on a per-command basis, whereas `return_buffers` applies to every command on a client. __Note__: This doesn't work properly with the pubsub mode. A subscriber has to either always return Strings or Buffers. |
| socket_keepalive | true | If set to `true`, the keep-alive functionality is enabled on the underlying socket. |
Expand Down Expand Up @@ -712,60 +712,80 @@ client.zadd(args, function (err, response) {
## Performance

Much effort has been spent to make `node_redis` as fast as possible for common
operations. As pipelining happens naturally from shared connections, overall
efficiency goes up.

Here are results of `multi_bench.js` which is similar to `redis-benchmark` from the Redis distribution.

hiredis parser (Lenovo T450s i7-5600U):
operations.

```
Client count: 1, node version: 4.2.2, server version: 3.0.3, parser: hiredis
PING, 1/1 min/max/avg: 0/ 2/ 0.02/ 2501ms total, 47503.80 ops/sec
PING, batch 50/1 min/max/avg: 0/ 2/ 0.09/ 2501ms total, 529668.13 ops/sec
SET 4B str, 1/1 min/max/avg: 0/ 2/ 0.02/ 2501ms total, 41900.04 ops/sec
SET 4B str, batch 50/1 min/max/avg: 0/ 2/ 0.14/ 2501ms total, 354658.14 ops/sec
SET 4B buf, 1/1 min/max/avg: 0/ 4/ 0.04/ 2501ms total, 23499.00 ops/sec
SET 4B buf, batch 50/1 min/max/avg: 0/ 2/ 0.31/ 2501ms total, 159836.07 ops/sec
GET 4B str, 1/1 min/max/avg: 0/ 4/ 0.02/ 2501ms total, 43489.80 ops/sec
GET 4B str, batch 50/1 min/max/avg: 0/ 2/ 0.11/ 2501ms total, 444202.32 ops/sec
GET 4B buf, 1/1 min/max/avg: 0/ 3/ 0.02/ 2501ms total, 38561.38 ops/sec
GET 4B buf, batch 50/1 min/max/avg: 0/ 2/ 0.11/ 2501ms total, 452139.14 ops/sec
SET 4KiB str, 1/1 min/max/avg: 0/ 2/ 0.03/ 2501ms total, 32990.80 ops/sec
SET 4KiB str, batch 50/1 min/max/avg: 0/ 2/ 0.34/ 2501ms total, 146161.54 ops/sec
SET 4KiB buf, 1/1 min/max/avg: 0/ 1/ 0.04/ 2501ms total, 23294.28 ops/sec
SET 4KiB buf, batch 50/1 min/max/avg: 0/ 2/ 0.36/ 2501ms total, 137584.97 ops/sec
GET 4KiB str, 1/1 min/max/avg: 0/ 2/ 0.03/ 2501ms total, 36350.66 ops/sec
GET 4KiB str, batch 50/1 min/max/avg: 0/ 2/ 0.32/ 2501ms total, 155157.94 ops/sec
GET 4KiB buf, 1/1 min/max/avg: 0/ 4/ 0.02/ 2501ms total, 39776.49 ops/sec
GET 4KiB buf, batch 50/1 min/max/avg: 0/ 2/ 0.32/ 2501ms total, 155457.82 ops/sec
INCR, 1/1 min/max/avg: 0/ 3/ 0.02/ 2501ms total, 43972.41 ops/sec
INCR, batch 50/1 min/max/avg: 0/ 1/ 0.12/ 2501ms total, 425809.68 ops/sec
LPUSH, 1/1 min/max/avg: 0/ 2/ 0.02/ 2501ms total, 38998.40 ops/sec
LPUSH, batch 50/1 min/max/avg: 0/ 4/ 0.14/ 2501ms total, 365013.99 ops/sec
LRANGE 10, 1/1 min/max/avg: 0/ 2/ 0.03/ 2501ms total, 31879.25 ops/sec
LRANGE 10, batch 50/1 min/max/avg: 0/ 1/ 0.32/ 2501ms total, 153698.52 ops/sec
LRANGE 100, 1/1 min/max/avg: 0/ 4/ 0.06/ 2501ms total, 16676.13 ops/sec
LRANGE 100, batch 50/1 min/max/avg: 1/ 6/ 2.03/ 2502ms total, 24520.38 ops/sec
SET 4MiB str, 1/1 min/max/avg: 1/ 6/ 2.11/ 2502ms total, 472.82 ops/sec
SET 4MiB str, batch 20/1 min/max/avg: 85/ 112/ 94.93/ 2563ms total, 210.69 ops/sec
SET 4MiB buf, 1/1 min/max/avg: 1/ 8/ 2.02/ 2502ms total, 490.01 ops/sec
SET 4MiB buf, batch 20/1 min/max/avg: 37/ 52/ 39.48/ 2528ms total, 506.33 ops/sec
GET 4MiB str, 1/1 min/max/avg: 3/ 13/ 5.26/ 2504ms total, 190.10 ops/sec
GET 4MiB str, batch 20/1 min/max/avg: 70/ 106/ 89.36/ 2503ms total, 223.73 ops/sec
GET 4MiB buf, 1/1 min/max/avg: 3/ 11/ 5.04/ 2502ms total, 198.24 ops/sec
GET 4MiB buf, batch 20/1 min/max/avg: 70/ 105/ 88.07/ 2554ms total, 227.09 ops/sec
Lenovo T450s, i7-5600U and 12gb memory
clients: 1, NodeJS: 6.2.0, Redis: 3.2.0, parser: javascript, connected by: tcp
PING, 1/1 avg/max: 0.02/ 5.26 2501ms total, 46916 ops/sec
PING, batch 50/1 avg/max: 0.06/ 4.35 2501ms total, 755178 ops/sec
SET 4B str, 1/1 avg/max: 0.02/ 4.75 2501ms total, 40856 ops/sec
SET 4B str, batch 50/1 avg/max: 0.11/ 1.51 2501ms total, 432727 ops/sec
SET 4B buf, 1/1 avg/max: 0.05/ 2.76 2501ms total, 20659 ops/sec
SET 4B buf, batch 50/1 avg/max: 0.25/ 1.76 2501ms total, 194962 ops/sec
GET 4B str, 1/1 avg/max: 0.02/ 1.55 2501ms total, 45156 ops/sec
GET 4B str, batch 50/1 avg/max: 0.09/ 3.15 2501ms total, 524110 ops/sec
GET 4B buf, 1/1 avg/max: 0.02/ 3.07 2501ms total, 44563 ops/sec
GET 4B buf, batch 50/1 avg/max: 0.10/ 3.18 2501ms total, 473171 ops/sec
SET 4KiB str, 1/1 avg/max: 0.03/ 1.54 2501ms total, 32627 ops/sec
SET 4KiB str, batch 50/1 avg/max: 0.34/ 1.89 2501ms total, 146861 ops/sec
SET 4KiB buf, 1/1 avg/max: 0.05/ 2.85 2501ms total, 20688 ops/sec
SET 4KiB buf, batch 50/1 avg/max: 0.36/ 1.83 2501ms total, 138165 ops/sec
GET 4KiB str, 1/1 avg/max: 0.02/ 1.37 2501ms total, 39389 ops/sec
GET 4KiB str, batch 50/1 avg/max: 0.24/ 1.81 2501ms total, 208157 ops/sec
GET 4KiB buf, 1/1 avg/max: 0.02/ 2.63 2501ms total, 39918 ops/sec
GET 4KiB buf, batch 50/1 avg/max: 0.31/ 8.56 2501ms total, 161575 ops/sec
INCR, 1/1 avg/max: 0.02/ 4.69 2501ms total, 45685 ops/sec
INCR, batch 50/1 avg/max: 0.09/ 3.06 2501ms total, 539964 ops/sec
LPUSH, 1/1 avg/max: 0.02/ 3.04 2501ms total, 41253 ops/sec
LPUSH, batch 50/1 avg/max: 0.12/ 1.94 2501ms total, 425090 ops/sec
LRANGE 10, 1/1 avg/max: 0.02/ 2.28 2501ms total, 39850 ops/sec
LRANGE 10, batch 50/1 avg/max: 0.25/ 1.85 2501ms total, 194302 ops/sec
LRANGE 100, 1/1 avg/max: 0.05/ 2.93 2501ms total, 21026 ops/sec
LRANGE 100, batch 50/1 avg/max: 1.52/ 2.89 2501ms total, 32767 ops/sec
SET 4MiB str, 1/1 avg/max: 5.16/ 15.55 2502ms total, 193 ops/sec
SET 4MiB str, batch 20/1 avg/max: 89.73/ 99.96 2513ms total, 223 ops/sec
SET 4MiB buf, 1/1 avg/max: 2.23/ 8.35 2501ms total, 446 ops/sec
SET 4MiB buf, batch 20/1 avg/max: 41.47/ 50.91 2530ms total, 482 ops/sec
GET 4MiB str, 1/1 avg/max: 2.79/ 10.91 2502ms total, 358 ops/sec
GET 4MiB str, batch 20/1 avg/max: 101.61/118.11 2541ms total, 197 ops/sec
GET 4MiB buf, 1/1 avg/max: 2.32/ 14.93 2502ms total, 430 ops/sec
GET 4MiB buf, batch 20/1 avg/max: 65.01/ 84.72 2536ms total, 308 ops/sec
```

The hiredis and js parser should most of the time be on the same level. But if you use Redis for big SUNION/SINTER/LRANGE/ZRANGE hiredis is faster.
Therefor the hiredis parser is the default used in node_redis. To use `hiredis`, do:

npm install hiredis redis

## Debugging

To get debug output run your `node_redis` application with `NODE_DEBUG=redis`.

This is also going to result in good stack traces opposed to useless ones otherwise for any async operation.
If you only want to have good stack traces but not the debug output run your application in development mode instead (`NODE_ENV=development`).

Good stack traces are only activated in development and debug mode as this results in a significant performance penalty.

___Comparison___:
Useless stack trace:
```
ReplyError: ERR wrong number of arguments for 'set' command
at parseError (/home/ruben/repos/redis/node_modules/redis-parser/lib/parser.js:158:12)
at parseType (/home/ruben/repos/redis/node_modules/redis-parser/lib/parser.js:219:14)
```
Good stack trace:
```
ReplyError: ERR wrong number of arguments for 'set' command
at new Command (/home/ruben/repos/redis/lib/command.js:9:902)
at RedisClient.set (/home/ruben/repos/redis/lib/commands.js:9:3238)
at Context.<anonymous> (/home/ruben/repos/redis/test/good_stacks.spec.js:20:20)
at callFnAsync (/home/ruben/repos/redis/node_modules/mocha/lib/runnable.js:349:8)
at Test.Runnable.run (/home/ruben/repos/redis/node_modules/mocha/lib/runnable.js:301:7)
at Runner.runTest (/home/ruben/repos/redis/node_modules/mocha/lib/runner.js:422:10)
at /home/ruben/repos/redis/node_modules/mocha/lib/runner.js:528:12
at next (/home/ruben/repos/redis/node_modules/mocha/lib/runner.js:342:14)
at /home/ruben/repos/redis/node_modules/mocha/lib/runner.js:352:7
at next (/home/ruben/repos/redis/node_modules/mocha/lib/runner.js:284:14)
at Immediate._onImmediate (/home/ruben/repos/redis/node_modules/mocha/lib/runner.js:320:5)
at processImmediate [as _immediateCallback] (timers.js:383:17)
```

## How to Contribute
- Open a pull request or an issue about what you want to implement / change. We're glad for any help!
- Please be aware that we'll only accept fully tested code.
Expand Down
7 changes: 3 additions & 4 deletions appveyor.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ environment:
- nodejs_version: "0.10"
- nodejs_version: "0.12"
- nodejs_version: "4"
- nodejs_version: "5"
- nodejs_version: "6"

pull_requests:
do_not_increment_build_number: true
Expand All @@ -21,11 +21,10 @@ install:
- redis-64\tools\redis-server.exe --service-install
- redis-64\tools\redis-server.exe --service-start
- '@ECHO Redis Started'
# Get the latest stable version of Node 0.STABLE.latest
# Get the required Node version
- ps: Install-Product node $env:nodejs_version
# Typical npm stuff. Use msvs 2013 for the hiredis parser
# Typical npm stuff
- npm install
- npm install hiredis --msvs_version=2013

# Post-install test scripts.
test_script:
Expand Down
14 changes: 7 additions & 7 deletions benchmarks/multi_bench.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ var run_time = returnArg('time', 2500); // ms
var pipeline = returnArg('pipeline', 1); // number of concurrent commands
var versions_logged = false;
var client_options = {
parser: returnArg('parser', 'hiredis'),
parser: returnArg('parser', 'javascript'),
path: returnArg('socket') // '/tmp/redis.sock'
};
var small_str, large_str, small_buf, large_buf, very_large_str, very_large_buf;
Expand Down Expand Up @@ -231,8 +231,8 @@ tests.push(new Test({descr: 'SET 4B buf', command: 'set', args: ['foo_rand000000
tests.push(new Test({descr: 'GET 4B str', command: 'get', args: ['foo_rand000000000000']}));
tests.push(new Test({descr: 'GET 4B str', command: 'get', args: ['foo_rand000000000000'], batch: 50}));

tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], batch: 50, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], client_options: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], batch: 50, client_options: { return_buffers: true} }));

tests.push(new Test({descr: 'SET 4KiB str', command: 'set', args: ['foo_rand000000000001', large_str]}));
tests.push(new Test({descr: 'SET 4KiB str', command: 'set', args: ['foo_rand000000000001', large_str], batch: 50}));
Expand All @@ -243,8 +243,8 @@ tests.push(new Test({descr: 'SET 4KiB buf', command: 'set', args: ['foo_rand0000
tests.push(new Test({descr: 'GET 4KiB str', command: 'get', args: ['foo_rand000000000001']}));
tests.push(new Test({descr: 'GET 4KiB str', command: 'get', args: ['foo_rand000000000001'], batch: 50}));

tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], batch: 50, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], client_options: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], batch: 50, client_options: { return_buffers: true} }));

tests.push(new Test({descr: 'INCR', command: 'incr', args: ['counter_rand000000000000']}));
tests.push(new Test({descr: 'INCR', command: 'incr', args: ['counter_rand000000000000'], batch: 50}));
Expand All @@ -267,8 +267,8 @@ tests.push(new Test({descr: 'SET 4MiB buf', command: 'set', args: ['foo_rand0000
tests.push(new Test({descr: 'GET 4MiB str', command: 'get', args: ['foo_rand000000000002']}));
tests.push(new Test({descr: 'GET 4MiB str', command: 'get', args: ['foo_rand000000000002'], batch: 20}));

tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], batch: 20, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], client_options: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], batch: 20, client_options: { return_buffers: true} }));

function next () {
var test = tests.shift();
Expand Down
23 changes: 21 additions & 2 deletions changelog.md
View file
Original file line number Diff line number Diff line change
@@ -1,18 +1,37 @@
Changelog
=========

## v.2.6.0 - 25 Mai, 2016

In addition to the pre-releases the following changes exist in v.2.6.0:

Features

- Updated [redis-parser](https://github.com/NodeRedis/redis-parser) dependency ([changelog](https://github.com/NodeRedis/redis-parser/releases/tag/v.2.0.0))
- The JS parser is from now on the new default as it is a lot faster than the hiredis parser
- This is no BC as there is no changed behavior for the user at all but just a performance improvement. Explicitly requireing the Hiredis parser is still possible.
- Added name property to all Redis functions (Node.js >= 4.0)
- Improved stack traces in development and debug mode

Bugfixes

- Reverted support for `__proto__` (v.2.6.0-2) to prevent and breaking change

Deprecations

- The `parser` option is deprecated and should be removed. The built-in Javascript parser is a lot faster than the hiredis parser and has more features

## v.2.6.0-2 - 29 Apr, 2016

Features

- Added support for the new `CLIENT REPLY ON|OFF|SKIP` command (Redis v.3.2)
- Added support for the new [CLIENT REPLY ON|OFF|SKIP](http://redis.io/commands/client-reply) command (Redis v.3.2)
- Added support for camelCase
- The Node.js landscape default is to use camelCase. node_redis is a bit out of the box here
but from now on it is possible to use both, just as you prefer!
- If there's any documented variable missing as camelCased, please open a issue for it
- Improve error handling significantly
- Only emit an error if the error has not already been handled in a callback
- Emit an error if a command would otherwise silently fail (no callback present)
- Improved unspecific error messages e.g. "Connection gone from end / close event"
- Added `args` to command errors to improve identification of the error
- Added origin to errors if there's e.g. a connection error
Expand Down
Loading

0 comments on commit 79fc400

Please sign in to comment.