Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancements of the redis-rb-cluster #6

Open
wants to merge 73 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
3f290d8
convert to 2 space indentation
Sep 11, 2015
60c8832
tests and local dev env build up
Sep 11, 2015
c12782f
skeleton code for a better cluster client
Sep 11, 2015
ae51180
custom exceptions for cluster
Sep 11, 2015
614e8da
lib for managing connections of nodes
Sep 11, 2015
5886a73
unit test for cmds
Sep 11, 2015
6045278
most of list commands. Cross keys operation will be implemented later
Sep 14, 2015
a40d69d
new exception: CrossSlotsError
Sep 15, 2015
1e250dc
str commands
Sep 15, 2015
5da60aa
test for str cmds
Sep 15, 2015
4ee6d00
refactor and improvement over list commands
Sep 15, 2015
f28479c
bug fix in bitop, it is a special command that the target key starts …
Sep 15, 2015
a7c962b
add test for bitop
Sep 15, 2015
c3bac21
remove useless code
Sep 15, 2015
08e0964
sets commands
Sep 15, 2015
1e0d57d
base test case. keep DRY
Sep 15, 2015
dc1a920
on_slot -> in_slot, better variable name
Sep 15, 2015
272e34c
use TestBase class in all test cases, remove usless utils.rb
Sep 15, 2015
571618e
sorted set commands
Sep 15, 2015
6d6e414
unit test for sorted set cmds
Sep 15, 2015
45334cf
hash commands
Sep 16, 2015
636e9ea
unit test for hash commands
Sep 16, 2015
0e108ae
keys commands
Sep 16, 2015
41f3280
unit test for keys commands
Sep 16, 2015
c03fbbe
raise NotImplementedError for not implemented commands
Sep 16, 2015
d044791
monitor command should raise NotImplementedError
Sep 16, 2015
9f8fdbf
fix unstable test
Sep 16, 2015
5e02f54
move local env built-up related to dev folder
Sep 16, 2015
acf98a4
fix local dev env
Sep 16, 2015
5632d6d
always use host instead of ip
Sep 16, 2015
d4367ce
put gems container into docker-compose
Sep 16, 2015
ca0ad20
do not access to the class attr directly, use a method to update it
Sep 17, 2015
d042b04
customised inpspect method, friendly debugging
Sep 17, 2015
86407e9
remove unused code
Sep 17, 2015
642b08e
clean startup nodes, ensure uniq works properly
Sep 17, 2015
05ea252
remove unused import
Sep 17, 2015
0bebeb6
add Rakefile for running all tests
Sep 17, 2015
30a70ac
ignore gem metadata
Sep 17, 2015
16f2d4f
install gemspec locally
Sep 17, 2015
e4f0fbc
cluster.rb -> rediscluster.rb
Sep 17, 2015
9561bec
remove unused import
Sep 17, 2015
e66d70d
update readme for building, installing
Sep 17, 2015
836f242
only support hostname atm
Sep 17, 2015
0af6ceb
make the initialise func readable a bit
Sep 17, 2015
c67c631
clean the code
Sep 17, 2015
9cef57e
switch to require_relative
Sep 17, 2015
9a5d473
make the init_slot_cache a bit faster
Sep 17, 2015
2f5aa66
moveout flushdb test from list command tests
Sep 18, 2015
0cd5963
unit tests for server commands
Sep 18, 2015
0612073
add config command log down the configuration change to each node
Sep 18, 2015
126f924
better info msg
Sep 18, 2015
d9ef68c
no log in keys command
Sep 18, 2015
144f8ca
fix unstable test
Sep 18, 2015
2d5ac82
more server cmds
Sep 18, 2015
122a232
improve the way of passing argments into execute_cmds_on_all_nodes me…
Sep 18, 2015
1efb077
connection commands
Sep 18, 2015
ddcea96
connection cmds unit test
Sep 18, 2015
025539f
enable reade from slave
Sep 24, 2015
a2c229e
remove unused code
Oct 1, 2015
a0c17a1
make inspect friendlier
Oct 1, 2015
e56a064
keep the options in redis-rb
Oct 1, 2015
8641e75
add comments to some key methods
Oct 2, 2015
41967b6
add mapped_hmget, mapped_hmset
Oct 13, 2015
456e980
slaves needs to be covered too
Oct 15, 2015
27c251f
clean the uesless comments
Oct 19, 2015
b240585
add mget mset series commands
Oct 19, 2015
8e3147a
clean the local dev setup
Oct 19, 2015
c73646d
clean local dev and remove unused files
Oct 21, 2015
c8b3fcc
switch back to 4 indentation spaces
Oct 30, 2015
9d3308c
add missing deps for the client
Oct 30, 2015
37448b7
add SSH to local dev container
iandyh May 19, 2016
f02d7bb
customise retry interval and retry when there is a temporary cluster …
iandyh May 19, 2016
44abe03
should be put outside of loop
iandyh May 19, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
enable reade from slave
  • Loading branch information
iandyh committed Oct 29, 2015
commit 025539f533ef3d30a09146474e2a144aeca05de1
74 changes: 54 additions & 20 deletions lib/connection_table.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,33 @@ def split_node(node)
[host, port]
end

def initialize(max_connections)
def initialize(max_connections, read_slave: false, timeout: nil)
@max_connections = max_connections
@table = {}
@read_slave = read_slave
@timeout = timeout
@master_conns = {}
@slave_conns = {}
@slots = {}
@pid = get_pid
end

def inspect
"#<#{self.class.name}: @table=#{@table}, @max_connections=#{@max_connections}>"
"#<#{self.class.name}: @max_connections=#{@max_connections}>"
end

def new_pool(node)
def new_pool(node, read_only: false)
host, port = split_node(node)
ConnectionPool.new(size: @max_connections) { Redis.new(:host => host, :port => port)}
ConnectionPool.new(size: @max_connections) {
r = Redis.new(:host => host, :port => port, :timeout => @timeout)
if read_only
r.readonly
end
r
}
end

def get_pool_by_node(node)
if [email protected]_key?(node)
pool = new_pool(node)
@table[node] = pool
return pool
end
@table[node]
@master_conns.fetch(node, nil) or @slave_conns.fetch(node)
end

def get_connection_by_node(node)
Expand All @@ -40,29 +45,58 @@ def get_connection_by_node(node)
end
end

def get_random_connection
random_node = @table.keys.sample
def get_random_connection(master_only)
keys = master_only ? @master_conns.keys : @master_conns.keys + @slave_conns.keys
random_node = keys.sample
get_connection_by_node(random_node)
end

def get_connection_by_slot(slot)
node = @slots[slot]
def get_connection_by_slot(slot, master_only)
nodes = @slots[slot]
node = @read_slave && !master_only ? nodes.sample: nodes[0]
get_connection_by_node(node)
end

def flush_slots_cache
@slots = {}
end

def update_slot!(newslot, node_name)
@slots[newslot] = node_name
def init_node_pool(nodes)
nodes.each do |n|
name = n[:name]
if @master_conns.has_key?(name) || @slave_conns.has_key?(name)
next
end
if n[:role] == 'master'
reset_master_node!(name)
next
end
reset_slave_node!(name)
end
end

def reset(node)
@table[node] = new_pool(node)
def update_slot!(newslot, node_names)
@slots[newslot] = node_names
end

def check_pid
def reset_master_node!(node)
@master_conns[node] = new_pool(node)
end

def reset_slave_node!(node)
@slave_conns[node] = new_pool(node, read_only: true)
end

def get_pid
Process.pid
end

def make_fork_safe(nodes)
if @pid != get_pid
@master_conns = {}
@slave_conns = {}
init_node_pool(nodes)
end
end

end
Loading