Skip to content

Commit

Permalink
Update docs on the deprecated overmoon
Browse files Browse the repository at this point in the history
  • Loading branch information
@doyoubi
doyoubi committed Dec 13, 2020
1 parent 3a07b3a commit 1ba7f72
Show file tree
Hide file tree
Showing 6 changed files with 195 additions and 166 deletions.
179 changes: 18 additions & 161 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ please refer to [Redis Cluster Protocol](./docs/redis_cluster_protocol.md).
##### Metadata Storage
Metadata storage stores all the metadata of the whole `undermoon` cluster,
including existing Redis instances, proxies, and exposed Redis clusters.
Now it's an in-memory storage server called `Memory Broker`
and there's [another implementation backed by Etcd](https://github.com/doyoubi/overmoon) in the future.
Now it's an in-memory storage server called `Memory Broker`.
When using [undermoon-operator](https://github.com/doyoubi/undermoon-operator),
this `Memory Broker` will change to use `ConfigMap` to store the data.

##### Coordinator
Coordinator will synchronize the metadata between broker and server proxy.
Expand All @@ -46,171 +47,27 @@ The design of chunk makes it very easy to build a cluster with a good topology f
Using [undermoon-operator](https://github.com/doyoubi/undermoon-operator)
is the easiest way to create Redis clusters if you have Kubernetes.

See the `README.md` of [undermoon-operator](https://github.com/doyoubi/undermoon-operator)
for more how to use it.

### Run Undermoon using Docker Compose
The following examples use docker to create an `undermoon` cluster.

Or you can set them up without docker following this docs: [setting up undermoon manually](docs/set_up_manually.md).

Requirements:

- docker-compose
- redis-cli
- [jq](https://stedolan.github.io/jq/)

#### Run the cluster in docker-compose
Download and run the cluster directly:
```bash
$ make docker-mem-broker-example
```

Or build it yourself and run the `undermoon` docker image:
```bash
$ make docker-build-image
$ make docker-mem-broker
```

#### Register Proxies
After everything is up, run the initialize script to register the storage resources through HTTP API:
```bash
$ ./examples/mem-broker/init.sh
```

We have 6 available proxies.
```bash
$ curl http://localhost:7799/api/v2/proxies/addresses
```

#### Create Cluster
Since every proxy has 2 corresponding Redis nodes, we have 12 nodes in total.
Note that the number of a cluster could only be the multiples of 4.
Let's create a cluster with 4 nodes.
```bash
$ curl -XPOST -H 'Content-Type: application/json' \
http://localhost:7799/api/v2/clusters/meta/mycluster -d '{"node_number": 4}'
```

Before connecting to the cluster, you need to add these hosts to you `/etc/hosts`:
```
# /etc/hosts
127.0.0.1 server_proxy1
127.0.0.1 server_proxy2
127.0.0.1 server_proxy3
127.0.0.1 server_proxy4
127.0.0.1 server_proxy5
127.0.0.1 server_proxy6
```

Let's checkout our cluster. It's created by some randomly chosen proxies.
We need to find them out first.
Note that you need to install the `jq` command to parse json easily for the command below.
helm install my-undermoon-operator undermoon-operator-<x.x.x>.tgz
```bash
# List the proxies of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy5:6005"
"server_proxy6:6006"
helm install \
--set 'cluster.clusterName=my-cluster-name' \
--set 'cluster.chunkNumber=2' \
--set 'cluster.maxMemory=2048' \
--set 'cluster.port=5299' \
my-cluster \
-n my-namespace \
undermoon-cluster-<x.x.x>.tgz
```

Pickup one of the proxy address above (in my case it's `server_proxy5:6005`) for the cluster `mycluster` and connect to it.

```bash
# Add `-c` to enable cluster mode:
$ redis-cli -h server_proxy5 -p 6005 -c
# List the proxies:
server_proxy5:6005> cluster nodes
mycluster___________d71bc00fbdddf89_____ server_proxy5:6005 myself,master - 0 0 7 connected 0-8191
mycluster___________8de73f9146386295____ server_proxy6:6006 master - 0 0 7 connected 8192-16383
# Send out some requests:
server_proxy5:6005> get a
-> Redirected to slot [15495] located at server_proxy6:6006
(nil)
server_proxy6:6006> get b
-> Redirected to slot [3300] located at server_proxy5:6005
(nil)
```
Great! We can use our created cluster just like the official Redis Cluster.

#### Scale Up
It actually has 4 Redis nodes under the hood.
```bash
# List the nodes of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].address'
"redis9:6379"
"redis10:6379"
"redis11:6379"
"redis12:6379"
```
Two of them are masters and the other two of them are replicas.

Let's scale up to 8 nodes:
```bash
# Add 4 nodes
$ curl -XPATCH -H 'Content-Type: application/json' \
http://localhost:7799/api/v2/clusters/nodes/mycluster -d '{"node_number": 4}'
# Start migrating the data
$ curl -XPOST http://localhost:7799/api/v2/clusters/migrations/expand/mycluster
```

Now we have 4 server proxies:
```bash
$ redis-cli -h server_proxy5 -p 6005 -c
server_proxy5:6005> cluster nodes
mycluster___________d71bc00fbdddf89_____ server_proxy5:6005 myself,master - 0 0 12 connected 0-4095
mycluster___________8de73f9146386295____ server_proxy6:6006 master - 0 0 12 connected 8192-12287
mycluster___________be40fe317baf2cf7____ server_proxy2:6002 master - 0 0 12 connected 4096-8191
mycluster___________9434df4158f3c5a4____ server_proxy4:6004 master - 0 0 12 connected 12288-16383
```

and 8 nodes:
```bash
# List the nodes of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].address'
"redis9:6379"
"redis10:6379"
"redis11:6379"
"redis12:6379"
"redis3:6379"
"redis4:6379"
"redis7:6379"
"redis8:6379"
```

#### Failover
If you shutdown any proxy, the replica will be promoted to master.
And as long as the whole `undermoon` cluster has remaining free proxies,
it can automatically replace the failed proxy,

```bash
# List the proxies of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy5:6005"
"server_proxy6:6006"
"server_proxy2:6002"
"server_proxy4:6004"
```

Let's shutdown one of the proxies like `server_proxy5:6005` here.
```bash
$ docker ps | grep server_proxy5 | awk '{print $1}' | xargs docker kill
```
See the `README.md` of [undermoon-operator](https://github.com/doyoubi/undermoon-operator)
for how to use it.

The `undermoon` will detect the failure, replace the failed proxy, promote the new master, and add new replica to the new master.
```bash
# server_proxy5 is replaced by server_proxy3
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy3:6003"
"server_proxy6:6006"
"server_proxy2:6002"
"server_proxy4:6004"
```
### Run Undermoon Using Docker Compose
See [docker compose example](./docs/docker_compose_example.md).

And we can remove the server_proxy3 from the `undermoon` cluster now.
```bash
$ curl -XDELETE http://localhost:7799/api/v2/proxies/meta/server_proxy3:6003
```
### Setup Undermoon Manually
Or you can set them up without docker following this docs: [setting up undermoon manually](docs/set_up_manually.md).

## Development
`undermoon` tries to avoid `unsafe` and some calls that could crash.
Expand Down
1 change: 1 addition & 0 deletions conf/server-proxy.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ backend_high_flush_interval = 600000
# When active_redirection is enabled,
# all the server proxies will handle the redirection inside.
# Clients don't need to be a Redis Cluster Client.
# NOTICE: This is an experimental feature and don't use it in production.
active_redirection = false
# This is only useful when active_redirection is true.
# Use 0 to disable limitation.
Expand Down
160 changes: 160 additions & 0 deletions docs/docker_compose_example.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Run Undermoon using Docker Compose
The following examples use docker to create an `undermoon` cluster.

Requirements:

- docker-compose
- redis-cli
- [jq](https://stedolan.github.io/jq/)

#### Run the cluster in docker-compose
Download and run the cluster directly:
```bash
$ make docker-mem-broker-example
```

Or build it yourself and run the `undermoon` docker image:
```bash
$ make docker-build-image
$ make docker-mem-broker
```

#### Register Proxies
After everything is up, run the initialize script to register the storage resources through HTTP API:
```bash
$ ./examples/mem-broker/init.sh
```

We have 6 available proxies.
```bash
$ curl http://localhost:7799/api/v2/proxies/addresses
```

#### Create Cluster
Since every proxy has 2 corresponding Redis nodes, we have 12 nodes in total.
Note that the number of a cluster could only be the multiples of 4.
Let's create a cluster with 4 nodes.
```bash
$ curl -XPOST -H 'Content-Type: application/json' \
http://localhost:7799/api/v2/clusters/meta/mycluster -d '{"node_number": 4}'
```

Before connecting to the cluster, you need to add these hosts to you `/etc/hosts`:
```
# /etc/hosts
127.0.0.1 server_proxy1
127.0.0.1 server_proxy2
127.0.0.1 server_proxy3
127.0.0.1 server_proxy4
127.0.0.1 server_proxy5
127.0.0.1 server_proxy6
```

Let's checkout our cluster. It's created by some randomly chosen proxies.
We need to find them out first.
Note that you need to install the `jq` command to parse json easily for the command below.

```bash
# List the proxies of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy5:6005"
"server_proxy6:6006"
```

Pickup one of the proxy address above (in my case it's `server_proxy5:6005`) for the cluster `mycluster` and connect to it.

```bash
# Add `-c` to enable cluster mode:
$ redis-cli -h server_proxy5 -p 6005 -c
# List the proxies:
server_proxy5:6005> cluster nodes
mycluster___________d71bc00fbdddf89_____ server_proxy5:6005 myself,master - 0 0 7 connected 0-8191
mycluster___________8de73f9146386295____ server_proxy6:6006 master - 0 0 7 connected 8192-16383
# Send out some requests:
server_proxy5:6005> get a
-> Redirected to slot [15495] located at server_proxy6:6006
(nil)
server_proxy6:6006> get b
-> Redirected to slot [3300] located at server_proxy5:6005
(nil)
```
Great! We can use our created cluster just like the official Redis Cluster.

#### Scale Up
It actually has 4 Redis nodes under the hood.
```bash
# List the nodes of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].address'
"redis9:6379"
"redis10:6379"
"redis11:6379"
"redis12:6379"
```
Two of them are masters and the other two of them are replicas.

Let's scale up to 8 nodes:
```bash
# Add 4 nodes
$ curl -XPATCH -H 'Content-Type: application/json' \
http://localhost:7799/api/v2/clusters/nodes/mycluster -d '{"node_number": 4}'
# Start migrating the data
$ curl -XPOST http://localhost:7799/api/v2/clusters/migrations/expand/mycluster
```

Now we have 4 server proxies:
```bash
$ redis-cli -h server_proxy5 -p 6005 -c
server_proxy5:6005> cluster nodes
mycluster___________d71bc00fbdddf89_____ server_proxy5:6005 myself,master - 0 0 12 connected 0-4095
mycluster___________8de73f9146386295____ server_proxy6:6006 master - 0 0 12 connected 8192-12287
mycluster___________be40fe317baf2cf7____ server_proxy2:6002 master - 0 0 12 connected 4096-8191
mycluster___________9434df4158f3c5a4____ server_proxy4:6004 master - 0 0 12 connected 12288-16383
```

and 8 nodes:
```bash
# List the nodes of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].address'
"redis9:6379"
"redis10:6379"
"redis11:6379"
"redis12:6379"
"redis3:6379"
"redis4:6379"
"redis7:6379"
"redis8:6379"
```

#### Failover
If you shutdown any proxy, the replica will be promoted to master.
And as long as the whole `undermoon` cluster has remaining free proxies,
it can automatically replace the failed proxy,

```bash
# List the proxies of the our "mycluster`:
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy5:6005"
"server_proxy6:6006"
"server_proxy2:6002"
"server_proxy4:6004"
```

Let's shutdown one of the proxies like `server_proxy5:6005` here.
```bash
$ docker ps | grep server_proxy5 | awk '{print $1}' | xargs docker kill
```

The `undermoon` will detect the failure, replace the failed proxy, promote the new master, and add new replica to the new master.
```bash
# server_proxy5 is replaced by server_proxy3
$ curl -s http://localhost:7799/api/v2/clusters/meta/mycluster | jq '.cluster.nodes[].proxy_address' | uniq
"server_proxy3:6003"
"server_proxy6:6006"
"server_proxy2:6002"
"server_proxy4:6004"
```

And we can remove the server_proxy3 from the `undermoon` cluster now.
```bash
$ curl -XDELETE http://localhost:7799/api/v2/proxies/meta/server_proxy3:6003
```
4 changes: 0 additions & 4 deletions docs/mem_broker_replica.md
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
# Setting Up Backup for Memory Broker
Even though the on going [broker backed by Etcd](https://github.com/doyoubi/overmoon) will be more reliable,
sometimes users may choose simplicity over reliability.
`Memory Broker` is still a good way to go at this point.
`Memory Broker` provides a naive solution for backing up the data.

## Setting Up Replica for Memory Broker
Build the binaries:
Expand Down
Loading

0 comments on commit 1ba7f72

Please sign in to comment.