Redis Cluster集群架构实现

Redis集群简介

通过前面三篇博客的介绍《Redis基础认识及常用命令使用(一)–技术流ken》，《Redis基础知识补充及持久化、备份介绍(二)–技术流ken》，《Redis主从复制、多实例、高可用(三)–技术流ken》，现在已经对redis的基础知识，常用命令，持久化，备份，主从复制，多实例的安装以及redis的高可用熟练掌握了。本篇博客将介绍redis cluster集群，也是一个比较复杂的内容，本篇博客将采用较为简洁的方式来呈现redis集群。

有关redis集群的介绍可以参考下面我摘自redis官网的简介。

Redis 集群是一个可以在多个 Redis 节点之间进行数据共享的设施（installation）。

Redis 集群不支持那些需要同时处理多个键的 Redis 命令，因为执行这些命令需要在多个 Redis 节点之间移动数据，并且在高负载的情况下，这些命令将降低 Redis 集群的性能，并导致不可预测的行为。

Redis 集群通过分区（partition）来提供一定程度的可用性（availability）：即使集群中有一部分节点失效或者无法进行通讯，集群也可以继续处理命令请求。

Redis 集群提供了以下两个好处：

将数据自动切分（split）到多个节点的能力。
当集群中的一部分节点失效或者无法进行通讯时，仍然可以继续处理命令请求的能力。 –摘自redis官网

总而言之，redis集群实现了数据的共享以及去中心化。

Redis集群实现

实现redis集群，现在采用一台服务器安装6个redis实例，有关如果安装多实例的详细讲解请参考我上篇博客《Redis主从复制、多实例、高可用(三)–技术流ken》

6个多实例，三个为主节点，三个为从节点

环境：

CentOS Linux release 7.5.1804 (Core)

redis-4.0.11

创建多实例

第一步：创建多实例数据目录

root@ken ~]# mkdir /ken

[root@ken ~]# cd /ken

[root@ken ken]# mkdir 6379 6380 6381 6382 6383 6384

[root@ken ken]# ls

6379  6380  6381  6382  6383  6384

第二步：上传redis安装包

这里使用的是redis-4.0.11的版本，可以在redis官方网站进行下载https://redis.io/

[root@ken ~]# rz

第三步：解压安装包

[root@ken ~]# tar xf redis-4.0.11.tar.gz

第四步：复制解压安装包的配置文件到6379目录下

[root@ken ~]# cp redis-4.0.11/redis.conf /ken/6379/

第五步：移动redis解压包至/usr/local/redis下

[root@ken ~]# mv redis-4.0.11 /usr/local/redis

第六步：编译安装

[root@ken ~]# cd /usr/local/redis
[root@ken redis]# make && make install

[root@ken ~]# ln /usr/local/redis/src/ /bin -s

第七步：修改配置文件

[root@ken ~]# grep -E -v "^#|^$" /ken/6379/redis.conf

bind 10.220.5.137       #绑定本机ip地址

protected-mode yes

port 6379               #监听端口

tcp-backlog 511

timeout 0

tcp-keepalive 300

daemonize yes           #后台运行

supervised no

pidfile /ken/6379/redis_6379.pid  #pid文件保存位置

loglevel notice

logfile ""

databases 16

always-show-logo yes

save 900 1

save 300 10

save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename dump.rdb

dir /ken/6379/            #rdb文件保存位置

slave-serve-stale-data yes

slave-read-only yes

repl-diskless-sync no

repl-diskless-sync-delay 5

repl-disable-tcp-nodelay no

slave-priority 100

lazyfree-lazy-eviction no

lazyfree-lazy-expire no

lazyfree-lazy-server-del no

slave-lazy-flush no

appendonly no

appendfilename "appendonly.aof"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

aof-use-rdb-preamble no

lua-time-limit 5000

cluster-enabled yes    #开启集群

cluster-config-file nodes-6379.conf  #集群文件名称

cluster-node-timeout 15000           #集群超时时间

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-ziplist-entries 512

hash-max-ziplist-value 64

list-max-ziplist-size -2

list-compress-depth 0

set-max-intset-entries 512

zset-max-ziplist-entries 128

zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

activerehashing yes

client-output-buffer-limit normal 0 0 0

client-output-buffer-limit slave 256mb 64mb 60

client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

aof-rewrite-incremental-fsync yes

第八步：把上面这个修改过的文件复制到其他的实例数据目录之下

[root@ken ~]# cp /ken/6379/redis.conf /ken/6380/

[root@ken ~]# cp /ken/6379/redis.conf /ken/6381/

[root@ken ~]# cp /ken/6379/redis.conf /ken/6382/

[root@ken ~]# cp /ken/6379/redis.conf /ken/6383/

[root@ken ~]# cp /ken/6379/redis.conf /ken/6384/

第九步:修改配置文件

只要使用sed即可进行修改

[root@ken ~]# sed -i 's/6379/6380/g' /ken/6380/redis.conf

[root@ken ~]# sed -i 's/6379/6381/g' /ken/6381/redis.conf

[root@ken ~]# sed -i 's/6379/6382/g' /ken/6382/redis.conf

[root@ken ~]# sed -i 's/6379/6383/g' /ken/6383/redis.conf

[root@ken ~]# sed -i 's/6379/6384/g' /ken/6384/redis.conf

第十步：启动各个实例

[root@ken ~]# redis-server /ken/6380/redis.conf

[root@ken ~]# redis-server /ken/6381/redis.conf

[root@ken ~]# redis-server /ken/6382/redis.conf

[root@ken ~]# redis-server /ken/6383/redis.conf

[root@ken ~]# redis-server /ken/6384/redis.conf

[root@ken ~]# ss -tnl

State      Recv-Q Send-Q                   Local Address:Port                                  Peer Address:Port

LISTEN     0      128                       10.220.5.137:16380                                            *:*

LISTEN     0      128                       10.220.5.137:16381                                            *:*

LISTEN     0      128                       10.220.5.137:16382                                            *:*

LISTEN     0      128                       10.220.5.137:16383                                            *:*

LISTEN     0      128                       10.220.5.137:16384                                            *:*

LISTEN     0      128                                  *:10050                                            *:*

LISTEN     0      128                                  *:10051                                            *:*

LISTEN     0      128                       10.220.5.137:6379                                             *:*

LISTEN     0      128                       10.220.5.137:6380                                             *:*

LISTEN     0      128                       10.220.5.137:6381                                             *:*

LISTEN     0      128                       10.220.5.137:6382                                             *:*

LISTEN     0      128                       10.220.5.137:6383                                             *:*

LISTEN     0      128                                  *:111                                              *:*

LISTEN     0      128                       10.220.5.137:6384                                             *:*

LISTEN     0      128                                  *:22                                               *:*

LISTEN     0      128                       10.220.5.137:16379                                            *:*

LISTEN     0      128                                 :::10050                                           :::*

LISTEN     0      128                                 :::10051                                           :::*

LISTEN     0      70                                  :::3306                                            :::*

LISTEN     0      128                                 :::111                                             :::*

LISTEN     0      128                                 :::80                                              :::*

LISTEN     0      128                                 :::22                                              :::*

安装ruby2.3

实现redis cluster功能，依赖redis-trib.rb，而这个工具是依赖一个ruby开发工具包的，所以需要安装ruby环境，并安装依赖包

第一步:上传解压安装包

[root@ken ~]# rz

[root@ken ~]# tar xf ruby-2.3.5.tar.gz

第二步：编译安装

建议虚拟机内存至少1个G以上

[root@ken ~]# cd ruby-2.3.5/

[root@ken ruby-2.3.5]# ./configure --prefix=/usr/local/ruby && make && make install

第三步：安装redis-trib.rb的依赖

[root@ken ~]# ln -s /usr/local/ruby/bin/gem /bin

[root@ken ~]# gem install -l redis-3.3.0.gem

Successfully installed redis-3.3.0

Parsing documentation for redis-3.3.0

Installing ri documentation for redis-3.3.0

Done installing documentation for redis after 0 seconds

1 gem installed
[root@ken ~]# ln -s /usr/local/ruby/bin/ruby  /bin

获取集群帮助

只需要输入redis-trib.rb回车即可

[root@ken ~]# redis-trib.rb

Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN

                  --replicas <arg>

  check           host:port

  info            host:port

  fix             host:port

                  --timeout <arg>

  reshard         host:port

                  --from <arg>

                  --to <arg>

                  --slots <arg>

                  --yes

                  --timeout <arg>

                  --pipeline <arg>

  rebalance       host:port

                  --weight <arg>

                  --auto-weights

                  --use-empty-masters

                  --timeout <arg>

                  --simulate

                  --pipeline <arg>

                  --threshold <arg>

  add-node        new_host:new_port existing_host:existing_port

                  --slave

                  --master-id <arg>

  del-node        host:port node_id

  set-timeout     host:port milliseconds

  call            host:port command arg arg .. arg

  import          host:port

                  --from <arg>

                  --copy

                  --replace

  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

创建redis集群

命令的意义如下：

给定 redis-trib.rb 程序的命令是 create ，这表示我们希望创建一个新的集群。
选项 --replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。
之后跟着的其他参数则是实例的地址列表，我们希望程序使用这些地址所指示的实例来创建新集群。

简单来说，以上命令的意思就是让 redis-trib 程序创建一个包含三个主节点和三个从节点的集群。

接着， redis-trib 会打印出一份预想中的配置给你看，如果你觉得没问题的话，就可以输入 yes ， redis-trib 就会将这份配置应用到集群当中：

[root@ken ~]# redis-trib.rb create --replicas 1 10.220.5.137:6379 10.220.5.137:6380 10.220.5.137:6381 10.220.5.137:6382 10.220.5.137:6383 10.220.5.137:6384

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

10.220.5.137:6379

10.220.5.137:6380

10.220.5.137:6381

Adding replica 10.220.5.137:6383 to 10.220.5.137:6379

Adding replica 10.220.5.137:6384 to 10.220.5.137:6380

Adding replica 10.220.5.137:6382 to 10.220.5.137:6381

>>> Trying to optimize slaves allocation for anti-affinity

[WARNING] Some slaves are in the same host as their master

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-10922 (5462 slots) master

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:10923-16383 (5461 slots) master

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

Can I set the above configuration? (type 'yes' to accept): yes   #输入yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join.....

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

Redis集群测试

测试 Redis 集群比较简单的办法就是使用 redis-rb-cluster 或者 redis-cli ，接下来将使用 redis-cli 为例来进行演示：

第一步：登录集群中

一定要加个-c，后面输入哪个端口都可以，这就是redis的区中心化思想

[root@ken ~]# redis-cli -c -h 10.220.5.137 -p 6379

第二步：创建key

可以发现创建的key被分配到了不同的节点

10.220.5.137:6379> keys *

(empty list or set)

10.220.5.137:6379> set name ken

-> Redirected to slot [5798] located at 10.220.5.137:6380

OK

10.220.5.137:6380> set addr jiangsu

-> Redirected to slot [12790] located at 10.220.5.137:6381

OK

10.220.5.137:6381> set tel 123445

-> Redirected to slot [7485] located at 10.220.5.137:6380

OK

10.220.5.137:6380> set ege 25

OK

10.220.5.137:6380> set gender male

-> Redirected to slot [15355] located at 10.220.5.137:6381

OK

10.220.5.137:6381> keys *

1) "addr"

2) "gender"

第三步：获取key

在端口6381的节点之上没有tel这个key,但是我们仍然可以使用get tel获取到值，这就实现了redis集群的数据共享

10.220.5.137:6381> keys *

1) "addr"

2) "gender"

10.220.5.137:6381> get tel

-> Redirected to slot [7485] located at 10.220.5.137:6380

"123445"

在集群中添加新的节点

第一步：创建一个新的实例

[root@ken ~]# mkdir /ken/6385

[root@ken ~]# cp /ken/6379/redis.conf /ken/6385/

[root@ken ~]# sed -i 's/6379/6385/g' /ken/6385/redis.conf

[root@ken ~]# redis-server /ken/6385/redis.conf

10461:C 15 Nov 21:21:11.646 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

10461:C 15 Nov 21:21:11.646 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=10461, just started

10461:C 15 Nov 21:21:11.646 # Configuration loaded

第二步：添加该节点到集群中

命令中的 add-node 表示我们要让 redis-trib 将一个节点添加到集群里面， add-node 之后跟着的是新节点的 IP 地址和端口号，再之后跟着的是集群中任意一个已存在节点的 IP 地址和端口号，这里我们使用的是 10.220.5.137:6379 。

[root@ken ~]# redis-trib.rb add-node 10.220.5.137:6385 10.220.5.137:6379

>>> Adding node 10.220.5.137:6385 to cluster 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

>>> Send CLUSTER MEET to node 10.220.5.137:6385 to make it join the cluster.

[OK] New node added correctly.

第三步:查看

通过 cluster nodes 命令，我们可以确认新节点10.220.5.137:6385 已经被添加到集群里面了

10.220.5.137:6379> CLUSTER nodes

57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379@16379 myself,master - 0 1542288244000 1 connected 0-5460

daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385@16385 master - 0 1542288242824 0 connected

ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381@16381 master - 0 1542288244000 3 connected 10923-16383

3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380@16380 master - 0 1542288242000 2 connected 5461-10922

640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383@16383 slave 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 0 1542288242000 5 connected

b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384@16384 slave ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 0 1542288244852 6 connected

1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382@16382 slave 57753754f4c89054ab14e8ec517604c4fc4c8ed5 0 1542288243844 4 connected

第四步：使添加的新节点为从节点

如果我们打算让新节点成为 10.220.5.137:6379 的从节点，那么我们只要用客户端连接上新节点，然后执行以下命令就可以了

[root@ken ~]# redis-cli -c -h 10.220.5.137 -p 6385

10.220.5.137:6385> CLUSTER REPLICATE 57753754f4c89054ab14e8ec517604c4fc4c8ed5

OK

其中57753754f4c89054ab14e8ec517604c4fc4c8ed5是主节点10.220.5.137:6379的id

对redis中的数据做重新分片

[root@ken ~]# redis-trib.rb reshard 10.220.5.137:6379

你只需要指定集群中其中一个节点的地址， redis-trib 就会自动找到集群中的其他节点。

How many slots do you want to move (from 1 to 16384)? 1000

我们将打算移动的槽数量设置为 1000 个。

What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

定目标需要使用节点的 ID ，而不是 IP 地址和端口。比如说，我们打算使用集群的第一个主节点来作为目标，它的 IP 地址和端口是 10.220.5.137:6380 ，而节点 ID 则是3a9aa9592afc594c7e4206cc82ffb37d46a5b23d ，那么我们应该向 redis-trib 提供节点的 ID

Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

接着， redis-trib 会向你询问重新分片的源节点（source node），也即是，要从哪个节点中取出 1000 个哈希槽，并将这些槽移动到目标节点上面。

Source node #2:done

Do you want to proceed with the proposed reshard plan (yes/no)? yes

输入 yes 并使用按下回车之后， redis-trib 就会正式开始执行重新分片操作，将指定的哈希槽从源节点一个个地移动到目标节点上面

Moving slot 12905 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12906 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12907 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12908 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12909 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12910 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12911 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12912 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12913 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12914 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12915 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12916 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12917 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12918 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12919 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12920 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12921 from 10.220.5.137:6381 to 10.220.5.137:6380:

Moving slot 12922 from 10.220.5.137:6381 to 10.220.5.137:6380:

在重新分片操作执行完毕之后，可以使用以下命令来检查集群是否正常

[root@ken ~]# redis-trib.rb check 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   2 additional replica(s)

S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:12923-16383 (3461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-12922 (7462 slots) master

   1 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

删除节点

如果节点中有slot，那么需要先将slot 执行reshard给其他节点，然后才能执行删除操作

1. 删除空slot

删除10.220.5.137:6385

[root@ken ~]# redis-trib.rb check 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   2 additional replica(s)

S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:12923-16383 (3461 slots) master

   1 additional replica(s)

...

执行如下命令

[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d

>>> Removing node daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d from cluster 10.220.5.137:6379

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

查看发现6385节点已经被移除

[root@ken ~]# redis-trib.rb check 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:12923-16383 (3461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-12922 (7462 slots) master

   1 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

2.删除带有slot的节点
提示有数据报错

[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

>>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379

[ERR] Node 10.220.5.137:6381 is not empty! Reshard data away and try again.

需要重新分片

[root@ken ~]# redis-trib.rb reshard 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381

   slots:12923-16383 (3461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-12922 (7462 slots) master

   1 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

How many slots do you want to move (from 1 to 16384)? 3461

What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

Please enter all the source node IDs.

  Type 'all' to use all the nodes as source nodes for the hash slots.

  Type 'done' once you entered all the source nodes IDs.

Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

Source node #2:done

再检查slot已经为空

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381  #已经为空

   slots: (0 slots) master

   0 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-16383 (10923 slots) master

   2 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

再执行删除操作

[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8

>>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

再次查看。6381节点已经被删除

[root@ken ~]# redis-trib.rb check 10.220.5.137:6379

>>> Performing Cluster Check (using node 10.220.5.137:6379)

M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380

   slots:5461-16383 (10923 slots) master

   2 additional replica(s)

S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384

   slots: (0 slots) slave

   replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d

S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382

   slots: (0 slots) slave

   replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.