Redis5-集群搭建实验

集群规划：

nodeA：192.168.29.22(22-master,23-slave)

nodeB：192.168.29.23(23-master,24-slave)

nodeC：192.168.29.24(24-master,22-slave)

下载：

http://download.redis.io/releases/redis-5.0.7.tar.gz

安装

yum install -y gcc

tar zxvf redis-5.0..tar.gz

mv redis-5.0. /usr/local/redis-5.0.

mkdir /usr/local/redis5

cd /usr/local/redis-5.0./

make

make install PREFIX=/usr/local/redis5

mkdir /usr/local/redis5/conf

mkdir /usr/local/redis5/log

mkdir /usr/local/redis5/data
echo 'export PATH=$PATH:/usr/local/redis5/bin' >> ~/.bash_profile
source ~/.bash_profile

设置开机启动

vim /usr/lib/systemd/system/redis.service

[Unit]

Description=Redis Server

After=network.target

[Service]

Type=simple

# User=redis

# Group=redis

PIDFile=/var/run/22m_6379.pid

ExecStart=/usr/local/redis5/bin/redis-server /usr/local/redis5/conf/redis-22m.conf --daemonize no

# ExecStop=/usr/local/redis5/bin/redis-cli shutdown

ExecStop=/usr/local/redis5/bin/redis-cli -p  shutdown

Restart=always

[Install]

WantedBy=multi-user.target

vim /usr/lib/systemd/system/redisb.service

[Unit]

Description=Redis Server

After=network.target

[Service]

Type=simple

# User=redis

# Group=redis

PIDFile=/var/run/23s_7379.pid

ExecStart=/usr/local/redis5/bin/redis-server /usr/local/redis5/conf/redis-23s.conf --daemonize no

# ExecStop=/usr/local/redis5/bin/redis-cli shutdown

ExecStop=/usr/local/redis5/bin/redis-cli -p  shutdown

Restart=always

[Install]

WantedBy=multi-user.target

systemctl daemon-reload

systemctl start redis

systemctl enable redis

systemctl start redisb

systemctl enable redisb

配置

cp redis.conf /usr/local/redis5/conf/redis-22m.conf

#修改配置文件redis.conf，IP、端口、配置文件类推

192.168.29.22: redis-22m.conf

192.168.29.22: redis-23s.conf

192.168.29.23: redis-23m.conf

192.168.29.23: redis-24s.conf

192.168.29.24: redis-24m.conf

192.168.29.24: redis-22s.conf

vim redis-22m.conf

port    # 监听端口(从节点改成7379)

bind 0.0.0.0  # 监听 ip

dir /usr/local/redis5/data-22m   # 指定文件存放路径 （ .rdb .aof nodes-xxxx.conf 这样的文件都会在此路径下）

cluster-enabled yes   # 启动集群模式

cluster-config-file redis-22m-.conf     # 集群节点配置文件

daemonize yes       # 后台启动
pidfile /var/run/22m_6379.conf

cluster-node-timeout        # 集群节点超时时间

appendonly yes      # 指定持久化方式，开启 AOF 模式

protected-mode no   # 非保护模式

创建集群

redis-cli --cluster create 192.168.29.22: 192.168.29.23: 192.168.29.24: 192.168.29.23: 192.168.29.24: 192.168.29.22: --cluster-replicas 1
#--replicas 1 表示为集群中的每个主节点创建一个从节点

连接集群

redis-cli -c -h 192.168.29.22 -p

# -c参数为连接集群节点使用，此选项可防止moved和ask异常。

#连接任意节点均可进入集群

查看集群信息

[root@redis-A redis5]# redis-cli -c -h 192.168.29.23

192.168.29.23:> cluster info

cluster_state:ok

cluster_slots_assigned:

cluster_slots_ok:

...........

查看节点信息

192.168.29.23:> cluster nodes

#可展示主从关系
或
[root@redis-A redis5]# redis-cli --cluster check 192.168.29.23:6379

集群崩溃条件：当存活的 Master 节点小于总节点数的一半时

添加master节点

redis-cli --cluster add-node 192.168.30.3: 192.168.29.23:

#192.168..3为新增节点，192.168..23为集群中任意一正常节点，无论主从

新增的节点没有分配slot，需要reshard分配slot

redis-cli --cluster reshard 192.168.29.22:6379
>>> Performing Cluster Check (using node 192.168.29.22:6379)
M: 9dce6a67f400b958692079779420af9008d4424d 192.168.29.22:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
......
M: 1a1bb454fce513283392a321147c975713fe6f55 192.168.30.3:6379
   slots: (0 slots) master
......
M: 564c09d468dcb372d93ff86e44971aede4748d1f 192.168.29.23:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 2048　　#想要分配的slot数
What is the receiving node ID? 1a1bb454fce513283392a321147c975713fe6f55  #分配给哪个node，这里选新增的节点
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all #从哪些节点分配，这里填all
......
Moving slot 309 from 9dce6a67f400b958692079779420af9008d4424d
......
Do you want to proceed with the proposed reshard plan (yes/no)?yes  #选yes
......
Moving slot 11587 from 192.168.29.24:6379 to 192.168.30.3:6379:
......

添加 Slave 节点

redis-cli --cluster add-node 192.168.30.3: 192.168.29.22: --cluster-slave --cluster-master-id 1a1bb454fce513283392a321147c975713fe6f55

#192.168.30.3: 为新增节点；192.168.29.22:6379为集群中任意正常节点；--cluster-master-id指定master节点

如果不指定会随机分配给一个Mater节点

平衡各节点槽数量

新加 Master 节点后各节点的槽可能会分配不均匀，可以重新分配。

#查看slots信息
redis-cli --cluster info 192.168.30.3:

192.168.30.3: (1a1bb454...) ->  keys |  slots |  slaves.

192.168.29.22: (9dce6a67...) ->  keys |  slots |  slaves.

192.168.29.23: (564c09d4...) ->  keys |  slots |  slaves.

192.168.29.24: (ae5b2268...) ->  keys |  slots |  slaves.

[OK]  keys in  masters.

0.00 keys per slot on average.


#平衡slots

redis-cli --cluster rebalance --cluster-threshold  192.168.30.3:

>>> Performing Cluster Check (using node 192.168.30.3:)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All  slots covered.

>>> Rebalancing across  nodes. Total weight = 4.00

Moving  slots from 192.168.29.24: to 192.168.30.3:

......


#再次查看

redis-cli --cluster info 192.168.30.3:

192.168.30.3: (1a1bb454...) ->  keys |  slots |  slaves.

192.168.29.22: (9dce6a67...) ->  keys |  slots |  slaves.

192.168.29.23: (564c09d4...) ->  keys |  slots |  slaves.

192.168.29.24: (ae5b2268...) ->  keys |  slots |  slaves.

[OK]  keys in  masters.

0.00 keys per slot on average.

Slave 节点重新分配

redis-cli -c -h 192.168.30.3 -p

192.168.30.3:> cluster replicate 9dce6a67f400b958692079779420af9008d4424d

OK

#（ID为目标Mater节点）

删除节点

slave节点

redis-cli  --cluster del-node 192.168.30.3: e34bbd847a135012582a7b2b3259aec2fd7668bd

>>> Removing node e34bbd847a135012582a7b2b3259aec2fd7668bd from cluster 192.168.30.3:

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

#ID为slave节点192.168.30.:7379的ID

Master节点

如果 Master 节点有 Slave 节点，则首先要将 Slave 节点重新分配到其他 Master 节点或者删除。
如果 Master 节点有槽(slot)，则需要去掉分配的槽(slot)，然后再删除主节点

#查看slot信息

redis-cli --cluster info 192.168.30.3:

192.168.30.3: (1a1bb454...) ->  keys |  slots |  slaves.

192.168.29.22: (9dce6a67...) ->  keys |  slots |  slaves.

192.168.29.23: (564c09d4...) ->  keys |  slots |  slaves.

192.168.29.24: (ae5b2268...) ->  keys |  slots |  slaves.

[OK]  keys in  masters.

0.00 keys per slot on average.

#将192.168.30.:6379的slots转移到另一个Master节点

redis-cli --cluster reshard 192.168.30.3:

>>> Performing Cluster Check (using node 192.168.30.3:)

M: 1a1bb454fce513283392a321147c975713fe6f55 192.168.30.3:

   slots:[-],[-],[-] ( slots) master

M: 9dce6a67f400b958692079779420af9008d4424d 192.168.29.22:

   slots:[-] ( slots) master

    additional replica(s)

......

M: ae5b2268339d38874cd3da6487c33aace4d24d3a 192.168.29.24:

   slots:[-] ( slots) master

    additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All  slots covered.

How many slots do you want to move (from  to )?     #192.168.30.3:6379的slots的个数

What is the receiving node ID? 9dce6a67f400b958692079779420af9008d4424d        #准备接手slots的192.168.29.:6379的ID

Please enter all the source node IDs.

  Type 'all' to use all the nodes as source nodes for the hash slots.

  Type 'done' once you entered all the source nodes IDs.

Source node #: 1a1bb454fce513283392a321147c975713fe6f55    #192.168.30.3:6379的ID

Source node #: done    #没有第二个节点

......

Moving slot  from 1a1bb454fce513283392a321147c975713fe6f55

......

Do you want to proceed with the proposed reshard plan (yes/no)? yes

Moving slot  from 192.168.30.3: to 192.168.29.22::


#再次查看节点信息可见节点已没有slots

redis-cli -p  cluster nodes

9dce6a67f400b958692079779420af9008d4424d 192.168.29.22:@ master -    connected - -

564c09d468dcb372d93ff86e44971aede4748d1f 192.168.29.23:@ master -    connected -

de62d7694375f615f6d4cfe7af10a3367cac0569 192.168.29.22:@ slave ae5b2268339d38874cd3da6487c33aace4d24d3a    connected

2e71ff865f43170d91802f12199a896632e2473e 192.168.29.24:@ slave 564c09d468dcb372d93ff86e44971aede4748d1f    connected

ec028f2f4d44ebfad7add0fcbc26f328f2ec55dd 192.168.29.23:@ slave 9dce6a67f400b958692079779420af9008d4424d    connected

1a1bb454fce513283392a321147c975713fe6f55 192.168.30.3:@ myself,master -    connected

ae5b2268339d38874cd3da6487c33aace4d24d3a 192.168.29.24:@ master -    connected -

#删除Master节点，ID为要删除的节点ID

redis-cli  --cluster del-node 192.168.29.22: 1a1bb454fce513283392a321147c975713fe6f55

>>> Removing node 1a1bb454fce513283392a321147c975713fe6f55 from cluster 192.168.29.22:

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

集群升级

升级 Slave 节点

升级 Slave 节点比较简单，只需要停止 Slave 节点，然后升级 Redis 程序，最后重启即可。

升级 Master 节点

升级 Master 节点

1、使用 debug segfault 使 Master 节点发生故障转换为 Slave 节点。

redis-cli -h 192.168.29.22 -p  debug segfault

2、然后再照升级 Slave 节点的方法升级此节点。

参考:Redis 5.0.7 讲解，单机、集群模式搭建