RabbitMQ单机多实例配置

由于某些因素的限制，有时候你不得不在一台机器上去搭建一个rabbitmq集群，当然这种集群只适合自己玩玩，验证下结论，这个有点类似zookeeper的单机版。真实生成环境还是要配成多机集群的。有关怎么配置多机集群的可以参考其他的资料，这里主要论述如何在单机中配置多个rabbitmq实例。

主要参考官方文档：https://www.rabbitmq.com/clustering.html

前提

配置的前提是你的rabbitmq可以运行起来，比如”ps aux|grep rabbitmq”你能看到相关进程，又比如运行“rabbitmqctl status”你可以看到类似如下信息，而不报错：

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl status
Status of node 'rabbit@hiddenzhu-8drdc' ...
[{pid,13014},
 {running_applications,[{rabbit,"RabbitMQ","3.4.0"},
                        {mnesia,"MNESIA  CXC 138 12","4.14.1"},
                        {os_mon,"CPO  CXC 138 46","2.4.1"},
                        {xmerl,"XML parser","1.3.12"},
                        {sasl,"SASL  CXC 138 11","3.0.1"},
                        {stdlib,"ERTS  CXC 138 10","3.1"},
                        {kernel,"ERTS  CXC 138 10","5.1"}]},
 {os,{unix,linux}},
 {erlang_version,"Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:30] [hipe] [kernel-poll:true]\n"},
 {memory,[{total,37276536},
          {connection_readers,0},
          {connection_writers,0},
          {connection_channels,0},
          {connection_other,2832},
          {queue_procs,2832},
          {queue_slave_procs,0},
          {plugins,0},
          {other_proc,13331064},
          {mnesia,66248},
          {mgmt_db,0},
          {msg_index,41184},
          {other_ets,778528},
          {binary,23152},
          {code,14637029},
          {atom,654241},
          {other_system,7739426}]},
 {alarms,[]},
 {listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,3301929779},
 {disk_free_limit,50000000},
 {disk_free,37108654080},
 {file_descriptors,[{total_limit,924},
                    {total_used,3},
                    {sockets_limit,829},
                    {sockets_used,1}]},
 {processes,[{limit,1048576},{used,126}]},
 {run_queue,0},
 {uptime,2143}]

为了简单化，这里也要保证rabbitmq的plugin没有开启（因为开启之后要占用一些端口，多实例配置起来会更加复杂，这里简单问题简化说明）。

情景

假设有三个rabbitmq节点，分别为rabbit1, rabbit2和rabbit3

主要开启命令如下：

RABBITMQ_NODE_PORT=5672 RABBITMQ_NODENAME=rabbit1 rabbitmq-server -detached
RABBITMQ_NODE_PORT=5673 RABBITMQ_NODENAME=rabbit2 rabbitmq-server -detached
RABBITMQ_NODE_PORT=5674 RABBITMQ_NODENAME=rabbit3 rabbitmq-server -detached

结束命令如下：

rabbitmqctl -n rabbit1 stop
rabbitmqctl -n rabbit2 stop
rabbitmqctl -n rabbit3 stop

rabbit1

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit1 stop_app
Stopping node 'rabbit1@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit1 reset
Resetting node 'rabbit1@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit1 cluster
Clustering node 'rabbit1@hiddenzhu-8drdc' with [] ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit1 start_app
Starting node 'rabbit1@hiddenzhu-8drdc' ...
...done.

TIPS

有些版本（比如3.4.0）在第一个节点（主节点）运行“rabbitmqctl -n rabbit1 cluster” 时会遇到：

Error: could not recognise command

的错误，可以不运行“rabbitmqctl -n rabbit1 cluster”这句。然后在从节点运行：

rabbitmqctl -n rabbit2 join_cluster rabbit1@`hostname -s`

这一句。

主要原因是：有些版本不识别cluster这个命令

rabbit2

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 stop_app
Stopping node 'rabbit2@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 reset
Resetting node 'rabbit2@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 cluster rabbit1@`hostname -s`
Clustering node 'rabbit2@hiddenzhu-8drdc' with ['rabbit1@hiddenzhu-8drdc'] ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 start_app
Starting node 'rabbit2@hiddenzhu-8drdc' ...
...done.

TIPS

如果略去“rabbitmqctl -n rabbit2 reset”这一句，可能会报错：

Error: {ok,already_member}

查看cluster信息

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit1 cluster_status
Cluster status of node 'rabbit1@hiddenzhu-8drdc' ...
[{nodes,[{disc,['rabbit1@hiddenzhu-8drdc']},
         {ram,['rabbit2@hiddenzhu-8drdc']}]},
 {running_nodes,['rabbit2@hiddenzhu-8drdc','rabbit1@hiddenzhu-8drdc']}]
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 cluster_status
Cluster status of node 'rabbit2@hiddenzhu-8drdc' ...
[{nodes,[{disc,['rabbit1@hiddenzhu-8drdc']},
         {ram,['rabbit2@hiddenzhu-8drdc']}]},
 {running_nodes,['rabbit1@hiddenzhu-8drdc','rabbit2@hiddenzhu-8drdc']}]
...done.

加入节点rabbit3

[root@hiddenzhu-8drdc rabbitmq]# RABBITMQ_NODE_PORT=5674 RABBITMQ_NODENAME=rabbit3 rabbitmq-server -detached
Activating RabbitMQ plugins ...
0 plugins activated:

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit3 stop_app
Stopping node 'rabbit3@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit3 reset
Resetting node 'rabbit3@hiddenzhu-8drdc' ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit3 cluster rabbit1@`hostname -s`
Clustering node 'rabbit3@hiddenzhu-8drdc' with ['rabbit1@hiddenzhu-8drdc'] ...
...done.
[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit3 start_app
Starting node 'rabbit3@hiddenzhu-8drdc' ...
...done.

最终查看集群状态

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit3 cluster_status
Cluster status of node 'rabbit3@hiddenzhu-8drdc' ...
[{nodes,[{disc,['rabbit1@hiddenzhu-8drdc']},
         {ram,['rabbit3@hiddenzhu-8drdc','rabbit2@hiddenzhu-8drdc']}]},
 {running_nodes,['rabbit1@hiddenzhu-8drdc','rabbit2@hiddenzhu-8drdc',
                 'rabbit3@hiddenzhu-8drdc']}]
...done.

TIPS

有可能遇到这样的情况：

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl status -n rabbit2
Status of node 'rabbit2@hiddenzhu-8drdc' ...
Error: {badarith,[{rabbit_vm,bytes,1,[]},
                  {rabbit_vm,'-mnesia_memory/0-lc$^0/1-0-',1,[]},
                  {rabbit_vm,mnesia_memory,0,[]},
                  {rabbit_vm,memory,0,[]},
                  {rabbit,status,0,[]},
                  {rpc,'-handle_call_call/6-fun-0-',5,
                       [{file,"rpc.erl"},{line,187}]}]}

有一种解决版本是将“/var/lib/rabbitmq/mnesia/”目录下的所有内容删掉（rm -rf *），然后重新启动再配置。

And….

很多时候也会遇到这样的情况：

[root@hiddenzhu-8drdc rabbitmq]# rabbitmqctl -n rabbit2 status
Status of node 'rabbit2@hiddenzhu-8drdc' ...
Error: unable to connect to node 'rabbit2@hiddenzhu-8drdc': nodedown
diagnostics:
- nodes and their ports on hiddenzhu-8drdc: [{rabbit1,44494},
                                             {rabbitmqctl2271,60458}]
- current node: 'rabbitmqctl2271@hiddenzhu-8drdc'
- current node home dir: /root
- current node cookie hash: VCwbL3S9/ydrGgVsrLjVkA==

这说明rabbitmq节点并未启动起来，需要进一步查看排除异常。

End…