部署ActiveMQ的Share File System Master-Slave
之前在项目里用MQ是用单节点，因为业务量不大没有主从。这样风险很大，会有单点问题。新项目起来了，需要一个高可用的MQ，故研究了下AMQ的几种master-slave方式：
1、Pure Master-Slave
5.8版本之后已经不支持了，主要原因是Master挂了之后再起来无法自动融入集群，需要停掉MQ重新启动才可以。这里略过。
http://activemq.apache.org/pure-master-slave.html
2、Share File System Master-Slave
通过共享文件系统实现主从节点，解决单点问题。可以理解为将数据放在第三方。
http://activemq.apache.org/shared-file-system-master-slave.html
3、JDBC Master-Slave
数据放在库里，严重依赖数据库的性能，要是数据库挂了更麻烦。
http://activemq.apache.org/jdbc-master-slave.html
3、zookeeper Master-Slave
用过Hadoop的都知道zookeeper是一个强大的调度工具。它可以自动选择可用节点来实现Master-Slave，但太费劲，感兴趣可以戳下面链接
http://activemq.apache.org/replicated-leveldb-store.html
我们准备采用第二种Share File System Master-Slave，简单粗暴，快捷有效。
在192.168.3.31上下载安装AMQ就不用说了，注意机器名hostname里面不能有下划线，一开始我被坑死了，死活启动不了。。。
在192.168.3.31上安装NFS：
yum install nfs-utils nfs4-acl-tools portmap
其中portmap是NFS依赖的服务，必须确认先安装：（记住名字rpcbind）
[root@bxcsweb data]# yum install portmap
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.btte.net
* extras: mirrors.btte.net
* updates: mirrors.btte.net
Setting up Install Process
Package rpcbind-0.2.-.el6.x86_64 already installed and latest version
Nothing to do
直接启动NFS会报错：
[root@bxcsweb data]# service nfs restart
关闭 NFS 守护进程： [失败]
关闭 NFS mountd： [失败]
Shutting down RPC idmapd: [失败]
启动 NFS 服务： [确定]
启动 NFS mountd： [失败]
启动 NFS 守护进程：rpc.nfsd: writing fd to kernel failed: errno (Connection refused)
rpc.nfsd: unable to set any sockets for nfsd [失败]
先启动依赖，再启动NFS:
service rpcbind restart
service nfs restart
然后在被挂载的主节点机器上设置共享哪些目录、以及共享策略(从节点不能设置,否则会有问题)：（编辑/etc/exports文件）
这个文件默认是空的，编辑的格式：
[共享的目录] [主机名1或IP1(参数1,参数2)] [主机名2或IP2(参数3,参数4)]
下面是一些NFS共享的常用参数： ro 只读访问
rw 读写访问
sync 所有数据在请求时写入共享
async NFS在写入数据前可以相应请求
secure NFS通过1024以下的安全TCP/IP端口发送
insecure NFS通过1024以上的端口发送
wdelay 如果多个用户要写入NFS目录，则归组写入（默认）
no_wdelay 如果多个用户要写入NFS目录，则立即写入，当使用async时，无需此设置。
hide 在NFS共享目录中不共享其子目录
no_hide 共享NFS目录的子目录
subtree_check 如果共享/usr/bin之类的子目录时，强制NFS检查父目录的权限（默认）
no_subtree_check 和上面相对，不检查父目录权限
all_squash 共享文件的UID和GID映射匿名用户anonymous，适合公用目录。
no_all_squash 保留共享文件的UID和GID（默认）
root_squash root用户的所有请求映射成如anonymous用户一样的权限（默认）
no_root_squas root用户具有根目录的完全管理访问权限 （不安全）
anonuid=xxx 指定NFS服务器/etc/passwd文件中匿名用户的UID
anongid=xxx 指定NFS服务器/etc/passwd文件中匿名用户的GID
后来编辑为：
使得31、32机器可以有NFS的/home/shareMQ目录的读写权限
/home/shareMQ 192.168.3.32(rw,sync) 192.168.3.31(rw,sync)
然后restart31机器的NFS服务 （切记要重启一下）
到32机器上尝试mount MQ的数据到31的NFS Server上：
结果报错，错误很明显，不识别-t后面的类型：
[root@bxcscache data]# mount -t nfs 192.168.3.31:/home/shareMQ /usr/local/apache-activemq-5.10./data/kahadb/
mount: wrong fs type, bad option, bad superblock on 192.168.3.31:/home/shareMQ,
missing codepage or helper program, or other error
(for several filesystems (e.g. nfs, cifs) you might
need a /sbin/mount.<type> helper program)
In some cases useful info is found in syslog - try
dmesg | tail or so
是因为需要安装NFS工具：
yum install nfs-utils nfs-utils-lib
然后主从节点都执行挂载命令：
mount -t nfs 192.168.3.31:/home/shareMQ /usr/local/apache-activemq-5.10./data/kahadb/
31机器上也挂载：
[root@bxcsweb shareMQ]# mount -t nfs 192.168.3.31:/home/shareMQ/ /usr/local/apache-activemq-5.10./data/kahadb/
查看已挂载的情况：
31(31是主节点,可以发现后面的服务端客户端两个ip都是自己 而且要在主节点新建/home/shareMQ目录,从节点不用建目录)
[root@bxcsweb kahadb]# mount
/dev/vda2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=,mode=)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /main type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
192.168.3.31:/home/shareMQ/ on /usr/local/apache-activemq-5.10./data/kahadb type nfs (rw,vers=,addr=192.168.3.31,clientaddr=192.168.3.31)
32
[root@bxcscache kahadb]# mount
/dev/vda2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=,mode=)
tmpfs on /dev/shm type tmpfs (rw)
/dev/vda1 on /main type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
192.168.3.31:/home/shareMQ on /usr/local/apache-activemq-5.10./data/kahadb type nfs (rw,vers=,addr=192.168.3.31,clientaddr=192.168.3.32)
启动两台机器的activeMQ：
发现有一台一直打出如下日志，注意红色部分。后启动这这个节点一直被当作slave节点，khaha的文件锁已经被先启动的master拿到了，所以10秒尝试获取一次锁：
./activemq start INFO: Using java '/usr/local/jdk1.6.0_45/bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
INFO: pidfile created : '/usr/local/apache-activemq-5.10.0/data/activemq-bxcsweb.pid' (pid '3439')
[root@bxcsweb bin]# tail -f ../data/activemq.log
2014-06-13 04:36:09,628 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2014-06-13 04:36:09,631 | WARN | Store limit is 102400 mb (current store usage is 0 mb). The data directory: /usr/local/apache-activemq-5.10.0/data/kahadb only has 7555 mb of usable space - resetting to maximum available disk space: 7555 mb | org.apache.activemq.broker.BrokerService | main
2014-06-13 04:36:09,645 | ERROR | Temporary Store limit is 51200 mb, whilst the temporary data directory: /usr/local/apache-activemq-5.10.0/data/localhost/tmp_storage only has 7554 mb of usable space - resetting to maximum available 7554 mb. | org.apache.activemq.broker.BrokerService | main
2014-06-13 04:36:10,104 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2014-06-13 04:36:10,167 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2014-06-13 04:36:10,439 | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | main
2014-06-13 04:45:26,441 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@68814013: startup date [Fri Jun 13 04:45:26 CST 2014]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2014-06-13 04:45:27,850 | INFO | PListStore:[/usr/local/apache-activemq-5.10.0/data/localhost/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2014-06-13 04:45:27,895 | INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/usr/local/apache-activemq-5.10.0/data/kahadb] | org.apache.activemq.broker.BrokerService | main
2014-06-13 04:45:27,906 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2014-06-13 04:45:37,911 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2014-06-13 04:45:47,920 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2014-06-13 04:45:57,928 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2014-06-13 04:46:07,936 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2014-06-13 04:46:17,940 | INFO | Database /usr/local/apache-activemq-5.10.0/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
如果要是master挂了，这个节点会立马获取到锁，然后启动起来：
比如我关掉master，在slave上看到：
-- ::, | INFO | Database /usr/local/apache-activemq-5.10./data/kahadb/lock is locked... waiting seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
-- ::, | INFO | Database /usr/local/apache-activemq-5.10./data/kahadb/lock is locked... waiting seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
-- ::, | INFO | Database /usr/local/apache-activemq-5.10./data/kahadb/lock is locked... waiting seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
-- ::, | INFO | Database /usr/local/apache-activemq-5.10./data/kahadb/lock is locked... waiting seconds for the database to be unlocked. Reason: java.io.IOException: File '/usr/local/apache-activemq-5.10.0/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
-- ::, | INFO | KahaDB is version | org.apache.activemq.store.kahadb.MessageDatabase | main
-- ::, | INFO | Recovering from the journal ... | org.apache.activemq.store.kahadb.MessageDatabase | main
-- ::, | INFO | Recovery replayed operations from the journal in 0.068 seconds. | org.apache.activemq.store.kahadb.MessageDatabase | main
-- ::, | INFO | Apache ActiveMQ 5.10. (localhost, ID:bxcsweb---:) is starting | org.apache.activemq.broker.BrokerService | main
-- ::, | INFO | Listening for connections at: tcp://bxcsweb:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
-- ::, | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
-- ::, | INFO | Listening for connections at: amqp://bxcsweb:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
-- ::, | INFO | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
-- ::, | INFO | Listening for connections at: stomp://bxcsweb:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
-- ::, | INFO | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
-- ::, | INFO | Listening for connections at: mqtt://bxcsweb:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
-- ::, | INFO | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
-- ::, | INFO | Listening for connections at ws://bxcsweb:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
-- ::, | INFO | Connector ws started | org.apache.activemq.broker.TransportConnector | main
-- ::, | INFO | Apache ActiveMQ 5.10. (localhost, ID:bxcsweb---:) started | org.apache.activemq.broker.BrokerService | main
-- ::, | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
-- ::, | WARN | Store limit is mb (current store usage is mb). The data directory: /usr/local/apache-activemq-5.10./data/kahadb only has mb of usable space - resetting to maximum available disk space: mb | org.apache.activemq.broker.BrokerService | main
-- ::, | ERROR | Temporary Store limit is mb, whilst the temporary data directory: /usr/local/apache-activemq-5.10./data/localhost/tmp_storage only has mb of usable space - resetting to maximum available mb. | org.apache.activemq.broker.BrokerService | main
-- ::, | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
-- ::, | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
-- ::, | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | main
至此，ActiveMQ的Share File System Master-Slave部署就算成功结束了。
然后是安全性配置：
activemq.xml已经默认打开了安全性。只用在conf/jetty-realm.properties配控制台登录用户名、密码。
另外连接队列的用户名密码在activemq.xml的<broker>节点中添加以下元素以提供对建立连接时的用户名/密码的支持：
参考了这篇博文：http://www.cnblogs.com/zhongkl/archive/2011/08/12/2136553.html
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="system" password="manager"
groups="users,admins"/>
<authenticationUser username="user" password="password"
groups="users"/>
<authenticationUser username="guest" password="password" groups="guests"/>
</users>
</simpleAuthenticationPlugin> <!-- lets configure a destination based authorization mechanism -->
<authorizationPlugin>
<map>
<authorizationMap>
<authorizationEntries>
<authorizationEntry queue=">" read="admins" write="admins" admin="admins" />
<authorizationEntry queue="USERS.>" read="users" write="users" admin="users" />
<authorizationEntry queue="GUEST.>" read="guests" write="guests,users" admin="guests,users" /> <authorizationEntry topic=">" read="admins" write="admins" admin="admins" />
<authorizationEntry topic="USERS.>" read="users" write="users" admin="users" />
<authorizationEntry topic="GUEST.>" read="guests" write="guests,users" admin="guests,users" /> <authorizationEntry topic="ActiveMQ.Advisory.>" read="guests,users" write="guests,users" admin="guests,users"/>
</authorizationEntries> <!-- let's assign roles to temporary destinations. comment this entry if we don't want any roles assigned to temp destinations -->
<tempDestinationAuthorizationEntry>
<tempDestinationAuthorizationEntry read="tempDestinationAdmins" write="tempDestinationAdmins" admin="tempDestinationAdmins"/>
</tempDestinationAuthorizationEntry>
</authorizationMap>
</map>
</authorizationPlugin>
</plugins>
接下来我们看看spring里怎么配：
<!--连接工厂-->
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<!--31、32两台机器连接，断线重连，10毫秒一次，最多尝试60次-->
<value>failover:(tcp://192.168.3.31:61616,tcp://192.168.3.32:61616)?initialReconnectDelay=10&maxReconnectAttempts=60</value>
</property>
<property name="userName" value="***" />
<property name="password" value="***" />
</bean>
OK 结束。
其中，各种搭建过程部分参考了：
http://harbey.iteye.com/blog/1756607
http://shensy.iteye.com/blog/1752529
部署ActiveMQ的Share File System Master-Slave的更多相关文章
- Shared File System Master Slave 全配置以及测试
在本机完成2个broker的共享文件测试 2个broker的完整配置文件如下 <!-- Licensed to the Apache Software Foundation (ASF) ...
- SharedFile System Master Slave(共享文件系统)做ActiveMQ集群
WINDOWS环境下:http://www.apache.org/dyn/closer.cgi?path=/activemq/apache-activemq/5.9.0/apache-activemq ...
- ActiveMQ使用Zookeeper+LevelDb配置Master/Slave集群
前言: 本文介绍的AMQ集群是Master-Slave模式的,官网介绍三种方案: (1)基于共享文件系统的,(2)基于JDBC,(3)基于可复制的LevelDB. 关于三种方式的对比网上已经有很多,本 ...
- ActiveMQ集群支持Master/Slave模式
现在ActiveMQ, 在Failover方面有两种解决方案:Pure Master Slave和Shared File System Master Slave. 先看Pure Master ...
- PatentTips – EMC Virtual File System
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generally relates to net ...
- Sharing The Application Tier File System in Oracle E-Business Suite Release 12.2
The most current version of this document can be obtained in My Oracle Support Knowledge Document 13 ...
- ActiveMQ新的Master/Slave存储共享机制Replicated LevelDB Store
ActiveMQ新的Master/Slave存储共享机制Replicated LevelDB Store 使用ZooKeeper协调选择一个node作为master.被选择的master broker ...
- 【摘录】JDBC Master Slave(JDBC方式的JMS集群)
JDBC Master Slave First supported in ActiveMQ version 4.1 If you are using pure JDBC and not using t ...
- NFS（Network File System）即网络文件系统 (转)
第1章 NFS介绍 1.1 NFS服务内容的概述 □ RPC服务知识概念介绍说明,以及RPC服务存在价值(必须理解掌握) □ NFS服务工作原理讲解(必须理解掌握) □ NFS共享文件系统使用原理讲解 ...
随机推荐
- office2016 部分截图
哈啊哈哈啊哈 看看吧~~~~~~~~~~~~~~~~~
- codeforces 388C Fox and Card Game
刚刚看到这个题感觉是博弈题: 不过有感觉不像,应该是个贪心: 于是就想贪心策略: 举了一个例子: 3 3 1 2 3 4 3 4 1 2 5 4 1 2 5 8 如果他们两个每次都拿对自己最有利的那个 ...
- 不改变中间层，如何玩转 .NET 的远程处理功能？
原文链接: https://msdn.microsoft.com/enus/library/aa289846(v=vs.71).aspx Visual Studio .NET 2003 该方案展示了传 ...
- 如何将CELERY放到后台执行？
在作正式环境,这个是必须的. 于是找了两小时文档, 以下这个方法,相对来说好实现. 就是要注意supervisord.conf的目录存放位置. 放在DJANGO的PROJ目录下,是最佳位置. http ...
- PHP unlink() 函数
定义和用法 unlink() 函数删除文件. 若成功,则返回 true,失败则返回 false. 语法 unlink(filename,context) 参数 描述 filename 必需.规定要删除 ...
- Uva 12361 File Retrieval 后缀数组+并查集
题意:有F个单词,1 <= F <=60 , 长度<=10^4, 每次可以输入一个字符串,所有包含该字串的单词会形成一个集合. 问最多能形成多少个不同的集合.集合不能为空. 分析:用 ...
- SPRING IN ACTION 第4版笔记-第四章ASPECT-ORIENTED SPRING-009-带参数的ADVICE2
一. 情景:有个魔术师会读心术,常人一想一事物他就能读到.以魔术师为切面织入常人的内心. 二. 1. // <start id="mindreader_java" /> ...
- 乐1/MACBOOK/ N1 Type-C接口新体验
经过在华强北电子市场排队一个小时,笔者顺利买到了期待已久的乐1,结合之前的NOKIA平板电脑N1,苹果全新MACBOOK,终于集齐了手机.平板.笔记本电脑三种TYPE-C接口设备(能兑换极品装备吗?^ ...
- Apache CXF 例子
来自:http://www.cnblogs.com/frankliiu-java/articles/1641949.html Apache CXF 是一个开放源代码框架,是在Xfire 跟Celtix ...
- 【HDOJ】4455 Substrings
5000ms的时限,还挺长的.算法是DP.思路是找到ans[1..n]的结果,然后Query就容易做了.问题是怎么DP?考虑:1 1 2 3 4 4 5w=1: 7, 7 = 1 * 7w=2: 10 ...