jboss eap 6.3 集群(cluster)配置

接上一篇继续，Domain模式解决了统一管理多台jboss的问题，今天我们来学习如何利用mod_cluster来实现负载均衡、容错。

mod_cluster是jboss的一个开源集群模块(基于apache 2.2.x)，官网地址为http://mod-cluster.jboss.org/ ，下面是使用步骤：

一、从官网下载binary文件

http://mod-cluster.jboss.org/downloads/1-2-6-Final-bin 这上面有各种OS的版本，本文以windows为试验环境，应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/windows/mod_cluster-1.2.6.Final-windows-amd64.zip 这个版本

(注：如果是linux环境，应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/linux-x86_64/mod_cluster-1.2.6.Final-linux2-x64.tar.gz)

二、解压安装

将下载下来的文件解压到jboss master server机器上的某个目录(假定master server的IP为172.21.129.181)，比如C:\Apache2.2\ ，mod_cluster其实就是一个包含集群模块的apache server，如果你的机器上之前有安装过apache server，且自动安装成了windows服务，强烈建议先卸载，否则启动mod_cluster对应的apache时，有可能启动的是之前的版本。

注：如果是linux环境安装，强烈建议将压缩包中的httpd目录，解压到/opt/jboss/目录下，因为linux版本的很多脚本和配置里，都是写死这个目录，如果修改了目录，apachectl、http.conf这些文件里的路径全都要改,十分麻烦

2.1 先执行C:\Apache2.2\bin\installconf.bat 生成默认配置 (注：linux环境下安装时，无需执行该步骤，直接进入2.2改http.conf文件即可)

2.2 修改C:\Apache2.2\conf\httpd.conf

a)

#Listen 12.34.56.78:80

Listen 80

40行，把原来的端口8000，改成80，这样方便直接用 http://172.21.129.181/xxx 来访问应用

b)

164行改成

ServerName 172.21.129.181:80

c)

最后的mod_cluster部分，参考下面修改

# MOD_CLUSTER_ADDS
# Adjust to you hostname and subnet.
<IfModule manager_module>
  Listen 172.21.129.181:6666
  ManagerBalancerName mycluster
  <VirtualHost 172.21.129.181:6666>
    <Location />
     Order deny,allow
     #Deny from all
     Allow from all
    </Location>

    KeepAliveTimeout 300
    MaxKeepAliveRequests 0
    #ServerAdvertise on http://127.0.0.1:6666
    AdvertiseFrequency 5
    #AdvertiseSecurityKey secret
    #AdvertiseGroup 224.0.1.105:23364
    EnableMCPMReceive

    <Location /mod_cluster_manager>
       SetHandler mod_cluster-manager
       Order deny,allow
       #Deny from all
       Allow from all
    </Location>

  </VirtualHost>
</IfModule>

主要是把127.0.0.1，改成master server对应的真实IP，另外把Deny from all注释掉，把紧贴着的一行改成Allow from all，去掉这些访问上的限制

下面是完整的http.conf参考内容:

 ServerRoot "C:/Apache2.2"
 Listen 80
 LoadModule actions_module modules/mod_actions.so
 LoadModule alias_module modules/mod_alias.so
 LoadModule auth_basic_module modules/mod_auth_basic.so
 LoadModule authz_host_module modules/mod_authz_host.so
 LoadModule autoindex_module modules/mod_autoindex.so
 LoadModule dir_module modules/mod_dir.so
 LoadModule log_config_module modules/mod_log_config.so
 LoadModule mime_module modules/mod_mime.so

 ServerAdmin jimmy@localhost.localdomain
 ServerName 172.21.129.181:80
 DocumentRoot "C:/Apache2.2/htdocs"

 <Directory />
     Options FollowSymLinks
     AllowOverride None
     Order deny,allow
     Allow from all
 </Directory>

 <Directory "C:/Apache2.2/htdocs">
     Options Indexes FollowSymLinks
     AllowOverride None
     Order allow,deny
     Allow from all
 </Directory>

 <IfModule dir_module>
     DirectoryIndex index.html
 </IfModule>

 <FilesMatch "^\.ht">
     Order allow,deny
     Deny from all
     Satisfy All
 </FilesMatch>

 ErrorLog "logs/error_log"
 LogLevel warn
 <IfModule log_config_module>
     LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
     LogFormat "%h %l %u %t \"%r\" %>s %b" common

     <IfModule logio_module>
       LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
     </IfModule>
     CustomLog "logs/access_log" common
 </IfModule>

 <IfModule alias_module>
     ScriptAlias /cgi-bin/ "C:/Apache2.2/cgi-bin/"
 </IfModule>

 <Directory "C:/Apache2.2/cgi-bin">
     AllowOverride None
     Options None
     Order allow,deny
     Allow from all
 </Directory>

 DefaultType text/plain

 <IfModule mime_module>
     TypesConfig conf/mime.types
     AddType application/x-compress .Z
     AddType application/x-gzip .gz .tgz
 </IfModule>

 LoadModule proxy_module modules/mod_proxy.so
 LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
 LoadModule proxy_http_module modules/mod_proxy_http.so
 LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
 LoadModule manager_module modules/mod_manager.so
 LoadModule slotmem_module modules/mod_slotmem.so
 LoadModule advertise_module modules/mod_advertise.so

 # MOD_CLUSTER_ADDS
 <IfModule manager_module>
   Listen 172.21.129.181:6666
   ManagerBalancerName mycluster
   <VirtualHost 172.21.129.181:6666>
     <Location />
      Order deny,allow
      Allow from all
     </Location>
     KeepAliveTimeout 300
     MaxKeepAliveRequests 0
     AdvertiseFrequency 5
     EnableMCPMReceive
     <Location /mod_cluster_manager>
        SetHandler mod_cluster-manager
        Order deny,allow
        Allow from all
     </Location>
   </VirtualHost>
 </IfModule>

2.3 将httpd安装成windows服务

C:\Apache2.2\bin\httpd -k install

再启动apache

C:\Apache2.2\bin\httpd -k start

顺利的话，浏览http://172.21.129.181:6666/mod_cluster_manager 应该能看到类似以下界面：(注：不要用google chrome浏览器，chrome打不开这个地址，不知道是啥原因，换成IE或Firefox就可以了)

这是很关键的一个界面，用来管理/监控所有节点，如果没成功，请仔细检查前面的步骤是否有误。

三、以domain模式启动所有jboss master/slave server

参考上一篇的做法，启动jboss master server / slave server，完成后，刷新 http://172.21.129.181:6666/mod_cluster_manager，会看到多出了好多节点

如果没有新的节点显示在该界面，检查jboss master server中创建的group，对应的profile是否为full-ha或ha

点击相关的链接，还能看到更详细的信息

下面是点击 show INFO output后的内容：

Node: [1],Name: d2c49277-af22-32b1-95be-b70e3ff20237,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [2],Name: 708b3ee3-fd4d-39eb-b24c-3384bd11958c,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [3],Name: c4051fb6-9cc9-30aa-967b-6ba1294026b8,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [4],Name: 17181717-8b5c-3289-b6ea-fedcee2a27cb,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 3,Read: 3355,Transfered: 0,Connected: 0,Load: 100
Node: [5],Name: 2465cf8d-9cb0-3248-b335-09c02f13b6fe,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1
Node: [6],Name: 68a1e90f-6645-38e0-8157-219ab2004148,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1
Node: [7],Name: b403bcec-5eb5-37a2-a956-51d273b87548,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [8],Name: 6e5ac603-5f19-3c90-8f97-d0bf01124c94,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Vhost: [2:1:1], Alias: example.com
Vhost: [2:1:2], Alias: localhost
Vhost: [2:1:3], Alias: default-host
Vhost: [4:1:4], Alias: default-host
Vhost: [4:1:5], Alias: localhost
Vhost: [4:1:6], Alias: example.com
Vhost: [1:1:7], Alias: example.com
Vhost: [1:1:8], Alias: localhost
Vhost: [1:1:9], Alias: default-host
Vhost: [3:1:10], Alias: default-host
Vhost: [3:1:11], Alias: localhost
Vhost: [3:1:12], Alias: example.com
Vhost: [6:1:13], Alias: localhost
Vhost: [6:1:14], Alias: default-host
Vhost: [7:1:15], Alias: example.com
Vhost: [8:1:16], Alias: example.com
Vhost: [8:1:17], Alias: localhost
Vhost: [8:1:18], Alias: default-host
Vhost: [7:1:19], Alias: localhost
Vhost: [7:1:20], Alias: default-host
Context: [2:1:1], Context: /SSO, Status: ENABLED
Context: [4:1:2], Context: /SSO, Status: ENABLED
Context: [1:1:3], Context: /SSO, Status: ENABLED
Context: [3:1:4], Context: /SSO, Status: DISABLED
Context: [7:1:5], Context: /SSO, Status: ENABLED
Context: [8:1:6], Context: /SSO, Status: ENABLED

注意一下：虽然我们有8台jboss server，但是mod_cluster貌似最多只能支持6个node的负载均衡，有2个node的转发因子是-1，即未生效(但大多数情况下，已经够用了)。mod_cluster还有其它一些限制，详情可参考 http://docs.jboss.org/mod_cluster/1.2.0/html_single/#d0e85

四、负载均衡、容错验证

在上一篇里，已经在my-group-1、my-group-2上部署了一个名为SSO的应用，现在可以直接用 http://172.21.129.181/SSO/ 来访问了，apache会智能转发到6个节点中的某一个，如果访问正常，说明负载均衡已经生效。

继续把其它server停掉几台，再继续访问http://172.21.129.181/SSO/ ，如果仍然正常，说明某几个node损坏，并不影响正常使用，即容错成功。

所有server全shutdown后，http://172.21.129.181/SSO/ 将无法访问。

最后附一张:架构图

参考文章：

[1] mod_cluster Documentation

[2] JBoss AS7 cluster with mod_cluster setup howto

[3] 使用mod_cluster进行apache httpd server和jboss eap 6.1集群配置