Rhel6-tomcat+nginx+memcached配置文档

理论基础：

User - > web ->nginx  ->tomcat1 ->*.jsp

80          8080

↓      -> tomcat2

html    -> tomcat3

用户通过web访问网页，默认访问的是80端口，即访问nginx。由于nginx只支持静态网页，如果用户访问的是类似*.jsp的动态网页，需要通过应用服务器tomcat先进行处理。tomcat相当与一个媒介，将动态网页经过处理之后交给nginx，最终返回给用户。

nginx后端的tomcat可以实现负载均衡，但是为了防止用户在访问过程出现类似下列情况：

访问登录页面时,帐号密码输入完成后，按下一步后马上又弹出一个登录界面

需要给nginx添加sticky模块，使得对于来自同一个用户的请求始终交给同一个tomcat处理

192.168.0.1 -> tomcat1

192.168.0.2 ->tomcat2 根据IP进行负载均衡

为了防止当用户在访问注册页面时，当用户填写注册信息到一半时，服务器宕机，当用户按“下一步”之后，用户之前提交的资料全部消失了，需要结合memcached来解决此问题。

nginx

|

T1      T2

| \     / |

|    \/   |

|    /\   |

|  /    \ |

M1       M2

交叉式存储，完美解决单点故障的问题

1号机子上有两个服务
T1和 M1

2号机子上有两个服务
T2 和 M2

默认的T1的数据往M2中存，T2的数据往M1中存，防止单点故障。

当T1服务挂掉之后，由T2接管，T2从M2中取得之前T1存在M2上的数据,之后提交的数据继续往M2上存（注：如果此时开启了另一个请求由T2接管，数据默认的往M1上存，而不是M2）；

接着如果连M2也挂掉了，但是由于T2是正常工作的，T2会将M2上所有的数据取出，并存到M1中，使得用户提交的数据仍然存在，不会丢失。

当1号机子挂掉后，T2存在M1上的数据将丢失

配置：

系统环境: rhel6 x86_64 iptables and selinux disabled

主机: n1:192.168.122.185:nginx tomcat memcached server85.example.com

n2:192.168.122.117:tomcat memcached server17.example.com

相关网址: rpm.pbone.net tomcat.apache.org

所需的包：memcache-2.2.5.tgz
memcached-1.4.15.tar.gz libevent-devel-1.4.13-4.el6.x86_64.rpm
libevent-doc-1.4.13-4.el6.noarch.rpm
libevent-headers-1.4.13-4.el6.noarch.rpm webbench-1.5.tar.gz
jdk-6u32-linux-x64.bin apache-tomcat-7.0.37.tar.gznginx-sticky-module-1.0.tar.gz
asm-3.2.jar kryo-1.04.jar kryo-serializers-0.10.jar
memcached-session-manager-1.6.3.jar
memcached-session-manager-tc7-1.6.3.jar minlog-1.2.jar
msm-kryo-serializer-1.6.3.jar reflectasm-1.01.jar
spymemcached-2.7.3.jar

1.安装memcached

[root@server85
kernel]# tar zxf memcache-2.2.5.tgz

[root@server85
kernel]# cd memcache-2.2.5

[root@server85
memcache-2.2.5]# phpize

[root@server85
memcache-2.2.5]# ./configure

[root@server85
memcache-2.2.5]# make && make install

[root@server85
memcache-2.2.5]# vim /usr/local/lnmp/php/etc/php.ini

extension=memcache.so

[root@server85
memcache-2.2.5]# /etc/init.d/php-fpm reload

[root@server85
kernel]# tar zxf memcached-1.4.15.tar.gz

[root@server85
kernel]# cd memcached-1.4.15

[root@server85
memcached-1.4.15]# ./configure

此时会出现以下错误:

checking
for libevent directory... configure: error: libevent is required.
You can get it from http://www.monkey.org/~provos/libevent/

解决方法如下:

[root@server85
memcached-1.4.15]# yum install libevent

[root@server85
kernel]# rpm -ivh libevent-*

注：libevent-*包含:libevent-devel-1.4.13-4.el6.x86_64.rpm
libevent-doc-1.4.13-4.el6.noarch.rpm
libevent-headers-1.4.13-4.el6.noarch.rpm

[root@server85
memcached-1.4.15]# make && make install

[root@server85
~]# memcached -m 50M -d -u root （若不制定大小默认64M,每次最大不超过2G)

[root@server85
~]# ps ax

6106 ? Ssl 0:00 memcached -m 50M -d -u root

[root@server85
~]# netstat -antple

tcp
0 0 0.0.0.0:11211 0.0.0.0:*
LISTEN 0 12791 6106/memcached

出现类似上方的内容表示安装成功.

[root@server85 kernel]# cd memcache-2.2.5/

[root@server85 memcache-2.2.5]# cp memcache.php /usr/local/lnmp/nginx/html/

[root@server85 memcache-2.2.5]# vim /usr/local/lnmp/nginx/html/memcache.php

define('ADMIN_PASSWORD','westos'); // Admin Password

#$MEMCACHE_SERVERS[] = 'mymemcache-server1:11211';

$MEMCACHE_SERVERS[] = '192.168.122.185:11211';

[root@server85 memcache-2.2.5]# nginx -s reload

访问server85.example.com/memcache.php:

进行压力测试:

[root@server85 kernel]# tar zxf webbench-1.5.tar.gz

[root@server85 kernel]# cd webbench-1.5

[root@server85 webbench-1.5]# make

此时会出现以下错误:

/bin/sh: ctags: command not found

解决方法如下:

[root@server85 webbench-1.5]# yum install ctags -y

[root@server85 webbench-1.5]# make clean

[root@server85
webbench-1.5]# make install

[root@server85
~]# webbench -c 500 -t 10 http://192.168.122.185/memcache.php

Webbench
- Simple Web Benchmark 1.5

Copyright
(c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking:
GET http://server85.example.com/memcache.php

500
clients, running 10 sec.

Speed=20442
pages/min, 102477 bytes/sec.

Requests:
3407 susceed, 0 failed.

或者访问server85.example.com/status:

2.安装tomcat

[root@server85 kernel]# chmod +x jdk-6u32-linux-x64.bin

[root@server85 kernel]# ./jdk-6u32-linux-x64.bin

[root@server85 kernel]# mv jdk1.6.0_32/ /usr/local/jdk

[root@server85 kernel]# vim /etc/profile

export JAVA_HOME=/usr/local/jdk

export CLASSPATH=:$JAVA_HOME/lib

export PATH=$PATH:$JAVA_HOME/bin

[root@server85 kernel]# source /etc/profile

执行echo $PATH能看到/usr/local/jdk/bin说明已生效.

[root@server85 ~]# vim test.java

public class test{

public static void main(String[] agre) {

System.out.println("Hello World!");

}

}

[root@server85 ~]# javac test.java

[root@server85 ~]# java test

打印出Hello World!说明安装成功!

[root@server85 kernel]# tar zxf apache-tomcat-7.0.37.tar.gz -C /usr/local/

[root@server85 kernel]# cd /usr/local/

[root@server85 local]# mv apache-tomcat-7.0.37/ tomcat

[root@server85 local]# cd tomcat/bin/

[root@server85 bin]# ./startup.sh

[root@server85 bin]# netstat -antple

tcp 0 0 :::8080 :::* LISTEN 0 57982 7465/java

端口说明启动成功.

访问server85.example.com:8080出现如下页面说明tomcat正常.

[root@server85 bin]# cd /usr/local/tomcat/webapps/ROOT/

[root@server85 ROOT]# vim test.jsp

the time is: <%=new java.util.Date() %>

访问server85.example.com:8080/test.jsp出现如下页面为正常.

[root@server85 ROOT]# vim /usr/local/lnmp/nginx/conf/nginx.conf

location ~ \.jsp$ {

proxy_pass http://127.0.0.1:8080;

端口处理)

[root@server85 ROOT]# nginx -s reload

以下步骤在server85上实施：

[root@server85 kernel]# tar zxf nginx-sticky-module-1.0.tar.gz

[root@server85 kernel]# tar zxf nginx-1.2.7.tar.gz

[root@server85 kernel]# cd nginx-1.2.7

[root@server85 nginx-1.2.7]# vim src/core/nginx.h

#define NGINX_VER "nginx"

[root@server85 nginx-1.2.7]# vim auto/cc/gcc

# debug

#CFLAGS="$CFLAGS -g"

[root@server85 nginx-1.2.7]# ./configure --prefix=/usr/local/lnmp/nginx/ --with-http_ssl_module --with-http_stub_status_module –add-module=/root/kernel/nginx-sticky-module-1.0

[root@server85 nginx-1.2.7]# make && make install

[root@server85 nginx-1.2.7]# vim /usr/local/lnmp/nginx/conf/nginx.conf

http {

upstream koen{

sticky;

server 192.168.122.185:8080;

server 192.168.122.117:8080;

}

server {

listen 80;

server_name server85.example.com;

location ~ \.jsp$ {

proxy_pass http://koen;

}

}

}

[root@server85 nginx-1.2.7]# nginx -s stop

[root@server85 nginx-1.2.7]# nginx

[root@server85 nginx-1.2.7]# memcached -d -u root

[root@server85 nginx-1.2.7]# cd /usr/local/tomcat/webapps/ROOT/

[root@server85 ROOT]# vim test.jsp

<%@ page contentType="text/html; charset=GBK" %>

<%@ page import="java.util.*" %>

<html><head><title>Cluster App Test</title></head>

<body>

Server Info:

<%

out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>

<%

out.println("<br> ID " + session.getId()+"<br>");

String dataName = request.getParameter("dataName");

if (dataName != null && dataName.length() > 0) {

String dataValue = request.getParameter("dataValue");

session.setAttribute(dataName, dataValue);

}

out.print("<b>Session list</b>");

Enumeration e = session.getAttributeNames();

while (e.hasMoreElements()) {

String name = (String)e.nextElement();

String value = session.getAttribute(name).toString();

out.println( name + " = " + value+"<br>");

System.out.println( name + " = " + value);

}

%>

<form action="test.jsp" method="POST">

name:<input type=text size=20 name="dataName">

<br>

key:<input type=text size=20 name="dataValue">

<br>

<input type=submit>

</form>

</body>

</html>

[root@server85 ROOT]# cd /usr/local/tomcat/lib/

把如下软件包放置在该目录下：

asm-3.2.jar

kryo-1.04.jar

kryo-serializers-0.10.jar

memcached-session-manager-1.6.3.jar

memcached-session-manager-tc7-1.6.3.jar

minlog-1.2.jar

msm-kryo-serializer-1.6.3.jar

reflectasm-1.01.jar

spymemcached-2.7.3.jar

[root@server85 lib]# vim /usr/local/tomcat/conf/context.xml

<Context>

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

memcachedNodes="n1:192.168.122.185:11211,n2:192.168.122.117:11211"

failoverNodes="n1"

requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"

/>

</Context>

[root@server85 lib]# /usr/local/tomcat/bin/shutdown.sh

[root@server85 lib]# /usr/local/tomcat/bin/startup.sh

以下步骤在server17上实施：

[root@server17
~]# memcached -d -u root

[root@server17
~]# cd /usr/local/tomcat/webapps/ROOT/

[root@server17
ROOT]# vim test.jsp

<%@
page contentType="text/html; charset=GBK" %>

<%@
page import="java.util.*" %>

<html><head><title>Cluster
App Test</title></head>

<body>

Server
Info:

<%

out.println(request.getLocalAddr()
+ " : " + request.getLocalPort()+"<br>");%>

<%

out.println("<br>
ID " + session.getId()+"<br>");

String
dataName = request.getParameter("dataName");

if
(dataName != null && dataName.length() > 0) {

String
dataValue = request.getParameter("dataValue");

session.setAttribute(dataName,
dataValue);

}

out.print("<b>Session
list</b>");

Enumeration
e = session.getAttributeNames();

while
(e.hasMoreElements()) {

String
name = (String)e.nextElement();

String
value = session.getAttribute(name).toString();

out.println(
name + " = " + value+"<br>");

System.out.println(
name + " = " + value);

}

%>

<form
action="test.jsp" method="POST">

name:<input
type=text size=20 name="dataName">

<br>

key:<input
type=text size=20 name="dataValue">

<br>

<input
type=submit>

</form>

</body>

</html>

[root@server17
ROOT]# cd /usr/local/tomcat/lib/

把如下软件包放置在该目录下：

asm-3.2.jar

kryo-1.04.jar

kryo-serializers-0.10.jar

memcached-session-manager-1.6.3.jar

memcached-session-manager-tc7-1.6.3.jar

minlog-1.2.jar

msm-kryo-serializer-1.6.3.jar

reflectasm-1.01.jar

spymemcached-2.7.3.jar

[root@server17
lib]# vim /usr/local/tomcat/conf/context.xml

<Context>

<Manager
className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

memcachedNodes="n1:192.168.122.185:11211,n2:192.168.122.117:11211"

failoverNodes="n2"

requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"

transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"

/>

</Context>

[root@server17
bin]# /usr/local/tomcat/bin/shutdown.sh

[root@server17
bin]# /usr/local/tomcat/bin/startup.sh

测试：

1.存储数据:

2.把server85上的m1宕掉然后存入user3

3.重置server85上的m1，存user4

4.把server17上的m2宕掉，存user5

5.重置server17上的m2,存user6

6.把server17上的tomcat关闭，存user7

测试完毕！！！