Nginx反向代理+负载均衡简单实现

一、基础环境：

负   载  机：A机器： 192.168.71.223
后端机器1：B机器：192.168.71.224
后端机器2：C机器：192.168.71.226

需求：
1）访问A机器的8080端口，反向代理到B机器的8080端口；
   访问A机器的8088端口，反向代理到C机器的8088端口；
   访问http://192.168.71.223:8090/ios，反向代理到B机器http://192.168.1.102:8090/ios/

2）访问A机器的80端口，负载均衡到后端的两台机器B和C的80端口

二、安装配置：

1、三个机器共同执行：

1）编译安装nginx
yum install -y pcre* openssl* gcc gcc+
mkdir /opt/src  && cd /opt/src
wget http://nginx.org/download/nginx-1.8.0.tar.gz && tar -zxvf nginx-1.8.0.tar.gz && cd nginx-1.8.0
useradd www -M -s /sbin/nologin  #添加www用户，其中-M参数表示不添加用户家目录，-s参数表示指定shell类型
vi /opt/src/nginx-1.8.0/auto/cc/gcc
#CFLAGS="$CFLAGS -g"  #将这句注释掉 取消Debug编译模式 大概在179行

./configure --prefix=/opt/nginx --user=www --group=www --with-http_stub_status_module --with-http_ssl_module  #我们再配置下nginx编译参数
make && make install clean

2、机器A配置：

 cd /opt/nginx/conf
 vi  nginx.conf 

将配置改为：

worker_processes  2;

events {
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    charset utf-8;

    log_format  main  '$http_x_forwarded_for $remote_addr $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_cookie" $host $request_time';
    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;
    keepalive_timeout  65;

    fastcgi_connect_timeout 3000;
    fastcgi_send_timeout 3000;
    fastcgi_read_timeout 3000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;

    client_header_timeout 600s;
    client_body_timeout 600s;

    client_max_body_size 100m;
    client_body_buffer_size 256k;

    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 9;
    gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
    gzip_vary on;

    include vhosts/*.conf;
}

nginx.conf

ulimit -n 65535
mkdir vhosts && cd vhosts
vi 8080.conf

将配置改为：

server {
    listen 8080;
    server_name localhost;
    index index.html index.php index.htm;
    root /var/www/html;

    access_log  /usr/local/nginx/logs/8080-access.log main;
    error_log  /usr/local/nginx/logs/8080-error.log;

location / {
    proxy_pass http://192.168.71.224:8080;
    proxy_redirect off ;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header REMOTE-HOST $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_connect_timeout 300;             #跟后端服务器连接超时时间，发起握手等候响应时间
    proxy_send_timeout 300;                #后端服务器回传时间，就是在规定时间内后端服务器必须传完所有数据
    proxy_read_timeout 600;                #连接成功后等待后端服务器的响应时间，已经进入后端的排队之中等候处理
    proxy_buffer_size 256k;                #代理请求缓冲区,会保存用户的头信息以供nginx进行处理
    proxy_buffers 4 256k;                  #同上，告诉nginx保存单个用几个buffer最大用多少空间
    proxy_busy_buffers_size 256k;          #如果系统很忙时候可以申请最大的proxy_buffers
    proxy_temp_file_write_size 256k;       #proxy缓存临时文件的大小
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
}
}

8080.conf

server {
    listen 8088;
    server_name localhost;
    index index.html index.php index.htm;
    root /var/www/html;

    access_log  /usr/local/nginx/logs/8088-access.log main;
    error_log  /usr/local/nginx/logs/8088-error.log;

location / {
    proxy_pass http://192.168.71.226:8088;
    proxy_redirect off ;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header REMOTE-HOST $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    proxy_read_timeout 600;
    proxy_buffer_size 256k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_temp_file_write_size 256k;
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
}
}

8088.conf

server {
    listen 8090;
    server_name localhost;
    index index.html index.php index.htm;
    root /var/www/html;

    access_log  /usr/local/nginx/logs/8090-access.log main;
    error_log  /usr/local/nginx/logs/8090-error.log;

    location /ios/ {                            #这种情况，这里一定要匹配的是/ios/，不能是/ios
    proxy_pass http://192.168.71.224:8090;       #一定要保证192.168.1.102机器8090端口站点目录下有ios目录！否则访问会报错404！
    proxy_redirect off ;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header REMOTE-HOST $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    proxy_read_timeout 600;
    proxy_buffer_size 256k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_temp_file_write_size 256k;
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
}
}

8090.conf

upstream lb {
    server 192.168.71.224:80 max_fails=3 fail_timeout=30s;   #max_fails = 3 为允许失败的次数，默认值为1
    server 192.168.71.226:80 max_fails=3 fail_timeout=30s;   #fail_timeout = 30s 当max_fails次失败后，暂停将请求分发到该后端服务器的时间
}

server {
    listen 80;
    server_name localhost;
    index index.html index.php index.htm;
    root /var/www/html;

    access_log  /usr/local/nginx/logs/80-access.log main;
    error_log  /usr/local/nginx/logs/80-error.log;

    location / {
    proxy_pass http://lb;
    proxy_redirect off ;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header REMOTE-HOST $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    proxy_read_timeout 600;
    proxy_buffer_size 256k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_temp_file_write_size 256k;
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
}
}

LB.conf

启动Nginx：

启动nginx：
/opt/nginx/sbin/nginx -t  检查配置是否正确 #如果报缺少某个日志文件，创建目录及文件就可以
nginx: the configuration file /opt/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /opt/nginx/conf/nginx.conf test is successful
/opt/nginx/sbin/nginx     启动nginx

3、机器B配置：

cd /opt/nginx/conf  && cp nginx.conf  nginx.conf.bak
vi  nginx.conf

将配置改为：

user  www;
worker_processes  2;

events {
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    charset utf-8;

    log_format  main  '$http_x_forwarded_for $remote_addr $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_cookie" $host $request_time';
    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;
    keepalive_timeout  65;

    fastcgi_connect_timeout 3000;
    fastcgi_send_timeout 3000;
    fastcgi_read_timeout 3000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;

    client_header_timeout 600s;
    client_body_timeout 600s;

    client_max_body_size 100m;
    client_body_buffer_size 256k;

    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 9;
    gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
    gzip_vary on;

    include vhosts/*.conf;
}

nginx.conf

ulimit -n 65535
mkdir vhosts && cd vhosts
vi 8080.conf

将配置改为：

server {
    listen 8080;
    server_name localhost;
    index index.html index.php index.htm;

    access_log  /usr/local/nginx/logs/8080-access.log main;
    error_log  /usr/local/nginx/logs/8080-error.log;

location ~ / {
    root /var/www/html/8080;
    index index.html index.php index.htm;
}
}

8080.conf

server {
    listen 8090;
    server_name localhost;
    index index.html index.php index.htm;

    access_log  /usr/local/nginx/logs/8090-access.log main;
    error_log  /usr/local/nginx/logs/8090-error.log;

location ~ / {
    root /var/www/html/8090;        #针对上面匹配ios的path代理，要保证站点目录/var/www/html/8080下有ios目录存在
    index index.html index.php index.htm;
}
}

8090.conf

server {
   listen 80;
   server_name localhost;
   index index.html index.php index.htm;

   access_log  /usr/local/nginx/logs/80-access.log main;
   error_log  /usr/local/nginx/logs/80-error.log;

location ~ / {
   root /var/www/html;
   index index.html index.php index.htm;
}
}

80.conf

启动nginx：
/opt/nginx/sbin/nginx -t  检查配置是否正确 #如果报缺少某个日志文件，创建目录及文件就可以
nginx: the configuration file /opt/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /opt/nginx/conf/nginx.conf test is successful
/opt/nginx/sbin/nginx     启动nginx

4、机器C配置：

cd /opt/nginx/conf  && cp nginx.conf  nginx.conf.bak
vi  nginx.conf

将配置改为：

user  www;
worker_processes  2;

events {
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    charset utf-8;

    log_format  main  '$http_x_forwarded_for $remote_addr $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_cookie" $host $request_time';
    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;
    keepalive_timeout  65;

    fastcgi_connect_timeout 3000;
    fastcgi_send_timeout 3000;
    fastcgi_read_timeout 3000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;

    client_header_timeout 600s;
    client_body_timeout 600s;

    client_max_body_size 100m;
    client_body_buffer_size 256k;

    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 9;
    gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
    gzip_vary on;

    include vhosts/*.conf;
}

nginx.conf

ulimit -n 65535
mkdir vhosts && cd vhosts
vi 8080.conf

将配置改为：

server {
    listen 8088;
    server_name localhost;
    index index.html index.php index.htm;

    access_log  /usr/local/nginx/logs/8088-access.log main;
    error_log  /usr/local/nginx/logs/8088-error.log;

location ~ / {
    root /var/www/html/8088;
    index index.html index.php index.htm;
}
}

8088.conf

server {
    listen 80;
    server_name localhost;
    index index.html index.php index.htm;

    access_log  /usr/local/nginx/logs/80-access.log main;
    error_log  /usr/local/nginx/logs/80-error.log;

location ~ / {
    root /var/www/html/;
    index index.html index.php index.htm;
}
}

80.conf

启动nginx：
/opt/nginx/sbin/nginx -t  检查配置是否正确 #如果报缺少某个日志文件，创建目录及文件就可以
nginx: the configuration file /opt/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /opt/nginx/conf/nginx.conf test is successful
/opt/nginx/sbin/nginx     启动nginx

5、创建访问目录：

机器B的目录结构为：
[root@localhost var]# tree www/
www/
└── html
    ├── 8080
    │   └── index.html
    ├── 8090
    │   ├── index.html
    │   └── ios
    │       └── index.html
    └── index.html

echo '192.168.71.224:/var/www/html/' >> index.html
mkdir -p /var/www/html/8080
echo '192.168.71.224:/var/www/html/8080' >> /var/www/html/8080/index.html
mkdir -p /var/www/html/8090
echo '192.168.71.224/var/www/html/8090' >> /var/www/html/8090/index.html
mkdir -p /var/www/html/ios
echo 'http://192.168.71.224:8090/ios/' >> /var/www/html/8090/ios/index.html
chown www:www -R /var/www/html

机器C的目录结构为：
[root@localhost var]# tree www/
www/
└── html
    ├── 8088
    │   └── index.html
    └── index.html
mkdir -p /var/www/html/8088
echo '192.168.71.226:/var/www/html' >> index.html
echo '192.168.71.226:/var/www/html/8088/' >> /var/www/html/8080/index.html
chown www:www -R /var/www/html

6、验证：

1、访问http://192.168.71.223/显示的是：192.168.71.224:/var/www/html/，再刷新变为：192.168.71.226:/var/www/html

2、访问http://192.168.71.223:8080/显示的是：192.168.71.224:/var/www/html/8080

3、访问http://192.168.71.223:8088/显示的是：192.168.71.226:/var/www/html/8088/

4、访问http://192.168.71.223:8090/ios/显示的是：http://192.168.71.224:8090/ios/

参考：http://www.cnblogs.com/kevingrace/p/5839698.html