minio 集群搭建

 

具体实际的取舍可以参考官方文档，我使用的是4 node 4 driver 模式

环境机器说明

192.168.31.2 192.168.31.3 192.168.31.4 192.168.31.5
每台机器单独挂在4块盘  data1 data2 data3 data4
minio 安装包参考github 网站

参考图

启动&&运行

• 配置key(实际可以直接配置到profile中)

export MINIO_ACCESS_KEY=<ACCESS_KEY>
export MINIO_SECRET_KEY=<SECRET_KEY>

• 启动(每台机器执行)

/usr/local/bin/minio server http://192.168.31.2/data1 http://192.168.31.2/data2 \
               http://192.168.31.2/data3 http://192.168.31.2/data4 \
               http://192.168.31.3/data1 http://192.168.31.3/data2 \
               http://192.168.31.3/data3 http://192.168.31.3/data4 \
               http://192.168.31.4/data1 http://192.168.31.4/data2 \
               http://192.168.31.4/data3 http://192.168.31.4/data4 \
               http://192.168.31.5/data1 http://192.168.31.5/data2 \
               http://192.168.31.5/data3  http://192.168.31.5/data4
等待片刻，节点加入成功，并会格式化磁盘

配置nginx 支持lb

我使用了lvs nginx 是在realserver 上，并使用了dr 模式

   upstream minio {
      server 192.168.31.2:9000 weight=10 max_fails=2 fail_timeout=30s;
       server 192.168.31.3:9000 weight=10 max_fails=2 fail_timeout=30s;
       server 192.168.31.4:9000 weight=10 max_fails=2 fail_timeout=30s;
       server 192.168.31.5:9000 weight=10 max_fails=2 fail_timeout=30s;
    }
   server {
    listen 9000;
        server_name localhost;
        charset utf-8;
        default_type text/html;
        location /{
           proxy_set_header Host $http_host;
                proxy_set_header X-Forwarded-For $remote_addr;
            client_body_buffer_size 10M;
                client_max_body_size 10G;
                   proxy_buffers 1024 4k;
                proxy_read_timeout 300;
                proxy_next_upstream error timeout http_404;
             proxy_pass http://minio;
        }
     }

访问测试

说明

实际使用可能需要使用systemd 进行管理，官方提供了脚本

• 参考脚本

[Unit]
Description=Minio
Documentation=https://docs.minio.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local

User=minio-user
Group=minio-user

PermissionsStartOnly=true

EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "[ -n \"${MINIO_VOLUMES}\" ] || echo \"Variable MINIO_VOLUMES not set in /etc/defaults/minio\""

ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# Let systemd restart this service only if it has ended with the clean exit code or signal.
Restart=on-success

StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop Minio
KillSignal=SIGTERM

SendSIGKILL=no

SuccessExitStatus=0

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

参考资料

https://docs.minio.io/docs/distributed-minio-quickstart-guide
https://github.com/minio/minio-service