Docker_CICD笔记

1 环境说明

1.1 机器配置

主机名称	IP地址	系统版本/内存/cpu核数/硬盘	安装软件
controlnode	172.16.1.70/24	centos7.4/4/2/60	docker、docker镜像制作
slavenode1	172.16.1.71/24	centos7.4/4/2/60	docker、harbor仓库、gitlab仓库、cadvisor
slavenode2	172.16.1.72/24	centos7.4/4/2/60	docker、jenkins、cadvisor
slavenode3	172.16.1.73/24	centos7.4/4/2/60	docker、grafana、prometheus

2 安装docker

2.1 安装docker注意事项

1 替换国内yum源

# curl -o /etc/yum.repos.d/docker-ce.repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
# yum clean all
# yum makecache

2 服务器时区和时间保持国内

# ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# (echo "*/5 * * * * /usr/sbin/ntpdate ntp.aliyun.com >/dev/null 2>&1"; crontab -l) | crontab

3 selinux和firewalld关闭

# 关闭selinux
# sed -i '/SELINUX/{s/enforcing/disabled/}' /etc/selinux/config
# setenforce 0
​
# 关闭firewalld
# systemctl stop firewalld.service
# systemctl disable firewalld.service

2.2 安装docker-ce

1 安装

# yum install docker-ce -y

2 启动docker并加入开机自启动

# systemctl start docker
# systemctl enable docker

3 查看docker信息

# docker info

2.3 替换 docker 镜像仓库源

# mkdir -p /etc/docker
# tee /etc/docker/daemon.json <<-'EOF'
{
   "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF
# systemctl daemon-reload
# systemctl restart docker

3 DockerFile 构建环境镜像

3.1 构建nginx镜像

1 Dockerfile

FROM centos:7
LABEL maintainer liuchang
RUN yum install -y gcc gcc-c++ make \
   openssl-devel pcre-devel gd-devel \
   iproute net-tools telnet wget curl && \
   yum clean all && \
   rm -rf /var/cache/yum/*
​
RUN groupadd -g 1200 nginx && \
   useradd -M -s /sbin/nologin -u 1200 -g nginx nginx
​
COPY nginx-1.19.1.tar.gz /
RUN tar -zxf nginx-1.19.1.tar.gz && \
   cd nginx-1.19.1 && \
   ./configure --prefix=/usr/local/nginx \
   --with-http_ssl_module \
   --with-http_stub_status_module \
   --user=nginx \
   --group=nginx && \
   make -j 4 && make install && \
   rm -rf /usr/local/nginx/html/* && \
   echo "ok" >> /usr/local/nginx/html/status.html && \
   cd / && rm -rf nginx* && \
   ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
​
ENV PATH $PATH:/usr/local/nginx/sbin
COPY nginx.conf /usr/local/nginx/conf/nginx.conf
WORKDIR /usr/local/nginx
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

2 构建

# docker build -t nginx:v1 .

3.2 构建php镜像

1 Dockerfile

FROM centos:7
LABEL maintainer liuchang
RUN yum install epel-release -y && \
    yum install -y gcc gcc-c++ make gd-devel libxml2-devel \
    libcurl-devel libjpeg-devel libpng-devel openssl-devel \
    libmcrypt-devel libxslt-devel libtidy-devel autoconf \
    iproute net-tools telnet wget curl && \
    yum clean all && \
    rm -rf /var/cache/yum/*

RUN groupadd -g 1200 nginx && \
    useradd -M -s /sbin/nologin -u 1200 -g nginx nginx

COPY php-7.2.19.tar.gz /
RUN tar -zxf php-7.2.19.tar.gz && \
    cd php-7.2.19 && \
    ./configure --prefix=/usr/local/php \
    --with-config-file-path=/usr/local/php/etc \
    --enable-fpm --enable-opcache \
    --with-mysql --with-mysqli --with-pdo-mysql \
    --with-openssl --with-zlib --with-curl --with-gd \
    --with-jpeg-dir --with-png-dir --with-freetype-dir \
    --enable-mbstring --with-mcrypt --enable-hash \
    --with-fpm-user=nginx \
    --with-fpm-group=nginx && \
    make -j 4 && make install && \
    cp -a php.ini-production /usr/local/php/etc/php.ini && \
    cp -a /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf && \
    cp -a /usr/local/php/etc/php-fpm.d/www.conf.default /usr/local/php/etc/php-fpm.d/www.conf && \
    sed -i 's/\;daemonize = yes/daemonize = no/' /usr/local/php/etc/php-fpm.conf && \
    sed -i 's/127.0.0.1:9000/0.0.0.0:9000/' /usr/local/php/etc/php-fpm.d/www.conf && \
    mkdir /usr/local/php/log && \
    cd / && rm -rf php* && \
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

ENV PATH $PATH:/usr/local/php/sbin
COPY php.ini /usr/local/php/etc/
COPY php-fpm.conf /usr/local/php/etc/
COPY www.conf /usr/local/php/etc/php-fpm.d/
WORKDIR /usr/local/php
EXPOSE 9000
CMD ["php-fpm"]

2 构建

# docker build -t php:v1 .

3.3 构建tomcat镜像

1 Dockerfile

FROM centos:7
LABEL maintainer liuchang

RUN yum install wget curl unzip iproute net-tools -y && \
    yum clean all && \
    rm -rf /var/cache/yum/* && \
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

COPY jdk-8u45-linux-x64.tar.gz /
RUN tar -xzf jdk-8u45-linux-x64.tar.gz && \
    mv jdk1.8.0_45 /usr/local/jdk && \
    rm -rf jdk-8u45-linux-x64.tar.gz

COPY apache-tomcat-8.5.43.tar.gz /
RUN tar -zxf apache-tomcat-8.5.43.tar.gz && \
    mv apache-tomcat-8.5.43 /usr/local/tomcat && \
    rm -rf apache-tomcat-8.5.43.tar.gz && \
    rm -rf /usr/local/tomcat/webapps/* && \
    mkdir -p /usr/local/tomcat/webapps/ROOT && \
    echo "ok" > /usr/local/tomcat/webapps/ROOT/status.html

ENV JAVA_HOME /usr/local/jdk
ENV CLASSPATH $JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar
ENV PATH $JAVA_HOME/bin:/usr/local/tomcat/bin:$PATH

COPY catalina.sh /usr/local/tomcat/bin
COPY server.xml /usr/local/tomcat/conf

RUN chmod +x /usr/local/tomcat/bin/catalina.sh

WORKDIR /usr/local/tomcat
EXPOSE 8080
CMD ["catalina.sh", "run"]

2 构建

# docker build -t tomcat:v1 .

3.4 构建jdk镜像

1 Dockerfile

FROM java:8-jdk-alpine
LABEL maintainer liuchang

ENV JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF8 -Duser.timezone=GMT+08"

RUN apk add -U tzdata && \
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

COPY ./target/eureka-service.jar ./

EXPOSE 8888
CMD java -jar $JAVA_OPTS /eureka-service.jar

2 构建

# docker build -t jdk:v1 .

4 使用dockerfile构建的镜像搭建环境

4.1 LNMP环境

1 自定义网络

# docker network create lnmp

2 环境设置

# mkdir -p /app/wwwroot/
# tar -xzf wordpress-5.4.2.tar.gz -C /app/wwwroot
# echo "<?php phpinfo(); ?>" > /app/wwwroot/status.php

3 创建php容器

docker run -d \
--name lnmp_php \
--net lnmp \
--mount type=bind,src=/app/wwwroot/,dst=/usr/local/nginx/html \
php:v1

4 创建nginx容器

docker run -d \
--name lnmp_nginx \
--net lnmp \
-p 888:80 \
--mount type=bind,src=/app/wwwroot/,dst=/usr/local/nginx/html \
nginx:v1

测试php页面状态

http://172.16.1.70:888/status.php

5 创建mysql容器

# mkdir -p /opt/mysql/data
# mkdir -p /opt/mysql/etc
# cp -a my.cnf /opt/mysql/etc

docker run -d \
    -p 3306:3306 --name lnmp_mysql \
    --net lnmp  \
    -v /opt/mysql/data:/var/lib/mysql \
    -v /opt/mysql/etc:/etc/mysql \
    -e MYSQL_ROOT_PASSWORD=123456 \
    mysql:5.7

docker exec -it lnmp_mysql ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

6 登录到mysql创建wp库

# mysql -h 172.16.1.70 -uroot -p123456
MySQL [(none)]> create database wp DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

MySQL [(none)]> select host,user from mysql.user;
+-----------+---------------+
| host      | user          |
+-----------+---------------+
| %         | root          |
| localhost | mysql.session |
| localhost | mysql.sys     |
| localhost | root          |
+-----------+---------------+
4 rows in set (0.00 sec)

7 设置/app/wwwroot/目录的属组属组和php-fpm.conf配置文件中配置的user,group一致

# chown -R 1200.1200 /app/wwwroot/

8 浏览器访问

http://172.16.1.70:888/wordpress

4.2 tomcat环境

1 环境设置

# mkdir -p /opt/tomcat/webapps/ROOT/
# unzip jpress-v3.2.5.war -d /opt/tomcat/webapps/ROOT/ &>/dev/null

2 创建数据库

# mysql -h 172.16.1.70 -uroot -p123456
MySQL [(none)]> create database jpress DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

3 创建tomcat容器

docker run -d \
--name tomcat \
-p 8080:8080 \
-v /opt/tomcat/webapps:/usr/local/tomcat/webapps \
tomcat:v1

4 浏览器访问

http://172.16.1.70:8080/

5 部署gitlab

5.1 部署gitlab

# mkdir -p /opt/gitlib
# cd /opt/gitlib/

docker run -d \
  --name gitlab \
  -p 8443:443 \
  -p 9999:80 \
  -p 9998:22 \
  -v $PWD/config:/etc/gitlab \
  -v $PWD/logs:/var/log/gitlab \
  -v $PWD/data:/var/opt/gitlab \
  -v /etc/localtime:/etc/localtime \
  gitlab/gitlab-ce:latest

访问地址：http://172.16.1.71:9999/

初次会先设置管理员密码 ，然后登陆，默认管理员用户名root，密码就是刚设置的。

5.2 创建项目，提交测试代码

进入后先创建java-demo项目，提交代码，以便后面测试。

# mkdir -p /tools
# cd /tools
# git clone http://172.16.1.71:9999/root/java-demo.git
# unzip tomcat-java-demo-master.zip &>/dev/null
# mv tomcat-java-demo-master/* java-demo/
# git add .
# git config --global user.email "you@example.com"
# git config --global user.name "Your Name"
# git commit -m 'all'
# git push origin master

提示：可以使用/root/.ssh中私钥访问gitlab。

6 部署Harbor镜像仓库

6.1 安装docker-compose

# curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# chmod +x /usr/local/bin/docker-compose

6.2 解压离线包部署

# tar -xzf harbor-offline-installer-v1.9.1.tgz
# mv harbor/ /usr/local/
# cd /usr/local/harbor
# vi harbor.yml
hostname: 172.16.1.71
harbor_admin_password: Harbor12345
# ./prepare
# ./install.sh

# docker-compose -ps
harbor停止和启动命令
# docker-compose stop
# docker-compose start

访问地址：http://172.16.1.71/

harbor 安装成功后默认用户名为admin

6.3 上传tomcat镜像到harbor仓库

1 由于harbor 未配置https，还需要在docker上配置可信任

# cat /etc/docker/daemon.json
{
    "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"],
    "insecure-registries": ["172.16.1.71"]
}
# systemctl daemon-reload
# systemctl restart docker

2 登录 harbor 仓库

# docker login -uadmin -pHarbor12345 172.16.1.71

3 给镜像打标签

# docker tag SOURCE_IMAGE[:TAG] 172.16.1.71/library/IMAGE[:TAG]
# docker tag tomcat:v1 172.16.1.71/library/tomcat:v1

4 上传镜像

# docker push 172.16.1.71/library/IMAGE[:TAG]
# docker push 172.16.1.71/library/tomcat:v1

5 在harbor中查看上传的镜像

7 部署Jenkins

7.1 由于harbor未配置https，还需要在jenkins上配置可信任

# cat /etc/docker/daemon.json
{
    "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"],
    "insecure-registries": ["172.16.1.71"]
}
# systemctl daemon-reload
# systemctl restart docker

7.2 准备JDK和Maven环境

# tar zxf jdk-8u45-linux-x64.tar.gz
# mv jdk1.8.0_45 /usr/local/jdk
# tar zxf apache-maven-3.5.0-bin.tar.gz
# mv apache-maven-3.5.0 /usr/local/maven

docker run -d --name jenkins -p 8080:8080 -p 50000:50000 -u root \
   -v /opt/jenkins_home:/var/jenkins_home \
   -v /var/run/docker.sock:/var/run/docker.sock \
   -v /usr/bin/docker:/usr/bin/docker \
   -v /usr/local/maven:/usr/local/maven \
   -v /usr/local/jdk:/usr/local/jdk \
   -v /etc/localtime:/etc/localtime \
   --name jenkins jenkins/jenkins:lts

访问地址：http://172.16.1.72:8080/

不安装推荐的插件，直接安装，插件后面再装

7.3、修改jenkins下载插件的下载源和maven构建时所需软件包的下载源

1 maven

# vim /usr/local/maven/conf/settings.xml +158
    <mirror>
      <id>central</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>

2 jenkins

# cd /opt/jenkins_home/updates
# sed -i.bak 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' default.json && \
sed -i.bak 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json
# docker restart jenkins

7.4 安装插件

系统管理-->插件管理-->Installed

搜索 git/pipeline，点击安装。

8 发布测试

8.1 创建一个流水线任务

8.2 添加gitlab和harbor认证凭据

1、添加拉取git代码凭据，并获取id替换到pipeline脚本中的git_auth变量值。

2、添加拉取harbor镜像凭据，并获取id替换到pipeline脚本中docker_registry_auth变量值。

8.3 添加参数化构建

This project is parameterized -> String Parameter

Name：Branch # 变量名，下面脚本中调用

Default Value：master # 默认分支

Description：发布的代码分支 # 描述

8.4 Pipeline脚本

#!/usr/bin/env groovy

def registry = "172.16.1.71"
def project = "library"
def app_name = "tomcat"
def image_name = "${registry}/${project}/${app_name}:${Branch}-${BUILD_NUMBER}"
def git_address = "http://172.16.1.71:9999/root/java-demo.git"
def docker_registry_auth = "3aadf18f-edca-4d22-a42b-56013fccb681"
def git_auth = "6b7d1b31-830e-416f-9bd6-e6c9e3c1df4b"

pipeline {
    agent any
    stages {
        stage('拉取代码'){
            steps {
              checkout([$class: 'GitSCM', branches: [[name: '${Branch}']], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
            }
        }

        stage('代码编译'){
           steps {
             sh """
                JAVA_HOME=/usr/local/jdk
                PATH=$JAVA_HOME/bin:/usr/local/maven/bin:$PATH
                mvn clean package -Dmaven.test.skip=true
                """
           }
        }

        stage('构建镜像'){
           steps {
                withCredentials([usernamePassword(credentialsId: "${docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
                sh """
                  echo '
                    FROM ${registry}/library/tomcat:v1
                    LABEL maitainer liuchang
                    RUN rm -rf /usr/local/tomcat/webapps/*
                    ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
                  ' > Dockerfile
                  docker build -t ${image_name} .
                  docker login -u ${username} -p '${password}' ${registry}
                  docker push ${image_name}
                """
                }
           }
        }

        stage('部署到Docker'){
           steps {
              sh """
              REPOSITORY=${image_name}
              docker rm -f tomcat-java-demo |true
              docker container run -d --name tomcat-java-demo -p 88:8080 ${image_name}
              """
            }
        }
    }
}

8.5 构建

1、构建流程图

2、代码构建时的工作空间

# ls /opt/jenkins_home/workspace/item-java
db  Dockerfile  LICENSE  pom.xml  README.md  src  target
# ls /opt/jenkins_home/workspace/item-java/target/
classes  generated-sources  ly-simple-tomcat-0.0.1-SNAPSHOT  ly-simple-tomcat-0.0.1-SNAPSHOT.war  maven-archiver  maven-status

3、查看镜像仓库

4、查看部署的docker容器是否运行

# docker images
REPOSITORY                   TAG                 IMAGE ID            CREATED             SIZE
172.16.1.71/library/tomcat   master-5            b2a8f33a5b69        3 minutes ago       784MB
172.16.1.71/library/tomcat   v1                  ca2e1ee6962b        3 days ago          765MB
# docker ps

5、访问网站

访问地址：http://172.16.1.72:88/

9 Prometheus+Grafana 监控 Docker

9.1 部署 prometheus

# mkdir -p /opt/prometheus/
# docker run -d \
  --name prometheus \
  -p 9090:9090 \
  -v /opt/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \
  prom/prometheus

9.2 部署 grafana

docker run -d \
  --name grafana \
  -p 3000:3000 \
  grafana/grafana

监控Docker主机模板：https://grafana.com/dashboards/193

9.3 部署 cadvisor

docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8090:8080 \
  --detach=true \
  --name cadvisor \
  --privileged \
  --device=/dev/kmsg \
  google/cadvisor:latest

9.4 展示

1 登录grafana

2 效果图

10 CI流程

1、拉取代码
2、代码编译（java项目），产出war包
3、打包项目镜像并推送到镜像仓库
4、部署镜像测试

AUTHOR：刘畅