Docker安装GitLab与Runner（网关），常规设置，自动化用到k8s+token

[转]图文详解k8s自动化持续集成之GitLab CI/CD

Windows里面使用Debian命令行工具完成

和Docker网络相关的命令

查看某一个容器的网络
docker inspect 容器ID

查看docker当前网络
docker network ls

首先创建一个网络（下面通过docker-compose的yml文件创建，这里只做命令的了解）
docker network create gitlab-network

将现有的容器连到创建的网络中（每个容器都要连到这个网络里）
docker network connect gitlab-network gitlab-server   #容器名
docker network connect gitlab-network gitlab-runner

查看网络内的容器信息
docker network inspect gitlab-network

也可以在运行时指定镜像
docker run --network gitlab-network 镜像名称

移除指定的网络
docker network rm gitlab-network

先删掉下面要用的bridge名称

---

使用Docker Compose安装GitLab-ce和GitLab-runner

Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要参考

Win10下使用Docker搭建Gitlab CI自动构建平台

docker rm & docker rmi & docker prune 的差异

• docker rm ： 删除一个或多个 容器
• docker rmi :  删除一个或多个 镜像
• docker prune： 用来删除不再使用的 docker 对象

---

docker+gitlab+gitlab-runner部署(CentOs7)重要参考

Docker Compose yml for Gitlab and Gitlab Runner

---

SMTP settings(备用参考)

#vim /etc/gitlab/gitlab.rb

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "developers@aaa.net"
gitlab_rails['smtp_password'] = "Del43@\#$@1"
gitlab_rails['smtp_domain'] = "exmail.qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true
gitlab_rails['gitlab_email_from'] = "developers@aaa.net"

测试邮件

#gitlab-ctl reconfigure
#gitlab-rails console
#Notify.test_email('315360007@qq.com', 'Message Subject', 'Message Body').deliver_now

配置本地windows的host文件：

127.0.0.1 gitlab-server
127.0.0.1 nexus3

浏览器访问：http://gitlab-server:9080/  或者 http://ip:9080/
用户名：root，密码：自己设置

用root用户登录

---

创建测试项目：mvcdockerrunner1

添加SSH密钥

本地windows需要在当前登录用户的目录里面找到.ssh目录下面的****.pub文件添加公钥，同时设置config文件

Host gitlab-server
HostName gitlab-server
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/315360007
Port 23

.ssh/config

克隆代码

---

手动设置激活的Runner(很重要)

1、注册gitlab-runner（按照提示一步一步做），熟悉了直接用下面第二种

docker exec -it gitlab-runner gitlab-runner register

2. 我们会输入 http://gitlab.local.net:9080 或 http://ip:port 也就是我们安装在本地的GitLab

3. Please enter the gitlab-ci token for this runner 要求输入 gitlab-ci token

在项目的 管理区域->runners中可以找到(这里注册的是share类型runner)

4. 输入描述,如：gitlab-runner

5. 输入tag（留空也可以，之后可以进行编辑 ）

6. 选择当遇到没有打标签的提交时是否会执行，我们选 true

7. 是否锁定此runner 到当前项目， 我们选 false

8. 选一个执行者 executor

这一步比较重要 (ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell)

我们选docker

9. 选择默认使用的镜像： docker:stable

2、注册gitlab-runner（直接传参数一次完成）

# 手动设置激活的Runner
docker exec -it gitlab-runner gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image docker:stable \
  --url "http://gitlab-server:9080/" \
  --registration-token "K6BiurzDBuzx23a-mV-f" \
  --description "gitlab-runner" \
  --tag-list "docker,company" \
  --run-untagged="true" \
  --locked="false" \
  --docker-privileged="true" \
  --docker-extra-hosts "gitlab-server:172.20.0.1" \
  --docker-extra-hosts "nexus3:172.20.0.1"

#解释很重要
#"docker:stable"这里之所以这样是因为基于这个镜像它包含了docker等工具，可以在gitlab-runner执行的.gitlab-ci.yml中有docker指令时
#而无需再安装docker 参考地址：https://docs.gitlab.com/ee/ci/docker/using_docker_build.html

#privileged=true #使用docker-in-docker时通常为true
#extra-hosts     #设置gitlab-server的网关地址

进入 gitlab-runner的容器里

 docker exec -it runner-container-id /bin/bash

查看/etc/gitlab-runner/config.toml文件

cat /etc/gitlab-runner/config.toml
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runner"
  url = "http://gitlab-server:9080/"
  token = "mWpRt6ry7HimmAyt86T1"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    extra_hosts = ["gitlab-server:172.20.0.1"]
    shm_size = 0

/etc/gitlab-runner/config.toml

五、提交项目代码完成CI/CD

Gitlab CI/CD 打包过程报 “ERROR: Uploading artifacts to coordinator... too large” 如何解决

Uploading artifacts for successful job
00:13
Uploading artifacts...
mevdockerrunner1.tar: found 1 matching files and directories
ERROR: Uploading artifacts as "archive" to coordinator... too large archive  id=34 responseStatus=413 Request Entity Too Large status=413 token=fhcnwRyE
FATAL: too large  

GitLab管理中心里设置CI/CD持续集成和部署里最大产物大小

---

设置和使用Gitlab-ci变量连接Kubernetes：

 1 .kube_deploy:
 2   image: boxboat/kubectl:1.17.4
 3   variables:
 4     HELM_TOKEN: ${HELM_USER_TOKEN}
 5     K8S_API_SERVER: ${KUBE_API_ADDRESS}
 6     K8S_API_SERVER_CERT: ${KUBE_API_CA}
 7   before_script:
 8     - mkdir -p /home/alpine/.kube
 9     - cp .ci/kube-config-template /home/alpine/.kube/config
10     - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_TOKEN}/g" /home/alpine/.kube/config
11     - sed -i "s/{{ KUBE_API_ADDRESS }}/${K8S_API_SERVER}/g" /home/alpine/.kube/config
12     - sed -i "s/{{ KUBE_API_CA }}/${K8S_API_SERVER_CERT}/g" /home/alpine/.kube/config

.gitlab-ci.yml

设置Kubernetes的config文件，kube-config-template文件内容：

 1 apiVersion: v1
 2 clusters:
 3 - cluster:
 4     certificate-authority-data: {{ KUBE_API_CA }}
 5     server: https://{{ KUBE_API_ADDRESS }}
 6   name: helm
 7 contexts:
 8 - context:
 9     cluster: helm
10     namespace: meshop-dev
11     user: helm
12   name: helm
13 current-context: "helm"
14 kind: Config
15 preferences: {}
16 users:
17 - name: helm
18   user:
19     token: {{ HELM_USER_TOKEN }}

.ci/kube-config-template

获取KUBE_API_CA和HELM_USER_TOKEN的值

#kubectl get sa -A   #查看所有security account
#kubectl get sa/helm -n kube-system -o yaml #查看在某一个命名空间下的某一个security account=helm的yaml，得到secrets的name
#kubectl get secrets/helm-token-wstgs  -o yaml -n kube-system #查看在某一个命名空间下的某一个secrets的name的yaml，得到token值和ca.crt值

使用变量

设置默认分支：

设置仓库不允许推送代码，只能自己创建分支提交，在服务器上合并分支

git流水线克隆代码

流水线触发器的设置，添加一个token

设置sourceTree自定义操作，设置token=上面的值

1 curl -X POST -F token=f21cbb2d6f8c10cad915a380c49b99 -F 'ref=dev' -F 'variables[MESHOP_BUILD_ONLY]=sso' https://git.runshopstore.com/api/v4/projects/2/trigger/pipeline

sso_restore.sh

设置ci里面使用触发器

1 workflow:
2   rules:
3     - if: $MESHOP_BUILD_ONLY != null
4       when: always
5     - if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/
6       when: always

.gitlab-ci.yml

合并请求必须保证编译通过：同时删除合并请求的分支

设置关闭仓库，测试可以使用的项目：

设置标记：

规范化git commit信息

团队敏捷实践 —— 使用 semantic-release 自动管理发布版本

单个项目最简单的自动化脚本（Config项目）

stages:
  - restore
  - compile
  - version
  - build
  - deploy

.load_environment_variables: &load_environment_variables
  - echo "init environment variables."
  - for environment in $(ls -d -1 .ci/environments/*.sh);
    do source $environment;
    done

default:
  image: docker.tidebuy.net/dotnet/sdk:5.0
  tags:
    - docker
    - company
  before_script:
    - *load_environment_variables

variables:
  # docker in docker
  DOCKER_DRIVER: "overlay2"
  DOCKER_HOST: tcp://localhost:2375
  # cache
  NUGET_PACKAGES_DIRECTORY: ".nuget"
  OBJECTS_DIRECTORY: "obj"
  SOURCE_CODE_PATH: "*/*/"

.nuget_cache:
  cache:
    key: "${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}"
    paths:
      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/project.assets.json"
      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/*.csproj.nuget.*"
      - "$NUGET_PACKAGES_DIRECTORY"

restore:
  stage: restore
  extends: .nuget_cache
  cache:
    policy: pull-push
  script:
    - echo "dotnet restore..."
    - dotnet restore
      $CI_PROJECT_DIR/MeShop.Config/MeShop.Config.sln
      --packages $NUGET_PACKAGES_DIRECTORY
      --runtime linux-x64
      --configfile $CI_PROJECT_DIR/.ci/nuget.config
  only:
    refs:
      - branches
      - merge_requests

compile:
  stage: compile
  before_script:
    - *load_environment_variables
  extends:
    - .nuget_cache
  cache:
    policy: pull
  script:
    - echo "compile project."
    - dotnet publish MeShop.Config/MeShop.View.Config/MeShop.View.Config.csproj --no-restore
      --runtime linux-x64 -c Release -o ${CI_PROJECT_DIR}/publish
  artifacts:
    name: "${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}"
    paths:
      - publish/
    expire_in: 1 days
  only:
    refs:
      - branches
      - merge_requests

version:
  stage: version
  image: meshop/semantic-release:17.1.1
  script:
    - if [ $CI_COMMIT_REF_NAME == 'alpha' -o $CI_COMMIT_REF_NAME  == 'master' ];
      then
      semantic-release;
      fi
    - if [ ! -f ./.version ];
      then
      echo "VERSION=${MESHOP_BUILD_VERSION:-$(date +%Y%m%d%H%M%S)}">.version;
      fi
    - cat .version
  artifacts:
    paths:
      - .version
    expire_in: 1 days
  needs:
    - job: compile
      artifacts: false
  rules:
    - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME == 'alpha'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  == 'master'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"
      when: on_success

build:
  stage: build
  image: "docker:stable"
  services:
    - docker:stable-dind
  variables:
    DOCKER_TLS_CERTDIR: ""
  script:
    - echo "Logging to GitLab Container Registry with CI credentials..."
    - echo "$MESHOP_BUILD_REGISTRY_PASSWORD" | docker login -u "$MESHOP_BUILD_REGISTRY_USER" --password-stdin "$MESHOP_BUILD_REGISTRY"
    - source ./.version
    - image_version="${VERSION}"
    - image_name="${MESHOP_BUILD_REGISTRY}/meshop/shop/config"
    - docker build -f .ci/Dockerfile
      --tag "$image_name:$image_version"
      .
    - docker push "$image_name:$image_version"
    - if [ $MESHOP_BUILD_LATEST = 'true' ];
      then
      docker tag $image_name:$image_version $image_name:latest &&
      docker push "$image_name:latest" ;
      fi
  needs:
    - job: version
      artifacts: true
    - job: compile
      artifacts: true
  rules:
    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME == 'alpha'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  == 'master'"
      when: on_success
    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"
      when: on_success

deploy:
  stage: deploy
  image: boxboat/kubectl:1.17.4
  script:
    - mkdir -p /home/alpine/.kube
    - cp .ci/kube-config-template /home/alpine/.kube/config
    - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_USER_TOKEN}/g" /home/alpine/.kube/config
    - sed -i "s/{{ KUBE_API_ADDRESS }}/${KUBE_API_ADDRESS}/g" /home/alpine/.kube/config
    - sed -i "s/{{ KUBE_API_CA }}/${KUBE_API_CA}/g" /home/alpine/.kube/config
    - kubectl rollout restart deployment/meshop-config
  rules:
    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"
      when: never
    - if: "$CI_COMMIT_REF_NAME == 'dev'"
      when: on_success

.gitlab-ci.yml