实战项目部署应用到kubernetes流程(jenkins+docker+k8s)
说明
通过jenkins构建java应用程序发布到k8s集群中
本文已一个大数据的java项目来演示构建部署过程
支持发布和回滚
支持一套模板应用不同项目
k8s基础准备
创建项目名称空间
[root@k8s-master ~]# kubectl create ns architect
namespace/architect created [root@k8s-master ~]# kubectl label namespace architect istio-injection=enabled
namespace/architect labeled
[root@k8s-master ~]# kubectl get namespace -L istio-injection
NAME STATUS AGE ISTIO-INJECTION
architect Active 87s enabled
default Active 39d enabled
istio-system Active 7d17h
kube-public Active 39d
kube-system Active 39d
[root@k8s-master ~]#
创建全局配置文件
＃创建大数据配置文件
[root@k8s-master ~]# ls bigdata-counter
core-site.xml dmp_operator1.keytab hbase-site.xml hdfs-site.xml krb5.conf
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl create configmap bigdata-config --from-file=/root/bigdata-counter/ -n architect
configmap/bigdata-config created
[root@k8s-master ~]#
#全局环境变量
[root@k8s-master ~]# cat yaml/global-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: global-config
namespace: architect
data:
CONFIG_SERVER_URL: http://config.ms.xx.com
CONFIG_SERVER_USER: myconfig
ILOG_BOOTSTRAP_SERVERS: bootstrap.servers=192.168.1.122:9092
CONFIG_SERVER_PWDS: mc123456 [root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f yaml/global-config.yaml
configmap/global-config created
[root@k8s-master ~]# kubectl get configmap -n architect
NAME DATA AGE
bigdata-config 4 2m33s
global-config 4 3s
[root@k8s-master ~]#
创建habror认证信息
[root@k8s-master ~]# kubectl create secret docker-registry registry-secret --docker-server=dev-hub.xx.net --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@dev-hub.xx.net -n architect
secret/registry-secret created
[root@k8s-master ~]# kubectl get secret -n architect
NAME TYPE DATA AGE
default-token-5wsxl kubernetes.io/service-account-token 3 2m24s
istio.default istio.io/key-and-cert 3 2m24s
registry-secret kubernetes.io/dockerconfigjson 1 10s
项目deployment文件
可以做成模本，通过变量替换形式．后面有讲到
[root@k8s-master ~]# cat bigdata-interface-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: bigdata-interface
version: default
name: bigdata-interface
namespace: architect
spec:
replicas: 1
selector:
matchLabels:
app: bigdata-interface
version: default
template:
metadata:
labels:
app: bigdata-interface
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: bigdata-interface
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b63
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: bigdata-interface
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
[root@k8s-master ~]#
jenkins pipeline 模板文件
node {
//项目名称
String APPNAME='bigdata-interface'
//名称空间
String NAMESPACE='architect'
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(name:'GIT_BRANCH_TAG', defaultValue: 'master',description:'', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.GIT_BRANCH_TAG]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git']]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
进一步改造pipeline
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git', description: 'git仓库地址', name: 'XBUILD_REPO', trim: false),
string(defaultValue: 'master', description: '版本名称', name: 'XBUILD_BRANCHE', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: params.XBUILD_REPO]]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins上创建自由风格项目
效果如下 支持参数化构建。但每项目单独JOB
查看镜像仓库已经推送正常
项目回滚
node {
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins效果如下：输入namespace和deployment 名称和上次构建版本
查看k8s集群
可以灵活的在已知版本回滚操作
[root@k8s-master ~]# kubectl get deploy -n architect -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
bigdata-interface 0/1 1 0 3h12m bigdata-interface dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b1 app=bigdata-interface,version=default
[root@k8s-master ~]# kubectl get pods -n architect
NAME READY STATUS RESTARTS AGE
bigdata-interface-6bf7b5646d-sf9j5 1/2 Running 0 24s
[root@k8s-master ~]#
通过以上实践，可以将deployment做成模板
deployment模板文件
-bash-4.2$ cat k8s-deployment.tpl
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: {APPNAME}
version: default
name: {APPNAME}
namespace: {NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
app: {APPNAME}
version: default
template:
metadata:
labels:
app: {APPNAME}
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: {APPNAME}
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: {HarborUrl}/{NAMESPACE}/{APPNAME}:{NAMESPACE}_b{BUILD_NUMBER}
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: {APPNAME}
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
-bash-4.2$
通过模板动态生成项目的deploy文件．进行升级回滚．
比如下面的：
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
command="sed -e 's#{APPNAME}#${APPNAME}#g;s#{NAMESPACE}#${NAMESPACE}#g;s#{HarborUrl}#${HarborUrl}#g;s#{BUILD_NUMBER}#${BUILD_NUMBER}#g' /data/jenkins_slave/k8s-deployment.tpl > /data/jenkins_slave/${APPNAME}-deploy.yaml"
sh "${command}"
sh "scp -P 22 /data/jenkins_slave/${APPNAME}-deploy.yaml ${KubernetHost}:/root/ "
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
sh "ssh -p 22 ${KubernetHost} kubectl apply -f /root/${APPNAME}-deploy.yaml"
//test
}
}
pipeline参数化构建
示例：
pipeline {
agent any
parameters {
gitParameter branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'XBUILD_BRANCHE', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG'
string defaultValue: 'git@git.xx.net:company/xx-xx.git', description: '', name: 'XBUILD_REPO', trim: false
}
stages {
stage('Example') {
steps {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bb', url: params.XBUILD_REPO]]])
}
}
}
}
构建效果如下：
实战项目部署应用到kubernetes流程(jenkins+docker+k8s)的更多相关文章
- docker 实战-项目部署
前面学习了 docker 的命令和实际的用法,现在来实战一下.编排一个服务,也就是项目部署. 目前我们在一个闭源环境下工作,介绍一些工作流程和部署项目的方法. 该工作流程比较特殊 所谓闭源环境就是 没 ...
- jenkins+docker+k8s项目发布
目录 一.简介 二.新建docker-build工程 三.项目部署 四.访问测试 一.简介 1.该章节基于jenkins.Harbor.pipeline.k8s来做发布,如对这些不熟悉,请按以下进入学 ...
- .Net 6 WebApi 项目部署到 Linux 系统上的 Docker 容器
1.创建一个基础的WebApi项目 注意:因为发布时候,Dockerfile文件必须和解决方案.cspro文件放在同级,所以建议勾上这个,当时遇到这个问题,导致打包镜像时找不到.cspro文件,搞了好 ...
- 第四十四章 微服务CICD（6）- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- kubernetes下jenkins实战maven项目编译构建
关于kubernetes环境的jenkins集群 在kubernetes环境部署的jenkins集群,执行任务时会新建pod,任务完成后pod被销毁,架构如下所示: 在kubernetes搭建jenk ...
- Kubernetes笔记（三）：Gitlab+Jenkins Pipeline+Docker+k8s+Helm自动化部署实践（干货分享！）
通过前面两篇文章,我们已经有了一个"嗷嗷待哺"的K8s集群环境,也对相关的概念与组件有了一个基本了解(前期对概念有个印象即可,因为只有实践了才能对其有深入理解,所谓"纸上 ...
- 【k8s实战一】Jenkins 部署应用到 Kubernetes
[k8s实战一]Jenkins 部署应用到 Kubernetes 01 本文主旨 目标是演示整个Jenkins从源码构建镜像到部署镜像到Kubernetes集群过程. 为了简化流程与容易重现文中效果, ...
- kubernetes部署jenkins(Docker in Docker)及认证
引言 Jenkins是一款开源 CI&CD 软件,用于自动化各种任务,包括构建.测试和部署软件. 本文将Jenkins的master与slave置于Pod中,部署在namespace:jenk ...
随机推荐
- MySQL5.7.29 和 Navicat ===> windows窗口式按装和使用
MySQL windows窗口式按装下载方法:官网: https://www.mysql.com/ ==> DOWNLOADS ==> MySQL Community (GPL) Down ...
- C语言：贪心算法之装箱问题
#include <stdio.h> #include <stdlib.h> #define N 6 #define V 100 typedef struct box // 使 ...
- Redis基本数据结构之ZSet
1.1Zset(有序集合) Zset保留了集合不能有重复成员的特性,但不同的是,有序集合中的元素可以排序.但是它和列表使用索引下标作为排序依据不同的是,它给每个元素设置一个分数(score)作为排序的 ...
- 100道Java高频面试题(阿里面试官整理)
我分享文章的时候,有个读者回复说他去年就关注了我的微信公众号,打算看完我的所有文章,然后去面试,结果我后来很长时间不更新了...所以为了弥补一直等我的娃儿们,给大家的金三银四准备了100道花时间准备的 ...
- Docker 一次性进程与对话进程
目录 一次性进程 对话进程 退出的方法 参考 Docker在运行程序的时候,需要区分运行的程序是一次性进程还是对话进程,不同的进程操作方式有差异. 一次性进程 一些简单进程是不需要交互的,比如hell ...
- FakeTaobaoDeepLink - 复制淘宝deeplink来拦截淘宝广告的自动拉起
Fake Taobao Deeplink 复制 ** com.taobao.tao.welcome.Welcome ** 的intent-filter来拦截误触广告后自动拉起淘宝app 完整工程 Gi ...
- 微信小程序input框去除空格
//去除空格 <input type='text' data-name='account' value='{{account}}' name="account" bindin ...
- java内存区域的划分
前言 之前我们探讨过一个.class文件是如何被加载到jvm中的.但是jvm内又是如何划分内存的呢?这个内被加载到了那一块内存中?jvm内存划分也是面试当中必被问到的一个面试题. 什么是JVM内存区域 ...
- POJ-1502(基本dijikstra算法)
MPI Maelstrom POJ-1502 这题是求最短路,但是因为一开始看错题目,导致我去使用prime算法求最小生成树 题意是指一台机器发出信息后,还可以向其他的机器发送信息,所以不能使用pri ...
- Pytorch1.7报错 Output 0 of UnbindBackward is a view and is being modified inplace
utils里内容改成 if scale_each is True: for idx, _ in enumerate([jj for jj in tensor]): t = tensor[idx] # ...