这些年我们一起搞过的持续集成~Jenkins+Perl and Shell script

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

部门用持续集成已经很久了,但其实使用起来还是很麻烦的,每当要给一个新项目set up持续集成的环境,虽然是Copy一些现有的jobs, 但是许多参数,变量需要去改,然后还有调试,少说3,4天搞一下,非常不方便。

最近比较空,就把现有的持续集成系统升级改造下,job用一套模板,全部参数化,只要修改配置文件,就可以为新项目配置好环境。

本文的重点是一些经验,想法的分享,并不是一篇手把手教你搭建持续集成环境的教程,而且阅读本文需要一定的Jenkins, perl, shell脚本的基础知识, 所用到的知识有:Jenkins+Maven+Git+Sonar+Perl+Shell script

下图很好的说明了我们现有的Job流, 以往是每个项目都有一套这样的job流,现在我改成这个Template_service job流,供所有的项目使用,项目名也是作为参数传递进来的,每个job都是用Jenkins Execute shell这个组件来实现功能的,像check out code,Email Notification, Publish Junit test results, code coverage report 等Jenkins自带组件都没有使用。

运行环境:  apache-maven-3.0.5, jdk1.7.0_15, git 1.8.2.1, SonarQube 4.0, perl v5.8.8

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

接下来简单的说明一下各个Job的功能,以及贴出详细的脚本供大家参考。

除了上图里面的job, 还有一个Config job, export一些环境变量

0)Template_Service_Config (one time run)

用来Export 环境变量的,比如Project list, project git 地址,

export PROJECT_LIST=PROJECTA,PROJECTB
export PROJECTA_GIT_PROJECT_URL=ssh://git@stash.xxx.com:7999/Project/ProjectA.git
export PROJECTA_BRANCH=development
export PROJECTA_SERVICE_NAME=China_Template_Service
export PROJECTA_EMAIL_LIST=wadexu@xxx.com,xxx@xxx.com
export PROJECTA_EMAIL_CC_LIST=wadexu@xxx.com
......
export PROJECTB_GIT_PROJECT_URL=ssh://git@stash.xxx.com:7999/abc/ProjectB.git
export PROJECTB_BRANCH=perf
export PROJECTB_SERVICE_NAME=China_ProjectB_DEV
export PROJECTB_EMAIL_LIST=wadexu@xxx.com,xxx@xxx.com
export PROJECTB_EMAIL_CC_LIST=wadexu@xxx.com
......

Build 这个job 的时候需要Parameters

代码如下:主要作用就是将输入的Parameters写入到一个文件,以便后续所有job 一上来source一下

TEST=${WORKSPACE}/test
CONFIG_FILE=Project_Config.txt rm -rf ${TEST}
git clone ssh://git@stash.xxx.com:7999/project/test.git
cd ${TEST} cd Config
if [ ! -f "${CONFIG_FILE}" ]; then
touch "${CONFIG_FILE}"
fi ##########Rewrite the config file base on the project list###########
CURRENT_TIME=`date +"%m/%d/%Y/%T"`
echo -e "export CONFIG_TIME=${CURRENT_TIME}\n
export PROJECT_LIST=${PROJECT_LIST}\n
export SHARED_FOLDER=~/jenkins/common\n" > ${CONFIG_FILE} ###########List the project1 parameters###########
echo -e "${PROJECTA_PARAMETERS}" >> ${CONFIG_FILE} ###########List the project2 parameters###########
echo -e "${PROJECTB_PARAMETERS}" >> ${CONFIG_FILE} git add ${CONFIG_FILE}
git commit -m "config file"
git push origin master cp ${CONFIG_FILE} ~/jenkins/common/Config
cd ..
cp Scripts/* ~/jenkins/common/Scripts
chmod 755 ~/jenkins/common/Scripts/*

还有就是将一些perl写的脚本从git clone下来放到某一个公共目录下,以便后续job使用, 有如下脚本,比如发送email, Unit test analyzer

部分脚本像gridService, packageService是操作公司一个网格云计算平台,安装rpm包,重启service用的,不方便贴出来。

[wadexu@vm10226 common]$ cd Scripts/
[wadexu@vm10226 Scripts]$ ll
total
-rwxr-xr-x wadexu wadexu Mar : gridService
-rwxr-xr-x wadexu wadexu Mar : packageService
-rwxr-xr-x wadexu wadexu Mar : queryDB.pl
-rwxr-xr-x wadexu wadexu Mar : Reporter.jar
-rwxr-xr-x wadexu wadexu Mar : report.pl
-rwxr-xr-x wadexu wadexu Mar : sendEmail.pl
-rwxr-xr-x wadexu wadexu Mar : ServiceInfoProvider.pl
-rwxr-xr-x wadexu wadexu Mar : UnitTestAnalyzer.pl

1)Template_Service_Initial

遍历Project list 列出的项目, 使用curl 调用下一个job (Commit_Auto_Build) 的Rest API, Build job with parameter接口

每个项目的Job流都是通过Project name + Initial job的Build number 来贯穿的, 这点很重要,多项目一起运行时各项目之间不会错乱,比如写Log文件,分析Log文件生成report

rm -rf ${WORKSPACE}/*

####### Import config file #######
cd ~/jenkins/common/Config
. Project_Config.txt
cd ~/jenkins/common/Temp ####### Get the Project list and traverse the list ######
id=1
list_size=`echo $PROJECT_LIST | awk -F "," '{print NF;}'`
let list_size=list_size+1
while [ $id -lt $list_size ]
do
projectName=`echo "${PROJECT_LIST}"|awk -v id=$id -F "," '{print $id}'`
if [ "$projectName " != ' ' ]; then
curl -X POST --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Commit_Auto_Build/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'$projectName'"},{"name":"PIPELINE_NUM","value":"'$BUILD_NUMBER'"}]}'
id=`expr $id + 1`
else
break
fi
done

2)Template_Service_Commit_Auto_Build

这个Job的作用是用来比较代码版本有没有更新,如果有更新(或者是第一次run这个job)就会调用后续的-- Build job

还有就是列出这次Build和上次Build之间改动过代码的作者。

如果将Initial job 设置成每几分钟run一下,其实这个job就起到了 每当有人提交代码,则自动Build的作用了。

### Import config file ###
cd ~/jenkins/common/Config
. Project_Config.txt ### Initial parameters ###
GIT_PROJECT_URL=${PROJECT_NAME}"_GIT_PROJECT_URL"
GIT_PROJECT_URL=$(eval echo \${$GIT_PROJECT_URL}) BRANCH=${PROJECT_NAME}"_BRANCH"
BRANCH=$(eval echo \${$BRANCH}) LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG_FOLDER=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME mkdir $BUILD_LOG_FOLDER GIT_PROJECT_NAME=${PROJECT_NAME}"_GIT_PROJECT_NAME"
GIT_PROJECT_NAME=$(eval echo \${$GIT_PROJECT_NAME}) #### setup ####
export CURRENT_RESOURCE_DIR=${WORKSPACE}/${GIT_PROJECT_NAME}
export CURRENT_MD5_FILE=${WORKSPACE}/${PROJECT_NAME}"_Code_Current"
export BASE_MD5_FILE=${WORKSPACE}/${PROJECT_NAME}"_Code_Base"
export CURRENT_AUTHOR_LIST=${WORKSPACE}/${PROJECT_NAME}"_Author_Current"
export BASE_AUTHOR_LIST=${WORKSPACE}/${PROJECT_NAME}"_Author_Base" #### cd resource,get log commit MD5 ####
cd ${WORKSPACE}
rm -rf ${CURRENT_RESOURCE_DIR}
git clone ${GIT_PROJECT_URL}
cd ${CURRENT_RESOURCE_DIR}
git checkout -b ci origin/${BRANCH} if [ ! -f "${BASE_AUTHOR_LIST}" ]; then
touch "${BASE_AUTHOR_LIST}"
fi if [ ! -f "${BASE_MD5_FILE}" ]; then
touch "${BASE_MD5_FILE}" ########call build job######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Build/build --data json='{"parameter":[{"name":"PROJECT_NAME","value":"'${PROJECT_NAME}'"},{"name":"PIPELINE_NUM","value":"'${PIPELINE_NUM}'"}]}' git log --pretty=format:"Author: %an" > ${BASE_AUTHOR_LIST}
sort -u ${BASE_AUTHOR_LIST} > ${BUILD_LOG} git log --max-count= --pretty=format:"%H" > ${BASE_MD5_FILE}
exit
else
git log --max-count= --pretty=format:"%H" > ${CURRENT_MD5_FILE}
git log --pretty=format:"Author: %an" > ${CURRENT_AUTHOR_LIST}
fi #### compare the md5 file, and run the job.####
## -a means && ##
if cmp -s ${CURRENT_MD5_FILE} ${BASE_MD5_FILE}
then
echo "same,no changes!"
else
mv ${CURRENT_MD5_FILE} ${BASE_MD5_FILE}
diff -b $CURRENT_AUTHOR_LIST $BASE_AUTHOR_LIST | grep "<" | sed 's/^< //g' > Update_Author.txt
mv ${CURRENT_AUTHOR_LIST} ${BASE_AUTHOR_LIST}
sort -u Update_Author.txt > ${BUILD_LOG} ########call build job######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_service_Build/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'${PROJECT_NAME}'"},{"name":"PIPELINE_NUM","value":"'${PIPELINE_NUM}'"}]}'
fi

这里用到了--pretty=format 这个参数, %H就是打印出哈希字串, %an是打印出Author, --max-count=1 取最近的一条log 记录

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

3) Template_Service_Build

这个job是最核心的job, 执行rpm build, 将rpm包丢到公司内部网格云计算平台的Repository下, 分析单元测试结果, 运行Sonar, 从Sonar的首页上拿到这个项目的一些信息,比如Sonar上 blocker, critical, major 等issues, 还有code coverage,全部打印到log里, 以便后续的脚本生成report, 另外就是Sonar issue, code coverage 不达标,则发送相应的email 给相应的project email list

rm -rf ${WORKSPACE}/*

###### Import config file ######
cd ~/jenkins/common/Config
. Project_Config.txt ###### Initial parameters ######
GIT_PROJECT_NAME=${PROJECT_NAME}"_GIT_PROJECT_NAME"
GIT_PROJECT_NAME=$(eval echo \${$GIT_PROJECT_NAME}) LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME BRANCH=${PROJECT_NAME}"_BRANCH"
BRANCH=$(eval echo \${$BRANCH}) GIT_PROJECT_URL=${PROJECT_NAME}"_GIT_PROJECT_URL"
GIT_PROJECT_URL=$(eval echo \${$GIT_PROJECT_URL}) REPOSITORY=${PROJECT_NAME}"_REPOSITORY"
REPOSITORY=$(eval echo \${$REPOSITORY}) INSTANCES=${PROJECT_NAME}"_INSTANCES"
INSTANCES=$(eval echo \${$INSTANCES}) COVERAGE_URL=${PROJECT_NAME}"_COVERAGE_URL"
COVERAGE_URL=$(eval echo \${$COVERAGE_URL}) SONAR_MIN_COVERAGE=${PROJECT_NAME}"_SONAR_MIN_COVERAGE"
SONAR_MIN_COVERAGE=$(eval echo \${$SONAR_MIN_COVERAGE}) MAX_MAJOR_ISSUE_NUM=${PROJECT_NAME}"_MAX_MAJOR_ISSUE_NUM"
MAX_MAJOR_ISSUE_NUM=$(eval echo \${$MAX_MAJOR_ISSUE_NUM}) MAX_CRITICAL_ISSUE_NUM=${PROJECT_NAME}"_MAX_CRITICAL_ISSUE_NUM"
MAX_CRITICAL_ISSUE_NUM=$(eval echo \${$MAX_CRITICAL_ISSUE_NUM}) SERVICE_NAME=${PROJECT_NAME}"_SERVICE_NAME"
SERVICE_NAME=$(eval echo \${$SERVICE_NAME}) EMAIL_LIST=${PROJECT_NAME}"_EMAIL_LIST"
EMAIL_LIST=$(eval echo \${$EMAIL_LIST}) author_list=`cat $BUILD_LOG | grep "Author"` || true ######### SET ENVIRONMENT VARIABLE ##############
export BASE_DIR=${WORKSPACE}/${GIT_PROJECT_NAME}
export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` ####### Build Job ######
cd ${WORKSPACE} START_TIME=`date +"%m/%d/%Y %T"`
echo "overview->startTime: ${START_TIME}" >> ${BUILD_LOG}
echo "overview->logURL: ${BUILD_URL}" >> ${BUILD_LOG}
echo "overview->serviceType: ${PROJECT_NAME} Service" >> ${BUILD_LOG} git clone ${GIT_PROJECT_URL} BUILD_FILE=rpmbuild.sh
cd ${BASE_DIR}
git checkout origin/${BRANCH} let BUILD_NUMBER=100+${BUILD_NUMBER}
sed -i -r "s/^RELEASE.+/RELEASE=${BUILD_NUMBER}_Dev/g" ${BUILD_FILE}
sh ${BUILD_FILE} cd ${WORKSPACE}
rm -rf *.rpm
mv `find . -type f -name "*.rpm" | egrep -v '(\.src\.)' | egrep -v 'BUILD'` .
RPM=`ls *.rpm` packageService --addPackages --repository ${REPOSITORY} --packages "${RPM}" ############# Unit test analyze ###############
UnitTestAnalyzer.pl --path "${BASE_DIR}/.rpm/BUILD" --sufix xml >> ${BUILD_LOG} ################Run Sonar################
cd ${BASE_DIR}
mvn clean install -f ${BASE_DIR}/pom.xml -DPASSWORD=$ENV{PASSWORD} -DUSERNAME=$ENV{USERNAME} -e -B sonar:sonar ################Get project in Sonar ################
sonarProjectName=`sed -ne '/name/{s/.*<name>\(.*\)<\/name>.*/\/p;q;}' pom.xml`
lowerProjectName=`echo $sonarProjectName | awk '{print tolower($0)}'` sonarId=`queryDB.pl --query="select * from resource_index where kee = '$lowerProjectName' order by resource_id;"`
echo $sonarId
sonar_path=http://vm10686.global.xxx.net:9000/dashboard/index/${sonarId}
echo "SonarPath: ${sonar_path}" >> ${BUILD_LOG} ################Function for get Sonar home page info################
##Key for what you want to find
##column to print function get_html_value() {
html=$
key=$
column=$
echo `cat $html | sed -n '/'$key'/p' | awk -v column=$column -F "[><]" '{print $column}'`
} ##############Sonar issues analyze ###############
curl ${sonar_path} > html.txt
blocker_num=`get_html_value html.txt m_blocker_violations `
critical_num=`get_html_value html.txt m_critical_violations `
major_num=`get_html_value html.txt m_major_violations `
minor_num=`get_html_value html.txt m_minor_violations `
info_num=`get_html_value html.txt m_info_violations ` echo "SonarIssues->Blocker: ${blocker_num}" >> ${BUILD_LOG}
echo "SonarIssues->Critical: ${critical_num}" >> ${BUILD_LOG}
echo "SonarIssues->Major: ${major_num}" >> ${BUILD_LOG}
echo "SonarIssues->Minor: ${minor_num}" >> ${BUILD_LOG}
echo "SonarIssues->Info: ${info_num}" >> ${BUILD_LOG} ##############Sonar Code Coverage analyze ###############
code_coverage=`get_html_value html.txt m_coverage `
code_coverage_integer=`echo $code_coverage | sed -n 's/%//p'` line_coverage=`get_html_value html.txt m_line_coverage `
branch_coverage=`get_html_value html.txt m_branch_coverage ` echo "CoverageInfo->UnitTestsCoverage: ${code_coverage}" >> ${BUILD_LOG}
echo "CoverageInfo->LineCoverage: ${line_coverage}" >> ${BUILD_LOG}
echo "CoverageInfo->BranchCoverage: ${branch_coverage}" >> ${BUILD_LOG}
echo "CoverageInfo->CoverageThreshold: ${SONAR_MIN_COVERAGE}%" >> ${BUILD_LOG} ################Send Email Code Coverage < Threshold ################
if (( $(echo "$code_coverage_integer < $SONAR_MIN_COVERAGE" | bc -l) ))
then
mail_subject=${PROJECT_NAME}"_Service_Build Job - Build # "${BUILD_NUMBER}" Failure"'!' mail_body="Hi, ${PROJECT_NAME}"" project member"$'\n'$'\n'"Alerts : Unit Tests Coverage: ${code_coverage} < CoverageThreshold: ${SONAR_MIN_COVERAGE}%, the latest build package cannot be allowed to install to service - ${SERVICE_NAME} [$INSTANCES]"'!'$'\n'$'\n'"For details please refer to SonarQube - ${sonar_path}"$'\n'$'\n'"The latest code author list as below:"$'\n'"${author_list}" sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --mailbody "$mail_body" --msg_type "text"
exit
fi ################Send Email when Sonar has major above issue################
if [ ${blocker_num} -gt ] || [ ${critical_num} -gt ${MAX_CRITICAL_ISSUE_NUM} ] || [ ${major_num} -gt ${MAX_MAJOR_ISSUE_NUM} ]
then
mail_subject=${PROJECT_NAME}"_Service_Build Job - Build # "${BUILD_NUMBER}" Failure"'!' mail_body="Hi, ${PROJECT_NAME}"" project member"$'\n'$'\n'"Alerts : Sonar major above issues != 0, the latest build package cannot be allowed to install to service - ${SERVICE_NAME} [$INSTANCES]"'!'$'\n'$'\n'"Blocker issues: ${blocker_num}"$'\n'"Critical issues: ${critical_num}"$'\n'"Major issues: ${major_num}"$'\n'"Minor issues: ${minor_num}"$'\n'"Info issues: ${info_num}"$'\n'$'\n'"For details please refer to SonarQube - ${sonar_path}"$'\n'$'\n'"The latest code author list as below:"$'\n'"${author_list}" sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --mailbody "$mail_body" --msg_type "text"
exit
fi ####### Call Deploy Job ######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Deploy/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'${PROJECT_NAME}'"},{"name":"PIPELINE_NUM","value":"'${PIPELINE_NUM}'"}]}'
mvn clean install -f ${BASE_DIR}/pom.xml -DPASSWORD=$ENV{PASSWORD} -DUSERNAME=$ENV{USERNAME} -e -B sonar:sonar

运行Sonar 的一些数据库配置信息,是放在maven/conf/settings.xml文件里的。

另外一点:每个项目在Sonar里的名称是 项目pom文件的<name></name>标签里的值,根据这个值去数据库里搜出id, 有了这个id就可以直接访问到Sonar上的这个项目 比如http://vm10686.global.xxx.net:9000/dashboard/index/1234

4)Template_Service_Deploy

Build完之后 就是要Deploy了, 将rpm包装到相应Service的instance上,然后重启service

rm -rf ${WORKSPACE}/*

###### Import config file ######
cd ~/jenkins/common/Config
. Project_Config.txt ###### Initial parameters ######
SERVICE_NAME=${PROJECT_NAME}"_SERVICE_NAME"
SERVICE_NAME=$(eval echo \${$SERVICE_NAME}) INSTANCES=${PROJECT_NAME}"_INSTANCES"
INSTANCES=$(eval echo \${$INSTANCES}) PATTERN=${PROJECT_NAME}"_RPM_GREP_PATTERN"
PATTERN=$(eval echo \${$PATTERN}) LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME ##########SET ENVIRONMENT VARIABLE##############
SCRIPTS_HOME=~/jenkins/common/Scripts
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` InstallTime=`date +"%m/%d/%Y %T"`
echo "Installation->InstallTime: ${InstallTime}" >> ${BUILD_LOG} ############ Deploy ###############
gridService --add-service --service ${SERVICE_NAME} --instance ${INSTANCES}
sleep 60
gridService --start-service --service ${SERVICE_NAME} Endpoint="${INSTANCES}:8080"
echo "Installation->Endpoint: ${Endpoint}" >> ${BUILD_LOG} echo "Installation->LogURL: ${BUILD_URL}" >> ${BUILD_LOG} UpdateTime=`date +"%m/%d/%Y %T"`
echo "ServiceInfo->UpdateTime: ${UpdateTime}" >> ${BUILD_LOG} ServiceInfoProvider.pl --service ${SERVICE_NAME} --pattern ${PATTERN} >> ${BUILD_LOG} ###### Call Polling Job ######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Polling/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'$PROJECT_NAME'"},{"name":"PIPELINE_NUM","value":"'$PIPELINE_NUM'"}]}'

5)Template_Service_Polling

接下来通过这个Polling job 轮询service有没有启动好, 每1分钟 判断一下,timeout时间10分钟, 脚本其实很简单,就是调一个接口,查看状态是否status=200

rm -rf ${WORKSPACE}/*

######## Import config file #########
cd ~/jenkins/common/Config
. Project_Config.txt ####### SET ENV #############
export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` ###### Initial parameters #############
SERVICE_NAME=${PROJECT_NAME}"_SERVICE_NAME"
SERVICE_NAME=$(eval echo \${$SERVICE_NAME}) LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME echo "pollingjob->logURL: ${BUILD_URL}" >> ${BUILD_LOG} ###### Polling service whether start #############
sleep 30
gridService --ping --timeout 600 --interval 60 --service ${SERVICE_NAME} ###### Call API Testing Job ######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_APITesting/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'${PROJECT_NAME}'"},{"name":"PIPELINE_NUM","value":"'${PIPELINE_NUM}'"}]}'

6)Template_Service_APITesting

Polling之后 就是调用借口测试的内部测试系统,如果export 的变量有 TestPlan 就会执行,否则直接调用最后一个job--Report

rm -rf ${WORKSPACE}/*

###### Import config file ######
cd ~/jenkins/common/Config
. Project_Config.txt ###### Initial parameters ######
LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME TESTPLAN_ID=${PROJECT_NAME}"_TESTPLAN_ID"
TESTPLAN_ID=$(eval echo \${$TESTPLAN_ID}) SERVICE_NAME=${PROJECT_NAME}"_SERVICE_NAME"
SERVICE_NAME=$(eval echo \${$SERVICE_NAME}) EMAIL_LIST=${PROJECT_NAME}"_EMAIL_LIST"
EMAIL_LIST=$(eval echo \${$EMAIL_LIST}) EMAIL_CC_LIST=${PROJECT_NAME}"_EMAIL_CC_LIST"
EMAIL_CC_LIST=$(eval echo \${$EMAIL_CC_LIST}) ###########Set Env #############
export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts/
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` ################Function################
##Key for what you want to find
##num occurrence of value if it multiple time function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
} ############### Main ###################### if [[ $TESTPLAN_ID == "" ]]; then
echo "TestPlan ID is blank"
###### Call Report Job ######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Report/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'$PROJECT_NAME'"},{"name":"PIPELINE_NUM","value":"'$PIPELINE_NUM'"}]}' else
processID=$(curl -X GET http://vm10765.global.xxx.net:8080/tms/project/testPlan/run.spring?ids=$TESTPLAN_ID) if [[ $processID != TestPlan-* ]]; then mail_subject=${PROJECT_NAME}"_Service_TMS_APITesting Job - Build # "${BUILD_NUMBER}" Failure"'!'
mail_body="TMS Server is not available now, please contact adminstrator."
echo $mail_body
sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --emailcclist "${EMAIL_CC_LIST}" --mailbody "$mail_body" --msg_type "text"
else while [ 1 ]
do
response=$(curl -X GET http://vm10765.global.xxx.net:8080/tms/project/getProcessResult.spring?processID=$processID) totalCounts=`echo $response | jsonValue totalCounts`
notStartCounts=`echo $response | jsonValue notStartCounts`
processingCounts=`echo $response | jsonValue processingCounts`
faildCounts=`echo $response | jsonValue faildCounts`
comparedCounts=`echo $response | jsonValue comparedCounts`
comparedFaildCounts=`echo $response | jsonValue comparedFaildCounts`
unComparedCounts=`echo $response | jsonValue unComparedCounts` if [ $notStartCounts == 0 ] && [ $processingCounts == 0 ]; then
echo "End of run."
break else
echo "API Testing is still running, please wait..."
sleep 5
fi
done ############# Print Log ###################
echo "APITestingInfo->TotalCounts: $totalCounts" >> ${BUILD_LOG}
echo "APITestingInfo->ErrorCounts: $faildCounts" >> ${BUILD_LOG}
echo "APITestingInfo->ComparedPassedCounts: $comparedCounts" >> ${BUILD_LOG}
echo "APITestingInfo->ComparedFailedCounts: $comparedFaildCounts" >> ${BUILD_LOG}
echo "APITestingInfo->UnComparedCounts: $unComparedCounts" >> ${BUILD_LOG} ###### Call Report Job ######
curl -d --user "jadmin:71103407" -s http://vm10686.global.xxx.net:8080/view/Template/job/Template_Service_Report/build --data json='{"parameter": [{"name":"PROJECT_NAME","value":"'$PROJECT_NAME'"},{"name":"PIPELINE_NUM","value":"'$PIPELINE_NUM'"}]}' fi
fi

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

7)Template_Service_Report

这里主要是生成Report并且发送Email,用Java写的一个jar放到linux上执行的,用html直接贴在邮件里发送email的,简单的其实可以用Perl写个excel,统计下测试情况,做个报表。

rm -rf ${WORKSPACE}/*

###### Import config file ######
cd ~/jenkins/common/Config
. Project_Config.txt ###### Initial parameters ######
LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG_FOLDER=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME SERVICE_NAME=${PROJECT_NAME}"_SERVICE_NAME"
SERVICE_NAME=$(eval echo \${$SERVICE_NAME}) INSTANCES=${PROJECT_NAME}"_INSTANCES"
INSTANCES=$(eval echo \${$INSTANCES}) EMAIL_LIST=${PROJECT_NAME}"_EMAIL_LIST"
EMAIL_LIST=$(eval echo \${$EMAIL_LIST}) EMAIL_CC_LIST=${PROJECT_NAME}"_EMAIL_CC_LIST"
EMAIL_CC_LIST=$(eval echo \${$EMAIL_CC_LIST}) TESTPLAN_ID=${PROJECT_NAME}"_TESTPLAN_ID"
TESTPLAN_ID=$(eval echo \${$TESTPLAN_ID}) export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts/
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd`
export XML_REPORT=${WORKSPACE}/interface.xml report.pl --log-path="$BUILD_LOG_FOLDER" --output ${XML_REPORT} REPORT_PATH=${WORKSPACE}/Report.html
echo ${REPORT_PATH} java -Dxmlpath="${XML_REPORT}" -Dreport="${REPORT_PATH}" -jar Reporter.jar
report_context=`cat $REPORT_PATH` errorCounts=`cat $BUILD_LOG | grep "APITestingInfo->ErrorCounts" | awk -F ": " '{print $2}'`
comparedFailedCounts=`cat $BUILD_LOG | grep "APITestingInfo->ComparedFailedCounts" | awk -F ": " '{print $2}'`
sonarPath=`cat $BUILD_LOG | grep "SonarPath" | awk -F ": " '{print $2}'` ############## Send Email ##################
if [ ${errorCounts} -gt 0 ] || [ ${comparedFailedCounts} -gt 0 ]
then
mail_subject=${PROJECT_NAME}"_Service_Report Job - Build # "${BUILD_NUMBER}" Failure"'!' mail_body="Hi, ${PROJECT_NAME} project member <BR><BR>The latest build package has been installed to service - ${SERVICE_NAME} [$INSTANCES], but the API testing has some test case failed.<BR><BR>Errored Test Case: ${errorCounts}<BR>Failed Test Case: ${comparedFailedCounts}<BR><BR>For details please refer to Test Management System - http://vm10765.global.xxx.net:8080/tms/login.spring<BR><BR>SonarQube - $sonarPath<BR><BR>Report as below:<BR><BR>""$report_context" sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --emailcclist "${EMAIL_CC_LIST}" --mailbody "$mail_body" --attached ${REPORT_PATH} else if [[ $TESTPLAN_ID == "" ]]; then
mail_subject=${PROJECT_NAME}"_Service_Report Job - Build # "${BUILD_NUMBER}" Success"'!'" [No TestPlan for API testing]"
mail_body="Hi, ${PROJECT_NAME} project member <BR><BR>The latest build package has been installed to service - ${SERVICE_NAME} [$INSTANCES].<BR><BR>SonarQube - $sonarPath<BR><BR>Report as below:<BR><BR>""$report_context" else
mail_subject=${PROJECT_NAME}"_Service_Report Job - Build # "${BUILD_NUMBER}" Success"'!'
mail_body="Hi, ${PROJECT_NAME} project member <BR><BR>The latest build package has been installed to service - ${SERVICE_NAME} [$INSTANCES] and API testing has passed.<BR><BR>For detailed test results, please refer to Test Management System - http://vm10765.global.xxx.net:8080/tms/login.spring<BR><BR>SonarQube - $sonarPath<BR><BR>Report as below:<BR><BR>""$report_context"
fi sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --emailcclist "${EMAIL_CC_LIST}" --mailbody "$mail_body" --attached ${REPORT_PATH} fi ########### Tear Down ###########
rm -rf ${BUILD_LOG_FOLDER}

我设置的是每天运行一次所有的project的持续集成, 收到的Email如下图所示:

Report中的很多信息都是来源于Sonar

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

到这里就结束了吗,还没有, 如果Build job,单元测试不通过或者其他原因导致Build Failure, 那怎么办?

再加一个Post Build job, 来监控Build job, 如果挂了,发email通知项目人员。

下图为Build job 加一个Action, 当Job Failed时触发Template_Service_Post_Build job

Template_Service_Post_Build 脚本如下:

rm -rf ${WORKSPACE}/*

######## Import config file #########
cd ~/jenkins/common/Config
. Project_Config.txt ####### SET ENV #############
export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` ###### Initial parameters #############
LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME EMAIL_LIST=${PROJECT_NAME}"_EMAIL_LIST"
EMAIL_LIST=$(eval echo \${$EMAIL_LIST}) EMAIL_CC_LIST=${PROJECT_NAME}"_EMAIL_CC_LIST"
EMAIL_CC_LIST=$(eval echo \${$EMAIL_CC_LIST}) author_list=`cat $BUILD_LOG | grep "Author"` || true ###### Monitor Service Build Job ########
build_job_url=`cat $BUILD_LOG | sed -n '/overview->logURL/p' | awk -F ": " '{print $2}'`
build_job_build_number=`echo $build_job_url | awk -F "/" '{print $(NF-1)}'`
build_job_log_url=$build_job_url"logText/progressiveText?start=0"
build_job_console_output=$build_job_url"console" mail_subject=${PROJECT_NAME}"_Service_Build Job - Build # "${build_job_build_number}" Failure"'!' mail_body="Hi, ${PROJECT_NAME}"" project member"$'\n'$'\n'"Build Failure"'!'$'\n'$'\n'"For details please refer to Console Output - ${build_job_console_output}"$'\n'$'\n'"The latest code author list as below:"$'\n'"${author_list}" sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --emailcclist "${EMAIL_CC_LIST}" --mailbody "$mail_body" --msg_type "text"

同理还需要加一个Template_Service_Post_Polling job来监控Template_Service_Polling job, 如果轮询Service的结果是重启失败,则邮件通知项目组人员。

脚本如下:

rm -rf ${WORKSPACE}/*

######## Import config file #########
cd ~/jenkins/common/Config
. Project_Config.txt ####### SET ENV #############
export SCRIPTS_HOME=${SHARED_FOLDER}/Scripts
cd ${SCRIPTS_HOME}
chmod 755 *
export PATH=$PATH:`pwd` ###### Initial parameters #############
LOG_NAME=${PROJECT_NAME}"_BUILD_"${PIPELINE_NUM}".log"
BUILD_LOG=${SHARED_FOLDER}/Log/${PROJECT_NAME}"_"${PIPELINE_NUM}/$LOG_NAME EMAIL_LIST=${PROJECT_NAME}"_EMAIL_LIST"
EMAIL_LIST=$(eval echo \${$EMAIL_LIST}) EMAIL_CC_LIST=${PROJECT_NAME}"_EMAIL_CC_LIST"
EMAIL_CC_LIST=$(eval echo \${$EMAIL_CC_LIST}) author_list=`cat $BUILD_LOG | grep "Author"` ###### Monitor Service Polling Job ########
polling_job_url=`cat $BUILD_LOG | sed -n '/pollingjob->logURL/p' | awk -F ": " '{print $2}'`
polling_job_build_number=`echo $polling_job_url | awk -F "/" '{print $(NF-1)}'`
polling_job_log_url=$polling_job_url"logText/progressiveText?start=0"
polling_job_console_output=$polling_job_url"console" mail_subject=${PROJECT_NAME}"_Service_Polling Job - Build # "${polling_job_build_number}" Failure"'!' mail_body="Hi, ${PROJECT_NAME}"" project member"$'\n'$'\n'"Service Restart Failure"'!'$'\n'$'\n'"For details please refer to Console Output - ${polling_job_console_output}"$'\n'$'\n'"The latest code author list as below:"$'\n'"${author_list}" sendEmail.pl --subject "${mail_subject}" --emaillist "${EMAIL_LIST}" --emailcclist "${EMAIL_CC_LIST}" --mailbody "$mail_body" --msg_type "text"

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

现在我们来看一下更新后的CI 流程图

那么,这个flow的前提是 所有的项目都是按这个流程来的,万一有些特别的项目呢? 比如一个项目,它在Build之前 需要Pre-Build一个CommonUtils包, 那怎么办呢,简单一点就是为了这个项目定制一下。

为了更具通用性,扩展性,方便的插入一些special job, 我们可以在每个job之间加入 Exit Point, 以适应一些特别的项目想在任意点Plug in一些特别的job,

除了要在Config job 加上一些规则, 比如 export ProjectX_EXIT_POINT=Template_Service_Commit_Auto_Build|Special ProjectX-PreBuild job

还要在每个job里加一些判断语句,比如当前run的job name 等于ProjectX_EXIT_POINT以竖线分割的第一个值,则表明要在这个job退出, 第二个值表明要去调的special job name,当然 special job 里做完了自己的事之后还是要回调原来flow的下一个job

流程图更新如下所示:

感谢阅读,如果您觉得本文的内容对您的学习有所帮助,您可以点击右下方的推荐按钮,您的鼓励是我创作的动力。

##转载注明出处:http://www.cnblogs.com/wade-xu/p/4378224.html

这些年我们一起搞过的持续集成~Jenkins+Perl and Shell script的更多相关文章

  1. 自动化持续集成Jenkins

    自动化持续集成Jenkins 使用Jenkins配置自动化构建http://blog.csdn.net/littlechang/article/details/8642149 Jenkins入门总结h ...

  2. 持续集成~Jenkins构建GitHub项目的实现

    有了前两讲的基础,这回我们就可以把github上的项目做到CI(jenkins)里了,让它自动去集成部署,持续集成~Jenkins里的NuGet和MSBuild插件,持续集成~Jenkins里的pow ...

  3. 1.jenkins持续集成-jenkins安装

    1.为什们要使用jenkins Jenkins是基于Java开发的一种持续集成工具,用于监控持续重复的工作,功能包括: 1.持续的软件版本发布/测试项目; 2.监控外部调用执行的工作. 2.安装jen ...

  4. 持续集成Jenkins + robot framework + git

    Jenkins + robot framework + git持续集成 一.Jenkins安装插件 进入系统管理—插件管理—可选插件下安装以下插件Git Client Plugin.GIT plugi ...

  5. 持续集成Jenkins+sonarqube部署教程

    1 引言 1.1 文档概要 本文主要介绍jenkins,sonar的安装与集成,基于ant,maven构建.用一个例子介绍jenkins的编译打包部署,代码检查.最后集成jenkins.(现阶段只是简 ...

  6. python+selenium自动化软件测试(第11章):持续集成jenkins和GitHub的使用

    11.1 jenkins持续集成环境 相关安装包下载链接:http://pan.baidu.com/s/1qYhmlg4 密码:dcw2赠送jenkins集成selenium环境视频链接http:// ...

  7. 持续集成-jenkins介绍与环境搭建

    什么是持续集成? 转自:https://blog.csdn.net/tanshizhen119/article/details/80328523 持续集成,俗称CI, 大师Martin Fowler对 ...

  8. 敏捷持续集成(Jenkins)

    在前面已经完成git和gitlab的相关操作 1.持续集成的概念: 1. 什么是持续集成: 持续集成是一种软件开发实践,即团队开发成员经常集成他们的工作,通过每个成员每天至少集成一次,也就意味着每天可 ...

  9. DevOps实践之持续集成Jenkins(最新版本测试)

    一.安装Jenkins (1)下载并安装配置Java Development Kit 8 [root@localhost ~]# rpm -ivh jdk-8u161-linux-x64.rpm Pr ...

随机推荐

  1. JavaScript Array对象 知识点总结

    1 isArray方法 该方法是Array对象的静态方法,用来判断一个值是否为数组,它可以弥补typeof运算符的不足. 用法是Array.isArray(array实例) 通用的判断对象数据类型的方 ...

  2. 2014年4月份第1周51Aspx源码发布详情

    基于Extjs4+MVC4权限管理源码  2014-3-31 [VS2012]源码描述: 20140331更新:修改部门管理中bug 20140303更新:增加部门管理模块,主要包含部门添加,编辑,删 ...

  3. Notification NotificationManager RemoteViews PendingIntent

    Notification和NotificationManager :可以用来实现可视化的消息通知等,比如:下载进度显示,消息显示,广播的内容等 RemoteViews :用于创建自定义的Notific ...

  4. angular 国际化

    今天mentor说让我看看angular国际化的问题,我看了看,发现这个帖子很有价值,但是手头上还有好几个issue没有解决,所以就先考皮在这将连接,解决完这几个bug我再去细细研究这个问题, htt ...

  5. ssd硬盘u盘装win7扩展文件时0x80070570错误

    搞了个minisata ssd 硬盘,80G正好,只用来装操作系统. 这个u盘装了很多系统了,没感觉会有什么问题,但在安装过程中出现0x80070570错误.见图 网上有很多方法. 最后的解决方法是: ...

  6. PAT (Basic Level) Practise:1037. 在霍格沃茨找零钱

    [题目链接] 如果你是哈利·波特迷,你会知道魔法世界有它自己的货币系统 —— 就如海格告诉哈利的:“十七个银西可(Sickle)兑一个加隆(Galleon),二十九个纳特(Knut)兑一个西可,很容易 ...

  7. HDU 2509 nim博弈

    Be the Winner Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: 32768/32768 K (Java/Others)Tot ...

  8. JAVA中精确计算金额BigDecimal

    package com.chauvet.utils; import java.math.BigDecimal; import java.text.DecimalFormat; import java. ...

  9. 第二篇T语言实例开发(版本5.3),福彩3D摇号器

    ---恢复内容开始--- 福彩3D摇号器 先验知识功能的含义与使用 功能是TC源程序的基本模块,通过对功能模块的调用实现特定的功能,TC语言中的功能相当于其他高级语言的子程序. 在TC里面,功能分为三 ...

  10. c++作用域运算符---7

    原创博客:转载请标明出处:http://www.cnblogs.com/zxouxuewei/ ::是C++里的“作用域运算符”. 比如声明了一个类A,类A里声明了一个成员函数void f(),但没有 ...