基于jmeter和shell的接口性能自动化

1. 总体需求

由于性能测试中涉及的查询接口多，版本迭代频繁，版本更新后自动跑一轮查询业务的性能，可以及时发现一些开发修复bug触发的非预期的bug，利用晚上时间快速重测性能接口可以解放人工测试的时间，让测试人员可以把精力集中在测试复杂接口，调优分析性能瓶颈上。

2. 实现流程

自动化的场景模拟真实手工测试，操作步骤和手工测试一样。

开始读取接口测试用例执行用例，发起服务器监控收集执行结果和监控结果执行完所有用例？整理结果，发送邮件结束yesno

3.准备工作

准备软件：

系统环境：CentOS release 6.7 (Final)
内核版本：Linux localhost 2.6.32-573.7.1.el6.x86_64
测试工具：apache-jmeter-2.13 http://jmeter.apache.org/download_jmeter.cgi
运行JDK环境：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
python环境：Python 2.6.6
服务器监控nmon：http://nmon.sourceforge.net/pmwiki.php?n=Site.Download

4.具体实现

4.1用例执行

Jmeter的执行支持命令行的方式执行，并且会有一个粗略报告，我们取这个报告里面的执行结果并保存下来

Jmeter的执行结果：

[root@localhost bin]# ./jmeter -n -t singin.jmx

Creating summariser <summary>

Created the tree successfully using singin.jmx

Starting the test @ Mon Dec 14 16:42:33 CST 2015 (1450082553651)

Waiting for possible shutdown message on port 4445

summary +      1 in     1s =    1.3/s Avg:   268 Min:   268 Max:   268 Err:     0 (0.00%) Active: 1 Started: 1 Finished: 0

summary +      2 in     0s =   50.0/s Avg:    17 Min:    14 Max:    20 Err:     0 (0.00%) Active: 0 Started: 1 Finished: 1

summary =      3 in     1s =    3.7/s Avg:   100 Min:    14 Max:   268 Err:     0 (0.00%)

Tidying up ...    @ Mon Dec 14 16:42:34 CST 2015 (1450082554551)

... end of run
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10

由sumary统计行可以得到我们需要收集的测试结果：

测试结果项	值
总请求数	3
平均tps	3.7
平均响应时间	100ms
请求失败率	0.00%

主流程脚本

#/bin/bash

source /etc/profile

Jmeter_Home='/usr/local/apache-jmeter-2.13'

TestReport='/data/loadtest/report'

LogDIR='/data/loadtest/log'

Date=`date +"%F"`

cd /data/loadtest/

>summary.txt

#清理上次执行结果

run_test()

{

#获取测试用例

 for i in `find ./testcase/ -name *.jmx|awk -F '.' '{print $2}'`

  do

  casename=`echo "$i"|awk -F '/' '{print $4}'`

  >log/${casename}.txt

  echo -n "$i ">>summary.txt

  #发起监控

  ./monitor.sh >/dev/null 2>&1 &

  #开始执行测试

  $Jmeter_Home/bin/jmeter -n -t /data/loadtest${i}.jmx >>log/${casename}.txt &

  sleep 310

  #如果执行310s还未结束，强制终止执行

  ps -ef | grep java |grep -v grep | awk '{print $2}' |xargs kill -9

  sleep 3

  #提取结果

  grep 'summary =' log/${casename}.txt| tail -1 |awk -F '[\t / (]+' '{if($7>10000 && $17<10.00){printf("%s %d %d %d %.2f% pass ",$7,$10,$3,$16,100-$17)}else{printf("%s %d %d %d %.2f%% fail ",$7,$10,$3,$16,100-$17)}}'>>summary.txt

  cat monitor.txt >>summary.txt

  echo '' >> summary.txt

  #获取关键日志

  ssh 10.1.30.54 'tail -n 300 /data/logs/fcuh-user/catalina.out'>${LogDIR}${i}.log

 done

}

run_test

sleep 3

#生成html报告

sh genHTML.sh

sleep 1

#发送邮件

python sendmail.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

4.2服务器监控

服务器资源收集方面，选用的是nmon监控工具，因为它可以后台收集结果保存到文件。由于每个用例测试5分钟，所以只需要监控300秒，每5s监控一次，对应命令：

nmon -f -t -s5 -c60 -F /data/test.nmon
1
1

测试用例跑完再读取这个结果文件，获取有用的信息

当前只统计了磁盘io和cpu的占用率信息，原始文件保存在本地目录，如果需要，可以手动查看到。

监控脚本：

#!/bin/bash

#读取监控服务器列表

SERVERLIST=`cat serverlist`

DATE=`date +'%F'`

mkdir -p /data/loadtest/monitor/$DATE

TIME=`date +'%T'`

#发起监控

for i in $SERVERLIST

do

  ssh $i 'nmon -f -t -s5 -c60 -F /data/test.nmon >/dev/null 2>&1 &'

done

#监控5分钟

sleep 303

>monitor.txt

#收集监控结果，保存到monitor.txt

for i in $SERVERLIST

do

 scp $i:/data/test.nmon /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon

 io=`cat /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon|grep "DISKBUSY,T" | awk -F ',' '{sum+=$3} END {printf("%.2f%",sum/NR)}'`

 cpu=`cat /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon|grep "CPU_ALL,T" | awk -F ',' '{sum+=$6} END {printf("%.2f%",100-sum/NR)}'`

 #net=`cat /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon|grep "NET,T"|awk -F ',' '{sum_r+=$4}{sum_w+=$6} END {print sum_r/NR,sum_w/NR}'`

 echo -n "${cpu} ${io} ">>monitor.txt

done
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

将用例执行结果和监控结果都汇总到summary.txt里

生成的格式如下，方便后面生成html格式的报告

[root@localhost loadtest]# cat summary.txt

/testcase/user/获取用户自己的信息 10748.6 8 3225296 0 100.00% pass 32.87% 5.34% 81.59% 2.12% 0.15% 1.78% 12.13% 32.83% 17.18% 6.14%

/testcase/user/未读消息数 11487.4 7 3446960 0 100.00% pass 32.33% 12.57% 69.02% 1.86% 0.18% 1.74% 13.55% 35.85% 18.56% 7.65% 
1
2
3
1
2
3

服务器监控脚本

#!/bin/bash

SERVERLIST=`cat serverlist`

DATE=`date +'%F'`

mkdir -p /data/loadtest/monitor/$DATE

TIME=`date +'%T'`

for i in $SERVERLIST

do

  ssh $i 'nmon -f -t -s5 -c60 -F /data/test.nmon >/dev/null 2>&1 &'

done

sleep 303

>monitor.txt

for i in $SERVERLIST

do

 scp $i:/data/test.nmon /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon

 io=`cat /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon|grep "DISKBUSY,T" | awk -F ',' '{sum+=$3} END {printf("%.2f%",sum/NR)}'`

 cpu=`cat /data/loadtest/monitor/$DATE/${i}_${TIME}.nmon|grep "CPU_ALL,T" | awk -F ',' '{sum+=$6} END {printf("%.2f%",100-sum/NR)}'`

 echo -n "${cpu} ${io} ">>monitor.txt

done


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

4.3生成html报告

#!/bin/sh

>index.html

echo "<html><head><META http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\"/><title>用户端自动化性能测试报告</title>">>index.html

echo `cat style.css`>>index.html

(

cat <<EOF

<script language="JavaScript">

               function show_detail(detail){

                           if(detail.style.display=="none"){

                           detail.style.display="";

                           }

                           else{

                           detail.style.display="none";

                           }

                           }

                           </script>

EOF

)>>index.html

echo "</head><body><h1>用户端自动化性能测试报告</h1><hr size="1">">>index.html

sum=`cat summary.txt | wc -l`

sucess=`cat summary.txt|grep pass |grep -v grep|wc -l`

fail=`expr $sum - $sucess`

rate=`echo "$sucess $sum"|awk '{printf("%.2f%%",$1/$2*100)}'`

(

cat <<EOF

<table><tr><td>

<h2>结果汇总</h2>

<table width="60%" cellspacing="2" cellpadding="5" border="0" class="details" align="left">

<tr><th>总接口数</th><th>成功接口数</th><th>失败接口数</th><th>测试通过率</th></tr>

<tr align="center"><td>$sum</td><td>$sucess</td><td>$fail</td><td>$rate</td></tr>

</tr></table>

</td></tr>

EOF

)>>index.html

(

cat <<EOF

<tr><td>

<h2>概要结果</h2>

<table width="95%" cellspacing="2" cellpadding="5" border="0" class="details" align="left">

<tr valign="top">

<th>测试接口</th><th>每秒请求数(tps)</th><th>平均响应时间(ms)</th><th>总事务数</th><th>失败事务数</th><th>事务成功率</th><th>测试结果</th>

</tr>

<tr valign="top" class="">

EOF

)>>index.html

cat summary.txt |while read line

do

  echo $line | awk '{if($7=="pass"){print "<tr><td>"$1"</td><td>"$2"</td><td>"$3"</td><td>"$4"</td><td>"$5"</td><td>"$6"</td><td class=\"Pass\">"$7"</td></tr>"}else{print "<tr><td>"$1"</td><td>"$2"</td><td>"$3"</td><td>"$4"</td><td>"$5"</td><td>"$6"</td><td class=\"Failure\">"$7"</td></tr>"}}'>>index.html

done

echo "</tr></table></td></tr>">>index.html

echo "<table><tr><td><font color="red"><b>测试结果pass标准：tps>10000且事务成功率>90%</b></font><td></tr><tr><td><h2><a href=\"javascript:show_detail(detail)\">详细结果查看附件</a></h2></td></tr></table>">>index.html

#echo "<div class=\"page_details_expanded\" id=\"detail\" style=\"display:none;\" width=\"95%\">">>index.html

(

cat <<EOF

<table width="95%" cellspacing="2" cellpadding="5" border="0" class="details" align="left" id="detail" style="display:none">

<tr valign="top">

<th>测试接口</th><th>每秒请求数tps</th><th>平均响应时间(ms)</th><th>总事务数</th><th>失败事务数</th><th>成功率</th><th>测试结果</th><th>nginx服务器cpu</th><th>nginx服务器io</th><th>web服务器cpu</th><th>web服务器io</th><th>service服务器cpu</th><th>service服务器io</th><th>主数据库服务器cpu</th><th>主数据库服务器io</th><th>从数据库服务器cpu</th><th>从数据库服务器io</th>

</tr>

<tr valign="top" class="">

EOF

)>>index.html

j=1

for i in `cat summary.txt`

do

   if [ `expr $j % 17 ` != 0 ]; then

      echo '<td align="left">'$i'</td>'>>index.html

   else

      echo '<td align="left">'$i'</td></tr>'>>index.html

   fi

   j=`expr $j + 1`

done

echo "</tr></table></td></tr></table></body></html>">>index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

html样式表

<style type="text/css">

body {

        font:normal 68% verdana,arial,helvetica;

        color:#000000;

     }

table tr td, table tr th {

         font-size: 78%;

     }

table.details tr th{

         color: #ffffff;

         font-weight: bold;

         text-align:center;

         background:#2674a6;

         white-space: nowrap;

     }

table.details tr td{

        background:#eeeee0;

        white-space: nowrap;

     }

h1 {

        margin: 0px 0px 5px; font: 265% verdana,arial,helvetica

   }

h2 {

        margin-top: 1em; margin-bottom: 0.5em; font: bold 185% verdana,arial,helvetica

   }

h3 {

        margin-bottom: 0.5em; font: bold 115% verdana,arial,helvetica

   }

.Failure {

        font-weight:bold; color:red;

   }

.Pass {

        font-weight:bold; color:green;

   }

</style>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

4.4发送测试结果邮件

发邮件脚本

#!/usr/bin/env python

#coding: utf-8

import string

import smtplib

import os

import datetime

from email.mime.multipart import MIMEMultipart

from email.mime.text import MIMEText

from email.mime.base import MIMEBase

from email.header import Header

today = datetime.date.today()

sender = 'xx@xx.com'

receiverlist = ["a@a.com","b@b.com","c@c.com"]

subject = '%s %s' % ('用户端自动化性能测试报告',today)

smtpserver = 'smtp.exmail.qq.com'

username = 'xx@xx.com'

password = 'xxx'

f = open('index.html',"r")

content = f.read()

#msg = MIMEText(content,'html','utf-8')

msg = MIMEMultipart()

msg.attach(MIMEText(content,'html','utf-8'))

msg['From'] = 'xx@xx.com'

msg['to'] = ','.join(receiverlist)

msg['Subject'] = subject

att=MIMEText(open('index.html','rb').read(),'base64','gb2312')

att["Conten-Type"]='application/octet-stream'

att["Content-Disposition"]='attachment;filename="Load test result.html"'

msg.attach(att)

smtp = smtplib.SMTP()

smtp.connect(smtpserver)

smtp.ehlo()

smtp.starttls()

smtp.ehlo()

#smtp.set_debuglevel(1)

smtp.login(username, password)

smtp.sendmail(msg['From'],msg['to'],msg.as_string())

smtp.quit()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

测试结果截图：