检测 web项目 404 500 状态的 页面
用于发版前自动化测试
用法
1、使用参数 -f 指定url配置文件
2、url文件简单配置, 每行一条URL 下面三种格式都可以，如果不声明 GET、POST 默认为GET请求
https://www.wmzy.com/api/rank/schList?sch_rank_type=QS2018
get https://www.wmzy.com/api/rank/schList
post https://www.wmzy.com/account/saveEduInfo
调用命令: ./testPage.py -f ./urls.txt
3、通过 list 配置带有参数和信息头的的URL
配置文件是严格的 python 字典类型数据，包含 option 和 urls 两个 key
option:字典类型, 可以配置 method,headers,data, 会被list 内 url的配置覆盖
urls: list 类型, 每个元素配置一条url，格式如下
简单的get请求可以是url字符串
带有额外配置信息的URL使用字典配置
url: url地址
method:缺省认为GET
headers：字段类型，请求头信息
data:数据，缺省默认为 None
调用命令: ./testPage.py -f ./urls2.txt
4、使用参数 -out 输出测试报告文件
调用命令: ./testPage.py -f ./urls2.txt -out ./report.txt
5、通过参数 -print 100 指定:请求测试通过时打印请求返回内容的前100个字符
6、通过参数 -headers 指定:请求测试通过时打印请求返回的头信息
调用命令: ./testPage2.py -f ./ajaxTestUrls.txt -out ./report.txt -print 200
脚本 testPage.py
#!/usr/bin/python
# -*- coding: utf-8 -*-
# author ecalf import sys
import urllib2
import urllib
from urllib2 import URLError, HTTPError
import zlib
import codecs def showMsg( msg='', isWarn=False):
if isWarn:
print '\033[1;91m '+msg+' \033[0m'
else:
print '\033[1;0m '+msg+' \033[0m' def printReport(text='',reportFile=None):
print text
if reportFile and isinstance(reportFile,file):
print >> reportFile, text def getCmdParams(params):
fileIn = ''
fileOut = ''
outPutConfig = {
'printBodyLength':0,
'printHeaders':False
} if len(params)==1:
fileIn = params[0] else:
for i,param in enumerate(params):
if param.lower() == '-f':
if i+1>=len(params):
showMsg('请指定url配置文件',True)
else:
fileIn = params[i+1]
elif param.lower() == '-out':
if i+1>=len(params):
showMsg('请指定测试报告输出文件路劲',True)
else:
fileOut = params[i+1]
elif param.lower() == '-print':
if i+1>=len(params):
showMsg('请指定打印数据的长度',True)
else:
try:
outPutConfig['printBodyLength'] = int(params[i+1])
except Exception,err:
showMsg('指定打印请求返回的数据长度应该输入数字',True)
elif param.lower()=='-headers':
outPutConfig['printHeaders'] = True return fileIn,fileOut,outPutConfig def getUrlList(fileIn):
try:
fileObj = open(fileIn,'r')
fileContent = fileObj.read()
except Exception,err:
showMsg('无法读取文件:'+fileIn,True)
finally:
if fileObj:
fileObj.close() if fileContent[0:3]==codecs.BOM_UTF8:
fileContent = fileContent[3:] dataType = 'string'
try:
urlsConfig = eval(fileContent)
if isinstance(urlsConfig,dict):
dataType = 'list' except Exception,err:
#showMsg('parse fileContent err>>>'+str(err),True)
pass urls = []
option = None
try:
if dataType=='string':
lines = fileContent.split('\n')
for url in lines:
if url.replace(' ', ''):
urls.append(url) elif dataType.lower()=='list':
option = urlsConfig['option']
urls = urlsConfig['urls'] except Exception,err:
showMsg('文件读取错误，请使用UTF8编码保存URL配置文件',True) return urls,option def getCommonHeaders():
headers = {
'Connection':'keep-alive',
'Cache-Control':'max-age=0',
'Accept': 'text/html,application/xhtml+xml,application/xml,application/json;q=0.9,image/webp,image/apng,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36',
'Pragma': 'no-cache',
'Cache-Control': 'no-cache',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.9'
} return headers def setHost(request,host):
request.add_header('Host', host) def setCookie(request,cookieStr):
request.add_header('Cookie', cookieStr) def initConfig(urlConfig,option):
method = 'GET'
headers = getCommonHeaders()
data = {} #from option
if isinstance(option,dict):
if 'method' in option:
method = option['method'] if 'headers' in option and isinstance(option['headers'],dict):
headers.update(option['headers']) if 'data' in option and isinstance(option['data'],dict):
data.update(option['data']) #from urlConfig
if 'method' in urlConfig:
method = urlConfig['method'] if 'headers' in urlConfig and isinstance(urlConfig['headers'],dict):
headers.update(urlConfig['headers']) if 'data' in urlConfig and isinstance(urlConfig['data'],dict):
data.update(urlConfig['data']) if len(data.keys())==0:
data = None return urlConfig['url'],method,headers,data def initRequest(url,data=None,method='GET',headers=None):
url = urllib.quote(url.strip(),safe="$#&+;/:,=?@")
if data:
data = urllib.urlencode(data) if method.upper()=='GET':
if data:
searchStartIndex = url.find('?')
if searchStartIndex>-1:
url = url[0:searchStartIndex]+'?'+data+'&'+url[searchStartIndex+1:]
else:
anchorIndex = url.find('#')
if anchorIndex>-1:
url = url[0:anchorIndex]+'?'+data+url[anchorIndex:]
else:
url = url+'?'+data
request = urllib2.Request(url,None,headers)
else:
if not data:
data = ''
request = urllib2.Request(url,data,headers) return request,url def startTest(urls,option,fileOut,outPutConfig):
if len(urls)==0:
showMsg('url list is empty',True)
return reportFile = ''
if fileOut:
try:
reportFile = open(fileOut,'w')
except Exception,err:
showMsg('创建报告文件失败:'+str(err),True) counter = {
'count200':[],
'count401':[],
'count404':[],
'count500':[],
'count502':[],
'countURLError':[],
'countException':[],
} for i,urlConfig in enumerate(urls):
i=i+1 try:
if isinstance(urlConfig,basestring):
url = urlConfig
urlConfig = {}
else:
url = urlConfig['url'] method = 'GET'
if 'method' in urlConfig:
method = urlConfig['method'] if url[:4].upper()=='POST':
method = 'POST'
url = url[4:]
elif url[:3].upper()=='GET':
method = 'GET'
url = url[3:] urlConfig['method'] = method
urlConfig['url'] = url.strip() url,method,headers,data = initConfig(urlConfig,option)
request,url = initRequest(url,data,method,headers)
response = urllib2.urlopen(request) body = response.read()
gzipped = response.headers.get('Content-Encoding')
if gzipped:
body =zlib.decompress(body, 16+zlib.MAX_WBITS)
#print 'zlib.decompress body' if outPutConfig['printHeaders']==True or outPutConfig['printBodyLength']>0:
printReport('----------------------------------------------------------------',reportFile) if outPutConfig['printHeaders']==True:
printReport(response.info(),reportFile) if outPutConfig['printBodyLength']>0:
printReport(body[:outPutConfig['printBodyLength']],reportFile) statusCode = response.getcode()
countKey = 'count'+str(statusCode)
if countKey not in counter:
counter[countKey] = [] counter[countKey].append(i)
msg = '['+str(i)+']'+'\t'+str(statusCode)+'\t\t'+method.upper()+' '+url if response.geturl()!=url:
msg = msg+' redirect =>'+response.geturl() showMsg(msg)
if reportFile and isinstance(reportFile,file):
reportFile.write(msg+'\n') except HTTPError, err:
#print err,err.code,err.reason
stateText = err.reason
statusCode = err.code countKey = 'count'+str(statusCode)
if countKey not in counter:
counter[countKey] = [] counter[countKey].append(i) #print err.reason
msg = '['+str(i)+']'+'\t'+str(statusCode)+'\t\t'+method.upper()+' '+url
showMsg(msg,True)
if reportFile and isinstance(reportFile,file):
reportFile.write(msg+'\n')
except URLError, err:
counter['countURLError'].append(i) msg = '['+str(i)+']'+'\tURLError'+'\t'+method.upper()+' '+url
showMsg(msg,True)
if reportFile and isinstance(reportFile,file):
reportFile.write(msg+'\n') except Exception,err:
counter['countException'].append(i) msg = '['+str(i)+']'+'\tException:'+str(err)+'\t'+method.upper()+' '+url
showMsg(msg,True)
if reportFile and isinstance(reportFile,file):
reportFile.write(msg+'\n') reportMsg = []
reportMsg = reportMsg+['\n\n------------ test report, Total:',str(i),' --------------','\n']
reportMsg = reportMsg+['status ','\t','count','\t','index','\n'] countKeys = counter.keys()
countKeys.sort()
for i,key in enumerate(countKeys):
statusColumn = key[len('count'):]+' '
statusColumn = statusColumn[:9]
countColumn = str(len(counter[key]))
if len(countColumn)<5:
countColumn = countColumn+' '
countColumn = countColumn[:5] reportMsg = reportMsg+[statusColumn,'\t',countColumn,'\t',str(counter[key]),'\n'] # reportMsg = reportMsg+['ok','\t',str(len(counter['countOk'])),'\t',str(counter['countOk']),'\n']
# reportMsg = reportMsg+['40*','\t',str(len(count400)),'\t',str(count400),'\n']
# reportMsg = reportMsg+['50*','\t',str(len(count500)),'\t',str(count500),'\n']
# reportMsg = reportMsg+['err','\t',str(len(countErr)),'\t',str(countErr),'\n']
# reportMsg = reportMsg+['??','\t',str(len(Exception)),'\t',str(Exception),'\n']
reportMsg = reportMsg+['---------------------------------------------------','\n']
reportMsg = reportMsg+['\n\n']
reportMsg = ''.join(reportMsg) printReport(reportMsg,reportFile)
if reportFile and isinstance(reportFile,file):
reportFile.close() params = sys.argv[1:]
fileIn,fileOut,outPutConfig = getCmdParams(params)
urls,option = getUrlList(fileIn)
startTest(urls,option,fileOut,outPutConfig)
URL配置文件（简单）
post http://www.wmzy.com/account/saveEduInfo
http://www.wmzy.com/api/school/getSchList?prov_filter=44&type_filter=0&diploma_filter=0&flag_filter=0&page=2&page_len=20&_=1529798214101
http://www.wmzy.com/api/school/getSchList?prov_filter=shenzhen&type_filter=0&diploma_filter=0&flag_filter=0&page=2&page_len=20&_=1529798214101
http://www.wmzy.com/api/school/3bzwryxx.html
http://www.wmzy.com/static/outer/js/aq_auth.js
http://cloud.tenxasdasdcent.com/deasdasdasdvelopera/aasdasdsk/asd42279/answer/63957
http://node-img.b0.upaiyun.com/gaokao/tvpPlayer.html?vid=z0692bc6wkb&auto=1&title=AI时代·如何完美定制你的志愿填报
absdad
URL配置文件（复杂）
{
'option':{
'headers':{
'Cache-Control':'no-cache',
'Cookie':'sessionid=s:7rn8WESdKKXcukeo03wQjTlw.QB7A4baInGRgBbS0BgqGf1Rz5TSwLlRc8MRyE6roeJ8;path=/'
}
},
'urls':[
'http://www.wmzy.com//index.html',
{
'method':'POST',
'url':"http://www.wmzy.com/account/getEduInfoUIConfig"
},
{
'method':'POST',
'url':"http://www.wmzy.com/account/saveEduInfo",
'data':{
'province_id': '',
'city_id': '',
'region_id': '',
'school_id': '5a9f64a8b97bd266633f28f9',
'enroll_year': 2017,
'enroll_type': 1
}
},
{
'method':'POST',
'url':'http://www.wmzy.com/account/ajaxLogin',
"headers":{
'X-Requested-With':'XMLHttpRequest'
},
'data':{
'account':'',
'password':'',
'forceBindeCard':'false'
}
},
{
'method':'POST',
'url':'http://www.wmzy.com/zhiyuan/score',
"headers":{
'X-Requested-With':'XMLHttpRequest'
},
'data':{
'prov':44,
'realScore':500,
'ty':'l',
'diploma_id':7,
'score_form':'scoreBox'
}
}
]
}
检测 web项目 404 500 状态的 页面的更多相关文章
- Spring MVC自定义403,404,500状态码返回页面
代码 HTTP状态码干货:http://tool.oschina.net/commons?type=5 import org.springframework.boot.web.servlet.erro ...
- hbase搭建web项目 报500错误 HTTP Status 500 - Unable to compile class for JSP
在昨天,用hbase做后台搭建web项目时,前边的进行的非常顺利,当运行时便 报错了,截图如下: 这是直接在jsp中接收参数报的错误,如果在servlet中,同样也是报500的错误,虽然显示的不太一样 ...
- HTTP 302 404 500 状态消息
1xx:信息 100 Continue 服务器仅接收到部分请求,但是一旦服务器并没有拒绝该请求,客户端应该继续发送其余的请求. 101 Switching Protocols 服务器转换协议:服务器将 ...
- web.xml 404 500 配置
web.xml <error-page> <error-code>404</error-code> <location>/error404.html&l ...
- 严重: Error in dependencyCheck java.io.IOException: invalid header field（tomcat启动成功可是訪问web项目404错误）
tomcat启动的时候出现 严重: Error in dependencyCheck java.io.IOException: invalid header field 而且tomcat也不自己主动r ...
- Java Web项目--使用Servlet生成一个页面
为了生成一个servlet对应的网页.我们需要新建一个web.xml,其中将会放置servlet的相关信息.web.xml文件放置在WebContent/WEB-INF/目录下.(我们在Eclipe中 ...
- Java Web项目--使用JSP生成一个页面
我们使用了servlet生成了一个网页,但是可以看到使用servlet生成网页必须将网页的内容全部嵌入到Java代码当中,不是很方便.所以有没有什么办法是将Java代码嵌入到html代码中,而不是像s ...
- 小型web项目的模块化(转)
背景 目前团队中新的 Web 项目基本都采用了 Vue 或 React ,加上 RN,这些都属于比较重量级的框架,然而对于小型 Web 页面,又显得过大.早期的一些项目则使用了较原始的 HTML ...
- 动静态web项目（三）
在Eclipse中将web项目分为了Dynamic Web Project和Static Web Project. 那么这两种有什么区别呢? 其实这里的Dynamic和Static是通过页面来区分的. ...
随机推荐
- centos命令安装
1.解决ifconfig命令失效:需要安装net-tools工具 yum install net-tools 2.免密码登录 (1)通过命令,产生公钥信息 ssh-keygen -t rsa 如果提示 ...
- dml并行
Enabling Parallel DMLA DML statement can be parallelized only if you have explicitly enabled paralle ...
- cocos2dx 开启控制台
打开 SimulatorWin.cpp 把这句启用: #define SIMULATOR_WITH_CONSOLE_AND_MENU 1 这句本来因为#define隐藏了
- linux 服务配置
1.基本的linux 服务器防火墙配置 2.配置之前如果需要将之前的所有规则清楚 iptables -F -------清除预设表filter中的所用规则链的规则 iptables -X ---- ...
- 洛谷 P2678 & [NOIP2015提高组] 跳石头
题目链接 https://www.luogu.org/problemnew/show/P2678 题目背景 一年一度的“跳石头”比赛又要开始了! 题目描述 这项比赛将在一条笔直的河道中进行,河道中分布 ...
- 抓取某网站信息时遇到的问题及解决 The character set provided in ContentType is invalid. Cannot read content as string using an invalid character set
var response = httpClient.SendAsync(requestMessage).Result; content = response.Content.ReadAsStringA ...
- [转] iphoneX、iphoneXS、iphoneXSMax、iphoneXR适配
基础知识 1. 关于iphoneX .iphoneXS.iphoneXSMax.iphoneXR机型的大小和像素 机型尺寸 注意:开发人员只需要记住开发尺寸 2. 屏幕组成 齐刘海(44px) + ...
- PHP金钱数字转金钱大写
/* * 数字金额转换成中文大写金额的函数 * String Int $num 要转换的小写数字或小写字符串 * return 大写数字 */ function num_to_rmb($num){ $ ...
- 【JavaScript】 使用Async 和 Promise 完美解决回调地狱
很久以前就学习过Async和Promise,但总是一知半解的. 今天在写NodeJS的时候,发现好多第三方库使用回调,这样在实际操作中会出现多重回调,这就是传说中的JS回调地狱. 举个例子 有一个方法 ...
- JetBrains系IDE的设置Pycharm PHPStorm
一.换号边界线 File -> Settings ->Editor -> Code Style -> Right margin (columns) 二.代码自动完成快捷键 ...