Tornado之笔记集合
目录
一、基本使用
1、最简使用
import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world") application = tornado.web.Application([
(r"/index", MainHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py示例
2、settings
settings = {
'template_path': 'template', # html文件夹
'static_path': 'static', # 静态文件夹
'static_url_prefix': '/static/', # html里的静态文件夹路径前缀
'ui_methods': mt,
'ui_modules': md,
}
application = tornado.web.Application([
(r"/index", MainHandler),
], **settings) # 在这使用**settings
（1）html里静态文件使用方法
建议用这种
<link href="{{static_url("css/common.css")}}" rel="stylesheet" /> <script src="{{static_url("jquery-3.3.1.js")}}"></script> 或 <link href="/static/css/common.css" rel="stylesheet" /> <script src="/static/jquery-3.3.1.js"></script>
静态文件缓存的实现（这段不知道是什么，先记录下来）
def get_content_version(cls, abspath):
"""Returns a version string for the resource at the given path. This class method may be overridden by subclasses. The
default implementation is a hash of the file's contents. .. versionadded:: 3.1
"""
data = cls.get_content(abspath)
hasher = hashlib.md5()
if isinstance(data, bytes):
hasher.update(data)
else:
for chunk in data:
hasher.update(chunk)
return hasher.hexdigest()
静态文件缓存的实现
路由系统
1、基本使用
路由系统其实就是 url 和 类 的对应关系，这里不同于其他框架，其他很多框架均是 url 对应 函数，Tornado中每个url对应的是一个类
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world") application = tornado.web.Application([
(r"/index", MainHandler),
(r"/story/([0-9]+)", StoryHandler),
])
import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world") class StoryHandler(tornado.web.RequestHandler):
def get(self, story_id):
self.write("You requested the story " + story_id) class BuyHandler(tornado.web.RequestHandler):
def get(self):
self.write("buy.wupeiqi.com/index") application = tornado.web.Application([
(r"/index", MainHandler),
(r"/story/([0-9]+)", StoryHandler),
]) application.add_handlers('buy.wupeiqi.com$', [
(r'/index',BuyHandler),
]) if __name__ == "__main__":
application.listen(80)
tornado.ioloop.IOLoop.instance().start()
示例
2、其他
reverse_url
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from tornado.web import RequestHandler class MainHandler(RequestHandler):
def get(self):
url1 = self.application.reverse_url("n1")
print(url1)
url2 = self.application.reverse_url("n2",666)
print(url2)
self.write("Hello, world") class HomeHandler(RequestHandler):
def get(self,*args,**kwargs):
print(args)
print(kwargs)
self.write("welcome home") application = tornado.web.Application([
(r"/index", MainHandler,{},"n1"),
(r"/home/(?P<pk>\d+)", HomeHandler,{},"n2"),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start() '''
/index
/home/666
'''
app.py
视图函数
模版语言
1、基本使用
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render("index.html", list_info = [11,22,33]) application = tornado.web.Application([
(r"/index", MainHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
</head>
<body> <div>
<ul>
{% for item in list_info %}
<li>{{item}}</li>
{% end %}
</ul>
</div> </body>
</html>
index.html
在模板中默认提供了一些函数、字段、类以供模板使用： escape: tornado.escape.xhtml_escape 的別名
xhtml_escape: tornado.escape.xhtml_escape 的別名
url_escape: tornado.escape.url_escape 的別名
json_encode: tornado.escape.json_encode 的別名
squeeze: tornado.escape.squeeze 的別名
linkify: tornado.escape.linkify 的別名
datetime: Python 的 datetime 模组
handler: 当前的 RequestHandler 对象
request: handler.request 的別名
current_user: handler.current_user 的別名
locale: handler.locale 的別名
_: handler.locale.translate 的別名
static_url: for handler.static_url 的別名
xsrf_form_html: handler.xsrf_form_html 的別名
其他方法
2、for、if使用
{% for item in list_info %}
<li>{{item}}</li>
{% end %}
{% if item > 22 %}
<h1>{{item}}</h1>
{% else %}
<li>{{item}}</li>
{% end %}
3、母版、插件
（1）母版
用法与django基本一致
{% extends 'xxxxxxxx.html'%}
{% block xxx %}{% end %}
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>老男孩</title>
{% block CSS %}{% end %}
</head>
<body> <div class="pg-header"> </div> {% block RenderBody %}{% end %} {% block JavaScript %}{% end %}
</body>
</html>
layout.html
{% extends 'layout.html'%}
{% block CSS %}
{% end %}
{% block RenderBody %}
<h1>Index</h1>
<ul>
{% for item in list_info %}
<li>{{item}}</li>
{% end %}
</ul>
{% end %}
{% block JavaScript %}
{% end %}
index.html
（2）插件
用法与django基本一致
{% include 'xxxxx.html' %}
<div>
<ul>
<li>1024</li>
<li>42区</li>
</ul>
</div>
header.html
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
</head>
<body>
<div class="pg-header">
{% include 'header.html' %}
</div>
</body>
</html>
index.html
4、自定义UIMethod、UIModule
类似于django的simply_tag
（1）自定义uimethods.py与uimodules.py。注意：这里的uimethods和uimodules只是py的文件名
def tab(self):
return 'UIMethod'
uimethods.py
#!/usr/bin/env python
# -*- coding:utf-8 -*-
from tornado.web import UIModule
from tornado import escape class custom(UIModule): def render(self, *args, **kwargs):
return escape.xhtml_escape('<h1>模版语言</h1>')
uimodules.py
（2）在主程序中注册settings
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from tornado.escape import linkify
import uimodules as md # 导入自定义模块
import uimethods as mt # 导入自定义模块 class MainHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html') settings = {
'template_path': 'template', # html文件夹
'static_path': 'static', # 静态文件夹
'static_url_prefix': '/static/',
'ui_methods': mt,
'ui_modules': md,
} application = tornado.web.Application([
(r"/index", MainHandler),
], **settings) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py
（3）在html上使用
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
<link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
<h1>hello</h1>
{% module custom(123) %}
{{ tab() }}
</body>
index.html
cookie
self.set_cookie("mycookie", "myvalue")
self.get_cookie("xxx")
self.set_secure_cookie("mycookie", "myvalue")
self.get_secure_cookie("xxx")
expires使用时间戳
import time
expire_time = time.time() + 300 # 时间戳
self.set_cookie("mycookie","myvalue",expires=expire_time)
1、基本使用
import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
self.set_cookie('test_cookie',"helloworld")
self.write("Hello, world") application = tornado.web.Application([
(r"/index", MainHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py
2、加密cookie（签名）
Cookie 很容易被恶意的客户端伪造。加入你想在 cookie 中保存当前登陆用户的 id 之类的信息，你需要对 cookie 作签名以防止伪造。Tornado 通过 set_secure_cookie 和 get_secure_cookie 方法直接支持了这种功能。 要使用这些方法，你需要在创建应用时提供一个密钥，名字为 cookie_secret。 你可以把它作为一个关键词参数传入应用的设置中。
签名cookie的本质：
写cookie过程： 将值进行base64加密
对除值以外的内容进行签名，哈希算法（无法逆向解析）
拼接 签名 + 加密值
读cookie过程： 读取 签名 + 加密值
对签名进行验证
base64解密，获取值内容
import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
if not self.get_secure_cookie("mycookie"):
self.set_secure_cookie("mycookie", "myvalue")
self.write("Your cookie was not set yet!")
else:
self.write("Your cookie was set!") application = tornado.web.Application([
(r"/index", MainHandler),
], cookie_secret="61oETzKXQAGaYdkL5gEmGeJJFuYh7EQnp2XdTP1o/Vo=") if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py
def _create_signature_v1(secret, *parts):
hash = hmac.new(utf8(secret), digestmod=hashlib.sha1)
for part in parts:
hash.update(utf8(part))
return utf8(hash.hexdigest()) # 加密
def _create_signature_v2(secret, s):
hash = hmac.new(utf8(secret), digestmod=hashlib.sha256)
hash.update(utf8(s))
return utf8(hash.hexdigest()) def create_signed_value(secret, name, value, version=None, clock=None,
key_version=None):
if version is None:
version = DEFAULT_SIGNED_VALUE_VERSION
if clock is None:
clock = time.time timestamp = utf8(str(int(clock())))
value = base64.b64encode(utf8(value))
if version == 1:
signature = _create_signature_v1(secret, name, value, timestamp)
value = b"|".join([value, timestamp, signature])
return value
elif version == 2:
# The v2 format consists of a version number and a series of
# length-prefixed fields "%d:%s", the last of which is a
# signature, all separated by pipes. All numbers are in
# decimal format with no leading zeros. The signature is an
# HMAC-SHA256 of the whole string up to that point, including
# the final pipe.
#
# The fields are:
# - format version (i.e. 2; no length prefix)
# - key version (integer, default is 0)
# - timestamp (integer seconds since epoch)
# - name (not encoded; assumed to be ~alphanumeric)
# - value (base64-encoded)
# - signature (hex-encoded; no length prefix)
def format_field(s):
return utf8("%d:" % len(s)) + utf8(s)
to_sign = b"|".join([
b"",
format_field(str(key_version or 0)),
format_field(timestamp),
format_field(name),
format_field(value),
b'']) if isinstance(secret, dict):
assert key_version is not None, 'Key version must be set when sign key dict is used'
assert version >= 2, 'Version must be at least 2 for key version support'
secret = secret[key_version] signature = _create_signature_v2(secret, to_sign)
return to_sign + signature
else:
raise ValueError("Unsupported version %d" % version) # 解密
def _decode_signed_value_v1(secret, name, value, max_age_days, clock):
parts = utf8(value).split(b"|")
if len(parts) != 3:
return None
signature = _create_signature_v1(secret, name, parts[0], parts[1])
if not _time_independent_equals(parts[2], signature):
gen_log.warning("Invalid cookie signature %r", value)
return None
timestamp = int(parts[1])
if timestamp < clock() - max_age_days * 86400:
gen_log.warning("Expired cookie %r", value)
return None
if timestamp > clock() + 31 * 86400:
# _cookie_signature does not hash a delimiter between the
# parts of the cookie, so an attacker could transfer trailing
# digits from the payload to the timestamp without altering the
# signature. For backwards compatibility, sanity-check timestamp
# here instead of modifying _cookie_signature.
gen_log.warning("Cookie timestamp in future; possible tampering %r",
value)
return None
if parts[1].startswith(b""):
gen_log.warning("Tampered cookie %r", value)
return None
try:
return base64.b64decode(parts[0])
except Exception:
return None def _decode_fields_v2(value):
def _consume_field(s):
length, _, rest = s.partition(b':')
n = int(length)
field_value = rest[:n]
# In python 3, indexing bytes returns small integers; we must
# use a slice to get a byte string as in python 2.
if rest[n:n + 1] != b'|':
raise ValueError("malformed v2 signed value field")
rest = rest[n + 1:]
return field_value, rest rest = value[2:] # remove version number
key_version, rest = _consume_field(rest)
timestamp, rest = _consume_field(rest)
name_field, rest = _consume_field(rest)
value_field, passed_sig = _consume_field(rest)
return int(key_version), timestamp, name_field, value_field, passed_sig def _decode_signed_value_v2(secret, name, value, max_age_days, clock):
try:
key_version, timestamp, name_field, value_field, passed_sig = _decode_fields_v2(value)
except ValueError:
return None
signed_string = value[:-len(passed_sig)] if isinstance(secret, dict):
try:
secret = secret[key_version]
except KeyError:
return None expected_sig = _create_signature_v2(secret, signed_string)
if not _time_independent_equals(passed_sig, expected_sig):
return None
if name_field != utf8(name):
return None
timestamp = int(timestamp)
if timestamp < clock() - max_age_days * 86400:
# The signature has expired.
return None
try:
return base64.b64decode(value_field)
except Exception:
return None def get_signature_key_version(value):
value = utf8(value)
version = _get_version(value)
if version < 2:
return None
try:
key_version, _, _, _, _ = _decode_fields_v2(value)
except ValueError:
return None return key_version 内部算法
内部算法
CSRF
Tornado中的跨站请求伪造和Django中的相似
（Ajax使用时，本质上就是去获取本地的cookie，携带cookie再来发送请求）
settings = {
"xsrf_cookies": True,
}
application = tornado.web.Application([
(r"/", MainHandler),
(r"/login", LoginHandler),
], **settings)
app.py
<form action="/new_message" method="post">
{{ xsrf_form_html() }}
<input type="text" name="message"/>
<input type="submit" value="Post"/>
</form>
html使用form表单post
function getCookie(name) {
var r = document.cookie.match("\\b" + name + "=([^;]*)\\b");
return r ? r[1] : undefined;
}
jQuery.postJSON = function(url, args, callback) {
args._xsrf = getCookie("_xsrf");
$.ajax({url: url, data: $.param(args), dataType: "text", type: "POST",
success: function(response) {
callback(eval("(" + response + ")"));
}});
};
AJAX
文件上传
1、Form表单上传
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>上传文件</title>
</head>
<body>
<form id="my_form" name="form" action="/index" method="POST" enctype="multipart/form-data" >
<input name="fff" id="my_file" type="file" />
<input type="submit" value="提交" />
</form>
</body>
</html>
html
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self): self.render('index.html') def post(self, *args, **kwargs):
file_metas = self.request.files["fff"]
# print(file_metas)
for meta in file_metas:
file_name = meta['filename']
with open(file_name,'wb') as up:
up.write(meta['body']) settings = {
'template_path': 'template',
} application = tornado.web.Application([
(r"/index", MainHandler),
], **settings) if __name__ == "__main__":
application.listen(8000)
tornado.ioloop.IOLoop.instance().start() Python
app.py
2、AJAX上传
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
</head>
<body>
<input type="file" id="img" />
<input type="button" onclick="UploadFile();" />
<script>
function UploadFile(){
var fileObj = document.getElementById("img").files[0]; var form = new FormData();
form.append("k1", "v1");
form.append("fff", fileObj); var xhr = new XMLHttpRequest();
xhr.open("post", '/index', true);
xhr.send(form);
}
</script>
</body>
</html>
HTML - XMLHttpRequest
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
</head>
<body>
<input type="file" id="img" />
<input type="button" onclick="UploadFile();" />
<script>
function UploadFile(){
var fileObj = $("#img")[0].files[0];
var form = new FormData();
form.append("k1", "v1");
form.append("fff", fileObj); $.ajax({
type:'POST',
url: '/index',
data: form,
processData: false, // tell jQuery not to process the data
contentType: false, // tell jQuery not to set contentType
success: function(arg){
console.log(arg);
}
})
}
</script>
</body>
</html>
HTML - jQuery
<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
<title></title>
</head>
<body>
<form id="my_form" name="form" action="/index" method="POST" enctype="multipart/form-data" >
<div id="main">
<input name="fff" id="my_file" type="file" />
<input type="button" name="action" value="Upload" onclick="redirect()"/>
<iframe id='my_iframe' name='my_iframe' src="" class="hide"></iframe>
</div>
</form> <script>
function redirect(){
document.getElementById('my_iframe').onload = Testt;
document.getElementById('my_form').target = 'my_iframe';
document.getElementById('my_form').submit(); } function Testt(ths){
var t = $("#my_iframe").contents().find("body").text();
console.log(t);
}
</script>
</body>
</html>
HTML - iframe
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self): self.render('index.html') def post(self, *args, **kwargs):
file_metas = self.request.files["fff"]
# print(file_metas)
for meta in file_metas:
file_name = meta['filename']
with open(file_name,'wb') as up:
up.write(meta['body']) settings = {
'template_path': 'template',
} application = tornado.web.Application([
(r"/index", MainHandler),
], **settings) if __name__ == "__main__":
application.listen(8000)
tornado.ioloop.IOLoop.instance().start()
app.py
验证成功，略。
<script type="text/javascript">
$(document).ready(function () {
$("#formsubmit").click(function () {
var iframe = $('<iframe name="postiframe" id="postiframe" style="display: none"></iframe>');
$("body").append(iframe);
var form = $('#theuploadform');
form.attr("action", "/upload.aspx");
form.attr("method", "post");
form.attr("encoding", "multipart/form-data");
form.attr("enctype", "multipart/form-data");
form.attr("target", "postiframe");
form.attr("file", $('#userfile').val());
form.submit();
$("#postiframe").load(function () {
iframeContents = this.contentWindow.document.body.innerHTML;
$("#textarea").html(iframeContents);
});
return false;
});
});
</script>
<form id="theuploadform">
<input id="userfile" name="userfile" size="" type="file" />
<input id="formsubmit" type="submit" value="Send File" />
</form>
<div id="textarea">
</div>
扩展：基于iframe实现Ajax上传示例
$('#upload_iframe').load(function(){
var iframeContents = this.contentWindow.document.body.innerText;
iframeContents = JSON.parse(iframeContents);
})
function bindChangeAvatar1() {
$('#avatarImg').change(function () {
var file_obj = $(this)[0].files[0];
$('#prevViewImg')[0].src = window.URL.createObjectURL(file_obj)
})
}
function bindChangeAvatar2() {
$('#avatarImg').change(function () {
var file_obj = $(this)[0].files[0];
var reader = new FileReader();
reader.readAsDataURL(file_obj);
reader.onload = function (e) {
$('#previewImg')[0].src = this.result;
};
})
}
function bindChangeAvatar3() {
$('#avatarImg').change(function () {
var file_obj = $(this)[0].files[0];
var form = new FormData();
form.add('img_upload', file_obj);
$.ajax({
url: '',
data: form,
processData: false, // tell jQuery not to process the data
contentType: false, // tell jQuery not to set contentType
success: function (arg) {
}
})
})
}
function bindChangeAvatar4() {
$('#avatarImg').change(function () {
$(this).parent().submit();
$('#upload_iframe').load(function () {
var iframeContents = this.contentWindow.document.body.innerText;
iframeContents = JSON.parse(iframeContents);
if (iframeContents.status) {
$('#previewImg').attr('src', '/' + iframeContents.data);
}
})
})
}
其他
异步非阻塞
1、同步阻塞和异步非阻塞对比
装饰器 + Future 从而实现Tornado的异步非阻塞
当发送GET请求时，由于方法被@gen.coroutine装饰且yield 一个 Future对象，那么Tornado会等待，等待用户向future对象中放置数据或者发送信号，如果获取到数据或信号之后，就开始执行doing方法。
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from tornado.web import RequestHandler
import time class SyncHandler(RequestHandler): def get(self):
self.doing()
self.write('sync') def doing(self):
time.sleep(10) application = tornado.web.Application([
(r"^/$", SyncHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
同步阻塞
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from tornado.web import RequestHandler
import time
from tornado.web import gen,Future # class SyncHandler(RequestHandler):
#
# def get(self):
# self.doing()
# self.write('sync')
#
# def doing(self):
# time.sleep(10) class AsyncHandler(tornado.web.RequestHandler):
@gen.coroutine
def get(self):
future = Future()
tornado.ioloop.IOLoop.current().add_timeout(time.time() + 5, self.doing)
yield future def doing(self, *args, **kwargs):
self.write('async')
self.finish() application = tornado.web.Application([
# (r"^/$", SyncHandler),
(r"^/test/$", AsyncHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
异步非阻塞
2、httpclient类库
Tornado提供了httpclient类库用于发送Http请求，其配合Tornado的异步非阻塞使用。
import tornado.ioloop
import tornado.web class MainHandler(tornado.web.RequestHandler):
def get(self):
import requests
requests.get('http://www.google.com')
self.write('xxxxx') class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.write("Index")
application = tornado.web.Application([
(r"/main", MainHandler),
(r"/index", IndexHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
同步阻塞
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.web
from tornado import gen
from tornado import httpclient # 方式一：
class AsyncHandler(tornado.web.RequestHandler):
@gen.coroutine
def get(self, *args, **kwargs):
print('进入')
http = httpclient.AsyncHTTPClient()
data = yield http.fetch("http://www.google.com")
print('完事',data)
self.finish('') # 方式二：
# class AsyncHandler(tornado.web.RequestHandler):
# @gen.coroutine
# def get(self):
# print('进入')
# http = httpclient.AsyncHTTPClient()
# yield http.fetch("http://www.google.com", self.done)
#
# def done(self, response):
# print('完事')
# self.finish('666') application = tornado.web.Application([
(r"/async", AsyncHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
httpclient的异步非阻塞
引入future.set_result 当有异步io阻塞的时候 使用future.set_result可以终止阻塞
import tornado.ioloop
import tornado.web
from tornado import gen
from tornado.concurrent import Future future = None
class MainHandler(tornado.web.RequestHandler):
@gen.coroutine
def get(self):
global future
future = Future()
future.add_done_callback(self.done) yield future def done(self, *args, **kwargs):
self.write('Main')
self.finish() class IndexHandler(tornado.web.RequestHandler):
def get(self):
global future
future.set_result(None)
self.write("Index") application = tornado.web.Application([
(r"/main", MainHandler),
(r"/index", IndexHandler),
]) if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
set_result终止异步IO阻塞
3、自定义异步非阻塞web框架
http://www.cnblogs.com/wupeiqi/p/6536518.html#3957994
#!/usr/bin/env python
# -*- coding:utf-8 -*-
import re
import socket
import select
import time class HttpResponse(object):
"""
封装响应信息
"""
def __init__(self, content=''):
self.content = content self.headers = {}
self.cookies = {} def response(self):
return bytes(self.content, encoding='utf-8') class HttpNotFound(HttpResponse):
"""
404时的错误提示
"""
def __init__(self):
super(HttpNotFound, self).__init__('404 Not Found') class HttpRequest(object):
"""
用户封装用户请求信息
"""
def __init__(self, conn):
self.conn = conn self.header_bytes = bytes()
self.header_dict = {}
self.body_bytes = bytes() self.method = ""
self.url = ""
self.protocol = "" self.initialize()
self.initialize_headers() def initialize(self): header_flag = False
while True:
try:
received = self.conn.recv(8096)
except Exception as e:
received = None
if not received:
break
if header_flag:
self.body_bytes += received
continue
temp = received.split(b'\r\n\r\n', 1)
if len(temp) == 1:
self.header_bytes += temp
else:
h, b = temp
self.header_bytes += h
self.body_bytes += b
header_flag = True @property
def header_str(self):
return str(self.header_bytes, encoding='utf-8') def initialize_headers(self):
headers = self.header_str.split('\r\n')
first_line = headers[0].split(' ')
if len(first_line) == 3:
self.method, self.url, self.protocol = headers[0].split(' ')
for line in headers:
kv = line.split(':')
if len(kv) == 2:
k, v = kv
self.header_dict[k] = v class Future(object):
"""
异步非阻塞模式时封装回调函数以及是否准备就绪
"""
def __init__(self, callback):
self.callback = callback
self._ready = False
self.value = None def set_result(self, value=None):
self.value = value
self._ready = True @property
def ready(self):
return self._ready class TimeoutFuture(Future):
"""
异步非阻塞超时
"""
def __init__(self, timeout):
super(TimeoutFuture, self).__init__(callback=None)
self.timeout = timeout
self.start_time = time.time() @property
def ready(self):
current_time = time.time()
if current_time > self.start_time + self.timeout:
self._ready = True
return self._ready class Snow(object):
"""
微型Web框架类
"""
def __init__(self, routes):
self.routes = routes
self.inputs = set()
self.request = None
self.async_request_handler = {} def run(self, host='localhost', port=9999):
"""
事件循环
:param host:
:param port:
:return:
"""
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((host, port,))
sock.setblocking(False)
sock.listen(128)
sock.setblocking(0)
self.inputs.add(sock)
try:
while True:
readable_list, writeable_list, error_list = select.select(self.inputs, [], self.inputs,0.005)
for conn in readable_list:
if sock == conn:
client, address = conn.accept()
client.setblocking(False)
self.inputs.add(client)
else:
gen = self.process(conn)
if isinstance(gen, HttpResponse):
conn.sendall(gen.response())
self.inputs.remove(conn)
conn.close()
else:
yielded = next(gen)
self.async_request_handler[conn] = yielded
self.polling_callback() except Exception as e:
pass
finally:
sock.close() def polling_callback(self):
"""
遍历触发异步非阻塞的回调函数
:return:
"""
for conn in list(self.async_request_handler.keys()):
yielded = self.async_request_handler[conn]
if not yielded.ready:
continue
if yielded.callback:
ret = yielded.callback(self.request, yielded)
conn.sendall(ret.response())
self.inputs.remove(conn)
del self.async_request_handler[conn]
conn.close() def process(self, conn):
"""
处理路由系统以及执行函数
:param conn:
:return:
"""
self.request = HttpRequest(conn)
func = None
for route in self.routes:
if re.match(route[0], self.request.url):
func = route[1]
break
if not func:
return HttpNotFound()
else:
return func(self.request) snow.py
自定义异步非阻塞web框架
from snow import Snow
from snow import HttpResponse def index(request):
return HttpResponse('OK') routes = [
(r'/index/', index),
] app = Snow(routes)
app.run(port=8012)
基本使用
from snow import Snow
from snow import HttpResponse
from snow import TimeoutFuture request_list = [] def async(request):
obj = TimeoutFuture(5)
yield obj def home(request):
return HttpResponse('home') routes = [
(r'/home/', home),
(r'/async/', async),
] app = Snow(routes)
app.run(port=8012)
异步非阻塞：超时
from snow import Snow
from snow import HttpResponse
from snow import Future request_list = [] def callback(request, future):
return HttpResponse(future.value) def req(request):
obj = Future(callback=callback)
request_list.append(obj)
yield obj def stop(request):
obj = request_list[0]
del request_list[0]
obj.set_result('done')
return HttpResponse('stop') routes = [
(r'/req/', req),
(r'/stop/', stop),
] app = Snow(routes)
app.run(port=8012)
异步非阻塞：等待 基于等待模式可以完成自定制操作
RESTFUL
利用Tornado 的原生restfull，轻松的开发API
需求：Haproxy服务化，即：提供API可以对Ha配置文件进行获取，添加，删除
global
log 127.0.0.1 local2
daemon
maxconn 256
log 127.0.0.1 local2 info
defaults
log global
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
option dontlognull listen stats :8888
stats enable
stats uri /admin
stats auth admin:1234 frontend oldboy.org
bind 0.0.0.0:80
option httplog
option httpclose
option forwardfor
log global
acl www hdr_reg(host) -i www.oldboy.org
use_backend www.oldboy.net if www backend www.oldboy.org
server 10.1.70.9 10.1.70.9 weight 20 maxconn 3000 配置文件
配置文件
#!/usr/bin/env python
# -*- coding:utf-8 -*- from __future__ import with_statement
import threading
import json
import os
import commands
from config import settings class BaseResponse(object):
""" 封装返回值""" def __init__(self):
self.status = False
self.message = ''
self.data = '' class BaseController(object):
""" 操作基础类 """
rLock = threading.RLock() def __init__(self, origin):
self.config_path = settings['haproxy_config']
self.config_path_bak = settings['haproxy_config_bak']
self.origin = origin def get_request(self):
try:
post_data = self.origin
if not post_data or not post_data.strip():
raise Exception('parameters can not be empty')
data = json.loads(post_data)
return data
except Exception, e:
raise Exception('parameter is not valid') def check_post_data_format(self, post_dict): if not post_dict.has_key('rules') or not post_dict.has_key('backend'):
raise Exception('parameter is not valid, not contain rules and backend')
for item in post_dict['rules']:
if not item.has_key('server') or not item.has_key('weight') or not item.has_key('maxconn') or not item.has_key('port'):
raise Exception('parameter is not valid, not contain server or weight or maxconn or port') def check_post_data_format_part(self, post_dict): if not post_dict.has_key('rules') or not post_dict.has_key('backend'):
raise Exception('parameter is not valid, not contain rules and backend')
for item in post_dict['rules']:
if not item.has_key('server') or not item.has_key('port'):
raise Exception('parameter is not valid, not contain server or port') def file_block_list(self, frontend_title, backend_title):
file_backend_list = []
file_frontend_list = []
file_backend_flag = False
file_frontend_flag = False with open(self.config_path, 'r') as f:
for line in f:
if line.strip() == 'frontend %s' % (frontend_title,):
file_frontend_flag = True
file_frontend_list.append('frontend %s' % (frontend_title,))
continue
elif line.strip() == 'backend %s' % (backend_title,):
file_backend_list.append('backend %s' % (backend_title,))
file_backend_flag = True
continue
exec_line = line.expandtabs(8)
if exec_line.startswith('%s' % (' '*8,)):
if file_frontend_flag:
file_frontend_list.append(exec_line.strip('\n'))
continue
if file_backend_flag:
file_backend_list.append(exec_line.strip('\n'))
continue
else:
file_frontend_flag = False
file_backend_flag = False return [file_frontend_list, file_backend_list] def check(self, new_config_path):
output = commands.getoutput('haproxy -f %s -c' % (new_config_path,))
if output.strip() == 'Configuration file is valid':
return True
else:
raise Exception('failed to check new config：%s' % (output.strip())) def reload(self):
status, output = commands.getstatusoutput('/etc/init.d/haproxy reload')
if status == 0:
return 1
else:
return output def confirm(self, new_config):
os.rename(self.config_path, self.config_path_bak)
os.rename(new_config, self.config_path) def rollback(self):
temp = self.config_path + '.error'
os.rename(self.config_path, temp)
os.rename(self.config_path_bak, self.config_path) class GetController(BaseController): def execute(self, backend_title):
response = BaseResponse()
try:
if not backend_title or not backend_title.strip():
raise Exception('parameters can not be empty')
#frontend_title = backend_title[backend_title.index('.')+1:]
frontend_title = 'oldboy'
file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title)
response.data = {'frontend': file_frontend_list, 'backend': file_backend_list}
response.status = True
except Exception, e:
response.message = str(e)
return response class AddController(BaseController):
""" 新建配置功能类 """ def process_frontend(self, backend_title, frontend_title, file_frontend_list):
if not file_frontend_list:
file_frontend_list.append('frontend %s' % (frontend_title,))
file_frontend_list.append('%sbind 0.0.0.0:8000' % (" "*8,))
file_frontend_list.append('%soption httplog' % (" "*8,))
file_frontend_list.append('%soption httpclose' % (" "*8,))
file_frontend_list.append('%soption forwardfor' % (" "*8,))
file_frontend_list.append('%slog global' % (" "*8,)) #pre = backend_title[0:backend_title.index('.')]
acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title)
if not file_frontend_list.__contains__(acl):
file_frontend_list.append(acl)
use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title)
if not file_frontend_list.__contains__(use_backend):
file_frontend_list.append(use_backend) def process_backend(self, post_data, backend_title, file_backend_list):
if not file_backend_list:
file_backend_list.append('backend %s' % (backend_title,))
for item in post_data['rules']:
temp = "%sserver %s %s:%s weight %s maxconn %s" % (" "*8, item['server'], item['server'], item['port'], item['weight'], item['maxconn'], )
if not file_backend_list.__contains__(temp):
file_backend_list.append(temp) def write_config(self, file_frontend_list, file_backend_list, is_new_frontend, is_new_backend):
in_file_path = self.config_path
out_file_path = in_file_path + '.new' frontend_title = file_frontend_list[0]
backend_title = file_backend_list[0]
frontend_flag = False
backend_flag = False
temp_flag = True
with open(in_file_path) as infile, open(out_file_path, 'w') as outfile:
for line in infile:
line = line.expandtabs(8) if frontend_flag:
if temp_flag:
for item in file_frontend_list:
outfile.write(item+'\n')
temp_flag = False
if line.startswith(' '*8):
continue
else:
frontend_flag = False
temp_flag = True if backend_flag:
if temp_flag:
for item in file_backend_list:
outfile.write(item+'\n')
temp_flag = False
if line.startswith(' '*8):
continue
else:
backend_flag = False
temp_flag = True if line.strip() == frontend_title.strip():
frontend_flag = True
continue if line.strip() == backend_title.strip():
backend_flag = True
continue outfile.write(line) if is_new_frontend:
outfile.write('\n')
for item in file_frontend_list:
outfile.write(item+'\n')
if is_new_backend:
outfile.write('\n')
for item in file_backend_list:
outfile.write(item+'\n')
return out_file_path def execute(self):
response = BaseResponse()
BaseController.rLock.acquire()
try:
post_data = self.get_request() self.check_post_data_format(post_data) backend_title = post_data['backend']
#frontend_title = backend_title[backend_title.index('.')+1:]
frontend_title = 'oldboy' file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title) is_new_frontend = False if file_frontend_list else True
is_new_backend = False if file_backend_list else True self.process_frontend(backend_title, frontend_title, file_frontend_list)
self.process_backend(post_data, backend_title, file_backend_list)
new_config = self.write_config(file_frontend_list, file_backend_list, is_new_frontend, is_new_backend) if self.check(new_config):
self.confirm(new_config)
result = self.reload()
if result == 1:
response.status = True
response.message = 'success'
else:
self.rollback()
raise Exception(result) except Exception, e:
response.message = str(e)
finally:
BaseController.rLock.release()
return response class DelController(BaseController):
""" 删除配置功能类 """ def process_backend(self, post_data, file_backend_list):
for item in post_data['rules']:
temp = "server %s %s:%s" % (item['server'], item['server'], item['port'],)
'''
if file_backend_list.__contains__(temp):
del file_backend_list[file_backend_list.index(temp)]
'''
for rule in file_backend_list:
if rule.strip().startswith(temp):
del file_backend_list[file_backend_list.index(rule)] def process_frontend(self, backend_title, file_frontend_list):
#pre = backend_title[0:backend_title.index('.')] acl = "%sacl %s hdr_reg(host) -i %s" % (' '*8, backend_title, backend_title) if file_frontend_list.__contains__(acl):
del file_frontend_list[file_frontend_list.index(acl)] use_backend = "%suse_backend %s if %s" % (' '*8, backend_title, backend_title)
if file_frontend_list.__contains__(use_backend):
del file_frontend_list[file_frontend_list.index(use_backend)] def del_config(self, file_frontend_list, file_backend_list, is_del_backend):
in_file_path = self.config_path
out_file_path = in_file_path + '.new' frontend_title = file_frontend_list[0]
backend_title = file_backend_list[0] frontend_flag = False
backend_flag = False
temp_flag = True
with open(in_file_path) as infile, open(out_file_path, 'w') as outfile:
for line in infile:
line = line.expandtabs(8) if frontend_flag:
if temp_flag:
for item in file_frontend_list:
outfile.write(item+'\n')
temp_flag = False
if line.startswith(' '*8):
continue
else:
frontend_flag = False
temp_flag = True if backend_flag:
if temp_flag:
if not is_del_backend:
for item in file_backend_list:
outfile.write(item+'\n')
temp_flag = False
if line.startswith(' '*8):
continue
else:
backend_flag = False
temp_flag = True if line.strip() == frontend_title.strip():
frontend_flag = True
continue if line.strip() == backend_title.strip():
backend_flag = True
continue outfile.write(line)
return out_file_path def execute(self):
response = BaseResponse()
BaseController.rLock.acquire()
try:
post_data = self.get_request() self.check_post_data_format_part(post_data) backend_title = post_data['backend']
#frontend_title = backend_title[backend_title.index('.')+1:]
frontend_title = 'oldboy' file_frontend_list, file_backend_list = self.file_block_list(frontend_title, backend_title) if not file_frontend_list or not file_backend_list:
raise Exception('config not exist') self.process_backend(post_data, file_backend_list)
is_del_backend = False if len(file_backend_list) == 1:
self.process_frontend(backend_title, file_frontend_list)
is_del_backend = True new_config = self.del_config(file_frontend_list, file_backend_list, is_del_backend)
if self.check(new_config):
self.confirm(new_config)
result = self.reload()
if result == 1:
response.status = True
response.message = 'success'
else:
self.rollback()
raise Exception(result) except Exception, e:
response.message = e.message
finally:
BaseController.rLock.release() return response lib/commons.py
lib/common.py
settings = {
'haproxy_config': '/etc/haproxy/haproxy.cfg',
'haproxy_config_bak': '/etc/haproxy/haproxy.cfg.bak',
}
settings.py
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from lib import commons
import json class MainHandler(tornado.web.RequestHandler): def get(self):
backend_title = self.get_argument('origin', None)
obj = commons.GetController(backend_title)
response = obj.execute(backend_title)
self.write(json.dumps(response.__dict__)) def post(self, *args, **kwargs):
data = self.get_argument('origin', None)
obj = commons.AddController(data)
response = obj.execute()
self.write(json.dumps(response.__dict__)) def delete(self, *args, **kwargs):
data = self.get_argument('origin', None)
obj = commons.DelController(data)
response = obj.execute()
self.write(json.dumps(response.__dict__)) application = tornado.web.Application([
(r"/haproxy/", MainHandler),
(r"/haproxy", MainHandler),
]) if __name__ == "__main__":
application.listen(port=9999, address='0.0.0.0')
tornado.ioloop.IOLoop.instance().start()
app.py
自定义组件
1、验证码
http://www.cnblogs.com/fat39/p/8527489.html
2、自制session
3、自制form
（1）回忆Django的Form表单
回忆Django的Form 1、widget插件：input、textarea、choice、multiplychoice等
2、field字段：charfield、emailfield、floatfield等
3、单字段验证：clean_xxx
4、所有字段验证：clean
5、html字段：单字段输入框、错误信息、as_p等
（2）自己参考Tyrion实现一个最简单的form
# -*- coding:utf-8 -*- import copy
import re ###### widget ######
class TextInput(object):
def __init__(self,attrs={}):
self.attrs = attrs def __str__(self):
tmp = []
for name,value in self.attrs.items():
tmp.append("{0}='{1}'".format(name,value))
return "<input type='text' {0} />".format(" ".join(tmp)) ##### field ##### class Field(object):
DEFAULT_WIDGET = TextInput def __init__(self,widget=None):
self.widget = widget if widget else self.DEFAULT_WIDGET def __str__(self):
return str(self.widget) class CharField(Field):
REGEX = "^\w+@\w+$" def field_valid(self,value):
if re.match(self.REGEX,value):
return True
else:
return False ##### form ##### class BaseForm(object): def __init__(self,data=None):
self.data = data
self.errors = {}
self.initial() def initial(self):
self.kv = {}
for k,v in type(self).__dict__.items():
print(k,v)
if isinstance(v,Field):
new_v = copy.deepcopy(v)
setattr(self,k,new_v)
self.kv[k] = new_v def is_valid(self):
# print(self.data.get_argument)
flag = True
for k,v in self.kv.items():
errors_dict = {}
self.errors[k] = errors_dict
field_value = self.data.get_argument(k) result = v.field_valid(field_value)
if not field_value:
errors_dict["require"] = "not empty"
flag = False
if not result:
errors_dict["invalid"] = "invalid"
flag = False
return flag
form.py
#!/usr/bin/env python
# -*- coding:utf-8 -*- import tornado.ioloop
import tornado.web
from tornado.web import RequestHandler
from form import BaseForm
from form import CharField
from form import TextInput class MyForm(BaseForm):
user = CharField(widget=TextInput(attrs={"name":"user"}))
email = CharField(widget=TextInput(attrs={"name":"email"})) class LoginHandler(RequestHandler):
def get(self,*args,**kwargs):
userinfo = MyForm(self)
self.render("login.html",userinfo=userinfo) def post(self, *args, **kwargs):
userinfo = MyForm(self) if userinfo.is_valid():
self.write("haha")
else:
self.render("login.html", userinfo=userinfo) application = tornado.web.Application([
(r"/login.html", LoginHandler),
]) # application.reverse_url("n1") if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
app.py
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<h1>登录页面</h1>
<form action="" method="POST">
user:{% raw userinfo.user %} {{ userinfo.errors.get('user') }}
email:{% raw userinfo.email %}{{ userinfo.errors.get('email') }}
<input type="submit">
</form>
</body>
</html>
html
（3）Tyrion学习（推荐使用）
原note：http://www.cnblogs.com/wupeiqi/p/5938916.html
github：https://github.com/WuPeiqi/Tyrion
参考或转发
http://www.cnblogs.com/wupeiqi/articles/5702910.html
Tornado之笔记集合的更多相关文章
- tornado 学习笔记1 引言
从事软件开发这行业也快5年啦,其实从事的工作也不完全是软件开发,软件开发只是我工作中的一部分.其中包括课题研究.信息化方案设计.软件开发.信息系统监理.项目管理等工作,比较杂乱.开发的软件比较多,但是 ...
- Tornado学习笔记(一) helloword/多进程/启动参数
前言 当你觉得你过得很舒服的时候,你肯定没有在进步.所以我想学习新的东西,然后选择了Tornado.因为我觉得Tornado更匹配目前的我的综合素质. Tornado学习笔记系列主要参考<int ...
- JavaScript基础笔记集合（转）
JavaScript基础笔记集合 JavaScript基础笔记集合 js简介 js是脚本语言.浏览器是逐行的读取代码,而传统编程会在执行前进行编译 js存放的位置 html脚本必须放在&l ...
- Tornado 学习笔记13 TCPServer
为了实现TCPServer的功能,定义一个类用于继承TCPServer并实现handle_stream方法.HttpServer就是一个很好的例子. 13.1 构造函数 def __init ...
- Tornado学习笔记12 tornado.httpserver-.非阻塞的Http服务器
是一个非阻塞的,单线程的Http 服务器. 一般地,应用程序很少与HttpServer类直接交互,除非在进程开始时启动服务时(甚至在使用tornado.web.Applicaiton.listen时也 ...
- tornado 学习笔记7 RequestHandler功能分析
在第5部分讲到,构建一个tornado网站,必须包含一个或者多个handler,这些handler是RequestHandler的子类.每个请求都会被映射到handler中进行处理,处理 ...
- tornado 学习笔记6 Application 源码分析
Application 是Tornado重要的模块之一,主要是配置访问路由表及其他应用参数的设置. 源代码位于虚拟运行环境文件夹下(我的是env),具体位置为env > lib>sit-p ...
- tornado 学习笔记5 构建Tornado网站应用
一个Tornado 网站应用通常由一个或多个RequestHanlde的子类.一个负责将请求路由至handlers的Application以及一个启动服务器的main()函数等组成. 一个最小的“he ...
- <代码整洁之道>、<java与模式>、<head first设计模式>读书笔记集合
一.前言 几个月前的看书笔记 ...
随机推荐
- node.js 连接 sql server 包括低版本的sqlserver 2000
利用tedious连接,github地址:https://github.com/tediousjs/tedious 废话不多时直接上代码. connection.js var Connection = ...
- K2 4.7 升级 数据库排序规则更改
介绍 在过去,K2没有指定安装过程中要在其数据库上使用的标准排序规则.然而,现在K2引入了标准排序规则,以便在之后使用(如果我没有错的话,它是在4.7). 因此, 问题出现在数据库的排序规则不是Lat ...
- Eclipse设置格式化每行字符的长度
Windows>>prefrence>>Java>>CodeStyle>>formatter>>edit>>line wrapp ...
- Linux中kafka部署和集群
1.下载kafka安装包kafka_2.12-1.1.0. tar -xzvf kafka_2.11-0.8.2.1.tgz #解压 mv kafka_2.11-0.8.2.1 /usr/local/ ...
- HTTP中常见的各种状态码详解及解决方案
总结了一些常见的http的状态码,以及常见的解决方案. 一.各范围内状态码大致含义 1xx:临时响应(Informational),需要请求者继续执行操作的状态代码,表示服务器正在接受请求. 2xx: ...
- 大数据入门第九天——MapReduce详解（六）MR其他补充
一.自定义in/outputFormat 1.需求 现有一些原始日志需要做增强解析处理,流程: 1. 从原始日志文件中读取数据 2. 根据日志中的一个URL字段到外部知识库中获取信息增强到原始日志 3 ...
- Android开发——代码中实现WAP方式联网
,移动和联通的WAP代理服务器都是10.0.0.172,电信的WAP代理服务器是10.0.0.200. 在Android系统中,对于获取手机的APN设置,需要通过ContentProvider来进行数 ...
- 5285: [Hnoi2018]寻宝游戏
5285: [Hnoi2018]寻宝游戏 链接 分析: 从下面依次确定运算符号,然后在确定的过程中,需要确定的位数会逐渐减少.比如最后有一个1,如果在从下往上确定了一个or 1,那么再往前可以随便选了 ...
- CodeForces 593D Happy Tree Party [LCA+并查集]
题意:给一棵树,每条边有一个权值,给两种操作,第一种是询问y向下整除从a到b的最短路径中每条边的权值后y的值,第二种是改变某条边的权值. 思路:y的最大值为1e18,最多除大于等于2的数不超过60次即 ...
- [HNOI2012]永无乡 线段树合并
[HNOI2012]永无乡 LG传送门 线段树合并练手题,写这篇博客只是为了给我的这篇文章找个板子题. 并查集维护连通性,对于不在同一个连通块内的合并操作每次直接合并两颗线段树,复杂度\(O(n \l ...