web新特性 之 WebSocket

详情参见：你真的了解WebSocket吗？     WebSocket系列教程   HTML5新特性之WebSocket

　　WebSocket协议是基于TCP的一种新的协议。WebSocket最初在HTML5规范中被引用为TCP连接，作为基于TCP的套接字API的占位符。它实现了浏览器与服务器全双工(full-duplex)通信。其本质是保持TCP连接，在浏览器和服务端通过Socket进行通信。

　　WebSocket是一种用于在服务器和客户端之间实现高效的双向通信的机制，通过WebSocket，能够实现在一个HTTP连接上自由地双向收发消息。

　　服务端与客户端的连接不断开，实现全双工的操作。及服务端或是客户端都会给对方发送消息。

 WebSocket（内部还是socket）
	- 本质（magic string）魔法字符串，通过内部封装的通信加密规则，进行通信。
	- 应用
		- 连接
		- 验证【握手信息】（magic string）
		- 收发消息

一、启动服务端

import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('127.0.0.1', 8002))
sock.listen(5)
# 等待用户连接
conn, address = sock.accept()
...
...
...

启动Socket服务器后，等待用户【连接】，然后进行收发数据。

二、 客户端连接

　　客户端：浏览器 （必须有websocket包） 在浏览器中已经高度封装，url地址前引用标志:ws

#客户端 通过websocket 创建一个对象，完成的是三个操作！
#创建连接
#发送消息
#接收验证消息
<script type="text/javascript">
    var socket = new WebSocket("ws://127.0.0.1:8002/xxoo");
    ...
</script>

　　当客户端向服务端发送连接请求时，不仅连接还会发送【握手】信息，并等待服务端响应，至此连接才创建成功！

三、建立连接【握手】

操作对象：服务端， socket

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('127.0.0.1', 8002))
sock.listen(5)
# 获取客户端socket对象
conn, address = sock.accept()
# 获取客户端的【握手】信息
data = conn.recv(1024)
...
...
...
conn.send('响应【握手】信息')

　　连接成功，在请求的信息中有用的信息为：Sec-WebSocket-Key:***的键值对，用于验证服务端有没有websocket通信的功能。这个功能的验证就是判断服务端能不能按照归则对这段字符串进行加密【叫做握手信息】。加密之后的字符串【握手信息】返回给客户端进行解密，进行通信认证。

　　加密规则：hmac1,base64加密，+ magic string

请求【握手】信息为：

GET /chatsocket HTTP/1.1
Host: 127.0.0.1:8002
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://localhost:63342
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: mnwFxiOlctXFN/DeMt1Amg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
...
...

请求和响应的【握手】信息需要遵循规则：

　　从请求【握手】信息中提取 Sec-WebSocket-Key
　　利用magic_string 和 Sec-WebSocket-Key 进行hmac1加密，再进行base64加密
　　将加密结果响应给客户端
　　注：magic string为：258EAFA5-E914-47DA-95CA-C5AB0DC85B11

import socket
import base64
import hashlib

def get_headers(data):
    """
    将请求头格式化成字典
    :param data:
    :return:
    """
    header_dict = {}
    data = str(data, encoding='utf-8')

    for i in data.split('\r\n'):
        print(i)
    header, body = data.split('\r\n\r\n', 1)
    header_list = header.split('\r\n')
    for i in range(0, len(header_list)):
        if i == 0:
            if len(header_list[i].split(' ')) == 3:
                header_dict['method'], header_dict['url'], header_dict['protocol'] = header_list[i].split(' ')
        else:
            k, v = header_list[i].split(':', 1)
            header_dict[k] = v.strip()
    return header_dict

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('127.0.0.1', 8002))
sock.listen(5)

conn, address = sock.accept()
data = conn.recv(1024)
headers = get_headers(data) # 提取请求头信息
# 对请求头中的sec-websocket-key进行加密
response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \
      "Upgrade:websocket\r\n" \
      "Connection: Upgrade\r\n" \
      "Sec-WebSocket-Accept: %s\r\n" \
      "WebSocket-Location: ws://%s%s\r\n\r\n"
magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
value = headers['Sec-WebSocket-Key'] + magic_string
ac = base64.b64encode(hashlib.sha1(value.encode('utf-8')).digest())
response_str = response_tpl % (ac.decode('utf-8'), headers['Host'], headers['url'])
# 响应【握手】信息
conn.send(bytes(response_str, encoding='utf-8'))
...
...
...
    

提取Sec-WebSocket-Key值并加密：

四、客户端和服务端收发数据

　　进行通信交互的话，就需要浏览器向服务端发送数据：socket.send(),而服务端接收到的数据是加密类型的字节数据【因为websocket内部有封包机制】。只有对数据进行进行解包，才能正常显示浏览器发送的信息。
　　客户端和服务端传输数据时，需要对数据进行【封包】和【解包】。客户端的JavaScript类库已经封装【封包】和【解包】过程，但Socket服务端需要手动实现。

1、针对服务端来说，获取客户端发送的数据进行解包。

解包规则：(数据头+数据全部加密) 用第二个字节的后7位判断(做位运算获取！与127做与判断)
　　获取所有的数据值，每个字节都与mask进行与运算进行解码

解包详细过程：

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-------+-+-------------+-------------------------------+
|F|R|R|R| opcode|M| Payload len |    Extended payload length    |
|I|S|S|S|  (4)  |A|     (7)     |             (16/64)           |
|N|V|V|V|       |S|             |   (if payload len==126/127)   |
| |1|2|3|       |K|             |                               |
+-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
|     Extended payload length continued, if payload len == 127  |
+ - - - - - - - - - - - - - - - +-------------------------------+
|                               |Masking-key, if MASK set to 1  |
+-------------------------------+-------------------------------+
| Masking-key (continued)       |          Payload Data         |
+-------------------------------- - - - - - - - - - - - - - - - +
:                     Payload Data continued ...                :
+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
|                     Payload Data continued ...                |
+---------------------------------------------------------------+

官方文档详细讲解：

The MASK bit simply tells whether the message is encoded. Messages from the client must be masked, so your server should expect this to be 1. (In fact, section 5.1 of the spec says that your server must disconnect from a client if that client sends an unmasked message.) When sending a frame back to the client, do not mask it and do not set the mask bit. We'll explain masking later. Note: You have to mask messages even when using a secure socket.RSV1-3 can be ignored, they are for extensions.

The opcode field defines how to interpret the payload data: 0x0 for continuation, 0x1 for text (which is always encoded in UTF-8), 0x2 for binary, and other so-called "control codes" that will be discussed later. In this version of WebSockets, 0x3 to 0x7 and 0xB to 0xF have no meaning.

The FIN bit tells whether this is the last message in a series. If it's 0, then the server will keep listening for more parts of the message; otherwise, the server should consider the message delivered. More on this later.

Decoding Payload Length

To read the payload data, you must know when to stop reading. That's why the payload length is important to know. Unfortunately, this is somewhat complicated. To read it, follow these steps:

Read bits 9-15 (inclusive) and interpret that as an unsigned integer. If it's 125 or less, then that's the length; you're done. If it's 126, go to step 2. If it's 127, go to step 3.
Read the next 16 bits and interpret those as an unsigned integer. You're done.
Read the next 64 bits and interpret those as an unsigned integer (The most significant bit MUST be 0). You're done.
Reading and Unmasking the Data

If the MASK bit was set (and it should be, for client-to-server messages), read the next 4 octets (32 bits); this is the masking key. Once the payload length and masking key is decoded, you can go ahead and read that number of bytes from the socket. Let's call the data ENCODED, and the key MASK. To get DECODED, loop through the octets (bytes a.k.a. characters for text data) of ENCODED and XOR the octet with the (i modulo 4)th octet of MASK. In pseudo-code (that happens to be valid JavaScript):

var DECODED = "";
for (var i = 0; i < ENCODED.length; i++) {
    DECODED[i] = ENCODED[i] ^ MASK[i % 4];
}

Now you can figure out what DECODED means depending on your application.

info = conn.recv(8096) #接收到的数据

    payload_len = info[1] & 127
    if payload_len == 126:
        extend_payload_len = info[2:4]
        mask = info[4:8] #解密的钥匙
        decoded = info[8:] #加密的数据
    elif payload_len == 127:
        extend_payload_len = info[2:10]
        mask = info[10:14]
        decoded = info[14:]
    else:
        extend_payload_len = None
        mask = info[2:6]
        decoded = info[6:]
    #对每个加密的字节进行循环，与解密钥匙进行与运算
    bytes_list = bytearray()
    for i in range(len(decoded)):
        chunk = decoded[i] ^ mask[i % 4]
        bytes_list.append(chunk)
    body = str(bytes_list, encoding='utf-8')
    print(body)

基于Python实现解包过程（未实现长内容）

2、而发送数据还需要进行封包：

def send_msg(conn, msg_bytes):
    """
    WebSocket服务端向客户端发送消息
    :param conn: 客户端连接到服务器端的socket对象,即： conn,address = socket.accept()
    :param msg_bytes: 向客户端发送的字节
    :return:
    """
    import struct

    token = b"\x81"
    length = len(msg_bytes)
    if length < 126:
        token += struct.pack("B", length)
    elif length <= 0xFFFF:
        token += struct.pack("!BH", 126, length)
    else:
        token += struct.pack("!BQ", 127, length)

    msg = token + msg_bytes
    conn.send(msg)
    return True

封包方法

五、基于Python实现简单示例

a. 基于Python socket实现的WebSocket服务端：

#!/usr/bin/env python
# -*- coding:utf-8 -*-
import socket
import base64
import hashlib

def get_headers(data):
    """
    将请求头格式化成字典
    :param data:
    :return:
    """
    header_dict = {}
    data = str(data, encoding='utf-8')

    header, body = data.split('\r\n\r\n', 1)
    header_list = header.split('\r\n')
    for i in range(0, len(header_list)):
        if i == 0:
            if len(header_list[i].split(' ')) == 3:
                header_dict['method'], header_dict['url'], header_dict['protocol'] = header_list[i].split(' ')
        else:
            k, v = header_list[i].split(':', 1)
            header_dict[k] = v.strip()
    return header_dict

def send_msg(conn, msg_bytes):
    """
    WebSocket服务端向客户端发送消息
    :param conn: 客户端连接到服务器端的socket对象,即： conn,address = socket.accept()
    :param msg_bytes: 向客户端发送的字节
    :return:
    """
    import struct

    token = b"\x81"
    length = len(msg_bytes)
    if length < 126:
        token += struct.pack("B", length)
    elif length <= 0xFFFF:
        token += struct.pack("!BH", 126, length)
    else:
        token += struct.pack("!BQ", 127, length)

    msg = token + msg_bytes
    conn.send(msg)
    return True

def run():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind(('127.0.0.1', 8003))
    sock.listen(5)

    conn, address = sock.accept()
    data = conn.recv(1024)
    headers = get_headers(data)
    response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \
                   "Upgrade:websocket\r\n" \
                   "Connection:Upgrade\r\n" \
                   "Sec-WebSocket-Accept:%s\r\n" \
                   "WebSocket-Location:ws://%s%s\r\n\r\n"

    value = headers['Sec-WebSocket-Key'] + '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
    ac = base64.b64encode(hashlib.sha1(value.encode('utf-8')).digest())
    response_str = response_tpl % (ac.decode('utf-8'), headers['Host'], headers['url'])
    conn.send(bytes(response_str, encoding='utf-8'))

    while True:
        try:
            info = conn.recv(8096)
        except Exception as e:
            info = None
        if not info:
            break
        payload_len = info[1] & 127
        if payload_len == 126:
            extend_payload_len = info[2:4]
            mask = info[4:8]
            decoded = info[8:]
        elif payload_len == 127:
            extend_payload_len = info[2:10]
            mask = info[10:14]
            decoded = info[14:]
        else:
            extend_payload_len = None
            mask = info[2:6]
            decoded = info[6:]

        bytes_list = bytearray()
        for i in range(len(decoded)):
            chunk = decoded[i] ^ mask[i % 4]
            bytes_list.append(chunk)
        body = str(bytes_list, encoding='utf-8')
        send_msg(conn,body.encode('utf-8'))

    sock.close()

if __name__ == '__main__':
    run()

b. 利用JavaScript类库实现客户端

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
    <div>
        <input type="text" id="txt"/>
        <input type="button" id="btn" value="提交" onclick="sendMsg();"/>
        <input type="button" id="close" value="关闭连接" onclick="closeConn();"/>
    </div>
    <div id="content"></div>

<script type="text/javascript">
    var socket = new WebSocket("ws://127.0.0.1:8003/chatsocket");

    socket.onopen = function () {
        /* 与服务器端连接成功后，自动执行 */

        var newTag = document.createElement('div');
        newTag.innerHTML = "【连接成功】";
        document.getElementById('content').appendChild(newTag);
    };

    socket.onmessage = function (event) {
        /* 服务器端向客户端发送数据时，自动执行 */
        var response = event.data;
        var newTag = document.createElement('div');
        newTag.innerHTML = response;
        document.getElementById('content').appendChild(newTag);
    };

    socket.onclose = function (event) {
        /* 服务器端主动断开连接时，自动执行 */
        var newTag = document.createElement('div');
        newTag.innerHTML = "【关闭连接】";
        document.getElementById('content').appendChild(newTag);
    };

    function sendMsg() {
        var txt = document.getElementById('txt');
        socket.send(txt.value);
        txt.value = "";
    }
    function closeConn() {
        socket.close();
        var newTag = document.createElement('div');
        newTag.innerHTML = "【关闭连接】";
        document.getElementById('content').appendChild(newTag);
    }

</script>
</body>
</html>

六、基于Tornado框架实现Web聊天室

　　关于推送消息：Tornado原生支持websocket，我们可以借助Tornado框架内部封装的方法实现消息的推送，把接收到的消息发送给所有的客户。

　　Tornado是一个支持WebSocket的优秀框架，其内部原理正如1~5步骤描述，当然Tornado内部封装功能更加完整。

以下是基于Tornado实现的聊天室示例：

import uuid
import json
import tornado.ioloop
import tornado.web
import tornado.websocket

class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        self.render('index.html')

class ChatHandler(tornado.websocket.WebSocketHandler):
    # 用户存储当前聊天室用户
    waiters = set() # 每个人的socket对象# self
    # 用于存储历时消息
    messages = []

    def open(self):
        """
        客户端连接成功时，自动执行
        :return:
        """
        ChatHandler.waiters.add(self)
        uid = str(uuid.uuid4())
        self.write_message(uid)

        for msg in ChatHandler.messages:
            content = self.render_string('message.html', **msg)
            self.write_message(content)

    def on_message(self, message):
        """
        客户端连发送消息时，自动执行
        :param message:
        :return:
        """
        msg = json.loads(message)
        ChatHandler.messages.append(msg)

        for client in ChatHandler.waiters:
            content = client.render_string('message.html', **msg)
            client.write_message(content)

    def on_close(self):
        """
        客户端关闭连接时，，自动执行
        :return:
        """
        ChatHandler.waiters.remove(self)

def run():
    settings = {
        'template_path': 'templates',
        'static_path': 'static',
    }
    application = tornado.web.Application([
        (r"/", IndexHandler),
        (r"/chat", ChatHandler),
    ], **settings)
    application.listen(8888,'0.0.0.0')
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    run()

app.py

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Python聊天室</title>
</head>
<body>
    <div>
        <input type="text" id="txt"/>
        <input type="button" id="btn" value="提交" onclick="sendMsg();"/>
        <input type="button" id="close" value="关闭连接" onclick="closeConn();"/>
    </div>
    <div id="container" style="border: 1px solid #dddddd;margin: 20px;min-height: 500px;">

    </div>

    <script src="/static/jquery-1.12.4.js"></script>
    <script type="text/javascript">
        $(function () {
            wsUpdater.start();
        });

        var wsUpdater = {
            socket: null,
            uid: null,
            start: function() {
                var url = "ws://192.168.11.211:8888/chat";
                wsUpdater.socket = new WebSocket(url);
                wsUpdater.socket.onmessage = function(event) {
                    if(wsUpdater.uid){
                        wsUpdater.showMessage(event.data);
                    }else{
                        wsUpdater.uid = event.data;
                    }
                }
            },
            showMessage: function(content) {
                $('#container').append(content);
            }
        };

        function sendMsg() {
            var msg = {
                uid: wsUpdater.uid,
                message: $("#txt").val()
            };
            wsUpdater.socket.send(JSON.stringify(msg));
        }

</script>

</body>
</html>

index.html

<div style="border: 1px solid #dddddd;margin: 10px;">
    <div>游客{{uid}}</div>
    <div style="margin-left: 20px;">{{message}}</div>
</div>

message.html