Python自动化【第八篇】：Python基础-Socket编程进阶

本节内容：

Socket语法及相关
SocketServer实现多并发

1.　　Socket语法及相关

　　sk = socket.socket(socket.AF_INET,socket.SOCK_STREAM,0)

　　参数一：地址簇

　　socket.AF_INET IPv4（默认）
　　socket.AF_INET6 IPv6

　　socket.AF_UNIX 只能够用于单一的Unix系统进程间通信

　　参数二：类型

　　socket.SOCK_STREAM　　流式socket , for TCP （默认）
　　socket.SOCK_DGRAM　　数据报式socket
, for UDP

　　socket.SOCK_RAW 原始套接字，普通的套接字无法处理ICMP、IGMP等网络报文，而SOCK_RAW可以；其次，SOCK_RAW也可以处理特殊的IPv4报文；此外，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP头。
　　socket.SOCK_RDM 是一种可靠的UDP形式，即保证交付数据报但不保证顺序。SOCK_RAM用来提供对原始协议的低级访问，在需要执行某些特殊操作时使用，如发送ICMP报文。SOCK_RAM通常仅限于高级用户或管理员运行的程序使用。
　　socket.SOCK_SEQPACKET 可靠的连续数据包服务

　　参数三：协议

　　0　　（默认）与特定的地址家族相关的协议,如果是 0 ，则系统就会根据地址格式和套接类别,自动选择一个合适的协议

sk.bind(address)　　将套接字绑定到地址。address地址的格式取决于地址族。在AF_INET下，以元组（host,port）的形式表示地址。
sk.listen(backlog)　　开始监听传入连接。backlog指定在拒绝连接之前，可以挂起的最大连接数量。backlog等于5，表示内核已经接到了连接请求，但服务器还没有调用accept进行处理的连接个数最大为5。这个值不能无限大，因为要在内核中维护连接队列。
sk.setblocking(bool)　　是否阻塞（默认True），如果设置False，那么accept和recv时一旦无数据，则报错。
sk.accept()　　接受连接并返回（conn,address）,其中conn是新的套接字对象，可以用来接收和发送数据。address是连接客户端的地址。接收TCP 客户的连接（阻塞式）等待连接的到来。
sk.connect(address)　　连接到address处的套接字。一般，address的格式为元组（hostname,port）,如果连接出错，返回socket.error错误。
sk.connect_ex(address)　　同上，只不过会有返回值，连接成功时返回 0 ，连接失败时候返回编码，例如：10061
sk.close()　　关闭套接字
sk.recv(bufsize[,flag])　　接受套接字的数据。数据以字符串形式返回，bufsize指定最多可以接收的数量。flag提供有关消息的其他信息，通常可以忽略。
sk.recvfrom(bufsize[.flag])　　与recv()类似，但返回值是（data,address）。其中data是包含接收数据的字符串，address是发送数据的套接字地址。
sk.send(string[,flag])　　将string中的数据发送到连接的套接字。返回值是要发送的字节数量，该数量可能小于string的字节大小。即：可能未将指定内容全部发送。
sk.sendall(string[,flag])　　将string中的数据发送到连接的套接字，但在返回之前会尝试发送所有数据。成功返回None，失败则抛出异常。内部通过递归调用send，将所有内容发送出去。
sk.sendto(string[,flag],address)　　将数据发送到套接字，address是形式为（ipaddr，port）的元组，指定远程地址。返回值是发送的字节数。该函数主要用于UDP协议。
sk.settimeout(timeout)　　设置套接字操作的超时期，timeout是一个浮点数，单位是秒。值为None表示没有超时期。一般，超时期应该在刚创建套接字时设置，因为它们可能用于连接的操作（如 client 连接最多等待5s ）
sk.getpeername()　　返回连接套接字的远程地址。返回值通常是元组（ipaddr,port）。
sk.getsockname()　　返回套接字自己的地址。通常是一个元组(ipaddr,port)
sk.fileno()　　套接字的文件描述符

2.　　SocketServer实现多并发

　　SocketServer内部使用 IO多路复用以及 “多线程” 和 “多进程” ，从而实现并发处理多个客户端请求的Socket服务端。

+------------+

| BaseServer |

+------------+

+-----------+ +------------------+

| TCPServer |------->| UnixStreamServer |

+-----------+ +------------------+

+-----------+ +--------------------+

| UDPServer |------->| UnixDatagramServer |

　　SocketServer简化了网络服务器的编写。它有4个类：TCPServer，UDPServer，UnixStreamServer，UnixDatagramServer。
　　这4个类是同步进行处理的，另外通过ForkingMixIn和ThreadingMixIn类来支持异步。

　　创建socketserver的步骤：

自己创建一个请求处理类，并且这个类要继承BaseRequestHandler,并且重写父类里的handle方法
必须实例化TCPserver，并且传递server ip和你上面创建的请求处理类给这个TCPserver
server.handle_request() #只处理一个请求（不建议用）;　　server.serve_forever() #处理多个请求，永远执行。

　　ThreadingTCPServer

　　ThreadingTCPServer实现的Soket服务器内部会为每个client创建一个 “线程”，该线程用来和客户端进行交互。

　　1）ThreadingTCPServer基础

　　使用ThreadingTCPServer:

import SocketServer
 
class MyTCPserver(SocketServer.BaseRequestHandler):
 
    def handle(self):
        pass
 
if __name__ == '__main__':
    server = SocketServer.ThreadingTCPServer(('127.0.0.1',5566), MyTCPserver)
    server.serve_forever()

Threading TCP server

　　2）ThreadingTCPServer源码剖析

　　ThreadingTCPServer的类图关系如下：

　　内部调用流程为：

启动服务端程序
执行 TCPServer.__init__ 方法，创建服务端Socket对象并绑定 IP 和端口
执行 BaseServer.__init__ 方法，将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给self.RequestHandlerClass
执行 BaseServer.server_forever 方法，While 循环一直监听是否有客户端请求到达 ...
当客户端连接到达服务器
执行 ThreadingMixIn.process_request 方法，创建一个 “线程” 用来处理请求
执行 ThreadingMixIn.process_request_thread 方法
执行 BaseServer.finish_request 方法，执行 self.RequestHandlerClass() 即：执行自定义 MyRequestHandler 的构造方法（自动调用基类BaseRequestHandler的构造方法，在该构造方法中又会调用 MyRequestHandler的handle方法）

　　ThreadingTCPServer相关源码：

　　BaseServer

class BaseServer:
 
    """Base class for server classes.
 
    Methods for the caller:
 
    - __init__(server_address, RequestHandlerClass)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you do not use serve_forever()
    - fileno() -> int   # for select()
 
    Methods that may be overridden:
 
    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - server_close()
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()
 
    Methods for derived classes:
 
    - finish_request(request, client_address)
 
    Class variables that may be overridden by derived classes or
    instances:
 
    - timeout
    - address_family
    - socket_type
    - allow_reuse_address
 
    Instance variables:
 
    - RequestHandlerClass
    - socket
 
    """
 
    timeout = None
 
    def __init__(self, server_address, RequestHandlerClass):
        """Constructor.  May be extended, do not override."""
        self.server_address = server_address
        self.RequestHandlerClass = RequestHandlerClass
        self.__is_shut_down = threading.Event()
        self.__shutdown_request = False
 
    def server_activate(self):
        """Called by constructor to activate the server.
 
        May be overridden.
 
        """
        pass
 
    def serve_forever(self, poll_interval=0.5):
        """Handle one request at a time until shutdown.
 
        Polls for shutdown every poll_interval seconds. Ignores
        self.timeout. If you need to do periodic tasks, do them in
        another thread.
        """
        self.__is_shut_down.clear()
        try:
            while not self.__shutdown_request:
                # XXX: Consider using another file descriptor or
                # connecting to the socket to wake this up instead of
                # polling. Polling reduces our responsiveness to a
                # shutdown request and wastes cpu at all other times.
                r, w, e = _eintr_retry(select.select, [self], [], [],
                                       poll_interval)
                if self in r:
                    self._handle_request_noblock()
        finally:
            self.__shutdown_request = False
            self.__is_shut_down.set()
 
    def shutdown(self):
        """Stops the serve_forever loop.
 
        Blocks until the loop has finished. This must be called while
        serve_forever() is running in another thread, or it will
        deadlock.
        """
        self.__shutdown_request = True
        self.__is_shut_down.wait()
 
    # The distinction between handling, getting, processing and
    # finishing a request is fairly arbitrary.  Remember:
    #
    # - handle_request() is the top-level call.  It calls
    #   select, get_request(), verify_request() and process_request()
    # - get_request() is different for stream or datagram sockets
    # - process_request() is the place that may fork a new process
    #   or create a new thread to finish the request
    # - finish_request() instantiates the request handler class;
    #   this constructor will handle the request all by itself
 
    def handle_request(self):
        """Handle one request, possibly blocking.
 
        Respects self.timeout.
        """
        # Support people who used socket.settimeout() to escape
        # handle_request before self.timeout was available.
        timeout = self.socket.gettimeout()
        if timeout is None:
            timeout = self.timeout
        elif self.timeout is not None:
            timeout = min(timeout, self.timeout)
        fd_sets = _eintr_retry(select.select, [self], [], [], timeout)
        if not fd_sets[0]:
            self.handle_timeout()
            return
        self._handle_request_noblock()
 
    def _handle_request_noblock(self):
        """Handle one request, without blocking.
 
        I assume that select.select has returned that the socket is
        readable before this function was called, so there should be
        no risk of blocking in get_request().
        """
        try:
            request, client_address = self.get_request()
        except socket.error:
            return
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except:
                self.handle_error(request, client_address)
                self.shutdown_request(request)
 
    def handle_timeout(self):
        """Called if no new request arrives within self.timeout.
 
        Overridden by ForkingMixIn.
        """
        pass
 
    def verify_request(self, request, client_address):
        """Verify the request.  May be overridden.
 
        Return True if we should proceed with this request.
 
        """
        return True
 
    def process_request(self, request, client_address):
        """Call finish_request.
 
        Overridden by ForkingMixIn and ThreadingMixIn.
 
        """
        self.finish_request(request, client_address)
        self.shutdown_request(request)
 
    def server_close(self):
        """Called to clean-up the server.
 
        May be overridden.
 
        """
        pass
 
    def finish_request(self, request, client_address):
        """Finish one request by instantiating RequestHandlerClass."""
        self.RequestHandlerClass(request, client_address, self)
 
    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        self.close_request(request)
 
    def close_request(self, request):
        """Called to clean up an individual request."""
        pass
 
    def handle_error(self, request, client_address):
        """Handle an error gracefully.  May be overridden.
 
        The default is to print a traceback and continue.
 
        """
        print
        '-' * 40
        print
        'Exception happened during processing of request from',
        print
        client_address
        import traceback
        traceback.print_exc()  # XXX But this goes to stderr!
        print
        '-' * 40

base server

　　TCPServer

class TCPServer(BaseServer):
 
    """Base class for various socket-based server classes.
 
    Defaults to synchronous IP stream (i.e., TCP).
 
    Methods for the caller:
 
    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you don't use serve_forever()
    - fileno() -> int   # for select()
 
    Methods that may be overridden:
 
    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()
 
    Methods for derived classes:
 
    - finish_request(request, client_address)
 
    Class variables that may be overridden by derived classes or
    instances:
 
    - timeout
    - address_family
    - socket_type
    - request_queue_size (only for stream sockets)
    - allow_reuse_address
 
    Instance variables:
 
    - server_address
    - RequestHandlerClass
    - socket
 
    """
 
    address_family = socket.AF_INET
 
    socket_type = socket.SOCK_STREAM
 
    request_queue_size = 5
 
    allow_reuse_address = False
 
    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):
        """Constructor.  May be extended, do not override."""
        BaseServer.__init__(self, server_address, RequestHandlerClass)
        self.socket = socket.socket(self.address_family,
                                    self.socket_type)
        if bind_and_activate:
            try:
                self.server_bind()
                self.server_activate()
            except:
                self.server_close()
                raise
 
    def server_bind(self):
        """Called by constructor to bind the socket.
 
        May be overridden.
 
        """
        if self.allow_reuse_address:
            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)
        self.server_address = self.socket.getsockname()
 
    def server_activate(self):
        """Called by constructor to activate the server.
 
        May be overridden.
 
        """
        self.socket.listen(self.request_queue_size)
 
    def server_close(self):
        """Called to clean-up the server.
 
        May be overridden.
 
        """
        self.socket.close()
 
    def fileno(self):
        """Return socket file number.
 
        Interface required by select().
 
        """
        return self.socket.fileno()
 
    def get_request(self):
        """Get the request and client address from the socket.
 
        May be overridden.
 
        """
        return self.socket.accept()
 
    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        try:
            # explicitly shutdown.  socket.close() merely releases
            # the socket and waits for GC to perform the actual close.
            request.shutdown(socket.SHUT_WR)
        except socket.error:
            pass  # some platforms may raise ENOTCONN here
        self.close_request(request)
 
    def close_request(self, request):
        """Called to clean up an individual request."""
        request.close()

TCP server

　　ThreadingMixIn

class ThreadingMixIn:
    """Mix-in class to handle each request in a new thread."""
 
    # Decides how threads will act upon termination of the
    # main process
    daemon_threads = False
 
    def process_request_thread(self, request, client_address):
        """Same as in BaseServer but as a thread.
 
        In addition, exception handling is done here.
 
        """
        try:
            self.finish_request(request, client_address)
            self.shutdown_request(request)
        except:
            self.handle_error(request, client_address)
            self.shutdown_request(request)
 
    def process_request(self, request, client_address):
        """Start a new thread to process the request."""
        t = threading.Thread(target=self.process_request_thread,
                             args=(request, client_address))
        t.daemon = self.daemon_threads
        t.start()

threading mix-in

　　ThreadingTCPServer

class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

threading TCP server

　　RequestHandler相关源码:

  BaseRequestHandler
 
class BaseRequestHandler:
 
    """Base class for request handler classes.
 
    This class is instantiated for each request to be handled.  The
    constructor sets the instance variables request, client_address
    and server, and then calls the handle() method.  To implement a
    specific service, all you need to do is to derive a class which
    defines a handle() method.
 
    The handle() method can find the request as self.request, the
    client address as self.client_address, and the server (in case it
    needs access to per-server information) as self.server.  Since a
    separate instance is created for each request, the handle() method
    can define arbitrary other instance variariables.
 
    """
 
    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
            self.handle()
        finally:
            self.finish()
 
    def setup(self):
        pass
 
    def handle(self):
        pass
 
    def finish(self):
        pass

request handler

　　SocketServer的ThreadingTCPServer之所以可以同时处理请求得益于 select 和 Threading 两个东西，其实本质上就是在服务器端为每一个客户端创建一个线程，当前线程用来处理对应客户端的请求，所以，可以支持同时n个客户端链接（长连接）。

　　socketserver 实例：

　　server side：

import socketserver
 
class MyTCPHandler(socketserver.BaseRequestHandler):
    """
    The request handler class for our server.
 
    It is instantiated once per connection to the server, and must
    override the handle() method to implement communication to the
    client.
    """
 
    def handle(self):
        # self.request is the TCP socket connected to the client
        self.data = self.request.recv(1024).strip()
        print("{} wrote:".format(self.client_address[0]))
        print(self.data)
        # just send back the same data, but upper-cased
        self.request.sendall(self.data.upper())
 
if __name__ == "__main__":
    HOST, PORT = "localhost", 9999
 
    # Create the server, binding to localhost on port 9999
    server = socketserver.ThreadingTCPServer((HOST, PORT), MyTCPHandler)
 
    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    server.serve_forever()

server side

　　client side：

import socket
import sys
 
HOST, PORT = "localhost", 9999
data = " ".join(sys.argv[1:])
 
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 
try:
    # Connect to server and send data
    sock.connect((HOST, PORT))
    sock.sendall(bytes(data + "\n", "utf-8"))
 
    # Receive data from the server and shut down
    received = str(sock.recv(1024), "utf-8")
finally:
    sock.close()
 
print("Sent:     {}".format(data))
print("Received: {}".format(received))

client side

Python自动化【第八篇】：Python基础-Socket编程进阶的更多相关文章

图解Python 【第八篇】：网络编程-进程、线程和协程
本节内容一览图: 本章内容: 同步和异步线程(线程锁.threading.Event.queue 队列.生产者消费者模型.自定义线程池) 进程(数据共享.进程池) 协程一.同步和异步你叫我去吃饭 ...
【python自动化第八篇：网络编程】
一.拾遗动态导入模块目的是为了在导入模块的过程中将模块以字符的格式导入. #!/usr/bin/env python # -*- coding:utf-8 -*- #Author:wanghui ...
【python自动化第十一篇】
[python自动化第十一篇:] 课程简介 gevent协程 select/poll/epoll/异步IO/事件驱动 RabbitMQ队列上节课回顾进程: 进程的诞生时为了处理多任务,资源的隔离, ...
Python之路,Day8 - Socket编程进阶
Python之路,Day8 - Socket编程进阶本节内容: Socket语法及相关 SocketServer实现多并发 Socket语法及相关 socket概念 socket本质上就是在2台 ...
python学习第八讲,python中的数据类型,列表,元祖,字典,之字典使用与介绍
目录 python学习第八讲,python中的数据类型,列表,元祖,字典,之字典使用与介绍.md 一丶字典 1.字典的定义 2.字典的使用. 3.字典的常用方法. python学习第八讲,python ...
进击的Python【第七章】：Python的高级应用（四）面向对象编程进阶
Python的高级应用(三)面向对象编程进阶本章学习要点: 面向对象高级语法部分静态方法.类方法.属性方法类的特殊方法反射异常处理 Socket开发基础一.面向对象高级语法部分静态方法 ...
python【第八篇】socket网络编程
内容大纲 1.socke基础两个程序通过一个双向的通信连接实现数据的交换,这个连接的一端称为一个socket. 建立网络通信连接至少要一对端口号(socket).socket本质是编程接口(API ...
Python开发【第八篇】：网络编程
Python之路[第六篇]:socket Socket socket通常也称作"套接字",用于描述IP地址和端口,是一个通信链的句柄,应用程序通常通过"套接字&quo ...
【python自动化第七篇：面向对象进阶】
知识点概览: 静态方法,类方法,属性方法类的特殊方法反射异常处理 socket开发基础一.静态方法:@staticmethod 只是名义上归类管理,实际上在静态方法里访问不了类或者实例中的任何 ...

随机推荐

OD调试2
通过视频学习,实现了一下简单的traceme爆破.这不是把它的序列号破译出来,只是识别了自己的号码,不算真正的爆破.(与期望有点差异) 先来看一下这款软件的逻辑结构. 先输入用户名以及序列号然后 ...
Struts2之过滤器和拦截器的区别
刚学习Struts2这个框架不久,心中依然有一个疑惑未解那就是过滤器和拦截器的区别,相信也有不少人跟我一样对于这个问题没有太多的深入了解那么下面我们就一起来探讨探讨过滤器,是在java web中, ...
Grand Central Dispatch（GCD）
GCD GCD是异步执行任务的技术之一. GCD使用很简洁的记述方法,实现了极为复杂繁琐的多线程编程. dispatch_async(queue, ^{ //长时间处理 ...
Insert Interval
在已经排好序的区间中,插入一个新的区间,与merge的做法类似 Given a set of non-overlapping intervals, insert a new interval into ...
不同vlan之间的相互访问
拓扑图: 用到的命令: 给端口的vlan <sw1>用户模式切换到系统模式 system-view 交换机名称 sysname swj1 创建vlan 3 端口模式选择int g0/0/ ...
如何让LinearLayout也有类似Button的点击效果？
有的时候,我们希望LinearLayout布局也有点击的效果,这时候我们不仅需要一个作为背景的selector,还要设置一些其它属性才行: android:clickable="true&q ...
jquery.cookie() 方法的使用(读取、写入、删除)
一个轻量级的cookie 插件,可以读取.写入.删除 cookie. jquery.cookie.js 的配置首先包含jQuery的库文件,在后面包含 jquery.cookie.js 的库文件. ...
SQL Server中常用的SQL语句（转）：
SQL Server中常用的SQL语句转自:http://www.cnblogs.com/rainman/archive/2013/05/04/3060428.html 1.概述名词笛卡尔积.主 ...
mysql 主从重新同步
mysql 主从同步一担出了问题之后,就会导致从库上的数据和主库不一样了.所以需要生新同步数据. 1.登录主库服务器,进入mysql,命令为:mysql -uroot -ppassword 2.执行: ...
jquery获得图片的真实大小
$(function(){ var imgSrc = $("#image").attr("src"); getImageWidth(imgSrc,functio ...

Python自动化 【第八篇】：Python基础-Socket编程进阶

Python自动化 【第八篇】：Python基础-Socket编程进阶的更多相关文章

随机推荐

热门专题

Python自动化【第八篇】：Python基础-Socket编程进阶

Python自动化【第八篇】：Python基础-Socket编程进阶的更多相关文章