Modules you should know in Python Libray

前两天被问到常用的python lib和module有哪些？最常用的那几个，其他的一下子竟然回答不上。想想也是，一般情况下，遇到一个问题，在网上一搜，顺着线索找到可用的例子，然后基本没有怎么深究。结果就导致了现在没有网络基本写不出代码的状况，搜索是程序员必须的技能，但是只依靠搜索例子是无法成长的。

今天抽出1个小时时间把一些重要的module的简要整理出来，逐步一个个熟悉。

0. Date Structures and Algorithms

abc

The abc module defines a metaclass and a pair of decorators for defining new abstract

base classes.

array 

This module defines an object type which can compactly represent an array of basic values: characters, integers, floating point numbers

The array module defines a new object type, array, that works almost exactly like a list, except that its contents are constrained to a single type.The type of an array is determined at the time of creation.

bisect

The bisect module provides support for keeping lists in sorted order. It uses a bisection algorithm to do most of its work.

This module provides support for maintaining a list in sorted order without having to sort the list after each insertion. For long lists of items with expensive comparison operations, this can be an improvement over the more common approach. The module is called 
bisect because it uses a basic bisection algorithm to do its work.

collections

The collections module contains high-performance implementations of a few useful container types, abstract base classes for various kinds of containers, and a utility function for creating name-tuple objects.

This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers, dict, list, set, and tuple.

namedtuple()	factory function for creating tuple subclasses with named fields	New in version 2.6.
deque	list-like container with fast appends and pops on either end	New in version 2.4.
Counter	dict subclass for counting hashable objects	New in version 2.7.
OrderedDict	dict subclass that remembers the order entries were added	New in version 2.7.
defaultdict	dict subclass that calls a factory function to supply missing values	New in version 2.5.

contextlib

The contextlib module provides a decorator and utility functions for creating context managers used in conjunction with the 
with statement.

functools

The functools module contains functions and decorators that are useful for creating higher-order functions, functional programming, and decorators.

The 
functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module.

heapq

The heapq module implements a priority queue using a heap.

This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm.

itertools

The itertools module contains functions for creating efficient iterators, useful for looping over data in various ways.

This module implements a number of 
iterator building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python.

operator

The operator module provides functions that access the built-in operators

The 
operator module exports a set of efficient functions corresponding to the intrinsic operators of Python. 

---

1. Files and Dictionary

bz2

The bz2 module is used to read and write data compressed according to the bzip2 compression algorithm.

filecmp

The filecmp module provides the functions, which can be used to compare files and directories.

fnmathch

The fnmatch module provides support for matching filenames using UNIX shell-style wildcard characters.This module only performs filename matching, whereas the glob module can be used to actually obtain file listings.

glob

The glob module returns all filenames in a directory that match a pattern specified using the rules of the UNIX shell (as described in the fnmatch module).

gzip

The gzip module provides a class, GzipFile, that can be used to read and write files compatible with the GNU gzip program. GzipFile objects work like ordinary files except that data is automatically compressed or decompressed.

shutil

The shutil module is used to perform high-level file operations such as copying, removing, and renaming.The functions in this module should only be used for proper files and directories. In particular, they do not work for special kinds of files on the file

system such as named pipes, block devices, etc. Also, be aware that these functions don't always correctly deal with advanced kinds of file metadata (e.g., resource forks, creator codes, etc.).

tarfile

The tarfile module is used to manipulate tar archive files. Using this module, it is possible to read and write tar files, with or without compression.

tempfile

The tempfile module is used to generate temporary filenames and files.

zipfile

The zipfile module is used to manipulate files encoded in the popular zip format

(originally known as PKZIP, although now supported by a wide variety of programs).

Zip files are widely used by Python, mainly for the purpose of packaging.

zlib

The zlib module supports data compression by providing access to the zlib library.

The zlib library is available at 
http://www.zlib.net. 

---

2. Operating System Services

commands

The commands module is used to execute simple system commands specified as a string and return their output as a string. It only works on UNIX systems.The functionality provided by this module is somewhat similar to using backquotes (`) in a UNIX shell script. For example, typing x = commands.getoutput('ls –l') is similar to saying x=`ls –l`.

ConfigParser 

The ConfigParser module (called configparser in Python 3) is used to read .ini format configuration files based on the Windows INI format.These files consist of named sections, each with its own variable assignments such as the following:

# A comment

; A comment

[section1]

name1 = value1

name2 = value2

[section2]

; Alternative syntax for assigning values

name1: value1

name2: value2

datetime 

The datetime module provides a variety of classes for representing and manipulating dates and times. Large parts of this module are simply related to different ways of creating and outputting date and time information. Other major features include mathematical operations such as comparisons and calculations of time deltas. Date manipulation is a complex subject, and readers would be strongly advised to consult Python's online documentation for an introductory background concerning the design of this module.

A 
date object represents a simple date consisting of a year, month, and day.The following four functions are used to create dates:

date(year, month, day)

time 

The time module provides various time-related functions. In Python, time is measured as the number of seconds since the epoch.The epoch is the beginning of time (the point at which time = 0 seconds).The epoch is January 1, 1970, on UNIX and can be determined by calling time.gmtime(0) on other systems.

errno 

The errno module defines symbolic names for the integer error codes returned by various

operating system calls, especially those found in the os and socket modules.These

codes are typically found in the errno attribute of an OSError or IOError exception.

fcntl

The fcntl module performs file and I/O control on UNIX file descriptors. File descriptors can be obtained using the fileno() method of a file or socket object.

io 

The io module implements classes for various forms of I/O as well as the built-in open() function that is used in Python 3.The module is also available for use in Python 2.6.

logging

The logging module provides a flexible facility for applications to log events, errors,

warnings, and debugging information.This information can be collected, filtered, written

to files, sent to the system log, and even sent over the network to remote machines.

This section covers the essential details of using this module for most common cases.

mmap

The mmap module provides support for a memory-mapped file object.This object behaves both like a file and a byte string and can be used in most places where an ordinary file or byte string is expected. Furthermore, the contents of a memory-mapped file are mutable.This means that modifications can be made using index-assignment and slice-assignment operators. Unless a private mapping of the file has been made, such changes directly alter the contents of the underlying file.

msvcrt

The msvcrt module provides access to a number of useful functions in the Microsoft Visual C runtime library.This module is available only on Windows.

optparse

The optparse module provides high-level support for processing UNIX-style command-line options supplied in sys.argv.

os

The os module provides a portable interface to common operating-system services. It does this by searching for an OS-dependent built-in module such as nt or posix and exporting the functions and data as found there. Unless otherwise noted, functions are available on Windows and UNIX. UNIX systems include both Linux and Mac OS X.

os.path

The os.path module is used to manipulate pathnames in a portable manner. It's imported by the os module.

signal

The signal module is used to write signal handlers in Python. Signals usually correspond to asynchronous events that are sent to a program due to the expiration of a timer, arrival of incoming data, or some action performed by a user.The signal interface emulates that of UNIX, although parts of the module are supported on other platforms.

subprocess

The subprocess module contains functions and objects that generalize the task of creating new processes, controlling input and output streams, and handling return codes. 

The module centralizes functionality contained in a variety of other modules such as os, popen2, and commands.

winreg

The winreg module (_winreg in Python 2) provides a low-level interface to the Windows registry.The registry is a large hierarchical tree in which each node is called a key.The children of a particular key are known as subkeys and may contain additional subkeys or values.

---

3. Thread and Concurrency

multiprocessing 

The multiprocessing module provides support for launching tasks in a subprocess, communicating and sharing data, and performing various forms of synchronization.The programming interface is meant to mimic the programming interface for threads in the threading module. However, unlike threads, it is important to emphasize that processes do not have any shared state.Thus, if a process modifies data, that change is local only to that process.

The features of the multiprocessing module are vast, making it one of the larger and most advanced built-in libraries.

processes

All of the features of the multiprocessing module are focused on processes.They are described by the following class

Process([group [, target [, name [, args [, kwargs]]]]])

A class that represents a task running in a subprocess.The arguments in the constructor should always been specified using keyword arguments. target is a callable object that will execute when the process starts, args is a tuple of positional arguments passed to target, and kwargs is a dictionary of keyword arguments passed to target. If args and kwargs are omitted, target is called with no arguments. name is a string that gives a descriptive name to the process. group is unused and is always set to None. Its presence here is simply to make the construction of a Process mimic the creation of a thread in the threading module.

threading 

The threading module provides a Thread class and a variety of synchronization primitives for writing multithreaded programs.

Thread Objects

The Thread class is used to represent a separate thread of control. A new thread can be created as follows:

Thread(group=None, target=None, name=None, args=(), kwargs={})

This creates a new Thread instance. group is None and is reserved for future extensions. target is a callable object invoked by the run() method when the thread starts. By default, it’s None, meaning that nothing is called. name is the thread name.

queue

The queue module (named Queue in Python 2) implements various multiproducer, multiconsumer queues that can be used to safely exchange information between multiple threads of execution.

The queue module defines three different queue classes:

Queue([maxsize])

Creates a FIFO (first-in first-out) queue. maxsize is the maximum number of items that can be placed in the queue. If maxsize omitted or 0, the queue size is infinite.

LifoQueue([maxsize])

Creates a LIFO (last-in, first-out) queue (also known as a stack).

PriorityQueue([maxsize])

Creates a priority queue in which items are ordered from lowest to highest priority.

When working with this queue, items should be tuples of the form (priority, data) where priority is a number.

---

4. Net Programming and Socket

asynchat

The asynchat module simplifies the implementation of applications that implement asynchronous networking using the asyncore module. It does this by wrapping the low-level I/O functionality of asyncore with a higher-level programming interface that is designed for network protocols based on simple request/response mechanisms (for example, HTTP).

asyncore

The 
asyncore module is used to build network applications in which network activity is handled asynchronously as a series of events dispatched by an event loop, built using the 
select() system call. Such an approach is useful in network programs that want to provide concurrency, but without the use of threads or processes.This method can also provide high performance for short transactions. All the functionality of this module is provided by the dispatcher class, which is a thin wrapper around an ordinary socket object.

select

The select module provides access to the select() and poll() system calls.

socket

The socket module provides access to the standard BSD socket interface. Although it's based on UNIX, this module is available on all platforms.The socket interface is designed to be generic and is capable of supporting a wide variety of networking protocols (Internet,TIPC, Bluetooth, and so on). However, the most common protocol is the Internet Protocol (IP), which includes both TCP and UDP. Python supports both IPv4 and IPv6, although IPv4 is far more common.

ssl

The ssl module is used to wrap socket objects with the Secure Sockets Layer (SSL), which provides data encryption and peer authentication. Python uses the OpenSSL library (http://www.openssl.org) to implement this module.

socketserver

This module is called socketserver in Python 3.The SocketServer module provides classes that simplify the implementation of TCP, UDP, and UNIX domain socket servers.

---

5. NETWORK

telnetlib 

The 
telnetlib module provides a 
Telnet class that implements the Telnet protocol.

imaplib

(NNTP protocol)

This module defines three classes, 
IMAP4, 
IMAP4_SSL and 
IMAP4_stream, which encapsulate a connection to an IMAP4 server and implement a large subset of the IMAP4rev1 client protocol as defined in

RFC 2060. It is backward compatible with IMAP4 (

RFC 1730) servers, but note that the 
STATUS command is not supported in IMAP4.

nntplib

(NNTP protocol) 

This module defines the class 
NNTP which implements the client side of the NNTP protocol. It can be used to implement a news reader or poster, or automated news processors. For more information on NNTP (Network News Transfer Protocol), see Internet

RFC 977.

poplib

This module defines a class, 
POP3, which encapsulates a connection to a POP3 server and implements the protocol as defined in

RFC 1725. 

smtpd

This module offers several classes to implement SMTP servers. One is a generic do-nothing implementation, which can be overridden, while the other two offer specific mail-sending strategies.