转：python获取linux系统及性能信息

原文：http://amitsaha.github.io/site/notes/articles/python_linux/article.html

In this article, we will explore the Python programming language as a tool to retrieve various information about a system running Linux. Let’s get started.

Which Python?

When I refer to Python, I am referring to CPython 2 (2.7 to be exact). I will mention it explicitly when the same code won’t work with CPython 3 (3.3) and provide the alternative code, explaining the differences. Just to make sure that you have CPython installed, type python orpython3 from the terminal and you should see the Python prompt displayed in your terminal.

Please note that all the programs have their first line as #!/usr/bin/env python meaning that, we want the Python interpreter to execute these scripts. Hence, if you make your script executable using chmod +x your-script.py, you can execute it using ./your-script.py (which is what you will see in this article).

Exploring the platform module

The platform module in the standard library has a number of functions which allow us to retrieve various system information. Let us start the Python interpreter and explore some of them, starting with the platform.uname() function:

>>> import platform
>>> platform.uname()
('Linux', 'fedora.echorand', '3.7.4-204.fc18.x86_64', '#1 SMP Wed Jan 23 16:44:29 UTC 2013', 'x86_64')

If you are aware of the uname command on Linux, you will recognize that this function is an interface of sorts to this command. On Python 2, it returns a tuple consisting of the system type (or Kernel type), hostname, version, release, machine hardware and processor information. You can access individual attributes using indices, like so:

>>> platform.uname()[0]
'Linux'

On Python 3, the function returns a named tuple:

>>> platform.uname()
 
uname_result(system='Linux', node='fedora.echorand',
release='3.7.4-204.fc18.x86_64', version='#1 SMP Wed Jan 23 16:44:29
UTC 2013', machine='x86_64', processor='x86_64')

Since the returned result is a named tuple, this makes it easy to refer to individual attributes by name rather than having to remember the indices, like so:

>>> platform.uname().system
'Linux'

The platform module also has direct interfaces to some of the above attributes, like so:

>>> platform.system()
'Linux'
 
>>> platform.release()
'3.7.4-204.fc18.x86_64'

The linux_distribution() function returns details about the Linux distribution you are on. For example, on a Fedora 18 system, this command returns the following information:

>>> platform.linux_distribution()
('Fedora', '18', 'Spherical Cow')

The result is returned as a tuple consisting of the distribution name, version and the code name. The distributions supported by your particular Python version can be obtained by printing the value of the _supported_dists attribute:

>>> platform._supported_dists
('SuSE', 'debian', 'fedora', 'redhat', 'centos', 'mandrake',
'mandriva', 'rocks', 'slackware', 'yellowdog', 'gentoo',
'UnitedLinux', 'turbolinux')

If your Linux distribution is not one of these (or a derivative of one of these), then you will likely not see any useful information from the above function call.

The final function from the platform module, we will look at is the architecture() function. When you call the function without any arguments, this function returns a tuple consisting of the bit architecture and the executable format of the Python executable, like so:

>>> platform.architecture()
('64bit', 'ELF')

On a 32-bit Linux system, you would see:

>>> platform.architecture()
('32bit', 'ELF')

You will get similar results if you specify any other executable on the system, like so:

>>> platform.architecture(executable='/usr/bin/ls')
('64bit', 'ELF')

You are encouraged to explore other functions of the platform module which among others, allow you to find the current Python version you are running. If you are keen to know how this module retrieves this information, the Lib/platform.py file in the Python source directory is where you should look into.

The os and sys modules are also of interest to retrieve certain system attributes such as the native byteorder. Next, we move beyond the Python standard library modules to explore some generic approaches to access the information on a Linux system made available via theproc and sysfs file systems. It is to be noted that the information made available via these filesystems will vary between various hardware architectures and hence you should keep that in mind while reading this article and also writing scripts which attempt to retrieve information from these files.

CPU Information

The file /proc/cpuinfo contains information about the processing units on your system. For example, here is a Python version of what the Linux command cat /proc/cpuinfo would do:

#! /usr/bin/env python
""" print out the /proc/cpuinfo
    file
"""
 
from __future__ import print_function
 
with open('/proc/cpuinfo') as f:
    for line in f:
        print(line.rstrip('\n'))

When you execute this program either using Python 2 or Python 3, you should see all the contents of /proc/cpuinfo dumped on your screen (In the above program, the rstrip() method removes the trailing newline character from the end of each line).

The next code listing uses the startswith() string method to display the models of your processing units:

#! /usr/bin/env python
 
""" Print the model of your
    processing units
 
"""
 
from __future__ import print_function
 
with open('/proc/cpuinfo') as f:
    for line in f:
        # Ignore the blank line separating the information between
        # details about two processing units
        if line.strip():
            if line.rstrip('\n').startswith('model name'):
                model_name = line.rstrip('\n').split(':')[1]
                print(model_name)

When you run this program, you should see the model names of each of your processing units. For example, here is what I see on my computer:

Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz

We have so far seen a couple of ways to find the architecture of the computer system we are on. To be technically correct, both those approaches actually report the architecture of the kernel your system is running. So, if your computer is actually a 64-bit computer, but is running a 32-bit kernel, then the above methods will report it as having a 32-bit architecture. To find the true architecture of the computer you can look for the lm flag in the list of flags in /proc/cpuinfo. The lm flag stands for long mode and is only present on computers with a 64-bit architecture. The next program shows how you can do this:

#! /usr/bin/env python
 
""" Find the real bit architecture
"""
 
from __future__ import print_function
 
with open('/proc/cpuinfo') as f:
    for line in f:
        # Ignore the blank line separating the information between
        # details about two processing units
        if line.strip():
            if line.rstrip('\n').startswith('flags') \
                    or line.rstrip('\n').startswith('Features'):
                if 'lm' in line.rstrip('\n').split():
                    print('64-bit')
                else:
                    print('32-bit')

As we have seen so far, it is possible to read the /proc/cpuinfo and use simple text processing techniques to read the data we are looking for. To make it friendlier for other programs to use this data, it is perhaps a better idea to make the contents of /proc/cpuinfoavailable as a standard data structure, such as a dictionary. The idea is simple: if you see the contents of this file, you will find that for each processing unit, there are a number of key, value pairs (in an earlier example, we printed the model name of the processor, here model name was a key). The information about different processing units are separated from each other by a blank line. It is simple to build a dictionary structure which has each of the processing unit’s data as keys. For each of the these keys, the value is all the information about the corresponding processing unit present in the file /proc/cpuinfo. The next listing shows how you can do so.

#!/usr/bin/env/ python
 
"""
/proc/cpuinfo as a Python dict
"""
from __future__ import print_function
from collections import OrderedDict
import pprint
 
def cpuinfo():
    ''' Return the information in /proc/cpuinfo
    as a dictionary in the following format:
    cpu_info['proc0']={...}
    cpu_info['proc1']={...}
 
    '''
 
    cpuinfo=OrderedDict()
    procinfo=OrderedDict()
 
    nprocs = 0
    with open('/proc/cpuinfo') as f:
        for line in f:
            if not line.strip():
                # end of one processor
                cpuinfo['proc%s' % nprocs] = procinfo
                nprocs=nprocs+1
                # Reset
                procinfo=OrderedDict()
            else:
                if len(line.split(':')) == 2:
                    procinfo[line.split(':')[0].strip()] = line.split(':')[1].strip()
                else:
                    procinfo[line.split(':')[0].strip()] = ''
 
    return cpuinfo
 
if __name__=='__main__':
    cpuinfo = cpuinfo()
    for processor in cpuinfo.keys():
        print(cpuinfo[processor]['model name'])

This code uses an OrderedDict (Ordered dictionary) instead of a usual dictionary so that the key and values are stored in the order which they are found in the file. Hence, the data for the first processing unit is followed by the data about the second processing unit and so on. If you call this function, it returns you a dictionary. The keys of dictionary are each processing unit with. You can then use to sieve for the information you are looking for (as demonstrated in the if __name__=='__main__' block). The above program when run will once again print the model name of each processing unit (as indicated by the statement print(cpuinfo[processor]['model name']):

Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz

Memory Information

Similar to /proc/cpuinfo, the file /proc/meminfo contains information about the main memory on your computer. The next program creates a dictionary from the contents of this file and dumps it.

#!/usr/bin/env python
 
from __future__ import print_function
from collections import OrderedDict
 
def meminfo():
    ''' Return the information in /proc/meminfo
    as a dictionary '''
    meminfo=OrderedDict()
 
    with open('/proc/meminfo') as f:
        for line in f:
            meminfo[line.split(':')[0]] = line.split(':')[1].strip()
    return meminfo
 
if __name__=='__main__':
    #print(meminfo())
 
    meminfo = meminfo()
    print('Total memory: {0}'.format(meminfo['MemTotal']))
    print('Free memory: {0}'.format(meminfo['MemFree']))

As earlier, you could also access any specific information you are looking for by using that as a key (shown in the if __name__==__main__block). When you execute the program, you should see an output similar to the following:

Total memory: 7897012 kB
Free memory: 249508 kB

Network Statistics

Next, we explore the network devices on our computer system. We will retrieve the network interfaces on the system and the data bytes sent and recieved by them since your system reboot. The /proc/net/dev file makes this information available. If you examine the contents of this file, you will notice that the first two lines contain header information - i.e. the first column of this file is the network interface name, the second and the third columns display information about the received and the transmitted bytes (such as total bytes sent, number of packets, errors, etc.). Our interest here is to extract the total data sent and recieved by the different network devices. The next listing shows how we can extract this information from /proc/net/dev:

#!/usr/bin/env python
from __future__ import print_function
from collections import namedtuple
 
def netdevs():
    ''' RX and TX bytes for each of the network devices '''
 
    with open('/proc/net/dev') as f:
        net_dump = f.readlines()
 
    device_data={}
    data = namedtuple('data',['rx','tx'])
    for line in net_dump[2:]:
        line = line.split(':')
        if line[0].strip() != 'lo':
            device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
                                                float(line[1].split()[8])/(1024.0*1024.0))
 
    return device_data
 
if __name__=='__main__':
 
    netdevs = netdevs()
    for dev in netdevs.keys():
        print('{0}: {1} MiB {2} MiB'.format(dev, netdevs[dev].rx, netdevs[dev].tx))

When you run the above program, the output should display your network devices along with the total recieved and transmitted data in MiB since your last reboot as shown below:

em1: 0.0 MiB 0.0 MiB
wlan0: 2651.40951061 MiB 183.173976898 MiB

You could probably couple this with a persistent data storage mechanism to write your own data usage monitoring program.

Processes

The /proc directory also contains a directory each for all the running processes. The directory names are the same as the process IDs for these processes. Hence, if you scan /proc for all directories which have digits as their names, you will have a list of process IDs of all the currently running processes. The function process_list() in the next listing returns a list with process IDs of all the currently running processes. The length of this list will hence be the total number of processes running on the system as you will see when you execute the above program.

#!/usr/bin/env python
"""
 List of all process IDs currently active
"""
 
from __future__ import print_function
import os
def process_list():
 
    pids = []
    for subdir in os.listdir('/proc'):
        if subdir.isdigit():
            pids.append(subdir)
 
    return pids
 
if __name__=='__main__':
 
    pids = process_list()
    print('Total number of running processes:: {0}'.format(len(pids)))

The above program when executed will show an output similar to:

Total number of running processes:: 229

Each of the process directories contain number of other files and directories which contain various information about the invoking command of the process, the shared libraries its using, and others.

Block devices

The next program lists all the block devices by reading from the sysfs virtual file system. The block devices on your system can be found in the /sys/block directory. Thus, you may have directories such as /sys/block/sda, /sys/block/sdb and so on. To find all such devices, we perform a scan of the /sys/block directory using a simple regular expression to express the block devices we are interested in finding.

#!/usr/bin/env python
 
"""
Read block device data from sysfs
"""
 
from __future__ import print_function
import glob
import re
import os
 
# Add any other device pattern to read from
dev_pattern = ['sd.*','mmcblk*']
 
def size(device):
    nr_sectors = open(device+'/size').read().rstrip('\n')
    sect_size = open(device+'/queue/hw_sector_size').read().rstrip('\n')
 
    # The sect_size is in bytes, so we convert it to GiB and then send it back
    return (float(nr_sectors)*float(sect_size))/(1024.0*1024.0*1024.0)
 
def detect_devs():
    for device in glob.glob('/sys/block/*'):
        for pattern in dev_pattern:
            if re.compile(pattern).match(os.path.basename(device)):
                print('Device:: {0}, Size:: {1} GiB'.format(device, size(device)))
 
if __name__=='__main__':
    detect_devs()

If you run this program, you will see output similar to as follows:

Device:: /sys/block/sda, Size:: 465.761741638 GiB
Device:: /sys/block/mmcblk0, Size:: 3.70703125 GiB

When I run the program, I had a SD memory card plugged in as well and hence you can see that the program detects it. You can extend this program to recognize other block devices (such as virtual hard disks) as well.

Building command line utilities

One ubiquitious part of all Linux command line utilities is that they allow the user to specify command line arguments to customise the default behavior of the program. The argparse module allows your program to have an interface similar to built-in Linux utilities. The next listing shows a program which retrieves all the users on your system and prints their login shells (using the pwd standard library module):

#!/usr/bin/env python
 
"""
Print all the users and their login shells
"""
 
from __future__ import print_function
import pwd
 
# Get the users from /etc/passwd
def getusers():
    users = pwd.getpwall()
    for user in users:
        print('{0}:{1}'.format(user.pw_name, user.pw_shell))
 
if __name__=='__main__':
    getusers()

When run the program above, it will print all the users on your system and their login shells.

Now, let us say that you want the program user to be able to choose whether he or she wants to see the system users (like daemon,apache). We will see a first use of the argparse module to implement this feature in by extending the previous listing as follows.

#!/usr/bin/env python
 
"""
Utility to play around with users and passwords on a Linux system
"""
 
from __future__ import print_function
import pwd
import argparse
import os
 
def read_login_defs():
 
    uid_min = None
    uid_max = None
 
    if os.path.exists('/etc/login.defs'):
        with open('/etc/login.defs') as f:
            login_data = f.readlines()
 
        for line in login_data:
            if line.startswith('UID_MIN'):
                uid_min = int(line.split()[1].strip())
 
            if line.startswith('UID_MAX'):
                uid_max = int(line.split()[1].strip())
 
    return uid_min, uid_max
 
# Get the users from /etc/passwd
def getusers(no_system=False):
 
    uid_min, uid_max = read_login_defs()
 
    if uid_min is None:
        uid_min = 1000
    if uid_max is None:
        uid_max = 60000
 
    users = pwd.getpwall()
    for user in users:
        if no_system:
            if user.pw_uid >= uid_min and user.pw_uid <= uid_max:
                print('{0}:{1}'.format(user.pw_name, user.pw_shell))
        else:
            print('{0}:{1}'.format(user.pw_name, user.pw_shell))
 
if __name__=='__main__':
 
    parser = argparse.ArgumentParser(description='User/Password Utility')
 
    parser.add_argument('--no-system', action='store_true',dest='no_system',
                        default = False, help='Specify to omit system users')
 
    args = parser.parse_args()
    getusers(args.no_system)

On executing the above program with the --help option, you will see a nice help message with the available options (and what they do):

$ ./getusers.py --help
usage: getusers.py [-h] [--no-system]
 
User/Password Utility
 
optional arguments:
  -h, --help   show this help message and exit
  --no-system  Specify to omit system users

An example invocation of the above program is as follows:

$ ./getusers.py --no-system
gene:/bin/bash

When you pass an invalid parameter, the program complains:

$ ./getusers.py --param
usage: getusers.py [-h] [--no-system]
getusers.py: error: unrecognized arguments: --param

Let us try to understand in brief how we used argparse in the above program. The statement: parser =argparse.ArgumentParser(description='User/Password Utility') creates a new ArgumentParser object with an optional description of what this program does.

Then, we add the arguments that we want the program to recognize using the add_argument() method in the next statement:parser.add_argument('--no-system', action='store_true', dest='no_system', default = False, help='Specify to omit systemusers'). The first argument to this method is the name of the option that the program user will supply as an argument while invoking the program, the next parameter action=store_true indicates that this is a boolean option. That is, its presence or absence affects the program behavior in some way. The dest parameter specifies the variable in which the value that the value of this option will be available to the program. If this option is not supplied by the user, the default value is False which is indicated by the parameter default = Falseand the last parameter is the help message that the program displays about this option. Finally, the arguments are parsed using theparse_args() method: args = parser.parse_args(). Once the parsing is done, the values of the options supplied by the user can be retrieved using the syntax args.option_dest, where option_dest is the dest variable that you specified while setting up the arguments. This statement: getusers(args.no_system) calls the getusers() function with the option value for no_system supplied by the user.

The next program shows how you can specify options which allow the user to specify non-boolean preferences to your program. This program is a rewrite of Listing 6, with the additional option to specify the network device you may be interested in.

#!/usr/bin/env python
from __future__ import print_function
from collections import namedtuple
import argparse
 
def netdevs(iface=None):
    ''' RX and TX bytes for each of the network devices '''
 
    with open('/proc/net/dev') as f:
        net_dump = f.readlines()
 
    device_data={}
    data = namedtuple('data',['rx','tx'])
    for line in net_dump[2:]:
        line = line.split(':')
        if not iface:
            if line[0].strip() != 'lo':
                device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
                                                    float(line[1].split()[8])/(1024.0*1024.0))
        else:
            if line[0].strip() == iface:
                device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
                                                    float(line[1].split()[8])/(1024.0*1024.0))
    return device_data
 
if __name__=='__main__':
 
    parser = argparse.ArgumentParser(description='Network Interface Usage Monitor')
    parser.add_argument('-i','--interface', dest='iface',
                        help='Network interface')
 
    args = parser.parse_args()
 
    netdevs = netdevs(iface = args.iface)
    for dev in netdevs.keys():
        print('{0}: {1} MiB {2} MiB'.format(dev, netdevs[dev].rx, netdevs[dev].tx))

When you execute the program without any arguments, it behaves exactly as the earlier version. However, you can also specify the network device you may be interested in. For example:

$ ./net_devs_2.py
 
em1: 0.0 MiB 0.0 MiB
wlan0: 146.099492073 MiB 12.9737148285 MiB
virbr1: 0.0 MiB 0.0 MiB
virbr1-nic: 0.0 MiB 0.0 MiB
 
$ ./net_devs_2.py  --help
usage: net_devs_2.py [-h] [-i IFACE]
 
Network Interface Usage Monitor
 
optional arguments:
  -h, --help            show this help message and exit
  -i IFACE, --interface IFACE
                        Network interface
 
$ ./net_devs_2.py  -i wlan0
wlan0: 146.100307465 MiB 12.9777050018 MiB

System-wide availability of your scripts

With the help of this article, you may have been able to write one or more useful scripts for yourself which you want to use everyday like any other Linux command. The easiest way to do is make this script executable and setup a BASH alias to this script. You could also remove the .py extension and place this file in a standard location such as /usr/local/sbin.

Other useful standard library modules

Besides the standard library modules we have already looked at in this article so far, there are number of other standard modules which may be useful: subprocess, ConfigParser, readline and curses.

What next?

At this stage, depending on your own experience with Python and exploring Linux internals, you may follow one of the following paths. If you have been writing a lot of shell scripts/command pipelines to explore various Linux internals, take a look at Python. If you wanted a easier way to write your own utility scripts for performing various tasks, take a look at Python. Lastly, if you have been using Python for programming of other kinds on Linux, have fun using Python for exploring Linux internals.

Resources

Python resources

System Information¶