Run Shell Commands in Python

subprocess.call

This is the recommended way to run shell commands in Python compared with old-fashioned os module.

This is a realtime method, which means you can get the shell output on the fly, compared with following "subprocess.check_output" method, which collect all output in its return value.

This method return the return value of the command, for example:

ret = subprocess.call('ls -l')

where ret=0, while

ret = subprocess.call('cd aaa')

ret=2 when there isn't "aaa" subfolder under CWD.

For ease of use, write a shorthand function:

import subprocess
def run(cmd):
    ret = subprocess.call(cmd, shell=True)
    if ret != 0:
        sys.exit('Exec cmd %s error, return value: %s' %(cmd, str(ret)))

Then you can simply use "run(cmd)" as a shell interface. "run" print command stdout stderr to console stdout, and if there's something wrong during execution, we interrupt it.

subprocess.check_output

A more safe way to run shell command is using "check_output" function. If the return value if not 0, a exception raised, otherwise return the command output.

$ cat myrun.py
import subprocess
def run(cmd):
    return subprocess.check_output(cmd, shell=True)
$ python -i myrun.py
>>> ret = run('ls -l|grep donno')
>>> ret
'drwxr-xr-x 6 chad chad 4096 Jan 26 18:18 donno-0.1.10\n-rw-r--r-- 1 chad chad 8716 Jan 27 15:53 donno-0.1.10.tar.gz\n'
>>> ret = run('cd aaa')
/bin/sh: 1: cd: can't cd to aaa
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "shtest.py", line 3, in run
    return subprocess.check_output(cmd, shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'cd aaa' returned non-zero exit status 2

subprocess.Popen

If you want some more powerful tools, use this. You can't use pipe directly in this form. Instead, You have to use subprocess.PIPE:

>>> import subprocess
>>> lsres = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)
>>> grepres = subprocess.Popen(['grep', 'Do'], stdin=lsres.stdout, stdout=subprocess.PIPE)
>>> res = grepres.communicate()

communicate() interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate. It returns a tuple (stdoutdata, stderrdata).

Full solution of running shell command

If you want realtime output, while saving output and return code in variables, you should use Popen:

from subprocess import Popen, PIPE, STDOUT
cmd = 'vmstat 2 3'
cmd2 = 'exit 3'

p = Popen(cmd, close_fds=True, shell=True, stdout=PIPE, stderr=STDOUT)

line = ''
while p.poll() is None:
    out = p.stdout.read(1)
    if out=='\n':
        print(line)
        line = ''
    else:
        line = line + out
print('--------\nret is: %d' % p.returncode)

The Popen.stdout is a file object, so its "read(size)" method here means read every 1 byte.

"close_fds=True" is maybe unnecessary, but I keep it for safe.

os.system

If it's unnecessary to save command output, this is most convenient way. The output will output to console. You can use space and pipe in command:

>>> import os

>>> ret = os.system('ls -l|grep D')

And it will return after the command complete:

>>> ret = os.system('vmstat 3 3')

os.popen

Use this form if you want to save command output.

>>> retfile = os.popen('pwd')

>>> ret = retfile.read()

>>> ret

'/home/lichao\n'

>>> retfile

<open file 'pwd', mode 'r' at 0xb74aad30>

or write it more compact:

>>> result = os.popen('ls|grep enex').read()

Deprecated

• commands.getoutput()

• commands.getstatusoutput()