Python subprocess

Filed Under: Python

Python subprocess module provides easy functions that allow us to spawn a new process and get their return codes. This module was introduced in Python v2.4.

In this lesson, we will study various functions available with the subprocess module and how we can put them to use.

Python subprocess

Python subprocess module is a powerful tool and was introduced to replace various old modules/functions present in Python, like:

  • os.system
  • os.spawn and related functions
  • os.popen and related functions
  • popen2.*
  • commands.*

Please note that replacements were not as is and some modifications in the program were needed to move to the subprocess module usage. Let’s start our journey with the functions of this module.

Python subprocess call()

This function is used to run a command and get the return code of the command. Let’s quickly look at an example code snippet:


import subprocess
print(subprocess.call(["pwd", "-P"]))

The output will be:
python subprocess call

Let’s understand what happened in above program:

  • When an argument list is passed, the first argument is interpreted as the executable.
  • The parameters from second param onwards are treated as the command line arguments to the program.
  • We could also have done:
  • 
    import subprocess
    print(subprocess.call('ls -l', shell=True))
    

    With shell being True, call() function treats this as command completely and run it as is. The output would have shown all files and directories in current folder.

Note that in POSIX based systems, a 0 return code is for success and 1 to 255 are for anything else. These exit codes are interpreted by machine scripts to evaluate the events of success and failures.

Python subprocess run()

This function works just like the call method and is used to run a command and get the return code of the command. Let’s quickly look at an example code snippet:


import subprocess
print(subprocess.run(["pwd", "-P"]))

The output will be:
python subprocess run

Note that the run() function was added in Python 3.5. A clear difference between the run() and the call() function is that the call() function doesn’t supports the input and check parameters.

Python subprocess check_call()

This function works like call() function but if there was an error in running the specified command, it raises a CalledProcessError exception. Let’s quickly look at an example code snippet:


import subprocess
print(subprocess.check_call("false"))

The output will be:
Python subprocess example check call

We used the false command as it always return with an error return code.

Python subprocess check_output()

When we use the call() function to run a command, the output is bound to the parent process and is unretrievable for the calling program. We can use the check_output() function to capture the output for later usage. Let’s quickly look at an example code snippet:


import subprocess
output = subprocess.check_output(['ls', '-1'])
print("Output is {} bytes long.".format(len(output)))

The output will be:
Python subprocess check output

Python subprocess communicate()

We can use the communicate() function in this Python module to read input and the output forom the process itself. stdout is the process output and stderr is populated in case of an error. Let’s quickly look at an example code snippet:


import subprocess
process = subprocess.Popen(
    ['cat', 'hello.py'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
print(stdout)

The output will be:
Python subprocess communicate

Note that we have a hello.py script which just says print("Hello").

Python subprocess Popen

Python subprocess Popen is used to execute a child program in a new process. We can use it to run some shell commands. Let’s look at Popen usage through simple example program.


import subprocess

process = subprocess.Popen(['ls', '-ltr'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

print(process.communicate())

process = subprocess.Popen(['echo', 'Pankaj'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=None)

print(process.communicate())

process = subprocess.Popen(['ping', '-c 1', 'journaldev.com'], stdout=subprocess.PIPE)

print(process.communicate())

When executed, it produces following output.


(b'total 8\n-rw-r--r--  1 pankaj  staff  396 Dec  9 22:00 subprocess_popen.py\n', b'')
(b'Pankaj\n', b'')
(b'PING journaldev.com (209.124.77.163): 56 data bytes\n64 bytes from 209.124.77.163: icmp_seq=0 ttl=53 time=474.153 ms\n\n--- journaldev.com ping statistics ---\n1 packets transmitted, 1 packets received, 0.0% packet loss\nround-trip min/avg/max/stddev = 474.153/474.153/474.153/0.000 ms\n', None)

In this lesson, we learned about various functions provided by subprocess module in Python and saw how they work.

Reference: API Doc

Comments

  1. artemisia says:

    this tutorial is awesome. Is there a way to show, or a link that shows, how you can take the output and put it back into the python code? For example, in the Python subprocess check_output() example, if you wanted to use that information about 725 bytes in the code again, what library/method would you use?

  2. Aswin says:

    This tutorial is simply awesome. Explains in details. Thank you and keep up the great work!

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages