Running External Commands with subprocess

March 13, 2026 · 3 min read · Updated March 13, 2026 · intermediate

python stdlib subprocess processes

The subprocess module is your go-to tool for running external commands from Python. Whether you need to invoke a shell script, run a system utility, or orchestrate other programs, subprocess gives you the control you need. This guide covers practical patterns for everyday tasks.

When to Use subprocess

You should use subprocess when you need to:

Run system commands like git, docker, or curl
Execute shell scripts or batch files
Chain together command-line tools
Capture or pipe data between processes

Avoid subprocess when you can use Python built-ins. For file operations, use os or pathlib. For environment variables, use os.environ. Subprocess is for spawning new processes, not for in-process work.

Your First Command

The simplest way to run an external command is with subprocess.run():

import subprocess

result = subprocess.run(["ls", "-la"])
print(result.returncode)

This runs ls -la and waits for it to finish. The return code tells you if it succeeded (0) or failed (non-zero).

Capturing Output

Most of the time, you need to see what the command produced. Use capture_output=True to grab both stdout and stderr:

import subprocess

result = subprocess.run(
    ["python", "--version"],
    capture_output=True,
    text=True
)

print(result.stdout)   # Python 3.12.0\n
print(result.stderr)   # (usually empty)
print(result.returncode)  # 0

The text=True argument decodes the bytes to strings automatically. Without it, you get raw bytes.

Running Shell Commands

Sometimes you need shell features like variable expansion, pipes, or glob patterns. Use shell=True for these cases:

import subprocess

result = subprocess.run(
    "echo $HOME && ls *.py 2>/dev/null | wc -l",
    shell=True,
    capture_output=True,
    text=True
)
print(result.stdout)

Be careful with shell=True and user input. Always validate input to avoid shell injection vulnerabilities.

Handling Errors

Commands can fail. Use check=True to raise an exception when a command returns a non-zero exit code:

import subprocess

try:
    subprocess.run(
        ["git", "commit", "-m", "fix bug"],
        check=True,
        capture_output=True,
        text=True
    )
except subprocess.CalledProcessError as e:
    print(f"Command failed: {e.stderr}")

This is cleaner than checking return codes manually.

Working with Long-Running Processes

For commands that take a while, you might want to show progress or interact with them. Use Popen for full control:

import subprocess

proc = subprocess.Popen(
    ["tail", "-f", "/var/log/syslog"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True
)

try:
    for line in proc.stdout:
        print(line, end="")
except KeyboardInterrupt:
    proc.terminate()
    proc.wait()

The communicate() method is safer for sending input:

proc = subprocess.Popen(
    ["python"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True
)

stdout, stderr = proc.communicate(input="print(hello from python)")
print(stdout)

##Timeouts

Prevent commands from hanging forever by setting a timeout:

import subprocess

try:
    result = subprocess.run(
        ["sleep", "30"],
        timeout=5,
        capture_output=True,
        text=True
    )
except subprocess.TimeoutExpired:
    print("Command took too long and was killed")

Piping Data Between Commands

You can connect processes together using pipes:

import subprocess

# First process: generate data
proc1 = subprocess.Popen(
    ["echo", "line1\nline2\nline3"],
    stdout=subprocess.PIPE,
    text=True
)

# Second process: filter the data
proc2 = subprocess.Popen(
    ["grep", "line1"],
    stdin=proc1.stdout,
    stdout=subprocess.PIPE,
    text=True
)

proc1.stdout.close()  # Allow proc1 to receive SIGPIPE if proc2 exits
output = proc2.communicate()[0]
print(output)  # line1

Environment Variables and Working Directory

Sometimes you need to control the environment or location:

import subprocess

result = subprocess.run(
    ["python", "-c", "import os; print(os.getcwd())"],
    cwd="/tmp",
    env={"PYTHONPATH": "/opt/mylib"},
    capture_output=True,
    text=True
)
print(result.stdout)  # /tmp

Best Practices

Always use a list of arguments rather than a shell string when possible. This avoids shell injection and is more portable:

# Good
subprocess.run(["ls", "-la"])

# Avoid unless you need shell features
subprocess.run("ls -la", shell=True)

Set capture_output=True instead of manually redirecting stdout and stderr. It is shorter and clearer.

Use text=True for human-readable output. It handles encoding for you.

Check out the shlex module if you need to parse shell-like strings into argument lists:

import shlex
import subprocess

cmd = echo Hello World
args = shlex.split(cmd)
subprocess.run(args)

Getting Started

The subprocess module handles most external command needs. Start with run() for simple cases, add error handling with check=True, and reach for Popen when you need streaming or complex process management.