Reading and Writing Files in Python

Most programs need to work with data that outlives a single execution. Files let you store data on disk and retrieve it later. Whether you’re saving user preferences, processing log files, or building a simple database, reading and writing files is a fundamental skill.

Python makes file handling straightforward with built-in functions and the pathlib module.

Opening Files

Before you can read or write a file, you need to open it. The open() function returns a file object that provides methods for interacting with the file.

# Open a file for reading (default mode is 'r')
file = open("example.txt", "r")

# Do something with the file
content = file.read()

# Always close the file when done
file.close()

That pattern works, but there’s a problem: if an error occurs before you call close(), the file stays open. Python provides a better way using the with statement:

with open("example.txt", "r") as file:
    content = file.read()
# File is automatically closed when you exit the with block

This is called a context manager. It guarantees the file closes properly, even if your code raises an exception. Always use with when working with files.

Reading Files

Once a file is open for reading, you have several options for getting its contents.

Reading the Entire File

The read() method loads the entire file into memory:

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

For small files, this is fine. For large files, reading everything at once can consume too much memory.

Reading Line by Line

The readline() method returns one line at a time:

with open("example.txt", "r") as file:
    line = file.readline()
    print(line)  # First line, including newline character
    print(file.readline())  # Second line

Reading All Lines

The readlines() method returns a list of all lines:

with open("example.txt", "r") as file:
    lines = file.readlines()
    for i, line in enumerate(lines):
        print(f"{i}: {line.strip()}")

Iterating Over a File

The most memory-efficient approach is to iterate directly over the file object:

with open("example.txt", "r") as file:
    for line in file:
        print(line.strip())

This reads one line at a time from disk, keeping memory usage low even for very large files.

Writing Files

To write to a file, open it with a write mode. Use "w" to overwrite the file or "a" to append to it.

Writing Text

# Create a new file or overwrite an existing one
with open("output.txt", "w") as file:
    file.write("Hello, World!\n")
    file.write("This is line two.")

The "w" mode creates the file if it doesn’t exist. If the file already exists, it truncates it to zero length before writing.

Appending to a File

Use "a" mode to add to the end of an existing file:

with open("log.txt", "a") as file:
    file.write("User logged in at 10:30\n")
    file.write("User downloaded a file at 10:35\n")

This preserves existing content and adds new content at the end.

Writing Multiple Lines

The writelines() method writes a list of strings:

lines = ["First line\n", "Second line\n", "Third line\n"]
with open("multi.txt", "w") as file:
    file.writelines(lines)

File Modes

The second argument to open() specifies the mode:

Mode	Description
"r"	Read (default)
"w"	Write (overwrites)
"a"	Append
"x"	Create new file, fail if exists
"r+"	Read and write
"rb"	Read binary
"wb"	Write binary

For text files, use the modes as shown. For binary files (images, audio, etc.), add "b" to the mode.

# Reading an image
with open("photo.jpg", "rb") as file:
    data = file.read()

Handling Errors

Files can fail to open for many reasons: the file doesn’t exist, you lack permissions, the disk is full, or the path is wrong. Use try and except to handle these cases gracefully:

try:
    with open("missing.txt", "r") as file:
        content = file.read()
except FileNotFoundError:
    print("The file doesn't exist.")
except PermissionError:
    print("You don't have permission to read this file.")

Reading and Writing JSON

A common pattern is storing structured data in JSON format. The json module makes this easy:

import json

# Data to save
data = {
    "name": "Alice",
    "scores": [95, 87, 92],
    "active": True
}

# Write JSON to a file
with open("data.json", "w") as file:
    json.dump(data, file, indent=2)

# Read JSON from a file
with open("data.json", "r") as file:
    loaded = json.load(file)

print(loaded["name"])  # Alice

The indent parameter formats the JSON with whitespace, making it readable. Omit it for compact storage.

Using pathlib (Modern Approach)

The pathlib module provides an object-oriented way to work with file paths. It’s included in Python 3.4+ and is often cleaner than the older os.path functions.

from pathlib import Path

# Read a file
content = Path("example.txt").read_text()
print(content)

# Write to a file
Path("output.txt").write_text("Hello from pathlib!")

# Check if file exists
if Path("example.txt").exists():
    print("File exists")

# List all files in a directory
for file in Path(".").iterdir():
    if file.is_file():
        print(file.name)

For reading and writing with explicit encoding control, use read_text() and write_text():

from pathlib import Path

# Write with specific encoding
Path("unicode.txt").write_text("Hello with émoji: 🎉", encoding="utf-8")

# Read with specific encoding
content = Path("unicode.txt").read_text(encoding="utf-8")

The pathlib approach is more readable for many common operations and handles path manipulation more elegantly.

Common Patterns

Copying a File

# Read entire file and write to new location
from pathlib import Path

source = Path("original.txt")
destination = Path("backup.txt")

destination.write_text(source.read_text())

Processing a File Line by Line

# Filter lines containing a keyword
with open("data.txt", "r") as file:
    for line in file:
        if "error" in line.lower():
            print(line.strip())

Reading a CSV File

The csv module handles comma-separated values:

import csv

with open("data.csv", "r", newline="") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)  # Each row is a list

Next Steps

You now know how to read and write files in Python. These skills are the foundation for data processing, configuration management, and building applications that persist data.

To continue learning, explore these topics:

• The os module for lower-level file operations
• The shutil module for copying and moving files
• Working with binary files and buffers
• The tempfile module for creating temporary files

The next tutorial covers error handling and exceptions, which will help you write more robust file-handling code.