Working with files in Python: read, write, and append

Working with files in Python comes up almost everywhere — log writers, config loaders, CSV importers, save systems for small tools. Python ships built-in support that covers the common cases without any extra library. This guide walks through the patterns you actually need: opening files in the right mode, reading them efficiently, writing or appending text and binary data, and recovering when things go wrong.

Opening files with open()

The open() function is the entry point. It takes a path and returns a file object. The second argument is the mode, which decides what the file object can do.

# Open a file for reading
file = open("example.txt", "r")

The mode characters combine direction (read, write, append) with format (text or binary):

Mode	Meaning
r	Read (default)
w	Write (overwrites existing content)
a	Append (adds to end of file)
rb	Read binary
wb	Write binary
r+	Read and write
w+	Read and write (overwrites)
a+	Read and append

Always pick the encoding explicitly. Python defaults to UTF-8 on most platforms, but Windows can still surprise you with cp1252.

with open("example.txt", "r", encoding="utf-8") as file:
    content = file.read()

Reading file contents

Python gives you three idioms for reading. Pick the one that matches your file size and the shape of the data.

Reading the entire file

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

The with statement closes the file when the block exits — even on exceptions. Use it.

Reading line by line

with open("example.txt", "r") as file:
    for line in file:
        print(line.strip())

Line-by-line iteration is memory-cheap. The file object is its own iterator, yielding one line at a time. This is the right pattern for log files, CSVs, or anything that doesn’t fit in RAM.

Reading all lines into a list

with open("example.txt", "r") as file:
    lines = file.readlines()

# Or more simply
with open("example.txt", "r") as file:
    lines = list(file)

readlines() keeps the trailing newline on each entry. Strip it if you don’t want it.

Reading a specific number of characters

with open("example.txt", "r") as file:
    first_100 = file.read(100)

Useful for sniffing the first few bytes (think: detecting a BOM or a header) or when chunk-processing.

Writing to files

Writing a single string

with open("output.txt", "w") as file:
    file.write("Hello, World!")

write() returns the number of characters written. Mode w truncates the file first, so existing content disappears.

Writing multiple lines

lines = ["First line\n", "Second line\n", "Third line\n"]

with open("output.txt", "w") as file:
    file.writelines(lines)

writelines() does not add newlines for you. Include them in the strings.

Appending to files

with open("output.txt", "a") as file:
    file.write("This gets appended\n")

Append mode is what you want for log files, audit trails, or anything where order matters and nothing should be lost.

Why use context managers?

The with statement handles cleanup automatically.

# Without context manager (not recommended)
file = open("example.txt", "r")
content = file.read()
file.close()  # Must remember to close!

# With context manager (recommended)
with open("example.txt", "r") as file:
    content = file.read()
# File automatically closed when exiting the with block

If you forget close(), the file stays open until garbage collection runs, which can leak file descriptors on long-running processes. Worse, write buffers may not flush, so data on disk is silently incomplete. The with block solves both problems.

Handling binary files

Images, audio, archives, and pickled objects need binary mode. The file object then yields bytes, not str.

# Reading binary data
with open("image.png", "rb") as file:
    data = file.read()

# Writing binary data
with open("copy.png", "wb") as file:
    file.write(data)

Reading a PNG in text mode corrupts the bytes the moment Python tries to decode them. Always pair binary data with rb or wb.

Working with file paths

The newer pathlib module is the modern way to build paths. It works across operating systems and reads more cleanly than string concatenation.

from pathlib import Path

# Using pathlib (recommended)
file_path = Path("data") / "output.txt"
with open(file_path, "w") as file:
    file.write("Hello!")

# Check if file exists
if file_path.exists():
    print("File exists")

# Get file size
print(f"Size: {file_path.stat().st_size} bytes")

For deeper coverage, see the pathlib guide and the Python virtual environments guide for isolating file-system-heavy projects.

Common patterns

Copying a file

with open("source.txt", "r") as source:
    with open("destination.txt", "w") as dest:
        dest.write(source.read())

For large files, switch to chunked reads or use shutil.copyfile() from the standard library.

Counting lines in a file

with open("example.txt", "r") as file:
    line_count = sum(1 for line in file)
print(f"Lines: {line_count}")

This streams. No matter how big the file gets, memory stays flat.

Reading CSV files

For comma-separated data, lean on the csv module. It handles quoting, escapes, and dialect quirks you would otherwise reinvent.

import csv

with open("data.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

Processing large files line by line

# Process a large log file without loading into memory
total_errors = 0
with open("server.log", "r") as file:
    for line in file:
        if "ERROR" in line:
            total_errors += 1
print(f"Found {total_errors} errors")

This pattern scales to multi-gigabyte logs. Each line lives in memory for only one iteration.

Error handling

File operations fail for a hundred reasons: missing path, locked file, full disk, wrong permission. Catch the specific exceptions and react.

try:
    with open("example.txt", "r") as file:
        content = file.read()
except FileNotFoundError:
    print("File does not exist")
except PermissionError:
    print("Permission denied")
except IOError as e:
    print(f"IO error: {e}")

Catch FileNotFoundError and PermissionError first because they are common and need different recovery (create the file, ask for elevation). The broader IOError/OSError catches the rest.

When to use these patterns

Use read() when the file is small enough to fit in memory and you need to scan or transform the whole thing.

Use line-by-line iteration when working with large files, streams, or anything you want to process in constant memory.

Use w mode when creating new files or replacing existing content. Remember that it truncates immediately.

Use a mode when logging or appending data to an existing file.

Use binary modes (rb, wb) for non-text files like images, PDFs, or pre-encoded payloads.

When not to use these patterns

Skip raw file handling when the data needs to be queried or updated frequently. A SQLite database or a key-value store handles concurrent reads and partial updates that plain files struggle with.

Don’t read whole large files into memory when you can stream them. The line-by-line idiom is almost always the right default for log analysis.

Don’t skip exception handling — file calls touch the operating system, and operating systems fail in ways your code cannot predict. A small try/except block is enough to turn a crash into a useful error message. Working with files in Python is straightforward once these patterns become reflex.