io module
The io module provides Python’s main facilities for handling various types of I/O. The three primary categories are text I/O, binary I/O, and raw I/O. The module supplies in-memory stream classes (StringIO, BytesIO) and the wrappers used behind the scenes when you call open(). It is part of the standard library and requires no installation.
Core Classes
StringIO
StringIO provides an in-memory text stream. It behaves like a file opened in text mode and is useful for building strings incrementally or for passing to APIs that expect a file-like object.
from io import StringIO
buf = StringIO()
buf.write("Hello, ")
buf.write("world!")
print(buf.getvalue())
# Hello, world!
buf.close()
You can also initialise it with existing content and read from it:
from io import StringIO
stream = StringIO("line one\nline two\nline three")
for line in stream:
print(line.strip())
# line one
# line two
# line three
BytesIO
BytesIO is the binary counterpart of StringIO. It operates on bytes objects and is essential when working with binary data such as images, serialised objects, or network payloads.
from io import BytesIO
buf = BytesIO()
buf.write(b"\x89PNG\r\n\x1a\n") # PNG magic bytes
buf.write(b"\x00\x00\x00\rIHDR")
print(buf.getvalue()[:8])
# b'\x89PNG\r\n\x1a\n'
A common use is reading binary content into a higher-level library without touching the filesystem:
from io import BytesIO
import json
data = {"key": "value"}
buf = BytesIO()
buf.write(json.dumps(data).encode("utf-8"))
buf.seek(0)
loaded = json.load(buf)
print(loaded)
# {'key': 'value'}
TextIOWrapper
TextIOWrapper wraps a binary stream and provides text-mode access with encoding and newline handling. This is the class behind file objects returned by open() in text mode.
from io import BytesIO, TextIOWrapper
raw = BytesIO(b"caf\xc3\xa9\n")
text_stream = TextIOWrapper(raw, encoding="utf-8")
print(text_stream.read())
# caf\u00e9
You can also use it to add encoding to sys.stdout:
from io import TextIOWrapper
import sys
# Re-wrap stdout with a specific encoding
wrapped = TextIOWrapper(sys.stdout.buffer, encoding="utf-8", line_buffering=True)
wrapped.write("Encoded output\n")
wrapped.detach() # prevent closing sys.stdout.buffer
Examples
Capturing print output
Redirect print() to a string for testing or logging:
from io import StringIO
import sys
capture = StringIO()
sys.stdout = capture
print("captured line one")
print("captured line two")
sys.stdout = sys.__stdout__
output = capture.getvalue()
print(repr(output))
# 'captured line one\ncaptured line two\n'
Using StringIO with csv
The csv module expects a file-like object. StringIO lets you parse CSV data from a string:
from io import StringIO
import csv
csv_data = "name,age\nAlice,30\nBob,25"
reader = csv.DictReader(StringIO(csv_data))
for row in reader:
print(row)
# {'name': 'Alice', 'age': '30'}
# {'name': 'Bob', 'age': '25'}
BytesIO with zipfile
Create a zip archive entirely in memory:
from io import BytesIO
import zipfile
buf = BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("readme.txt", "Hello from memory!")
buf.seek(0)
with zipfile.ZipFile(buf, "r") as zf:
print(zf.read("readme.txt").decode())
# Hello from memory!
Common Patterns
Context manager usage: StringIO and BytesIO support the context manager protocol, ensuring the buffer is closed after use:
from io import StringIO
with StringIO() as buf:
buf.write("temporary data")
result = buf.getvalue()
print(result)
# temporary data
Seeking and rewinding: After writing to a stream, you must call seek(0) before reading from the beginning. Forgetting this is a frequent source of bugs:
from io import BytesIO
buf = BytesIO()
buf.write(b"some data")
buf.seek(0) # rewind to start
content = buf.read()
print(content)
# b'some data'
Testing file-processing functions: Pass a StringIO or BytesIO instead of opening real files in unit tests:
from io import StringIO
def count_lines(f):
return sum(1 for _ in f)
fake_file = StringIO("a\nb\nc\n")
assert count_lines(fake_file) == 3