Async File I/O with aiofiles in Python

Python’s built-in open() is blocking. When you call f.read() or f.write(), your asyncio event loop sits idle until the operating system finishes moving bytes to or from disk. For I/O-heavy applications that also manage network connections, this blocking behaviour undermines the concurrency you’re trying to build.

aiofiles addresses this by wrapping the standard file API in coroutines that delegate to a thread pool. The event loop stays free to handle other tasks while a worker thread moves your data.

Installing aiofiles

pip install aiofiles

The library requires Python 3.7 or later. It works with any asyncio event loop.

Reading and writing files

The core API mirrors Python’s built-in open() exactly. Files are opened with aiofiles.open(), and every method on the returned file object is a coroutine that must be awaited.

Here is a complete example that writes a file and then reads it back, demonstrating the full read-write cycle.

import aiofiles
import asyncio

async def main():
    # Writing to a file
    async with aiofiles.open('example.txt', 'w') as f:
        await f.write('Hello, async world!\n')
        await f.write('Second line.')

    # Reading the file back
    async with aiofiles.open('example.txt', 'r') as f:
        contents = await f.read()
    print(contents)

asyncio.run(main())

The async with block is not optional. It ensures the file handle is closed promptly, even if an exception occurs mid-operation. Without it, you risk leaking file descriptors.

Open files support all the usual methods as coroutines: read(), readline(), readlines(), write(), writelines(), seek(), tell(), flush(), and close(). Binary mode is important when working with non-text data. Opening in 'rb' returns bytes; using 'r' returns strings, which can cause encoding errors on binary input. Always match the mode to the data type you are handling.

async with aiofiles.open('data.bin', 'wb') as f:
    await f.write(b'\x00\x01\x02\x03')

async with aiofiles.open('data.bin', 'rb') as f:
    data = await f.read()
# data is bytes

Async iteration over lines

For large files, reading line by line keeps memory usage flat. The async for construct lets you process a file one line at a time without loading it entirely into RAM. This is especially useful for log files, CSVs, or any dataset that might exceed available memory.

async def count_lines(filepath):
    count = 0
    async with aiofiles.open(filepath) as f:
        async for line in f:
            count += 1
    return count

Each line is returned as a string, ready to be parsed, filtered, or counted. The loop fetches lines lazily from the thread pool, so memory stays constant regardless of file size.

Chunked reads for large files

When you need to process a binary file in fixed-size chunks, call read() with an explicit size argument. This avoids loading the entire file into memory at once and is the standard approach for handling large media files, database backups, or any data that does not fit comfortably in RAM.

The following example copies a file using a 64KB chunk size, which is a good balance between memory usage and syscall frequency on most systems.

async def process_large_file(src, dst, chunk_size=65536):
    async with aiofiles.open(src, 'rb') as fin, \
               aiofiles.open(dst, 'wb') as fout:
        while True:
            chunk = await fin.read(chunk_size)
            if not chunk:
                break
            await fout.write(chunk)

The critical part here is the if not chunk: break check. When read() returns an empty bytes object, you have reached end-of-file. Calling read() with no argument or a negative size reads to the end of the file in one call, which defeats the purpose of chunking for large files.

Parallel file operations

Because file operations are now non-blocking coroutines, you can process many files concurrently using asyncio.gather(). Each file operation runs in the thread pool while the event loop handles the others. This approach scales well when your per-file work is mostly I/O-bound, such as reading configuration files, processing uploads, or aggregating data from multiple sources.

import aiofiles
import asyncio

async def process_file(filepath):
    async with aiofiles.open(filepath, 'r') as f:
        contents = await f.read()
    lines = contents.split('\n')
    return filepath, len(lines)

async def main(filepaths):
    tasks = [process_file(p) for p in filepaths]
    results = await asyncio.gather(*tasks)
    for path, line_count in results:
        print(f"{path}: {line_count} lines")

asyncio.run(main(['file1.txt', 'file2.txt', 'file3.txt']))

If each file requires significant CPU time to process, the thread pool becomes a bottleneck and you should consider moving the computation to a process pool instead.

The aiofiles.os module

Beyond file contents, aiofiles provides async versions of many os module functions that deal with the file system. These are useful when you need to coordinate operations like checking whether a file exists before writing to it, renaming files as part of a pipeline, or creating directories alongside other async work. Each function is a coroutine that must be awaited.

import aiofiles.os

async def backup_file(src, dst):
    if await aiofiles.os.path.exists(dst):
        await aiofiles.os.remove(dst)
    await aiofiles.os.rename(src, dst)

Available functions include stat, rename, remove/unlink, mkdir, makedirs, rmdir, listdir, path.exists, path.isfile, path.isdir, access, getcwd, and sendfile. These cover most common file system operations you would need in an async context.

Temporary files

The aiofiles.tempfile module gives you async temporary files with the same context manager and iteration support as regular files. This is useful when you need to stage data before writing it to its final destination, or when you want to avoid writing intermediate results to disk synchronously.

import aiofiles.tempfile

async with aiofiles.tempfile.NamedTemporaryFile('wb+') as f:
    await f.write(b'temp data')
    await f.seek(0)
    content = await f.read()

It exposes TemporaryFile, NamedTemporaryFile, SpooledTemporaryFile, and TemporaryDirectory. All work as async context managers and can be iterated with async for just like regular files.

What aiofiles does not do

It is important to set realistic expectations. aiofiles does not make disk I/O faster than using threads directly. Under the hood, it still uses a thread pool. What it gives you is the ability to write file I/O in an async style without blocking your event loop. The benefit is concurrency with network I/O, not raw I/O throughput.

For truly CPU-bound tasks like parsing a large CSV file or applying transformations to many rows, you will still block the worker thread. In those cases, offload the work to a process pool so the event loop remains responsive. The run_in_executor() method on the event loop handles this cleanly.

def parse_csv_sync(filepath):
    import csv
    rows = []
    with open(filepath) as f:
        reader = csv.DictReader(f)
        for row in reader:
            rows.append(process_row(row))
    return rows

async def main():
    loop = asyncio.get_running_loop()
    rows = await loop.run_in_executor(None, parse_csv_sync, 'big.csv')
    print(f'Processed {len(rows)} rows')

run_in_executor(None, ...) uses the default process pool executor. This keeps your async event loop completely free while the CPU-intensive parsing runs in a subprocess.

Common mistakes

Forgetting await is the most common error when switching from synchronous code. The file methods are coroutines, not regular methods, so calling them without await returns a coroutine object rather than your data.

# Wrong: returns a coroutine object, not the content
content = f.read()

# Correct: awaits the coroutine to get the actual data
content = await f.read()

Not using async with is the second most common mistake. It leads to leaked file handles when exceptions occur, and those handles may not be released until the garbage collector runs.

# Wrong: file never closed if an exception is raised
f = await aiofiles.open('file.txt')
content = await f.read()
await f.close()

# Correct: file is always closed when exiting the block, even on exceptions
async with aiofiles.open('file.txt') as f:
    content = await f.read()

Using text mode for binary data causes encoding errors. Always use 'rb' when reading image files, archives, or any non-text data.

# Wrong
async with aiofiles.open('image.png', 'r') as f:
    data = await f.read()

# Correct
async with aiofiles.open('image.png', 'rb') as f:
    data = await f.read()

Conclusion

aiofiles brings non-blocking file I/O to asyncio applications with an API that feels familiar if you already know Python’s file interface. Use it when you need to perform file operations alongside network I/O, process multiple files concurrently, or keep your event loop responsive while reading large files. Remember that it still uses threads under the hood, so for CPU-heavy file processing, reach for run_in_executor() with a process pool instead.

See Also

• Async Asyncio Basics — event loops, tasks, and asyncio.run() from the ground up
• Async Patterns — cancellation, timeouts, and advanced async control flow
• Async Aiohttp Client — async HTTP requests used alongside file I/O in a real application
• Context Managers — how async with works under the hood