Binary Data with struct and memoryview

June 13, 2026 · 13 min read ·Updated June 13, 2026 ·intermediate

pythonstdlibstructmemoryviewbinarybuffer-protocolbytes

Python ints and strings don’t fly across the wire or land in a PNG header as themselves. To work with binary data, you turn values into bytes, give those bytes a layout, and read the layout back on the other side. Two tools in the standard library do almost all of that work: the struct module for the layout language, and the memoryview builtin for handling the bytes without copying them. This guide walks through both, with the gotchas that bite people in production.

TL;DR

Use the struct module to convert Python values to and from bytes with a layout you control via a format string.
Always lead a format string with <, >, or ! to lock in byte order for binary data. The default @ is native-only and silently breaks across machines.
Cache a Struct instance in a hot path so the format string is parsed once, not on every call.
Use memoryview for zero-copy slicing. Views of bytes are read-only; views of bytearray are writable.
Combine struct.unpack_from with memoryview slices to parse a fixed-layout binary protocol without per-record allocation.

Why Python needs two tools for binary data

bytes and bytearray are flat sequences of integers from 0 to 255. They are the wire format, but they don’t tell you what those integers mean. Is byte 0 part of a 4-byte integer? Two 2-byte shorts? The first character of a 12-byte string field? You decide that with a format string, and struct is the interpreter.

memoryview solves the other problem. Slicing a bytes produces a copy of those bytes, every time. If you’re parsing a 4 MB network buffer into 40 KB chunks, that’s a lot of allocation. A memoryview of the same buffer gives you slices that share storage with the original, with no copy at all. The downside is the API: a memoryview is a wrapper, not a bytes, and the rules around when you can write through it catch people off guard.

The rest of this guide is mostly about getting comfortable with both.

How do you pack and unpack values with struct?

The struct module turns Python values into bytes and back, using a format string as the contract.

import struct

# Pack three integers into 7 bytes
blob = struct.pack(">bhl", 1, 2, 3)
# b'\x01\x00\x02\x00\x00\x00\x03'

# And read them back
struct.unpack(">bhl", blob)
# (1, 2, 3)

struct.calcsize(">bhl")
# 7

struct.pack returns a fresh bytes object sized to the format. struct.unpack always returns a tuple, even for a single field, and you destructure that single value with a trailing comma:

value, = struct.unpack(">H", b"\x01\x00")
# 1

If the value is out of range for the format, struct.error is raised at pack time (since Python 3.1, you no longer get silent wrap-around) and the call never returns a partial result:

struct.pack(">h", 99999)
# struct.error: 'h' format requires -32768 <= number <= 32767

The format characters you will actually use

You will reach for maybe a dozen format characters in practice. The full table lives in the struct format characters docs, but the workhorses are:

Format	C type	Python type	Size (bytes)	Notes
`b`	`signed char`	`int`	1
`B`	`unsigned char`	`int`	1
`h`	`short`	`int`	2
`H`	`unsigned short`	`int`	2
`i`	`int`	`int`	4
`I`	`unsigned int`	`int`	4
`q`	`long long`	`int`	8
`Q`	`unsigned long long`	`int`	8
`f`	`float`	`float`	4	IEEE 754 binary32
`d`	`double`	`float`	8	IEEE 754 binary64
`s`	`char[]`	`bytes`	n	Count is a byte length: `'10s'` is one 10-byte field.
`c`	`char`	`bytes` of length 1	1	Count repeats the field: `'10c'` is ten 1-byte arguments.
`?`	`_Bool`	`bool`	1

The s vs c distinction is the single biggest footgun:

struct.pack("@3s", b"abc")
# b'abc'   # one 3-byte field

struct.pack("@3c", b"a", b"b", b"c")
# b'abc'   # three 1-byte fields

'4s' expects a single 4-byte bytes. '4c' expects four separate 1-byte arguments. Mixing them up wastes hours.

Repeat counts work for everything except s and p: '4h' is the same as 'hhhh'. Whitespace inside a format string is ignored, which makes longer layouts more readable.

Which byte order should you use in struct format strings?

The first character of a format string is the byte order, size, and alignment. Picking the wrong one is the most common struct bug in the wild.

Prefix	Byte order	Size	Alignment
`@`	native	native	native
`=`	native	standard	none
`<`	little-endian	standard	none
`>`	big-endian	standard	none
`!`	network (= big-endian)	standard	none

The default is @, which means whatever the host machine happens to be. Code that works on x86 silently produces wrong output when a coworker runs it on a big-endian machine, or when you read a file written by a different process.

Rule: for any cross-machine format, always start the format string with <, >, or !. Use ! for network protocols (it’s big-endian by definition, per RFC 1700). Use < or > for explicit choice and stick with it.

Native padding makes the same layout take different amounts of space depending on the order of the fields:

# On a 64-bit little-endian machine
struct.pack("@ci", b"#", 0x12131415)
# b'#\x00\x00\x00\x15\x14\x13\x12'   # 8 bytes, with 3 pad bytes for the int

struct.pack("@ic", 0x12131415, b"#")
# b'\x15\x14\x13\x12#'                 # 5 bytes, no padding needed

Reordering fields changes the encoded size. With native alignment off (<, >, !, =), this stops being a problem because struct no longer inserts pad bytes.

When should you cache a compiled struct instance?

If you call struct.pack in a tight loop, every call re-parses the format string. The Struct class compiles it once and stores the result on the instance:

from struct import Struct

record = Struct("<I 2s f q")   # 4 + 2 + 4 + 8 = 18 bytes, little-endian
record.size
# 18

blob = record.pack(1, b"hi", 3.14, -1)
record.unpack(blob)
# (1, b'hi', 3.140000104904175, -1)

Struct instances have the same method set as the module-level functions, plus two handy attributes: format (the original string) and size (the cached calcsize result). They are picklable and thread-safe for the pack family, so you can stash one on a module-level constant and share it across threads without a lock.

In-place packing with pack_into and unpack_from

pack_into writes into an existing buffer at an offset. unpack_from reads from one. This matters when the buffer came from somewhere else (a socket, an mmap, a file read) and you want to avoid an extra allocation.

import struct

buf = bytearray(64)
struct.pack_into(">II", buf, 0, 0xDEADBEEF, 0xCAFEBABE)
struct.pack_into("10s",  buf, 8, b"header")

id1, id2 = struct.unpack_from(">II", buf, 0)
header,  = struct.unpack_from("10s", buf, 8)

pack_into requires offset; there is no default, and forgetting it is a TypeError. The buffer has to be writable, so bytes won’t work. Use bytearray, array.array, or anything else that implements the buffer protocol as writable.

Why use memoryview for parsing raw bytes?

memoryview wraps any object that implements the buffer protocol (bytes, bytearray, array.array, NumPy arrays, ctypes buffers, and more) and lets you slice it without copying:

data = b"shave and a haircut, two bits"
view = memoryview(data)

chunk = view[12:19]    # another memoryview, no copy
chunk.tobytes()
# b'haircut'
chunk.obj              # the original bytes object
chunk.nbytes           # 7
chunk.readonly         # True (data is a bytes object)

The last line is the catch. A memoryview of bytes is read-only. Slicing and trying to assign raises TypeError: cannot modify read-only memory. To write through a view, the backing object has to be writable. bytearray works:

buf = bytearray(b"row, row, row your boat")
mv = memoryview(buf)
mv[3:13] = b"-10 bytes-"
buf
# bytearray(b'row-10 bytes- your boat')

This is the reason memoryview exists. Slicing a bytes is O(n) and returns a new object. Slicing a memoryview is O(1) and returns another view. For a 4 MB buffer parsed into hundreds of small chunks, the difference is the difference between “fast” and “GC-thrashing slow”.

Reinterpreting bytes with memoryview.cast

Sometimes you don’t want a new slice; you want to look at the same bytes through a different type. memoryview.cast reinterprets the buffer as a sequence of a different element size:

buf = bytearray(b'\x01\x00\x02\x00\x03\x00\x04\x00')   # 8 raw bytes
mv = memoryview(buf).cast("H")    # unsigned short (LE on most machines)
mv.tolist()
# [1, 2, 3, 4]

The new view shares storage with the old one, so a write through one view is visible through the other. That is the whole point of cast: a single allocation, two perspectives:

mv[0] = 0xFFFF
bytes(buf)
# b'\xff\xff\x02\x00\x03\x00\x04\x00'

cast requires nbytes % itemsize == 0. Casting a 7-byte buffer to 'H' raises ValueError: memoryview: cannot cast between two different size types without copying. For multi-dimensional reinterpretation, pass a shape and the same view turns into a 2-D or N-D array of the new element type:

raw = struct.pack(">6h", *range(1, 7))   # 12 bytes
matrix = memoryview(raw).cast("h", shape=(2, 3))
matrix[0, 0]      # 1
matrix[1, 2]      # 6
matrix.tolist()   # [[1, 2, 3], [4, 5, 6]]

How do you parse a binary protocol with struct and memoryview?

The real payoff is putting struct and memoryview together to walk a buffer of fixed-layout binary data records. The header is a known-size prefix and the body length comes from inside that header, so the rest of the buffer is a memoryview slice of zero-copy bytes. The pattern looks like this:

import struct
from struct import Struct

HEADER = Struct("<IHBB")   # type (4), length (2), flags (1), version (1)

def parse_frame(buf: bytes) -> tuple[tuple, memoryview]:
    header = HEADER.unpack_from(buf)
    msg_type, length, flags, version = header
    # memoryview slice of the body; no copy
    body = memoryview(buf)[HEADER.size : HEADER.size + length]
    return header, body

raw = b"\x01\x00\x00\x00\x07\x00\x00\x03" + b"payload"
header, body = parse_frame(raw)
header
# (1, 7, 0, 3)
bytes(body)
# b'payload'

For a streaming use case, fill a bytearray once with socket.recv_into or RawIOBase.readinto, then walk the buffer with memoryview slices and struct.unpack_from. No per-record allocation, no per-record copy. See the socket module reference and io module reference for the receiving half.

struct.iter_unpack handles the case where your buffer is a sequence of fixed-size records, which is the common shape for log files and binary streams. You give it the format once and it walks the buffer:

records = b""
for i in (1, 2, 3):
    records += struct.pack("<I 2s f q", i, b"hi", 3.14, -i)

The accumulated buffer is 54 bytes (3 × 18). Once you have a flat bytes object, iter_unpack returns one tuple per record with no Python-level loop and no per-record slicing:

for record in struct.iter_unpack("<I 2s f q", records):
    print(record)
# (1, b'hi', 3.140000104904175, -1)
# (2, b'hi', 3.140000104904175, -2)
# (3, b'hi', 3.140000104904175, -3)

If your records arrive in chunks, build them with memoryview(buf)[offset:offset+18] and append to a reusable list. The buffer length must be an exact multiple of the format size; otherwise iter_unpack raises struct.error mid-iteration. Pad or slice the input first.

What are the most common struct and memoryview mistakes?

Most of the bugs that show up in code that uses struct and memoryview for binary data are not exotic — they are a handful of surprises that every reader eventually hits. Skim the list once and you will save yourself the debugging session later. A few things that show up over and over:

Forgetting the byte-order prefix. A format string starting with b, h, or i is in native order. Always lead with <, >, or ! for portable layouts.
Mixing s and c count semantics. '10s' is one 10-byte field; '10c' is ten 1-byte arguments.
Assuming unpack returns a scalar. It always returns a tuple. Use a trailing comma to destructure.
Trying to write through a read-only memoryview. Views of bytes are read-only. Use bytearray if you need writes.
Casting to a size that doesn’t divide the buffer. cast("H") on 7 bytes fails. Either pad or pick a different type.
Using pack for one integer. If all you have is a single integer, int.to_bytes and int.from_bytes are shorter and clearer. struct earns its keep when you have a multi-field layout.
Trying to use 'P' with explicit byte order. The native-pointer format is only available with native byte order. With <, >, !, or =, it raises struct.error.
Subclassing memoryview. CPython rejects subclass instances at the type level. Don’t bother trying.

A safe wrapper around struct for one specific message type looks like this in practice, with the format declared once on a class attribute so the encode and decode sides cannot drift apart:

import struct
from struct import Struct

class Pinger:
    fmt = Struct("<Hd")  # seq (uint32), timestamp_ms (double)

    def encode(self, seq: int, ts: float) -> bytes:
        return self.fmt.pack(seq, ts)

    def decode(self, blob: bytes) -> tuple[int, float]:
        seq, ts = self.fmt.unpack(blob)
        return seq, ts

The compile happens once when the class is defined, every encode and decode shares the cached size and format, and there is no per-call parsing overhead.

When to reach for something else

struct and memoryview cover a lot of ground for fixed-layout binary data, but they are general-purpose tools, and general-purpose tools are rarely the best choice for a specific problem. Reach for the libraries below when your problem has outgrown struct:

A homogeneous typed buffer of numbers. The standard library’s array module stores C types directly, and you can memoryview over an array.array and cast it to a matching struct format. For two or more dimensions, use NumPy.
Numeric work (vectors, matrices, FFTs). Reach for NumPy. Its whole point is typed, multidimensional buffers with vectorized operations.
Portable serialization across languages or upgrades. Use json for text, pickle for Python-only objects. struct is for fixed layouts, not evolution.
Fixed-size binary digests. Use hashlib to produce bytes; struct is for unpacking the digest (or the prefix you wrote alongside it). The hashlib guide has a worked example.
Calling C code with pointer-shaped data. The 'P' format exists but isn’t a substitute for a real FFI. Use the ctypes or cffi package, depending on your project.
Compressed binary payloads. See the zlib compression guide for the post-struct step.

Frequently asked questions

What is the difference between `struct.pack` and `int.to_bytes`?

int.to_bytes converts a single integer to bytes; struct.pack handles a layout of one or more typed fields. For one integer with a known size, int.to_bytes(n, 'big') is shorter and clearer. For multi-field binary data layouts, struct wins because the format string documents the shape of the buffer and calcsize tells you the byte count up front.

Why is my `memoryview` read-only?

A view inherits read-only status from the object it wraps. bytes objects are immutable, so a memoryview of bytes is read-only and mv[0] = 1 raises TypeError: cannot modify read-only memory. Build the view on a bytearray (or any other writable buffer-protocol object) to assign through it.

When do you use `struct` versus `json` or `pickle`?

struct is for fixed-layout binary data where every byte position has a meaning, like a network header, a binary file format, or a sensor sample. json is for portable text data. pickle is for Python-only object graphs. If your format needs to evolve or be read by other languages, struct is the wrong tool — it has no schema, no versioning, and no field names.

How do you read a binary file in Python?

Open the file in 'rb' mode, read into a bytearray if you’ll be parsing records in place, and walk the buffer with memoryview slices and struct.unpack_from. For a streaming socket, socket.recv_into accepts a memoryview directly so you can fill a pre-allocated buffer without an extra copy. The socket module reference covers the receiving half.

What is the buffer protocol?

The buffer protocol is the C-level contract that lets one object expose its raw bytes to another. bytes, bytearray, array.array, NumPy arrays, and ctypes buffers all implement it. memoryview is the Python-level wrapper for that protocol, which is why it works across all of these types without copying.

Conclusion

Learn the format characters and the byte-order prefixes, cache a Struct instance for any hot path, and treat memoryview as the default view type when you’ll be slicing the result. Once that pattern clicks, reading and writing fixed-layout binary data with struct and memoryview becomes a one-screen function, and you stop fighting the language every time you need to talk to something that doesn’t speak Python natively.

For the network side of the same problem, see the socket module reference and the io module reference. For fixed-size digests, the hashlib guide is a good next read.