Reading and Writing JSON Files in Python

JSON (JavaScript Object Notation) is the backbone of data exchange in modern applications. Whether you are reading configuration files, consuming REST APIs, or storing structured data, working with JSON is a skill every Python developer needs. Python’s standard library makes this task straightforward with the built-in json module, which handles parsing, serialization, and custom encoding without external dependencies.

What is JSON

JSON represents data in a format that humans can read and machines can parse. It supports several data types:

• Objects: Curly braces containing key-value pairs {"key": "value"}
• Arrays: Ordered lists [1, 2, 3]
• Strings: Double-quoted text "hello"
• Numbers: Integers and floats 42, 3.14
• Booleans: true or false
• Null: null

Python maps JSON types to native types:

JSON Type	Python Type
object	dict
array	list
string	str
number (int)	int
number (float)	float
true/false	True/False
null	None

Reading JSON from a File

The most common task is reading JSON data from a file:

import json

# Read JSON from a file
with open("config.json", "r") as f:
    data = json.load(f)

print(data)
# {'debug': True, 'database': {'host': 'localhost', 'port': 5432}}

The json.load() function reads directly from a file object. It parses the JSON content and returns the corresponding Python object — a dictionary for JSON objects, a list for arrays, and native types for strings, numbers, booleans, and null.

What if your JSON file does not exist or contains invalid syntax? Real applications must handle these cases gracefully. You will get an error:

import json

try:
    with open("missing.json", "r") as f:
        data = json.load(f)
except FileNotFoundError:
    print("File does not exist")
except json.JSONDecodeError as e:
    print(f"Invalid JSON: {e}")

Handling errors explicitly gives you control over what happens when a file is missing or malformed. The FileNotFoundError catches cases where the file path is wrong or the file was deleted, while json.JSONDecodeError catches structural problems like trailing commas, unquoted keys, or truncated content. Without these handlers, your program would crash with a stack trace that is less informative than a clear error message. Catching the specific exception types lets you log the problem, fall back to a default configuration, or prompt the user instead of failing silently.

Reading JSON from a String

Sometimes you have JSON data as a string rather than a file. This is common when working with API responses:

import json

json_string = '{"name": "Alice", "age": 30, "active": true}'

# Parse JSON string to Python object
data = json.loads(json_string)

print(data)
# {'name': 'Alice', 'age': 30, 'active': True}

# Access the values like a normal dictionary
print(data["name"])  # Alice
print(data["age"])   # 30

The function name loads (with an “s”) stands for “load string”. It takes a string and returns a Python object.

Real-world JSON responses rarely stay flat. API payloads typically nest objects inside objects, embed arrays of items, or include optional fields that may be present in some responses but not others. Being able to navigate these nested structures confidently is essential when consuming REST APIs or processing configuration files with multiple levels of hierarchy.

What if the JSON is nested or contains lists?

import json

api_response = '''
{
    "user": {
        "id": 123,
        "name": "Alice",
        "roles": ["admin", "editor"]
    },
    "status": "success"
}
'''

data = json.loads(api_response)

# Navigate nested structures
user_name = data["user"]["name"]        # Alice
first_role = data["user"]["roles"][0]   # admin

print(f"User: {user_name}, Role: {first_role}")

The nested access pattern — chaining dictionary keys and list indices with square brackets — works well for known structures, but it throws KeyError or IndexError if any intermediate key or index is missing. When the JSON shape is unpredictable, consider using the .get() method with a default value or wrapping the access in a try/except block. This defensive approach prevents your program from crashing when an API response changes structure between versions.

Writing JSON to a File

Saving data as JSON is just as easy. Use json.dump() to write directly to a file:

import json

data = {
    "name": "Bob",
    "age": 25,
    "skills": ["Python", "JavaScript"],
    "active": True
}

with open("output.json", "w") as f:
    json.dump(data, f, indent=2)

This creates a file called output.json with formatted content. The indent=2 parameter controls both indentation and readability: it specifies two-space nesting levels that make the structure visually obvious at a glance. Without indent, the entire JSON would be compressed into a single line, which saves space but makes the file unreadable for humans who need to inspect or debug it.

{
  "name": "Bob",
  "age": 25,
  "skills": [
    "Python",
    "JavaScript"
  ],
  "active": true
}

The indent parameter makes the output readable. Without it, the JSON would be compressed to a single line.

Reading and writing files covers the most common JSON workflow — loading data from disk, processing it in memory, and saving results back. But many applications never touch the filesystem. When you send JSON over a network, store it in a database, or pass it between microservices, you work with strings instead of file handles. Python’s string-oriented functions handle this case just as cleanly.

Writing JSON to a String

To convert a Python object to a JSON string, use json.dumps():

import json

data = {"name": "Charlie", "score": 95.5}

# Convert to JSON string
json_string = json.dumps(data)

print(json_string)
# {"name": "Charlie", "score": 95.5}

# Pretty-printed version
pretty = json.dumps(data, indent=4)
print(pretty)
# {
#     "name": "Charlie",
#     "score": 95.5
# }

This is useful when sending JSON data over HTTP or including it in a message queue.

The dumps() function gives you fine-grained control over the output format through optional parameters. The indent parameter you’ve already seen controls nested indentation, but separators and sort_keys offer additional formatting options that become important when output size matters or deterministic ordering is required.

Pretty printing and compact output

For debugging, pretty printing helps visualize nested structures:

import json

data = {
    "users": [
        {"id": 1, "name": "Alice"},
        {"id": 2, "name": "Bob"}
    ],
    "count": 2
}

# Human-readable format
print(json.dumps(data, indent=2))

The pretty-printed output below shows how indent=2 creates a consistent two-space nesting pattern for every level of the hierarchy. Notice that both the top-level keys and the nested user objects receive the same two-space indent, producing a clean visual alignment that makes it easy to scan for specific fields or spot structural errors.

Output:

{
  "users": [
    {
      "id": 1,
      "name": "Alice"
    },
    {
      "id": 2,
      "name": "Bob"
    }
  ],
  "count": 2
}

Pretty printing is ideal during development and debugging, where readability matters more than file size. But when you are building an API that serves thousands of requests per second, every byte counts. Compact serialization strips all optional whitespace to produce the smallest possible JSON representation, reducing bandwidth usage and improving response times under load.

For production environments where file size matters, use compact formatting:

import json

data = {"a": 1, "b": 2}

# Compact output (no whitespace)
compact = json.dumps(data, separators=(",", ":"))
print(compact)
# {"a":1,"b":2}

The basic json.load/loads and json.dump/dumps functions handle dictionaries, lists, strings, numbers, booleans, and None without any extra work. But real Python programs use richer data types: datetime objects for timestamps, Decimal for precision arithmetic, custom classes for domain models, and sets or tuples for collections. None of these map to a native JSON type, so attempting to serialize them raises a TypeError unless you tell the encoder how to convert them.

Working with complex objects

JSON has limited types. Not every Python object can be serialized directly:

import json
from datetime import datetime

# This will fail
data = {"created": datetime.now()}

try:
    json.dumps(data)
except TypeError as e:
    print(f"Error: {e}")
    # Object of type datetime is not JSON serializable

The error message tells you exactly what went wrong — the encoder does not know how to turn a datetime into JSON. Since JSON has no date type, you must decide on a serialization format before the encoder can proceed. The three approaches below offer different tradeoffs between simplicity and reusability.

You have several options to handle this:

Option 1: Convert manually

import json
from datetime import datetime

data = {"created": datetime.now()}

# Convert datetime to string before serializing
data["created"] = data["created"].isoformat()

json_string = json.dumps(data)
print(json_string)
# {"created": "2024-01-15T10:30:45.123456"}

Manual conversion is the simplest approach: convert the problematic value to a JSON-compatible type before calling json.dumps. This works well for one-off serializations where you control exactly which fields need conversion. For larger data structures with many datetime fields scattered across nested objects, however, manually converting each one becomes tedious and error-prone.

Option 2: Use the default parameter

import json
from datetime import datetime

def serialize_datetime(obj):
    if isinstance(obj, datetime):
        return obj.isoformat()
    raise TypeError(f"Type {type(obj)} not serializable")

data = {"created": datetime.now()}

json_string = json.dumps(data, default=serialize_datetime)

The default parameter accepts a function that the encoder calls whenever it encounters an unrecognized type. Your function receives the unknown object and must either return a JSON-compatible value or raise TypeError to propagate the error. This approach centralizes the conversion logic in one place, making it easy to reuse across multiple json.dumps calls throughout your codebase.

Option 3: Use a custom encoder class

import json
from datetime import datetime

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        return super().default(obj)

data = {"created": datetime.now()}

json_string = json.dumps(data, cls=CustomEncoder)

Subclassing json.JSONEncoder is the most extensible approach. You override the default method to handle your custom types, and the encoder inherits all the standard serialization logic for built-in types automatically. This pattern integrates cleanly with libraries and frameworks that accept an encoder class — for example, Flask’s jsonify function and Django’s JSONResponse both support custom encoder classes.

Reading JSON from APIs

When working with HTTP APIs, you often receive JSON responses. The requests library handles the parsing for you:

import requests
import json

# Fetch data from an API
response = requests.get("https://api.github.com/users/octocat")

# Method 1: Use response.json() (most common)
data = response.json()

# Method 2: Parse manually
data = json.loads(response.text)

print(f"Name: {data['name']}")
print(f"Repos: {data['public_repos']}")

The response.json() method is equivalent to json.loads(response.text).

Sending data to an API reverses the flow: you serialize Python objects into JSON strings and include them in the request body. The requests library supports both manual serialization with explicit content-type headers and a convenience json parameter that handles the conversion automatically. The manual approach gives you control over the serialization details, while the json parameter is shorter and less error-prone for typical use cases.

For POST requests with JSON payloads:

import requests
import json

payload = {"username": "alice", "password": "secret123"}

# Send JSON data
response = requests.post(
    "https://api.example.com/login",
    data=json.dumps(payload),  # Convert dict to JSON string
    headers={"Content-Type": "application/json"}
)

# Or use the json parameter (requests does the conversion)
response = requests.post(
    "https://api.example.com/login",
    json=payload
)

Even experienced developers occasionally make mistakes with JSON in Python. The three pitfalls below — forgetting to parse responses, mishandling Unicode characters, and losing integer precision — are among the most common and can produce subtle bugs that are hard to track down. Each one has a straightforward fix once you know what to look for.

Common Pitfalls

Forgetting to parse

import requests

response = requests.get("https://api.example.com/data")

# Wrong: response is a Response object, not the data
print(response["name"])  # TypeError

# Correct: parse the JSON first
data = response.json()
print(data["name"])  # Works

The response.json() method returns a parsed dictionary directly, so attempting to index into the raw response object fails with a TypeError. Always call .json() first — or check response.status_code before parsing to handle HTTP errors gracefully.

Unicode and encoding

By default, json.dumps() escapes non-ASCII characters:

import json

data = {"name": "日本語"}

print(json.dumps(data))
# {"name": "\u65e5\u672c\u8a9e"}

print(json.dumps(data, ensure_ascii=False))
# {"name": "日本語"}

The ensure_ascii=False flag tells the encoder to write non-ASCII characters directly into the output rather than escaping them as Unicode escape sequences. This makes the output readable for humans working with non-Latin scripts and slightly reduces the serialized string length. The tradeoff is that some older systems or strict JSON parsers may expect ASCII-only content, so keep ensure_ascii=True (the default) when you need maximum compatibility.

Float precision

JSON represents all numbers as floats or integers. Large integers may lose precision:

import json

# Python int can be arbitrarily large
data = {"big": 9007199254740993}  # Larger than JavaScript's MAX_SAFE_INTEGER

json_string = json.dumps(data)
parsed = json.loads(json_string)

print(data["big"] == parsed["big"])  # False - precision lost!

JSON numbers use IEEE 754 double-precision floating-point format, which can precisely represent integers only up to 2^53 (about 9 quadrillion). Python integers have no such limit — they grow to whatever size memory allows. When you serialize a Python integer larger than JavaScript’s Number.MAX_SAFE_INTEGER and parse it back, the lower bits may be truncated, producing a different value. If you need exact large-integer fidelity across JSON boundaries, serialize the integer as a string and convert it back after parsing.

Quick Reference

Task	Code
Read JSON file	json.load(f)
Read JSON string	json.loads(string)
Write JSON file	json.dump(data, f)
Write JSON string	json.dumps(data)
Pretty print	json.dumps(data, indent=2)
Compact output	json.dumps(data, separators=(",", ":"))
Custom serialization	json.dumps(data, default=func)

See Also

• json-module — Complete reference for the json module
• csv-module — Working with CSV files
• urllib-parse-module — URL parsing utilities