Pythonic Data Structures

The Pythonic Way

You already know how to store data in Python's fundamental data structures. Now, let's focus on doing it idiomatically. Writing 'Pythonic' code means leveraging the language's features to write clear, concise, and efficient solutions. It’s the difference between speaking a language fluently and just translating words.

Good code is about more than just correctness; it's about communicating your intent clearly to others and to your future self.

We'll start by replacing clunky for loops with a more elegant and often faster alternative: comprehensions.

Beyond the Loop

Comprehensions are a hallmark of Pythonic code. They allow you to create a new list, set, or dictionary from an existing iterable in a single, readable line. Compare the traditional loop approach to a list comprehension for creating a list of squared numbers.

# Traditional for loop
squares = []
numbers = [1, 2, 3, 4, 5]
for num in numbers:
    squares.append(num * num)
# Result: [1, 4, 9, 16, 25]

# List comprehension
squares_comp = [num * num for num in numbers]
# Result: [1, 4, 9, 16, 25]

The comprehension is more than just shorter; it clearly states what you are creating (a list of squared numbers) rather than how you are creating it (initialize an empty list, loop, append). This declarative style is easier to read at a glance.

You can also include conditional logic. Let's get the squares of only the even numbers:

even_squares = [num * num for num in numbers if num % 2 == 0]
# Result: [4, 16]

This pattern extends to sets and dictionaries. A set comprehension is perfect for creating a set of unique values from an iterable, and a dictionary comprehension is great for transforming or filtering existing dictionaries.

# Set comprehension to get unique word lengths
words = ["hello", "world", "python", "is", "fun"]
unique_lengths = {len(word) for word in words}
# Result: {2, 3, 5, 6}

# Dictionary comprehension to create a price map with tax
prices = {'apple': 1.0, 'banana': 0.5, 'orange': 0.75}
prices_with_tax = {item: price * 1.18 for item, price in prices.items()}
# Result: {'apple': 1.18, 'banana': 0.59, 'orange': 0.885}

While comprehensions are powerful, nesting them can quickly reduce readability. A nested comprehension can flatten a list of lists, but for anything more complex, a standard loop might be clearer. And for very large datasets, a related concept, , can be more memory-efficient.

Slicing and Dicing

You're likely familiar with basic slicing like my_list[1:4]. Python's slicing is more powerful than it first appears, thanks to an optional third argument: the step.

numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

# Get every second element
numbers[::2]  # [0, 2, 4, 6, 8]

# Get every second element from index 1 to 7
numbers[1:8:2]  # [1, 3, 5, 7]

# Reverse a list with a negative step
numbers[::-1]  # [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

That last one, [::-1], is a common and highly readable idiom for reversing any sequence in Python. Using the step parameter allows for complex data selection without needing a loop.

Specialised Tools

Python's standard library includes the collections module, which provides specialised data structures that solve common problems elegantly. Let's look at two: namedtuple and defaultdict.

namedtuple

noun

A factory function for creating tuple subclasses with named fields. It improves code readability by allowing you to access elements by name instead of index.

Imagine you are working with 2D points stored as tuples, like (10, 20). Accessing them via point[0] and point[1] isn't very descriptive. A namedtuple fixes this.

from collections import namedtuple

Point = namedtuple('Point', ['x', 'y'])

p1 = Point(10, 20)

print(p1.x)  # Output: 10
print(p1.y)  # Output: 20

A defaultdict is another useful tool. It's a subclass of the standard dictionary that calls a factory function to supply missing values. This completely avoids KeyError exceptions when you try to access or modify a key that's not present.

from collections import defaultdict

# Count occurrences of words in a list
sentence = "the quick brown fox jumps over the lazy dog"
words = sentence.split()

word_count = defaultdict(int)  # int() returns 0
for word in words:
    word_count[word] += 1

print(word_count['the'])  # Output: 2
print(word_count['cat'])  # Output: 0 (no KeyError!)

Without defaultdict, you'd need to write extra code to check if a key exists before incrementing its value. This is a much cleaner solution for tasks like grouping or counting items.

Objects in Memory

Understanding how Python handles objects in memory is crucial for avoiding subtle bugs. The key concept is mutability. Some objects, like lists and dictionaries, are mutable, meaning they can be changed in place. Others, like tuples, strings, and integers, are immutable; any 'change' actually creates a new object.

This distinction becomes critical when you copy objects. A simple assignment (new_list = old_list) doesn't create a copy; it just makes both variables point to the same list object in memory.

Modifying the list through one variable will affect the other. To create an independent copy, you need to be explicit. A shallow copy (using my_list.copy() or my_list[:]) creates a new container but fills it with references to the same items. This is fine for lists of immutable objects like numbers. But if your list contains other mutable objects, like other lists, both the original and the shallow copy will share those inner objects.

A deep copy, from the copy module, is the solution here. It recursively traverses the object and copies everything, creating a completely independent clone.

import copy

old_list = [1, [2, 3]]

# Shallow copy
shallow_copy = old_list.copy()
shallow_copy[1].append(4)

print(f"Old list after shallow copy mod: {old_list}")
# Output: Old list after shallow copy mod: [1, [2, 3, 4]] -> It changed!

# Deep copy
old_list = [1, [2, 3]] # Reset
deep_copy = copy.deepcopy(old_list)
deep_copy[1].append(4)

print(f"Old list after deep copy mod:    {old_list}")
# Output: Old list after deep copy mod:    [1, [2, 3]] -> It's safe!

Choosing the right data structure involves knowing these nuances. Performance often depends on the task. A set offers incredibly fast membership testing (if item in my_set:), much faster than a list. A dictionary provides near-instant lookups by key. Understanding the of these operations helps you write truly efficient code.

One of the most effective ways to optimize Python code is by using efficient data structures.

Mastering these Pythonic structures and principles is a big step. It moves you from simply writing code that works to writing code that is performant, readable, and robust.

Mastering Pythonic Development