Intermediate Python Application and Design
Advanced Data Structures
More Than Just Loops
Once you've got the hang of Python's basic for loops, you might notice yourself writing the same patterns over and over. You initialize an empty list, loop through some data, and append a modified item to your list on each pass. It works, but it's not the most direct way to express your intent.
Pythonic code is about writing code that is clear, concise, and readable. It leverages the language's features to express ideas simply.
Let's say you want to create a list of the first ten perfect squares. The standard loop approach looks like this:
squares = []
for i in range(10):
squares.append(i * i)
# squares is [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
This is fine, but we can do better. A list comprehension collapses this entire block into a single, elegant line. It follows the structure [expression for item in iterable].
squares = [i * i for i in range(10)]
# same result, one line
This isn't just shorter, it's more descriptive. It reads like plain English: "make a list of i * i for each i in the range of 10." You can also add conditions to filter items. To get only the squares of even numbers, you add an if clause at the end.
even_squares = [i * i for i in range(10) if i % 2 == 0]
# even_squares is [0, 4, 16, 36, 64]
Comprehensions can even be nested. To flatten a matrix (a list of lists) into a single list, you can use two for clauses.
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]
# flattened is [1, 2, 3, 4, 5, 6, 7, 8, 9]
Comprehending Dictionaries and Sets
This powerful syntax isn't limited to lists. You can create dictionaries and sets the same way. A dictionary comprehension uses curly braces {} and a key-value pair, like {key_expression: value_expression for item in iterable}.
# Create a dictionary of numbers and their squares
number_squares = {x: x*x for x in range(5)}
# number_squares is {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Set comprehensions are nearly identical to list comprehensions but use curly braces. They're a great way to create a set from an existing collection, automatically handling duplicates.
numbers = [1, 2, 2, 3, 4, 4, 5]
unique_squares = {x*x for x in numbers}
# unique_squares is {1, 4, 9, 16, 25}
Using comprehensions makes your code more declarative. You're describing what you want, not the step-by-step process of how to create it. This aligns with guidelines, which emphasize code readability.
Generators for Big Data
List comprehensions are fantastic, but they have one significant drawback: they build the entire list in memory at once. If you're working with millions of items, this can consume a huge amount of RAM. What if you only need to process one item at a time?
This is where generator expressions come in. They look almost identical to list comprehensions, but they use parentheses () instead of square brackets [].
# A list comprehension (uses memory for 1 million integers)
list_comp = [i for i in range(1_000_000)]
# A generator expression (uses a tiny, constant amount of memory)
gen_exp = (i for i in range(1_000_000))
The generator expression doesn't create a million numbers. It creates a special generator object that knows how to produce those numbers when asked. You can loop over it just like a list, but the values are generated on the fly. This is incredibly useful for tasks like summing a large sequence or writing data to a file, where you don't need to hold everything in memory.
Use a list comprehension when you need the list itself. Use a generator expression when you just need to iterate over the values once, especially with large datasets.
Specialized Collections
Python's built-in collections module offers even more powerful data structures. Two of the most useful are defaultdict and namedtuple.
Have you ever tried to add a value to a list inside a dictionary, only to get a KeyError because the key doesn't exist yet? The defaultdict solves this. It's like a regular dictionary, but when you try to access a key that isn't there, it automatically creates a default value for it. This is perfect for grouping items.
from collections import defaultdict
colors = [('apple', 'red'), ('banana', 'yellow'), ('cherry', 'red'), ('orange', 'orange')]
by_color = defaultdict(list)
for fruit, color in colors:
by_color[color].append(fruit)
# by_color['red'] is ['apple', 'cherry']
# by_color['yellow'] is ['banana']
# No KeyError checks needed!
A is another handy tool. It lets you create simple, tuple-like objects that have named fields. This makes your code more readable because you can access elements by name instead of by index. They're lightweight and use no more memory than a regular tuple.
from collections import namedtuple
# Define the structure
Point = namedtuple('Point', ['x', 'y'])
# Create an instance
p1 = Point(10, 20)
# Access by name or index
print(p1.x) # Output: 10
print(p1[1]) # Output: 20
Ready to test your comprehension of these advanced structures?
What is the primary advantage of using a generator expression like (i for i in range(1000000)) over a list comprehension like [i for i in range(1000000)]?
Which of the following code snippets correctly creates a dictionary mapping each number from 0 to 4 to its square?
By moving beyond basic loops and embracing these more advanced structures, you can write Python code that is not only faster and more memory-efficient but also cleaner and easier for others to understand.