No history yet

Pythonic Data Structures

More Than Just Loops

You already know how to build a list by starting with an empty one and repeatedly using .append() inside a for loop. It works, but it's not the most expressive or efficient way to do it in Python. A more 'Pythonic' approach uses comprehensions, which let you build a new collection from an existing one in a single, readable line.

Let's say we want to create a list of the squares of the first ten numbers. The classic loop approach looks like this:

# The traditional for loop
squares = []
for i in range(10):
    squares.append(i**2)

print(squares)
# Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

A list comprehension achieves the same result more concisely:

# The list comprehension way
squares = [i**2 for i in range(10)]

print(squares)
# Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

The syntax is descriptive: [expression for item in iterable]. You can even add a conditional clause to filter items. For example, to get the squares of only the even numbers:

even_squares = [i**2 for i in range(10) if i % 2 == 0]

print(even_squares)
# Output: [0, 4, 16, 36, 64]

This same powerful syntax applies to dictionaries and sets. To create a dictionary mapping numbers to their squares, you use curly braces and a key-value pair:

square_map = {i: i**2 for i in range(5)}

print(square_map)
# Output: {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}

And for a set of unique squared values from a list with duplicates, it's just as simple. Notice how the duplicate 4 is automatically handled.

numbers = [1, 2, -1, 2, 3, -2]
unique_squares = {n**2 for n in numbers}

print(unique_squares)
# Output: {1, 4, 9}

Comprehensions are not just for cleaner code. Because they are implemented in C and optimized for the Python interpreter, they are often faster than building the same collection with an explicit for loop. When you see a chance to use one, it's usually the right choice.

Specialised Tools for the Job

Python's built-in lists, tuples, and dictionaries are great general-purpose tools. But sometimes you need something more specialised. The collections module provides high-performance container datatypes that solve specific problems cleanly.

Think of the collections module as a workshop full of specialised tools. You can build a house with just a hammer and saw, but a drill and a level make the job much easier and the result more robust.

One such tool is namedtuple. It allows you to create tuple-like objects that have named fields, much like a struct in other languages. This makes your code self-documenting and easier to read, as you can access elements by name instead of by index.

from collections import namedtuple

# Define the structure of our namedtuple
Point = namedtuple('Point', ['x', 'y'])

# Create an instance
p1 = Point(10, 20)

# Access data by name or index
print(f"Access by name: x={p1.x}, y={p1.y}")
print(f"Access by index: x={p1[0]}, y={p1[1]}")

Next up is , or 'double-ended queue'. While lists are efficient for adding or removing items from the end (appending and popping), they are slow at doing so from the beginning. Removing the first element from a large list requires shifting all other elements one spot to the left, which is an expensive operation. A deque is designed to make additions and removals from both ends incredibly fast.

from collections import deque

# Create a deque
d = deque(['task2', 'task3', 'task4'])

# Add to the right (like list.append)
d.append('task5')

# Add to the left (fast!)
d.appendleft('task1')

print(d)
# Output: deque(['task1', 'task2', 'task3', 'task4', 'task5'])

# Remove from the left (fast!)
d.popleft()
print(d)
# Output: deque(['task2', 'task3', 'task4', 'task5'])

Finally, defaultdict is a subclass of the standard dictionary that simplifies handling missing keys. Normally, trying to access a key that doesn't exist raises a KeyError. With defaultdict, you provide a function (like int, list, or set) that is called to supply a default value for any missing key. This is perfect for grouping or counting items.

from collections import defaultdict

# Create a defaultdict that returns 0 for missing keys
word_counts = defaultdict(int)

sentence = "the quick brown fox jumps over the lazy dog"
for word in sentence.split():
    word_counts[word] += 1 # No need to check if the key exists!

print(word_counts['the'])
# Output: 2

print(word_counts['cat'])
# Output: 0 (and 'cat' is now in the dictionary)

Thinking About Memory

List comprehensions are great, but they have one potential drawback: they build the entire list in memory all at once. If you're working with a massive dataset, this can consume a lot of RAM. What if you only need to process the items one by one?

This is where come in. They look almost identical to list comprehensions, but they use parentheses instead of square brackets. Instead of creating a full list, a generator expression creates a special 'generator' object.

This object doesn't hold all the values. Instead, it knows how to produce them on demand, one at a time, as you loop over it. This approach is far more memory-efficient for large sequences.

import sys

# A list comprehension (uses more memory)
list_comp = [i**2 for i in range(10000)]
print(f"List size: {sys.getsizeof(list_comp)} bytes")

# A generator expression (uses very little memory)
gen_exp = (i**2 for i in range(10000))
print(f"Generator size: {sys.getsizeof(gen_exp)} bytes")

# You can still iterate over the generator to get all the values
# total = sum(gen_exp)

Let's review these concepts before moving on.

Now, check your understanding with a few questions.

Quiz Questions 1/5

What is the output of the following Python code?

print([x*x for x in range(5) if x % 2 != 0])
Quiz Questions 2/5

Which collection type is specifically designed for fast additions and removals from both the beginning and the end of a sequence?

By using comprehensions, specialised collections, and generators, you can write Python code that is not only more efficient but also clearer and more expressive. These are the idioms that distinguish proficient Python programmers.