No history yet

Pythonic Data Structures

Beyond Basic Lists

You're already familiar with building lists using simple for loops. But Python offers a more concise and often faster way to create lists: comprehensions. They pack the logic of a loop and conditional statements into a single, readable line.

Imagine you have a list of numbers and you want a new list containing only the squares of the even numbers. The traditional approach requires several lines of code.

numbers = [1, 2, 3, 4, 5, 6, 7, 8]
squared_evens = []
for num in numbers:
    if num % 2 == 0:
        squared_evens.append(num ** 2)
# squared_evens is now [4, 16, 36, 64]

A list comprehension achieves the same result in one go. The structure is [expression for item in iterable if condition].

numbers = [1, 2, 3, 4, 5, 6, 7, 8]
squared_evens = [num ** 2 for num in numbers if num % 2 == 0]
# squared_evens is also [4, 16, 36, 64]

This isn't just about saving space. List comprehensions are often implemented in C at a lower level in Python's source code, which can make them faster than an equivalent for loop with .append() calls. You can even nest them to work with more complex data, like flattening a matrix (a list of lists).

matrix = [[1, 2], [3, 4], [5, 6]]
flat_list = [num for row in matrix for num in row]
# flat_list is [1, 2, 3, 4, 5, 6]

Generators for Big Data

List comprehensions are great, but they have one drawback: they create the entire list in memory at once. This is fine for small lists, but what if you're processing a file with millions of records? Loading everything into memory could crash your program.

This is where generator expressions come in. They look almost identical to list comprehensions but use parentheses instead of square brackets. The key difference is that they practice —they generate values one at a time, on demand.

# List comprehension (eager evaluation)
# This builds a list of 100 million numbers in memory
list_comp = [i for i in range(100_000_000)]

# Generator expression (lazy evaluation)
# This creates a generator object, using almost no memory
gen_exp = (i for i in range(100_000_000))

The generator expression doesn't hold the numbers. It holds the recipe for producing the numbers. You can iterate over it just like a list, but each value is computed just in time and then discarded from memory.

Use a list comprehension when you need the full list immediately, like for sorting or slicing. Use a generator expression when you're working with large datasets or chaining operations, and only need to process one item at a time.

The Collections Powerhouse

Python's built-in data types are powerful, but sometimes you need something more specialized. The collections module provides high-performance container datatypes that act as powerful alternatives to the standard dict, list, set, and tuple.

Lesson image

One of the most useful is NamedTuple. It allows you to create simple, immutable classes for grouping data without the boilerplate of a full class definition. It makes your code more readable by allowing you to access elements by name instead of by index.

from collections import namedtuple

# Define the structure
Point = namedtuple('Point', ['x', 'y'])

# Create an instance
p1 = Point(10, 20)

# Access data by name (more readable)
print(f"X is {p1.x}, Y is {p1.y}")

# Access by index still works
print(f"The sum is {p1[0] + p1[1]}")

Another workhorse is defaultdict. Have you ever written code to count items and had to check if a key already exists before incrementing it? defaultdict solves this elegantly. When you create one, you provide a default factory (like int or list). If you try to access a key that doesn't exist, it automatically creates it using that factory.

from collections import defaultdict

# Create a defaultdict where the default value for a new key is an integer (0)
word_counts = defaultdict(int)

sentence = "the quick brown fox jumps over the lazy dog"
for word in sentence.split():
    word_counts[word] += 1 # No need to check if the key exists!

# word_counts['the'] will be 2

Two other handy tools are Counter and deque.

Counter is essentially a defaultdict(int) on steroids, purpose-built for tallying objects. It even includes a .most_common() method.

deque (pronounced 'deck') stands for double-ended queue. It's like a list but optimized for adding and removing elements from both the front and the back, offering near constant-time O(1) performance for these operations, whereas a list's pop(0) is a costly O(n) operation.

from collections import Counter, deque

# Counter example
counts = Counter('mississippi')
# counts will be {'i': 4, 's': 4, 'p': 2, 'm': 1}
print(counts.most_common(2)) # -> [('i', 4), ('s', 4)]

# Deque example
d = deque([1, 2, 3])
d.appendleft(0)   # Fast append to the left
d.pop()           # Fast pop from the right
# d is now deque([0, 1, 2])

Sets and Smarter Dictionaries

When it comes to performance, understanding the right data structure can make a world of difference. For checking if an item exists in a collection, sets are vastly superior to lists. A set uses a hash table internally, which allows it to check for membership in O(1) average time, regardless of the size. A list, on the other hand, has to check every single element, an O(n) operation.

If your code frequently uses item in my_collection, and the order doesn't matter, use a set, not a list.

Sets also give you powerful mathematical operations like union (|), intersection (&), and difference (-) right out of the box.

set_a = {1, 2, 3, 4}
set_b = {3, 4, 5, 6}

# Union: items in either set
print(set_a | set_b)  # {1, 2, 3, 4, 5, 6}

# Intersection: items in both sets
print(set_a & set_b)  # {3, 4}

# Difference: items in set_a but not in set_b
print(set_a - set_b)  # {1, 2}

Finally, let's look at dictionaries. When you call .keys(), .values(), or .items(), you don't get a list. You get a special dictionary view object. These views are dynamic windows into the dictionary's entries. If the dictionary changes, the view reflects those changes immediately.

my_dict = {'a': 1, 'b': 2}

items_view = my_dict.items()
print(items_view)  # dict_items([('a', 1), ('b', 2)])

# Modify the original dictionary
my_dict['c'] = 3

# The view updates automatically!
print(items_view)  # dict_items([('a', 1), ('b', 2), ('c', 3)])

And for a common task, merging dictionaries, Python 3.9 introduced a clean new syntax using the union operator. Before that, the standard was to use dictionary unpacking.

dict1 = {'a': 1, 'b': 2}
dict2 = {'b': 3, 'c': 4}

# Modern way (Python 3.9+)
merged = dict1 | dict2
# merged is {'a': 1, 'b': 3, 'c': 4}
# Note: values from the right-hand dict win

# Older way (still works!)
merged_older = {**dict1, **dict2}
# result is the same

Choosing the right data structure is a key step in writing professional, efficient Python code. By moving beyond basic lists and dictionaries, you can leverage these more advanced tools to make your programs faster, cleaner, and more robust.