No history yet

Advanced Data Structures

Beyond Basic Lists and Dictionaries

You've already got a handle on Python's workhorses: lists and dictionaries. They're great for general-purpose storage, but relying on them for every problem is like using a single wrench to build a car. Sometimes you need a specialized tool for a specific job, one that offers better performance, readability, or memory efficiency.

Let's start by upgrading how we create these structures. Instead of looping and appending, we can build them in a single, expressive line using comprehensions.

# Standard loop to create a list of squares of even numbers
numbers = [1, 2, 3, 4, 5, 6]
squares = []
for num in numbers:
    if num % 2 == 0:
        squares.append(num * num)
# squares is now [4, 16, 36]

# The comprehension equivalent is cleaner and faster
squares_comp = [num * num for num in numbers if num % 2 == 0]
# squares_comp is also [4, 16, 36]

Comprehensions aren't just for lists. You can use them to build dictionaries with logic, too. This is incredibly useful for reshaping data, like when you're preparing information to be sent to a web API.

# Create a dictionary from a list, mapping items to their length
words = ['apple', 'banana', 'cherry']
word_lengths = {word: len(word) for word in words}
# {'apple': 5, 'banana': 6, 'cherry': 6}

# Filter an existing dictionary to create a new one
prices = {'milk': 2.50, 'bread': 3.00, 'eggs': 4.75}
cheap_items = {item: price for item, price in prices.items() if price < 3.50}
# {'milk': 2.50, 'bread': 3.00}

The Collections Toolbox

Python comes with a built-in module called collections that provides high-performance, specialized container types. Think of it as a professional workshop for your data. When a standard list or dictionary feels clumsy or slow, there's probably a tool in here that's perfect for the job.

Let's look at namedtuple. Tuples are memory-efficient, but accessing data by index (employee[1]) can make code hard to read. A namedtuple solves this by giving you the best of both worlds: the low overhead of a tuple with the readability of object attribute access (employee.name).

from collections import namedtuple

# Define the structure of our 'Employee' namedtuple
Employee = namedtuple('Employee', ['id', 'name', 'role'])

# Create an instance
emp1 = Employee(101, 'Alice', 'Engineer')

# Access data by name, which is much clearer than by index
print(f"{emp1.name} is an {emp1.role}.")
# Output: Alice is an Engineer.

# You can still access by index if you need to
print(f"ID: {emp1[0]}")
# Output: ID: 101

Another powerful tool is the deque. While lists are great, they are slow if you need to add or remove items from the beginning. Doing my_list.pop(0) requires shifting every single element in the list, which is inefficient for large lists.

A deque (pronounced 'deck') is a 'double-ended queue' designed for fast appends and pops from both ends. It's ideal for implementing queues or keeping a running history of the last few items seen.

from collections import deque

# Create a deque, optionally with a max length
history = deque(maxlen=3)

history.append('action1')
history.append('action2')
history.append('action3')
print(history)  # deque(['action1', 'action2', 'action3'], maxlen=3)

# Add a new item; the oldest one ('action1') is automatically dropped
history.append('action4')
print(history)  # deque(['action2', 'action3', 'action4'], maxlen=3)

# Efficiently add to the left side
history.appendleft('action0') # 'action4' is dropped
print(history) # deque(['action0', 'action2', 'action3'], maxlen=3)

Sets and Generators

Sometimes you don't care about the order of items, but you need to know if an item exists in a collection or find the unique items between two collections. This is where sets shine. Based on set theory from mathematics, sets provide lightning-fast membership testing (in) and operations to find unions, intersections, and differences.

# A set of users who liked an article
article_likers = {'Alice', 'Bob', 'Charlie', 'David'}

# A set of users who commented
commenters = {'Charlie', 'Eve', 'Frank', 'Alice'}

# Who both liked and commented? (Intersection)
engaged_users = article_likers.intersection(commenters)
# {'Alice', 'Charlie'}

# Who liked the article but did not comment? (Difference)
lurkers = article_likers.difference(commenters)
# {'Bob', 'David'}

# All unique users who interacted in any way (Union)
all_users = article_likers.union(commenters)
# {'Eve', 'Charlie', 'Bob', 'Frank', 'David', 'Alice'}

Finally, let's talk about memory. A list comprehension is great, but it builds the entire list in memory at once. If you're working with millions of items, this can consume a huge amount of RAM.

This is where come in. A generator expression looks almost identical to a list comprehension, but it uses parentheses instead of square brackets. Instead of building a full list, it creates a generator object that yields one item at a time, on demand. This is incredibly memory-efficient for processing large datasets.

import sys

# A list comprehension that uses more memory
list_comp = [i for i in range(100000)]
print(f"List size: {sys.getsizeof(list_comp)} bytes")

# A generator expression that is memory efficient
gen_exp = (i for i in range(100000))
print(f"Generator size: {sys.getsizeof(gen_exp)} bytes")

# The generator doesn't produce values until you loop over it
# total = sum(gen_exp)

By moving beyond the basics, you can write Python code that is not only cleaner and more readable but also significantly more performant and memory-conscious. Choosing the right data structure is a key step in leveling up as a developer.