Python Programming Comprehensive Path
Data Structure Patterns
Smarter, Not Harder, with Comprehensions
You already know how to build a list from a loop using a basic list comprehension. It’s clean, readable, and Pythonic. But we can push them further to handle more complex logic in a single, expressive line.
Imagine you have a list of numbers and you want to create a new list labelling them as 'even' or 'odd'. A standard loop with an if/else block works, but a comprehension is more direct.
numbers = [1, 2, 3, 4, 5, 6]
# Using a conditional expression inside a list comprehension
labels = ['even' if x % 2 == 0 else 'odd' for x in numbers]
print(labels)
# Output: ['odd', 'even', 'odd', 'even', 'odd', 'even']
This pattern extends to dictionaries, too. Dictionary comprehensions are perfect for transforming or filtering existing dictionaries. For instance, you can easily swap keys and values or create a new dictionary containing only items that meet a certain condition.
stock_prices = {'AAPL': 170, 'GOOG': 2800, 'MSFT': 300}
# Create a new dictionary with only stocks priced over 💲200
expensive_stocks = {ticker: price for ticker, price in stock_prices.items() if price > 200}
print(expensive_stocks)
# Output: {'GOOG': 2800, 'MSFT': 300}
Efficiency Is Everything
Writing clean code is important, but writing efficient code is what separates a good programmer from a great one. When you’re working with large datasets, the choice of data structure can be the difference between a script that runs in seconds and one that takes hours.
To measure this efficiency, we use Big O notation to describe how the runtime or memory usage of an algorithm grows as the input size increases. It gives us a high-level way to compare different approaches.
| Operation | list | set | dict |
|---|---|---|---|
Item Lookup (e.g., x in my_container) | O(n) | O(1) | O(1) |
| Item Insertion | O(1)* | O(1) | O(1) |
| Item Deletion | O(n) | O(1) | O(1) |
*List insertion is O(1) only when appending to the end (.append()). Inserting elsewhere is O(n).
Why are sets and dictionaries so fast for lookups? The secret is hashing. When you add an item to a set or dictionary, Python runs the key through a hash function, which converts it into a memory address. To check if the item exists, Python just re-calculates the hash and looks at that specific memory spot. This is much faster than checking every single element in a list, one by one.
This makes sets incredibly useful for tasks like finding unique items or checking for membership in a large collection. If you need to repeatedly check if an item is in a list, converting that list to a set first can dramatically speed up your code.
# Scenario: Check for thousands of user IDs in a large list of banned IDs.
banned_ids_list = [123, 456, 789, ...] # A very long list
# Slow approach: O(n) for each check
for user_id in users_to_check:
if user_id in banned_ids_list:
# ... take action
# Fast approach: Convert to a set for O(1) checks
banned_ids_set = set(banned_ids_list) # O(n) conversion, but done only once
for user_id in users_to_check:
if user_id in banned_ids_set:
# ... take action (this check is now super fast)
The Collections Powerhouse
Python's standard library includes the collections module, which provides specialized, high-performance data structures that solve common problems more effectively than the general-purpose list, dict, and tuple.
First up is namedtuple. It lets you create simple, immutable objects for bundling data, much like a read-only class, but with less memory overhead. It makes your code more readable by allowing you to access fields by name instead of by index.
from collections import namedtuple
# Define the structure
Point = namedtuple('Point', ['x', 'y'])
# Create an instance
p1 = Point(10, 20)
print(p1.x, p1.y) # Access by name
# Output: 10 20
print(p1[0], p1[1]) # Can still access by index
# Output: 10 20
Next is deque (pronounced 'deck'), which stands for double-ended queue. While a list is efficient for adding or removing items from the end, it's very slow for doing the same at the beginning, because all other elements must be shifted. A deque is optimized for fast appends and pops from both ends, making it perfect for implementing queues or keeping a running history of recent items.
from collections import deque
# Create a deque with a max length of 3
last_three_actions = deque(maxlen=3)
last_three_actions.append('login')
last_three_actions.append('view_page')
last_three_actions.append('edit_profile')
print(last_three_actions)
# Output: deque(['login', 'view_page', 'edit_profile'], maxlen=3)
last_three_actions.append('logout') # 'login' is automatically pushed out
print(last_three_actions)
# Output: deque(['view_page', 'edit_profile', 'logout'], maxlen=3)
# Efficiently add to the left
last_three_actions.appendleft('admin_login')
print(last_three_actions)
# Output: deque(['admin_login', 'view_page', 'edit_profile'], maxlen=3)
Finally, the Counter object is a specialized dictionary subclass for counting hashable objects. It’s a simple but powerful tool for data analysis and tallying.
from collections import Counter
words = ['apple', 'banana', 'apple', 'orange', 'banana', 'apple']
word_counts = Counter(words)
print(word_counts)
# Output: Counter({'apple': 3, 'banana': 2, 'orange': 1})
# Find the 2 most common words
print(word_counts.most_common(2))
# Output: [('apple', 3), ('banana', 2)]
By understanding the performance characteristics of these built-in and collections data structures, you can start writing more professional, scalable Python code. Now, let's test what you've learned.
What is the primary purpose of Big O notation in the context of algorithms?
Why are membership checks (e.g., item in my_collection) significantly faster for a set than for a list when dealing with large datasets?