Scalable Python and Application Architecture
Pythonic Patterns
Beyond the Basics
You've mastered functions, loops, and variables. Now it's time to write code that isn't just correct, but also clean, efficient, and reusable. The goal is to follow the DRY principle: Don't Repeat Yourself. Instead of copying and pasting logic across your application, you can abstract it away into reusable components.
Two powerful tools for this are decorators and context managers. They allow you to add functionality and manage resources elegantly, separating core business logic from common, repetitive tasks like logging, timing, or connecting to a database.
Functions That Modify Functions
In Python, functions are first-class objects. This means you can pass them around just like any other variable. A decorator is essentially a function that takes another function as an argument, adds some functionality, and returns a new, enhanced function.
Let's say you want to time how long several different functions take to run. The clumsy way would be to add timing code inside each one. The Pythonic way is to write a decorator.
import time
def timing_decorator(func):
def wrapper(*args, **kwargs):
start_time = time.time()
result = func(*args, **kwargs) # Call the original function
end_time = time.time()
print(f"{func.__name__} ran in: {end_time - start_time:.4f} secs")
return result
return wrapper
@timing_decorator
def slow_function():
time.sleep(2)
print("Function finished!")
slow_function()
The @timing_decorator syntax is just a shortcut, or "syntactic sugar," for slow_function = timing_decorator(slow_function). The wrapper function has access to func because of a principle called and is known as a closure. This is what allows the decorator to "wrap" the original function without permanently altering it.
Decorators are ideal for handling cross-cutting concerns—features like logging, timing, or authentication that are needed in many different parts of an application.
Decorators with State
Sometimes you need a decorator that keeps track of some information, or state. For example, what if you wanted to count how many times a function is called? While you could use global variables, a cleaner solution is a class-based decorator.
A class can be a decorator if it implements the __call__ method. This allows an instance of the class to be called like a function.
class CallCounter:
def __init__(self, func):
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"Call {self.count} of {self.func.__name__}")
return self.func(*args, **kwargs)
@CallCounter
def say_hello():
print("Hello!")
say_hello() # Output: Call 1 of say_hello
say_hello() # Output: Call 2 of say_hello
One small problem with our decorators so far is that they obscure the metadata of the original function. If you check slow_function.__name__, you'll find it's wrapper, not slow_function. The solution is to use a decorator that copies the original function's name, docstring, and other attributes to the wrapper function.
Managing Resources Safely
Properly managing resources like file handles, network connections, or database sessions is critical. You need to ensure they are always closed, even if errors occur. The classic way to do this is with a try...finally block.
file = open('example.txt', 'w')
try:
file.write('Hello, world!')
finally:
file.close()
This works, but it's verbose. Python offers a much cleaner solution: the with statement and context managers. The with statement guarantees that cleanup code is executed, no matter what happens in the block.
Any object that can be used with a with statement is a context manager. It just needs to follow a specific protocol by implementing two special methods: __enter__ and __exit__.
| Method | Purpose |
|---|---|
__enter__(self) | Called when entering the with block. Its return value is assigned to the variable after as, if there is one. |
__exit__(self, exc_type, exc_val, exc_tb) | Called when exiting the block. It handles cleanup. If an exception occurred, the details are passed as arguments. |
Let's create a simple context manager that simulates a database connection.
class DatabaseConnection:
def __enter__(self):
print("Connecting to the database...")
# In a real scenario, this would return a connection object
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print("Closing the database connection.")
def query(self, sql):
print(f"Executing query: {sql}")
with DatabaseConnection() as db:
db.query("SELECT * FROM users")
Notice how __exit__ is called automatically. This makes your code more readable and robust. You can even write a context manager without a class, using a special decorator from the library.
from contextlib import contextmanager
@contextmanager
def managed_file(name, mode):
f = None
try:
f = open(name, mode)
yield f
finally:
if f:
f.close()
print(f"File '{name}' closed.")
with managed_file('hello.txt', 'w') as f:
f.write('hello, world')
print("Writing to file...")
Everything before the yield statement is treated as the __enter__ method. The yield passes control back to the with block. When the block finishes, the code after the yield runs as the __exit__ method.
What is the primary principle that decorators and context managers help enforce in Python programming?
Consider the following code. What will be printed to the console?
def my_decorator(func):
def wrapper(*args, **kwargs):
# ... some logic ...
return func(*args, **kwargs)
return wrapper
@my_decorator
def say_hello():
"""A simple greeting function."""
print("Hello!")
print(say_hello.__name__)
By mastering decorators and context managers, you can write Python code that is not only functional but also elegant, robust, and easy to maintain.