No history yet

Advanced Functional Tools

Functions on Functions: Decorators

In Python, functions are first-class citizens. You can pass them as arguments, return them from other functions, and assign them to variables. This flexibility opens up powerful techniques, one of which is the decorator.

A decorator is a function that takes another function as an argument, adds some functionality, and returns a new function without altering the original function's code. Think of it like gift wrapping. You take an item (the function), wrap it in decorative paper (the decorator), and you're left with a more festive version of the same item.

Let's say you want to time how long a function takes to run. Instead of adding timing logic inside every function you want to measure, you can write a single decorator.

import time

def timer_decorator(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"{func.__name__} ran in: {end_time - start_time:.4f} secs")
        return result
    return wrapper

@timer_decorator
def process_data(size):
    """A sample function that simulates data processing."""
    time.sleep(size)
    print(f"Processed {size} items.")

process_data(2)

# Output:
# Processed 2 items.
# process_data ran in: 2.0035 secs

The @timer_decorator syntax is just a cleaner way of writing process_data = timer_decorator(process_data). It swaps our original function with the decorated version.

This works, but there's a subtle problem. The decorator replaces our original function with the wrapper function. If you check process_data.__name__ or process_data.__doc__, you'll get the details for wrapper, not process_data. This can cause issues with debugging and introspection tools.

To fix this, we use @wraps from Python's built-in functools module. It's a decorator for your decorator that copies the original function's metadata to the wrapper function.

import time
from functools import wraps

def timer_decorator(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"{func.__name__} ran in: {end_time - start_time:.4f} secs")
        return result
    return wrapper

@timer_decorator
def process_data(size):
    """A sample function that simulates data processing."""
    time.sleep(size)
    print(f"Processed {size} items.")

print(process_data.__name__)    # Output: process_data
print(process_data.__doc__)     # Output: A sample function that simulates data processing.

Decorators are possible because of a concept called . A closure is an inner function that remembers and has access to variables from the local scope in which it was created, even after the outer function has finished executing. In our example, wrapper is a closure that

'remembers' the func variable passed to timer_decorator.

Memory-Wise Iteration: Generators

When you need to process a large sequence of items, like lines in a multi-gigabyte file, creating a list of all items can consume a huge amount of memory. Generators solve this problem by producing items one at a time, on demand.

Instead of return, a generator function uses the keyword. When the function is called, it returns a generator object but doesn't start running yet. The code only executes when you iterate over the generator, for example in a for loop. Each time the loop asks for the next item, the function runs until it hits a yield statement, which passes the value back. It then pauses, preserving its local state, waiting for the next call.

def count_up_to(max_num):
    print("Generator started")
    count = 1
    while count <= max_num:
        yield count
        count += 1

counter = count_up_to(3)
print(counter) # <generator object count_up_to at 0x...>

print(next(counter)) # Generator started, prints 1
print(next(counter)) # prints 2
print(next(counter)) # prints 3

You can also create simple generators on the fly using generator expressions. They look like list comprehensions but use parentheses instead of square brackets. This creates a generator object without building the full list in memory.

# List comprehension (uses more memory)
squares_list = [x*x for x in range(1000)]

# Generator expression (memory efficient)
squares_generator = (x*x for x in range(1000))

# Both can be iterated over the same way
for i in squares_generator:
    # process i
    pass

Managing Resources: Context Managers

Certain tasks, like working with files or network connections, require setup and teardown actions. You need to open a file and, crucially, ensure it's closed afterward, even if errors occur. Forgetting to close a file can lead to resource leaks.

The with statement simplifies this by using . A context manager is an object that defines the methods __enter__() for setup and __exit__() for teardown. The with statement guarantees that __exit__() is called, no matter what happens inside the block.

# The old, error-prone way
f = open('my_file.txt', 'w')
try:
    f.write('Hello, world!')
finally:
    f.close()

# The clean, safe way with a context manager
with open('my_file.txt', 'w') as f:
    f.write('Hello, world!')
# The file is automatically closed here.

You can create your own context managers using classes, but Python provides a more convenient way with the contextlib module and a generator. By decorating a generator function with @contextmanager, you can create a context manager where the code before the yield is the setup (__enter__) and the code after is the teardown (__exit__).

from contextlib import contextmanager

@contextmanager
def managed_resource(name):
    print(f"Setting up resource: {name}")
    try:
        yield name # The value passed to the 'as' variable
    finally:
        print(f"Tearing down resource: {name}")

with managed_resource("Database Connection") as res:
    print(f"Using {res}")
    # raise ValueError("Something went wrong") # Uncomment to see teardown still works

# Output:
# Setting up resource: Database Connection
# Using Database Connection
# Tearing down resource: Database Connection

These functional tools help you write code that is more efficient, robust, and readable. They are staples in modern Python development, especially for data processing and resource management.

Let's check your understanding of these advanced concepts.

Quiz Questions 1/7

What is the primary purpose of a decorator in Python?

Quiz Questions 2/7

Consider the following code:

@my_decorator
def say_hello():
    print("Hello!")

This syntax is equivalent to which of the following lines of code?