Applied Python for Data Science
NumPy Numerical Computing
Beyond Python Lists
You're already familiar with Python lists, which are incredibly flexible. They can hold different data types and change size dynamically. But this flexibility comes at a cost: performance. Each item in a Python list is a full-fledged Python object, stored in scattered memory locations. For numerical computing, this is slow.
NumPy's core data structure, the ndarray, or n-dimensional array, solves this problem. It's a grid of values, all of the same type, packed into one continuous block of memory. This compact storage allows NumPy to perform mathematical operations on the entire array at once, using highly optimized, pre-compiled C code instead of slower Python loops.
The key takeaway: NumPy arrays trade the flexibility of Python lists for a massive gain in speed and efficiency for numerical tasks.
import numpy as np
# A Python list
python_list = [1, 2, 3, 4, 5]
# Creating a NumPy array from the list
numpy_array = np.array(python_list)
print(numpy_array)
# Expected Output: [1 2 3 4 5]
# Notice the lack of commas; this is a NumPy array, not a list.
The Power of Vectorization
The most significant advantage of using NumPy is . This is the practice of replacing explicit loops with array expressions. Instead of iterating through an array element by element, you can apply an operation to the entire array at once.
Let's say we want to multiply every number in a large dataset by 1.5. With a standard Python list, you'd write a for loop. With NumPy, you just do it directly.
large_array = np.arange(1_000_000)
# The NumPy way (vectorized)
fast_result = large_array * 1.5
# The Python list way (loop)
slow_list = list(range(1_000_000))
slow_result = []
for item in slow_list:
slow_result.append(item * 1.5)
# Both produce the same result, but the NumPy version is vastly faster.
These vectorized operations are performed by Universal Functions, or ufuncs. These are functions that operate on ndarrays in an element-by-element fashion. Common mathematical operators like +, -, * have corresponding ufuncs (np.add, np.subtract, np.multiply), as do more complex functions like np.sin, np.log, and np.sqrt.
Smart Operations with Broadcasting
What happens when you want to perform an operation on two arrays of different shapes? For example, adding a single number to every element in an array. NumPy handles this with a powerful mechanism called .
Instead of creating a new array filled with that single number, NumPy virtually "stretches" or "broadcasts" the smaller array to match the shape of the larger one. This is done without actually using extra memory.
matrix = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
# Add a 1D array to a 2D array
# The 1D array [10, 20, 30] is broadcast across each row of the matrix.
row_vector = np.array([10, 20, 30])
result = matrix + row_vector
print(result)
# Expected Output:
# [[11 22 33]
# [14 25 36]
# [17 28 39]]
Accessing Your Data
Accessing elements in NumPy arrays is similar to Python lists but far more powerful. You can use slicing and indexing across multiple dimensions.
data = np.array([[10, 20, 30, 40],
[50, 60, 70, 80],
[90, 100, 110, 120]])
# Get a single element (row 1, column 2)
element = data[1, 2] # Result: 70
# Slice to get the first two rows and columns 1 through 3
sub_array = data[:2, 1:3]
# Result:
# [[20 30]
# [60 70]]
# Get a specific column (all rows, column 3)
column = data[:, 3] # Result: [ 40 80 120]
One of the most useful techniques is boolean masking. You can create a boolean array based on a condition and use it to select only the elements that meet the condition. This is an efficient way to filter your data without writing a loop.
ages = np.array([22, 35, 18, 45, 29, 60, 21])
# Create a boolean mask for ages over 30
mask = ages > 30
# The mask will be: [False, True, False, True, False, True, False]
# Use the mask to select elements
over_30 = ages[mask]
print(over_30)
# Expected Output: [35 45 60]
Linear Algebra and Statistics
NumPy is the foundation for scientific computing in Python, so it includes a rich library of mathematical and statistical functions. You can quickly get summaries of your data.
measurements = np.array([1.2, 1.5, 1.1, 1.8, 1.4, 2.1, 0.9])
print(f"Mean: {measurements.mean()}")
print(f"Standard Deviation: {measurements.std()}")
print(f"Sum: {measurements.sum()}")
It also has a powerful linear algebra module, linalg, for tasks like solving systems of equations, finding determinants, and performing matrix inversions. For example, to solve the system of equations:
We can represent this as a matrix equation and solve for .
# Coefficients matrix A
A = np.array([[2, 3],
[4, 1]])
# Constants vector b
b = np.array([8, 6])
# Solve the system for x and y
solution = np.linalg.solve(A, b)
print(solution)
# Expected Output: [1. 2.] (meaning x=1, y=2)
What is the primary reason NumPy's ndarray is more performant than a standard Python list for numerical computations?
The practice of replacing explicit loops with array expressions, such as my_array * 2 instead of iterating through each element, is known as:
With these fundamentals, you have the building blocks for nearly all numerical work in Python. The speed of vectorization and the convenience of broadcasting make NumPy an indispensable tool for data science.