Introduction to Python

The Building Blocks of Python

Python is a versatile programming language, popular in data engineering because its syntax is clean and easy to read. Unlike many other languages that use brackets or keywords to define blocks of code, Python uses simple indentation. This forces code to be organized and readable.

Think of it as an outline. The indentation shows which lines of code belong to a particular section. This structure is not just for looks; it's a rule the language enforces. You also use comments to explain your code. In Python, comments start with a hash symbol (#) and are ignored by the computer.

# This is a comment. It explains the code but doesn't run.

# The print() function displays text to the screen.
print("Hello, World!")

Handling Data

At its core, programming is about working with data. Python has several built-in data types to handle different kinds of information. The most basic types include numbers for calculations, strings for text, and booleans for true/false logic.

For more complex needs, Python offers data structures that can hold collections of data. Two of the most common are lists and dictionaries.

Lists are ordered collections of items. You can store anything in a list, and you can change it after you create it. Think of a shopping list where you can add or remove items.
Dictionaries are unordered collections of key-value pairs. Instead of using an index number, you access a value using its unique key. It's like a real dictionary, where you look up a word (the key) to find its definition (the value).

Data Type	Description	Example
Integer	Whole numbers, positive or negative.	`42`, `-100`
Float	Numbers with a decimal point.	`3.14`, `-0.001`
String	A sequence of characters, used for text.	`"hello"`, `'Data Engineering'`
Boolean	Represents one of two values: True or False.	`True`, `False`

Here's how you might create a list of data sources and a dictionary with details about a specific dataset.

# A list of data sources
data_sources = ["PostgreSQL", "API", "CSV file"]

# Accessing the first item (indexing starts at 0)
print(data_sources[0]) # Output: PostgreSQL

# A dictionary describing a dataset
customer_data = {
  "name": "Customer Transactions",
  "format": "CSV",
  "records": 15000
}

# Accessing a value by its key
print(customer_data["format"]) # Output: CSV

Controlling the Flow

Data pipelines often require logic to handle different scenarios. Control structures allow you to direct the flow of your program based on certain conditions. The if statement is used to make decisions. It runs a block of code only if a specific condition is true.

You can also repeat actions using loops. The for loop is perfect for iterating over a sequence, like a list. For each item in the sequence, it executes a block of code. This is incredibly useful for processing records one by one.

# An if-elif-else statement
file_format = "JSON"

if file_format == "CSV":
  print("Processing CSV file.")
elif file_format == "JSON":
  print("Processing JSON file.")
else:
  print("Unsupported file format.")

Here is how you would use a for loop to process each item in our previous list.

# A for loop to iterate over a list
data_sources = ["PostgreSQL", "API", "CSV file"]

for source in data_sources:
  print(f"Connecting to {source}...")

Functions and Error Handling

As your scripts get more complex, you'll find yourself writing the same code over and over. Functions solve this by letting you package a block of code that you can name and reuse. A function can take inputs, called parameters, and can return a result.

This makes your code more organized, readable, and easier to maintain. Instead of copying and pasting code, you just call the function.

# Define a function that takes two parameters
def add_numbers(a, b):
  result = a + b
  return result

# Call the function with two arguments
sum_result = add_numbers(5, 10)
print(sum_result) # Output: 15

Even with perfect code, things can go wrong. A network connection might fail, or a file might be missing. Error handling allows you to manage these exceptions gracefully. In Python, you use a try...except block.

The code that might cause an error goes inside the try block. If an error occurs, the code in the except block is executed. This prevents your entire script from crashing and allows you to handle the problem, perhaps by logging the error or trying an alternative action.

# Handle a potential ZeroDivisionError
numerator = 10
denominator = 0

try:
  result = numerator / denominator
  print(result)
except ZeroDivisionError:
  print("Error: Cannot divide by zero.")

Let's review the core concepts we've covered.

Time to test your knowledge.

Quiz Questions 1/6

What does Python use to define code blocks, such as the body of a loop or function?

Quiz Questions 2/6

Which data structure is an unordered collection of key-value pairs?

With these fundamentals, you have the basic tools to start writing simple Python scripts for data tasks. You can define variables, use different data types, control your script's logic, and handle errors.

Python and SQL for Data Engineering Foundations