At its core, calculus is the study of change. But before we can talk about change, we need a way to describe relationships between things. That's where functions come in. A function is simply a rule that takes an input and produces a single, predictable output.

Think of a coffee machine. You put in coffee beans (the input), and it gives you coffee (the output). The machine is the function. For any given input of beans, you get a specific output of coffee. It won't surprise you with tea. In mathematics, we often write a function as $f(x)$, where $f$ is the rule, $x$ is the input, and $f(x)$ (read as "f of x") is the output.

For example, the function $f(x) = x + 5$ is a simple rule that says "take any number, $x$, and add 5 to it." If you input 3, the output is $f(3) = 3 + 5 = 8$. If you input -10, the output is $f(-10) = -10 + 5 = -5$.

Every function has two important sets of values associated with it: its domain and its range.

For our simple function $f(x) = x + 5$, you can plug in any real number for $x$ without breaking the rule. So, its domain is all real numbers. Similarly, the output can be any real number, so its range is also all real numbers.

But consider the function $g(x) = \sqrt{x}$. Since we can't take the square root of a negative number (in the realm of real numbers), the domain is restricted to all non-negative numbers ($x \ge 0$). The output will also always be non-negative, so the range is also all numbers greater than or equal to zero.

Understanding a function's domain and range is critical in machine learning. It helps define the valid inputs for a model and anticipate the kinds of outputs it will generate, preventing errors and nonsensical results.

Now for one of the biggest ideas in all of calculus: the limit. A limit describes the value that a function approaches as the input gets closer and closer to a certain number. The key idea is that we don't care what happens exactly at the number, only what happens in its immediate neighborhood.

We write the limit of a function $f(x)$ as $x$ approaches a value $c$ like this:

This reads as "the limit of $f(x)$ as $x$ approaches $c$ equals $L$." It means that as $x$ gets closer and closer to $c$ (from both sides), the output value $f(x)$ gets closer and closer to $L$.

For many simple functions, finding the limit is as easy as plugging the number in. These are called continuous functions. A continuous function is one you can draw without lifting your pencil from the paper; it has no jumps, holes, or gaps.

For the function $f(x) = x + 5$, what is the limit as $x$ approaches 3? We can just plug it in: $\lim_{x \to 3} (x+5) = 3 + 5 = 8$. The value the function approaches is the same as the function's value at that point.

But what about a function like this?

If we try to find the limit as $x$ approaches 2, we run into a problem. Plugging in $x=2$ gives us $\frac{0}{0}$, which is undefined. The function has a hole at $x=2$. This is where limits shine. We can see what value the function is approaching from either side of 2.

Let's try a number very close to 2, like 2.001: $f(2.001) = \frac{(2.001)^2 - 4}{2.001 - 2} = \frac{4.004001 - 4}{0.001} = \frac{0.004001}{0.001} = 4.001$. Now let's try a number just below 2, like 1.999: $f(1.999) = \frac{(1.999)^2 - 4}{1.999 - 2} = \frac{3.996001 - 4}{-0.001} = \frac{-0.003999}{-0.001} = 3.999$.

The closer we get to 2 from either side, the closer the output gets to 4. We can also simplify the function algebraically for any $x \ne 2$: $f(x) = \frac{(x-2)(x+2)}{x-2} = x+2$. This simplified function makes it clear that as $x$ approaches 2, the value approaches $2+2=4$. So, even though the function is undefined at $x=2$, its limit is 4.

A function is continuous at a point $c$ if three conditions are met: $f(c)$ is defined, the limit as $x \to c$ exists, and the limit equals the function's value. In short: $\lim_{x \to c} f(x) = f(c)$.

These concepts of functions, limits, and continuity form the bedrock of calculus. They allow us to handle ideas of infinite closeness and rates of change, which are essential for building and optimizing the complex algorithms used in machine learning.

With these foundational ideas in place, we're ready to explore how calculus measures change.