Optimization is about finding the best possible result under certain conditions. In mathematics, this usually means finding the maximum or minimum value of a function. The function we want to maximize or minimize is called the objective function.

Formulating this function is the first step. For example, if a company wants to maximize profit, its objective function would model that profit. Let's say it costs $50 to produce an item, and the selling price $p$ depends on the quantity $q$ produced, following the demand equation $p(q) = 200 - 2q$. The revenue is price times quantity, $R(q) = p(q) \times q = (200 - 2q)q$. The total cost is $C(q) = 50q$. The profit function, our objective function, is the difference between revenue and cost.

Our goal is to find the value of $q$ that makes $P(q)$ as large as possible.

To find where a function reaches a maximum or minimum, we look for its critical points. These are the points where the function's slope is zero, meaning the tangent line is horizontal. Since the first derivative, $f'(x)$, gives us the slope, we find these points by setting the derivative to zero and solving for the variable.

For our profit function, $P(q) = 150q - 2q^2$, the first derivative is:

Setting this to zero gives us $150 - 4q = 0$, which solves to $q = 37.5$. This is our critical point. But is it a maximum, a minimum, or something else? The second derivative test helps us figure that out.

The second derivative, $f''(x)$, tells us about the concavity of the function.

• If $f''(x) < 0$ at a critical point, the function is concave down (like a frown), indicating a local maximum.
• If $f''(x) > 0$, the function is concave up (like a cup), indicating a local minimum.
• If $f''(x) = 0$, the test is inconclusive.

Let's find the second derivative of our profit function:

Since $P''(q) = -4$ is negative, our critical point at $q = 37.5$ corresponds to a maximum profit. The company should produce 37.5 units. In a real-world scenario, they would likely produce either 37 or 38 units.

Often, we don't just want to maximize or minimize a function in a vacuum. We have limitations, or constraints. For example, a farmer wants to maximize the area of a rectangular field but only has a limited amount of fencing. This is a constrained optimization problem.

The method of Lagrange multipliers is a powerful technique for solving these kinds of problems. It allows us to find the maximum or minimum of a multivariable function $f(x, y, ...)$ subject to a constraint $g(x, y, ...) = c$.

The core idea is that at an optimal point, the gradient of the objective function is parallel to the gradient of the constraint function. The gradient, denoted by $\nabla$, is a vector of partial derivatives that points in the direction of the steepest ascent of a function. If the gradients are parallel, one must be a scalar multiple of the other. We call this scalar multiplier $\lambda$ (lambda).

Let's use our farmer example. They want to maximize the area $A(x,y) = xy$ of a rectangular pen. They have 100 meters of fencing, so the perimeter must be $2x + 2y = 100$.

Our objective function is $f(x, y) = xy$. Our constraint is $g(x, y) = 2x + 2y = 100$.

We need to solve a system of equations derived from $\nabla f = \lambda \nabla g$ and the constraint itself:

1. $\frac{\partial f}{\partial x} = \lambda \frac{\partial g}{\partial x} \implies y = \lambda(2)$
2. $\frac{\partial f}{\partial y} = \lambda \frac{\partial g}{\partial y} \implies x = \lambda(2)$
3. $g(x, y) = 100 \implies 2x + 2y = 100$

From the first two equations, we see that $x = y$. Substituting this into the third equation gives $2x + 2x = 100$, or $4x = 100$, so $x=25$. Since $x=y$, we also have $y=25$. The dimensions that maximize the area are 25m by 25m, a perfect square.

These techniques form the foundation of optimization. They are used everywhere, from engineering design and financial modeling to machine learning algorithms, helping us find the most efficient and effective solutions to complex problems.