Linear regression is powerful for modeling relationships that follow a straight line. But most real-world data isn't that tidy. To capture complex, non-linear patterns, we need a more flexible structure. This is where neural networks, specifically the Multi-Layer Perceptron (MLP), come into play.

An MLP is composed of simple computational units called neurons, organized into layers. Think of it like a team of specialists. The first layer receives the raw data, just like a receptionist collects initial information. Each subsequent layer builds on the work of the previous one, finding more abstract and complex patterns, until the final layer makes a decision or prediction.

The three main types of layers are:

• Input Layer: This is the entry point. It has one neuron for each feature in your dataset. If you're predicting house prices from size and location, your input layer has two neurons.
• Hidden Layers: These are the layers between the input and output. They are where the network learns to represent the data in increasingly complex ways. An MLP can have one, several, or many hidden layers.
• Output Layer: This is the final layer. It produces the network's prediction. The number of neurons here depends on the task. For predicting a single value like a price, it's one neuron. For classifying an image into one of ten categories, it's ten neurons.

Each neuron in a hidden or output layer performs a two-step calculation. First, it computes a weighted sum of its inputs, then adds a bias. Think of weights as dials that tune the importance of each input signal, and the bias as a way to shift the output up or down. A neuron's initial calculation is essentially a linear regression.

If we stopped here, stacking layers would be pointless. A series of linear operations can always be simplified into a single linear operation. To introduce the non-linearity needed to model complex data, we pass this result, $z$, through an activation function.

Activation functions are the key to a neural network's power. They are simple, non-linear transformations applied to the neuron's output. By breaking up the linearity, they allow the network to learn intricate curves and boundaries in the data.

One of the most common activation functions is the Rectified Linear Unit, or ReLU for short. Its rule is simple: if the input is positive, the output is the input itself. If the input is negative, the output is zero.

Another classic is the Sigmoid function, which squashes any input value into a range between 0 and 1. This makes it useful for the output layer when predicting probabilities.

A related function, Tanh (hyperbolic tangent), squashes values into a range between -1 and 1. Its zero-centered nature can sometimes help networks learn faster than Sigmoid.

The entire process of passing data from the input layer, through the hidden layers, and to the output layer is called forward propagation. At each step, neurons calculate their weighted sums and apply their activation functions, passing the results to the next layer. This forward pass produces the network's initial guess or prediction for a given input.

This architecture of layers, weights, biases, and activation functions gives neural networks their remarkable ability to approximate any continuous function. The learning process, which we'll cover next, is all about finding the right values for those weights and biases to make the network's predictions as accurate as possible.