A neural network processes information in a way that's loosely inspired by the human brain. It's made up of layers of interconnected nodes, or "neurons." Information flows from one layer to the next, getting transformed at each step. This process allows the network to recognize complex patterns in data.

There are three main types of layers:

1. Input Layer: This is the network's front door. It receives the initial data, like the pixels of an image or the words in a sentence. Each node in this layer typically represents a single feature of the input data.
2. Hidden Layers: These are the intermediate layers between the input and output. A network can have one or many hidden layers, and this is where most of the computation happens. Deeper networks (those with more hidden layers) can learn more complex patterns.
3. Output Layer: This is the final layer that produces the result. For a classification task, it might have one node for each possible category. For a regression task, it might have a single node that outputs a numerical value.

So how does a single neuron process information? It performs a simple two-step calculation. First, it takes all the outputs from the neurons in the previous layer and computes a weighted sum. Each connection has a weight, which signifies the importance of that connection. Think of it as turning a knob—a higher weight means that input has more influence.

After summing the weighted inputs, the neuron adds a bias. A bias is a constant value that allows the neuron to shift its output up or down. This helps the network fit the data better, much like the y-intercept in a linear equation.

This first step is a linear transformation. But if that's all a network did, it would be no more powerful than simple linear regression. The real magic comes from the second step.

The second step in a neuron's calculation is to pass the result of the weighted sum ($z$) through an activation function. This function introduces non-linearity, which is what allows neural networks to learn incredibly complex, real-world patterns that aren't just straight lines.

There are many types of activation functions, but two are extremely common:

ReLU (Rectified Linear Unit): This is one of the most popular activation functions used in hidden layers. It's very simple: if the input is positive, it passes the value along. If it's negative, it outputs zero. This simple rule helps networks learn quickly and efficiently.

Softmax: This function is typically used in the output layer of a classification network. It takes a vector of arbitrary real-valued scores and squashes them into a vector of values between 0 and 1 that sum to 1. In other words, it converts the network's raw output scores into a probability distribution over the possible classes.

The entire process of passing data from the input layer through the hidden layers to the output layer is called forward propagation. It's a one-way street where information flows forward, and at each layer, the data is transformed by weights, biases, and activation functions.

Let's walk through it:

1. The input layer receives the raw data.
2. This data is passed to the first hidden layer. Each neuron in this layer calculates its weighted sum and bias, then applies its activation function (e.g., ReLU).
3. The outputs of the first hidden layer become the inputs for the second hidden layer, and the process repeats.
4. This continues until the data reaches the output layer.
5. The output layer's neurons perform their final calculations. For a classifier, this often involves applying the Softmax function to produce probabilities.

This final output is the network's prediction. The ability to stack layers allows the network to build a hierarchy of features. Early layers might learn simple features like edges or colors, while deeper layers combine these to recognize more complex concepts like faces or objects.

This forward pass is just half the story. It explains how a network makes a prediction, but not how it learns. The learning happens by comparing the network's prediction to the actual correct answer and then adjusting the weights and biases to reduce the error, a process you'll explore next.