Your brain is a massive network of cells called neurons. They connect to each other at junctions called synapses, passing electrical signals back and forth. This intricate web is how you think, learn, and remember. Artificial neural networks take inspiration from this biological design.

An artificial neuron, or node, is a mathematical function that mimics a biological neuron. It receives one or more inputs, processes them, and passes an output to other neurons. Just like in the brain, the connections between these artificial neurons are what give the network its power.

So how does a single neuron work? It starts by taking a weighted sum of all its inputs. Each input has a weight, which is just a number. A higher weight means the network considers that input more important. Think of it like a group project: the person with the most expertise on a topic has their opinion (input) weighed more heavily.

After summing the weighted inputs, the neuron adds a bias. A bias is another number that helps shift the output up or down. It gives the neuron more flexibility, allowing it to produce an output even when all inputs are zero. The formula for this step looks like this, where $x$ represents inputs and $w$ represents weights:

This result is then passed through an activation function. This function's job is to introduce non-linearity, which allows the network to learn complex patterns. Without it, a neural network would just be a simple linear regression model. The activation function determines the final output of the neuron, deciding whether it should be “activated” or “fire.”

A common choice is the sigmoid function, which squashes any number into a range between 0 and 1. This is useful for predicting probabilities. It looks like an S-shaped curve.

A neural network learns by adjusting its weights and biases. This process happens in a cycle of two main phases: feedforward and backpropagation.

First, an input (like the pixels of an image) is fed into the first layer of neurons. Each neuron does its calculation and passes its output to the next layer. This is the feedforward process. The data flows forward through the network until it reaches the final output layer, which produces a prediction.

Next, the network compares its prediction to the correct answer. The difference between the two is calculated using a loss function. This function gives us a single number that tells us how wrong the network was. The goal is to make this number as small as possible.

Once the loss is calculated, the backpropagation process begins. The network works backward from the loss, calculating how much each weight and bias contributed to the error. It uses calculus (specifically, the chain rule) to find the gradient, or the slope of the loss function with respect to each parameter.

This gradient tells the network which direction to adjust the weights and biases to decrease the loss. An optimization technique, like Gradient Descent, uses this information to take a small step in the right direction. This entire cycle repeats thousands or millions of times, with the network slowly getting better at its task with each iteration.

By repeatedly going through the feedforward and backpropagation cycle, the network fine-tunes its internal parameters, effectively learning to map inputs to correct outputs.

Let's check your understanding of these core concepts.