A Markov chain is a model for a system that changes over time. It describes a sequence of events where the probability of each event depends only on the state of the system at the previous event. Think of it as a series of steps or jumps between different situations, which we call "states."

The entire collection of possible states a system can be in is called its state space. This can be anything from the weather (sunny, cloudy, rainy) to a player's position on a board game.

The core idea behind a Markov chain is the Markov property. It's a rule of memorylessness. This property states that the future state depends only on the current state, not on the sequence of states that came before it. In other words, the past is irrelevant to predicting the future, given the present.

Imagine a very simple weather model. Let's say the state space is just {Sunny, Rainy}. If today is Sunny, there's a certain chance it will be Sunny again tomorrow, and a certain chance it will be Rainy. According to the Markov property, these chances don't depend on whether it was rainy for the past week or sunny all month. All that matters is that today is Sunny.

The movement from one state to another is called a transition. Each transition has a probability associated with it, called a transition probability. These probabilities tell us how likely the system is to move from its current state to any other state in the next step.

For our weather example, we might define the following transition probabilities:

• If it's Sunny today, the probability of it being Sunny tomorrow is 0.9.
• If it's Sunny today, the probability of it being Rainy tomorrow is 0.1.
• If it's Rainy today, the probability of it being Sunny tomorrow is 0.5.
• If it's Rainy today, the probability of it being Rainy tomorrow is 0.5.

Notice that for any given starting state, the probabilities of all possible next states must add up to 1. If it's sunny today, it must be either sunny (0.9) or rainy (0.1) tomorrow. There are no other options, and $0.9 + 0.1 = 1$.

This kind of diagram helps visualize the entire system. We can see all the states and how they connect to one another through probabilities. This is the essence of a Markov chain: a set of states, and a set of rules for moving between them.

Markov chains are surprisingly common. Consider a simple board game like Snakes and Ladders. Your position on the board is a state. The state space is all the squares on the board, from 1 to 100.

When it's your turn, you roll a die and move. Your new position is a new state. The probability of landing on a specific square depends only on where you are now (your current state) and the roll of the die. It doesn't matter how you got to your current square; the path you took through the board is irrelevant for your next move.

If you land on a ladder, you immediately transition to a different state (a higher square). If you land on a snake, you slide down to a lower state. These are deterministic transitions within the model, but the overall process follows the Markov property. Your next move is always determined from your current square, not your previous ones.

This simplification, ignoring the past, is what makes Markov chains such a powerful and widely used tool for modeling random processes.