Stochastic Dynamic Systems Explained
Introduction to Stochastic Processes
Systems That Change Randomly
Many things in the world change over time, but not in a perfectly predictable way. Think about the number of customers in a coffee shop, the price of a stock, or the weather tomorrow. These are all systems that evolve with some element of chance. A stochastic process is simply a way to mathematically describe such a system.
stochastic process
noun
A collection of random variables, representing the evolution of some system of random values over time.
Imagine you're tracking the number of birds at a feeder in your backyard. Each time you look, the number might be different. The number of birds at any given moment is a random variable. The collection of all these observations over a week forms a stochastic process. It's a sequence of random events.
A Matter of Time
The way we measure time in these processes is important. It splits them into two main categories: discrete-time and continuous-time.
A discrete-time process is measured at separate, distinct points in time. Think of it as a series of snapshots. For example, counting the number of cars that pass an intersection every minute. You get a value at 1:00 PM, 1:01 PM, 1:02 PM, and so on, but not in between.
A continuous-time process, on the other hand, can change at any instant. Imagine tracking the temperature in a room. It doesn't jump from one value to the next; it changes smoothly and can be measured at any moment. The system is always evolving.
| Type | Time Measurement | Example |
|---|---|---|
| Discrete-Time | Specific intervals (e.g., every hour, day) | The outcome of a daily coin flip |
| Continuous-Time | At any moment | The speed of a car on a highway |
Memoryless Systems
Now let's look at a specific type of stochastic process called a Markov chain. Its defining feature is something called the Markov property, or "memorylessness."
This means that to predict the future of the system, all you need to know is its current state. You don't need any information about how it got there. The past is irrelevant.
Consider a simple weather model. The weather today can be Sunny, Cloudy, or Rainy. If it's sunny today, the probability of it being sunny tomorrow might be 80%. This probability doesn't change whether the past week was a monsoon or a heatwave. All that matters is that today is sunny. This is a discrete-time Markov chain because the state (the weather) changes from day to day.
Counting Random Events
Another important type of stochastic process is the Poisson process. This is a model for counting the number of times an event happens over a period of time. It's useful when events occur randomly and independently of each other.
For something to be a Poisson process, two main conditions should apply:
- The number of events in any two non-overlapping time intervals are independent.
- The probability of an event occurring in a very short time interval is proportional to the length of the interval.
Think about receiving emails. The number of emails you get between 9 AM and 10 AM doesn't affect the number you get between 2 PM and 3 PM. Also, you're more likely to receive an email in a 10-minute window than in a 1-minute window. The Poisson process is a continuous-time process because an email can arrive at any instant.
Stochastic processes give us the tools to model and understand the randomness that is all around us.