Modeling environmental dynamics directly from high-dimensional observations like images is computationally intractable and information-theoretically inefficient. The core challenge is to distill the rich, often redundant sensory stream into a compact representation that captures the underlying causal structure of the world. This process, known as semantic compression, is the foundation of modern world models.

Instead of operating in pixel space, these models map observations into a lower-dimensional latent manifold. The goal is for this manifold's topology to mirror the environment's true state dynamics. Every point in this latent space should encode a belief about the state of the world. A trajectory through this space represents the evolution of that state over time, driven by actions and intrinsic dynamics.

The classical Markov assumption, where the current state $s_t$ contains all necessary information for the future, often fails in partially observable environments. An agent might not see the cause of an event, only its effect, breaking the Markov property. To address this, we move from simple state representations to belief states, which are probability distributions over possible world states.

Predictive State Representations (PSRs) offer a powerful alternative. Instead of modeling a hidden state, a PSR defines the state as a vector of predictions about future observable events, conditioned on a history of actions and observations. The state is what you can predict. This approach sidesteps the need to explicitly model unobservable variables, grounding the state directly in future empirical outcomes.

However, learning PSRs can be challenging. A more common approach in deep learning is the Recurrent State-Space Model (RSSM). The RSSM maintains a belief state by integrating new observations with a prior belief derived from the previous state and action. It combines a deterministic transition model with a stochastic component, allowing it to capture both predictable and surprising elements of the environment.