Building an artificial general intelligence (AGI) that can think and reason like a human is one of the most ambitious goals in science. It's not just a matter of creating bigger and faster versions of the AI we have today. It requires a different way of thinking about intelligence itself.

Two major theoretical questions guide this quest. First, how should an AGI learn? Should it be programmed with knowledge, or should it discover the world on its own? Second, what should it want? If an AGI becomes powerful, how do we ensure its goals align with our own best interests? Let's explore two influential frameworks that tackle these very questions.

Human infants aren't born knowing how to speak, read, or solve math problems. They learn gradually, by observing, experimenting, and interacting with their environment. Some researchers believe this developmental process is the key to creating true intelligence.

This idea is at the heart of a framework called the Ontogenetic Architecture of General Intelligence (OAGI). The term "ontogenetic" refers to the development of a single organism from its earliest stage to maturity.

The OAGI model proposes that an AGI shouldn't be pre-programmed with a vast library of facts. Instead, it should start with a basic, flexible structure—much like an infant's brain—and build its cognitive abilities from the ground up. It would learn abstract concepts by grounding them in real-world sensory experiences, just as a child learns the concept of "heavy" by trying to lift different objects.

This approach emphasizes continuous learning and adaptation. The AGI would constantly refine its understanding of the world, developing more complex skills and knowledge over time, not because it was explicitly told to, but because its architecture is designed to grow through experience.

Once an AGI can learn, what should its ultimate goals be? Simply telling it to "make humans happy" is vague and could lead to disastrous misinterpretations. This is where another theoretical framework, Coherent Extrapolated Volition (CEV), comes in.

CEV is an attempt to define a safe and beneficial goal for a superintelligent AI. The core idea is to have the AGI act on what we would want if we were better versions of ourselves—if we knew more, thought faster, and were more coherent in our values.

Imagine humanity is asked to decide on its future. Individually, we have conflicting desires, biases, and a limited understanding of the world. The CEV framework suggests the AGI's task would be to figure out our collective will by extrapolating from our current state. It would first learn about our values, then determine where those values would converge if we had more time to reflect and grow. It's not about what we want right now, in this moment of confusion, but about our idealized, collective desire for the future.

This isn't a simple task. It requires the AGI to understand the nuances of human morality, culture, and aspirations. The goal of CEV is to create a guiding principle that ensures an AGI acts in the best interest of humanity as a whole, as we would define it from a place of greater wisdom.

These frameworks represent two different but complementary pieces of the AGI puzzle. OAGI focuses on the "how" of learning, while CEV addresses the "why" of its actions. Together, they provide a glimpse into the profound theoretical challenges researchers face in the journey toward creating a true artificial general intelligence.