Most large language models you've used operate like a quick-witted expert. You ask a question, and they generate an answer, word by word, based on patterns learned from vast amounts of text. This is a form of rapid, intuitive cognition, often called System 1 thinking. It's fast and remarkably effective for tasks like summarisation or drafting an email.

But for complex problems in science, mathematics, or programming, a simple, instant response often isn't enough. These challenges require deliberation, planning, and self-correction. This is where a new class of AI, sometimes called reasoning models, comes in. Models like OpenAI's o1 series, Anthropic's Claude 3.5 with its 'extended thinking' mode, and Google's Gemini 2.5 Pro are designed to engage in a slower, more methodical process known as System 2 thinking before providing an answer.

The key difference isn't just the model's architecture, but how it's used at the moment of a query, a concept called inference-time scaling. Instead of generating an immediate response, the model is given a larger 'compute budget' to think. Think of it like an exam. A standard model has to answer each question the second it reads it. A reasoning model gets extra time to work through the problem on scrap paper, check its work, and even start over if it finds a mistake.

This 'thinking' process isn't random. It often follows a structured technique like Chain-of-Thought (CoT) prompting. The model is encouraged to break down a problem into smaller, sequential steps, articulating its reasoning along the way. It then evaluates its own chain of thought, creating self-correction loops that refine its path towards the solution. It's essentially talking itself through the problem to avoid jumping to a flawed conclusion.

How do we know this extra thinking time pays off? New, more challenging benchmarks have been developed to test these advanced reasoning abilities. Standard benchmarks were often 'contaminated,' meaning their questions were present in the model's training data, making it hard to assess true problem-solving skill.

Benchmarks like GPQA Diamond and SWE-bench are different. GPQA contains graduate-level questions in physics, biology, and chemistry written by experts, which are extremely unlikely to have been in any training set. SWE-bench tests a model's ability to solve real-world software engineering problems from GitHub, requiring it to understand complex codebases and fix bugs. Reasoning models excel here, often outperforming standard models by a significant margin.

This enhanced capability comes with trade-offs. Granting a model more inference time directly increases two things: latency and cost. A quick question to a standard model might take a second and cost a fraction of a cent. Posing a complex coding challenge to a reasoning model might take over a minute and cost several pence or more.

The choice depends entirely on the task. For a live conversation or quick data extraction, a fast model is ideal. But for tasks where accuracy is paramount and the cost of an error is high, like identifying security vulnerabilities in code or conducting scientific research, the higher latency and cost of a reasoning model are a worthwhile investment.