No history yet

Spring AI Model Abstractions

The Core Abstraction

When you build an application that interacts with a large language model, you want to focus on your application's logic, not the specific details of a vendor's API. Spring AI provides a powerful abstraction layer that decouples your code from any single AI provider. This means you can write your code once and switch between models from OpenAI, Anthropic, Ollama, or others just by changing your configuration.

At the heart of this design are two key interfaces: ChatModel and StreamingChatModel. These interfaces define a common way to interact with any compatible AI model.

Spring AI is an application framework or starter library for AI engineering.

The ChatModel interface is for simple request-response interactions. You send a prompt and get a complete response back. The StreamingChatModel is for when you want to receive the response as a stream of tokens, which is great for creating a more responsive, real-time user experience in a chatbot.

// A simplified view of the core interfaces
public interface ChatModel {
    ChatResponse call(Prompt prompt);
}

public interface StreamingChatModel {
    Flux<ChatResponse> stream(Prompt prompt);
}

Crafting the Prompt

You don't send a raw string to the model. Instead, you construct a Prompt object. A Prompt is a wrapper around a list of Message objects. This structure allows you to build complex, multi-turn conversations and provide context to the model. There are three main types of messages.

Message TypePurpose
SystemMessageSets the context or persona for the AI. It's the initial instruction.
UserMessageRepresents a message from the user to the AI.
AssistantMessageRepresents a previous response from the AI. Used to maintain conversation history.

For example, a SystemMessage might tell the AI, "You are a helpful assistant that translates English to French." Then, each UserMessage would be a phrase to translate. By including previous UserMessage and AssistantMessage pairs in subsequent prompts, you provide the model with the conversation's history, enabling it to give context-aware responses.

import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.prompt.Prompt;

// Create a system message to define the AI's role
var systemMessage = new SystemMessage("You are a helpful coding assistant.");

// Create a user message with their request
var userMessage = new UserMessage("Write a simple Java function to reverse a string.");

// Construct the final prompt
Prompt prompt = new Prompt(List.of(systemMessage, userMessage));

Configuring Model Behavior

Different AI models offer various parameters to control how they generate text. You might want a more creative, unpredictable response, or you might need a very deterministic, factual one. Spring AI handles this through ChatOptions.

The Prompt object can include an instance of ChatOptions, where you can set common parameters like temperature or top-k. While the options are standardized, Spring AI also allows for provider-specific options, giving you fine-grained control without locking you into a specific SDK.

Temperature

noun

A parameter that controls the randomness of the model's output. A higher temperature (e.g., 0.8) results in more creative and varied responses, while a lower temperature (e.g., 0.2) makes the output more focused and deterministic.

Here’s how you would configure options for an OpenAI model, setting the model name and temperature. You pass these options when creating the Prompt.

import org.springframework.ai.openai.OpenAiChatOptions;
import org.springframework.ai.chat.prompt.Prompt;

// Define model-specific options
OpenAiChatOptions options = OpenAiChatOptions.builder()
    .withModel("gpt-4-turbo")
    .withTemperature(0.7f)
    .build();

// Get the user message
var userMessage = new UserMessage("Tell me a fun fact about the Roman Empire.");

// Create a prompt that includes the options
Prompt prompt = new Prompt(userMessage, options);

Because Spring AI abstracts these details, switching to a different provider like Ollama is seamless. You would use OllamaOptions instead of OpenAiChatOptions, but the Prompt structure and the chatModel.call() method remain exactly the same. Your application code doesn't need to change.

Ready to check your understanding?

Quiz Questions 1/4

What is the primary benefit of using Spring AI's abstraction layer for interacting with large language models?

Quiz Questions 2/4

You are building a feature where the AI's response appears to the user token-by-token, creating a real-time typing effect. Which Spring AI interface should you use?

This powerful abstraction layer is what makes Spring AI so flexible for building robust, provider-agnostic AI applications.