Beau

Okay, so last time we really dug into sequential models, right? Treating user history like a sentence with BERT4Rec. My brain is still kind of stuck on that idea… that an item click is basically a 'word' in the 'sentence' of a user's journey.

Jo

It’s a powerful metaphor. And it's the perfect bridge, actually. Because for a long time, that 'word'—that item—was just an ID. Item 734. We learned its embedding, its latent factors, next to all the other item IDs. But the model never actually knew *what* item 734 was.

Beau

Right, it just knew how it related to other IDs based on user interactions. That's the whole foundation of collaborative filtering we've been building on.

Jo

Exactly. But what if we could give the model the item's title, its description, its reviews… and have it understand the item on a deep, semantic level? What if instead of learning an embedding for an arbitrary ID, we could use a massive, pre-trained model to just... create one for us based on the content?

Beau

And there it is. We're talking about Large Language Models.

Jo

We are. It’s a complete shift. We're moving from learning relationships in a closed world of user-item interactions to leveraging a model that has a general understanding of... well, everything. That's the first major use case: LLMs as incredible feature extractors.

Beau

So, help me picture this. I have a new movie I want to add to my streaming service. It's called 'Cosmic Echoes'. In the old world, it's a new item ID, and it has the ultimate cold-start problem. No one has watched it, so it doesn't exist in our interaction matrix.

Jo

Perfect example. How do you recommend it? You can't. But with an LLM, you feed it the movie's description: 'A sci-fi thriller about an astronaut who discovers a signal that appears to be an echo of her own future distress call.' The LLM, say, Mistral, reads that and generates a rich vector embedding.

Beau

And that embedding… it's not random. It's located in vector space near other movies with similar themes.

Jo

Exactly. It will naturally be close to 'Interstellar' or 'Arrival' or maybe even 'Primer' because the LLM understands concepts like 'sci-fi', 'time paradoxes', 'space'. So you can immediately find users who liked similar items and recommend it. The cold-start problem... it basically evaporates for content-based filtering. This is a zero-shot recommender.

Beau

Zero-shot because it required zero interaction data to make a meaningful recommendation. Okay, I'm with you. And at scale, you're not doing a brute-force search through millions of these vectors. This is where vector databases come in, right? The Pinecones and Milvuses of the world.

Jo

Yep. They're built for exactly this: incredibly fast approximate nearest neighbor search. You take a user's recent watch history, generate embeddings for those items, average them to get a 'taste vector' for the user's current session, and then query the vector DB to find the 'K' most similar item vectors in your entire catalog in milliseconds.

Beau

That's semantic search, but applied to recommendations. Instead of a user typing a query, their behavior *is* the query. That's a huge leap from just using the embeddings from a two-tower model.

Jo

It is, but you can also combine them. You can use these rich LLM-generated embeddings as features that you feed *into* a two-tower model. So you get the best of both worlds: semantic understanding from the LLM and interaction-based learning from the collaborative filtering architecture.

Beau

Okay, that makes sense. But what if the general-purpose LLM doesn't quite get the nuance of my domain? Like, for a fashion retailer, the difference between 'bohemian' and 'cottagecore' is huge, but a general LLM might see them as pretty similar.

Jo

Fantastic question. That leads to the next step: fine-tuning. A pre-trained Llama model is a great starting point, but you can make it an expert in your specific domain. You take your user interaction logs—user A clicked on item X, user B bought item Y—and you continue training the LLM on that data.

Beau

But you're not retraining the whole thing from scratch, that would be... insane.

Jo

God, no. You'd need a server farm the size of a small country. You use parameter-efficient fine-tuning methods. Think LoRA or P-Tuning. You're freezing the vast majority of the model's weights and only training a tiny fraction of new parameters—like little adapter modules. It teaches the model your specific data patterns without it forgetting all its general world knowledge. It's how you teach it that for your users, 'cottagecore' and 'bohemian' are, in fact, very different concepts.

Beau

So we've gone from using LLMs as off-the-shelf feature creators to tuning them to become domain-specific experts. Okay... that's a powerful progression. But so far, we're still kind of using them as a component *in* a traditional recommender system. It feels like we're not using the 'L' for 'Language' part enough.

Jo

And that is the final, most exciting frontier right now: Generative Recommendation. What if the LLM isn't just a part of the system? What if the LLM *is* the system?

Beau

How... how would that even work?

Jo

Through prompt engineering. Instead of just feeding the model item data, you feed it a natural language prompt that describes the user and the task. Imagine a prompt that says: 'This user has a history of buying these items: [Item A Title, Item B Title, Item C Title]. They recently searched for 'lightweight running shoes'. Based on this, predict the title of the next item they are most likely to click on from this list: [Candidate 1, Candidate 2, Candidate 3].'

Beau

Whoa. So you're turning recommendation into a text-completion task. The LLM's answer *is* the recommendation.

Jo

Precisely. The model leverages all its latent knowledge to reason about the user's intent and the item's properties and just... generates the answer. It completely blurs the line between search and recommendation. A user could literally type, 'Find me a murder mystery novel that's not too gory but has a really clever twist, like something Agatha Christie would write today.'

Beau

And the LLM can parse that entire nuanced request, translate it into conceptual space, and generate a list of books. That's... that's wild. But it also sounds computationally expensive for ranking millions of items.

Jo

It is. Which is why you often see a hybrid approach. You use a faster, traditional model—maybe even one using LLM embeddings—for the retrieval step to narrow down millions of items to a few hundred candidates. Then, you use the big generative LLM as a powerful, but expensive, re-ranker on just that small set.

Beau

The classic multi-stage ranking funnel. That makes sense. But there's one more thing... what about user profiles? We've talked about items and interactions. Can LLMs help there?

Jo

Absolutely. Think about summarizing a user's entire history. You can feed an LLM a long list of a user's clicked articles, watched movies, or purchased products and ask it to synthesize a profile. The prompt could be, 'Summarize this user's interests in a few sentences based on their activity.'

Beau

And it might output something like 'This user is a tech enthusiast interested in vintage synthesizers and Japanese sci-fi from the 1980s.'

Jo

Exactly. That generated text profile can then become a powerful feature in its own right, fed into other models. You can even use generative agents to simulate how different user personas might react to a new item before you even launch it. It's a whole new world for offline evaluation.

Beau

So we're not just predicting the next click anymore. We're building a system that can actually... understand the user and the catalog on a human level. It's moving from statistical pattern matching to something that feels a lot more like reasoning.

Jo

That’s the goal. And the amazing part is, all those foundational concepts we've covered—latent factors, two-tower models, sequential awareness—they don't go away. They just become part of a much larger, more intelligent, language-aware architecture.