How AI Assistants Decide Which Sources to Cite: A Technical Primer on RAG Retrieval

When a user asks ChatGPT, Claude, or Perplexity about your product category, the assistant does not simply "know" the answer. It retrieves external sources, processes them, and synthesizes a response. This is retrieval-augmented generation—RAG.

For GEO practitioners, understanding this pipeline is not optional. It determines whether your content appears, gets cited, or gets ignored entirely.

The RAG Pipeline in Three Stages

Most production RAG systems follow a consistent architecture. Knowing each stage clarifies where your content can succeed or fail.

Stage 1: Embedding Similarity

The first filter is purely mathematical. Your content gets converted into a high-dimensional vector. The user's query becomes another vector. Cosine similarity determines how "close" they are.

This stage is unforgiving. If your content does not embed near the query, it never enters the candidate pool.

What affects embedding similarity:

Research on dense retrieval indicates that semantic similarity models like those based on BERT architectures outperform keyword matching for complex queries (Karpukhin et al., 2020, Facebook AI Research).

Stage 2: Reranking

Initial retrieval returns candidates. Reranking decides the final order.

Cross-encoder rerankers like those in the Cohere Rerank API or open-source models (e.g., BGE-Reranker) score each document against the query using a transformer that processes both together. This is computationally expensive, which is why it happens after retrieval narrows the pool.

Reranking introduces signals beyond pure semantic similarity:

• Relevance depth. Does the document answer the query directly, or merely mention related terms?
• Recency. For time-sensitive queries, fresher content often ranks higher. Perplexity explicitly surfaces publication dates.
• Source authority. Some systems weight domains by historical link signals or editorial trust scores.

The exact weighting is proprietary. But the principle holds: making the initial candidate pool is necessary. Surviving reranking is what gets you cited.

Stage 3: Context Window Limits

Here is the hard constraint most GEO practitioners underestimate.

Large language models have finite context windows. GPT-4 Turbo supports 128,000 tokens (OpenAI, 2023). Claude 3 supports up to 200,000 tokens (Anthropic, 2024). But in practice, RAG systems inject far less—often 4,000 to 16,000 tokens of retrieved context.

Why? Cost, latency, and diminishing returns. Research from Stanford's HELM benchmark suggests that model performance degrades when context is cluttered with marginally relevant documents (Liang et al., 2022).

This means even if your content survives retrieval and reranking, it competes for limited slots. A system retrieving 50 candidates may only inject 5-8 into the final prompt.

What GEO Practitioners Can Influence

Not everything is within your control. Model architectures, reranking weights, and context allocation are decided by AI providers.

But the inputs are yours:

1. Semantic clarity. Write content that embeds close to how users phrase queries.
2. Chunking-friendly structure. Use clear headings, concise paragraphs. Make each section self-contained.
3. Direct answers. Content that states claims plainly reranks better than content that buries them in caveats.
4. Freshness. Update pages. Publication dates matter for recency-weighted queries.
5. Crawlability. If AI systems cannot access your content, none of this applies.

Measuring What Matters

Understanding the pipeline is one step. Knowing whether your content actually appears in AI answers is another.

Mentio tracks whether your brand is mentioned and cited across ChatGPT, Claude, and Perplexity. No speculation about embeddings—just direct visibility data. You see which queries surface your content, which competitors appear alongside you, and where you are absent.

The RAG pipeline is a black box. But the outputs are observable.