Skip to content
Back to Generative AIAll lessons

Generative AI

Vector Search / RAG Retrieval

O(n · d) time for a plain scan over n stored vectors of dimension d. Real systems use special vector indexes to search millions of vectors faster than checking every single one.

The idea, in plain English

RAG stands for Retrieval-Augmented Generation. It is like giving an AI an open-book exam. Instead of answering purely from memory, the AI first looks up the most relevant notes, then writes its answer using them. The 'lookup' step is called vector search. Every stored chunk of text has an embedding, which is a location in meaning-space. The query gets an embedding too. You retrieve whichever stored chunk's embedding sits closest, or most similar, to the query's embedding.

How it works

  1. 1Ahead of time, turn every chunk of text in your knowledge base into an embedding vector, and store them.
  2. 2When a query comes in, turn it into an embedding vector using the same method.
  3. 3Compare the query vector to every stored vector using cosine similarity. Retrieve the chunk with the highest score. This is the 'retrieval' part of Retrieval-Augmented Generation (RAG). A real system then hands that chunk to a language model, which writes the final answer.

When you'd use it

Use this to let an AI answer questions about your own documents, a knowledge base, or anything outside what it memorized during training, without retraining the model itself.

Common beginner mistakes

  • Don't assume retrieval always finds a relevant chunk. If nothing in the knowledge base is actually related, it still returns whichever chunk scored highest, even if that is a poor match.
  • Don't skip the generation step. Retrieval only finds supporting text. A language model still needs to turn it into a real answer.

Try it — edit and run

Click the code to edit · press ⌘/Ctrl+↵ to run

Editable code. Tab and Shift+Tab indent. Press Escape, then Tab, to move focus out of the editor.

Expected output — hit Run to try it
Query: furry pets
Score 0.96: Cats and dogs are popular furry pets.
Score 0.00: Python is a popular programming language for data science.
Score 0.80: The sun is a star at the center of our solar system.
Retrieved chunk: Cats and dogs are popular furry pets.

Not sure this is the right topic? See the learning paths → or where this leads →