AI Agent Guide Interactive
Module 2 · RAG Retrieval & Grounding Qdrant + TypeScript 3 Interactive Labs

Embeddings & Vector Search Fundamentals

Your documents are now split into clean DocumentChunk objects, but the machine still has no idea that "cats and dogs" are close while "cats and quantum mechanics" are worlds apart. This step turns text into coordinates, stores them in a vector database, and retrieves nearest neighbors by distance.

30-Second Version · Remember Just These 3 Rules All lab numbers are either cited or marked illustrative
  1. 01 Cosine similarity measures direction only, never vector length.
  2. 02 Skip MTEB overall averages—check the Retrieval split first, then benchmark on your own queries.
  3. 03 Low scores? Inspect your chunks first. Setting limit without a threshold feeds noise straight to the LLM.

What Embeddings Actually Do: Encoding Meaning as Coordinates

An embedding model does not generate text; it outputs a fixed-length numeric array—a point in semantic space. Retrieval is simply finding the points closest to the query.

The vector dimensionality is fixed by the model: OpenAI text-embedding-3-small outputs 1,536 dimensions, text-embedding-3-large outputs 3,072, while Alibaba Cloud Model Studio's text-embedding-v3 and open-source BGE-M3 default to 1,024 dimensions.

For text retrieval, "closest" usually means highest cosine similarity, ranging theoretically from -1 to 1. Drag the orange arrow in the 2D explorer below—or —and watch whether the ranking changes.

Lab 01 · Vector Space Intuition
2D Semantic Space & Cosine Similarity Probe
Drag the orange query arrow to see how angle θ drives ranking and which documents get dropped by score_threshold = 0.65.Illustrative: real vectors have 1,024+ dims, projected here to 2D
Semantic Dim X Semantic Dim Y
Threshold: 0.65
Top-K Similarity Ranking (cos θ = (A · B) / (‖A‖ ‖B‖))

Choosing an Embedding Model: MTEB Is a Starting Point, Not the Finish Line

MTEB overall averages blend classification, clustering, and other tasks irrelevant to RAG. Start with the Retrieval sub-score, then decide based on Recall@5 on your own query set.

MTEB (GitHub repo) spans over a hundred datasets, and the Hugging Face live leaderboard is usually the first stop. The catch: sorting by the headline overall score often picks the wrong model.

Predict First On the March 2026 English MTEB leaderboard, NV-Embed-v2 scores 72.31 overall while Gemini Embedding 001 scores 68.32. For an enterprise knowledge base, which should you test first?
Under the hood: NV-Embed-v2 scores only 62.65 on Retrieval, whereas Gemini Embedding 001 hits 67.71. The overall leader is actually weaker at retrieval.
Lab 02 · Selection Trade-Off Simulator
Three Selection Strategies, Three Outcomes
Pick the strategy you'd reach for first and inspect the engineering consequences.
Task: Build RAG over internal engineering docs (~20,000 chunks, bilingual Chinese/English).
Current setup: Qdrant collection already provisioned for text-embedding-v3 with size: 1024.
Public English MTEB Scores Source · 2026-03 Board
Only the two models with cited leaderboard numbers are shown. Dimensions from official model cards.
Select a strategy above
This panel shows which model gets picked, how it performs on retrieval, whether you must rebuild the Qdrant collection, and what to do next.

Why do leaderboard winners underperform on private corpora? Three main reasons (see Pinecone's embedding rundown): MTEB tests general text like news and Wikipedia whose vocabulary diverges from internal docs; some models have training overlap with public benchmarks; and aggregate averages hide per-task variance.

Smaller models routinely beat larger ones in embedding retrieval. BAAI's BGE-M3 (arXiv:2402.03216, model card) has only 568M parameters yet supports 100+ languages and an 8,192-token context window. Alignment between training objectives and retrieval matters far more than parameter count. For multilingual and Chinese workloads, Alibaba's open-source Qwen3-Embedding family (0.6B / 4B / 8B, official blog) is also worth benchmarking.

End-to-End Ingestion and Search with Qdrant

Batch your embedding requests and store the original text in the payload during ingestion; at query time, combine limit with score_threshold and refuse cleanly when the result array is empty.

Qdrant Ingestion & Retrieval Pipelines Architecture overview
Offline Ingestion Pipeline DocumentChunk Batched (BATCH_SIZE=50) Batch Embedding Multi-input, lower RPM vector + payload Store text / source / section Qdrant Collection Cosine · 1024-dim locked Online Retrieval Pipeline User Query Same embedding model Filter + Cosine Pre-filter by metadata limit + threshold Drop below score_threshold Hits → Pass to LLM Empty → Safe Refusal

Among vector databases, Qdrant is written in Rust, launches with a single Docker command, ships with a built-in Web Dashboard (http://localhost:6333/dashboard), and provides a strongly typed TypeScript SDK (@qdrant/js-client-rest)—scaling cleanly from local prototype to production.

API Pricing & Dimensions Reference (Default Book Setup)
The examples in this chapter default to Alibaba Cloud Model Studio's text-embedding-v3 (¥0.5 / 1M tokens, 1,024 dims, 8,192 context; billing docs). Other common options: OpenAI text-embedding-3-small ($0.02 / 1M, 1,536 dims; docs), text-embedding-3-large ($0.13 / 1M, 3,072 dims), Voyage AI voyage-4-lite ($0.02 / 1M, 1,024 dims; pricing), and Google Gemini Embedding 001 ($0.15 / 1M, 3,072 dims; Vertex AI pricing).

Two engineering details trip up almost every first ingestion script: first, pack multiple texts into a single input array (batch upsert) instead of one HTTP call per chunk, or you will hit rate limits fast; second, store the raw text alongside source and section inside each point's payload, otherwise a vector hit gives you an ID with no text to hand back to the LLM.

Full TypeScript implementation: Qdrant batch upsert & filtered search (embed-and-store.ts)
examples/rag/embedding-vector-search/embed-and-store.ts
import { QdrantClient } from "@qdrant/js-client-rest";
import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.ANTHROPIC_API_KEY, // Reuses DashScope API key
  baseURL: process.env.EMBEDDING_BASE_URL,
});
const qdrant = new QdrantClient({ host: "localhost", port: 6333 });

const COLLECTION = "knowledge_base";
const EMBEDDING_MODEL = process.env.EMBEDDING_MODEL ?? "text-embedding-v3";
const VECTOR_SIZE = 1024; // Must match the embedding model output dimension
const BATCH_SIZE = 50;

async function ensureCollection() {
  const { collections } = await qdrant.getCollections();
  if (!collections.some((c) => c.name === COLLECTION)) {
    await qdrant.createCollection(COLLECTION, {
      vectors: { size: VECTOR_SIZE, distance: "Cosine" },
    });
  }
}

async function ingestChunks(chunks: Array<{ text: string; metadata: { source: string; section?: string; chunkIndex: number } }>) {
  for (let i = 0; i < chunks.length; i += BATCH_SIZE) {
    const batch = chunks.slice(i, i + BATCH_SIZE);
    const response = await openai.embeddings.create({
      model: EMBEDDING_MODEL,
      input: batch.map((c) => c.text), // Batch requests to reduce RPM pressure
    });

    const points = batch.map((chunk, j) => ({
      id: i + j,
      vector: response.data[j].embedding,
      payload: {
        text: chunk.text,
        source: chunk.metadata.source,
        section: chunk.metadata.section ?? "",
        chunkIndex: chunk.metadata.chunkIndex,
      },
    }));

    await qdrant.upsert(COLLECTION, { points });
  }
}

async function search(query: string, limit = 5, scoreThreshold = 0.65, sourceFilter?: string) {
  const res = await openai.embeddings.create({ model: EMBEDDING_MODEL, input: query });
  const queryVector = res.data[0].embedding;

  const results = await qdrant.search(COLLECTION, {
    vector: queryVector,
    limit,
    score_threshold: scoreThreshold, // Drop anything below the quality floor
    with_payload: true,
    ...(sourceFilter ? { filter: { must: [{ key: "source", match: { value: sourceFilter } }] } } : {}),
  });

  if (results.length === 0) {
    return "No relevant context found to answer this question."; // Refuse instead of feeding noise to the LLM
  }
  return results;
}

Top-K, Similarity Scores, and Chunk Quality

In production RAG, cosine scores of 0.70–0.80 are already strong. When scores stay stubbornly low, inspect whether your chunks mix too many topics before swapping embedding models.

Under Cosine distance, higher Qdrant score values mean higher similarity (Qdrant Search docs). Because conversational user queries rarely match formal doc phrasing word for word, scores above 0.85 are uncommon.

If precise queries fail to break 0.70, the bottleneck is usually bloated, mixed-topic chunks rather than a weak model. Cram a dozen concepts into one chunk and its embedding collapses into a diluted "average meaning," pulling down similarity for every single question.

Predict First In a controlled run on the same vector-DB notes: packing 4 topics into one large chunk yields a score of 0.53 for "What is the range of cosine similarity?" Split into single-concept chunks using the exact same model—what does the score jump to?
Actual experiment output (chunk-quality-experiment.ts): it jumps from 0.53 straight to 0.85. The focused chunk contains only the cosine similarity definition, so the vector represents that concept cleanly instead of being diluted by three unrelated topics.

In Lab 03 below, switch across all 3 real benchmark queries to compare "large & mixed" vs. "small & focused" chunks, then tune limit, score_threshold, and source filtering together. Try to see how the threshold gate triggers safe refusal.

Lab 03 · Chunk Dilution & Parameter Tuner
Chunk Semantic Dilution & Top-K / Threshold Gate
Top: Measured · chunk-quality-experiment.ts real scores. Bottom: Simulated · Qdrant filter logic synced live to the TypeScript snippet above.
Qdrant Retrieval Parameter Tuner (syncs live to the TS code above)
Result cap limit 5
Quality floor score_threshold 0.65

Selection & Engineering Decision Matrix

Every rule from this chapter distilled into a filterable cheat sheet. Click a scenario tag to highlight the relevant playbooks.

Default Start
Ship end-to-end with text-embedding-v3 first

Reuse your existing DashScope key with balanced 1,024-dim bilingual vectors. Getting ingestion, payloads, and search working beats agonizing over leaderboards upfront.

Multilingual & Local
BGE-M3 or Qwen3-Embedding

568M BGE-M3 also outputs 1,024 dims, so swapping from text-embedding-v3 requires no Qdrant collection rebuild. Benchmark Qwen3-Embedding-4B when you need higher accuracy.

Benchmark Trap
Retrieval split + private query eval

Ignore MTEB overall averages. Collect 100–200 real user questions and measure Recall@5 and MRR—your private corpus score is the only verdict that counts.

Low-Score Triage
Scores < 0.70? Audit chunks first

Oversized, mixed-topic chunks dilute vector semantics. Splitting a multi-topic block into single-concept chunks jumped the exact same model from 0.53 to 0.85.

Hallucination Guard
Always pair limit with score_threshold

Without a reranker, start at limit=5 and threshold=0.65. When Qdrant returns an empty array, refuse directly instead of feeding 0.5-score noise to the LLM.

Known Boundary
Dense vectors miss exact identifiers

Asking for "GDPR Article 17" when the doc says "right to erasure" (or querying error codes and SKUs) can fall short on pure dense search. That's the gap Hybrid Search & Reranking closes next.

With offline ingestion and online vector search in place, the retrieval backbone is standing. Pure dense retrieval still has one structural blind spot: exact terms, error codes, and clause numbers. The next chapter adds BM25 keyword search, Reciprocal Rank Fusion (RRF), and Cross-Encoder reranking to push both recall and precision another level higher.