Local Versus Global Questions: When Vector Search Fails and GraphRAG Takes Over
Why vector embeddings struggle with holistic corpus summarization, and how entity knowledge graphs and community reports solve it.
On this page
Retrieval-Augmented Generation (RAG) is the default architecture for connecting large language models to proprietary enterprise documents.
In standard vector RAG, the formula is familiar:
- Chop documents into 500-token text chunks.
- Embed chunks into high-dimensional vectors.
- Index them in a vector database (Pinecone, Qdrant, Milvus).
- At query time, embed the user query, perform Approximate Nearest Neighbor (ANN) search, retrieve the top 5 chunks, and feed them into the prompt.
For Local Queries (“What was our Q3 European marketing expenditure?” or “What is the warranty policy for Model X?”), standard vector RAG performs exceptionally well.
However, if an executive asks a Global Question (“What are the top three systemic organizational risks facing our supply chain across all international divisions?”), standard vector RAG fails completely.
The failure is mathematical: a global question does not have a single matching 500-token chunk. The answer is diffused across thousands of unrelated sentences spanning hundreds of documents. Vector similarity lookup pulls five random paragraphs that happen to share semantic keywords, hallucinating an incomplete and misleading summary.
Solving this required Microsoft Research to introduce GraphRAG: combining knowledge graphs, hierarchical community detection, and pre-computed community summaries.
+-------------------------------------------------------------------------+
| LOCAL VS. GLOBAL RETRIEVAL PARADIGM |
| |
| Local Question: "What is John's salary?" |
| -> Vector Search excels (Pinpoints specific chunk in <10ms). |
| |
| Global Question: "What are the core themes of corruption in this leak?"|
| -> Vector Search fails (No single chunk holds the answer). |
| -> GraphRAG succeeds (Traverses entity graph & community summaries). |
+-------------------------------------------------------------------------+
How GraphRAG Builds Knowledge
GraphRAG replaces naive chunk-and-embed pipelines with an automated graph extraction and hierarchical clustering pipeline:
[Raw Document Corpus (Thousands of PDFs)]
|
v
+---------------------------+
| LLM Entity & Relation Ext.| ---> Extracts: (Entity A) -[Relationship]-> (Entity B)
+---------------------------+
|
v
+---------------------------+
| Leiden Community Detection| ---> Clusters graph into hierarchical modules
+---------------------------+
|
v
+---------------------------+
| Community Summary Reports | ---> LLM pre-summarizes every entity cluster
+---------------------------+
1. Entity and Relationship Extraction
The pipeline passes chunks through an LLM instructed to extract domain entities (people, organizations, locations, system concepts) and their directed relationships, producing structured triples:
("API Gateway", "enforces_rate_limits_on", "Downstream Microservices")
2. Hierarchical Community Detection (Leiden Algorithm)
Once the knowledge graph is populated, GraphRAG applies the Leiden Community Detection Algorithm. This algorithm partitions the graph into dense, interconnected clusters at multiple hierarchical levels:
- Level 0 (Global Themes): Broad corporate concepts (Logistics, Human Resources, Regulatory Compliance).
- Level 1 (Sub-Communities): Specific operations within Logistics (Cold Storage, Trucking Fleets, Customs Clearance).
- Level 2 (Leaf Micro-Entities): Individual warehouses, vehicle part numbers, and contract names.
3. Pre-Generated Community Summaries
For every detected community cluster at every hierarchical level, an LLM generates a comprehensive Community Report:
- A structured executive summary describing the cluster’s key entities, historical claims, and vulnerabilities.
- These reports are stored as permanent artifacts in the retrieval index.
Query Execution: Local Search vs. Global Search
When a query arrives, GraphRAG supports two completely distinct retrieval modes:
[User Natural Language Query]
|
+--------------+--------------+
| |
(Local Specific Query) (Global Abstract Query)
v v
[Graph Local Search] [Graph Global Search]
| |
- Embed query vector - Map-Reduce over Community Reports
- Fetch nearest entities - Shuffled batches scored by relevance
- Pull 1-hop relationships - Intermediate points synthesized
- Pack focused context - Comprehensive macro answer emitted
Local Search (Detailed Grounding)
For queries targeting specific facts:
- Vector search identifies the top matching entities in the graph.
- The engine traverses the graph along 1-hop and 2-hop edges to retrieve related entities, claims, and original source text chunks.
- This creates a context window containing both textual evidence and structural relationships, avoiding the hallucination traps of raw text chunking.
Global Search (Macro Synthesis)
For queries asking for themes, aggregations, or corpus-wide trends:
- The engine bypasses raw document chunks entirely.
- It fetches the pre-computed Community Reports at Level 0 and Level 1.
- It executes a Map-Reduce evaluation:
- Map Phase: Shuffled batches of community reports are evaluated by an LLM, generating candidate points with importance ratings (0 to 100).
- Reduce Phase: High-importance points are consolidated, deduplicated, and synthesized into a final, exhaustive macro-report.
The Cost Equation: Full GraphRAG vs. LazyGraphRAG
The primary obstacle to adopting GraphRAG is indexing cost:
- Full GraphRAG requires multiple LLM extraction passes across every document, followed by dozens of LLM calls to generate community reports.
- Indexing a 100-megabyte document corpus can easily cost $2,000 in API tokens and take hours to compute.
- If documents update frequently, the knowledge graph becomes stale.
To solve this, researchers developed LazyGraphRAG:
- Replaces heavy LLM entity extraction at index time with fast, classical Natural Language Processing (NLP) noun-phrase extractors.
- Skips upfront community summarization entirely.
- Defers LLM reasoning to query time, evaluating only the graph paths touched by the active question.
- Result: Drops indexing cost by 99.9% (over 700x cheaper) while retaining 95% of the global reasoning accuracy of full GraphRAG.
The RAG Evolution Ladder
| Architecture | Best Used For | Indexing Cost | Query Latency | Primary Failure Mode |
|---|---|---|---|---|
| Naive Vector RAG | Specific facts, FAQ lookup | Near-Zero | Sub-20 ms | Fails completely on global / thematic queries |
| Advanced RAG | Hybrid search (BM25 + Dense) | Low | ~50 ms | Still chunk-bound; cannot synthesize across 50 files |
| Full GraphRAG | Thematic synthesis, investigations | Very High | ~2 to 5 seconds | Expensive indexing; complex graph maintenance |
| LazyGraphRAG | Dynamic enterprise knowledge | Low | ~1 to 2 seconds | Slightly lower reasoning depth than full reports |
Vector search is a microscope: it zooms in on a single slide. GraphRAG is a satellite: it surveys the entire continent. Robust enterprise systems deploy both: routing local queries to vector indexes and macro-questions to knowledge graphs.
/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.