Where LLMs Sit in the Search Stack: DoorDash, Instacart, and Uber Eats Architectures Compared
Comparing offline enrichment, real-time query parsing, and two-tower dense retrieval across three food delivery giants.
On this page
Search in food delivery and grocery is deceptively difficult. Unlike text document retrieval or web search, food search breaks keyword-matching systems in nearly every dimension:
- Synonyms and Regional Slang: A user searching for “soda”, “pop”, “coke”, or “soft drink” expects identical inventory.
- Dietary Constraints as Hard Filters: When someone searches for “vegan gluten-free pizza”, returning a delicious pepperoni pizza with 98% lexical match is an absolute failure.
- Bilingual and Code-Switching Queries: In multilingual markets, users seamlessly blend languages (e.g., “pan con avocado” or “spicy ramen near me”).
- The Cold-Start Long Tail: More than 20% of daily queries represent long-tail, un-cached phrases (“easy dinner for four under twenty dollars”) that have never been typed before.
When DoorDash, Instacart, and Uber Eats integrated large language models into their search engines, they did not pick the same architecture. Each company placed the LLM at a radically different depth in its runtime stack.
The core architectural lesson is that your existing infrastructure dictates where the model belongs.
+-------------------------------------------------------------------------+
| THE SEARCH INTEGRATION SPECTRUM |
| |
| [DoorDash] [Instacart] [Uber Eats] |
| Offline Enrichment -> Query Understanding -> Embedding Backbone|
| (Catalog Graph) (LLM RAG & Rewrite) (Two-Tower Dense) |
| |
| Near-Zero Runtime Latency ~300ms Tail Latency Sub-50ms HNSW |
+-------------------------------------------------------------------------+
Architecture 1: DoorDash (Offline Graph Enrichment)
DoorDash possessed a mature, highly structured Product Knowledge Graph built over years of merchant ingestion. Their catalog had hundreds of millions of dishes mapped to ingredients, merchant IDs, and price points.
Rather than injecting an LLM into the real-time request-response loop, DoorDash used LLMs primarily offline to enrich the graph:
[Raw Merchant SKU / Menu PDF]
|
v
+------------------+
| Offline LLM Job | ---> Extracts: [Cuisine, Flavor, Texture, Dietary, Allergen]
+------------------+
|
v
[Knowledge Graph] <--- Populated with structured entity edges
^
|
[Runtime Query] -> [Fast Slot Parser] -> Direct Cypher / SQL Filter
Runtime Mechanics
- Offline Extraction: LLMs process raw restaurant menus, parsing unformatted dishes (“Grandma’s secret pasta”) into structured attributes:
{cuisine: italian, dietary: vegetarian, spicy_level: low}. - Runtime Query Segmentation: A lightweight, deterministic parser segments user queries into known taxonomy slots.
- Retrieval-as-Guardrail: When an LLM is asked to classify a borderline term, its output is strictly constrained to the top 100 nearest existing taxonomy concepts in the knowledge graph.
Because the LLM runs offline or as a lightweight taxonomy matcher, DoorDash achieved a 30% lift in carousel trigger rates with almost zero added latency to their primary search path.
Architecture 2: Instacart (Real-Time Query Understanding)
Instacart faced a different challenge: multi-intent conversational grocery search. A user searching for “protein” does not want just chicken breast; they want protein bars, Greek yogurt, whey powder, and plant-based shakes.
Previously, Instacart maintained separate machine learning microservices for spell checking, query rewriting, classification, and brand tagging. This fragmented pipeline suffered from cascading errors.
Instacart consolidated these fragmented stages into a tiered real-time Query Understanding layer:
[User Search: "protein snack no sugar"]
|
v
+-----------------------+
| Head Query Cache Check| ---> (Hit: Return precomputed rewrite in 2ms)
+-----------------------+
| (Miss: Long Tail Query)
v
+---------------------------+
| Fine-Tuned Llama-3-8B | <--- Conditioned with RAG context
| (H100 Serving Cluster) | Rewrites: "high protein low sugar snacks"
+---------------------------+
|
v
+---------------------------+
| Semantic Guardrail Filter | ---> Strips hallucinations not in store inventory
+---------------------------+
|
v
[Classical Elasticsearch / Solr Inverted Index]
Key Engineering Decisions
- Head vs Tail Partitioning: Top queries (the head) are precomputed and served from an in-memory cache in under 2 milliseconds.
- Dedicated Tail LLM: The bottom 20% of long-tail queries route to a fine-tuned 8-billion-parameter Llama model hosted on H100 GPU clusters, operating under a strict 300 ms budget.
- Impact: Query rewrite coverage jumped from ~50% to over 95%, and scroll depth on the bottom 2% of difficult queries dropped by 6%, proving users found items immediately.
Architecture 3: Uber Eats (The Two-Tower Dense Vector Backbone)
Uber Eats operates across hundreds of international cities with extreme linguistic diversity. Maintaining localized keyword taxonomies or language-specific rewrite models for every market was unsustainable.
Uber Eats made the boldest architectural bet: replacing the lexical core with an end-to-end Two-Tower Neural Retrieval Backbone, where fine-tuned LLMs serve as the embedding generators for both queries and documents.
QUERY TOWER (Online) DOCUMENT TOWER (Offline)
[User Query: "tasty vegan pho"] [Merchant Dishes / Menu Items]
| |
v v
+---------------------+ +---------------------+
| Fine-Tuned Qwen LLM | | Fine-Tuned Qwen LLM |
+---------------------+ +---------------------+
| (Matryoshka Truncation) | (Matryoshka Truncation)
v (256-dim Vector) v (256-dim Vector)
[Online Query Embedding] [Offline Precomputed HNSW Index]
\ /
\ /
\---> [Approximate Nearest Neighbor] <---/
(Cosine Similarity)
|
v
[Top-K Candidate Delivery Dishes]
Engineering Optimizations for Scale
Serving an LLM directly in the retrieval loop for millions of concurrent requests is computationally impossible without aggressive vector optimizations:
- Matryoshka Representation Learning (MRL): High-dimensional embeddings (1,536 dimensions) are trained so that the first 256 dimensions capture nearly all the variance. Uber truncates vectors from 1,536 down to 256 dimensions with less than 0.3% loss in retrieval recall, slashing memory consumption by 83%.
- int7 Scalar Quantization: Vector components are quantized to 7-bit integers, allowing vector similarity calculations to run using hardware-accelerated integer SIMD instructions.
- Pre-Filtering on Hard Metadata: Before performing HNSW graph traversal, candidates are pre-filtered on physical constraints (is the restaurant open, and is it within the 5-mile delivery radius?).
Comparison Matrix
| Dimension | DoorDash | Instacart | Uber Eats |
|---|---|---|---|
| LLM Placement | Offline Catalog Enrichment | Real-Time Query Rewriter | Vector Retrieval Backbone |
| Runtime Serving Overhead | Negligible (~0 ms) | Moderate (~200 to 300 ms on tail) | Low (~25 to 45 ms via HNSW) |
| Existing Infra Anchor | Product Knowledge Graph | Elasticsearch + Tagging | Vector Indexing + Global Footprint |
| Primary Strength | Zero hallucination risk | Deep understanding of complex queries | Multi-language, unified global search |
| Main Vulnerability | Unmapped items stay hidden | High GPU compute cost on tail | Hard filters require vector pre-masking |
There is no universally superior architecture. When adding LLM intelligence to your search stack, do not ask “which model is best?” Ask where your existing infrastructure gives you an unfair advantage: in your catalog graph, in your query understanding pipeline, or in your vector index.

/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.