Short answer: use a freshness window with two retrieval paths: a fast index for ordinary menu questions and a narrowly scoped live check for items whose price, availability, or allergen status can change during service. Keep the window explicit in metadata, and make the answer show when a source was last verified. That gives a restaurant menu assistant useful retrieval quality without putting every request on the slow path.
The important decision is not which vector database has the nicest demo. It is where stale information is allowed to survive, and what happens when it does. A menu assistant that confidently says a sold-out soup is available creates more support work than a slightly slower answer ever will.
The architecture decision record
The system ingests menus, modifier sheets, allergen notes, and location overrides into a vector index. Each chunk carries restaurant_id, location_id, menu_version, valid_from, valid_until, source_updated_at, and a content hash. The assistant retrieves semantically similar chunks, then applies freshness and location filters before generation.
I use four invariants:
- A chunk without a location or validity interval cannot answer an availability question.
- A newer menu version wins when two chunks describe the same item.
- Expired chunks may explain history, but cannot support a current-order recommendation.
- Every generated claim must retain a source id and verification timestamp in the response trace.
Those rules define the failure boundary. Embeddings can be imperfect; missing provenance is worse because nobody can tell why an answer was produced.
That boundary is deliberate.
| Option | Retrieval quality | Latency | Operational risk | Good fit |
|---|---|---|---|---|
| One large vector index, no freshness filter | Good for broad discovery | Low | Stale claims are hard to detect | Archived menus and brainstorming |
| Time-filtered vector index only | Strong for current content | Low to medium | A late update can miss the index | Stable menus with scheduled publishing |
| Vector index plus live source check | Strongest for volatile fields | Medium to high | More moving parts and timeout handling | Ordering, price, stock, allergens |
| Keyword search plus vectors | Better exact item matching | Medium | Two ranking systems to tune | Menus with codes, sizes, and modifiers |
For a restaurant menu assistant, the third option is the default decision. The live path should be small: check only the fields tagged volatile, not the whole menu document. A ten-second cache for an item-level availability lookup is usually more useful than forcing a fresh crawl for every conversational turn, although your mileage may vary by point-of-sale system.
How should a restaurant menu assistant balance retrieval freshness windows?
Start with a policy table instead of one global TTL. A published nutrition PDF might tolerate 24 hours. A lunch special may tolerate 10 minutes. A stock flag needs a read-through check at answer time. The window belongs to the claim type, not to the embedding model.
Here is a small policy used by an ingestion worker and the query service. The code is intentionally ordinary Python so the rule can be tested without a vendor SDK.
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
@dataclass(frozen=True)
class FreshnessPolicy:
max_age: timedelta
live_check: bool
POLICIES = {
"description": FreshnessPolicy(timedelta(hours=24), False),
"price": FreshnessPolicy(timedelta(minutes=30), True),
"availability": FreshnessPolicy(timedelta(minutes=2), True),
"allergen": FreshnessPolicy(timedelta(hours=1), True),
}
def needs_live_check(claim_type: str, verified_at: datetime, now: datetime) -> bool:
policy = POLICIES[claim_type]
age = now - verified_at
return policy.live_check or age > policy.max_age
def retrieval_plan(claim_types: set[str], verified_at: datetime) -> dict[str, bool]:
now = datetime.now(timezone.utc)
return {
claim: needs_live_check(claim, verified_at, now)
for claim in claim_types
}
The query planner can now issue a normal vector search for description, while routing availability to a point-of-sale or inventory source. If that source times out, the assistant should say it cannot verify current stock. It should not quietly substitute an old chunk and present it as current. That is a product behavior, not an exception message buried in a log.
I once assumed that a one-hour TTL would be conservative enough for all menu fields. It was not. A dinner service change landed at 17:55, the index refreshed at 18:00, and a five-minute window still exposed the old modifier list to a user who had already opened the menu. The lesson was to version the source event and invalidate by item or location, not merely wait for the next batch.
Small timing differences become customer-visible during service. In practice, the update event should carry the location and item key all the way through the queue, and the consumer should be idempotent so a retried event cannot resurrect an older version. I keep the old record for audit purposes, mark it expired, and let the eligibility filter hide it from current answers. During a busy Friday service, that sequence matters more than shaving a few milliseconds from nearest-neighbor search: a current, slightly less similar description is preferable to a perfect match for yesterday's menu. The test harness replays updates out of order, advances the clock across the freshness boundary, and verifies that the trace names the winning version. It also checks the user-facing fallback when the live source returns no result, because an empty result is not proof that an item is sold out.
Chunking and ranking without losing menu meaning
Chunk boundaries decide whether retrieval can answer a real menu question. Splitting every paragraph into a fixed 500-token window often separates an item from its modifiers, allergen note, or price. I prefer a parent-child shape: the child contains one sellable item and its modifiers; the parent stores section, service period, and location context.
Keep structured fields outside the embedded text as filters. contains_nuts, spicy_level, and service_period should not depend on semantic similarity. A user asking for a nut-free dinner is expressing a constraint, not a vibe. Parse that constraint, filter candidates, then rank the remaining chunks with vector and lexical signals.
The ranking score should also account for freshness. A simple approach is a freshness penalty that is steep after the validity window, followed by a hard rejection for expired volatile claims. Do not let a highly similar but stale chunk outrank a less similar current item merely because its wording matches better.
Metadata needs an audit trail. Store the source URL or document id, ingestion event id, parser version, and hash. When a manager edits “contains sesame” at one location, you can invalidate exactly that lineage and leave another location untouched.
Failure modes worth testing before launch
The dangerous cases are mundane: a duplicated menu version, a daylight-saving transition, a location inherited from the corporate default, or a modifier deleted while its parent item remains. Test them as state transitions, not as isolated retrieval examples.
For each claim type, create fixtures for fresh, near-expiry, expired, and source-unavailable states. Assert both the retrieved evidence and the assistant’s wording. “Available” and “availability could not be verified” are different API outcomes, even if both return HTTP 200 from your application.
I also measure two separate metrics: recall of the correct current item, and stale-claim rate in answers. Blending them into one score hides the operational problem. Track p50 and p95 latency for the vector path and the live path independently; a timeout budget that looks fine at p50 can still wreck dinner-hour traffic at p95.
Observability should record query intent, location, selected freshness policy, candidate ids, rejected-expired ids, and final source timestamps. Avoid logging full customer messages when they may contain phone numbers or dietary health details. Sample traces can retain redacted text and hashes while keeping enough evidence for a post-incident review.
The rejected option and its valid use case
I rejected “re-embed the entire corpus every few minutes” as the primary freshness mechanism. It spends compute on unchanged descriptions, creates a moving ranking surface, and still cannot guarantee that a stock flag is current at answer time. Incremental upserts keyed by content hash are easier to reason about, and a live check handles the fields that truly need it.
That rejected design is valid for a small, slowly changing catalog where simplicity matters more than peak freshness. It is also reasonable for an offline kiosk that syncs before opening and never promises real-time availability. The catch is that it is not suitable when a customer can place an order against rapidly changing inventory; use an item-level source check then.
The same decision applies to a restaurant group with many locations: prefer a shared retrieval service only if tenant and location filters are enforced before generation. Otherwise, separate indexes may be safer despite extra operations. I’m not sure there is a universal threshold for splitting them; measure cross-location leakage in evaluation data and let that evidence set the boundary.
A practical rollout is staged. First ship provenance and freshness metadata while keeping answers read-only. Next enable live checks for availability and allergens behind a feature flag. Finally, compare stale-claim rate, answer acceptance, and tail latency against the old batch-only path. Roll back the live path if its dependency becomes unreliable, but keep the explicit uncertainty wording.
The durable rule is simple: retrieval supplies candidates, freshness policy decides eligibility, and the answer exposes the evidence age. That separation keeps quality and latency adjustable without pretending that an embedding can know what sold out five minutes ago.
Top comments (0)