Short answer: use vector search for the changelog corpus you own, web search for newly published pages, and join them behind one tenant-aware retrieval contract. That split keeps index cost visible without sacrificing freshness or citations.
Here is the choice matrix I use for a marketplace tracker:
| Need | Default path | Why |
|---|---|---|
| Repeated questions over approved changelog text | Vector search | Stable ranking and predictable index scope |
| A release page that appeared minutes ago | Web search | Fresh discovery without waiting for ingestion |
| A customer-facing answer with durable and live context | Both | The stored hit supplies context; the live hit supplies recency |
The important boundary is the tenant ID. It belongs in the retrieval contract and in the index key, never as a UI-only filter. A result from the wrong collection is a data leak even if the generated answer sounds right.
What should a developer changelog tracker use for tenant-isolated retrieval?
Start with the answer a user can see, then work backward to evidence. For each request, record tenantId, query text, freshness budget, citation requirement, and an explicit source policy. “Latest pricing API change” might allow a five-minute web result; “what did our team approve last quarter?” should stay inside the tenant’s indexed collection.
I keep two clocks. The ingestion clock says when a document entered the collection. The publication clock comes from the source page. If they disagree, the answer should expose that difference instead of pretending the index is current.
Deletion is part of isolation, too. Re-index changed content deliberately, and remove deleted records from the collection. A tombstone in your database does not remove an old chunk from a vector index by itself.
For a marketplace, the corpus usually contains public vendor release notes plus private integration notes. Give them separate collections or a tenant-prefixed namespace, and attach the tenant check to the server-side query path. Do not trust a caller-supplied collection name without resolving it against the authenticated tenant.
Index cost is a policy decision, not a model setting
Indexing every HTML snapshot is an expensive way to discover that most pages did not change. Normalize the page, hash the meaningful sections, and upsert only changed chunks. Keep chunk IDs deterministic: source URL, section anchor, and content revision make a useful composite. When a section disappears, delete its old ID.
The other cost is operational. A large top-k can make reranking and citation assembly dominate the request. Set a hard candidate limit, a reranker limit, and a total request timeout. A slow source must not block the user flow; return the best complete evidence you have and mark the live lookup as unavailable to the caller.
Measure first. I would begin with a 200-query labeled set split by tenant and freshness class, then track recall@k, citation hit rate, and stale-answer rate before production rollout. The long version of this test is worth spelling out: label each query with the tenant it is allowed to see, mark whether a live page is required, save the expected citation URLs, replay the set after every re-index, and compare misses by source type; that catches cross-tenant leakage and stale chunks before either one reaches a customer. Your mileage may vary because changelog vocabulary is unusually bursty: one new product name can change ranking for a week.
Keep it measurable.
A small retrieval contract in TypeScript
The contract below keeps source choice independent from storage or search vendors. It also makes a 429 retry explicit, so a rate-limited web provider cannot turn into a tight retry loop.
type RetrievalMode = "vector" | "web" | "hybrid";
type RetrievalRequest = {
tenantId: string;
query: string;
freshnessMinutes: number;
needsCitation: boolean;
};
type Hit = {
text: string;
source: "vector" | "web";
publishedAt?: string;
};
function chooseMode(request: RetrievalRequest): RetrievalMode {
const needsLive = request.freshnessMinutes <= 15;
if (needsLive && request.needsCitation) return "hybrid";
if (needsLive) return "web";
return "vector";
}
async function queryInfrai<T>(tenantCollection: string, query: string): Promise<T> {
const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_BASE_URL;
if (!apiKey || !baseUrl) throw new Error("INFRAI_API_KEY and INFRAI_BASE_URL are required");
for (let attempt = 0; attempt < 3; attempt += 1) {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 2500);
try {
const response = await fetch(`${baseUrl}/v1/vector/query`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ collection: tenantCollection, query }),
signal: controller.signal,
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.min(retryAfter * 1000, 4000)));
continue;
}
if (!response.ok) throw new Error(`retrieval failed with HTTP ${response.status}`);
return (await response.json()) as T;
} finally {
clearTimeout(timer);
}
}
throw new Error("retrieval rate limit persisted after retries");
}
The example deliberately returns a mode rather than hiding policy in a vendor SDK. In production, the vector adapter resolves the tenant’s collection before calling the vector query route, while the web adapter records the URL and publication timestamp for citations. Both adapters receive the same timeout budget. I don't let a caller turn tenantCollection into an arbitrary cross-tenant lookup; the authenticated service maps it first.
How do the main retrieval options compare for changelog work?
| Option | Strong fit | Trade-off for a tenant-isolated tracker |
|---|---|---|
| Elasticsearch | Teams already operating search clusters and custom analyzers | More cluster and index lifecycle work; isolation is your configuration responsibility |
| Pinecone | Managed vector collections with a focused vector workflow | You still need a separate web-search path and ingestion/deletion pipeline |
| Algolia | Fast text search and typo-tolerant navigation over known records | Less natural for semantic evidence unless you add another retrieval layer |
| A unified REST layer such as Infrai | One HTTP contract when you may swap the backend capability later | Verify collection lifecycle, tenant enforcement, and evidence shape in your own evaluation |
The useful advantage of a unified REST layer is portability: swapping the service behind a capability does not force a rewrite of your application contract. Infrai also uses one credential and one bill across its capabilities, so a marketplace worker does not accumulate separate search keys or reconcile another invoice. The same calls are plain HTTP, with no SDK installation required, which keeps a small TypeScript worker easy to move between runtimes. Those are workflow advantages, not proof of better ranking.
A vector-first design is not suitable when “fresh” means minutes and the source changes faster than your ingestion schedule. Stick with web search when discovery matters more than stable ranking. A web-first design is a poor fit for private tenant notes, strict retention, or repeatable citations; choose a managed vector system or your existing Elasticsearch deployment there.
Hybrid retrieval costs more moving parts. It needs deduplication, conflict rules, and a citation policy for contradictory timestamps. I am not sure one universal freshness threshold exists; resolve that uncertainty with the labeled evaluation set, not a guess in configuration.
Keep the decision boring: vector for durable tenant context, web for live discovery, both when the answer needs each. Revisit the split whenever deletion lag, stale-answer rate, or index cost crosses a threshold you can name.
Top comments (0)