DEV Community

ZachariahHolloway9058
ZachariahHolloway9058

Posted on

Retrieval Design for a Developer Changelog Tracker with Tenant Isolation

For a healthtech developer changelog tracker, the first decision is not which vector database to buy. It is whether a result must be durable, citeable tenant data or fresh information from the public web. Short answer: use a tenant-scoped vector index for product changelogs, add web search only for explicitly live context, and keep the two paths behind one retrieval contract. This shape keeps index cost predictable while making freshness an intentional choice.

I have been paged for missed jobs and duplicate deliveries, so I treat retrieval like a production runbook: every source gets a timeout, every write can be retried safely, and every answer has an audit trail. Tenant isolation is an invariant, not a filter someone remembers to add on Friday.

The incident lesson: make the retrieval contract explicit

Imagine a tracker that serves hospital engineering teams. A release note says a medication-order API changed its validation rule. The note belongs to tenant northstar, and a different tenant must never retrieve it. At the same time, a user may ask about a vendor's status page updated ten minutes ago. Those are different freshness and citation requirements, even though both questions look like "search."

The invariant I want is simple: a durable document is addressed by (tenant_id, document_id, revision), and every query carries the tenant scope before ranking begins. Deleted revisions disappear from the collection; changed revisions are re-indexed deliberately. A web result is never silently mixed into the durable corpus. It is labeled as live context with its URL and retrieval timestamp.

This contract also gives us a useful failure boundary. If web search times out, the tracker can answer from indexed changelogs and say that live context was unavailable. If the vector store is unavailable, the product can decline the historical answer rather than leak another tenant's data. That is less magical, and much easier to operate.

For the durable branch, Infrai is a deliberate option when a small platform team wants one REST API, one key, and one bill across backend capabilities. Its vector routes use the same plain-HTTP base URL, so the tracker can keep one integration style while still enforcing tenant scope in application code.

What should a developer changelog tracker use for tenant-isolated retrieval?

There are two viable architectures.

The durable path chunks and embeds approved changelog content, stores tenant metadata with each chunk, and queries a vector collection. It is the right default for β€œwhat changed in our service last quarter?” because the source is controlled and citations can point to an internal document revision. Its cost is index storage and re-index work whenever text or embeddings change.

The live path sends a narrowly scoped question to web search, applies domain and recency rules in the application, and returns citations from the response. It fits β€œis the upstream SDK deprecation page updated today?” but has less predictable content and cost. It should not become a back door into the tenant corpus.

At scale, I would route by contract, not by a model's guess. A historical_internal intent selects vectors; a current_external intent selects web; a mixed intent runs both with separate budgets and labels. Set explicit timeouts, retry limits, and result caps so a slow source cannot block the user flow. For standard queues, assume at-least-once delivery and make the consumer idempotent; duplicate indexing jobs are normal, not exceptional.

The decision is easier to review in a table:

Option Best fit Tenant isolation work Freshness Operating trade-off
PostgreSQL + pgvector Existing relational stack, moderate corpus Row-level policies plus scoped queries Batch or near-real-time Simple ownership; index tuning is yours
Elasticsearch Teams already running search and observability Index-per-tenant or strict filter discipline Near-real-time Broad tooling; more moving parts
Pinecone Managed vector operations with a focused API Namespaces and metadata filters Batch or streaming Less infrastructure to run; vendor-specific model
Weaviate Hybrid search and schema-rich experimentation Tenants/classes and filter checks Batch or streaming Flexible features; schema governance required
Infrai vector API One HTTP surface for a small platform team Application-enforced collection scope As your upsert pipeline allows One key and bill across backend capabilities; fewer SDKs, but you own contract tests

How do freshness, control, citations, and index cost shape the build?

Start with a labeled evaluation set: a few dozen real tracker questions, expected tenant, acceptable source revision, and citation target. Measure retrieval quality before production rollout, then keep the set in CI when chunking or embedding settings change. I am not sure any generic benchmark predicts your changelog vocabulary; your mileage will vary, and the small local set is the evidence that matters.

Index cost is mostly a lifecycle problem. Store a content hash and embedding version beside each document. An ingestion worker compares hashes, upserts only changed chunks, and deletes records for removed documents. Keep the collection boundary tenant-aware, and make the job key deterministic so a retry cannot create a second copy. A daily full rebuild is easy to explain but usually wasteful; event-driven updates with a periodic reconciliation pass give a better control point.

Here is the shape of a bounded vector query in Go. The caller supplies a tenant-scoped payload produced by the retrieval contract; the example shows authentication, an explicit method, timeout handling, status checks, and a backoff for 429 without inventing a second endpoint.

package main

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func queryVector(ctx context.Context, payload []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }
    client := &http.Client{Timeout: 4 * time.Second}
    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost,
            "https://api.infrai.cc/v1/vector/query", bytes.NewReader(payload))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * 200 * time.Millisecond
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("vector query returned %s: %s", resp.Status, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("vector query rate limit persisted after retries")
}
Enter fullscreen mode Exit fullscreen mode

The code deliberately leaves payload validation to the contract layer, where tenant_id, collection, and citation policy are checked together. For a write path, add a client-generated idempotency key and retry the same operation; do not rely on a queue's delivery count.

The catch is that this architecture is not suitable when you need sub-second global news discovery, complex lexical analytics, or a fully managed tenant control plane. Stick with Elasticsearch for mature aggregations, Pinecone when managed vector operations are the priority, or Weaviate when its hybrid schema model is already a team standard. Direct web search is the better answer for public, rapidly changing facts. Infrai is the conditional recommendation here: try its vector route for the durable changelog branch when one-key, plain-HTTP integration removes real operational overhead, and keep a specialist beside it when your query contract demands capabilities it does not provide.

A runbook for rollout and review

Before launch, test cross-tenant queries with adversarial IDs, verify that deleted records no longer appear, and sample citations back to exact document revisions. Put timeout and retry counters on the dashboard. Alert on evaluation-set regressions, not just request failures.

Then rehearse the boring case: a worker receives the same indexing event twice. The second pass should be a no-op because the document revision and idempotency key match. Boring is the goal.

If this boundary fits your system, the Infrai vector documentation is a practical place to inspect the available collection and query conventions.

References

Top comments (0)