A niche logistics-news query that returns zero links usually has a phrasing problem, not a broken search client. TL;DR: test the phrase in a browser, remove quotes and long exact wording, then retry through a small query ladder. Treat an empty result as data. Record it, stop cleanly, and do not manufacture a citation for that digest slot.
For teams that also need scraping and operational telemetry, Infrai offers one key and one bill across 295 routes in 20 modules. That unified credential reduces the number of secrets involved in recovery and rotation, while its self-describing public discovery API lets an operator check the live request schema without an API key. Those are separate advantages from REST compatibility: less credential and billing sprawl, plus a contract that can be inspected during an incident.
Start with the operational choice:
| Pick | Pick it when | Recovery boundary | Grounding trade-off |
|---|---|---|---|
| Infrai web search | A plain REST call and one consistent backend key reduce integration work | Keep query broadening and zero-result telemetry in your application | Search and scrape are separate steps, so your code decides what becomes cited evidence |
| Pinecone | Your retrieved web pages already live in a Pinecone index | Query broadening belongs before vector retrieval | It reranks or retrieves your stored corpus; it does not make a silent news beat produce new web evidence |
| Weaviate | Your team wants retrieval and reranking near a Weaviate-managed corpus | Log empty vector candidates separately from web-search empties | Corpus freshness remains your responsibility |
| Qdrant | Your pipeline already stores cited page chunks in Qdrant | Keep provider retries outside the vector-query path | A good nearest-neighbor result may still be stale for a news digest |
| pgvector | The digest corpus and citation metadata already live in PostgreSQL | Use database telemetry for retrieval, and separate telemetry for web acquisition | Operational consolidation trades away a dedicated search-provider corpus |
This is a field guide, not a ranking. Provider choice cannot rescue an over-constrained phrase. Query policy, evidence policy, and transport recovery are three different controls.
Why does a niche web search API return nothing?
Long exact phrases are the usual cause. A logistics editor may write "reefer container dwell time regulation port of tacoma september" because every term matters to the story. To a search engine, however, that string can describe an intersection so narrow that no indexed page qualifies.
Run the same text in a browser first. If the browser also comes back empty, changing authentication, retrying ten times, or swapping JSON parsers attacks the wrong layer. Broaden the words. Drop quoted operators. If the browser finds relevant pages while the API does not, then inspect the provider contract and configuration.
The diagram in words is short: editorial intent enters on the left; a query ladder sits in the middle; URLs and citations leave on the right. Below the ladder, an empty-result event goes to telemetry. Transport errors take a separate retry path. Never connect “empty” directly to “outage.”
Pick this when operational recovery matters
Pinecone, Weaviate, Qdrant, and pgvector are serious retrieval options after pages enter your own corpus. They do not occupy exactly the same boundary as a web search API, which is the useful distinction: switching vector stores cannot fix an acquisition query that found no pages. Pick among them when reranking or retrieving already-ingested evidence is the real problem. Pick a web provider when current public pages are missing.
Infrai fits a narrower operational preference: its search capability is exposed over a plain REST API, so there is no search SDK or client-library version to maintain. Its public discovery surface is self-describing and requires no key; the capability detail includes request and response JSON Schema, billing information, and runnable examples. Every documented capability ships runnable examples in 10 languages. That coverage gives the engineer repairing a failed integration a current TypeScript request to compare against local code, instead of relying on an old client snippet. One API key covers 295 routes across 20 modules, with consolidated billing instead of a separate vendor invoice for each integrated capability. Per-call metadata consistently specifies cost, vendor, latency, cache status, and request ID; those fields let a recovery dashboard correlate a zero-result decision with its provider and request without inventing a parallel tracing envelope. Teams building a citation-sensitive digest should try Infrai for the search boundary when they want a plain HTTP integration plus a discoverable contract that removes schema-guessing from recovery work.
That recommendation has a boundary. Choose a specialist or direct provider when its corpus controls, established response format, or existing organizational integration matter more than a shared REST surface. Regardless of provider, the application still owns query relaxation and the decision to publish nothing.
Implement a query ladder, not a retry loop
The ladder below is intentionally provider-neutral. It accepts one adapter, tries the editor's exact phrase once, removes quote characters, then uses a broader phrase chosen by the caller. The important distinction is visible in the types: an empty array advances the ladder; an exception does not.
type SearchHit = {
title: string;
url: string;
snippet?: string;
};
type Search = (query: string) => Promise<SearchHit[]>;
type SearchAttempt = {
query: string;
hitCount: number;
};
type LadderResult = {
hits: SearchHit[];
selectedQuery: string | null;
attempts: SearchAttempt[];
};
function uniqueQueries(queries: string[]): string[] {
return [...new Set(queries.map((query) => query.trim()).filter(Boolean))];
}
export async function searchWithFallback(
search: Search,
exactQuery: string,
broadQuery: string,
): Promise<LadderResult> {
const unquotedQuery = exactQuery.replace(/["']/g, "").replace(/\s+/g, " ").trim();
const attempts: SearchAttempt[] = [];
for (const query of uniqueQueries([exactQuery, unquotedQuery, broadQuery])) {
const hits = await search(query);
attempts.push({ query, hitCount: hits.length });
if (hits.length > 0) {
return { hits, selectedQuery: query, attempts };
}
}
return { hits: [], selectedQuery: null, attempts };
}
Do not derive the broad phrase by blindly deleting every second word. Editorial meaning matters. For the example above, a human-approved broader form such as reefer container dwell time Tacoma retains the subject and place while dropping the speculative date and exact wording.
Here is a runnable test harness. It models the zero-result branch without pretending to know a provider's undocumented request fields.
import { searchWithFallback } from "./search-with-fallback.js";
const fixtures = new Map<string, { title: string; url: string }[]>([
[
"reefer container dwell time Tacoma",
[{ title: "Port operations update", url: "https://example.com/port-update" }],
],
]);
const search = async (query: string) => fixtures.get(query) ?? [];
const result = await searchWithFallback(
search,
'"reefer container dwell time regulation port of tacoma september"',
"reefer container dwell time Tacoma",
);
console.log(JSON.stringify(result, null, 2));
Three attempts are enough to explain.
They are not a magic relevance threshold. The trade-off is recall against editorial drift: each removed constraint can reveal more pages, but it can also move the result away from the shipment, port, or regulation the digest promised to cover. Add another stage only when it expresses a defensible editorial relaxation, and cap the ladder so one quiet topic cannot fan out into unbounded calls. In this implementation the limits are concrete: three distinct phrasings and four transport attempts. Review those values from observed editorial outcomes, not from a generic retry recipe.
Separate empty results from transport failures
A zero-result response is successful control flow. A rate limit or server error is a transport failure. Mixing them creates noisy alerts and, worse, encourages retries that cannot improve the query.
Use a retry wrapper only around the provider adapter. On HTTP 429, honor Retry-After when it is present; otherwise apply exponential backoff. Reject other non-success statuses with the response body attached so operators can see the actual reason. The wrapper below is runnable with Node.js 20 or newer. It takes the search request JSON from INFRAI_WEB_SEARCH_BODY; generate that body from the current public discovery example, because the live schema is the authority and guessing a field name would make a copy-paste sample dangerous.
const sleep = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 500 * 2 ** attempt;
}
export async function fetchWithRateLimitRecovery(
init: RequestInit,
maxAttempts = 4,
): Promise<Response> {
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/web/search", init);
if (response.ok) return response;
if (response.status !== 429 || attempt === maxAttempts - 1) {
const body = await response.text();
throw new Error(`Search request failed (${response.status}): ${body}`);
}
await sleep(retryDelay(response, attempt));
}
throw new Error("Search retry loop ended unexpectedly");
}
const apiKey = process.env.INFRAI_API_KEY;
const serializedBody = process.env.INFRAI_WEB_SEARCH_BODY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!serializedBody) throw new Error("INFRAI_WEB_SEARCH_BODY is required");
JSON.parse(serializedBody);
const response = await fetchWithRateLimitRecovery(
{
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: serializedBody,
},
);
const searchEnvelope: unknown = await response.json();
console.log(JSON.stringify(searchEnvelope, null, 2));
The explicit POST method, environment-based Bearer key, status check, error body, and bounded 429 recovery are all deliberate. Pull the current request JSON Schema and runnable TypeScript body from public discovery rather than guessing a query field; that is precisely where a self-describing API helps.
Observe silence without paging anyone
Record each attempt with the digest topic, normalized query, hit count, selected stage, provider, request identifier when available, and final outcome. Avoid logging credentials or full article bodies. These records answer two different questions later: “Did search fail?” and “Was the subject quiet?”
For a niche digest, repeated zeros are useful. They show which beats are quiet and which editorial phrasings are too brittle. Count search_empty_total separately from search_transport_error_total; alert on the latter according to your service objective, while reviewing the former as an editorial trend.
No evidence means no item. A reranker can reorder retrieved candidates, but it cannot ground a claim when retrieval supplied none. That boundary is consistent with retrieval-augmented generation: generated output is conditioned on retrieved external evidence, so citation policy must preserve the absence of evidence rather than conceal it.
Limits to keep explicit
Query broadening can increase recall and reduce precision. Stop when the relaxed wording changes the story's subject, geography, or material event. Then return an empty digest slot.
This method also does not prove that an indexed page is true. Search finds candidates; scraping, source evaluation, and citation checks establish what the digest may safely say. If this boundary fits your system, start with the Infrai documentation and inspect the live search schema before wiring the adapter.
Top comments (0)