This is a debugging story where every error message was technically true and completely misleading.
I run a local AI assistant built on the Claude Agent SDK. The architecture has a gateway layer: the desktop client talks to my server, and the server decides which upstream LLM provider handles the request. Anthropic direct, an OpenAI-compatible endpoint, whatever the user configured.
Everything worked against Anthropic. Then I pointed it at a DeepSeek gateway that speaks the OpenAI protocol, and every single request failed:
Model does not exist. Please check the model name.
The model name was correct. I checked it four times — deepseek-chat, spelled right, available on the provider's model list, working fine in a plain curl. The error was a lie, or at least a badly chosen message for what was actually happening.
Three separate bugs conspired to produce it. Here they are, because each one is a trap you will hit if you build an OpenAI-compatibility layer.
Bug 1: A Query String That Broke Routing
The Claude Agent SDK builds its request URL from a configured base URL, and it appends ?beta=true to certain endpoints. That's harmless against the Anthropic API. It's poison for a gateway that routes on path.
The OpenAI-compatible gateway matches on the path prefix — /v1/chat/completions — and ?beta=true is a query parameter, which shouldn't matter at all. But the routing layer in front of it was doing exact-ish matching on the full path string, so /v1/chat/completions?beta=true fell through the route table and landed on a default handler that returned... "model does not exist."
The fix is one line: strip the query string before routing.
// Normalize the incoming path for routing — SDK query params are not meaningful upstream
const url = new URL(req.url, 'http://localhost');
const routePath = url.pathname; // /v1/chat/completions, no ?beta=true
The lesson: never assume a query parameter is harmless just because the HTTP spec says it is. If any layer does string matching on URLs, unknown parameters can change behavior. Normalize at the boundary, once, explicitly.
Bug 2: Assistant Messages Must Be Merged, Not Appended
The Claude Agent SDK represents a turn with tool use as separate content blocks — one for the text, one for the tool call, one for the tool result. The Anthropic Messages API accepts that natively:
[
{ "role": "assistant", "content": [
{ "type": "text", "text": "Let me look at that file." },
{ "type": "tool_use", "id": "tu_1", "name": "read_file", "input": { "path": "a.ts" } }
]},
{ "role": "user", "content": [
{ "type": "tool_result", "tool_use_id": "tu_1", "content": "..." }
]}
]
The OpenAI Chat Completions API does not. Its assistant message has a content string field and a separate tool_calls array. If you naively translate block-by-block, you emit two consecutive assistant messages:
[
{ "role": "assistant", "content": "Let me look at that file." },
{ "role": "assistant", "content": null, "tool_calls": [...] }
]
Some gateways reject this. Some accept it and quietly lose the text. DeepSeek's gateway was in the first camp, and its error surfaced as the same unhelpful "model does not exist."
The fix is a translation step that merges all blocks from one assistant turn into one message:
function toOpenAIMessages(blocks: SdkMessage[]): OpenAIMessage[] {
const out: OpenAIMessage[] = [];
let pendingAssistant: any = null;
for (const m of blocks) {
if (m.role === 'assistant') {
if (!pendingAssistant) pendingAssistant = { role: 'assistant', content: '', tool_calls: [] };
for (const b of m.content) {
if (b.type === 'text') {
pendingAssistant.content += b.text;
} else if (b.type === 'tool_use') {
pendingAssistant.tool_calls.push({
id: b.id,
type: 'function',
function: { name: b.name, arguments: JSON.stringify(b.input) },
});
}
}
} else {
if (pendingAssistant) { // flush before switching roles
if (pendingAssistant.tool_calls.length === 0) delete pendingAssistant.tool_calls;
out.push(pendingAssistant);
pendingAssistant = null;
}
// user turn: text and each tool_result become their own message
for (const b of m.content) {
if (b.type === 'text') out.push({ role: 'user', content: b.text });
else if (b.type === 'tool_result') {
out.push({ role: 'tool', tool_call_id: b.tool_use_id, content: String(b.content) });
}
}
}
}
if (pendingAssistant) {
if (pendingAssistant.tool_calls.length === 0) delete pendingAssistant.tool_calls;
out.push(pendingAssistant);
}
return out;
}
The lesson: when you translate between two protocols, translate at the semantic level (a turn), not the syntactic level (a block). The shapes look similar enough that block-by-block mapping almost works, and "almost" is where the debugging hours go.
Bug 3: Errors That Describe the Wrong Layer
The third bug isn't in my code — it's that the failure message pointed at the wrong thing.
"Model does not exist" is what the gateway says when any part of its request validation fails, probably because the model field is validated first and that's the only error it knows how to articulate. My request had a perfectly valid model name; the actual problem was in the message array's shape.
This is a recurring pattern with OpenAI-compatible gateways: broad error messages with a single, specific-sounding cause. When you see "model not found" from a compatibility layer, don't just re-check the model name — dump the exact request body you sent and compare it against what a known-good curl sends.
What I ended up adding to my gateway was a request echo on failure:
try {
const upstream = await fetch(target, { method: 'POST', body: JSON.stringify(payload), ... });
if (!upstream.ok) {
const detail = await upstream.text();
logger.warn('upstream rejected', {
status: upstream.status,
detail,
sentPath: routePath,
sentMessageCount: payload.messages.length,
roles: payload.messages.map(m => m.role),
});
}
return upstream;
}
Logging the roles array is what actually cracked it. Once I saw ["user","assistant","assistant","user"] with two adjacent assistant entries, the merge bug was obvious in five seconds, after an hour of staring at the model name.
One More Trap: Usage That Silently Reports Zero
This one came from a reader, and it is nastier than either bug above because nothing fails.
Most OpenAI-compatible shims emit usage only in the final streamed chunk — or not at all. A naive accumulator that reads usage per chunk, or that starts from {} and sums whatever shows up, ends up recording zero tokens. Your cost dashboard then reports $0.00 for a workload that is clearly running.
The reason this survives for weeks is that the failure mode looks healthy: no errors, no latency spike, bills that look great. Nobody investigates a system that appears to be saving money.
So the gateway now asserts on usage instead of trusting it:
function assertUsage(usage, requestId) {
const total = usage?.total_tokens;
// Streaming shims frequently omit usage entirely. Zeros are not "cheap" —
// they are a broken accumulator, and a broken accumulator means the cost
// pipeline is silently lying to you.
if (!Number.isFinite(total) || total <= 0) {
logger.error('usage missing or zero', {
requestId,
raw: usage, // what upstream actually sent
promptTokens: usage?.prompt_tokens ?? null,
completionTokens: usage?.completion_tokens ?? null,
});
metrics.increment('gateway.usage.invalid');
throw new UsageError(requestId, usage);
}
return total;
}
Two details matter more than the code:
-
Log the raw object, not the normalized one. Normalization is where the discrepancy is introduced. If you only log
prompt_tokens ?? 0, you can no longer tell "upstream sent 0" apart from "upstream sent nothing and we invented a zero." Those are different bugs with different fixes. -
Fail loudly, and count it. A thrown error with a counter attached is the only version of this that shows up on a dashboard. A silent
?? 0is not a fallback, it is data corruption with good manners.
The general shape of the lesson is the same as the routing bug: a compatibility layer's job is to tell you when it cannot translate, not to smooth the difference away. An invented zero and a lie are the same thing.
The Checklist I Now Use
Wire a new OpenAI-compatible upstream:
- [ ] Strip or normalize query strings before any path-based routing
- [ ] Merge consecutive same-role messages into one before sending upstream
- [ ] Assistant text and
tool_callsgo in the same message, not two - [ ] Every
tool_resultbecomes arole: "tool"message with matchingtool_call_id - [ ] Log the full request shape (roles array, message count) on any non-2xx
- [ ] Assert
usage.total_tokens > 0on every response; log the raw usage object when it fails - [ ] Verify against a known-good
curlpayload, not just "the docs"
The deeper point: compatibility layers fail at the seams, not in the middle. A gateway that "speaks OpenAI" accepts the happy path your curl test used, and falls over on the message shapes your agent actually produces — because agents build messages incrementally, across turns, with tool calls and results interleaved, and no human hand-writes those.
If you're building an agent on top of an OpenAI-compatible endpoint, test with a real multi-turn tool-use conversation in your very first integration run. The one-shot curl test will pass and tell you nothing.
The gateway lives in @blm/server in github.com/lujianfu2008/xiaoqingting. Previous post: teaching the assistant new skills at runtime.
Top comments (2)
That's the sharper framing — thank you. The fallback route should have been a 404 echoing the path; returning a model error from a route miss is exactly the kind of message that costs you an hour of staring at the wrong thing.
The
usagepoint is a genuinely nasty one and I hadn't handled it. In a streaming shim,usageoften arrives only in the final chunk (or not at all), so a naive accumulator records zero and the cost pipeline silently reports $0. Worse, that failure looks healthy — cheap bills, no errors, nobody investigates. I'm adding an assertion on the aggregated final usage and a loud failure whentotal_tokensis missing or 0, plus logging the raw and normalized path per request as you suggested.Appreciate both — the "route miss lies" diagnosis is going into my checklist verbatim.
Great catch on the query string. The deeper bug is that the default route lied: a route miss should never come back as a model error, because that message sends you to the model catalogue instead of the router. Two cheap guards: log (raw path, normalized path, matched route) per request, and make the fallback a 404 that echoes the path.
One more item for the compat checklist: usage. Plenty of OpenAI-compatible shims return usage missing or zero on streamed responses - your cost pipeline then records $0 and nobody notices for weeks. Assert usage.total_tokens > 0 on every response and fail loudly.