Skip to content

DEV Community

InferenceKey

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

inferencekey.com Joined on Oct 14, 2026

Nov 14

Clover-35B-A3B on llama.cpp: what "AR mode only" means, and a GGUF that answers in Gs

#ai #llm #gpu #selfhosted

4 min read

Nov 13

Serving Hy-SimulMT live on one 48 GB GPU: the startup spike, the FP8 trap, what to measure

#ai #machinelearning #gpu #selfhosted

5 min read

Nov 12

Serving Groot-19B-A3B at 256K on one RTX 5090: the memory budget and the thinking trap

#ai #llm #gpu #selfhosted

5 min read

Nov 11

Parsing a whole PDF in one request with Unlimited OCR 2: the crop_mode setting and a silent cut

#ai #machinelearning #python #gpu

4 min read

Nov 10

Nemotron 3.5 Transcribe on one 32 GB Blackwell card: the prompt is your API, so test it

#ai #machinelearning #gpu #python

5 min read

Nov 9

Serving a stateful video world model on one RTX 4090: memory, flags and seeds in Hy-Nomas

#ai #machinelearning #gpu #opensource

4 min read

Nov 8

Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure

#ai #llm #opensource #gpu

5 min read

Nov 7

Ming-3.0-tts on a 16 GB card: the GPU sat mostly idle, and why a bigger one barely helps

#ai #machinelearning #python #gpu

4 min read

loading...