DEV Community

#llminference

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
When a single GPU isn't sufficient to host or compute the model, how would you split a large LLM across multiple GPUs or…
technaom profile

When a single GPU isn't sufficient to host or compute the model, how would you split a large LLM across multiple GPUs or…

#aiinterview #llminference #genai #aiarchitect
Comments
2 min read
H200 vs H100 for LLM inference: which GPU to buy, and where the B200 fits
azrty profile

H200 vs H100 for LLM inference: which GPU to buy, and where the B200 fits

#gpuselection #h200 #h100 #llminference
Comments
13 min read
FreeToken: Run Frontier MoE Models on Your Own GPU, and the RAM It Takes
itxtayab profile

FreeToken: Run Frontier MoE Models on Your Own GPU, and the RAM It Takes

#opensource #localllm #mixtureofexperts #llminference
Comments
9 min read
What optimizations would you employ to achieve high throughput and low latency?
technaom profile

What optimizations would you employ to achieve high throughput and low latency?

#aiinterview #llminference #genai #aiarchitect
Comments
2 min read
LLM cold starts on HyperPod: from 27 minutes to seconds with model caching
fernando_azevedo_6844e930 profile

LLM cold starts on HyperPod: from 27 minutes to seconds with model caching

#aiagents #sagemakerhyperpod #llminference #coldstart
Comments
9 min read
vLLM v0.28.0: the breaking change small GPU users must read
induwara_ashinsana_9e4d5b profile

vLLM v0.28.0: the breaking change small GPU users must read

#vllm #llminference #opensource
Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.