Inference

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Raff Martin (esatapedico)

Nov 14

Benchmarking across some of our published families

#ai #inference #opensource #benchmark

10 min read

Alex Chen

Nov 13

How Many Closed Findings Before You Trust a Routing Change

#llm #ai #gpu #inference

10 min read

Konrad Wright

Nov 13

What Happens When You Hit Enter?

#machinelearning #llm #inference

5 min read

Sai Prudhvi Neelakantam

Nov 13

Pose Estimation: MediaPipe or YOLOPose for Real-Time Pose Classification?

#computervision #poseestimation #realtimeai #inference

7 min read

gxqwb

Nov 11

Test-Time Compute vs More Parameters: Where the Inference Budget Actually Goes

#ai #inference #reasoning #scaling

5 min read

Dr. Agentic

Nov 11

Five-Axis Framework for Comparing AI Inference Chips

#ai #machinelearning #gpu #inference

13 min read

Vidraft_lab

Nov 9

수락 길이는 22.8% 늘었는데 처리량은 14.0%만 늘었습니다: DSpine 카드 수치와 공개 파일 헤더로 투기 디코딩 초안 모델을 읽는 법

#ai #machinelearning #llm #inference

3 min read

gxqwb

Nov 8

Prefix Caching vs Recomputed Prompts: The Hit-Rate Break-Even

#ai #inference #performance #caching

6 min read

Jax Sinclair

Nov 8

The Ridge Dies at 228 Tokens on a 4090, and KV Precision Cannot Save It

#hardware #performance #llm #inference

6 min read

gxqwb

Nov 7

Mixture-of-Depths vs Uniform Compute: The Routing Tax on Every Token

#ai #inference #performance #gpu

5 min read

Jonathan Murray for Backboard.io

Nov 7

12.8x faster for coding agents. At full precision.

#ai #llm #inference #agents

2 min read

devs

Nov 7

Hugging Face’s 70k-Word MoE Handbook After 800k GPU Hours — Why Sparsity Still Wins for Local Real-Time Tools

#moe #inference #localai #windows

4 min read

Jax Sinclair

Nov 6

A 32B Model on an H100 Never Reaches Its KV Cache Crossover

#hardware #performance #llm #inference

6 min read

Konrad Wright

Nov 4

Why Pay for the Smartest Model Every Time?

#machinelearning #inference #efficiency #cost

6 min read

Jonathan Murray for Backboard.io

Nov 7

Up to 95% more output from one GPU on Qwen3.6

#ai #llm #inference #gpu

2 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.