DEV Community

Edge AI in Practice Series' Articles

Back to GINIGEN AI's Series
Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy
Cover image for Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy

Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy

Comments
6 min read
Building an AI That Refuses to Say "Safe": Reliability Engineering for an Offline Disaster App
Cover image for Building an AI That Refuses to Say "Safe": Reliability Engineering for an Offline Disaster App

Building an AI That Refuses to Say "Safe": Reliability Engineering for an Offline Disaster App

Comments
7 min read
4-bit GGUF Quality for MoE Models: Why Only 3B of 180B Params Fire, and How to Prove Parity

4-bit GGUF Quality for MoE Models: Why Only 3B of 180B Params Fire, and How to Prove Parity

Comments
7 min read
One Self-Improving Model, Eleven Number-One Titles: What That Takes

One Self-Improving Model, Eleven Number-One Titles: What That Takes

Comments 1
2 min read
Zero-Token Decisions in the Browser: On-Device Multimodal Classification with EmbeddingGemma 2 and WebGPU

Zero-Token Decisions in the Browser: On-Device Multimodal Classification with EmbeddingGemma 2 and WebGPU

Comments
3 min read
LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget
Cover image for LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

Comments
11 min read
0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs
Cover image for 0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

Comments
14 min read
The Inference Hardware You Already Own Might Train Models Too: A 325M Pretraining Run on an NPU
Cover image for The Inference Hardware You Already Own Might Train Models Too: A 325M Pretraining Run on an NPU

The Inference Hardware You Already Own Might Train Models Too: A 325M Pretraining Run on an NPU

Comments
3 min read
From 46M to 325M: An Inference NPU Now Pretrains a Language Model at GPU Quality
Cover image for From 46M to 325M: An Inference NPU Now Pretrains a Language Model at GPU Quality

From 46M to 325M: An Inference NPU Now Pretrains a Language Model at GPU Quality

Comments
2 min read
LFM3-1.7B On-Device: The 256K KV Cache Math, Which GGUF to Ship, and the Safety Test to Run First
Cover image for LFM3-1.7B On-Device: The 256K KV Cache Math, Which GGUF to Ship, and the Safety Test to Run First

LFM3-1.7B On-Device: The 256K KV Cache Math, Which GGUF to Ship, and the Safety Test to Run First

Comments
14 min read
Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model
Cover image for Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model

Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model

Comments
5 min read
Asymmetric Control for AI Agents: Why the Stop Path Must Be Independent of the Model
Cover image for Asymmetric Control for AI Agents: Why the Stop Path Must Be Independent of the Model

Asymmetric Control for AI Agents: Why the Stop Path Must Be Independent of the Model

Comments
3 min read
A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise
Cover image for A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

Comments
15 min read
A 763B MoE on a 12 GB GPU: Our Speed Ceiling Versus What Strata Reports, and What Closes the Gap
Cover image for A 763B MoE on a 12 GB GPU: Our Speed Ceiling Versus What Strata Reports, and What Closes the Gap

A 763B MoE on a 12 GB GPU: Our Speed Ceiling Versus What Strata Reports, and What Closes the Gap

Comments
13 min read
Kimi K3 in 8 GB of RAM: Why 28x More Memory Buys Only 1.7x Speed, and the Byte Math That Predicts It
Cover image for Kimi K3 in 8 GB of RAM: Why 28x More Memory Buys Only 1.7x Speed, and the Byte Math That Predicts It

Kimi K3 in 8 GB of RAM: Why 28x More Memory Buys Only 1.7x Speed, and the Byte Math That Predicts It

Comments
12 min read
Does 2-Bit Quantization Break On-Device Safety? What an EMNLP 2026 Study Found, and a GGUF Release Gate
Cover image for Does 2-Bit Quantization Break On-Device Safety? What an EMNLP 2026 Study Found, and a GGUF Release Gate

Does 2-Bit Quantization Break On-Device Safety? What an EMNLP 2026 Study Found, and a GGUF Release Gate

1
Comments
14 min read
One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up
Cover image for One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up

One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up

Comments
12 min read
Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200
Cover image for Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200

Cutting 64 of 256 Experts from a GGUF by Byte Copy: What the Headers Say About GLM-5.3-SLIM on One B200

Comments
13 min read