DEV Community

e=mc²: Build an LLM From Scratch, One Lesson a Day Series' Articles

Back to AI Explore's Series
Build an LLM From Scratch in 5 Minutes on a Free GPU: I Trained a 5.9-Million-Parameter Storyteller, and Here Is What It Wrote
Cover image for Build an LLM From Scratch in 5 Minutes on a Free GPU: I Trained a 5.9-Million-Parameter Storyteller, and Here Is What It Wrote

Build an LLM From Scratch in 5 Minutes on a Free GPU: I Trained a 5.9-Million-Parameter Storyteller, and Here Is What It Wrote

Comments
5 min read
How Attention Works: I Read All 24 Heads of a Tiny Language Model and Found the One That Watches the Main Character's Name
Cover image for How Attention Works: I Read All 24 Heads of a Tiny Language Model and Found the One That Watches the Main Character's Name

How Attention Works: I Read All 24 Heads of a Tiny Language Model and Found the One That Watches the Main Character's Name

Comments 1
6 min read
Why Small LLMs Memorize: I Trained the Same Model at Four Sizes and Watched It Start Reciting
Cover image for Why Small LLMs Memorize: I Trained the Same Model at Four Sizes and Watched It Start Reciting

Why Small LLMs Memorize: I Trained the Same Model at Four Sizes and Watched It Start Reciting

Comments
6 min read
Catastrophic Forgetting, Measured: I Fine-Tuned a Small LLM on One New Character and Its Old Stories Went From Perplexity 9 to 156
Cover image for Catastrophic Forgetting, Measured: I Fine-Tuned a Small LLM on One New Character and Its Old Stories Went From Perplexity 9 to 156

Catastrophic Forgetting, Measured: I Fine-Tuned a Small LLM on One New Character and Its Old Stories Went From Perplexity 9 to 156

Comments
5 min read
Instruction Tuning From Scratch: Two Special Tokens and One Masked Loss Turn a Story Generator Into a Prompt Follower
Cover image for Instruction Tuning From Scratch: Two Special Tokens and One Masked Loss Turn a Story Generator Into a Prompt Follower

Instruction Tuning From Scratch: Two Special Tokens and One Masked Loss Turn a Story Generator Into a Prompt Follower

Comments
5 min read
LoRA Explained in Plain Words: I Fine-Tuned a Small LLM by Training 2% of Its Weights, and Here Is What It Cost
Cover image for LoRA Explained in Plain Words: I Fine-Tuned a Small LLM by Training 2% of Its Weights, and Here Is What It Cost

LoRA Explained in Plain Words: I Fine-Tuned a Small LLM by Training 2% of Its Weights, and Here Is What It Cost

Comments
6 min read