Open Documentation
Fine-Tuning Cookbooks
& Engineering Guides
A growing repository of utility scripts, bug fixes, tutorials, and architectural frameworks to support the community navigating model fine-tuning and agentic deployments.
How to See What a Tiny Transformer Actually Learned
Three ways to see what a tiny character-level transformer learned: watch the rules emerge during training, test whether it generalized or memorized, and ablate the attention head doing the work.
Implementing LoRA from Scratch
Learn exactly how Low-Rank Adaptation works by writing the A and B matrices in vanilla PyTorch, training them, and merging the weights.
Gradient Checkpointing with LoRA: The Detached Tensor Problem
Wondering why your training loop no longer crashes with a detached tensor error? We break down the infamous Issue #137, Hugging Face's silent PEFT patch, and how to fix the computational graph manually in custom architectures.
RAG vs. Fine-Tuning: When to use which
A technical framework for deciding between RAG for dynamic knowledge retrieval and SFT for behavioral logic.
What is the Model Context Protocol?
A technical guide to MCP architecture, how it solves the M x N integration problem, and when to use it for deterministic tool calling.
How to Train Custom Tokens with LoRA
Fix untrained embedding errors and prevent model gibberish when adding custom tokens to a vocabulary during PEFT fine-tuning.