Open Documentation
Fine-Tuning Cookbooks
& Engineering Guides
A growing repository of utility scripts, bug fixes, tutorials, and architectural frameworks to support the community navigating model fine-tuning and agentic deployments.
RAG vs SFT: When to Use Which | SR Cookbooks
A technical breakdown of when to use Retrieval-Augmented Generation (knowledge) versus Supervised Fine-Tuning (behavior) in enterprise AI pipelines.
Self-Hosting an Open-Weight Coding Agent on AWS | SR Cookbooks
A complete guide to self-hosting open coding models on AWS EC2 with vLLM, connecting VS Code Continue, running VPC-isolated MCP tools, and analyzing team infrastructure costs.
The Math of DPO, Explained | SR Cookbooks
A step-by-step mathematical derivation of Direct Preference Optimization (DPO), showing how the partition function cancels and verifying the implicit reward gradient in Python.
Building Local MCP Servers for VS Code & Cursor with FastMCP | SR Cookbooks
A step-by-step technical guide to building Python MCP servers with FastMCP, configuring VS Code and Cursor mcp.json, and debugging stdio JSON-RPC streams.
Speculative Decoding From Scratch | SR Cookbooks
Implement speculative decoding from scratch using NumPy to verify losslessness via rejection sampling and calculate wall-clock speedup bounds.
Power Analysis for Benchmark Design | SR Cookbooks
Calculate statistical power and minimum detectable accuracy gaps for LLM benchmarks using Python to avoid reporting sampling noise as signal.
Deduplication at Scale: MinHash & LSH in Python | SR Cookbooks
Implement MinHash and Locality-Sensitive Hashing (LSH) in pure Python to eliminate the quadratic bottleneck of near-duplicate detection for ML dataset curation.
Fixing EOS Errors: Why Fine-Tuned Models Talk to Themselves | SR Cookbooks
A technical guide to fixing the infinite generation bug in SFT by properly mapping EOS tokens and chat templates in Hugging Face.
What is the Model Context Protocol (MCP)? | SR Cookbooks
A technical architecture guide to the Model Context Protocol (MCP). Learn how it solves the M x N integration problem for AI agents and tool calling.
PEFT Explained: LoRA vs. QLoRA | SR Cookbooks
Understand the architectural differences between LoRA and QLoRA, and learn when to use each Parameter-Efficient Fine-Tuning technique based on your VRAM limits.
How to Train Custom Tokens with LoRA | SR Cookbooks
Learn how to fix untrained embedding errors when adding custom tokens to an LLM vocabulary during PEFT and LoRA fine-tuning.
Catastrophic Forgetting in Fine-Tuning | SR Cookbooks
What catastrophic forgetting is, why it happens when you fine-tune an LLM, what it costs you, and how to spot it, plus when a specialized small model can safely ignore it.
Inspecting What a Tiny Transformer Actually Learned | SR Cookbooks
A technical guide to probing a character-level PyTorch Transformer. Learn how to measure rule acquisition, test generalization, and ablate attention heads.
How to Check a Fine-Tuning Dataset Before You Train | SR Cookbooks
Avoid common fine-tuning failures. Learn how to validate chat templates, prevent silent truncation, verify loss masking, and check for data leakage before spending GPU hours.
Implementing LoRA from Scratch in PyTorch | SR Cookbooks
Learn how to write a custom LoRA wrapper in pure PyTorch. Freeze a base model, train low-rank A and B matrices, and merge the weights without using PEFT.
LoRA & Gradient Checkpointing: The Phantom Bug | SR Cookbooks
Wondering why combining LoRA with gradient checkpointing no longer crashes your script? A deep dive into PyTorch detached tensors, Hugging Face PEFT fixes, and manual graph attachment.
Knowledge Distillation from Scratch: Pure PyTorch | SR Cookbooks
Learn the underlying math of Knowledge Distillation. Move beyond standard SFT by implementing temperature scaling and Kullback-Leibler divergence from scratch in PyTorch.
Implementing a KV Cache From Scratch: Pure PyTorch | SR Cookbooks
Learn the mechanics of autoregressive LLM optimization. Implement a transformer Key-Value (KV) cache from scratch in PyTorch to massively speed up decoding.
Tokenizers From Scratch: BPE vs MaxMatch | SR Cookbooks
Stop treating tokenization as a black box. Learn how Byte Pair Encoding (BPE) and MaxMatch split text differently, and what mathematically happens when you add new tokens for fine-tuning.