Standalone essay
The Student That Beat Its Teacher
A guide to knowledge distillation, and why an AI copy can sometimes outperform the original.
Writing
Essays and technical notes on machine learning, language models, and the systems around them.
Thinking in Transformers
Chapter 1: Tokenization — Where Language Becomes Discrete
Why tokenization is the first information bottleneck in LLMs, shaping compression, context length, multilingual behavior, and symbolic performance.
Thinking in Transformers
Chapter 0: The Design Space of Language Models
A working model for understanding LLMs as compressed, lossy, differentiable databases of language.