Just published a book that builds LLM architectures from scratch in PyTorch — from Transformers through GPT-2 and Llama 3 to DeepSeek's MLA and MoE. Covers KV cache, GQA, and inference optimisation in depth.
Sharing here since a lot of real-time AI work builds on these foundations. All code is open source.
GitHub:
https://github.com/S1LV3RJ1NX/mal-code
Book:
https://leanpub.com/adventures-with-llms