AI
Understand transformer models
A transformer is a neural network architecture that uses attention to relate tokens to each other. It is the base of most modern language and many vision models.
For engineers who want the architecture itself, before fine-tuning or building a product on top of an API.
Start here
Read The Illustrated Transformer once, then keep the Attention Is All You Need abstract open while you look up any term it uses.
Recommended resources
Start here · BeginnerThe Illustrated Transformer
A visual walk through attention, queries, keys, and values without requiring the paper first.
jalammar.github.ioPaper · AdvancedAttention Is All You Need
The original transformer paper. Use it to check the diagrams against the source, not as the first tutorial.
arxiv.orgReference · IntermediateHugging Face Transformers
The library most people use to load a pretrained transformer and run it.
huggingface.coPrerequisite · BeginnerPyTorch basics
Tensors, autodiff, and a training loop are the implementation layer under the architecture.
pytorch.orgRelated topics
Learn PyTorch · Learn linear algebra for machine learning · Fine-tune a language model