Definition
A transformer is a model architecture used to process sequences using mechanisms such as attention and feedforward neural networks.
It is important to distinguish a transformer from an LLM:
- An LLM describes the language-modeling product or system trained to perform language tasks such as next-token prediction.
- A transformer describes a particular architecture used to build such models.
A transformer can contain multiple layers. Each layer can process representations through attention and feedforward components, progressively capturing more complex relationships in the input.
The transformer architecture became a foundational technology for modern language models.