`llm_runner` Part I: Implementing an Encoder in Rust
Recently, I’ve started a journey to “understand how LLMs work”. Implementing something is a good way to understand it, so I’ve started the llm-runner project. To start this project, it was natural to start with the architecture described in the fundamental paper Attention is all you need Vaswani et al., 2017. [Vas17] was not using the transformer architecture in an LLM yet, but it was the first to introduce the attention mechanism and the transformer architecture. ...