![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dmm"This is a one-file Rust implementation of Llama2 that works pretty well. It's Rust port of Karpathy's llama2.c"
"I don't actually know Rust, but it seems good. I had gpt translate the code, and then I fixed the compiler errors until it ran fast. Would love a code-review if anyone has time."
"Generally you get a lot of memory safety things for free in the conversion. One unsafe part is memory mapping to load in the model, my code is a bit sloppy (but llama2.c is way grosser)."
"One part that is cool is that Rust data parallel (rayon) forces you to protect against double writes with the borrow checker. This fixed one of my bugs in parallel multiheaded attention. (We're not all karpathy-level)"
"I don't actually know Rust, but it seems good. I had gpt translate the code, and then I fixed the compiler errors until it ran fast. Would love a code-review if anyone has time."
"Generally you get a lot of memory safety things for free in the conversion. One unsafe part is memory mapping to load in the model, my code is a bit sloppy (but llama2.c is way grosser)."
"One part that is cool is that Rust data parallel (rayon) forces you to protect against double writes with the borrow checker. This fixed one of my bugs in parallel multiheaded attention. (We're not all karpathy-level)"
