BERT is just a Single Text Diffusion Step
October 20, 2025A while back, Google DeepMind unveiled Gemini Diffusion, an experimental language model that generates text using diffusion. …
October 20, 2025Local SGD and DiLoCo Research Musings
October 14, 2025Originally, I was doing my master’s thesis on this topic. I eventually changed my focus, but these are some of the notes I …
Running GPT-2 in WebGL with Classic GPGPU Programming
May 24, 2025A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News …
Intro to Autograd Engines: Karpathy's Micrograd in Go
November 11, 2023For a while, I wanted to build a complete autograd engine. What is an autograd engine, you might ask? To find the answer, we …