A while back, Google DeepMind unveiled Gemini Diffusion, an experimental language model that generates text using diffusion. Unlike traditional GPT-style models that generate one word at a time, Gemini Diffusion creates whole blocks of text by refining …

I’m doing my master’s thesis around distributed low-communication training. Essentially, how can we train large models efficiently across distributed nodes and not be utterly destroyed by network latency and bandwidth? Below is some of what I’ve learned …

A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News (discussion). This is a short write-up over what I learned about old-school general-purpose GPU programming over the course of this …

My notes over Mark Maxwell’s course, Introduction to Mathematical Statistics, and his textbook, Probability & Statistics with Applications, Second Edition. Sampling Distributions and Estimation: Normally in a probability experiment, we don’t know the true …

An overview of common discrete and continuous distributions found in probability and statistics, from Mark Maxwell’s textbook, Probability & Statistics with Applications, Second Edition. Common Discrete Distributions: Discrete Uniform: A random variable …

Lately, I’ve been coming across many blogs that have weird font-size rendering issues for code blocks on iOS. Basically, in a code snippet, the text-size would sometimes be much larger for some lines than others. Below is a screenshot of the issue from a …

For a while, I wanted to build a complete autograd engine. What is an autograd engine, you might ask? To find the answer, we first must know what a neural network is. Neural Network Crash Course: A neural network can just be seen as a black-box function. …

Recently I was going through Thorsten Ball’s “Writing An Interpreter in Go”. In this book, you create a basic interpreted language and write a lexer, parser, evaluator, and REPL for it. A Lexer takes in source code and turns it into an intermediate …

Below are all the books I’ve read since middle school, roughly in order. Those highlighted in blue were those I particularly enjoyed :) 2025 – Age 22: Willpower - Roy F. Baumeister & John Tierney I was at a ZFellows event in SF where one of the speakers, …

Here are a few of my favorite quotes I’ve liked over the years. Life: “I believe that a man should strive for only one thing in life, and that is to have a touch of greatness” — Félix Martí-Ibáñez “In Three Words, I Can Sum Up Everything I’ve Learned About …