This blog has some stuff I've written over the years - on topics like ML, math, algorithms, pedagogy, low-level programming, Linux, compilers and other things I like to talk about.
For any comments, reach out to me at nor [dot] xor [dot] nor at gmail [dot] com. I am also on Twitter (@norxornor), but might be less responsive there.
To subscribe via RSS, copypaste the link to the RSS page in your favourite RSS reader. Each tag also has its own RSS feed.
Some of my (mostly throwaway) code can be found at nor-git.pages.dev.
A new deterministic, unconditional algorithm for enumerating primes in nearly linear time on a multitape Turing machine, matching Harvey's previously conditional bound relying on a conjecture, with GPT-6 Astra.
A short note on proving optimal packings of 15, 16, 17, 18 and 20 circles in a circle: history, how the proofs were found with an LLM, and the main ideas.
A discussion of MoE load balancing, understanding quantile balancing, and generalizing and improving upon it. We show two improvements - one for improving load balancing/steering, and another for improving loss by making load balancing strength configurable.
An elementary finite-time account of why wide networks trained on the same minibatches develop matching local loss fluctuations, and how width and batch size control initialization, data, and interaction noise, with implications on scaling.
A research note on what breaks in long-context attention, deriving a logit scaling, with QK-norm, hybrid/local attention, gating, and small-scale experiments.
An intuition-building tour of dynamical systems, using canonical examples to connect feedback, thresholds, coupling, noise, reinforcement, spatial structure, and phase transitions.
A writeup of porting the modded nanoGPT speedrun to pure JAX on TPU v6e, including hardware bottlenecks, bugs, optimizations, and open performance questions.