Fan Pu Zeng

Hello! I am Fan Pu (曾繁朴). I am currently on a personal research sabbatical. Previously, I spent 3 happy years at Jane Street, where I worked on LLM research and training, and developing on our in-house coding models and evals.

I graduated with a B.S (2022) and M.S (2023) in Computer Science from Carnegie Mellon University.

At CMU, I was actively involved in the open-source programming assignment auto-grading platform Autolab from 2018-2023. I served as the Masters Student Liaison for the Singapore Students Association. I also used to play Capture-The-Flag (CTF) competitions with PPP. I previously interned at Jane Street, Meta, Asana, and Saleswhale (acquired 2022). I was a TA for 10-708 Probabilistic Graphical Models in the Spring 2023 semester.

My academic interests lie in understanding the theoretical foundations for the success of deep learning, particularly in optimization and generalization.

In my free time, I enjoy climbing, running, skiing, reading and learning new things, and writing things for my blog. I used to do sprint canoe competitively. If I have an extended break I enjoy traveling, especially hiking and exploring the great outdoors. Most of the banner pictures on my blog posts were taken during these hikes. My favorite classroom in CMU is GHC 4303.

I grew up in my hometown Singapore before moving to the US for college and work. I try to go back and visit once a year.

Feel free to reach out to me at fzeng[at]alumni[dot]cmu[dot]edu. I am happy to chat and provide advice.

Regrettably, I am unable to provide referrals for people that I have not directly collaborated with, as I cannot write you a meaningful recommendation.

I have a Technician amateur radio license, with callsign KC3UFE.

This blog was originally started on 24 June 2018, although it has taken many forms since then. All banner pictures on the blog are taken by yours truly!

Talks

Slides I developed for talks on various LLM-related topics. You are free to share, adapt, and reuse these materials, provided that you give appropriate credit.

• (2025-09-19) Average Gradient Outer Product as a Mechanism for Feature Learning
• (2025-05-13) Learning Dynamics of LLM Finetuning
• (2025-05-08) DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
• (2025-02-21) Convergence and Generalization of Neural Networks: Neural Tangent Kernels and Beyond
• (2024-11-18) A Statistical Approach to Language Model Evaluations
• (2024-10-08) Advanced Retrieval Augmented Generation Techniques
• (2024-07-24) Superalignment, or how to train models smarter than us
• (2024-05-03) Rotary Positional Embeddings (RoPE)
• (2024-04-30) Parameter-Efficient Fine-Tuning
• (2024-03-01) Understanding Transformers

Starred Blog Posts

Some of my more popular posts:

Technical Posts

• Score-Based Diffusion Models
• Bounding Mixing Times of Markov Chains via the Spectral Gap
• An Intuitive Introduction to Gaussian Processes
• (Paper Summary) Zero-shot Image-to-Image Translation
• (Paper Summary) The Implicit Bias of Gradient Descent on Separable Data
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers: A Guided Walkthrough
• The Delightful Consequences of the Graph Minor Theorem
• Universal types, and your type checker doesn’t suck as much as you think

General

• The Art of LaTeX: Common Mistakes, and Advice for Typesetting Beautiful, Delightful Proofs
• Against Government Scholarships
• Notes On Founding A Startup To My Future Self

CMU

• CMU 15-712 Advanced Operating Systems and Distributed Systems Course Review
• CMU 15-441/641 Computer Networks Course Review
• My Sharing at the Hwa Chong Undergrad Alumni Forum, i.e. why study Computer Science at CMU

Artifacts

Interactive explainers I made with Claude while trying to understand something. A few favorites are below, and the rest are on the artifacts page.

Nov 09, 2026	Does Adam favor a basis? Gradient descent doesn't care which basis your parameters are written in, but Adam does. Why that happens, and how it ties into the outlier features you see in transformers trained with Adam.
Nov 08, 2026	Chaos, and why retraining is never the same An intro to chaos theory (the Lorenz system, Lyapunov exponents, the butterfly effect) starting from basic calculus, and why training a neural net behaves like a chaotic system.
Oct 25, 2026	InfiniBand field guide An intro to InfiniBand, the network that connects GPUs in big training clusters, for people with no networking background. How data gets from one GPU to another, why the network never drops packets, and how that differs from Ethernet.
Oct 03, 2026	Pre-LN vs post-LN gradients How where you put LayerNorm in a transformer changes the gradients at each layer, and why that means post-LN needs learning rate warmup while pre-LN mostly doesn't.

Games

Some games that I wish existed but didn’t, so I made them with Claude Code

• FlyJack (a simulated fruit fly brain playing blackjack)
• Blackjack trainer
• Beginner xiangqi (Chinese chess) trainer
• The Floating Cup (for learning classical Chinese phrases)
• Model Parallelism
• Fermi Estimates