Self-Supervised Learning and CollapseArticle

How self-supervised learning collapses, from the obvious to the hidden.

Interactive Stochastic Differential EquationsStudy

Random walks and Brownian motion, stochastic processes and the Markov property, Itô calculus and SDEs, and finally the Fokker–Planck equation and the probability flow ODE.

Interactive Differential EquationsStudy

ODEs and their numerical solvers — direction fields, initial value problems, phase portraits, Euler and Runge–Kutta, and numerical stability — then PDEs: the transport, continuity and diffusion equations.

Interactive CalculusStudy

Functions, limits, derivatives and the chain rule, Taylor expansion and integrals, then partial derivatives, the gradient, Jacobian, Hessian, vector fields, divergence and the Laplacian.

Interactive Information TheoryStudy

Self-information, entropy and coding length, cross-entropy and KL divergence, mutual information, and the losses built on them — cross-entropy, perplexity, the ELBO, and InfoNCE.

Interactive ProbabilityStudy

Random variables, joint and conditional distributions, expectation, the distributions machine learning actually uses, and the road from likelihood to NLL, Bayes' rule, and MAP.

Interactive Linear AlgebraStudy

Vectors, inner products, eigenvalues, the four fundamental subspaces, change of basis, and PyTorch einsum/permute/view/reshape.