Neural Math LabUNDERSTAND THE LEARNING

A small network. A deeper understanding.

Don’t just watch it learn.
Understand why.

Follow a prediction from input to output. Replay a learning experiment. Then change a weight and see what the mathematics predicts.

Start learning New to networks? Start with XOR
  • No account needed
  • Calculations in your browser
  • No telemetry
FROM AN INPUT TO A PREDICTIONx → h → p
A feedforward neural network Two input values connect to four hidden neurons, which connect to one sigmoid output. Lines represent learned weights. x₁x₂h₁h₂h₃h₄p
FORWARDp = sigmoid(tanh(xW₁ + b₁)W₂ + b₂)
Every connection is a number you can inspect. Illustration shown; the labs use recorded model parameters.

01 / EXPERIMENTS

Three ways to see the idea.

Start with a small truth table, then explore patterns that need a curved boundary.

THE FOUNDATION01

Four points. One useful lesson.

Meet XOR. Follow the actual weights, activations, and outputs of a tiny network, with the chain rule close at hand.

Explore XOR Recorded introduction · training truth table
START EXPLORING02

A boundary that bends.

Replay the moons experiment. Inspect predictions, compare a linear baseline, and take your first temporary gradient step.

Explore moons Interactive · replay, decisions & gradients
TEST YOUR INTUITION03

When a straight line won’t do.

Explore concentric classes and see what hidden neurons add. Check whether a better fit carries over to held-out examples.

Explore circles Interactive · replay, decisions & gradients

THE GRADIENT LAB

Predict a step.
Then test your prediction.

Choose any weight or bias. Compare backpropagation with a numerical derivative, inspect the loss curve, and try a step. Overshot? Undo it and reduce the learning rate.

Open the gradient lab

θ′ = θ − η∇J

  1. Inspect the derivative and its numerical check.
  2. Compare the predicted and actual loss change.
  3. Experiment with one parameter or the full network.

Up to 50 temporary steps, with undo and reset. Saved models and evaluation results stay fixed.

02 / FOLLOW THE MATHEMATICS

A derivation beside every experiment.

Read the explanation, make a prediction, and check your reasoning against the worked answers.