← Be the Model

Be the model: train the pond machine

619 parameters (152 embedding numbers, 432 weights, 35 biases, the biases starting at 0), 45 training examples. Measure, get the gradient, take a step, repeat. Skip the first two and the step does them itself.
average loss
0.000
an average loss of 2.944 is what guessing every word equally would score
examples where the guess matches the target
0 of 45
training steps taken
0
run all 45 training examples and average the losses
the trainer works out every slope by backpropagation
nudge every parameter: nudge = - step size × slope (measures the gradient first if you skipped it)
repeat the cycle
back to the same untrained machine
0.30
0.01 to 5.00
no gradient measured yet

Average loss against training steps

Six parameters you can nudge by hand

parameter (drag the slider under it)valueslopeuseful way
average loss now since you started this drag: no drag yet

The 45 training examples: input → target, each with its own loss

What the machine guesses now

inputtop guesssharemost common follower in the textguess agrees?