V vision
A convolutional VAE compresses each 64×64 frame into 32 numbers, z. Only its decoder ships to the browser.
… parameters
Agent is drivingstep 0
What is running
M memory
An LSTM with 256 units and a mixture-density head, five Gaussians per latent dimension, predicts zt+1 from zt and the action. τ scales its uncertainty.
… parameters
C controller
One linear layer on [z, h] with 867 parameters, found by CMA-ES in the real environment. It drives the dream the same way it drives the game.
… parameters
This is the recipe from World Models (Ha & Schmidhuber, 2018), re-implemented in PyTorch and trained on my DGX Spark. The models were exported to ONNX and run here with onnxruntime-web. The page is static and never contacts the Spark.
In the real environment
| Loading… |
Walk the latent space
Pick a real frame from a held-out rollout. V encodes it into z and decodes it back. Then push the 32 sliders to see what each direction of z means to the decoder.