Gemini 4 Argon SF Hackathon, Google DeepMind × Cerebral Valley

Built solo at the Gemini 4 Argon SF Hackathon, hosted by Google DeepMind and Cerebral Valley at SHACK15 in San Francisco on October 25, 2026. I took Problem Statement Two, More Dimensions: render what you made, watch it, critique it against the goal, fix it in a loop. 128 projects were submitted. Six made the final. Mine didn’t.

The problem I chose

You’re bringing up a two-arm robot station on a table you’ve never used. Before anything moves on the real arms, you want to try the task in sim.

Every new table means a new sim. Measure the table. Measure the objects. Write the scene file. Tune the grasp. Days, every time. So the step gets skipped, and untested motions run on real hardware.

ArgonTwin starts from a 10-second phone video instead.

What shipped

• The twin. 52 frames from a 10.4-second iPhone walkthrough of my desk. COLMAP solved all 52 camera poses, and a Gaussian splat trained on a Colab T4 in 10 seconds. Cropped to the table, 17,157 Gaussians ship. The screens and laptop became five collision boxes, with plane fits between 1.6 and 3.7 mm RMS.
• The rehearsal. Two ALOHA arms in MuJoCo pick up the actual Celsius can from my desk and hand it over. The can’s own 943 Gaussians ride on the simulated can.
• The critic. On a fail, Gemini 4 Argon gets 20 rendered keyframes from four cameras plus the measured numbers, about 27,000 tokens in one call. It explains the failure, cites the frames it used and returns a typed plan patch.
• The referee. Pass or fail is never the model’s call. MuJoCo’s measurements against fixed thresholds decide.

The demo run: attempt 1 put the hand-over point inside the screens. The arm hit them at 216 N and the can fell off the table. Argon read it in 12.2 seconds and moved the hand-over. Attempt 2 passed, 5 mm from target, zero contact. One click reproduced it live: same fail, a fresh repair, a pass at 4.2 mm.

Five live repairs. All five went from fail to pass.

Toolset

• Gemini 4 Argon: the critic. Frames plus metrics in, a validated plan patch out.
• Antigravity managed agent: a 20-trial robustness sweep in Google’s sandbox, accepted only after proof-of-execution checks and a local MuJoCo re-check. 20 of 20 passed.
• Gemma 4 E4B, fine-tuned: a small local repair model. Trained on 36 MuJoCo-verified repairs, it went from 4 of 12 held-out cases to 11 of 12. No cloud call.
• MuJoCo and MuJoCo Menagerie: the physics referee and the ALOHA arms. Same inputs, same result, every time.
• COLMAP, gsplat and Colab: phone video to Gaussian splat.
• Three.js: the viewer. The public demo replays a recorded run. Live runs stay on my laptop, so no keys and no model calls on the web.

What was hard

• A splat isn’t a collision surface. MuJoCo can’t push an arm against a cloud of Gaussians. I picked the screen corners by hand in one frame and fit planes to them. Surfaces the video never saw are still a guess.
• Grasping cheats a little. Once a gripper closes, a weld holds the can instead of fingertip friction. The viewer flags it on every attempt.
• The one-pass world build didn’t land. The plan was for Argon to watch the whole video and write the scene in one long call. In my calls the Interactions API rejected image input for Argon, and no world build ran end to end. Every repair so far starts from a hand-written scene.

What I learned

• The model proposes. Physics decides. A critic is only useful when something it can’t argue with keeps score.
• Give the critic numbers, not just pictures. Frames plus measured newtons and millimeters is what let Argon tie a failure to one plan parameter.
• Keep live and replay apart. The hosted demo plays a recorded run, labeled as recorded.
• Receipts over claims. Every number on this page has a run ID behind it in the repo.