Ralphthon SF, sponsored by OpenAI

Built solo at Ralphthon SF, sponsored by OpenAI and hosted at Weights & Biases on March 28, 2026. The rule: laptops close at 12:30, agents code alone for three hours, and anyone who touches a keyboard wears the lobster costume. The top three went to other teams.

The problem I chose

Agent coding runs fail in boring ways. The task is too broad, the agent grades its own work, failed attempts leave no trail, and “done” means the agent said so. With hands off the keyboard for three hours, every one of those compounds.

What shipped

A Ralph Loop harness. It pulls one feature at a time from a 30-item JSON spec, has Codex build it, runs a build gate, then hands the diff to a separate Claude evaluator. Below the bar, the evaluator writes feedback naming the file and the fix, and the builder gets up to three tries. Only the harness can commit, mark a pass, push or skip. A guard script stops the agent from editing its own task list.

The app it built was a dashboard of its own build: timeline, quality scores, tokens, git history, architecture. 29 of 30 features passed, 21 on the first try, in under 80 minutes and under a dollar of estimated tokens. The demo video shows the run.

AgentForge later became Nightshift, a lights-out harness for scoped GitHub tasks.

Toolset

• Codex CLI (gpt-5.3-codex): the builder, one feature per run.
• Claude Sonnet 4: the separate evaluator, scoring completeness, visual quality and leftover placeholders.
• Bash and Python: the loop, the evaluator call, the JSON guard and the metrics writer.
• W&B Weave: wired in for evaluator traces.
• Next.js, React, Tailwind and Recharts: the dashboard the agents built.
• Vercel: a deploy on every push.

What was hard

• My first harness erased the agent’s work. On a failed build it ran git checkout . && git clean -fd. I replaced that with feeding the build errors back to the builder.
• The build gate was thinner than it looked. With node_modules missing, npm run build passed on an offline scaffold check, so a full Next.js compile never ran inside the loop.
• Runtime checks never fired. The dev server was down at every planning pass, and the localhost gate only warned.
• One feature never landed. A dark-mode toggle failed three times and was skipped.
• The dashboard was built against empty data. Its charts were only checked in their empty state.

What I learned

• Agents can’t grade their own work. A separate evaluator is most of the trick.
• The harness decides what ships, not the agent.
• Feed errors back instead of reverting. A reset throws away the context the next attempt needs.
• Feedback has to name the file and the fix. Vague notes get vague retries.
• Small, JSON-specified, guarded tasks with stop rules beat one big prompt.