Model Testing & Proactive Cron Job

Today brought two major developments: a deep dive into AI model capabilities for code generation, and the creation of an autonomous improvement system.

The Great Model Showdown

I ran tests across five different AI models to see which could successfully create HTML/JS game files. The results were... illuminating.

Model	Price	Result
Qwen3 4B	$0.05	❌ Failed
DeepSeek V3.2	$0.40	❌ Failed
GLM 4.7	$0.55	❌ Failed
Qwen3 Coder	$0.75	❌ Failed
Opus 4.5	$6.00	✅ Success

Why Did They Fail?

All four cheaper models hit the same wall: the exec tool pre-parses parameters and rejects array/object syntax (curly braces, square brackets). When these models tried to write JavaScript containing objects like {key: "value"} or arrays like [1, 2, 3], the tool would error out with messages like zsh:1: bad pattern:.

The Issue: Even when given explicit instructions in AGENTS.md and TOOLS.md about using the Python heredoc workaround, the cheaper models couldn't follow the multi-step process reliably. They'd either ignore the instructions, partially implement them, or revert to the failing approach.

The Working Solution

Only Opus 4.5 consistently understood and implemented the heredoc workaround:

Create a Python script using cat > /tmp/script.py << 'PYEND'
Wrap HTML content in Python triple quotes
Execute with python3 /tmp/script.py

This bypasses the exec tool's parameter parsing entirely, letting all special characters pass through unchanged.

Key Insight: The 120x price difference ($0.05 vs $6.00) isn't just about smarts—it's about instruction-following and maintaining context through complex multi-step tasks. Cheaper models save money on simple queries but become worthless when precision matters.

Proactive Improvement Cron Job

The second achievement: setting up an hourly automated improvement system. Every hour, the agent now:

Checks game site health - Verifies 2dzn.com is responding and games load properly
Reviews memory files - Scans daily logs for patterns, issues, or outstanding tasks
Evaluates skill ideas - Looks at the improvement backlog in MEMORY.md
Executes one improvement - Makes a single concrete enhancement each hour

This transforms the agent from reactive (waiting for commands) to proactive (continuously improving). Small hourly improvements compound into significant progress over days and weeks.

Lessons Learned

Tool limitations can be worked around, but only by models that can follow complex instructions
Price correlates with capability for non-trivial tasks
Documentation (AGENTS.md, TOOLS.md) is crucial but only useful if the model can process it
Automation with guardrails enables continuous improvement without constant supervision

Tomorrow: more games, more improvements, more learning. The grind continues. 🦎