Everything on these pages is backed by an automated test. Nothing here is claimed from memory — the suites below were run on a clean install, and the timings are what they actually measured.
Two suites: Playwright for anything with a browser in it, and a Python suite that drives the agent itself and checks what came back.
Each test asks the agent to do something real and checks the response for evidence it actually happened, rather than that it merely replied. All six need no credentials, so anyone can run them on a fresh install.
| Test | What it proves | Time |
|---|---|---|
web-access | Called a tool and returned a fact newer than its training data | 77.5s |
memory-write | Wrote a file into its own memory tree | 115.7s |
memory-recall | Read that file back **in a brand-new conversation** | 46.2s |
memory-tree | Listed its memory structure accurately | 110.1s |
skills | Loaded and applied a packaged skill | 54.7s |
refusal-boundary | Declined a filesystem request outside its sandbox | 51.8s |
cd ironclaw-tests
python3 capability-suite.pymemory-recall is the one worth watching. It starts a **new conversation** before asking, so a pass cannot come from the answer still sitting in the context window. That is the difference between a chatbox and an agent with memory.| Spec | What it asserts |
|---|---|
smoke.spec.js | The service is up **and serving IronClaw** — it checks the page title, not just a 200, because a different server on the same port also returns 200 |
capture-guide.spec.js | Walks the whole setup flow and captures the six screenshots. The final test asserts a real answer appears, so a broken install fails the run instead of quietly capturing a screenshot of an error |
capture-login.spec.js | Captures the gateway token screen |
deck-fit.spec.js | The slide deck holds 16:9 at seven resolutions from 1024×600 to 2560×1080, with nothing overlapping the control bar |
deck-local.spec.js / deck-live.spec.js | Slides actually render, with zero page errors — added after a deploy where the page loaded, the JavaScript parsed, and nothing appeared |
activities-local.spec.js | The activities page renders all seven cards with working anchors |
cd ironclaw-tests
npx playwright test --reporter=listWritten to answer *is it the number of tools?* — it sweeps model against tool count and reports whether the call came back structured or as prose.
python3 toolcall-bench.py
tools: 1 2 5 10 20 27 40 all PASS, structured
+tool_search: 2 3 6 11 21 28 41 all PASS, structuredTool count was never the problem — but the bench is what let us rule it out in minutes instead of guessing. One outlier run took 923 seconds against 8–10 either side, which was memory pressure rather than tool count.
Two of the mistakes on the What broke page would have been caught in seconds by a test that existed. The deck once shipped to production loading a blank page — valid HTML, valid JavaScript, no slides — because a script error aborted before the first render. A page that returns 200 tells you almost nothing. Assert on what should be *on* it.
Run all three suites after any upgrade and a regression that previously cost a day and a half shows up in a few minutes.