Arcana AI can write and run Python to answer hard analytical questions. But for a hedge fund, the hard part was never the code — it was trust. No one acts on a number they can’t verify. Code Mode is the design layer that makes an AI-computed answer auditable enough to act on.
The 60-second version. Arcana AI runs Python when math is more reliable than an LLM “reasoning” about numbers. Engineering had the execution working; what was missing was any reason for a skeptical analyst to believe the result. I designed a computation receipt — a status chip that expands into what data was used, what was checked, and what didn’t reconcile — plus honest caveat and failure states. After a dev review, I took it further: a multi-step reasoning timeline, and an edit-and-rerun model where validation re-runs with every change. Everything below is the long version.
Self-contained, single-file HTML built on the live Arcana AI layout with the product’s real design tokens. Use the switcher inside the prototype to move between states; in v2, edit the code and rerun to watch validation re-bind. (Shown full-width — even better full-screen.)
The Code Mode prototype is a full desktop analytics interface — it needs a wider screen to be usable.
Some questions are best answered by running code, not by asking a language model to do arithmetic in its head. “Take my last six months of returns, find the 10 worst drawdown days, and show which positions drove each one” is a join, a sort, and a reconciliation — deterministic work an LLM should never freehand.
So Code Mode generates and runs Python against Arcana’s data, then feeds the validated output back into the answer. By the time I joined the problem, engineering had the mechanics working: a collapsed block showing code and its output. The feature was technically real.
It also had a trust problem that no amount of engineering would fix on its own. The users are portfolio managers and analysts at hedge funds — many of them not technical. When someone like that sees generated code, gets a number back, and doesn’t understand how it got there, one of two things happens: they take it on faith (dangerous), or they stop using the feature (worse). As one engineer put it in review, “users who lose trust early are unlikely to return.” And the stakes are asymmetric — in a financial workflow, a confidently wrong number that someone trades on is far more costly than no answer at all.
The existing direction answered “how do we show code and output?” The more useful question was: how do we show reasoning, data, checks, and limits in one coherent flow — so a non-technical user can trust the answer, and a technical one can audit it?
That reframe is the spine of the whole project. The code block stops being the feature. The feature becomes the evidence around the answer: where the numbers came from, what was validated, what didn’t reconcile, and what to be careful about — surfaced by default, with the code itself one layer deeper for the people who want it.
The organizing idea: however messy the AI’s reasoning gets — multiple steps, failed attempts, retries — the interface should always resolve to three stable layers. This model is what kept the design coherent as the feature grew.
The reasoning timeline — ephemeral. It holds the plan, the data pulls, and any retries (“Attempt 1 failed, refined, re-ran”). Narrates how the answer was reached.
The canonical, copyable, auditable code block — durable. Later, editable and re-runnable. In a financial tool, this is the trust anchor; it never dissolves into the transcript.
The plain-language answer, headed by a status chip that expands into the validation receipt — bound to a specific run of the code.
A non-technical PM should never be forced past the answer. A quant should be able to get all the way to the code in a few clicks. So the default view is calm — answer plus a one-line status chip — and every layer of depth is exactly one expand away: answer → plain-English method → validation receipt → code → full reasoning with attempts.
The hardest calls were about honesty. It would have been easy to hide the messy parts. Instead, caveats and failures are first-class:
The work moved in four passes — each one a different mode of design: untangling a spec, prototyping the core idea, absorbing a hard critique, then pushing the concept forward.
For anyone who wants past the summary — the three design problems that took the most thought.
Trust comes from traceability, not from a green checkmark. The receipt is the answer to “why should I believe this?” — a fixed structure so an analyst always knows where to look. It lives behind the status chip, so it’s there when wanted and invisible when not.
The code itself and the full tolerance formula sit one layer deeper, behind “View code” and “Methodology.”
When users can edit and re-run code, validation stops being a description and becomes a contract bound to a specific run. The most dangerous moment in the whole feature is the one in between: a user edits the code, but the green “passed” chip from the previous run is still showing. That’s false confidence — the exact thing the feature exists to prevent.
So I designed that in-between state first. The instant code is touched, the receipt grays out and the answer is marked stale: “You’ve edited the code — rerun to revalidate.” Re-running produces a new run with its own receipt, and if the result changed, a “What changed” diff explains how (“you widened the tolerance from 5bps to 10bps; the two flagged dates now reconcile; the caveat is cleared”). A run-history control lets you compare or revert. Validation becomes live — it tells the truth about the code in front of you, not the code from a minute ago.
The review surfaced a genuine fork, owned jointly with the eng lead: as the product moves toward multi-step reasoning “like Claude,” does the code live inside the thought process, or stay a separate object?
My position: both, with a clear division of labor. The thinking timeline narrates process — including failed attempts, phrased as refinement rather than raw stack traces so they reassure rather than alarm. But the final, successful code graduates out into the durable code block. In a financial tool, the canonical, copyable, auditable artifact is the trust anchor; dissolving it into a scrolling transcript just because a general-purpose chatbot does would cost more than it gains. That framing turned an open-ended “how should this look” into a concrete decision engineering and the eng lead could build against.
Code Mode hasn’t shipped to clients, so there’s no adoption data yet. Two things matter here instead: what the design work has already produced, and the measurement plan that ships with it — because designing the instrumentation is part of the job, not an afterthought.
One guardrail I pushed for: because the users are hedge funds, no holdings, values, or tickers go into analytics payloads — magnitudes are bucketed, identities are hashed. Worth getting right before the events go in, not after.
Code Mode sits inside a broader theme in my Arcana work: making dense, high-stakes, increasingly AI-driven information legible enough to act on. Same discipline as the RMS and research work — just pointed at the question of whether you can trust what the machine just told you.