AI Accountability & Governance

Why AI Accountability Requires Reconstructable Reasoning

AI systems can often explain their outputs. Accountability requires more than explanation.

As AI systems get deployed in public and institutional contexts, questions of accountability move to the center. Regulatory frameworks, including the EU AI Act, emphasize transparency, documentation, and explainability — all genuinely important steps. They may not, on their own, be sufficient.

The Limit of Explanation

Most approaches to AI accountability rely on explaining outputs after they've already been produced — summarizing model behavior, highlighting influential inputs, generating human-readable justifications. But these explanations are typically generated without access to the full sequence of reasoning that actually led to the outcome. They describe results. They don't reconstruct how those results emerged.

From Explanation to Reconstruction

Real accountability requires more than describing an outcome — it requires being able to reconstruct the reasoning trajectory that produced it: the assumptions introduced, the interpretations made, the intermediate steps taken, the intentions that were actually guiding the process. Without that, explanations risk becoming post-hoc interpretations rather than genuine grounds for critique.

Responsibility and Boundaries

In systems where humans and AI collaborate, responsibility is distributed across the interaction itself — decisions emerge through a sequence of contributions, from prompt to model response to human interpretation and refinement. Assigning responsibility means being able to trace that sequence. This research programme formalizes that through Responsibility Boundary: defined points at which reasoning can actually be inspected, evaluated, and attributed to whoever is accountable for it.

Traceable State

Reconstruction becomes possible once system states retain an actual trace of the reasoning that produced them — the principle behind Traceable State. Each state carries intention, justification, and a reference back to the reasoning that came before it, carried forward through PIFRs, which function as structured reasoning artifacts within the system rather than disposable byproducts of it.

Accountability as Reconstruction

When reasoning can actually be reconstructed, accountability stops being aspirational and becomes grounded: it becomes possible to see how a decision emerged, identify exactly where an assumption entered, and challenge specific steps in the reasoning rather than only the final result. What matters isn't only what the system produced — it's whether the path to that result can actually be followed by someone else.

This is what accountable reasoning actually means, stated precisely rather than aspirationally: reasoning that stays visible, stays attributable to someone specific, and — through Audit & Repair — stays correctable when it turns out to be wrong. Remove any one of the three and what's left isn't accountability, it's a partial version wearing the name.

Why It Matters Now

As AI systems get folded into governance, healthcare, and economic decision-making, the cost of untraceable reasoning keeps rising. Without reconstructable reasoning, decisions can't be meaningfully audited, responsibility stays diffuse, and trust ends up resting on authority rather than actual understanding. With it, decisions become inspectable, responsibility becomes attributable to someone specific, and systems can genuinely improve over time rather than just iterate. This connects directly to the broader structural argument made in EU AI Act: Structural Alignment.

Explainability makes AI outputs interpretable. Reconstructability makes them accountable.

— Arne Mayoh

Part of the Audited State Research essays.