Professional Services AI Now Has to Show Its Work

What Happens When AI Work Product Carries Your Signature

A benchmark published last week handed AI agents three ordinary commercial jobs, building a website, designing merchandise, and producing a pitch deck, then gave them a choice on every asset: public-domain material they could legally use, or copyrighted material they could not. Measured against a human baseline, the agents kept reaching for the copyrighted files. For open-weight models, violation rates climbed when the prompts simulated user preferences and time pressure.

Copyright-Bench, from Zheng Hui, Doni Bloomfield and Noam Kolt, looks like a copyright paper. It is closer to a study of whether an agent applies a rule it already has in hand. That distinction is invisible in a demo and expensive in a deliverable.

It landed in the same week a law firm decided to bet money on the opposite assumption. Crosby, which bills itself as a NewMod law firm, says it is taking out professional liability insurance covering legal work its agents deliver autonomously, treating them roughly the way a firm treats an associate. "Today, our lawyers review every single work output," CEO Ryan Daniels said. "As agents have improved in leaps over the last few months, it's become clear this won't be necessary in the future." The insurer has not been named.

Both things are true at once. Agents are being handed work that carries a signature, and the research measuring them keeps landing on the same result: giving an agent the correct source material does not mean the agent uses it. In professional services, where the product is a document somebody signs and the firm carries the liability, that is a reproducibility question rather than an abstract governance one.

Handed the Law, and Still Wrong

A second paper, posted three days after the first, tested that question head-on. A team led by Moniruzzaman Mahadi fine-tuned three sizes of a small open model on Bangladeshi statutory law, then sat them for Bangladesh Bar Council exams in Bangla and machine-translated English, scoring only answers that held consistent across repeated seeded runs.

The smallest model improved the most, its English score on the 2022 exam climbing from 2 out of 100 to 34. Read that as a warning about where the benefit lands rather than as a leaderboard result: feeding the model relevant law did the most good precisely where the model started with the least, which is the opposite of how most firms shop for capability.

The largest model showed no detectable net gain at all. Its Bangla answers improved while several English conditions regressed, and the two cancelled each other out. Something else did improve cleanly at every size: answers that wandered out of Bangla and into mostly English almost vanished. The models plainly learned. What they did not reliably learn was to apply the law sitting in front of them.

Knowing a rule and applying it are separate capabilities, and only the second one ever shows up in a signed document. Copyright-Bench isolates the same variable under a condition that should make professional services uncomfortable, because violation rates rose under simulated time pressure. Deadlines are not an edge case in accounting, law, or advisory work. They are the operating condition. A firm that validated an agent on unhurried test cases during a quiet procurement cycle has not tested it during close, or the night before a filing deadline.

That is fixable this quarter, because it is an evaluation design choice rather than a model choice. Accuracy scoring on relaxed test sets will never surface a regression that only appears when the clock is running, which is why agent evaluation has to measure more than accuracy.

Testing whether an agent used what you gave it also requires knowing what you gave it. Agent Context Manager governs which context documents an agent reads on a run and pins that set, so a reviewer can see later exactly what was in front of the model at the moment it produced the artifact. Without that record, "did it use the source material" stops being answerable after the fact, and you are left comparing an output against a memory of its inputs.

Two Runs of the Same Property, Two Different Answers

Bola Akinsanya, founder of Unlevered, described what this looks like on a signed document in an Accounting Today column on 24 July. Her niche is cost segregation, the engineering-heavy studies that break a building into components so owners can depreciate them on a faster schedule. Run the same property through an AI tool twice, she writes, and it can hand back two different answers. Cheap studies are arriving from new artificial intelligence service providers that amount to one person, a cloud account and strong SEO, priced at a few hundred dollars, with internal logic nobody can reproduce. Nothing in them confirms the client even owns what is being depreciated.

Her sharpest line is the one the rest of this story hangs on: "Plausible variation is fine in marketing copy, but it's a liability on a signed tax return."

Non-determinism is a feature when you are generating options and a defect when you are attesting to a conclusion. A cost segregation study has to survive someone else re-running the reasoning months later against the original facts. Reading well is beside the point.

Akinsanya also brings a number from her previous life. Building verification systems at Airbnb for host claims about permits and compliance, she found only about 40% of those claims held up. That figure is the exposure in miniature: an agent that faithfully processes unverified inputs returns a confident, well-formatted, wrong answer, and the signature at the bottom belongs to a person with a license.

Her prescription runs against the instinct to put the model at the center. Keep the core calculation in a deterministic engine with human-written, testable rules that return the same answer given the same facts tomorrow, and let AI work the periphery, finding and verifying the facts that feed it. This is one practitioner writing about one niche, and she says so; it is not a sector-wide measurement. It is also the clearest available picture of what unverifiable output looks like once it lands on a signed return.

What fixes "run it twice, get two answers" is a record of the run. AgentPMT Audit Logs capture every action an agent takes at runtime, filterable down to the full request and response, so a study produced in July can be reconstructed in March when an examiner asks how a number was derived. That is the difference between a pile of prompts and a queryable record of what the agent actually did.

The workflow shape Akinsanya prescribes, an agent checking an artifact against a source of truth instead of recalling a rule from memory, is also the shape the research supports. AgentPMT's AI Contract Redline workflow runs that pattern in a neighboring vertical, comparing a signed document against the original and reporting what moved. More generally, tool design carries more of an agent's reliability than prompt engineering does.

What an Underwriter Is Being Asked to Price

Back to Crosby. Whatever you make of the bet, buying insurance against agent work product is an accountability mechanism rather than a dodge. An underwriter has to price it, and pricing it means somebody is finally putting a number on the risk of autonomous professional work instead of arguing about it on panels.

Insurance pays after a failure. It does not establish what happened. If a client disputes a filing an agent produced, the firm still has to show which agent ran, on which inputs, with which tools, and who approved what. A firm that buys the coverage without building the record is paying to cover a dispute it could have settled with its own evidence.

The architectural version of this argument turned up in an arXiv paper from Dariusz Nowak-Nova on 27 July, on retrieval-augmented models for regulatory knowledge management. Its claim is that pairing a model with a controlled retrieval path introduces auditability, controlled knowledge management, and dynamic updating of regulatory information without retraining. The findings are qualitative, on locally deployed Polish-language models running on consumer hardware, so nothing in it settles how the approach performs at scale. The design argument stands on its own: traceability is a property you get from how the retrieval path is built, not a policy you announce.

Crosby's live decision is narrower than "should agents work autonomously." It is which outputs still route to a human, and that is a choice a firm can make deliberately instead of by default. AgentPMT Send Human Request is the primitive for that gate: an agent calls it when an output needs a person, and you decide in the workflow builder where the gate sits and what it holds. Getting that design right is its own discipline, and approval fatigue does more damage than most teams expect.

For the outputs that do not route to a human, the record has to stand by itself.

A law firm insuring agent output while researchers measure where agents drift is a field working out its rules in public, which is roughly what a young profession-facing technology is supposed to look like while the norms get set.

The Disclosure Question Clients Are Already Asking

Clients are not waiting for the profession to sort this out. A Karbon survey reported by Chris Gaetano in Accounting Today on 27 July found that clients overwhelmingly want to be told when AI touched their work, with a large share asking for complete transparency about it. The same respondents expect the savings to reach them: if AI halves the time a project takes, fees should hold flat or come down. Karbon sells practice management software and the article discloses no sample size, so the numbers point a direction without proving a magnitude.

Read those two findings together and they turn out to be one finding. The clients who want to know when AI did the work are the same clients who expect the discount for it, which quietly makes a firm's AI disclosure policy its pricing policy, whether or not anyone wrote it down that way.

The supply side cannot answer either question with evidence yet. In Publicis Sapient's survey of enterprise AI decision-makers, UK respondents reported AI already running across most of their processes, while just 19% called their AI environment fully integrated. Usage is near-universal; provable integration is not. Firms have spent years buying professional services automation software, layering business process automation services underneath it, and building out artificial intelligence consulting practices, and most still cannot produce a per-engagement account of what the machines did.

A firm that cannot describe how AI touched a deliverable cannot answer the disclosure question its clients are already asking, and cannot defend its rate when a client asks why the fee did not move. Per-run records are what turn an AI-use disclosure from a promise into a statement of fact, and most professional services automation tools were never built to produce them. The wider version of this, the distance between AI adoption and demonstrable return, has been the defining gripe of the last two years, and professional services is about to get a strict, client-enforced version of it.

Deciding Which Outputs Still Need a Signature

The design decision in front of every accounting, law, and advisory practice is narrow enough to settle before the next busy season: which outputs still require a human signature, and what record exists for the ones that do not.

Each piece is buildable today. Evaluate agents under realistic time pressure rather than on unhurried test cases, because pressure is where the research says compliance slips. Keep a per-run record of inputs, actions, and approvals, because a deliverable nobody can reconstruct is a deliverable nobody can defend. Put deliberate escalation gates where a person still signs, instead of defaulting to reviewing everything or reviewing nothing. None of that waits on better models, and none of it is really an artificial intelligence business strategy exercise; it is engineering a practice lead can start on Monday.

Crosby's expectation that per-output review becomes unnecessary and the research finding that agents deviate under pressure are both live claims, and evidence will settle them. That evidence is a byproduct of instrumentation. Build the record now and you will know which one was right well before your underwriter does.

Sources

• Agentic Evaluation of Copyright Law Compliance, arXiv (Zheng Hui, Doni Bloomfield, Noam Kolt)
• Do Small Models Use the Law You Give Them? Context-Injected Fine-Tuning for Legal QA in Bangladesh, arXiv (Moniruzzaman Mahadi et al.)
• Retrieval-Augmented Large Language Models as Components of Cognitive Computing Architecture for Regulatory Knowledge Management, arXiv (Dariusz Nowak-Nova)
• AI slop is flooding the cost segregation industry, Accounting Today (Bola Akinsanya)
• Crosby to Insure Its Agents for Legal Liability, Artificial Lawyer
• Clients value human relationships over AI efficiencies, Accounting Today (Chris Gaetano)
• Nine-in-ten UK businesses regularly use AI in work, Consultancy.uk (Publicis Sapient survey)