Whitepaper
The credibility crisis coming for AI-generated business cases
An AI-generated business case can have correct calculations and still be opaque reasoning. This whitepaper argues for a split: let AI accelerate the volume work, keep humans accountable for assumptions and judgment, and design every model so its logic, assumptions and sources can be traced.
AI is transforming how business cases are built. Models that took days can now be generated in minutes. But speed without explainability is creating a new credibility crisis. CFOs can't trust what they can't understand.
The argument of this whitepaper is that the answer is explainable economics, not less AI. AI should handle data gathering, benchmark research, scenario generation and sensitivity analysis; people should own assumption selection, logic design and the judgment calls a CFO will question. The sections below define the four dimensions a business case has to satisfy to count as explainable—logic transparency, assumption visibility, source traceability and outcome attribution—explain why auditability is becoming a requirement from boards, CFOs and regulators rather than overhead, and set out the five components of an explainable system, from assumption architecture to human checkpoints.
AI tools can now generate impressive-looking business cases in minutes. This is both an opportunity and a threat:
The Opportunity: Faster iteration, broader coverage, more sophisticated analysis.
The Threat: Black-box outputs that can't be explained, validated, or trusted.
CFOs are already asking: "Did a human build this? Can you explain how it works? What assumptions are baked in that I can't see?"
The organizations that thrive will be those that harness AI's speed while maintaining the explainability that trust requires.
Explainability isn't just about being able to trace calculations. It has four dimensions:
Logic Transparency: Can someone follow the reasoning from inputs to outputs? Are the "physics" of value creation clear?
Assumption Visibility: Are all assumptions explicit, sourced, and modifiable? Or are some hidden in algorithms?
Source Traceability: Can every data point and benchmark be traced to its origin? Is that origin trustworthy?
Outcome Attribution: When results differ from projections, can you identify which assumptions were wrong and why?
AI-generated business cases often fail on multiple dimensions. The calculations are correct, but the reasoning is opaque.
As AI generates more business cases, auditability becomes non-negotiable:
Regulatory Pressure: In regulated industries, decisions based on unexplainable models may not be compliant.
Stakeholder Demand: Boards, CFOs, and procurement increasingly require documentation of how projections were derived.
Accountability Requirements: When projects underperform, someone needs to explain what went wrong. Black boxes don't allow for learning.
Trust Preservation: Relationships survive when you can explain honestly why a projection was off. They don't survive when you can't.
Auditability isn't overhead—it's the foundation of sustainable value work.
The right mental model for AI in value engineering:
AI Accelerates: Data gathering, benchmark research, scenario generation, sensitivity analysis. These tasks benefit from AI speed without requiring deep explainability.
Humans Govern: Assumption selection, logic design, stakeholder communication, judgment calls. These require human accountability and explainability.
Collaboration Wins: The best outcomes come from AI handling volume and humans handling judgment. Neither alone is optimal.
This means building workflows where AI does heavy lifting but humans remain in the loop for decisions that matter.
Explainability must be designed in, not bolted on:
Assumption Architecture: Every model should have a clear assumption layer that's visible and modifiable, regardless of how the model was generated.
Logic Documentation: The reasoning chain from inputs to outputs should be documentable in plain language, not just formulas.
Source Libraries: Benchmarks and data points should come from maintained, sourced repositories—not hallucinated by AI.
Audit Trails: Every change, every version, every decision should be traceable.
Human Checkpoints: Critical decisions should require human review and approval, with documentation.
These principles apply whether you're using AI, spreadsheets, or purpose-built tools.
Organizations that invest in explainability gain competitive advantage:
Faster Approval: CFOs approve faster when they understand and trust the model.
Better Relationships: Customers trust vendors who can explain their value claims clearly.
Improved Learning: When you can trace why projections were right or wrong, you get better over time.
Reduced Risk: Explainable models are less likely to contain hidden errors or inappropriate assumptions.
Talent Attraction: Strong practitioners want to work with systems they can understand and improve.
Explainability isn't a constraint on AI—it's what makes AI-accelerated value work trustworthy.
The components required for a business case to be truly explainable.
How to appropriately divide work between AI and human judgment.
Design elements required for building explainable value systems.
Assess your current models against the Four Dimensions of Explainability
Evaluate how you currently use AI in value work against the Role Framework
Audit your systems for the Explainable System Components
Identify gaps where explainability is missing or weak
Plan improvements prioritized by impact on trust and approval speed
Build explainability requirements into any new tools or processes
If your reps already draft ROI with ChatGPT, Claude, Gemini or Copilot, ValueNova vs ChatGPT and AI tools compares that approach with a governed platform on the points this whitepaper raises: hallucinated benchmarks, audit trails and consistency across deals.
To test an existing model against the same principles, the free ROI Defensibility Checker asks nine questions about how baselines and improvement rates were sourced, whether assumptions have named owners, whether scenarios and sensitivity were tested, and which dependencies the return relies on.
Download the PDF version to reference offline or share with your team.
Download PDF Version