AI Readiness Audit: a 12-Point Checklist 

A board somewhere decides the company needs AI in the product by next quarter, and three layers down, an engineering lead ends up on a call with us asking how fast we can build a copilot. We've had a version of that call often enough to know the honest first answer: we don't know yet, and neither do you. Not because the ambition is wrong, but because the distance between a demo and a dependable feature is decided by things that have nothing to do with models: where your data lives, who's allowed to see it, how fast an answer has to arrive, and what each query is allowed to cost. That's why engagements on our AI and data engineering side start with a readiness audit: two to three weeks of structured checking before anyone commits to a build. This post is that audit written down, including the checklist we work through and the honest signs that say wait.

Data Access and Quality Come First, Always

The first week is mostly spent answering one unglamorous question: can a service account actually read the data this feature depends on, today, without a quarter of integration work? The answer is no more often than anyone expects. The knowledge the copilot needs turns out to live in a vendor system with no export API, in attachments on a shared drive, or in a database whose owner left. Then we sample quality rather than assume it: how complete the records are, how many duplicates exist, how stale the most-referenced documents have become. An assistant that retrieves from a wiki nobody has updated since 2023 will answer confidently from 2023, and users will catch it within a week. None of this is exciting work, which is exactly why it gets skipped, and skipping it is how AI projects become demos that never ship.

Permissions and Privacy Are Where Audits Fail

The single most common blocker we find isn't data quality. It's permissioning. Retrieval-based features inherit the access model of the data underneath them, and if that model is effectively 'everyone on staff sees everything,' the AI layer will cheerfully surface the CEO's compensation memo to anyone who phrases the question nicely. We check whether document-level or row-level access rules exist, whether they're enforceable at query time, and whether anyone can state them out loud. Alongside that sits the privacy map: which fields are personal data, which are regulated, and which markets impose residency rules that decide where inference can even run. A platform serving Saudi customers has data that can't leave the Kingdom, and that constraint shapes the architecture before any model gets chosen. Finding this out in an audit costs two weeks. Finding it out in a compliance review after launch costs a rebuild.

Latency Budgets, Cost Ceilings, and Who Checks the Answer

Three questions get asked in every audit that product teams rarely ask themselves. First, the latency budget: is this feature useful at two seconds, at ten, at forty? A support assistant can think for a while; an autocomplete can't. The budget decides which models and retrieval depths are even candidates. Second, the cost ceiling: what's a query allowed to cost, and what's the monthly bill allowed to reach at projected volume, signed off by someone in finance rather than estimated by an enthusiast? Per-query costs that look trivial in a demo multiply alarmingly at production traffic. Third, the human-in-the-loop points: where does a person review output before it acts on the world? Drafting an email is one risk class; issuing a refund is another. We make the team place every planned capability on that spectrum, and the placement often changes the scope more than any technical finding does.

The 12-Point Checklist

• A named business workflow the feature serves, with a measurable outcome attached
• A data inventory: every source system involved, with a named owner for each
• Quality sampled, not assumed: completeness, duplication, and staleness measured on real records
• A working access path: a service account can read the data today, not after a future integration project
• A permission model the AI layer can enforce, stated precisely enough to implement
• Privacy constraints mapped: personal data, regulated categories, and residency rules per market
• A latency budget per interaction, agreed with product, in seconds
• A cost ceiling per query and per month, owned by finance
• Eval criteria: a written definition of a good answer, plus 50 to 100 graded example questions
• Human-in-the-loop points marked: which outputs ship directly and which get reviewed
• Failure behavior designed: what users see when the model is wrong, slow, or down
• A named owner accountable for the feature after launch, not just during the project

Signs You're Not Ready Yet

Some findings end audits early, and that's a good outcome, not a failure. If the source data lives in personal spreadsheets and inbox attachments, you have a data engineering project first, and pretending otherwise just relocates it into the AI budget. If nobody on the team can write twenty example questions with verifiably correct answers, there's no definition of success to build toward, and evals can't referee what nobody can describe. If the permission answer is a shrug, the feature is a data leak with a chat interface. And if no one will own the cost ceiling, the first invoice at real traffic will trigger the kind of executive conversation that kills programs. None of these mean never. They mean the sequence is wrong, and sequence is cheap to fix before a build and expensive after.

What You Get at the End

The audit produces three things: the checklist above filled in with evidence rather than optimism, a short list of blockers ranked by how long each takes to clear, and a scoped recommendation, which is sometimes 'build this now,' sometimes 'fix these two things first,' and occasionally 'the prompting baseline already does this, build nothing.' We think that last answer is the strongest argument for doing the audit at all: it's the cheapest moment in the whole lifecycle to be told no. If you're weighing an AI feature and want the checklist run against your actual systems rather than your slide deck, that conversation takes half an hour and costs nothing but candor.