Global Community

Here's the full post, trimmed. I cut the repetition and the throat-clearing so it reads tight - an auditor's room rewards economy.

Subject: Handling unverifiable controls in an ISO 42001 AI risk assessment - insufficient objective evidence, or something cleaner?

Hi everyone, first post and a genuine methodology question I'd value your view on.

I've recently started a business doing compliance and risk assessment for general practice in Australia, focused on the fast adoption of AI "scribes" (ambient tools that record and summarise the doctor-patient consultation). I've completed my ISO 9001 and ISO 42001 lead auditor training and I'm now applying it where the tooling is moving faster than the guidance. Practices here need a documented risk assessment before adopting these tools, with the Privacy Act / APPs and the RACGP Standards sitting underneath.

Here's the problem I keep hitting. Some of the most material questions - where data is processed, whether audio is transiently retained, what a platform integration can access - can't always be verified. The vendor declines to confirm, or points to public documentation that turns out to be incomplete or internally contradictory. So I'm left with a risk item I can neither confirm nor close out.

My working approach is to treat this as insufficient objective evidence in the ISO 19011 sense: record exactly what was asked, of whom, and when; note the evidence was insufficient to determine conformity; and carry the item forward as a rated residual risk with a recommended treatment (obtain written confirmation before adoption, or document acceptance of the residual risk at practice level) - rather than marking it "compliant" or pretending the gap isn't there. ISO 42001's emphasis on transparency and documented information seems to support treating a vendor's non-disclosure as itself a recordable input to the risk picture.

What sits uncomfortably with me is that the unverifiable items are precisely the most material ones. A deliverable that's thorough on everything except the questions that matter most hasn't addressed the risk - it's documented that the risk is unaddressed. I'm not comfortable calling "we couldn't find out" a quality outcome when the unknowns are the very things a practice most needs answered.

So, two questions for the room:

1. Is recording insufficient evidence + rated residual risk + treatment the soundest way to handle a genuinely unverifiable control, or is there a cleaner convention I'm missing?
2. Where a cluster of material items comes back unverifiable, at what point does "unable to verify" become a finding about the tool's adoptability itself, rather than just a list of open items?

Grateful for any steers, references, or "here's how we do it" from other domains.

Thanks for having me.