Global Community

 View Only

Expand all | Collapse all

Use of AI in Auditing

  • 1.  Use of AI in Auditing

    Posted 5 days ago

    I am curious how fellow auditors are using AI for conducting internal audits and writing audit reports.  I am still hesitant to use AI other than to verify what I already believe to be a finding or gap.  I also will use AI to edit my report to make it concise. Please share how you use AI in your auditing practices.

    Regards,

    Nanda



    ------------------------------
    Nanda Filkin
    QA Scientist
    Arete Biosciences
    ------------------------------


  • 2.  RE: Use of AI in Auditing

    Posted 5 days ago

    DETERMINATION OF MAJOR OR MINOR NONCONFORMITIES.

    I worked for the ANSI National Accreditation Body (ANAB) on the Accreditation Council from 2000 to 2020 and served as Chair of the Council in the early 2000's.  I was also an Accreditation Auditor for ANAB from 2005 to 2008.

    I worked as a Lead Auditor for DQS Certification Body (CB) which is the fifth largest CB in the world from 2013 - 2022.  At the end of that period, I led a 55-auditor group auditing ISO 9001, ISO 14001, ISO 45001, ISO 50001, and RC 14001.  We did not use AI to verify audit findings.  Granted, I did not audit medical devices or automotive, but I don't know if that would matter.  I found during that period that the key to the determination of nonconformity is a matter if a requirement in the Standard has not been met. then there is a nonconformity.  The Auditor needs to be able to identify the particular requirement that is not being met.

    The determination of whether it was a minor or a major nonconformity was based on whether the classification is major or minor depending on the severity, scope, and potential impact of the nonconformity.   This includes:

    1. Impact on the management system's effectiveness

    • Major: A breakdown or absence of a key requirement that undermines the system's ability to meet the standard's objectives. This can be a total absence of a required process, a systemic failure, or repeated issues across multiple areas.
    • Minor: An isolated or limited deviation that does not significantly affect the system's overall effectiveness.

    2. Risk to quality, safety, or compliance

    • Major: Noncompliance that could lead to product failure, safety hazards, environmental harm, or regulatory penalties.
    • Minor: No immediate risk to safety, quality, or compliance, though it may indicate a procedural lapse.

    3. Scope and recurrence

    • Major: Often involves multiple minor nonconformities against the same requirement, indicating a widespread problem.
    • Minor: Single or infrequent occurrences, such as a one-off document error or training record omission.

    4. Potential for certification impact

    • Major: If unresolved, can lead to suspension or withdrawal of certification.
    • Minor: Does not immediately threaten certification but still requires correction to prevent escalation.

    5. Judgment and experience

    • Auditors and management use judgment and experience to decide if a nonconformity is likely to cause failure of the system or materially reduce its ability to assure controlled processes.

    I hope this provides some clarification on management system nonconformities.



    ------------------------------
    Chris Spire
    MSIA Associates, LLC
    ------------------------------



  • 3.  RE: Use of AI in Auditing

    Posted 4 days ago

    Thank you Chris.  This is very informative.  I will use this to help guide me in the future. 

    It looks like you have a lot of experience as an auditor and leading auditing groups.  As someone who is relatively new to auditing, I have found AI to be a useful tool to confirm that my findings are relevant and to provide additional reasoning if the auditee needs more than I can provide.  As I gain experience, I realize that the decision to give a major/minor nonconformances will become easier.  Have you used AI for anything in your career as a lead auditor?  



    ------------------------------
    Nanda Filkin
    QA Scientist
    Arete Biosciences
    ------------------------------



  • 4.  RE: Use of AI in Auditing

    Posted 2 days ago

    I really don't have experience using AI in auditing.  I was performing Certification Body Audits for DQS from 2013-2022.  In the North American Operaation, we had 250 auditors with a large support staff.  Many of our auditors like me had over 20 years' experience conducting management system audits.  The new auditors had lead auditors on their audits who could answer their questions and also a manager to call if they could not obtain an answer in the field.  It is a much different situation for the smaller organizations or solo auditors.  Also, AI in auditing is a relatively new tool and I am sure it will continue to work its way into management system audits in the future at large and small organizations and certainly in internal audits. 



    ------------------------------
    Chris Spire
    MSIA Associates, LLC
    ------------------------------



  • 5.  RE: Use of AI in Auditing

    Posted 5 days ago
    Edited by Godly Yao 5 days ago
    You are right, I wonder how auditors are using Ai at work. I think Ai can do more than making writing concise — it can assist with many audit tasks, for example, Sampling: it can suggest statistically valid sample sizes and select random or stratified samples.
    Monitoring continuous audit: it can run scheduled checks and alert on threshold breaches.

    Never, it can't replace professional judgment. Cannot replace auditor professional skepticism, judgment, or attestation responsibilities.

    ---------------------------------
    Godly Yao
    ---------------------------------





  • 6.  RE: Use of AI in Auditing

    Posted 4 days ago

    Thank you Godly for your insightful response.  I agree that AI cannot replace professional judgement but it can be used as an aid to help auditors make decisions.



    ------------------------------
    Nanda Filkin
    QA Scientist
    Arete Biosciences
    ------------------------------



  • 7.  RE: Use of AI in Auditing

    Posted 4 days ago

    Hi @Nanda Filkin,

    Very good question! As we explore the role of AI in the auditing field, it is crucial to remember that AI is a supportive tool, not a replacement for human judgment (as mentioned earlier by @Godly Yao).

    For example, when drafting a report on a major nonconformity and asking AI to "soften" the language, there is a risk that the severity of the issue could be unintentionally minimized. We should always maintain human oversight to ensure our reports remain accurate, objective, and transparent.



    ------------------------------
    Frank Higley-Sanchez
    ------------------------------



  • 8.  RE: Use of AI in Auditing

    Posted 2 days ago

    I would argue that a large number are using AI, but hardly any will likely admit to it. Think PwC and KPMG in Australia.

    A UKAS-accredited pilot demonstrated that AI can be integrated into certification audits-supporting evidence collection, risk analysis, and reporting-in compliance with ISO/IEC 17021-1, confirming that AI can enhance audit effectiveness while remaining standards-compliant under controlled conditions.
    As already mentioned, and it should be unnecessary to say, AI should augment, not automate. That aside, I think ABs and CBs will be late to the party on this one, and unless they act soon, will likely play catch-up. 
    To your question, I have conducted an audit with AI after a non-AI audit for comparison. This was with MS365 Copilot and not a specifically designed or trained model for the task. It performed extremely well at identifying potential issues (both NCs and OFIs), but lacked the nuance of auditing a real-life business, particularly as standards often use words to the effect of "to the extent necessary". However, it performed magically to assist with analysis of data and spreadsheets (as we should expect), and did well in digesting large volumes of information to answer targeted questions.
    With the introduction of technology such as AI glasses that give the ability to record and analyse video, audio, and text, I can see those who embrace the technology, for example, internal auditing, will start to question the results from CBs with traditional methods.
    As both a consultant and an auditor for CBs, I have been investigating various methods and approaches to be able to have conversations with clients. A simple example was a question by a lawyer trying to figure out a pricing method that isn't time-based, as now documents can be done in minutes rather than hours or days.
    It is certainly an interesting area and topic and i look forward to more of it.


    ------------------------------
    David New
    Executive
    Dyadic Consultancy
    ------------------------------



  • 9.  RE: Use of AI in Auditing

    Posted yesterday

    Nanda hanım; çalışmalarınızda başarılar dilerim, günümüz tekneloji içerisinde kendi düşüncem ve yaşadıklarımla beraber her konuyu yapay zeka ile çözüleceği ve netlik konusunda güvenemiyorum.



    ------------------------------
    LEVENT SERHAT DEDEOGLU
    ------------------------------



  • 10.  RE: Use of AI in Auditing

    Posted 11 hours ago
    Edited by Robert TANNER JR 11 hours ago

    As a certified ISO 27001:2022 ISMS Internal Auditor, I have been working with AI since 2024 and have tested the databanks, Corporate and Trainer agendas and bias, hallucinations, response accuracy and inquiries of the AI Platform sourcing references (footnotes) for the past two years and do not use them as a complete "Go To" reliable source. In 2024, I took a training course provided by the FBI that is similar to what the FBI Field Case Agents take to determine the accuracy of AI responses, the integrity of the reference data provided to the agents and with comments that law enforcement should not take the AI provided information as a trusted source but utilize for potential leads and pursuit/discovery pathways.

    CONS:  I have since continuously tested the AI Platforms and have determined that even with advances in AI LLMs, and other types of AI Platforms, I have concluded most AI Platforms are not ready to as more than an 85-percent reliable resource.  In addition, some AI Platforms have limited access to internet (World Wide Web) (that sometimes have past cut-off dates up to several years ago programmed by the company for internal and external safety reasons) with some not able to research past some containment firewalls at all (to find out, one must ask the AI Platform's what their programmed limitations and constraints are before proceeding).

    PROS: For audit summary transcribing during meetings and interviews, when scanning documents with an AI that has an optical capability, and when creating summary reports from an auditor's notes and reports, the AI Platforms can be an invaluable and time saving resource.



    ------------------------------
    Robert Tanner Jr.
    Director of Quality & ISMS Policy Manager
    Herzog Technologies Incorporated
    rtanner@herzog.com
    ------------------------------



  • 11.  RE: Use of AI in Auditing
    Best Answer

    Posted 10 hours ago

    The use of AI in auditing is not only inevitable, it's also needed to improve accuracy and consistency, and to keep pace with the complexity, breadth and depth of audits. It may be helpful to view AI as an augmented resource rather than an automation or simplification tool. It's especially important for auditors not to lose sight of the fact that they are ultimately responsible for everything that goes into the audit report. As @Godly Yao commented: AI cannot replace auditor professional skepticism, judgment, or attestation responsibilities.

    As a cautionary reminder, make sure how you're using AI doesn't violate confidentiality and a data protection requirements. Also keep in mind that some AI tools, for example those used to record meeting notes, have been linked to privacy issues, including one case where a transcription tool was reportedly used to monitor a sensitive human rights meeting without participant consent (see Politico's 2022 investigation into Otter.ai).

    As with most things, AI is only as good or useful as you instruct it, and fortunately, AI can help you help it. Setup takes time and is ongoing if you intend to make regular use of AI in audits. In addition to establishing the basic settings like tone and perspective (e.g. skeptical, third-party, independent auditor), AI can also advise on how to structure your requests depending on the complexity and format of the data. In fact, I have several AI-generated instruction sets, written by AI itself, that span several pages when saved in Word.

    Routinely challenge the results you get. You should expect to regularly ask for clarification, correct matters of fact, and even argue with your AI tool in order to improve the output. The results AI generates should support, enhance, and highlight the higher-level themes you've already identified as an audit team. Be skeptical of any surprising findings from AI. At most, these create new audit trails to pursue with standard auditing skills, and additional evidence and follow-up requirements.



    ------------------------------
    William Huggett
    ------------------------------



  • 12.  RE: Use of AI in Auditing

    Posted 10 hours ago

    Thank you @William Huggett for this insightful summary of how AI can be a helpful tool but should be used with caution.  I appreciate your perspective.



    ------------------------------
    Nanda Filkin
    QA Scientist
    Arete Biosciences
    ------------------------------



  • 13.  RE: Use of AI in Auditing

    Posted 10 hours ago
    Edited by Robert TANNER JR 10 hours ago

    I agree with the future enthusiasm regarding the "possibilities" and applications of AI, but as Auditors, our "guard" must be up when performing audits on companies, corporations and agencies.  As Auditors, it is essential to be trained to determine what AI Platform is the authentic Evidence Resource that contains the authentic quality records, documentation and performance indicators that have not been influenced by the Training/Algorithmic Programming of the AI Platform by the company, corporation or agency

    Did the customer provide the 3rd Party/External Auditor access to an alternate AI Platform setup specifically for 3rd Party/External Audits?

    This was my first question two years ago involving a scenario where am I auditing Customer AI Platform "A" (setup as the authentic repository used by company approval workflows, operations, designs, calibrations, test samples and performance reports), or have I been sat down at the Customer's AI Platform "B" (setup specifically for 3rd Party/External Audits where documents, results and reports have been scrubbed by an algorithm or with more preferential data?).  

    As in all past and current audits, it is the query skill of the Auditor than can determine a gap or make a discovery that additional sampling and information may need to be pursued.  As Auditors, in the near future, it may be necessary to attend additional query training to determine the authenticity of the requested evidence information is being obtained from a company's, corporate's or agency's only AI Platform or presented from a Partitioned or Separate AI Platform utilized specifically for External Audits.



    ------------------------------
    Robert Tanner Jr.
    Director of Quality & ISMS Policy Manager
    Herzog Technologies Incorporated
    rtanner@herzog.com
    ------------------------------



  • 14.  RE: Use of AI in Auditing

    Posted 9 hours ago

    Thank you for this response @Robert TANNER JR.   I appreciate your statement regarding the query skill of the Auditor.  I agree that query training will be imperative for the future of auditing, especially in differentiating authentic documents and records from those generate by AI.



    ------------------------------
    Nanda Filkin
    QA Scientist
    Arete Biosciences
    ------------------------------