Why vendor AI due diligence is a regulatory obligation, not just a procurement preference

Regulators across jurisdictions are consistent on one point: accountability for AI systems cannot be outsourced to vendors. The EU AI Act Articles 25 and 26 establish a clear value chain of responsibilities, providers retain liability for the systems they place on the market, and deployers (the organisations that use AI systems in their own products and services) carry accountability for how those systems are deployed and monitored. Under Australia's CPS 230, AI vendors supporting critical operations are material service providers subject to mandatory due diligence, contractual protections, and ongoing monitoring. APRA (Australian Prudential Regulation Authority)'s April 2026 letter found that some regulated entities lacked adequate visibility over their AI supply chain, including fourth-party dependencies.

Inadequate vendor due diligence creates direct regulatory exposure. An organisation that cannot demonstrate it assessed its AI vendor's data practices, security posture, model governance, and regulatory compliance before procurement is in a poor position when a regulator reviews an incident, a bias complaint is filed, or an enforcement action is initiated against a product built on that vendor's system.

The five areas every AI vendor assessment must cover

1. Data practices and privacy compliance. How does the vendor use your data, including the inputs you send through their API? Does the vendor use customer data to train or fine-tune models? Under what circumstances, and with whose consent? Who are the vendor's subprocessors? Where is data processed and stored? Does the vendor's data processing agreement meet the requirements of the Privacy Act 1988 (Australian entities), GDPR (EU/UK entities processing EU resident data), or other applicable privacy law? For vendors processing sensitive categories of information, health data, financial data, biometric data, what specific controls apply?

2. Model governance and documentation. Can the vendor provide a model card or technical documentation describing the model's intended use, training data, known limitations, and performance characteristics? Has the model been independently evaluated for bias? What demographic groups were used in testing, and across what tasks? Is the model subject to ongoing monitoring by the vendor after deployment? What is the vendor's process for identifying and addressing model drift, bias, and degradation? For EU AI Act high-risk AI systems, providers must produce technical documentation meeting Articles 11 and 13 requirements, can the vendor supply this?

3. Security and resilience. What is the vendor's security certification status, SOC 2 Type II, ISO 27001, or equivalent? Has the vendor conducted adversarial testing of the AI system, including prompt injection testing for LLM-based systems? What are the vendor's contractual uptime and availability commitments for systems used in critical operations? What is the vendor's incident response and notification process, specifically, how quickly will they notify you of a security incident or significant model failure? APRA's CPS 230 requires contractual incident notification obligations for material service providers.

4. Regulatory compliance status. What jurisdictions does the vendor operate in, and what regulatory obligations do they acknowledge? Has the vendor conducted an EU AI Act risk classification assessment for their systems? For Annex III high-risk AI systems, does the vendor have a conformity assessment and CE marking (where required)? Is the vendor's system registered in the EU AI Act database for high-risk systems? What is the vendor's position on liability under applicable AI regulation, do they accept deployer-facing obligations or attempt to disclaim all regulatory accountability?

5. Contractual protections. Several contractual protections are either legally required or operationally critical. Required under CPS 230 for APRA-regulated entities with material service providers: audit rights allowing the regulated entity or its auditor to assess the vendor's controls; incident notification timelines; adequate liability provisions; and exit and transition plans. Operationally critical for any AI vendor relationship: ownership of outputs (who owns the content or decisions produced by the AI system, the organisation or the vendor?); prohibited uses (what does the vendor's acceptable use policy prohibit, and are any of those prohibitions inconsistent with your intended use?); change notification (will the vendor notify you before making model updates that could change system behaviour?); and data deletion on termination.

Questions to ask AI vendors before procurement

The following are the most important questions to put directly to a vendor during procurement evaluation. They are designed to distinguish vendors that have genuinely invested in governance from those offering marketing claims:

On data: "If I send customer data through your API, can you confirm that it will not be used to train or improve your model? Where is it processed and for how long is it retained? Who are your subprocessors for AI processing?"

On model behaviour: "Can you provide a model card or technical documentation for this system? What demographic groups were represented in your bias testing, and what were the results? What is your process for detecting and addressing model drift after deployment?"

On incidents: "What is your definition of a significant incident for this system, and what is your contractual commitment for notifying us? Have there been any significant incidents in the past 12 months involving this system?"

On regulatory status: "Have you classified this system under the EU AI Act risk framework? If it is high-risk, do you have technical documentation and a conformity assessment we can review? What is your position on your obligations as a provider under the EU AI Act relative to our obligations as a deployer?"

On exit: "What does our data and model configuration look like at termination? What is the transition period, and what assistance do you provide to migrate to an alternative system?"

Ongoing monitoring, due diligence is not a one-time exercise

Vendor AI due diligence does not end at procurement. Regulatory expectations require ongoing monitoring: CPS 230 requires regular review and testing of material service provider arrangements; the EU AI Act requires post-market monitoring of high-risk AI systems by deployers (Article 72); and APRA's April 2026 letter specifically flagged concentration risk where entities rely heavily on a single AI vendor for multiple use cases.

An annual vendor review cycle should cover: any model updates made since the previous review and their impact on system behaviour; updated security certifications; regulatory compliance status under evolving law; incident and near-miss history; and continued alignment with the organisation's risk appetite. For material service providers under CPS 230, the review should also assess the vendor's subcontractor and fourth-party chain for changes that could affect supply chain risk.

Related reading

Further reading: ISO/IEC 42001