Two roles, both underdeveloped
Procurement teams occupy two distinct positions in AI governance: users of AI for contract analysis, supplier due diligence, spend analytics and market intelligence; and governors of AI procurement across the organisation. Most organisations have the first by accident and the second inadequately designed.
Negotiating AI-specific contract provisions
Standard software contract terms often fail to address the specific risks of AI tools. Key provisions procurement teams should ensure are addressed:
Data handling: Where is data processed? Who can access it? Is it used to train the model? Does processing comply with APP 8 and sector-specific data sovereignty requirements?
Incident notification: The Privacy Act requires notification of eligible data breaches to the OAIC and affected individuals. Your contract must ensure you receive timely vendor notice to meet these obligations within required timeframes.
Audit rights: Can you audit the vendors AI governance practices, security controls, and contract compliance? For AI tools supporting critical functions, this right is increasingly expected by auditors and regulators.
AI output accuracy: Standard uptime SLAs do not capture AI-specific performance requirements. Consider whether contracts should include commitments around accuracy, bias monitoring, and model drift management.
Liability: Most vendor contracts limit liability significantly. Assess whether additional indemnification or insurance is required for high-stakes AI use cases where AI-generated errors could cause material loss.
Governing AI procurement across the organisation
Shadow AI — business units accessing AI tools on a credit card or through software subscription channels without procurement review — is a significant governance problem. Establish a lightweight AI procurement review process covering data handling, security, privacy compliance, and alignment with the AI governance framework. The National AI Centres AI system register template provides a useful structure for documenting each tool in use.
Supplier risk in an AI-enabled landscape
Your suppliers are increasingly using AI in their operations. Supplier due diligence questionnaires should now ask whether suppliers use AI in delivering contracted services; what oversight exists; and whether AI-assisted delivery is disclosed to and consented to by clients. Suppliers using AI that creates WHS risks for their workers — including under NSWs Digital Work Systems Act 2026 — may have compliance obligations affecting your supply chain risk profile.
