ASD ISM 2089 Monitor AI model performance and investigate anomalies
What it says
AI model performance metrics are monitored and anomalies are investigated.
Quoted as published. Everything else on this page is written by Control Stack.
What it means
In plain EnglishKeep track of AI model performance and check any unusual behaviours.
This control is about continuously checking how well your AI systems are working and quickly digging into anything unusual. It matters because if you miss odd behaviour, you could end up with poor decisions, unhappy customers, or even legal issues if things go wrong.
Free. Save your status, tasks and notes.
Why it matters
If AI performance issues are ignored, businesses risk making poor decisions, damaging customer trust, and facing legal trouble.
How to implement
The IT team should regularly review AI model performance data.
They should set up automated alerts to notify them of any unexpected changes in the model's outputs.
Managers need to designate a specific staff member to investigate anomalies.
They should ensure this person has a clear process for identifying and examining unusual model behaviour.
System owners should define 'normal' performance metrics for their AI models.
They should work with analysts to specify what variances from these norms require further investigation.
The IT department should implement logging of AI model outputs.
They should do this by using software that records each decision or prediction made by the AI for audit and troubleshooting.
Business leaders should review anomaly reports with their IT team at regular intervals.
This ensures that any identified issues are being timely addressed and lessons are learned for future prevention.
Keep it running
Regularly update your AI model monitoring processes to keep pace with changes in technology and business needs.
Evidence and audit
- Check 1 of 5What an auditor asks for
Ask for
Ask for the protocol or procedure that outlines the steps for monitoring AI model performance.
Look at
Check whether it includes periodic reviews and anomaly checks, with clear roles and a defined frequency.
Good evidence
A detailed document with clear roles and review frequency, approved and signed by management.
- Check 2 of 5What an auditor asks for
Ask for
Request logs of AI model decisions and predictions.
Look at
Check for consistency in how these logs are maintained and whether anomalies are flagged within them.
Good evidence
Organised logs showing regular entries and flagged anomalies.
- Check 3 of 5What an auditor asks for
Ask for
Ask for anomaly investigation reports.
Look at
Confirm each report includes a summary of findings and the actions taken.
Good evidence
A comprehensive report that outlines the incident, the investigation steps and the corrective actions.
- Check 4 of 5What an auditor asks for
Ask for
Request the list of performance metrics used for AI models.
Look at
Confirm the metrics are tailored to the specific AI model's objectives and are reviewed over time.
Good evidence
Metrics that align with business goals and known risks, and are regularly updated.
- Check 5 of 5What an auditor asks for
Ask for
Ask about training materials for staff on handling AI anomalies.
Look at
Look for detailed roles and expected response times for investigating anomalies.
Good evidence
Documented training sessions and attendance records demonstrating proactive education.
Mappings
Closest match in ISO/IEC 27001: A.8.15, Logging of activities and events, which this control partially meets.
ISO/IEC 27001: 2 mappings
This control partially meets A.8.15.
ISM-2089 requires organisations to monitor AI model performance metrics and investigate anomalies. Annex A 8.15 requires logs of activities, exceptions, faults and other relevant events to be produced, stored, protected and analysed, which provides the logging and analysis foundation for detecting and investigating AI model anomalies. ISM-2089 focuses on AI-specific signals (e.g. drift, unexpected output patterns, abnormal error rates), while Annex A 8.15 is broader logging governance across all systems and applications.
This control partially meets A.8.16.
ISM-2089 requires organisations to monitor AI model performance metrics and investigate anomalies in model behaviour or outputs. Annex A 8.16 requires monitoring networks, systems and applications for anomalous behaviour and taking action to evaluate potential information security incidents, which can include AI services and their supporting platforms. ISM-2089 is a specific, AI-focused instantiation of Annex A 8.16's broader anomaly monitoring and evaluation requirement with emphasis on model metrics and output integrity rather than general system telemetry.
ISO/IEC 42001: 1 mapping
This control depends on A.6.2.8.
Annex A 6.2.8 requires enabling AI system event logging at defined lifecycle phases (at least during operational use) to ensure traceability of system activity. ISM-2089 requires monitoring AI model performance metrics and investigating anomalies, which is enabled by having event logging turned on during the operational phase and often during testing/validation phases to establish baseline behaviour. ISM-2089 focuses on monitoring and investigation actions, while Annex A 6.2.8 focuses on the lifecycle decision and minimum operational enablement of logging.
Mappings show how controls relate across frameworks. They do not imply full equivalence or certification.