Skip to content
Security assurance
ASD ISMISM-2117

ASD ISM 2117 AI models augment cyber security event detection

What it says

Suitable AI models are used to augment the detection of cyber security events and the identification of cyber security incidents.

Quoted as published. Everything else on this page is written by Control Stack.

What it means

In plain English

Only AI models proven suitable (validated for detection efficacy, false-negative behaviour, provenance and adversarial robustness) are used to augment, not replace, the detection of cyber security events and the identification of incidents.

This control lets you use artificial intelligence to help your monitoring tools spot suspicious activity faster and find incidents your analysts might miss, but only if the model is genuinely fit for the job. The key word is "suitable": before you trust a model in detection, you have to test how well it actually catches threats, measure how often it misses them, know where it came from, and check it cannot be easily fooled. AI here is an assistant to your monitoring, not a substitute for it, so a poor model must never become a single blind spot.

Why it matters

If an unsuitable or unvalidated AI model is wired into detection, it can silently produce false negatives (genuine intrusions, lateral movement or data exfiltration are scored as benign and never raised as events or incidents) while staff assume the AI has it covered. Equally, an over-tuned model floods analysts with false positives, causing alert fatigue so real detections are dismissed. Undetected model drift, or adversarial evasion crafted to slip past the model, then leaves attacks running for longer with no human catching the gap the AI created.

How to implement

  1. Define written suitability criteria for any AI model before it touches detection, including minimum recall (acceptable false-negative rate), maximum false-positive rate, supported log sources and the detection use cases it is approved for.

  2. Build a labelled validation dataset of true incidents and benign activity drawn from your own environment, then measure each candidate model's precision, recall and false-negative rate against it and reject models that miss the threshold.

  3. Capture model provenance for every model you deploy: record the vendor, version, training-data origin and documented limitations for sourced models, and maintain lineage and training records for any in-house models.

  4. Run adversarial robustness testing by feeding the model crafted and evasion-style inputs designed to be misclassified as benign, and only deploy models that hold up or have compensating controls for known weaknesses.

  5. Deploy the model to augment existing monitoring rather than replace it: keep signature, heuristic and human review in the detection path so AI scores enrich alerts instead of being the only gate on raising an event or incident.

  6. Instrument continuous drift monitoring against the validated baseline, wire drift breaches and a fixed retraining cadence to a revalidation run before promotion, and revalidate whenever new threat types, techniques or log sources appear.

Keep it running

Treat the model as a monitored asset: continuously track detection precision and recall (especially the false-negative rate) against labelled and red-team-generated samples, and trigger retraining when performance decays past a defined threshold. Set a fixed retraining and revalidation cadence, and additionally revalidate whenever new threat types, log sources or attack techniques emerge so the model does not silently fall behind the threat landscape. Watch for model drift by comparing live input distributions and detection rates against the validated baseline, and test resilience to adversarial evasion (crafted inputs designed to be misclassified as benign). Always keep AI augmentation behind human-led monitoring and signature/heuristic detection so a model failure degrades coverage rather than eliminating it.

Evidence and audit

  1. Check 1 of 5What an auditor asks for

    Ask for

    For each AI model in the detection pipeline, ask for its efficacy test report against a labelled dataset and the documented acceptance threshold the model had to meet before deployment.

    Look at

    Check that the report states a measured false-negative rate (recall) and a false-positive rate, and that the acceptance threshold was set and recorded before the model went live.

    Good evidence

    Every model has a measured recall and false-positive rate against a labelled dataset, and the results met a documented acceptance threshold before deployment.

  2. Check 2 of 5What an auditor asks for

    Ask for

    Ask to trace one or more AI-raised detections and one AI-suppressed sample through the detection pipeline.

    Look at

    Follow each sample to see whether the AI output feeds into human monitoring and existing signature or heuristic detection, or whether the AI alone decides if an event is raised.

    Good evidence

    AI output augments human monitoring and existing signature or heuristic detection rather than being the sole arbiter of whether a cyber security event is raised.

  3. Check 3 of 5What an auditor asks for

    Ask for

    Ask for provenance and assurance evidence for each sourced model, such as the model card, version, supplier attestation and training-data origin, and equivalent lineage documentation for any in-house models.

    Look at

    Confirm each sourced model has documented provenance and assurance, and that in-house models have equivalent lineage documentation establishing their suitability.

    Good evidence

    Every model in use, sourced or in-house, has documented provenance and lineage that establishes it is suitable for detecting cyber security events.

  4. Check 4 of 5What an auditor asks for

    Ask for

    Ask for adversarial robustness test results for each model.

    Look at

    Confirm the model was evaluated against evasion or crafted-input attacks, and check whether any identified weaknesses were recorded and tracked to remediation.

    Good evidence

    The model has been tested against evasion or crafted-input attacks, and any weaknesses found are tracked through to remediation.

  5. Check 5 of 5What an auditor asks for

    Ask for

    Ask for drift-monitoring reports, the retraining log and the latest revalidation record for each model.

    Look at

    Confirm detection performance is measured continuously against the validated baseline, that retraining or revalidation was triggered on schedule and when new threat types appeared, and that the model version in production matches the one in the latest revalidation record.

    Good evidence

    Performance is continuously monitored against the baseline, retraining or revalidation occurs on schedule and in response to new threat types, and the deployed model version is the one that passed suitability testing.

Mappings

Closest match in ASD Essentials for enterprise IT: IT.DET-03.2, Correlate signals across identities, applications, data and infrastructure to identify incidents, which partially overlaps with this control.

ISO 27001

ISO/IEC 27001: 2 mappings

  • A.8.15Logging of activities and eventsSupports
  • A.8.16Monitoring networks and systems for anomalous behaviourSupports
Essential Eight

ASD Essential Eight: 1 mapping

  • E8-AH-ML2.15Timely analysis of cyber security events to identify incidentsSupports
ASD Essentials

ASD Essentials for enterprise IT: 1 mapping

Mappings show how controls relate across frameworks. They do not imply full equivalence or certification.