ASD ISM 1924 Detect and mitigate adversarial prompts in generative AI applications
What it says
Generative AI applications evaluate user prompts to detect and mitigate adversarial inputs or suffixes designed to elicit unintended behaviour or assist in the generation of sensitive or harmful content.
Quoted as published. Everything else on this page is written by Control Stack.
What it means
In plain EnglishGenerative AI applications must check user prompts to catch and block adversarial inputs that try to trick the AI into harmful or unintended behaviour.
If your organisation runs a generative AI application (software that creates text, images or code in response to a typed request, like a chatbot), people may deliberately craft tricky inputs to make it misbehave. These tricks, often called prompt injection or jailbreak attempts, can include hidden instructions or strange text added to the end of a request (a "suffix") designed to make the AI ignore its safety rules. This control requires the application to actively examine every user prompt, spot these manipulations, and stop the AI from being fooled into producing sensitive or harmful content.
Free. Save your status, tasks and notes.
Why it matters
If adversarial prompts are not detected, attackers can jailbreak the AI into leaking sensitive data or producing harmful content, exposing the organisation to data breaches and reputational damage.
How to implement
The development team should add a prompt-screening step in front of the generative AI application that inspects each user request before it reaches the AI model, checking for known manipulation patterns and unusual suffixes that try to override the AI's instructions.
The IT or AI team should keep a regularly updated list of known adversarial techniques (such as 'ignore previous instructions' phrasing and common jailbreak suffixes) and feed it into the screening tool so newly discovered attacks are caught.
The development team should configure the application to block, sanitise, or flag suspicious prompts rather than passing them straight to the AI, and return a safe refusal message when a prompt is identified as adversarial.
The AI team should run regular adversarial testing (sometimes called red teaming), deliberately feeding the application tricky prompts to confirm it detects and refuses attempts to produce sensitive or harmful content.
The application owner should ensure every blocked or flagged prompt is logged so the security team can review attack attempts, measure how often they occur, and tune the detection rules over time.
Keep it running
Adversarial techniques evolve constantly, so keep detection rules current and re-test the application regularly rather than treating prompt screening as a one-time configuration.
Evidence and audit
- Check 1 of 5What an auditor asks for
Ask for
to see how the generative AI application screens incoming user prompts.
Look at
the design or configuration of the prompt-checking layer.
Good evidence
shows a documented mechanism that inspects every prompt for adversarial inputs and suffixes before the AI responds.
- Check 2 of 5What an auditor asks for
Ask for
the list of adversarial patterns or jailbreak techniques the application defends against and when it was last updated.
Look at
the update history.
Good evidence
shows a maintained, current list rather than a one-off setup left untouched.
- Check 3 of 5What an auditor asks for
Ask for
the results of recent adversarial testing or red-team exercises against the application.
Look at
the test cases and outcomes.
Good evidence
shows tricky prompts were tried, the system blocked them, and any failures were fixed.
- Check 4 of 5What an auditor asks for
Ask for
to be shown what happens when an adversarial prompt is submitted.
Look at
a live or recorded demonstration.
Good evidence
shows the application detects the attempt, refuses, and does not generate sensitive or harmful content.
- Check 5 of 5What an auditor asks for
Ask for
logs of detected or blocked prompts and who reviews them.
Look at
recent log entries and review records.
Good evidence
shows attempts are captured, regularly reviewed, and used to improve detection.
Mappings
Closest match in ISO/IEC 27001: A.8.26, Defining security requirements for applications, which this control supports.
ISO/IEC 27001: 3 mappings
This control supports A.8.26.
ISM-1924 requires generative AI solutions to evaluate user prompts and mitigate adversarial inputs intended to elicit unintended behaviour or sensitive/harmful content. Annex A 8.26 is a prerequisite because defining and approving security requirements for applications is where prompt-injection resistance and safety requirements should be specified for an AI application. Annex A 8.26 applies to all application acquisitions/development, whereas ISM-1924 is one concrete requirement that should be captured within those application security requirements.
This control supports A.8.28.
ISM-1924 requires organisations to build AI applications that can identify and mitigate adversarial prompt content (e.g. injection strings, malicious suffixes) at the point of use. Annex A 8.28 is a prerequisite because secure coding practices are typically needed to implement robust input validation, policy enforcement, guardrails, and safe prompt-handling patterns without introducing bypasses. Annex A 8.28 is broader across all software development, while ISM-1924 is a specialised secure-by-design requirement for generative AI prompt interfaces.
This control depends on A.8.12.
ISM-1924 focuses on preventing prompt injection so the AI does not generate or disclose sensitive or harmful content due to adversarial prompts. Annex A 8.12 supports this objective by requiring data leakage prevention measures that can reduce the chance of sensitive information being exposed even if a prompt-injection attempt partially succeeds. Annex A 8.12 is broader and applies across systems and data flows, while ISM-1924 is specifically about adversarial prompt detection/mitigation at the AI interface.
Mappings show how controls relate across frameworks. They do not imply full equivalence or certification.