What it means
- This is a technical security control. It protects AI systems against attacks that ordinary application security does not catch:
- Prompt injection: attacker text that hijacks the model's instructions. Direct (typed by the user) or indirect (hidden in a web page, email, PDF or retrieved document the model reads).
- Data leakage: the model revealing personal data, secrets, system prompts or other customers' data.
- Model abuse: jailbreaks to produce unlawful content, using your AI to generate deepfakes, stripping watermarks, running up costs, or tricking agents into harmful actions.
- Poisoning and supply chain: tampered training data, fine-tuning sets, embeddings or third-party models.
- ISO/IEC 42001 does NOT contain technical prompt-injection controls. It requires you to assess and treat AI risks, but leaves the technical "how" to you. Use these practical guides:
- owasp top 10 for llm applications (2025 edition): LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, LLM10 Unbounded Consumption.
- MITRE ATLAS: the attacker-technique knowledge base for AI, like ATT&CK for machine learning. LLM prompt injection is AML.T0051; indirect prompt injection is AML.T0051.001.
- ISO 27001 reuse: Partial. Secure development, security testing, vulnerability management, data leakage prevention and monitoring from ISO 27001 give the framework and team. The attack types (prompt injection, jailbreaks, poisoning, excessive agency) and the defences are new for AI.
Why it matters
- RBI FREE-AI asks regulated entities to identify AI-specific security risks (adversarial attacks, data poisoning, model manipulation) before deployment and keep assessing after go-live.
- The 2026 IT Rules require technical measures stopping users from creating unlawful synthetic content and from removing labels or metadata; jailbreak and watermark-stripping defences sit here.
- An LLM connected to email, files, CRM or payments turns a prompt injection into a data breach or fraud, which then also becomes a DPDP Act incident.
Checklist
- Threat model every LLM or agent feature against the OWASP LLM Top 10 and relevant MITRE ATLAS techniques; record results in the AI risk assessment (SC-AI-005 AI risk assessment).
- Treat all model inputs as untrusted, including retrieved documents, web pages, emails and tool outputs (indirect injection, AML.T0051.001).
- Separate system instructions from user and retrieved content; never place secrets or credentials in system prompts.
- Apply input and output filtering (prompt injection classifiers, PII and secret detection, unlawful content filters) via an AI gateway or guardrail layer.
- Treat model output as untrusted before it reaches browsers, SQL, shells or APIs (prevents improper output handling, LLM05).
- Limit agent tools and permissions to the minimum; require human approval for high-impact actions (SC-AI-030 Access control for AI systems and agents, SC-AI-026 Human oversight and override).
- Enforce tenant isolation in retrieval and vector stores so one customer cannot retrieve another's data.
- Set rate limits, token budgets and cost alerts to stop unbounded consumption and model extraction attempts.
- Verify integrity and source of models, datasets and plugins; keep an AI bill of materials (SC-AI-007 Third-party and vendor AI due diligence).
- Protect labels and provenance metadata from tampering (SC-AI-023 Label AI-generated and synthetic content).
- Run adversarial and prompt injection testing before go-live and after major changes (SC-AI-018 Robustness and adversarial testing (red teaming)).
- Log prompts, retrieved sources, tool calls and blocked attempts, and alert on attack patterns (SC-AI-032 Logging and record keeping).
Evidence examples
- Threat model mapped to OWASP LLM Top 10 categories and MITRE ATLAS technique IDs.
- AI gateway or guardrail configuration (filters, blocked categories, rate limits).
- Prompt injection and jailbreak test report with findings and fixes.
- Agent permission matrix showing allowed tools and approval steps.
- Vector store access design showing tenant isolation.
- SIEM alert rules for AI abuse and a sample triaged alert.
- AI bill of materials with model and dataset hashes or versions.
Common pitfalls
- Assuming the model vendor's safety filters are enough.
- Relying on "please ignore malicious instructions" in the system prompt as the main defence.
- Testing only direct prompts and forgetting documents, emails and web content the model reads.
- Giving agents admin API keys because scoping took too long.
- Believing ISO 42001 certification covers prompt injection; it does not.
Satisfies
- BindingEU AI Act
- VoluntaryNIST AI RMF and GenAI Profile
- SupervisoryNYDFS AI Cybersecurity Letter 2024
- BindingTexas TRAIGA
- SupervisoryOSFI E-23
- SupervisoryMeitY AI and Deepfake Advisories
- VoluntaryRBI FREE-AI
Also relevant to
- NITI Aayog Responsible AI for Allguidance mapping (privacy and security; no obligation IDs)
- Tamil Nadu Safe and Ethical AI Policyguidance mapping (misuse scoring; no obligation IDs)
Standards crosswalk
Clause references only. Check the standard itself for the exact wording.
ISO/IEC 42001
- 6.1.2 AI risk assessment
- 6.1.3 AI risk treatment
- A.6.2.4 verification and validation
- A.6.2.6 operation and monitoring
ISO/IEC 27001
- A.5.7 threat intelligence
- A.8.8 management of technical vulnerabilities
- A.8.12 data leakage prevention
- A.8.16 monitoring activities
- A.8.25 secure development life cycle
- A.8.28 secure coding
- A.8.29 security testing in development and acceptance
SOC 2
- CC6.1
- CC6.6
- CC6.8
- CC7.1
- CC7.2
- CC8.1
Key takeaways
- This is hands-on technical security for AI: prompt injection, data leakage and model abuse.
- ISO/IEC 42001 asks you to manage AI risk but has no technical prompt-injection controls; use OWASP LLM Top 10 (2025) and MITRE ATLAS.
- Indirect prompt injection (AML.T0051.001) through documents and web content is the biggest blind spot; limit agent permissions.