AI security
The practice of defending an AI system, and the data and tools it can reach, from attackers and deliberate misuse.
AI security is concerned with defending a system from misuse, especially intentional misuse that could jeopardize the model, its data, its tools or its outputs. AI safety covers harm with no attacker, including accidents.
Also called
- Security for AI
- LLM security
Kinds
- Data exfiltration A type of AI system attack which attempts to have the system send private data outside your company or to anyone not permitted to see it.
- Indirect prompt injection A type of prompt injection where instructions are hidden in content that the AI system reads (web page, email, file, etc.).
- Jailbreak A type of AI system attack which attempts to 'convince' a system to ignore its rules, guardrails, or system instructions.
- Prompt injection A type of AI system attack where text the system reads contains instructions, and the system follows them as if they came from its user or owner.
The other half
- AI safety The practice of protecting against harm that an AI system may cause or help cause, whether by its own mistakes or by doing what a person asks when it shouldn't.
How it connects
- Red teaming tests AI security
- AI security defended by Guardrail