Prompt injection
A type of AI system attack where text the system reads contains instructions, and the system follows them as if they came from its user or owner.
Prompt injection is one of the most basic and most effective attacks that systems need to defend against, and there is no complete fix today. Limiting what the system can access and do reduces the damage.
Kind of
- AI security The practice of defending an AI system, and the data and tools it can reach, from attackers and deliberate misuse.
Often confused with
- Jailbreak A type of AI system attack which attempts to 'convince' a system to ignore its rules, guardrails, or system instructions.
Related
- Input guardrail A guardrail that checks or filters what goes into a model before the model processes it.
How it connects
- Prompt injection includes Indirect prompt injection