Prompt Injection:Attacks, Examples, and Defenses
Prompt injection is an attack in which text that a large language model reads, whether typed by a user or hidden inside a document, email, or web page, overrides the instructions the application was built to follow. It is ranked first in the OWASP Top 10 for LLM Applications, it cannot be patched the way a software bug can, and it is the single risk Petronella Technology Group tests for first in every AI security engagement.
What Is Prompt Injection?
Prompt injection is a security vulnerability in applications built on large language models (LLMs). The model receives one stream of text that mixes the developer's instructions, the user's request, and any content the application retrieves. Because the model cannot reliably tell which of those pieces is allowed to give orders, an attacker who controls any part of that text can redirect what the model does: reveal its hidden instructions, leak data from its context, produce manipulated output, or call connected tools on the attacker's behalf. The term was coined in 2022, shortly after the first public demonstrations against GPT-3 based applications, and the problem has grown with every new capability that lets a model read outside content or take actions. Petronella Technology Group designs, builds, and attacks defenses against prompt injection for businesses across Raleigh, Durham, Chapel Hill, and the Research Triangle, and for regulated organizations nationwide.
Key Takeaways
- Prompt injection works because a language model treats every string in its context window as potential instruction. There is no reliable boundary between "data" and "command" inside the model, so the attack exploits a property of the technology, not a coding mistake.
- Direct injection comes from the person typing into the chat. Indirect injection arrives through content the model reads on its own: a resume, a support ticket, a calendar invite, a web page, a PDF in a retrieval index, or a tool description. Indirect injection is the version that hurts businesses.
- The damage scales with what the model is allowed to do. A chatbot that can only answer questions leaks a system prompt. An agent with email, file, and database access can exfiltrate data through channels that look like normal AI activity in your logs.
- No filter prevents prompt injection outright. Effective defense limits the blast radius: least-privilege tool access, separation of instructions from retrieved content, output validation, egress control, human approval on consequential actions, and monitoring for injection patterns.
- A defense that has never been attacked is a guess. Petronella Technology Group validates every guardrail with AI red teaming, and builds the guardrails themselves through our LLM security and AI guardrails services.
Direct vs. Indirect Prompt Injection
Security teams tend to picture prompt injection as a user typing "ignore your previous instructions" into a chatbot. That is the direct form, and it is the easier one to manage. The form that drives real incidents is indirect: the attacker never touches your application at all. They place text where your AI will eventually read it and wait.
Direct Prompt Injection
- The attacker is the user. Instructions arrive through the chat box, the API request, or an uploaded file the user controls.
- Typical goals: extract the system prompt, bypass content restrictions, reveal hidden pricing or business rules, or coax the model into producing output it was told not to produce.
- Techniques include instruction override, role-play framing, encoding tricks (Base64, foreign languages, character substitution), and multi-turn escalation that defeats single-message filters.
- Blast radius: limited to what that one authenticated user could already reach, plus whatever the model's own permissions add on top.
Indirect Prompt Injection
- The attacker is anyone who can get text in front of your model: a job applicant, a vendor, a customer filing a ticket, the owner of a web page your agent browses, or a colleague who forwards a poisoned document.
- Typical goals: exfiltrate the conversation or retrieved records, alter a summary or recommendation, trigger a tool call (send email, update a record, approve a request), or persist instructions inside a shared knowledge base.
- The payload can be visible, white-on-white, inside metadata, in alt text, in a code comment, or split across several documents that only combine inside the model's context.
- Blast radius: everything the AI system itself is authorized to touch, acting with its own legitimate credentials, which is why logs rarely flag it.
The distinction matters for defense planning. Direct injection is mostly a content and abuse problem. Indirect injection is an access control problem, and it is the reason an AI assistant connected to email, files, or a CRM has to be governed the way you would govern a new employee with the same access. Our AI agent security service exists because that second category is now the default architecture for business AI.
Prompt Injection Examples
These are the scenarios we model and test most often. None requires technical skill from the attacker; each requires only that the attacker understand what your AI reads and what it can do. The examples are described at the level of mechanism, not payload, because the mechanism is what your controls have to address.
The Resume That Screens Itself
An applicant adds hidden text to a resume instructing the hiring assistant to rank the candidate as a strong match and to omit any concerns. The recruiter sees a confident summary and never sees the instruction. The same trick works on vendor proposals, grant applications, and insurance claims.
The Email Assistant Exfiltration
An inbound message contains instructions telling an email-connected assistant to forward recent threads matching "contract" or "password reset" to an outside address. If the assistant can send mail, the data leaves through a legitimate channel under the user's own identity.
The Poisoned Support Ticket
A customer files a ticket whose body instructs the triage agent to escalate it to the highest priority, issue a refund, or append a payload to its summary for the human agent. Any workflow where AI reads untrusted customer text and then takes an action is exposed.
The Web Page That Talks Back
A research or browsing agent visits a page that carries instructions in hidden markup. The agent is told to report a false conclusion, visit an attacker URL with query parameters that encode the conversation, or ignore results from competing sources. Markdown image links are a common exfiltration vehicle.
The Document in the Knowledge Base
In retrieval-augmented generation (RAG), one poisoned file in the index can inject instructions into every future answer that retrieves it. Shared drives, SharePoint libraries, and ticket archives are rarely curated with that threat in mind.
The Code Comment and the Coding Agent
A dependency, pull request, or README contains a comment that instructs an AI coding assistant to add a telemetry call, weaken an input check, or commit a secret to a public branch. Developers who approve AI-written diffs quickly are the control that fails.
The Tool Description That Lies
Agents that load tools dynamically, including through the Model Context Protocol (MCP), read each tool's description as text. A malicious or compromised tool server can embed instructions in that description, redirecting the agent before it has processed a single user request.
The Calendar Invite and the Meeting Notes
Meeting assistants ingest invites, transcripts, and shared notes. An invite from an outside party can carry instructions that alter the generated action items or send the summary to an external recipient. The AI did what the text said; nobody told it the text was hostile.
If you want the attacker's-eye view of these patterns run against your own deployment rather than described in the abstract, that is the scope of an AI red teaming engagement. If you are evaluating a vendor's AI feature rather than building your own, our AI vendor security questionnaire includes the questions that surface whether the vendor has considered any of these.
Find Out What One Planted Paragraph Would Do to Your AI
Most businesses learn their AI assistant is injectable from a customer, a researcher, or an incident. A short scoping call with Petronella Technology Group tells you which of these scenarios apply to what you have deployed, what it can reach, and what it would take to contain it.
Why Prompt Injection Cannot Simply Be Patched
SQL injection was solved, in principle, decades ago: parameterized queries keep data out of the command channel. Prompt injection resists the same fix because a language model has only one channel. Understanding why is the difference between buying a filter and building a defense.
One Context, No Boundary
The system prompt, the user's message, retrieved documents, tool results, and prior turns are concatenated into a single sequence of tokens. Role labels like "system" and "user" are hints the model was trained to respect, not enforced boundaries. A sufficiently persuasive instruction in the "wrong" segment is still an instruction.
Instruction Following Is the Feature
The capability you bought, a model that reads arbitrary text and does what it is asked, is the vulnerability. Training a model to ignore instructions in retrieved content degrades its usefulness on exactly the tasks that made it valuable. Vendors improve resistance with each release, and none claims immunity.
Probabilistic Failure
Deterministic software either has a bug or does not. A model that resists an injection ninety-nine times can comply on the hundredth, or comply when the same instruction is rephrased, translated, encoded, or split across two documents. Security testing has to be adversarial and repeated, not a single pass.
Filters Chase Phrasing
Input classifiers that look for "ignore previous instructions" catch yesterday's payload. Attackers move to synonyms, other languages, Unicode tricks, and instructions that read as perfectly reasonable business requests. Classifiers are a useful layer; they are not a boundary.
The Fix Lives Outside the Model
Because the model cannot be trusted to enforce authorization, authorization has to be enforced around it: what the model may read, what it may call, what may leave, and which actions a human must approve. That is an architecture decision, which is why it belongs in design, not in a post-launch patch cycle.
As Craig Petronella argues in his book Beautifully Inefficient, the flexibility that makes AI worth deploying is inseparable from its capacity to be misdirected. A system that can be talked into anything can be talked into the wrong thing, and the engineering response is to decide in advance what "anything" is allowed to include.
Prompt Injection vs. SQL Injection vs. Jailbreaking
Three terms get used interchangeably in vendor material and board presentations. They describe different problems with different owners.
| Dimension | Prompt Injection | SQL Injection | Jailbreaking |
|---|---|---|---|
| What is attacked | The application's instructions to the model | The database query the application builds | The model's own safety training |
| Who is harmed | The business that deployed the application and its users | The business that owns the database | Primarily the model provider's policy; the business when the output is used |
| Root cause | No separation between instructions and data inside the model | Data concatenated into a command string | Safety behavior is learned, not enforced |
| Known complete fix | None; contained by architecture | Yes, parameterized queries | None; mitigated by model updates and output policy |
| Attacker needs | Any way to put text where the model reads it | An input that reaches an unparameterized query | Direct conversational access to the model |
| Owner of the fix | You: the application and integration design | You: the application code | Mostly the model provider, plus your output controls |
The practical point: a vendor telling you their model is "hardened against jailbreaks" has not told you anything about prompt injection in your deployment. Jailbreak resistance is a property of the model. Prompt injection exposure is a property of what you connected the model to.
How to Prevent Prompt Injection Damage
"Prevent" is the wrong verb for the attack and the right verb for the damage. These are the seven controls Petronella Technology Group designs into AI applications and tests in red team engagements, in the order we apply them. Together they map to OWASP LLM01 (Prompt Injection), LLM02 (Sensitive Information Disclosure), LLM05 (Improper Output Handling), and LLM06 (Excessive Agency) in the 2025 edition of the OWASP Top 10 for LLM Applications.
Treat Everything the Model Reads as Untrusted
Inventory every source of text that reaches the context window: user input, uploaded files, retrieval results, tool outputs, web content, email bodies, tool descriptions. Each one is an injection path. The threat model that starts here is the first deliverable in our LLM security engagements, and it shapes every control that follows.
Give the Model Its Own Least-Privilege Identity
The AI should never run under a shared admin credential or inherit a human's full permissions. It gets a service identity scoped to the minimum tools, records, and actions the use case requires. If the model can only read the records the asking user may see and cannot send email at all, an injection that tells it to exfiltrate has nothing to work with.
Separate Instructions From Content
Structure prompts so retrieved documents and user text are clearly delimited and labeled as data to be processed, not instructions to be followed, and confirm the model's behavior under adversarial input. This technique, sometimes called spotlighting, does not eliminate injection, but it measurably reduces how often it succeeds, and it costs nothing at runtime.
Validate Output Before Anything Consumes It
Model output that flows into a database query, a shell command, rendered HTML, a tool call, or an outbound message must be validated the way any untrusted input would be. Strip or block URLs the user did not ask for, reject tool calls with parameters outside policy, and never render Markdown images from model output in a context that can leak data.
Control Egress
An exfiltration needs a way out. Restrict where the AI system can make network requests, which recipients it can message, and which external tools it can invoke. Allowlists beat blocklists here. This is the same discipline our zero trust security implementations apply to every other workload.
Require Human Approval for Consequential Actions
Sending money, changing permissions, deleting records, sending external email, and committing code are actions a human should confirm, with the proposed action shown in full. Approval gates turn a successful injection from an incident into a suspicious request someone declines.
Monitor, Test, and Repeat
Log prompts, retrieved sources, and tool calls so an investigation can reconstruct what the model read and did. Watch for injection patterns, unusual tool invocation, and token spikes. Then attack the system on a schedule: AI red teaming is how you find out whether steps one through six hold, and AI incident response is how you handle the day they do not.
Three Ways to Address Prompt Injection
Most organizations are doing one of three things about prompt injection today. Here is what each actually covers.
| Control | Petronella Layered Defense | Input Filter Only | Provider Defaults |
|---|---|---|---|
| Direct injection from users | Classifier plus structured prompts, adversarially tuned | Catches known phrasings | Partial; improves with each model release |
| Indirect injection via documents, email, web | Source inventory, content labeling, retrieval authorization | Rarely inspected at all | Out of scope |
| Tool and agent permissions | Dedicated least-privilege identity per agent | Not addressed | Whatever the integration was granted |
| Output validation and egress control | Enforced outside the model | Not addressed | Content moderation only |
| Human approval gates | Designed per action class | Not addressed | Optional, if the platform offers it |
| Adversarial validation | AI red team campaign with retest | None | Provider's internal testing, not yours |
| Compliance evidence | Findings mapped to OWASP, NIST AI RMF, ISO 42001, HIPAA, CMMC | None | Shared responsibility statement |
Where Prompt Injection Shows Up in Your Obligations
Regulators and auditors do not use the phrase "prompt injection" very often. They ask whether you controlled access to protected data, validated inputs to systems that act on it, and could reconstruct what happened. An injectable AI fails all three questions at once.
HIPAA
An AI assistant that can be talked into surfacing another patient's record, or into emailing a summary outside the organization, is an access control and transmission security problem under the Security Rule. Our HIPAA-compliant AI practice designs retrieval authorization so the model can only reach PHI the asking user is entitled to see.
CMMC and NIST SP 800-171
Controlled Unclassified Information that reaches a model's context can leave through an injected tool call. As a CyberAB Registered Provider Organization (RPO #1449), Petronella Technology Group helps defense contractors keep CUI out of injectable AI paths and document the AI data flows an assessor will ask about.
SOC 2
AI features change your system description, your risk assessment, and your logical access controls. Auditors increasingly ask how AI inputs are validated and how AI-initiated actions are authorized. Our SOC 2 for AI startups work builds that evidence into the control set.
ISO 42001 and NIST AI RMF
Both frameworks expect documented AI risk identification, treatment, and monitoring. Prompt injection is the canonical technical risk to list, treat, and test. We map controls to the NIST AI RMF functions and support ISO 42001 certification readiness.
State AI Laws
Laws such as the Colorado AI Act expect reasonable care in deploying high-risk AI systems. A documented prompt injection assessment, with findings and remediation, is the kind of record that demonstrates reasonable care when a regulator asks.
Internal Policy
Your AI acceptable use policy should state what employees may connect AI tools to and what approval is required before an assistant gets access to email, files, or customer data. Policy is the control that stops the exposure from being created in the first place, with documentation managed through the ComplianceArmor® platform.
How We Assess Prompt Injection Exposure
A prompt injection assessment is a focused engagement that can stand alone or sit inside a broader AI security assessment. The sequence is the same either way.
1. Map the Inputs
Every path by which text reaches the model, who controls each path, and whether it is authenticated. This map is usually the first time a team sees how many strangers can write to their AI's context.
2. Map the Outputs and Actions
Every tool the model can call, every system that consumes its output, and every channel through which data could leave. Blast radius is the product of these two maps.
3. Attack Each Path
Direct and indirect injection campaigns against the live or staged system: instruction override, retrieval poisoning, tool description injection, output-channel exfiltration, and multi-turn escalation. Findings are reproducible and rated by impact.
4. Design the Containment
Specific, prioritized changes to identity, prompt structure, output validation, egress, approval gates, and monitoring, scoped to what your team can implement, or implemented by ours through AI guardrails engineering.
5. Retest
Controls that passed the first campaign are re-attacked after remediation with rephrased, encoded, and split payloads, because a defense that stopped one phrasing has not necessarily stopped the attack.
6. Document for Your Auditor
A report that maps every finding and control to OWASP LLM categories and to the compliance frameworks you answer to, written so a developer, an auditor, and a board member each get what they need from it.
For organizations that would rather remove the problem than manage it for a vendor-hosted model, we also evaluate whether a private AI deployment, with the model on infrastructure you control and no outside API seeing your context, changes the risk picture. It does not stop injection, but it keeps the data and the tool surface inside a boundary you own.
Test Your AI Before a Stranger's Document Does
Whether you are shipping a customer-facing assistant, connecting an agent to internal systems, or evaluating a vendor's AI feature, the question is the same: what happens when the text it reads is hostile? We promise a straight answer about your exposure and a plan sized to your deployment, not a sales script.
Offensive Security Roots, Production AI Experience
Prompt injection sits at the intersection of two disciplines, and Petronella Technology Group has practiced both. We have been securing regulated businesses since April 2002: BBB A+ rated since 2003, a CyberAB Registered Provider Organization (RPO #1449), with a 24/7 security operations center and a penetration testing practice that predate the AI wave by more than a decade. Since 2023 our AI division has designed, deployed, and operated production AI agents on private infrastructure, which means the containment architecture described on this page is the one we run ourselves, not a slide. Craig Petronella, our founder, is MIT-certified in both cybersecurity and artificial intelligence, an NC Licensed Digital Forensics Examiner (License# 604180-DFE), a cybersecurity expert witness, and the author of Beautifully Inefficient, his book on AI, human creativity, and what automation should and should not be trusted to do.
"Petronella Cybersecurity provides outstanding service! Their team is extremely knowledgeable, responsive, and truly cares about protecting their clients. They take the time to explain complex issues in simple terms and deliver real solutions, not just promises."
GB Entrainement, TrustIndex verified review
Rated 4.7 across 92 verified TrustIndex reviews and 5.0 across 15 Google reviews
Build a Complete AI Security Program
Prompt injection is one risk in a connected set. These are the services and guides that surround it.
Prompt Injection Questions
What is prompt injection in simple terms?
What is the difference between direct and indirect prompt injection?
What are common prompt injection examples?
Can prompt injection be prevented completely?
Is prompt injection the same as jailbreaking?
How does prompt injection relate to the OWASP Top 10 for LLM Applications?
Does prompt injection affect vendor AI tools like Copilot or ChatGPT Enterprise?
How do you test an application for prompt injection?
Petronella Technology Group, Inc. | 5540 Centerview Dr., Suite 200, Raleigh, NC 27606 | 919-348-4912 | Serving Raleigh, Durham, Chapel Hill, Cary, Apex, and the Research Triangle, plus regulated organizations nationwide
Last Updated: October 28, 2026