Treat AI model output as untrusted input: improper output handling explained
Treat AI output as untrusted data before it reaches web pages, databases, commands or other systems that interpret content.
A common shortcut in AI features: take what the model returns and pass it straight to the next part of the system, such as a web page, a database or a command line. OWASP's Top 10 for LLM Applications (2025) calls the failure to check that output LLM05, Improper Output Handling. We read the project's page in full on 15 November 2026. We did not read its references. This article summarises that one source.
The idea
OWASP defines the risk as insufficient validation, sanitisation and handling of model output before it goes downstream. Because model output can be steered by the prompt, passing it on unchecked is "similar to providing users indirect access to additional functionality." Our article on prompt injection covers how text steers a model; this one covers what happens after the model answers. OWASP separates it from overreliance, which is about trusting the accuracy of answers, not about where the output goes.
Per the page, successful exploitation can mean cross-site scripting (XSS) and cross-site request forgery (CSRF) in browsers, and server-side request forgery, privilege escalation or remote code execution on backends.
Examples the page lists
- Model output goes into a shell or a function like exec or eval.
- The model returns JavaScript or Markdown that the browser interprets.
- Model-written SQL runs without parameterisation.
- Model output builds file paths without sanitising them (path traversal).
- Model output goes into email templates without escaping.
What OWASP recommends
- Treat the model like any other user (zero trust) and validate what it sends to backend functions.
- Follow OWASP ASVS guidance on validation and sanitisation.
- Encode model output before showing it to users.
- Use context-aware encoding: HTML encoding for HTML text. For SQL values, use parameterised queries or prepared statements rather than relying on manual escaping. Parameterization does not make an arbitrary model-written SQL command authorized: restrict allowed operations and database permissions as well.
- Use strict Content Security Policies.
- Log and monitor model outputs for unusual patterns while applying redaction, access and retention controls. Do not create a second leak by retaining private prompts or credentials in raw logs.
The page also says impact rises when the app gives the model more privileges than a normal user, when third-party extensions do not validate input, and when there is no rate limiting or anomaly detection on model use.
Scenarios from the page
- A chatbot's reply is passed unchecked to an admin extension, which shuts the system down for maintenance.
- A website summariser reads a page containing a hidden instruction and sends sensitive data to an attacker's server.
- A chat feature that writes SQL runs a request to delete every table.
- A crafted prompt returns a JavaScript payload that runs in a victim's browser.
- A generated email template carries malicious JavaScript.
- Code generation may suggest software packages that do not exist, which an attacker could then publish as malware. The page says to review code and verify packages.
A short checklist
- Where does each piece of model output end up: screen, query, file path, shell, email?
- Is it encoded or validated for that exact place?
- Does the model hold more access than the person using it?
- Can we see odd outputs in logs?
Related reading on this site: SQL injection prevention, input validation basics, Content Security Policy basics, CSRF prevention basics.
Limits
One OWASP page, read in full; references and ASVS were not read. No claims about specific models or products, and no statistics.
By Omnistu.