OOmnistu

Blog / AI Development

AI Development

Can an AI feature leak private data? Sensitive information disclosure explained

Review how AI features can expose private information and where data controls, permissions and testing can reduce risk.

OOmnistu
Reviewed 2026-11-16 · 2 min read

OWASP's Top 10 for LLM Applications (2025) lists Sensitive Information Disclosure as LLM02. We read the project's page in full on 15 November 2026, and not the references it links. This article summarises that one page. It is not legal advice; if personal data law applies to your product, ask counsel.

What counts as sensitive

The page lists personal identifiable information (PII), financial details, health records, confidential business data, security credentials and legal documents. It also says proprietary model details, such as training methods and source code, can be sensitive. The risk covers the model and the application around it. Leaks happen through the model's output: unauthorised data access, privacy violations and intellectual property breaches.

Three kinds of exposure, per the page

  1. PII leakage during interactions.
  2. Proprietary algorithm exposure. Poorly configured outputs can reveal algorithms or data. The page discusses training-data exposure, model inversion and extraction as related risks. This overview does not independently establish any incident or vulnerability identifier.
  3. Business data disclosure. Responses may include confidential information by accident.

Mitigations OWASP lists

These techniques address different threat models and are not interchangeable guarantees. Concealing a prompt cannot replace access control, and specialist privacy techniques need separate design review. On system-prompt rules like "never return personal data", the page warns that such restrictions may not always be honoured and can be bypassed through prompt injection. So they are one layer, not the control.

Scenarios on the page

Questions for a team

Related: prompt injection explained, AI agent integration risks, secrets management basics.

Limits

One OWASP page read in full; references and MITRE ATLAS entries unread. No vendor claims, no statistics. The page names techniques such as federated learning and homomorphic encryption without detail, and we do not add any.

By Omnistu.

Sources

Previous
Data and model poisoning explained: what it means for teams using AI models