Skip to main content
XenonStack

Cyber Security

XenonStack Blog LLM Security: Risks, Controls and OWASP Top 10

and | Published | Updated

Key takeaway

  • LLM security explained for CISOs and architects: the OWASP 2025 risks, a layered control map, agent risks and a six-step adoption plan for enterprises.

LLM security is the set of controls that stop a large language model, and the applications and agents built around it, from leaking data, being manipulated or taking actions it should not. The reference most security teams start from is the OWASP Top 10 for LLM Applications, whose 2025 edition lists ten risks, from prompt injection to unbounded consumption. Most of them are failures of the application around the model, not of the model itself.

This guide is for CISOs, CTOs and architects who need a practical control set. It covers the main risks, a control map, what changes when LLMs become agents, and a sequence for adoption. For the technology background, see our guide to large language models, and for runtime visibility see LLM observability.

What are the main security risks of LLM applications?

The OWASP Top 10 for LLM Applications (2025 edition) names ten risks. The table groups them by what a leader needs to decide, with the practical control for each. The control column is our guidance, not OWASP wording.

Two-column diagram pairing four LLM risks (prompt injection, information disclosure, supply chain and poisoning, excessive agency) with their first controls
OWASP 2025 riskWhat it means in practiceFirst control to put in place
LLM01 Prompt InjectionCrafted input, or content the model reads, changes its behaviourTreat all model input as untrusted; separate instructions from data; test with adversarial prompts
LLM02 Sensitive Information DisclosureThe model reveals personal, confidential or proprietary dataClassify data before it reaches the model; filter outputs; enforce access at retrieval
LLM03 Supply ChainCompromised models, datasets or dependenciesKeep an inventory of models and sources; verify provenance; scan artefacts before use
LLM04 Data and Model PoisoningTampered training, fine-tuning or embedding dataRestrict and log who can change training data; review data sources
LLM05 Improper Output HandlingModel output is passed to other systems without checksValidate and encode output before it reaches browsers, databases or tools
LLM06 Excessive AgencyThe model or agent has more tools, permissions or autonomy than neededLeast privilege per tool; human approval for high-impact actions
LLM07 System Prompt LeakageHidden instructions or secrets are exposedKeep secrets and authorisation logic out of prompts
LLM08 Vector and Embedding WeaknessesRetrieval stores leak or are manipulatedApply access control at document level in the vector store
LLM09 MisinformationConfident but wrong output is acted onGround answers in sources; evaluate; keep a human in the loop for decisions
LLM10 Unbounded ConsumptionAbuse drives cost, denial of service or model extractionRate limits, quotas and spend alerts per user and per key

How do these risks change when LLMs become agents?

An assistant that only answers questions can leak or mislead. An agent that can call tools can also change records, send messages and move money. That is why excessive agency (LLM06) and prompt injection (LLM01) combine into the highest-impact scenario: an injected instruction arrives in an email or document, and the agent acts on it with its own permissions.

For agent platforms the controls shift from filtering text to governing actions. Give each agent a narrow identity, scope every tool to the minimum permission, isolate tenants, and require a human decision for high-severity actions. Our own security work for LLM and agent systems follows this pattern: adversarial testing, prompt-injection assessment, tool-misuse testing and remediation.

Governing actions needs a decision layer that records what the agent was allowed to do and why. AgentGRC maps policies and approval rules to agent actions before they reach business systems, so each action can be allowed, changed, escalated or blocked and the decision is recorded. It runs on ElixirData Context OS, which keeps governed context and decision traces together. For building and governing agents with humans in control, see ElixirClaw.

Which controls protect an LLM application end to end?

Think in layers. No single filter stops prompt injection, so defence in depth and zero trust apply: assume any layer can fail and limit what a failure can reach.

LayerControlWhy it matters
Identity and accessStrong authentication including multi-factor, role-based access, least privilege for models, data and toolsLimits who can query, change or export the model and its data
DataEncryption in transit and at rest, key management, classification, retention limitsReduces exposure if storage or a pipeline is breached
Model supply chainAuthenticated model registry, scanning before retraining, fine-tuning, evaluation or inference, checksums and signaturesProtects integrity and authenticity of model files
Input and outputInput validation, output filtering and encoding, content policiesReduces injection and improper output handling
RuntimeLogging of prompts, tool calls and decisions; anomaly alerts; rate limitsDetects abuse and supports investigation
ProcessSecure coding, code review, patching, vendor assessment, incident response and backupKeeps the surrounding software and third parties in check

Evidence matters as much as prevention. Keep an audit trail of what each model and agent did, under which policy and who approved it. The Audit Agent assembles the history, policy version and approvals for a past decision, but only for decisions made through Context OS and within the retention policy. Our guide to generative AI data security risks covers the data side in more depth.

How should an enterprise adopt LLM security step by step?

The NIST AI Risk Management Framework is voluntary guidance for managing AI risk, and NIST published a Generative AI Profile (NIST-AI-600-1) as a companion in July 2024. It is a useful frame for governance. The sequence below is our suggested order of work.

StepActionOwner
1. InventoryList every model, prompt, data source, tool and vendor in use, including shadow useSecurity and platform
2. ClassifyRate each use case by data sensitivity and by what actions it can takeCISO and risk
3. Baseline controlsAccess control, encryption, logging, rate limits and secret handlingPlatform and engineering
4. TestAdversarial and prompt-injection testing before launch and after each major changeSecurity and QA
5. Govern actionsApproval rules for high-impact agent actions, with recorded decisionsRisk and product
6. Monitor and respondRuntime monitoring, incident playbooks, regular review of logsSecurity operations

Related reading: AI in cyber security, dynamic application security testing and insider threats.

What are the trade-offs, and when are controls not enough?

Filters block some attacks and also some legitimate use, so measure false positives. Human approval slows agents, so reserve it for high-severity actions. Open-source and third-party models cut cost but widen the supply chain, so verify provenance. Embedded and edge deployments have fewer resources for scanning and monitoring, so decide up front what must run locally. None of these controls makes a model correct, so evaluation and human review stay in place for consequential decisions.

To map one decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions

What is LLM security?

LLM security is the set of controls that protect a large language model, its data and the applications and agents around it from leakage, manipulation and misuse. The OWASP Top 10 for LLM Applications 2025 is a common reference list.

What is prompt injection?

Prompt injection is the first risk on the OWASP 2025 list (LLM01). Crafted input, or content the model reads, changes the model's behaviour. Treat all model input as untrusted and test with adversarial prompts.

Why is excessive agency a concern?

OWASP lists excessive agency as LLM06. When a model or agent has more tools, permissions or autonomy than it needs, an injected or mistaken instruction can cause real actions. Use least privilege and human approval for high-impact actions.

How do we protect models in storage and transit?

Store models in an authenticated registry, scan them before use, encrypt them at rest and in transit, and verify checksums and signatures to confirm integrity and authenticity.

Does NIST cover generative AI?

Yes. NIST released the Generative AI Profile (NIST-AI-600-1) on 26 July 2024 as a companion to the AI Risk Management Framework. Both are voluntary guidance.

Where should a team start?

Start with an inventory of models, data sources, tools and vendors, classify each use case by risk, put baseline access, encryption and logging controls in place, then test adversarially before launch.

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue