Skip to main content
XenonStack

Autonomous Agents

XenonStack Blog Embodied Agents: Architecture and Governance in 2027

and | Published | Updated

Embodied Agents: Architecture and Governance in 2027
9:47

Key takeaway

  • Embodied agents explained: how generative agents, vision-language-action models and robots fit together, with controls and a 3 to 5 year outlook.

An embodied agent is an AI system with a body, real or simulated, that perceives its surroundings through sensors, decides what to do and acts on the world, then learns from the result. A generative agent is different: it is a software agent that uses a large language model to simulate believable human behaviour. The two ideas are often mixed up, and enterprise teams need to keep them apart because they carry very different risk.

Generative agents give an embodied system its memory, reflection and planning. A body adds sensors, actuators and the cost of being wrong in the physical world. This guide explains how the two fit together, what the research shows, which controls to put around them, and what is likely to change over the next three to five years. It builds on our guides to agentic workflows and edge AI for autonomous systems.

What is the difference between a generative agent and an embodied agent?

A generative agent reasons and plans in language. An embodied agent must also sense and act. Most real systems combine the two, with a language model supplying memory and planning and a body supplying perception and action.

DimensionGenerative agentEmbodied agent
What it isSoftware agent that uses a language model to simulate believable behaviourAgent with a physical or simulated body, sensors and actuators
What it acts onText, tools, a simulated world or business systemsObjects and spaces, through movement or manipulation
Main weaknessReliability of language model outputReliability plus physical safety and sensor error
Where it runsCloud or on-premises serversOften at the edge, close to the device
Typical governance needTool permissions, approvals, audit trailThe same, plus hard safety limits and physical-world testing

How do generative agents remember, reflect and plan?

The best-known description comes from the paper Generative Agents: Interactive Simulacra of Human Behavior (Park et al., arXiv 2304.03442, submitted 7 April 2023). The authors extend a large language model so that an agent keeps a full record of its experiences in natural language, turns those memories into higher-level reflections and retrieves them to plan its behaviour. They tested the design with 25 agents in an interactive sandbox inspired by The Sims, and an ablation study found that observation, planning and reflection each contribute significantly to believability. The agents lived in a simulated town, not in a physical body, which is why the paper is a model for the agent's mind rather than proof of safe action in the real world.

Two-column diagram mapping agent capabilities (memory, reflection and planning, tool access, physical action) to controls: decision record, simulation, least privilege, human approval and hard limits.

Memory and retrieval

The memory stream stores experience in natural language. Retrieval decides which records the model sees when it must act, so the quality of retrieval shapes the quality of behaviour.

Reflection

Reflection synthesises stored memories into higher-level inferences over time, which gives the agent a view of itself and others rather than a flat log.

Planning and reacting

Memories are retrieved dynamically to plan, for example the next day's activities. The plan is revised when the environment changes or a user gives a new instruction in natural language.

How do language models become embodied?

Two Google research papers from 2023 show the main routes. Both are research results, not product claims, and neither gives a basis for a production safety case on its own.

ApproachWhat it doesCurrent status
PaLM-E (arXiv 2303.03378, submitted 6 March 2023)Feeds images and robot state readings into a language model together with text, trained end to end. The largest version has 562B parameters and handles robotic manipulation planning, visual question answering and captioning.Research model described in a paper
RT-2 (arXiv 2307.15818, submitted 28 July 2023)A vision-language-action model that writes robot actions as text tokens and is co-fine-tuned on robot trajectories and web vision-language tasks. The authors report better generalisation to novel objects, based on 6k evaluation trials.Research model described in a paper
Generative agent architecture (Park et al.)Memory, reflection and planning around a language model, in a simulated townResearch prototype in simulation

Which design principles make an agent genuinely embodied?

Generative agents on their own produce surface-level behaviour in language. To derive meaning from goals and sub-goals, they need to be coupled with an embodied cognitive architecture. The principles below are the design framing our team uses when reviewing such systems.

PrincipleWhat it means in design
EmbodimentThe body and its sensors shape what the agent can perceive and do
HierarchyLow-level sensing and control sit under high-level reasoning and planning, with sub-goals in between
ConstructivismKnowledge is built and updated from experience and new input, not fixed in advance
SituatednessBehaviour depends on the context and environment, and on other agents
GenerativityBehaviour is not fully predefined, which allows new solutions and also new failures
Sub/symbolismNeural networks are combined with symbolic rules and logic
LearnabilityThe agent improves from feedback, so each change needs a release gate

Why affordances and goal-directed perception matter

Language models are built to produce fluent text, not to detect what an environment allows. An agent that grasps an object has to look for features that aid grasping. Perception is not the discovery of every property of a scene but the identification of properties relevant to the current goal. A system that lacks this will sound confident and still act on the wrong thing, which is why embodied designs add a perception layer and explicit goals instead of relying on a language model alone. Our guide to edge AI for autonomous vehicles shows the same split between perception and decision.

Where do embodied agents fit in the enterprise today?

Physical robots are already common in factories. The International Federation of Robotics reports in World Robotics 2025 (25 September 2025) that 542,000 industrial robots were installed in 2024 and that 4,664,000 were in operation. Most of those are programmed machines, not language-model agents, so the decision for a leader is where an agent layer adds value on top of proven automation.

UseWhen it fitsCurrent status
Simulation and testing of agent behaviourYou want to study multi-agent behaviour before any physical deploymentDemonstrated in research (Park et al.)
Language-guided robot manipulationTasks vary and new commands must be followed without retraining for each oneResearch results (PaLM-E, RT-2)
Agent layer above existing robots and devicesFixed automation exists and an agent must plan, escalate and log decisionsEngineering pattern; depends on the site and vendor
Fully autonomous action in shared human spacesOnly where hard safety limits, testing and approvals are in placeTreat as high risk; see the controls below

NexaStack describes itself on its site as an agentic operating platform for physical AI that runs across on-premises, cloud and hybrid environments, with monitoring, audit trails, policy controls and human oversight; see NexaStack for the detail.

What governance and security controls does an embodied agent need?

The OWASP GenAI project names Excessive Agency (LLM06:2025) as a vulnerability that lets an LLM-based system take damaging actions in response to unexpected, ambiguous or manipulated outputs. It lists three root causes: excessive functionality, excessive permissions and excessive autonomy. An embodied agent adds a physical consequence to each of them.

RiskControlSource or basis
Excessive functionalityGive the agent only the functions the task needs; avoid open-ended tools such as arbitrary command executionOWASP LLM06:2025
Excessive permissionsRun actions under a scoped identity with minimal rightsOWASP LLM06:2025
Excessive autonomyRequire human approval before high-impact actionsOWASP LLM06:2025
Trusting the model to decide what is allowedEnforce authorisation in the downstream system, not in the modelOWASP LLM06:2025 (complete mediation)
Physical harm from a wrong actionHard limits enforced outside the model, tested before release, with a stop pathOur design practice; not a standard quotation
Unclear accountabilityMap, measure and manage AI risk across the lifecycle, and keep a decision recordNIST AI Risk Management Framework 1.0 (26 January 2023) and the Generative AI Profile (26 July 2024)

OWASP adds that logging, monitoring and rate limiting do not prevent excessive agency, but they can limit the damage. For governance, AgentGRC says on its site that it lets organisations define which actions agents may take and checks them against policy before they reach core business systems, logging each decision with its context and approvals. Our guide to agentic AI risk management covers the control framework in more depth.

How should autonomy be set for an embodied agent?

Treat autonomy as a setting per agent and per action, not a property of the model. ElixirClaw says on its site that autonomy is set per agent, domain and action across four levels (L1 to L4), with humans setting boundaries and approving high-impact actions, and each decision logged as a decision trace. That pattern suits embodied work: start where a person approves every physical action, then move specific, well-tested actions to supervision of outcomes. See ElixirClaw for its own description. The same idea applies to software-only agents that operate screens, as in our guide to GUI agents.

What will change in embodied agents over the next 3 to 5 years?

Three things are on the record: the installed base of robots is growing, regulation of AI in products has dates, and the main risk framework is being revised. The table gives the sourced facts. The last row is our own judgement and is marked as such.

AreaWhat the source saysWhat it means for planning
Robot deploymentIFR expects installations to grow 6% to 575,000 in 2025 and to pass 700,000 by 2028 (World Robotics 2025)More physical endpoints for agents to coordinate, so fleet governance matters
EU AI Act, general applicationThe Act became applicable on 2 August 2026, with some exceptions, per the European CommissionCheck your use case against the Act now
EU AI Act, high-risk deadlinesHigh-risk obligations for Annex III use cases apply from 2 December 2027; for AI embedded in regulated products (Annex I) from 2 August 2028, after the AI Omnibus (in force 27 July 2026)Products with embedded AI have a longer runway, but design evidence takes time to build
NIST AI RMFNIST states that AI RMF 1.0 is being revised as part of the White House AI Action PlanMap controls to the framework in a way that survives revision
Language-guided robot controlRT-2 and PaLM-E show models that link language, vision and action (2023 papers)Our view: expect pilots of an agent layer above existing robots before open-ended autonomy

What is a sensible adoption sequence?

  1. Define the decisions. List which actions the agent may take, which need approval and which are forbidden.
  2. Simulate first. Test memory, planning and failure handling in simulation before any physical trial.
  3. Add a perception layer and hard limits. Keep safety limits outside the model.
  4. Pilot with a person approving every action. Record every decision and its context.
  5. Widen autonomy action by action. Use evidence from the pilot, not model confidence.
  6. Review against NIST and local regulation. Repeat when models, sensors or rules change.

When should you not use an embodied agent?

  • When fixed automation already does the job reliably and tasks do not vary.
  • When a wrong physical action cannot be stopped or reversed and there is no tested safety limit.
  • When you cannot log what the agent perceived and decided.
  • When the case rests on a research result with no testing in your environment.

The generative AI guide, the edge computing guide and the smart factory guide cover the surrounding architecture.

To map one physical-world decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

What should you plan for in 2027?

Rules for AI in physical products now have dates, and the European Commission page on the AI Act sets them out.

  • 2 December 2027: obligations for Annex III high-risk use cases apply, as extended by the AI Omnibus. Check whether any agent you plan to deploy falls under an Annex III use case (Commission AI Act page).
  • 2 August 2028: obligations for AI embedded in regulated products (Annex I) apply. The AI Omnibus entered into force on 27 July 2026, so robots and devices with embedded agents have a longer runway, but design evidence still takes time to build (same page).

Keep autonomy per action, keep hard safety limits outside the model, and keep a decision record from the first pilot.

Frequently asked questions about embodied agents

What is an embodied agent?

An embodied agent is an AI system with a physical or simulated body that senses its environment, decides and acts, and learns from the outcome.

Are generative agents the same as embodied agents?

No. A generative agent uses a language model with memory, reflection and planning to simulate believable behaviour, and the original study ran in a simulated town. An embodied agent adds a body, sensors and actuators.

What do PaLM-E and RT-2 show?

PaLM-E feeds images and robot state into a language model, and RT-2 writes robot actions as text tokens. Both are 2023 research papers, not production safety evidence.

What is the main security risk of an agent that can act?

OWASP calls it Excessive Agency: damaging actions caused by excessive functionality, permissions or autonomy. Least privilege, human approval and downstream authorisation reduce it.

What will change in embodied agents over the next 3 to 5 years?

IFR expects robot installations to pass 700,000 a year by 2028, EU AI Act high-risk obligations for embedded products apply from 2 August 2028, and NIST is revising its AI RMF. Plan for more devices and firmer rules.

Next Steps with Embodied Agents

Talk to our experts about implementing compound AI system, How Industries and different departments use Agentic Workflows and Decision Intelligence to Become Decision Centric. Utilizes AI to automate and optimize IT support and operations, improving efficiency and responsiveness.

 

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue