Key takeaway
- How personalised AI agents run on the Databricks lakehouse in 2026: Agent Bricks, Genie Agents, Unity Catalog governance, costs and an adoption plan.
Personalised AI agents for a Databricks lakehouse are agents that answer questions, check data and run routine management tasks for a specific person or team, while seeing only what that person is allowed to see. In 2026 Databricks documents three building blocks for this: Genie Agents for no-code, domain-specific questions over curated data, Agent Bricks for agents that developers write in code, and Unity Catalog with Unity Gateway to govern the data, models and tools those agents use.
The original version of this post centred on LakehouseIQ, which Databricks announced in 2023. That name no longer appears in the current Databricks documentation, so this guide maps the old idea to today's offerings, shows which one fits which job, and sets out the governance, cost and adoption choices that enterprise data and AI leaders need to make.
What is a personalised AI agent on the Databricks lakehouse?
A lakehouse keeps structured, semi-structured and unstructured data in open storage with warehouse-style management on top. An AI agent for data management adds a reasoning loop: it reads a request, chooses tools such as SQL queries or APIs, acts, and reports back. "Personalised" means the agent works in the context of one user or team: their questions, their business terms and, most importantly, their permissions.
On Databricks, that personal context comes from the platform rather than from the agent. Genie Agents are built from datasets registered in Unity Catalog, plus example SQL, SQL expressions for business terms and text instructions written by data analysts (Databricks, Genie Agents). Genie One, the business-user interface previously called Databricks One, does not have a separate permission system and enforces existing Unity Catalog and workspace permissions (Databricks, Genie One).
For leaders, the practical point is simple: an agent on the lakehouse is only as personal, and only as safe, as the catalogue, permissions and business definitions underneath it. Our guide to a governed data lake covers that foundation.
What happened to LakehouseIQ, and what has changed by 2026?
Databricks introduced LakehouseIQ on 28 June 2023 as "a knowledge engine that learns the unique nuances of your business and data", stating that it was "built on and governed by Unity Catalog" and that features would roll out through the year (Databricks blog, June 2023). The current Databricks documentation does not list a product with that name. The jobs LakehouseIQ was meant to do are now documented under other names.
| 2023 idea | Where it is documented now | Status in the docs (read November 2026) |
|---|---|---|
| Ask questions of business data in plain language | Genie Agents (formerly Genie spaces) and Genie One (formerly Databricks One) | No overall release stage stated; some parts in Public Preview or Beta |
| Assistant that understands your data in notebooks and the SQL editor | Genie Code is named among Databricks AI assistive features | Listed on the trust and safety page |
| Build custom agents grounded in enterprise data | Agent Bricks; the 2024 Agent Framework announcement is now titled "Agent Bricks Custom Agents" | Announced in Beta in June 2025; current page states no stage |
| Governance of data and AI assets | Unity Catalog and Unity Gateway | Unity Catalog also available as open source |
| Knowledge Assistant and Supervisor Agent | Agent Bricks FAQ | Described as legacy offerings |
Two smaller naming changes matter for anyone reading older material. Databricks' Data + AI Summit 2025 post introduced "Agent Bricks in Beta" (Databricks, June 2025), and its July 2024 post on the Mosaic AI Agent Framework is now titled "Announcing Agent Bricks Custom Agents and Agent Evaluation" (Databricks blog). The current Agent Bricks pages do not use the Mosaic AI name, so new architecture documents should use the current product names. The direction is clear: agents are now treated as governed assets alongside tables and models, and the same catalogue that controls data also controls the models and tools agents call.
Which Databricks offering fits which agent job?
Most enterprises will use more than one of these. The table below is a starting point for choosing, based on what each page documents.
| Offering | What it is | When it fits | What to check |
|---|---|---|---|
| Genie Agents | Domain-specific natural-language chat that returns SQL, result tables and charts | A team asks recurring questions of a known set of curated tables | Quality depends on example SQL, business terms, instructions and verified answers |
| Genie One | One chat entry point for business users across dashboards, Genie Agents, apps and tables | Broad self-service for business users with the Consumer entitlement | Some features are Beta or Public Preview; tracing is an admin setting |
| Agent Bricks | Developer platform: bring any framework or harness, choose the model, add tools, state and a managed runtime | Multi-step agents that call APIs, MCP servers or run code | Engineering ownership, evaluation with MLflow, deployment through Agent Runtime |
| Unity Gateway | Governance for runtime calls between models, agents, MCP servers and tools | Any agent that calls models or tools you need to control | Service policies, consumption limits, usage system tables |
| Unity Catalog | Governance for tables, views, volumes, functions, models and services | Always; it is the permission and lineage layer for everything above | Privileges, attribute-based policies, row and column filters, audit log system table |
Agent Bricks lists its components as the agent loop (for example LangGraph or the OpenAI Agents SDK), the model accessed through Unity Gateway and Foundation Model APIs, tools such as MCP servers, AI Search and Genie Agents, agent state backed by Lakebase, a durable agent server and the managed Agent Runtime, with MLflow and Unity Catalog for observability and governance (Databricks, What is Agent Bricks?).
How do personalised agents stay inside your governance rules?
The Databricks documentation describes four controls that together decide what an agent can see and do. Leaders should ask for each one by name in any design review.
Identity-bound queries. SQL that Genie One runs executes as the user, so Unity Catalog privileges, row filters and column masks apply (Genie One).
Permission-aware model input. Databricks states that all data sent to AI assistive feature models respects Unity Catalog permissions, while metadata such as table and column names and descriptions is sent to produce relevant answers (trust and safety).
Governed models and tools. Unity Gateway governs runtime interactions between models, agents, MCP servers and tools. Service policies implement guardrails, consumption limits manage capacity and cost, and requests, tokens and latency are tracked in system tables (Unity Gateway).
Traces and lineage. MLflow Tracing records each agent step, and Unity Catalog provides lineage and an audit log system table (Agent Bricks, Unity Catalog). For Genie One, admins can enable tracing to a unified trace table; deleting a conversation does not delete its traces.
These controls govern data access and runtime calls. They do not, on their own, record why an agent took a business decision or which policy version allowed it. Many enterprises add a decision layer for that: ElixirData Context OS keeps governed context, versioned policy and decision traces in one decision layer above AI agents, and AgentGRC keeps autonomous agent actions inside policy, with the audit trail risk teams need.
Which lakehouse management jobs can agents take on?
The original post described processing, query, security, insight and resource agents in general terms. The table below ties each job to a documented Databricks capability and to the human control we recommend keeping. It describes patterns, not packaged Databricks agents.
| Management job | What the agent does | Documented building block | Human control to keep |
|---|---|---|---|
| Answer business questions | Turns a plain-language question into SQL and returns results with the query shown | Genie Agents, Genie One | Analyst curates tables, terms and verified answers |
| Watch data quality | Summarises quality signals and drafts a ticket for the table owner | Data quality monitoring in Unity Catalog | Owner approves any fix to pipelines or data |
| Review access | Reads audit records and flags unusual access for review | Audit log system table | Security team decides on revocation |
| Track AI usage and cost | Reports token use, latency and spend by team or agent | Unity Gateway usage system tables, consumption limits, budgets | Platform owner sets limits and budgets |
| Run multi-step tasks | Calls APIs, MCP tools or a sandbox to complete a task end to end | Agent Bricks with Agent Runtime and MLflow Tracing | Approval step before any write to production systems |
For data teams that want a ready agent for plain-language analytics, AgentAnalyst lets teams ask questions of enterprise data in plain language and get real-time insights. Related XenonStack guides cover AI-powered data quality monitoring in Databricks and auto-scaling clusters for AI inference.
Video: Governed AI Analytics, Compliant Field Insights in One Hour (ElixirClaw), showing governed, plain-language analytics over enterprise data.
What should leaders check on cost, security and risk?
Cost
Agents add model, compute and storage cost on top of queries. Use Unity Gateway consumption limits, budgets and usage system tables from the first pilot, and report cost per answered question or completed task rather than per token.
Security
Check that every agent runs with the end user's identity or a narrowly scoped service identity, that MCP connections use the user's own OAuth credentials where Genie One documents this, and that sensitive columns are masked in Unity Catalog before any agent can query them.
Risk and compliance
Decide where traces live and how long they are kept, because Genie One traces survive conversation deletion. Note also that account-level Genie One metadata may be processed in the United States, which admins can switch off, according to the Genie One page. Align both with your data-residency and retention policies.
How should an enterprise adopt agents on the lakehouse?
When our team scopes this work with enterprise buyers, the first two questions are which decision or question they want fixed and which data platform holds the data. Starting there keeps the pilot small and measurable.
Pick one decision and one owner. For example, weekly margin questions for a finance team.
Curate the data. Register the tables in Unity Catalog, add descriptions, owners, row filters and column masks.
Start with a Genie Agent. Add example SQL, business terms and verified answers, then benchmark it on real questions.
Move to Agent Bricks only when needed. Use coded agents when the task needs several tools, APIs or actions.
Govern runtime calls. Put models and MCP servers behind Unity Gateway with limits and usage tracking.
Add approvals and decision records. Require a human step before writes, and record which policy allowed each action.
Scale by domain. Repeat for the next team, reusing the same catalogue and policies.
For agents that must act across systems outside the lakehouse, such as ticketing or ERP, ElixirClaw is an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control.
When should you not use an agent on the lakehouse?
When a dashboard answers it. Fixed, repeated KPIs are cheaper and more consistent in an AI/BI dashboard.
When definitions are unsettled. If teams disagree on what "active customer" means, an agent will return confident but contested numbers. Fix the semantics first.
When permissions are coarse. If everyone can read everything, an agent makes that exposure easier to exploit. Tighten Unity Catalog first.
When the action is irreversible. Keep a human approval step for deletes, payments and production changes.
If you are comparing lakehouse platforms, our guide to the AWS data lakehouse covers the equivalent service choices on AWS.
To map one lakehouse decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.
Frequently asked questions about AI agents on Databricks
What is a personalised AI agent on the Databricks lakehouse?
It is an agent that answers questions or takes actions over lakehouse data on behalf of a specific person or team, using only the data, tools and models that person is allowed to use. On Databricks, Unity Catalog permissions decide what the agent can see.
Is LakehouseIQ still a Databricks product?
Databricks announced LakehouseIQ in June 2023 as a knowledge engine built on and governed by Unity Catalog. The current Databricks documentation does not list a product called LakehouseIQ. Natural-language access to data is now documented under Genie Agents and Genie One.
What is the difference between Agent Bricks and Genie Agents?
Agent Bricks is the Databricks platform for developers who write agent code with a framework of their choice. Genie Agents, formerly Genie spaces, are no-code, domain-specific chat interfaces that analysts build over Unity Catalog datasets.
Do agents on Databricks bypass Unity Catalog permissions?
Not according to the Databricks documentation. Genie One runs SQL as the user, so Unity Catalog privileges, row filters and column masks apply, and data sent to AI assistive feature models respects Unity Catalog permissions.
How do you control which models and tools an agent can call?
Databricks documents Unity Gateway for this. It governs runtime calls between models, agents, MCP servers and tools, with service policies for guardrails, consumption limits and usage tracking through system tables.
Where should an enterprise start with lakehouse agents?
Start with one decision or question that a named team owns, curate the Unity Catalog tables it needs, build a Genie Agent or a coded agent for that scope, evaluate it on real questions, and only then widen access or allow actions.
Next Steps to Implement AI Agents in Databricks Lakehouse
Talk to our experts about implementing AI agents in the Databricks Lakehouse. Discover how industries and different departments leverage Agentic Workflows and Decision Intelligence to enhance data-driven decision-making. Learn how AI automates and optimizes data processing, analytics, and IT operations, improving efficiency, scalability, and responsiveness.
Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE