Skip to main content
XenonStack

Elixirdata

XenonStack Blog Databricks AI Agents: Agent Bricks, Genie and Unity Catalog

and | Published | Updated

Databricks AI Agents: Agent Bricks, Genie and Unity Catalog
15:24

Key takeaway

  • How personalised AI agents run on the Databricks lakehouse in 2026: Agent Bricks, Genie Agents, Unity Catalog governance, costs and an adoption plan.

Personalised AI agents for a Databricks lakehouse are agents that answer questions, check data and run routine management tasks for a specific person or team, while seeing only what that person is allowed to see. In 2026 Databricks documents three building blocks for this: Genie Agents for no-code, domain-specific questions over curated data, Agent Bricks for agents that developers write in code, and Unity Catalog with Unity Gateway to govern the data, models and tools those agents use.

The original version of this post centred on LakehouseIQ, which Databricks announced in 2023. That name no longer appears in the current Databricks documentation, so this guide maps the old idea to today's offerings, shows which one fits which job, and sets out the governance, cost and adoption choices that enterprise data and AI leaders need to make.

What is a personalised AI agent on the Databricks lakehouse?

A lakehouse keeps structured, semi-structured and unstructured data in open storage with warehouse-style management on top. An AI agent for data management adds a reasoning loop: it reads a request, chooses tools such as SQL queries or APIs, acts, and reports back. "Personalised" means the agent works in the context of one user or team: their questions, their business terms and, most importantly, their permissions.

On Databricks, that personal context comes from the platform rather than from the agent. Genie Agents are built from datasets registered in Unity Catalog, plus example SQL, SQL expressions for business terms and text instructions written by data analysts (Databricks, Genie Agents). Genie One, the business-user interface previously called Databricks One, does not have a separate permission system and enforces existing Unity Catalog and workspace permissions (Databricks, Genie One).

For leaders, the practical point is simple: an agent on the lakehouse is only as personal, and only as safe, as the catalogue, permissions and business definitions underneath it. Our guide to a governed data lake covers that foundation.

What happened to LakehouseIQ, and what has changed by 2026?

Databricks introduced LakehouseIQ on 28 June 2023 as "a knowledge engine that learns the unique nuances of your business and data", stating that it was "built on and governed by Unity Catalog" and that features would roll out through the year (Databricks blog, June 2023). The current Databricks documentation does not list a product with that name. The jobs LakehouseIQ was meant to do are now documented under other names.

Two-column diagram mapping 2023 LakehouseIQ ideas (plain-language questions, data-aware assistant, custom agents, governance) to Genie Agents and Genie One, Genie Code, Agent Bricks, and Unity Catalog with Unity Gateway.
2023 ideaWhere it is documented nowStatus in the docs (read November 2026)
Ask questions of business data in plain languageGenie Agents (formerly Genie spaces) and Genie One (formerly Databricks One)No overall release stage stated; some parts in Public Preview or Beta
Assistant that understands your data in notebooks and the SQL editorGenie Code is named among Databricks AI assistive featuresListed on the trust and safety page
Build custom agents grounded in enterprise dataAgent Bricks; the 2024 Agent Framework announcement is now titled "Agent Bricks Custom Agents"Announced in Beta in June 2025; current page states no stage
Governance of data and AI assetsUnity Catalog and Unity GatewayUnity Catalog also available as open source
Knowledge Assistant and Supervisor AgentAgent Bricks FAQDescribed as legacy offerings

Two smaller naming changes matter for anyone reading older material. Databricks' Data + AI Summit 2025 post introduced "Agent Bricks in Beta" (Databricks, June 2025), and its July 2024 post on the Mosaic AI Agent Framework is now titled "Announcing Agent Bricks Custom Agents and Agent Evaluation" (Databricks blog). The current Agent Bricks pages do not use the Mosaic AI name, so new architecture documents should use the current product names. The direction is clear: agents are now treated as governed assets alongside tables and models, and the same catalogue that controls data also controls the models and tools agents call.

Which Databricks offering fits which agent job?

Most enterprises will use more than one of these. The table below is a starting point for choosing, based on what each page documents.

OfferingWhat it isWhen it fitsWhat to check
Genie AgentsDomain-specific natural-language chat that returns SQL, result tables and chartsA team asks recurring questions of a known set of curated tablesQuality depends on example SQL, business terms, instructions and verified answers
Genie OneOne chat entry point for business users across dashboards, Genie Agents, apps and tablesBroad self-service for business users with the Consumer entitlementSome features are Beta or Public Preview; tracing is an admin setting
Agent BricksDeveloper platform: bring any framework or harness, choose the model, add tools, state and a managed runtimeMulti-step agents that call APIs, MCP servers or run codeEngineering ownership, evaluation with MLflow, deployment through Agent Runtime
Unity GatewayGovernance for runtime calls between models, agents, MCP servers and toolsAny agent that calls models or tools you need to controlService policies, consumption limits, usage system tables
Unity CatalogGovernance for tables, views, volumes, functions, models and servicesAlways; it is the permission and lineage layer for everything abovePrivileges, attribute-based policies, row and column filters, audit log system table

Agent Bricks lists its components as the agent loop (for example LangGraph or the OpenAI Agents SDK), the model accessed through Unity Gateway and Foundation Model APIs, tools such as MCP servers, AI Search and Genie Agents, agent state backed by Lakebase, a durable agent server and the managed Agent Runtime, with MLflow and Unity Catalog for observability and governance (Databricks, What is Agent Bricks?).

How do personalised agents stay inside your governance rules?

The Databricks documentation describes four controls that together decide what an agent can see and do. Leaders should ask for each one by name in any design review.

  • Identity-bound queries. SQL that Genie One runs executes as the user, so Unity Catalog privileges, row filters and column masks apply (Genie One).

  • Permission-aware model input. Databricks states that all data sent to AI assistive feature models respects Unity Catalog permissions, while metadata such as table and column names and descriptions is sent to produce relevant answers (trust and safety).

  • Governed models and tools. Unity Gateway governs runtime interactions between models, agents, MCP servers and tools. Service policies implement guardrails, consumption limits manage capacity and cost, and requests, tokens and latency are tracked in system tables (Unity Gateway).

  • Traces and lineage. MLflow Tracing records each agent step, and Unity Catalog provides lineage and an audit log system table (Agent Bricks, Unity Catalog). For Genie One, admins can enable tracing to a unified trace table; deleting a conversation does not delete its traces.

These controls govern data access and runtime calls. They do not, on their own, record why an agent took a business decision or which policy version allowed it. Many enterprises add a decision layer for that: ElixirData Context OS keeps governed context, versioned policy and decision traces in one decision layer above AI agents, and AgentGRC keeps autonomous agent actions inside policy, with the audit trail risk teams need.

Which lakehouse management jobs can agents take on?

The original post described processing, query, security, insight and resource agents in general terms. The table below ties each job to a documented Databricks capability and to the human control we recommend keeping. It describes patterns, not packaged Databricks agents.

Management jobWhat the agent doesDocumented building blockHuman control to keep
Answer business questionsTurns a plain-language question into SQL and returns results with the query shownGenie Agents, Genie OneAnalyst curates tables, terms and verified answers
Watch data qualitySummarises quality signals and drafts a ticket for the table ownerData quality monitoring in Unity CatalogOwner approves any fix to pipelines or data
Review accessReads audit records and flags unusual access for reviewAudit log system tableSecurity team decides on revocation
Track AI usage and costReports token use, latency and spend by team or agentUnity Gateway usage system tables, consumption limits, budgetsPlatform owner sets limits and budgets
Run multi-step tasksCalls APIs, MCP tools or a sandbox to complete a task end to endAgent Bricks with Agent Runtime and MLflow TracingApproval step before any write to production systems

For data teams that want a ready agent for plain-language analytics, AgentAnalyst lets teams ask questions of enterprise data in plain language and get real-time insights. Related XenonStack guides cover AI-powered data quality monitoring in Databricks and auto-scaling clusters for AI inference.

Video: Governed AI Analytics, Compliant Field Insights in One Hour (ElixirClaw), showing governed, plain-language analytics over enterprise data.

What should leaders check on cost, security and risk?

Cost

Agents add model, compute and storage cost on top of queries. Use Unity Gateway consumption limits, budgets and usage system tables from the first pilot, and report cost per answered question or completed task rather than per token.

Security

Check that every agent runs with the end user's identity or a narrowly scoped service identity, that MCP connections use the user's own OAuth credentials where Genie One documents this, and that sensitive columns are masked in Unity Catalog before any agent can query them.

Risk and compliance

Decide where traces live and how long they are kept, because Genie One traces survive conversation deletion. Note also that account-level Genie One metadata may be processed in the United States, which admins can switch off, according to the Genie One page. Align both with your data-residency and retention policies.

How should an enterprise adopt agents on the lakehouse?

When our team scopes this work with enterprise buyers, the first two questions are which decision or question they want fixed and which data platform holds the data. Starting there keeps the pilot small and measurable.

  1. Pick one decision and one owner. For example, weekly margin questions for a finance team.

  2. Curate the data. Register the tables in Unity Catalog, add descriptions, owners, row filters and column masks.

  3. Start with a Genie Agent. Add example SQL, business terms and verified answers, then benchmark it on real questions.

  4. Move to Agent Bricks only when needed. Use coded agents when the task needs several tools, APIs or actions.

  5. Govern runtime calls. Put models and MCP servers behind Unity Gateway with limits and usage tracking.

  6. Add approvals and decision records. Require a human step before writes, and record which policy allowed each action.

  7. Scale by domain. Repeat for the next team, reusing the same catalogue and policies.

For agents that must act across systems outside the lakehouse, such as ticketing or ERP, ElixirClaw is an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control.

When should you not use an agent on the lakehouse?

  • When a dashboard answers it. Fixed, repeated KPIs are cheaper and more consistent in an AI/BI dashboard.

  • When definitions are unsettled. If teams disagree on what "active customer" means, an agent will return confident but contested numbers. Fix the semantics first.

  • When permissions are coarse. If everyone can read everything, an agent makes that exposure easier to exploit. Tighten Unity Catalog first.

  • When the action is irreversible. Keep a human approval step for deletes, payments and production changes.

If you are comparing lakehouse platforms, our guide to the AWS data lakehouse covers the equivalent service choices on AWS.

To map one lakehouse decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions about AI agents on Databricks

What is a personalised AI agent on the Databricks lakehouse?

It is an agent that answers questions or takes actions over lakehouse data on behalf of a specific person or team, using only the data, tools and models that person is allowed to use. On Databricks, Unity Catalog permissions decide what the agent can see.

Is LakehouseIQ still a Databricks product?

Databricks announced LakehouseIQ in June 2023 as a knowledge engine built on and governed by Unity Catalog. The current Databricks documentation does not list a product called LakehouseIQ. Natural-language access to data is now documented under Genie Agents and Genie One.

What is the difference between Agent Bricks and Genie Agents?

Agent Bricks is the Databricks platform for developers who write agent code with a framework of their choice. Genie Agents, formerly Genie spaces, are no-code, domain-specific chat interfaces that analysts build over Unity Catalog datasets.

Do agents on Databricks bypass Unity Catalog permissions?

Not according to the Databricks documentation. Genie One runs SQL as the user, so Unity Catalog privileges, row filters and column masks apply, and data sent to AI assistive feature models respects Unity Catalog permissions.

How do you control which models and tools an agent can call?

Databricks documents Unity Gateway for this. It governs runtime calls between models, agents, MCP servers and tools, with service policies for guardrails, consumption limits and usage tracking through system tables.

Where should an enterprise start with lakehouse agents?

Start with one decision or question that a named team owns, curate the Unity Catalog tables it needs, build a Genie Agent or a coded agent for that scope, evaluate it on real questions, and only then widen access or allow actions.

Next Steps to Implement AI Agents in Databricks Lakehouse

Talk to our experts about implementing AI agents in the Databricks Lakehouse. Discover how industries and different departments leverage Agentic Workflows and Decision Intelligence to enhance data-driven decision-making. Learn how AI automates and optimizes data processing, analytics, and IT operations, improving efficiency, scalability, and responsiveness.

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue