Skip to main content
XenonStack

Embedded Analytics

XenonStack Blog Amazon Lex Analytics: Logs, Metrics and Dashboards

| Published | Updated

Key takeaway

  • Learn how Amazon Lex V2 conversation logs, analytics dashboards and CloudWatch metrics show where your bot fails, and how to turn them into fixes.

Amazon Lex analytics is the practice of using conversation logs, built in analytics dashboards and CloudWatch metrics to see how a bot performs and where it fails. It turns each session into evidence for improving intents, slots and fulfilment.

This guide is for developers and product owners running Amazon Lex bots who want to know what data exists, where it is stored and how to act on it. It covers versions, logs, dashboards, metrics, privacy and what changes when bots become agents.

What is conversation analytics in Amazon Lex?

Amazon Lex is the AWS service for building voice and text bots. Conversation analytics is what you do with the records those bots leave behind: which intents users trigger, which utterances the bot misses, where sessions end and how the runtime behaves. It matters because a bot that nobody reviews stops improving. For the wider AWS picture, see our guide to real time analytics on AWS.

Conversational data is data extracted from a digital conversation, across channels such as chat, email and calls. The platform guidance below is taken from the AWS documentation for Amazon Lex V2, and it explains where each kind of data comes from.

Which version of Amazon Lex should you use?

Use Lex V2. The AWS migration guide says AWS is no longer adding new features to V1 and strongly recommends V2 for all new bots. V2 also supports multiple languages inside one bot, where V1 needed a separate bot per language.

If you are moving an older bot, check what carries over. The same page says migration copies custom intents and slot types but does not migrate aliases, Lambda functions, conversation log settings or messaging channels such as Slack, so logging has to be set up again on the new bot. Everything in this guide describes V2.

What data does Amazon Lex capture?

The Lex V2 conversation logs page explains that text logs hold text input, transcripts of audio input and metadata such as intent name and slot values, and are stored in CloudWatch Logs. Audio logs hold the audio input and are stored in Amazon S3. Logs are configured per alias, and each alias can have its own text and audio settings.

Two column diagram. Data sources: text logs, audio logs, analytics dashboards, CloudWatch metrics. Review actions: find misses, fix design, set alarms, protect data.
SourceWhat it holdsWhere it livesTypical use
Text conversation logsUser text, transcripts, intent name, slot valuesCloudWatch LogsFind missed utterances and failing intents
Audio logsAudio input from voice sessionsAmazon S3Review speech recognition problems
Analytics dashboardsSuccess and failure metrics, conversation flowsLex V2 consoleSpot patterns without writing queries
CloudWatch metricsRequest counts, errors, latency, throttlingAWS/Lex namespaceAlarm on runtime health

A session has a session ID, and you can use it to retrieve the full transcript of the conversation, including user utterances and bot responses. Logs can be encrypted with customer managed AWS KMS keys.

How do you analyse Lex conversations?

Start in the console. The Lex V2 analytics page describes dashboards that evaluate bot performance through success and failure rates and visualise conversation flows. It lists an overview, a conversation dashboard and a performance dashboard covering intent and utterance metrics, with filters to find problems involving intents, slots, utterances and conversations.

For runtime health, the CloudWatch metrics page lists metrics in the AWS/Lex namespace, including RuntimeRequestCount, RuntimeUserErrors, RuntimeSystemErrors, RuntimeThrottledEvents and RuntimeLambdaErrors. CloudWatch keeps these statistics for 15 months and supports alarms that notify you when a threshold is crossed. Note that the page spells one latency metric RuntimeSucessfulRequestLatency, so copy the name exactly when you build alarms.

If you want near real time pipelines or long term storage outside CloudWatch, a streaming service such as Amazon Kinesis can sit downstream of exported logs. The pages read for this guide do not cover sentiment scoring, so confirm in the current documentation which service you will use for it before promising it to stakeholders.

How should you design bots to produce useful data?

Analytics quality follows design quality. Define each intent as one goal the bot can complete, and each slot type as the data it needs, such as a date or a city. Clear intents and slot types give cleaner logs, and sample utterances that reflect how people really phrase requests make misses easier to diagnose. Keep prompts short and give the user a clear first message.

Fulfilment matters too. When a bot calls AWS Lambda or an external system, errors there show up in the Lambda error metric, so you can separate language understanding problems from back end problems. For product background on building bots, see our guide to enterprise chatbot platforms.

What privacy and compliance steps apply?

Because slot values are written to the logs, treat the logs as sensitive. Encrypt them with KMS, restrict who can read the log group and bucket, and set retention to match your policy. AWS states that conversation logs cannot be used with a bot subject to the Children's Online Privacy Protection Act (COPPA). Check the rules that apply to your sector and region with your compliance team.

How do AI agents change conversation analytics?

A conventional bot answers within the intents it was given. An AI agent plans steps, calls tools and may hand a case to a person, so the log you need is a decision trail: what the agent knew, which tool it called, and why it escalated. That trail is more useful when intents, outcomes and policies share one vocabulary. XenonStack's ElixirClaw is the AI agents computing platform for teams building and running agents, and the same review habits in this guide, such as sampling failed sessions and testing changes against a fixed set of utterances, apply to agents. See also our overview of conversational AI, machine learning and A/B testing tools for comparing conversation flows.

QuestionRule based botAI agent
What is loggedIntent, slots, responsePlan, tool calls, outcome, escalation
What you tuneUtterances and slot typesInstructions, tools, policies, evaluation sets
Failure signalMissed utteranceWrong action, loop, unsafe or unapproved step

What should you do first?

  1. Confirm the bot runs on Lex V2 and plan a migration if it does not.
  2. Enable text conversation logs on the production alias, with KMS encryption.
  3. Open the analytics dashboards and list the five intents or utterances that fail most.
  4. Create CloudWatch alarms for user errors, system errors and throttling.
  5. Review a sample of failed sessions weekly and record each change you make.

Related reading: data intelligence solutions.

To map one decision your agents make, and the data and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions

Does Amazon Lex have built in analytics?

Yes. The Lex V2 console includes analytics dashboards for overview, conversations and performance, with filters for intents, slots, utterances and conversations.

Where are Amazon Lex conversation logs stored?

Text logs are stored in CloudWatch Logs and audio logs in Amazon S3. They are configured per bot alias and can be encrypted with customer managed KMS keys.

Should new bots use Lex V1 or V2?

Use V2. AWS says it is no longer adding features to V1 and strongly recommends V2 for all new bots.

Do Lex logs contain sensitive data?

They can. Slot values and transcripts are part of the logged metadata, so encrypt the logs, restrict access and set retention to match your policy.

How long does CloudWatch keep Lex metrics?

The Lex V2 documentation says CloudWatch statistics are kept for 15 months, and you can set alarms on thresholds.

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Ready to put AI agents to work? Explore ElixirClaw, the agentic OS platform to build, orchestrate and govern AI agents.

Table of Contents

Get the latest articles in your inbox

Subscribe Now

Related Articles

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue