Skip to main content
XenonStack

XAI

XenonStack Blog AI in Autonomous Driving: Safety Case and Regulation

and | Published | Updated

AI in Autonomous Driving: Safety Case and Regulation
12:09

Key takeaway

  • AI in autonomous driving for enterprise leaders: where AI fails, safety evidence, NHTSA and UN rules, a safety case checklist and a 3 to 5 year outlook.

AI in autonomous driving is the set of perception, prediction and planning models that let a vehicle perform the driving task, and the safety case and regulation that decide whether it may do so on public roads. For an enterprise this is a risk and evidence problem as much as a technology one. According to NHTSA, no fully automated vehicle is currently for sale to US consumers and every vehicle on sale needs the driver's full attention, so the real decisions sit in fleets, pilots and supplier programmes. This guide takes the safety and regulatory angle; for the levels of automation and ADAS, read our guide to ADAS and autonomous vehicles.

The short answer for leaders: judge an AI driving system by its operational design domain, the evidence behind it and the controls around it, not by the demo. The regulatory picture changed in 2026 with new UN rules for driverless systems, while US oversight rests on crash reporting and vehicle safety standards. Below we cover where AI fails, what the evidence shows, which rules apply, what a safety case contains and how to sequence adoption. For the in-vehicle compute side, see Edge AI for autonomous vehicles and edge AI agents in the automotive industry.

How does AI drive a vehicle, and where can it fail?

An autonomous driving stack turns sensor data into vehicle actions in stages. Each stage has its own failure mode and its own control. The table below is our engineering view of the stack, not a standard.

Two-column diagram mapping AI driving stack stages (sensing, perception, prediction, planning and control) to controls: malfunction detection, scenario coverage, incident replay and a defined operational design domain
StageWhat the AI doesTypical failure modeControl to ask for
SensingCameras, radar and LiDAR capture the sceneObstructed or degraded sensorsMalfunction detection and a visible warning to the driver
Perception and localisationModels detect objects and place the vehicle on a mapMisclassification of rare objects or conditionsScenario coverage in simulation, track and road testing
PredictionModels estimate what other road users will do nextWrong assumption about intentReplay of real incidents as regression tests
Planning and controlSelects a path and commands steering, speed and brakingUnsafe action outside the tested domainA defined operational design domain and a fallback behaviour
MonitoringLogs decisions and reports incidentsGaps that stop root-cause analysisIn-service monitoring and stored safety data

Computer vision sits at the heart of the perception stage; our guide to computer vision applications covers the techniques. Real-time inference usually runs in the vehicle, which is why edge computing matters for latency-critical decisions, and connectivity is covered in 5G for automated ground vehicles.

What does the safety evidence actually show?

Safety claims for AI driving differ sharply by level. Evidence for driver assistance (Levels 0 to 2) and for driverless robotaxis (Level 4) should not be mixed. The sources below are the ones we would put in front of a risk committee.

EvidenceWhat it saysWho published itHow to read it
Driverless operator dataThrough June 2026, Waymo reports 271.3 million rider-only miles across Phoenix, the San Francisco Bay Area, Los Angeles, Austin and Atlanta, with 95% fewer serious-injury-or-worse crashes and 82% fewer any-injury crashes than its human benchmarkWaymo safety impact pageOperator's own data, using a benchmark it describes; no snowfall in its current cities
Independent study of robotaxisDuring driverless operation, Waymo's police-reportable crash rate was 68% lower and its injury crash rate 81% lower than human drivers in the same cities and years (Teoh et al., 2026)IIHSIndependent, but limited to one operator and its cities
Partial automation (Level 2)A 2025 study found no crash-reduction advantage for partial automation over crash-avoidance technology alone; regular users often hold a false sense of securityIIHSTreat Level 2 as driver assistance with a monitoring duty
Consumer availabilityLevel 3 to 5 technologies are not available for consumer purchaseNHTSAEnterprise exposure is mostly through fleets, suppliers and pilots

The pattern is consistent: strong results for narrow, geofenced, driverless service, and no proven advantage from letting drivers disengage in partial automation. Neither result transfers automatically to a new city, vehicle or weather condition, so any programme needs its own evidence.

Which rules apply to AI driving systems?

There is no single global rulebook yet. The table lists the instruments an enterprise is most likely to meet, with their current status according to the sources we read. This is not legal advice; confirm the position with counsel and the regulator in each market.

InstrumentWhat it coversCurrent statusSource
US federal oversightNHTSA oversees safe testing, development and deployment of automated driving systems; manufacturers must comply with Federal Motor Vehicle Safety StandardsTesting is limited to restricted locations and conditions; a 2025 USDOT framework and exemption programme existNHTSA
Standing General OrderCrash reporting for automated driving systems and Level 2 driver assistanceThird amended order issued 24 April 2025, effective 16 June 2025; the most severe crashes are reported within five daysStanding General Order
FMVSS No. 127Automatic emergency braking, pedestrian braking and forward collision warning on light vehiclesFinal compliance date 1 September 2029 for all vehicles; 1 September 2030 for small-volume manufacturersFederal Register notice for FMVSS No. 127
UN regulation on driverless systemsSafety case, safety management system, validation, in-service monitoring and data storage for ADSAdopted by WP.29 on 24 June 2026; as reported by Connected Automated Driving, entry into force was expected about a month laterConnected Automated Driving
UK Automated Vehicles Act 2024Domestic regime for authorisation and operator licensingAs reported by Shoosmiths, secondary legislation and guidance are the next steps and the UK is expected to align with the UN approachShoosmiths
General AI risk managementVoluntary framework: Govern, Map, Measure, ManageAI RMF 1.0 released 26 January 2023NIST AI Risk Management Framework

What does a safety case for an AI driving system contain?

The new UN regulation is built around a safety case. As reported by Shoosmiths, the framework has six pillars. We map each to the evidence an enterprise buyer or fleet operator should ask a supplier for.

Pillar (as reported)What to ask for
Performance within the operational design domainA written domain definition and results for the full driving task inside it
Safety management system across the lifecycleNamed owners, change control for model updates and post-deployment review
Approval based on a structured safety caseThe argument and the evidence, not a summary slide
Validation by simulation, controlled testing and real-world dataCoverage reports for each of the three, including how simulators were validated
In-service monitoring and incident reportingIncident rates, near-miss data and the reporting route to the regulator
In-vehicle data storageWhat is recorded, for how long and who can access it

The same sources report the safety bar: the system must reach a level at least equal to a competent human driver. That is a regulatory statement of intent, and meeting it still has to be shown with evidence for each domain, as Connected Automated Driving describes.

What governance, security and cost issues apply?

Governance starts with decision records. Every automated action that affects safety needs a trace that links the model version, the input, the policy in force and the outcome, otherwise incident review and regulator reporting are slow and contested. NexaStack describes itself as an agentic operating platform for physical AI that runs across on-premises, cloud and hybrid environments, and AgentGRC keeps autonomous agent actions inside policy, with the audit trail risk teams need, according to its site. The ideas in agentic AI risk management apply directly: limit what an agent may do, who approves it and how each action is recorded.

AreaQuestion to settleWhy it matters
Model change controlWho approves a model or map update before it reaches vehicles?A new version can change behaviour in the tested domain
Data and loggingWhich events are stored and for how long?Supports incident analysis and regulatory reporting
CybersecurityWho owns security for sensors, connectivity and over-the-air updates?A compromised update path affects every vehicle on it
CostWhat is the cost of validation, simulation and monitoring per domain?Evidence collection often costs more than the model work
Liability and insuranceWho is responsible when the system is driving?Responsibility shifts at higher levels; confirm contracts and cover

Which technology trends are changing the stack?

Two shifts matter for safety teams. First, end-to-end learned driving models take raw sensor data and output driving actions. They can simplify the stack, yet they make it harder to explain a decision, so testing and traceability must be stronger rather than weaker. Second, multimodal and generative models are being explored for scene understanding and for creating test scenarios. Treat both as areas to pilot with evidence gates, not as proven. We have not cited vendor-specific claims here because we could not confirm them from the vendors' own pages.

What are the trade-offs, and when should you not go further?

SituationRecommendationReason
Supplier cannot define its operational design domainDo not proceed to a pilotSafety evidence has no boundary to attach to
Evidence comes from another city or climateRun local validation firstBenchmarks are reweighted to where an operator drives; snow is not represented in current data
Level 2 system marketed as self-drivingKeep driver monitoring and training in placePartial automation never replaces the driver
No incident reporting routePauseRegulators in several regions expect reporting and monitoring
Model updates shipped without change controlPauseBehaviour can change outside the tested domain

How should an enterprise sequence adoption?

StepActionExit criterion
1. Define the use casePick one route, site or duty cycle and its operational design domainDomain written and signed off
2. Map the rulesList NHTSA, state, UN and local requirements for that use caseLegal and risk review complete
3. Ask for the safety caseRequest the evidence listed above from the supplierGaps logged with owners
4. Pilot with monitoringRun supervised operation with incident loggingIncident and near-miss data reviewed on a fixed cadence
5. Expand by domainAdd routes, weather or vehicles one at a timeNew evidence for each addition

What will change in AI-driven autonomous driving over the next 3 to 5 years?

The table separates what the sources show from our own view. Dates are those stated by the sources; the outlook column is our judgement.

AreaWhat the sources showOur outlook (view)
UN rules for driverless systemsAdopted by WP.29 on 24 June 2026, endorsed by Canada, China, the EU, Japan, the UK and the US, as reported by Connected Automated Driving; the FIA says real-world performance should be monitored after deploymentType approval routes built on the safety case begin to appear in national law; FIA also expects the framework to evolve as evidence builds
US vehicle standardsFMVSS No. 127 compliance by 1 September 2029 (2030 for small-volume makers)Emergency braking becomes a baseline feature, so Level 2 differentiation shifts to driver monitoring
US crash reportingStanding General Order covers ADS and Level 2 crashes with five-day reporting for the most severePublic incident data keeps feeding independent studies
Driverless service footprintFive US metro areas in the Waymo data through June 2026Growth city by city, each needing local validation; snow and complex weather are the open test
Consumer availabilityLevel 3 to 5 not available for consumer purchase todayConsumer Level 4 within five years is unlikely; fleet and geofenced service lead

To connect the vehicle, edge and governance layers in one programme, see NexaStack for physical AI operations and ElixirData Context OS for governed context and decision traces; for agent automation around the programme, see ElixirClaw.

To map one driving decision your AI should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions

What is AI in autonomous driving?

It is the set of perception, prediction and planning models that perform the driving task, together with the safety evidence and controls that allow the system to operate. Responsibility and approval depend on the automation level.

Can I buy a self-driving car today?

According to NHTSA, no fully automated vehicle is currently for sale to US consumers, and every vehicle sold requires the driver's full attention at all times. Level 3 to 5 technologies are not available for consumer purchase.

Is driverless operation safer than human driving?

The Waymo safety page reports 95% fewer serious-injury-or-worse crashes and 82% fewer any-injury crashes than its human benchmark through June 2026, and an IIHS study found a 68% lower police-reportable crash rate. Both relate to one operator in its own cities, so local validation is still needed.

What does the Standing General Order require?

Identified manufacturers and operators must report certain crashes involving automated driving systems or Level 2 driver assistance. Under the third amended order, the most severe crashes are reported within five days and less severe ones monthly.

What did the UN adopt for driverless vehicles in 2026?

As reported by Connected Automated Driving, WP.29 adopted a regulation on 24 June 2026 requiring a safety case, a safety management system, validation by simulation, track and road testing, in-service monitoring and data storage, with safety at least equal to a competent human driver.

What does the next 3 to 5 years look like for driverless regulation?

Sourced dates point to UN rules moving into national approval routes and FMVSS No. 127 becoming mandatory by 1 September 2029. Our view is that driverless service will grow city by city, with local evidence needed each time.

What’s Next for Autonomous Vehicles?

Talk to our experts about implementing compound AI systems in autonomous driving and how industries and departments leverage Agentic workflow and Decision Intelligence to become decision-centric. Harness AI to automate and optimize vehicle operations, enhancing efficiency, safety, and responsiveness.

 

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue