Key takeaway
- Human-centred AI designs systems around people. Learn the core principles, NIST AI RMF and EU AI Act oversight points, and an enterprise adoption sequence.
Human-centred AI is the practice of designing AI systems around what people need, can understand and can control, and then checking that the technology and the business case support that. In an enterprise, it means named human owners, clear explanations, a way to override or stop the system, and care with personal data.
This guide gives general readers and enterprise leaders the principles, the human oversight points in the NIST AI Risk Management Framework and the EU AI Act, and a practical adoption sequence. For the wider governance picture, see our guide to ethical standards for AI governance.
What is human-centred AI?
Human-centred AI starts with people and asks what they want and what they need, then asks what is technically feasible and what is viable for the organisation. A product is not built just because the technology is interesting. The people who will use it, and the people it will affect, shape the design from the first step.
The idea has a research history. In 2019 Microsoft Research welcomed the launch of Stanford's Human-Centered AI Institute, and noted that the design of human-AI interaction experiences was called out as an important area requiring further study. Microsoft also publishes 18 guidelines for human-AI interaction, applied at four points: first use, during use, when the system is wrong, and over time.
Many people prefer the word augmentation to the word artificial. The aim is for AI to extend what people can do, with the person still accountable for the outcome.
What are the principles of human-centred AI when designing with data?
Eight principles are widely used when teams design with data. The table turns each into a design question and a practical step. The right-hand column points to the NIST characteristic it most relates to; the mapping is our own reading, not NIST wording.
| Principle | Design question | Practical step | Related NIST characteristic |
|---|---|---|---|
| Make lives better | Does the system help the person, or only the process? | Define the human benefit before the model metric. | Valid and reliable |
| Better inform and guide organisations | Does it help a decision-maker act with better evidence? | Show the evidence and the confidence, not only the answer. | Explainable and interpretable |
| Unlock new insights | Can it surface patterns people cannot review at scale? | Have domain experts check each new pattern before it drives action. | Valid and reliable |
| Make the previously hidden visible | Does it reveal links people would miss? | Keep the reasoning traceable so a reviewer can challenge it. | Accountable and transparent |
| Support positive behaviour change | Does it nudge with consent, and support the person? | Pair data insight with human support, and let people opt out. | Safe |
| Best use of time, effort and resources | Does it free people for judgement and creative work? | Automate routine steps; keep people on the exceptions. | Valid and reliable |
| Include vulnerable populations | Who is missing from the data, and who bears the risk? | Test with the groups affected, and review for harmful bias. | Fair with harmful bias managed |
| Protect privacy | Do people know what is collected, and can they choose? | Collect the minimum, state the purpose, record consent. | Privacy-enhanced |
Do not treat data as the truth. People create data, and people choose the data sets used to train models, so bias and blind spots can enter at every step. Review what a data set leaves out before relying on it.
What do NIST and the EU AI Act say about human roles?
The NIST AI Risk Management Framework was released on 26 January 2023 and is for voluntary use. NIST says it is being revised, so check the NIST page for the current version. Its guidance on trustworthy AI characteristics names seven: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.
On human roles, NIST says human judgement should be used when choosing the metrics for these characteristics, that deploying AI is the joint responsibility of all AI actors, and that trade-offs between characteristics should be resolved transparently and justifiably. It also says human intervention may be needed when a system cannot detect or correct its own errors.
Article 14 of the EU AI Act requires high-risk AI systems to be designed so that natural persons can effectively oversee them while in use. As appropriate and proportionate, overseers should be able to understand the system and monitor it, stay alert to over-reliance on its output, interpret that output, decide not to use it or override it, and intervene or stop it safely. Application dates for high-risk rules have been changing, so confirm the current timetable on the official EU pages.
| Human role | What the design must provide | Source |
|---|---|---|
| Understand and monitor | Documentation and live performance views | EU AI Act Article 14 |
| Guard against over-trust | Confidence signals and prompts to check the output | EU AI Act Article 14 |
| Override or reverse | A clear way to reject or undo an output | EU AI Act Article 14 |
| Stop or intervene | A stop control or equivalent procedure | EU AI Act Article 14; NIST (shut down, modify or intervene) |
| Set the metrics | Human judgement on thresholds for each characteristic | NIST AI RMF |
How do you balance what people want with what is feasible and viable?
Three questions need to be answered together. Is it desirable for the people involved? Is it feasible with the data and the technology? Is it viable for the business? A system can be technically possible and still fail on the first or the last. Because AI makes feasibility look easy, teams should also ask whether a problem is better fixed by a change to the process or the team rather than by a model.
How does human-centred AI apply to AI agents?
Agents act, not only advise, so the human controls must sit in the workflow. Useful patterns are recommend-only modes, approval for high-impact actions, least-privilege access and a decision record. Our guide to agentic AI risk management sets these out, and agentic AI governance and compliance covers the policy side. AgentGRC keeps autonomous agent actions inside policy, with the audit trail risk teams need, and ElixirData Context OS keeps governed context, versioned policy and decision traces in one decision layer above AI agents.
How should an enterprise adopt human-centred AI?
| Step | What to do | Output |
|---|---|---|
| 1. Name the people | List users, affected groups and the accountable owner | Stakeholder map |
| 2. Set the human benefit | State the benefit and the harm to avoid | Success and harm criteria |
| 3. Review the data | Check sources, gaps, consent and purpose | Data review record |
| 4. Design oversight | Add explanation, override and stop controls | Oversight design |
| 5. Test with real users | Include the groups most affected | Test findings |
| 6. Monitor over time | Track errors, complaints and drift; review on a schedule | Review cadence |
Impact also arrives at scale. A routing app that sends many drivers through one quiet neighbourhood helps each driver and burdens the neighbours. Review effects on groups and communities, not only on individual users. Adoption problems of this kind are covered in AI adoption challenges; for a regulated sector, see ethical AI in healthcare, and for service teams, responsible AI in customer service.
When is human-centred design not enough?
Good design does not replace controls. A human in the loop who cannot see the evidence, has no time to review, or is rewarded for speed gives little real oversight. Design can also be sound for the intended user and harmful when data is reused for another purpose later, so state the purpose and review it when strategy or ownership changes. Consider the benevolent use and the non-benevolent use of the same data before you collect it.
To map one decision your AI should make, the human who owns it, and the policy it depends on, book a Context Workshop. See what happens in the workshop.
FAQ
What is human-centred AI?
Human-centred AI means designing AI systems around the needs, understanding and control of the people who use them or are affected by them, rather than building a product only because the technology is interesting.
What are the principles of human-centred AI when designing with data?
Eight principles: make lives better, better inform and guide organisations, unlock new insights, make the previously hidden visible, support positive behaviour change, make the best use of time, effort and resources, include vulnerable populations, and protect privacy.
Which framework should enterprises use for trustworthy AI?
The NIST AI Risk Management Framework is a voluntary framework. Its guidance names seven characteristics: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. NIST says the framework is being revised, so check its page.
What does the EU AI Act say about human oversight?
Article 14 requires high-risk AI systems to be designed so people can effectively oversee them, including understanding outputs, guarding against over-reliance, overriding the system and stopping it. Confirm current application dates on the official EU pages.
How is human-centred AI different from keeping a human in the loop?
A human in the loop is one control. Human-centred AI covers the whole design, from the problem and the data to explanation, override, stop controls and review over time.
Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.
Ready to put AI agents to work? Explore ElixirClaw, the agentic OS platform to build, orchestrate and govern AI agents.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE