A knowledge graph for an LLM is a store of entities and the relationships between them that the model queries before it answers or acts. It gives the model facts it can cite and relationships it can follow, which a plain text index cannot. Build one when your questions depend on how things connect, such as which supplier feeds which plant, or which policy applies to which product. For a wider view of the technology, see our guide to graph analytics and knowledge graphs.
The build is a sequence of seven decisions: the questions it must answer, the schema, extraction, entity resolution, validation, storage and retrieval. This guide walks through each, compares vector RAG, a knowledge graph and GraphRAG, and sets out the governance and cost points that decide whether it reaches production. It is written for CTOs, CDOs, architects and platform leads.
An LLM is fluent but does not hold your enterprise facts, and it cannot show where an answer came from. A knowledge graph supplies the facts as entities and relationships, so the model answers from governed records and the reasoning can be traced. The table maps each common limitation to what the graph adds.
| LLM limitation | What a knowledge graph supplies | What you still need |
|---|---|---|
| Plausible but wrong answers | Facts held as entities and relationships that the answer can be checked against | Source records that are accurate and current |
| Knowledge fixed at training time | Facts updated in the graph without retraining the model | A refresh process with an owner |
| No link to enterprise policy or data | Policies, assets and people modelled as connected nodes | Access control on the graph itself |
| Opaque reasoning | A path through named relationships that can be shown to a reviewer | Logging of which paths were used for each answer |
| Weak multi-hop questions | Traversal across several relationships in one query | A schema that includes the relationships people ask about |
Vector RAG retrieves text passages that look similar to the question. A knowledge graph retrieves connected facts. GraphRAG uses an LLM to build a graph from your documents and then answers from it. The Microsoft Research GraphRAG paper (first submitted 24 April 2024, revised 19 February 2025) describes GraphRAG as aimed at global questions over a whole corpus, such as the main themes in a dataset, where conventional RAG struggles. On datasets of about 1 million tokens, the authors report that GraphRAG beats a conventional RAG baseline on the comprehensiveness and diversity of answers.
| Approach | Best when | Main trade-off | Status to note |
|---|---|---|---|
| Vector RAG | Questions are answered by one or two passages | Weak on relationships and corpus-wide questions | The default starting point; the GraphRAG documentation keeps it as Basic Search |
| Curated knowledge graph | Entities and relationships are known, governed and queried often | Schema design and upkeep need people | Built on RDF or property graph standards |
| GraphRAG (LLM-built graph) | Questions span many documents and need themes or community summaries | The LLM runs over the corpus at indexing time | Open source from Microsoft; the documentation lists Global, Local, DRIFT and Basic search |
| Hybrid | Some questions are lookups, others need connections | Two retrieval paths to test and monitor | Common once a pilot shows where vector search falls short |
Start with vector RAG and add the graph where evaluation shows it fails, for example on multi-hop or corpus-wide questions. Our note on how graph databases work covers the storage side.
Write ten to twenty real questions from the business first. They decide which entities and relationships you model, and they become your evaluation set. A graph with no target questions grows without a test.
Define entity types, relationship types and the properties that matter. Keep the first version small and reuse an existing vocabulary where one exists in your industry. Record who owns each entity type, because ownership decides who fixes bad data later.
Structured sources map to the schema directly. For documents, an LLM can extract entities and relationships from text. The GraphRAG indexing pipeline in the GraphRAG documentation splits the corpus into text units, extracts entities, relationships and key claims, clusters the graph with the Leiden technique, then summarises each community. The documentation recommends prompt tuning, because default prompts may not suit your data.
The same supplier may appear under three spellings. Entity resolution merges them into one node. The neo4j-graphrag package lists semantic-match and fuzzy-match resolvers among its graph-construction extras. Whatever tool you use, keep a log of merges so that a wrong merge can be undone.
Check the graph against rules before it feeds answers. For RDF data, SHACL, a W3C Recommendation dated 20 July 2017, is a language for checking that an RDF graph meets a set of conditions written as shapes. For property graphs, use database constraints and your own test queries drawn from the questions in step 1.
ISO/IEC 39075:2024 is the published ISO standard for property graphs and the GQL query language (Edition 1, April 2024; the ISO page lists a corrigendum issued in 2026 and a revision in development). Check whether your chosen store supports GQL, or its own dialect, before you commit, because that affects portability.
Common patterns are to run a graph query for entities named in the question, to combine vector search with graph traversal, or to let the model write a query that your code validates before running. Whichever you pick, return the path used with the answer so that a reviewer can see it.
| Option | Use it when | Current status |
|---|---|---|
| RDF with SHACL | You need shared vocabularies, linked data and formal validation | SHACL is a W3C Recommendation (2017) |
| Property graph with GQL | Application teams want node and edge properties and traversal queries | ISO/IEC 39075:2024 published April 2024 |
| Microsoft GraphRAG | You want an LLM to build a graph and community summaries from documents | Documented with Global, Local, DRIFT and Basic search; no version number given on the documentation site |
| neo4j-graphrag (Python) | You run Neo4j and want construction pipelines and retrievers | Needs Neo4j 5.18.1 or later (Aura 5.18.0 or later) and Python 3.10 to 3.14; replaces the deprecated neo4j-genai package |
Agents need to act, not only answer, so the graph sits between the model and the systems it can change. A common layered pattern puts the graph in the middle, checking and enriching what the model proposes.
| Layer | Role | Control to add |
|---|---|---|
| Input | Captures queries, events and alerts | Authentication and logging |
| LLM | Interprets intent and summarises | Prompt and output filtering |
| Knowledge graph | Validates claims, adds context, links systems | Access control, versioned schema |
| Agent orchestration | Plans tasks and coordinates agents | Approval rules for disruptive actions |
| Execution | Runs actions in IT, support, security and operations | Least-privilege credentials |
| Feedback | Writes confirmed outcomes back to the graph | Review before write-back |
This is the context layer described by ElixirData Context OS, which keeps governed context, versioned policy and decision traces in one decision layer above AI agents. ElixirClaw is an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control. For retrieval over unstructured material, AgentSearch offers context-aware search across enterprise knowledge.
| Use case | What the graph connects | Fit |
|---|---|---|
| Supply chain risk | Suppliers, shipments, sites and external events | Strong: multi-hop by nature |
| Security operations | Alerts, assets, identities and threat patterns | Strong: relationships carry the signal |
| Compliance and governance | Regulations, policies, controls and evidence | Strong where answers must cite sources |
| Customer support | Accounts, products, contracts and history | Medium: often solved with RAG first |
| Enterprise search | Documents, people, projects and topics | Medium: GraphRAG for corpus-wide questions |
Related reading: how a Google Knowledge Graph is built and exposed, which is a different problem from an enterprise graph.
The timings are our planning guide, not a benchmark. Adjust them to your data volume and team size.
To map one decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.
It is a store of entities and relationships that the model queries to ground its answers in governed facts and to show the path behind them.
It depends on the question. The Microsoft Research paper reports better comprehensiveness and diversity on global questions over datasets of about 1 million tokens. Its own documentation keeps Basic (vector) search for queries that baseline RAG serves well.
SHACL, a W3C Recommendation from 2017, validates RDF graphs. ISO/IEC 39075:2024 standardises property graphs and the GQL query language.
Tune the extraction prompts, resolve duplicate entities, validate against rules, record the source of each fact and sample the output before agents act on it.
Avoid it when single documents answer your questions, when no team owns the data or when the relationships are few enough for relational joins.