DEV Community

Cover image for GraphRAG in Practice: Entity Graphs, Provenance, and When the Cost Is Worth It
Hu Jin
Hu Jin

Posted on

GraphRAG in Practice: Entity Graphs, Provenance, and When the Cost Is Worth It

Classic hybrid RAG is strong at local questions: a policy clause, an error code, a single API page. It is weaker when the question needs a map of the corpus—themes across hundreds of documents, or an answer that stitches several entities together. GraphRAG-style pipelines address that gap by building a knowledge graph from text, keeping provenance back to source chunks, and retrieving over entities, relations, and community summaries—not only over embedded passages.

This article is the advanced companion to the beginner RAG series. It focuses on extraction, provenance, query modes, and honest cost tradeoffs. For the research framing and open-source system, start with Microsoft’s GraphRAG paper and repository (linked at the end).

What GraphRAG adds on top of vector RAG

Vector RAG retrieves chunks by similarity. GraphRAG also materializes:

  • Entities (people, products, teams, regulations, systems).
  • Relations (owns, depends-on, located-in, supersedes).
  • Provenance (which chunk and document justified each claim).
  • Optionally communities—clusters of densely connected entities—with precomputed summaries for global questions.

You still keep a vector (or hybrid) index. The graph is an extra index optimized for structure and sensemaking, not a full replacement for BM25 + embeddings.

Extract entities, relations, and provenance into a knowledge graph

Index-time: extraction with provenance

At ingest, an LLM (or a mix of NER + LLM) reads each chunk and emits structured triples. A practical schema:

entity: {id, name, type, description, source_chunk_ids[]}
relation: {src_id, dst_id, type, description, source_chunk_ids[], confidence?}
Enter fullscreen mode Exit fullscreen mode

Provenance is non-negotiable. Every node and edge should point back to chunk IDs (and ideally offsets). Without that, graph answers become un-auditable summaries—the opposite of what RAG is for.

Worked sketch (pseudocode):

def extract_graph(chunk_id: str, text: str) -> dict:
    prompt = f"""Extract entities and relations from the text.
Return JSON with entities[].name|type|description and
relations[].src|dst|type|description.
Only use facts supported by the text. Text:\n{text}"""
    raw = llm_json(prompt)
    for e in raw["entities"]:
        e.setdefault("source_chunk_ids", []).append(chunk_id)
    for r in raw["relations"]:
        r.setdefault("source_chunk_ids", []).append(chunk_id)
    return raw

def build_index(chunks):
    nodes, edges = {}, []
    for c in chunks:
        g = extract_graph(c.id, c.text)
        # upsert/merge entities by normalized name+type
        # append edges with provenance
    communities = cluster(nodes, edges)          # e.g. hierarchical community detection
    summaries = {cid: summarize_community(cid) for cid in communities}
    return GraphIndex(nodes, edges, communities, summaries, chunks)
Enter fullscreen mode Exit fullscreen mode

Merge carefully. The same real-world entity appears under many surface forms (AZURE AI SEARCH, Azure AI Search, AI Search). Normalization, alias tables, and human review for high-value entity types reduce duplicate nodes that fragment retrieval.

Query-time: local vs global

Local entity retrieval versus global community summarization

Local / entity-centric questions (“Who owns service X?”, “What depends on database Y?”):

  1. Resolve entities mentioned in the question (NER + vector match over entity descriptions).
  2. Walk a bounded neighborhood (1–2 hops) of relations.
  3. Pull source chunks via provenance—not only the LLM-written entity blurbs.
  4. Generate an answer that cites those chunks.

Global / theme questions (“What are the main risk themes in this corpus?”):

  1. Retrieve relevant community summaries (often via map-reduce over many communities, as in GraphRAG).
  2. Optionally descend into lower-level communities or source text for detail.
  3. Cite summary IDs and underlying chunk IDs when claims get specific.

Global mode is where graph methods tend to outperform naïve top-k chunk RAG, because no small set of chunks “is” the theme of a million-token corpus.

Hybrid graph + vector is the usual production shape

A durable pattern:

  1. Hybrid search over chunks for precision facts.
  2. Graph neighborhood for multi-entity structure.
  3. Community summaries for corpus-level sensemaking.
  4. Rerank or fuse candidates, then generate with strict citation rules.

Do not force every query through the most expensive path. Route by question type (classifier or cheap heuristic: entity count, words like “themes/overview/landscape”).

Cost, latency, and when it is worth it

Index-time cost versus when GraphRAG pays off

Index-time cost dominates. Extracting entities/relations with an LLM over every chunk can cost more than embedding alone by a large factor, and community summarization adds another LLM pass. Re-ingestion after doc updates is heavier too: you must refresh affected nodes, edges, and summaries.

Query-time cost varies. Local neighborhood reads can be cheap. Global map-reduce over many community summaries can be token-heavy per question.

Worth it when:

  • Users ask cross-document, multi-entity, or “what are the themes” questions.
  • You need explainable structure (impact analysis, ownership graphs).
  • The corpus is large enough that stuffing or naïve RAG fails sensemaking.

Defer or skip when:

  • Most traffic is single-doc FAQ / exact ID lookup (hybrid RAG + rerank is enough).
  • You cannot fund extraction + ongoing graph maintenance.
  • Provenance requirements are strict and your extraction quality is still noisy—fix chunking and hybrid search first.

Limitations (realistic)

  • Extraction errors become graph facts; bad merges create false hubs.
  • Graphs drift as documents change; without incremental update discipline they rot.
  • Community summaries are themselves LLM generations—treat them as indexes into evidence, not as ground truth.
  • Benchmarks from public papers may not transfer to your domain; measure on your own global vs local question sets.

Practical checklist

  • Store source_chunk_ids on every entity and relation.
  • Prefer retrieving raw chunks for final answers; use summaries for navigation.
  • Route local vs global queries explicitly.
  • Budget extraction tokens separately from query tokens.
  • Evaluate with separate local-fact and global-sensemaking test slices.

Related reading


About the author

I write about Applied AI, MCP, Azure AI Search, and RAG. Follow me on Dev.to and Hashnode for more.

Top comments (0)