Microsoft is making publicly available a new technology called GraphRAG, which enables chatbots and answer engines to connect the dots across an entire dataset, outperforming standard Retrieval-Augmented Generation (RAG) by large margins.

What’s The Difference Between RAG And GraphRAG?

RAG (Retrieval-Augmented Generation) is a technology that enables an LLM to reach into a database like a search index and use that as a basis for answering a question. It can be used to bridge a large language model and a conventional search engine index.

The benefit of RAG is that it can use authoritative and trustworthy data in order to answer questions. RAG also enables generative AI chatbots to use up to date information to answer questions about topics that the LLM wasn’t trained on. This is an approach that’s used by AI search engines like Perplexity.

The upside of RAG is related to its use of embeddings. Embeddings is a way of representing the semantic relationships between words, sentences, and documents. This representation enables the retrieval part of RAG to match a search query to text in a database (like a search index).

But the downside of using embeddings is that it limits the RAG to matching text at a granular level (as opposed to a global reach across the data).

Microsoft explains:

“Since naive RAG only considers the top-k most similar chunks of input text, it fails. Even worse, it will match the question against chunks of text that are superficially similar to that question, resulting in misleading answers.”

The innovation of GraphRAG is that it enables an LLM to answer questions based on the overall dataset.

What GraphRAG does is it creates a knowledge graph out of the indexed documents, also known as unstructured data. The obvious example of unstructured data are web pages. So when GraphRAG creates a knowledge graph, it’s creating a “structured” representation of the relationships between various “entities” (like people, places, concepts, and things) which is then more easily understood by machines.

GraphRAG creates what Microsoft calls “communities” of general themes (high level) and more granular topics (low level). An LLM then creates a summarization of each of these communities, a “hierarchical summary of the data” that is then used to answer questions. This is the breakthrough because it enables a chatbot to answer questions based more on knowledge (the summarizations) than depending on embeddings.

This is how Microsoft explains it:

“Using an LLM to summarize each of these communities creates a hierarchical summary of the data, providing an overview of a dataset without needing to know which questions to ask in advance. Each community serves as the basis of a community summary that describes its entities and their relationships.

…Community summaries help answer such global questions because the graph index of entity and relationship descriptions has already considered all input texts in its construction. Therefore, we can use a map-reduce approach for question answering that retains all relevant content from the global data context…”

Examples Of RAG Versus GraphRAG

The original GraphRAG research paper illustrated the superiority of the GraphRAG approach in being able to answer questions for which there is no exact match data in the indexed documents. The example uses a limited dataset of Russian and Ukrainian news from the month of June 2023 (translated to English).

Simple Text Matching Question

The first question that was used an example was “What is Novorossiya?” and both RAG and GraphRAG answered the question, with GraphRAG offering a more detailed response.

The short answer by the way is that “Novorossiya” translates to New Russia and is a reference to Ukrainian lands that were conquered by Russia in the 18th century.

The second example question required that the machine make connections between concepts within the indexed documents, what Microsoft calls a “query-focused summarization (QFS) task” which is different than a simple text-based retrieval task. It requires what Microsoft calls, “connecting the dots.”

The question asked of the RAG and GraphRAG systems:

“What has Novorossiya done?”

This is the RAG answer:

“The text does not provide specific information on what Novorossiya has done.”

GraphRAG answered the question of “What has Novorossiya done?” with a two paragraph answer that details the results of the Novorossiya political movement.

Here’s a short excerpt from the two paragraph answer:

“Novorossiya, a political movement in Ukraine, has been involved in a series of destructive activities, particularly targeting various entities in Ukraine [Entities (6494, 912)]. The movement has been linked to plans to destroy properties of several Ukrainian entities, including Rosen, the Odessa Canning Factory, the Odessa Regional Radio Television Transmission Center, and the National Television Company of Ukraine [Relationships (15207, 15208, 15209, 15210)]…

…The Office of the General Prosecutor in Ukraine has reported on the creation of Novorossiya, indicating the government’s awareness and potential concern over the activities of this movement…”

The above is just some of the answer which was extracted from the limited one-month dataset, which illustrates how GraphRAG is able to connect the dots across all of the documents.

GraphRAG Now Publicly Available

Microsoft announced that GraphRAG is publicly available for use by anybody.

“Today, we’re pleased to announce that GraphRAG is now available on GitHub, offering more structured information retrieval and comprehensive response generation than naive RAG approaches. The GraphRAG code repository is complemented by a solution accelerator, providing an easy-to-use API experience hosted on Azure that can be deployed code-free in a few clicks.”

Microsoft released GraphRAG in order to make the solutions based on it more publicly accessible and to encourage feedback for improvements.

Read the announcement:

GraphRAG: New tool for complex data discovery now on GitHub

Featured Image by Shutterstock/Deemerwha studio