Evolution of RAG: Baseline RAG, GraphRAG, and KAG

Posted on In Views:

Evolution of RAG: Baseline RAG, GraphRAG, and KAG

Enhancing LLM accuracy with structured knowledge, inverted indexes, and dynamic retrieval

This article focuses on the evolution of RAG, so technical details will be skipped to avoid this article being too geeky.

Using LLM for chatbot is the most common application nowadays, but the answer of LLM is too generalized to serve specific application scenarios.

Therefore, Retrieval-Augmented Generation, aka RAG, has been developed.

Simply put, it is to store the specialization data needed by LLM, so that LLM can refer to it when answering.

Baseline RAG

A traditional RAG process is as follows.

Indexing in the left side means to process the data that LLM needs to refer to in advance, and the processing flow is to first divide the chunks and then vectorize the chunks into the vector database.

When the user asks a question, he also needs to vectorize the query and take the first few similar results from the vector database, which is usually the order of vector product. These chunks are fed to the LLM along with the query, and the LLM generates a “good” answer.

From the above flow, it is obvious that traditional RAG encounters several problems.

  1. Context fragmentation due to chunks.
  2. Vector product is too simple, losing reasoning ability.
  3. Vectors do not have verification capability, so it is easily to answer the wrong question.

In order to improve the reasoning ability and overcome the problem, GraphRAG was born.

GraphRAG

To avoid getting into too much in depth theory, let’s look directly at the process.

From the diagram, we can see that GraphRAG has more Graph support (as the name suggests) than traditional RAG.

In particular, this Graph consists of these three elements.

  • Entity
  • Relationship
  • Statement description

The following is an example.

  • (OpenAI) –[developed]–> (ChatGPT)
  • (ChatGPT) –[is a]–> (LLM)
  • (ChatGPT) –[published in]–> (2022)
  • (ChatGPT) –[uses]–> (Transformer)
  • (Transformer) –[is a]–> (deep learning architecture)
  • (Transformer) –[published in]–> (2017)
  • (Transformer) –[is developed by]–> (Google)

By extracting key information from chunks and constructing a Knowledge Graph, aka KG, the process is similar to the human brain.

For example, the hippocampus converts short-term memories into long-term memories by filtering and translating information, so that it can use these memories to get better ideas when it encounters similar situations in the future.

These KGs are graphs, which are not always easy for LLM to use, so there is another important process in GraphRAG called summarization, where these KGs are summarized into paragraph and paragraph of text.

With summarization, LLM can have a more full picture and get a relatively accurate answer.

Nevertheless, GraphRAG has its shortcomings.

  1. Summaries and KGs can be helpful, but they also have the potential to diffuse specialized knowledge.
  2. Although summarization can help to expand context, the reasoning ability is still insufficient.
  3. The noise introduced by summarization can still create hallucination.

Therefore, the idea of converting from KG to summarization is not feasible. However, KG itself is a valuable design, as it simulates the process of knowledge generation.

What if we could make AI work more like the human brain? Therefore, we meet the Knowledge Augmented Generation, also known as KAG.

KAG

The DIKW pyramid is a famous model that describes the process of wisdom concentration, and it is what we are trying to do to make RAG accurate.

So, in the GraphRAG implementation, we filter Information from Data, and then try to build a Knowledge Graph. However, the human brain does not think in one direction, any wisdom is generated with multiple references.

The KAG paper introduces a lot of complex theories, but in short, KAG tries to allow KG to connect back to the original chunk and add more semantic references to the query rather than just vector references, which results in a significant improvement in domain-specific knowledge.

In terms of process, KAG adds inverted indexing and full-text search, so it can be more semantically precise. In addition, the extracted KG will become the source of input for LLM, so that LLM can have a more comprehensive knowledge base.

In GraphRAG, only those summaries are provided to LLM, but KAG does not do summaries, but submits the complete KG together with original text to LLM.

The ability to accept such a large number of inputs is also due to recent modeling advances, such as Gemini 2.0 Flash.

KAG has a lot of in-depth technical tuning, which makes it difficult to fully implement. However, just adding KG and full text searching over traditional RAG will already achieve good results, and it’s not too difficult to do both, which is the core value of KAG.

Wrap Up

Making LLM accurate is a difficult task, especially if we want to turn LLM into a commercial product, it is even more important to answer the question correctly.

Traditional RAGs have limited ways to improve the accuracy of LLMs, only chunking, embedding and prompting, and then adding rerank and other mechanisms to allow the Top N to find better results.

GraphRAG adds the concepts of KG and summarization to the traditional RAG to extend the context.

KAG, on the other hand, is based on GraphRAG and adds KG’s bi-directional links and inverted indexes.

All of these techniques are designed to expand the contextual information when LLM generates answers, and gradually enable LLM to be accurate enough in specific scenarios.

In fact, there are some additional variants, such as Corrective RAG aka CRAG. However, this kind of variants do not directly enhance the thinking ability of LLM, but rather assist LLM through some external mechanism, so I did not include them.

In summary, the evolutions of RAG are as follows.

LLM -> RAG -> GraphRAG -> KAG

RAG is a concept that has only been around for two years, and it’s exciting to see how quickly it has evolved.

Originally published on Medium