This post is not intended to describe/ talk about the codebase(may contain live hosted link), but rather a timeline, documentation/notes of development in brief. For understanding the source code, you’ll have better luck doing a gitingest, or maybe just clone the repo and then use whatever frontier-LLM is achieving AGI this week(/s) and not breaking your wallet at the moment, to generate a summary.

Why GraphRAG

I interned at IIM Mumbai, and the work we did there resulted in a publication. You guessed, it had something to do with graphs! Check here. In the time that followed, RAG was all the hype adjacent to LLMs. RAG and graphs -> GraphRAG! I came across Microsoft’s GraphRAG, and a few other projects that I can’t recall at the time of writing this. I studied these implementations to a half-decent depth. I believe Knowledge Graphs / GraphDBs, and consequently GraphRAG will be even more critical in trying to build intelligent systems that respect user privacy and work offline. And so, as a learning exercise, I decided to create my own.

Timeline/About the project

For the record, I’m a total 0 at frontend, so that is entirely the courtesy of Gemini/Claude. Most of the development took place in Google’s Antigravity.

Earliest version I remember: Simple Chunking(fixed-length based), no user authentication/db(ephemeral instances, no memory), monolithic architecture (single Dockerfile), basic UI with no controls.

I decided to spend time thinking about chunking methods because if a chunk is too small, then you miss out on any surrounding context that would otherwise help in richer and complete chunks(Lost in the Middle problem). On the other hand if you feed too-large chunks, you may get relatively lower quality ingestion/triples, repeated triples if you have a high overlap in chunks.

I proposed perplexity-based chunking. The idea was to utilise a Small Language Model(SLM) for chunking, which you could also possibly be using for Speculative Decoding. Without the math, perplexity measures how perplexed/surprised it is about the next token, provided the previous ones. We can extend this, and apply it at a sentence level by averaging the perplexities for the tokens in the current sentence. We calibrate and set a threshold for splitting(ending a chunk) for these perplexity values. On the other hand, for a more dynamic threshold, we can utilise moving averages, percentiles. See here.

The latest version got cosmetic changes, VLLM backend for SLM instead of a HuggingFace backend. Modular codebase and architecture(separate containers for backend, frontend & VLLM). For local testing and deployment after I started using VLLM, I provisioned VMs on gcloud. Hosting was done using Google Cloud Run. Early versions using their CI/CD, watching the github repo. Later versions, were deployed using gcloud.yaml using the gcloud-cli from the VM.

For DB, I went with KuzuDB because it was serverless, embeddable, hot in the graphdb community at the time. Too bad the open-source project got archived (read). For the our enthusiast-project it is no biggie however.

And, that’s a wrap :/ (might update this post with more stuff later!) Apologies for this not reading like a story, but some random incoherent pieces of text. The portfolio website came into being much later than the project itself, so a script wasn’t being cooked in my head while I worked.