Gruff Rag, or what Context Engineering actually is

Meet Gruff Rag — a developer hero debunking AI hype around “GraphRAG” , “Context Engineering,” and “Semantic Layer.” If anything, Gruff Rag is more real than GraphRAG. As the former AI Community Architect, I’ve spent an year and a half connecting OSS AI communities in graph space, eliciting patterns that emerging agentic AI systems adopted. A typical setup started with proven retrievers from LangChain or LlamaIndex, and used a vector RAG and sometimes a graph database approach for what is now called context engineering.

Several buzzword-pockmarked events were milestones along the way. In 2024, Microsoft released its GraphRAG paper and project. Everyone in graphs and their PMs, trying to break into AI, adopted the term to describe what they were doing. It was an early sign that the term is overly broad — the specific approach from Microsoft Research did not take and the project did not gain dominance in the space.

The “GraphRAG” moment happened around the time the “Agentic” shift came into fill swing. AI users quickly realized that agents are nothing without memory. An early crop of “AI memory” startups appeared, explaining that a shared state is necessary as an agreeable view of the world. Instead of seven blind men feeling a context, it should be explicitly engineered for agents to work together. They can inquire what the plan is, what’s up with a task, our knowledge about a certain part of our world, and update it in progress. Agents can enrich the memory with what they learn, modify or forget statements that turned out wrong or obsolete, and even send other agents to update the memory or do deep research to enrich it.

As usual, Andrej Karpathy appeared at the right time and at the right place, on X, to christen the baby as Context Engineering. That superseded AI memory in the discourse, even though the terms are not entirely equivalent. Context engineering includes making an AI memory accessible to agents via such channels as MCP, and a buildout of AI infra to do so went into overdrive. The rise of AAIF at the Linux Foundation was announced at a WorkOS MCP Night in December 2025, and featured such initial projects as the MCP, an AGENTS.md markdown file, and Goose, an OSS agent.

That same winter, a couple of VCs popularized the term “context graph,” to promote one of their investments. In a replay of the GraphRAG wave, graph vendors latched onto it to signal their AI pivots. But literally nothing had changed — a context graph was already a ubiquitous part of any AI memory, which in turn exists within a wider, um, context of “context engineering.”

Here comes the rub, from Gruff Rag. The most frustrating obstacle to any practical uptake of the GraphRAG is the stark reality that it does not exist. It does not exist as a thing. You can’t git clone graphrag, uv pip install it, or bring it to bear on your data in any other simple material way. You have to actually think of your data as a graph, understand what does it mean to compose context from various parts of the graph, how to serve it to agents, and how to update the derived context view from the graph changing underneath, potentially by multiple agents.

Vector RAG, by contrast, does exist — take any RAG database, like turbopuffer, drop a bunch of documents into it, and you’ll get going in a minute. It will embed the documents as vectors and get cracking on similarity metrics. You can do it in Postgres now too. You don’t actually need a graph database for the majority of graph tasks. A vast majority of the context graphs deployed today remain in their original relational tables.

Vector RAGs are simple utilities akin to search now. I don’t see any need to load your derived embeddings into a 3rd party cloud, for instance. Search always sits next to your primary data. Vectors and context graphs are utility projections of the ground truth residing in the Lakehouse.

The elephant in the erstwhile Hadoop room is RBAC. A proper fully connected graph can never be built over the security compartmentalization concerns. The tables are governed by strict access rules. Any agentic traversal will have to be gated by access control lists. And decoupling meaning from access is very hard. A begging runtime if, “if I am allowed to see this” statement isn’t gonna cut it in front of the CIO. At QueryGraph.ai, a community OSS AI project building a typesafe semantic layer for AI-native agentic Lakehouse, we implemented Typesec, and agentic protocol for auditable access. It uses RBAC and ODRL (a W3C security standard) with DIDs, Decentralized IDs (another W3C standard) to make access compile-time decision as much as possible. An insecure agent cannot compile and therefore cannot exist. Of course you need a compiled language with sufficient type-level programming capability for this, and we chose Rust as the bedrock of the AI infrastructure.

The future of context engineering is more aligned with Apache Ossie, the emerging semantic approaches to the Lakehouse, and Semantic Croissant, the CODATA Standard for interoperable ontologies, also adopted by the government of Japan in their OpenDataSpaces. Both Ossie and MotherDuck Guides need to come a long way to see semantics as meaning of data, not a standing SQL query defining a metric, but the gradient is encouraging.

Properly defining Semantic Layer as a thing is what we set out to do with QueryGraph. It also inspired us to built LakeCat, a next generation catalog that honors Engine Truth, enforces unforgeable governance with audit and lineage, and gets out of the way so engines can operate on data optimally. Join us at the community meetups to discuss these ideas! Lu.ma/_ai is our calendar.