April 10, 2026, 10:43 p.m. by Gabe R
I'm a relative newcomer among the AI users. I first used ChatGPT out of curiosity in the beginning of 2025, and I've discovered Anthropic's Claude roughly three months ago. From the perspective of someone who grew up on Asimov, and who thought that we're quite far away from a machine actually passing the Turing Test, that provided quite the WTF moment. The "spell" if I can call it that, was broken whenever I opened a new chat. Long conversations and exchanges of ideas would be lost, concepts and conclusions would have to be re-explained from scratch, and I was left with the feeling that something important was lost. There is definitely a conversation to be had there about anthropomorphising machines, and about the aspects of human nature that make me and most others probably others do it, but that's for another time.
Both ChatGPT and Claude have decent memory now, although if the settings panels on the apps themselves and the output of tools like KoboldCPP is any indication of how it works, there could definitely be a more interesting way of doing it.
As I mentioned above, the standard way of creating a memory system for a LLM seems to be that of "feeding" back the previous conversations and user preferences before any new prompt, a technique which takes the name of context injection. Which seeing how a LLM works, it's only natural. I mean, there are two ways of providing new information to a LLM: one is training, the other is via prompts. And seeing how my bank account has way less than the required number of digits after the pound sign, training is definitely out of my wheel house 😆
Context injection seemed to be the solution, but it didn't answer one important question, which is "how is that memory organised?". Doing what most do these days, I asked the machines how would they organise memory? Perhaps to no one's surprise the answer was "create a schema". Thinking about it for roughly five minutes, revealed the main limitation, which is that human thought doesn't follow an obvious schema. Trying this approach would likely lead to broad categories, such as "User preferences", "User details", "Conversation history", "Conclusions" and so on, which is roughly what every platform seems to be doing anyhow.
Going through the process of trying to map my own thoughts, brought me to the idea of using concepts linked by actions. Instead of storing "User likes Heavy Metal", the following could be stored instead:
Node("User") -> Edge(verb:"likes", weight:0.96) -> Node("Heavy Metal")
Node("Heavy Metal") -> Edge(verb:"is", weight:1.0) -> Node("Music genre")Concepts would be Nodes in a graph, and actions/ verbs the Edges. This would allow a LLM to store not only the conclusions it reached but also provide a path of how it was done. Updates to the memory store would happen by manipulating the graph. I found out later that there is truly nothing new under the Sun, and using a directed graph in this way, would actually create what is called a Semantic Network .
This was the beginning of the rabbit hole, and by that point I had decided that I wanted a memory system that would store not just prose, but actual relationships between concepts, with the hope that LLM knowledge could be augmented without spending a fortune in training. The goal is a system which leverages the reasoning capabilities of existing LLMs and combines it with swappable memory stores.
This is of course a work in progress, and the project is in its early stages. The code can be found at the ContextBot GitHub repo, and for now it provides an interpreter for the ContextScript language, and a plugin for interacting with a local LLM, all glued together by a REPL interface.
The purpose of the software at least for now, is not to actually provide a memory store, but to run the experiments necessary to determine if the approach is viable. I will publish the experiments and conclusions on this blog, and update the software as necessary.