We’ve been working on a problem that kept showing up while building AI agents: managing domain context.
MCP gives an agent access to tools. Skills help define how it should behave. Memory can store information from previous interactions.
But a domain-specific agent also needs to know how the things in its world fit together.
A support agent needs to understand customers, accounts, tickets, products and known issues. An SRE agent deals with services, databases, deployments, incidents, and runbooks. A sales agent has companies, people, opportunities, meetings, and decisions.
We built Open Index to manage this layer.
It’s an open-source framework for representing domain knowledge as structured entities and relationships that an agent can navigate.
How did we actually land here?
Our earlier knowledge setup used embeddings and Markdown files.
It was a good place to start. As the amount of knowledge grew, though, we started seeing problems that retrieval alone wasn't solving.
The first was non-determinism.
We could replay the same or a very similar scenario and see the agent pick completely different documents. That changed the investigation path and sometimes the final result.
Then there was navigation.
A retrieved document might tell the agent something about a service, but the agent still had to figure out what to inspect next.
Which database does this service use? What else depends on that database? Which deployment touched the service recently? Have we seen a similar incident before?
That information existed, but the connections between those pieces weren't always explicit.
We also started seeing context poisoning. As more documents and information were added, the agent had more opportunities to pick up something irrelevant or outdated and follow the wrong path.
Updating the knowledge base became difficult too after a certain amount of knowledge.
When the agent learns something new, you need to know where that information belongs, whether it already exists, whether an older version should be replaced, and whether it conflicts with something else.
A folder of Markdown files gets difficult to manage once both humans and agents are continuously reading and writing to it.
From Markdown to structured context
We started adding indexes and structured data around our existing context.
That fixed some problems, but created a new set of things we had to handle:
updates and de-duplication
stale or conflicting information
relationships between entities
version history
tracking which context the agent used
Eventually, this became its own context layer.
We turned that work into Open Index.
How Open Index works
Open Index lets you define the objects that exist in your domain and the relationships between them.
For example, an SRE setup could have:
Service → Database → Deployment → Incident → Runbook
A support setup could have:
Customer → Account → Ticket → Product → Known Issue
You define the models and schemas. You decide how information gets added or updated. You define how entities are connected.
The agent can then use those connections while working through a task.
Say an investigation starts with a service.
Instead of searching the entire knowledge base every time it needs more information, the agent can follow known relationships from that service to its dependencies, deployments, previous incidents or runbooks.
The context itself gives the agent a map of where it can go next.
What Open Index handles
There are three parts we care about.
Structured memory: Domain knowledge is stored as structured entities rather than only documents or chunks. The structure is defined by the domain. Open Index doesn't require every agent to use the same schema.
Navigation: Relationships between entities give the agent paths through the context. An agent investigating a service can move through the objects connected to that service instead of repeatedly trying to rediscover those connections through retrieval.
Relationships: The connections between entities can represent things such as dependencies, ownership, decisions and correlations. That gives the agent information about how two things are related, not only whether their text happens to be similar.
Getting started with Open Index
Open Index is domain agnostic. You can define the entities and relationships based on whatever your agent works with: marketing, sales, legal, support, security, SRE, or something else.
The basic flow is:
Define your entities and schemas → populate and update context → connect entities → let the agent navigate it.
You can run Open Index with a single command and see how the setup works before building it into an agent.
The tool is open source:
github.com/DrDroidLab/open-index
If you're working on one, we'd like to hear what your context setup looks like and what breaks as it grows.
