Beyond RAG: How the OKF + RAG Hybrid Architecture Makes AI Agents Actually Reliable

Ask your AI agent the simplest business question, “What’s our refund window?”,  and watch what happens. It answers instantly. Confidently. And completely wrong. It never opened your policy file. It guessed from training data.

That’s the problem RAG was supposed to fix. And it did, partly. Retrieval-Augmented Generation cut hallucinations dramatically by grounding models in your actual documents. But “dramatically better” and “reliable enough for legal, financial, or medical work” are two different thresholds. Stanford’s RegLab found that production legal AI tools still hallucinated on 17–34% of queries even after implementing RAG. That’s not a product footnote. For contract review or patient records, it’s a liability.

Here’s the finding that changes the conversation: aggregated enterprise deployment data from 2025، including studies by Vectara, Galileo, and multiple Fortune 500 internal audits, shows that when the same model queries ungoverned, untagged data via standard RAG pipelines, up to 52% of responses contain fabricated or unverifiable information. The same model on governed, structured data with clear ownership and freshness metadata drops to near-zero. Same model. Different data layer.

That gap, 52% versus zero, is the entire architectural argument for what comes next.

In June 2026, Google Cloud published the Open Knowledge Format (OKF), a deliberately simple specification that formalizes the “LLM wiki” pattern into a portable, version-controlled standard. Used alongside RAG, not instead of it, OKF creates a dual-layer system: high-stakes queries hit deterministic, curated knowledge; open-ended questions get the full power of semantic search. A router in front decides which path every query takes.

By the end of this guide, you’ll understand how to wire that architecture together, when each layer should take over, and how organizations are already using this to move from production hallucination crises to near-zero error rates, with the same underlying model they started with.

The Problem RAG Was Supposed to Solve – But Didn’t Fully

RAG was a genuine breakthrough. Before it, language models answered from frozen training weights, no access to your documents, your policies, your live business data. Hallucination was structural because the model had no choice but to fill gaps with plausible-sounding guesses.

RAG changed that. Agents now retrieve relevant chunks from your actual documents and pass them to the model at inference time. According to multiple 2025 research compilations, properly implemented RAG reduces hallucinations by 40–71% in controlled settings, a real improvement. Gartner projects that 80% of GenAI applications will be built on RAG by 2028.

But RAG has a ceiling. And enterprise deployments keep finding it.

Semantic Proximity Is Not Truth

RAG’s core mechanic is vector search. Your query becomes a vector, a list of numbers encoding its meaning, and the database returns the chunks mathematically nearest to it. The magic is meaning-based matching: different words, same idea, RAG still finds it.

The problem is that “semantically nearest” and “factually correct” are not the same thing.

Ask about your current refund policy and RAG might surface an archived version from two years ago, the text is similar. Ask about a price and it retrieves a promotional page that’s been superseded. The model gets three temporally mismatched fragments, assembles a confident answer, and the user has no idea the information is stale.

Research from MIT CSAIL and collaborating institutions (2025) found that large language models consistently use more assertive, confident language when generating hallucinated content than when producing accurate responses, making retrieval errors harder, not easier, to catch without systematic verification frameworks.

Chunking Destroys Structure

RAG works by slicing documents into pieces,  typically a few hundred tokens each. A table gets split across two chunks. A numbered procedure loses its sequence. A contract clause arrives stripped of the surrounding clauses that gave it legal meaning.

The model gets three fragments from what was one coherent document and fills the gaps between them. That gap-filling, done with fluency and confidence, is exactly what hallucination looks like in production.

The Governance Black Box

Change a policy. Update a price. Revise a compliance rule. In a RAG system, that change requires re-chunking the document, generating new embeddings, updating the vector store, and waiting for the pipeline to complete. You can’t diff what changed. You can’t review the update in a pull request. You can’t audit what the agent “knew” at any specific point in time.

McKinsey’s 2025 State of AI survey found that 88% of organizations now use AI regularly, yet only 39% report enterprise-level EBIT impact. The most common reported problem: AI inaccuracy. The organizations struggling most aren’t running bad models, they’re running on ungoverned data pipelines where the model can’t distinguish authoritative from outdated.

The 52% Gap Nobody Talks About

This is the number that reframes the architecture problem. Enterprise research consistently shows that the same model produces fabricated information on 52% of responses when querying ungoverned RAG data sources, and drops to near-zero on governed, structured data. Not a different model. Not different training. Just a different data layer.

That finding is the architectural case for everything in this guide.

The Two Types of Hallucination – And Why Only One Is a Model Problem

Most teams treat hallucination as a single problem with a single fix. It isn’t. There are two fundamentally different failure modes, and they need different solutions.

Model-level hallucinations originate inside the LLM itself:

• Factual hallucination — the model invents information that doesn’t exist anywhere
• Faithfulness hallucination — the model contradicts the context it was given
• Temporal hallucination — the model presents outdated information as current

These are genuine model limitations. Better training, RLHF, and reasoning improvements chip away at them over time.

Context failure hallucinations happen before the model even generates a word:

• Missing metadata — the retrieved chunk has no date, no owner, no source attribution
• Stale lineage — the document was updated in your CMS but the old chunk is still in the vector index
• Conflicting sources — two chunks make opposite claims and the model has no way to know which is authoritative
• Untrusted data — the source is a forum post or an unverified internal draft, but the model treats it as fact

This second category is entirely fixable in the governance layer, before retrieval even happens. No model upgrade required. No fine-tuning. Just structured, governed data with clear ownership and freshness signals.

OKF eliminates context failure hallucinations for the knowledge it covers by design: every concept has an owner, a timestamp, and a traceable source link. RAG governance, source tagging, freshness metadata, trust scoring, addresses the same problem in the long-tail layer.

Note: Most hallucination benchmarks measure model-level failures in controlled settings. They significantly undercount context failure hallucinations, which dominate real production deployments on messy enterprise data. That’s why the 52% figure from ungoverned enterprise RAG is so much higher than controlled benchmark numbers suggest it should be.

The variance isn’t in the model. It’s in the infrastructure underneath it.

Enter OKF: When Knowledge Works Like Code

On June 12, 2026, Google Cloud published Open Knowledge Format (OKF) v0.1, authored by Sam McVeety and Amir Hormati, Tech Leads from Google’s Data Analytics and BigQuery teams.

The spec fits on a single page. That’s intentional.

OKF represents organizational knowledge as a directory of markdown files with YAML frontmatter. One concept per file. Each file linked to others with ordinary markdown links. The whole bundle lives in git. No database. No embeddings. No runtime. No SDK. As the official spec puts it: “just markdown, just files, just YAML frontmatter.”

“Format, Not Platform”: Why This Is the Key Insight

Every previous enterprise knowledge solution was a platform: proprietary catalog, custom API, vendor lock-in. Every new agent you built required rebuilding the same context-assembly pipeline for that platform from scratch.

OKF’s contribution is a specification, not a product. Knowledge written by one producer can be consumed by any agent without translation. The same OKF bundle that Claude reads can be consumed by GPT-4, Gemini, or any future model, without rebuilding the knowledge layer. It will, as the Google Cloud blog states, “never require a proprietary account or SDK to read, write, or serve.”

OKF formalizes what practitioners like Andrej Karpathy had already identified: an LLM wiki, a curated, linked markdown knowledge base that agents read, update, and maintain, delivers dramatically better context than repeated document search. The format writes that pattern down as one open, interoperable standard.

What an OKF Bundle Actually Looks Like

A bundle is a folder structure any developer would recognize:

acme-knowledge/
├── index.md           ← Root navigation map
├── log.md             ← Change history
├── tables/
│   ├── index.md
│   ├── orders.md      ← One concept per file
│   └── customers.md
├── metrics/
│   ├── index.md
│   └── monthly_revenue.md
└── playbooks/
    └── incident_response.md

Each concept file has a minimal YAML block at the top, then clean markdown:

---
type: BigQuery Table
title: Customer Orders
description: One row per completed customer order. Source of truth for revenue.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
tags: [sales, revenue, source-of-truth]
timestamp: 2026-05-28T14:30:00Z
owner: data-engineering@acme.com
---

The only required field is type. Everything else is optional. This deliberate minimalism means any team can adopt OKF without a three-month schema design project. The markdown body holds the real knowledge: schemas, business rules, join paths, deprecation warnings. Concepts link to each other with ordinary markdown links, turning a flat folder into a navigable knowledge graph.

The Progressive Disclosure Pattern

One of OKF’s most practical features is the index.md file at every directory level. When an agent enters a section of the knowledge base, it reads the index first, getting a map of what exists before drilling into specifics. This solves the context-stuffing problem: instead of loading the entire knowledge base, the agent navigates like a developer browsing a codebase. Start at root. Read the overview. Follow the links.

Why Git Version Control Transforms the Economics

This is where OKF diverges sharply from every other knowledge approach.

When an OKF bundle lives in a git repository, every change becomes a commit: diffable, reviewable, attributed to an owner, rollback-able in seconds. You can run a pull request for a knowledge update the same way you review code changes.

Compare that to a RAG pipeline. Change a policy and you re-chunk, re-embed, re-index, and wait. No diff. No pull request. No audit trail. The old chunks sit in the index for hours after the truth has moved on.

For regulated industries, healthcare, finance, legal, this isn’t a developer convenience. It’s the difference between an architecture that survives a compliance audit and one that doesn’t.

Section summary: OKF doesn’t replace RAG. It makes the 20% of knowledge that absolutely cannot be wrong, policies, prices, procedures, regulations, deterministic, traceable, and human-reviewable. Everything else stays in RAG.

The Router: The Intelligence Layer Between Both Systems

OKF and RAG are two powerful but incomplete systems. Precision without scale. Scale without precision. The component that makes them work together is the router, a query classifier that sits in front of both.

Every query gets assessed before any retrieval happens. The router asks two questions:

1. How high are the stakes if this answer is wrong?
2. Does this need a specific canonical fact, or contextual, exploratory knowledge?

Based on those two dimensions, the query routes to OKF, RAG, or both.

The Stakes Taxonomy

High-stakes queries have a single correct answer that exists in your defined knowledge. Getting it wrong creates direct, measurable harm:

• “What is our refund window?” → goes to OKF. Exact answer. Policy linked.
• “What’s the current dosage for this medication?” → goes to OKF. Never guessed.
• “What does clause 14 of the enterprise agreement say?” → goes to OKF. Verbatim.

Open-ended queries span thousands of documents and have no single correct answer. Scale is the asset:

• “Has anyone seen this billing error before?” → goes to RAG. Search 40,000 tickets.
• “What are customers saying about our checkout flow?” → goes to RAG.
• “Summarize the key themes from last quarter’s support calls.” → goes to RAG.

Hybrid queries need both: the authoritative facts from OKF plus contextual depth from RAG. The agent retrieves the canonical answer first, uses it to verify or correct what RAG surfaces.

Router Implementation: Simple to Sophisticated

The simplest router is rule-based: any query containing keywords associated with your known canonical concepts routes to OKF first. This is fast, auditable, and a correct starting point.

More sophisticated implementations use a semantic classifier that compares query embeddings against pre-defined route embeddings — no LLM call required, very fast. Enterprise deployments use multi-level classification:

1. Level 1 – Compliance check: Can this query be answered at all? Does it require authentication?
2. Level 2 – Domain routing: Is this about HR policy, product specs, or engineering documentation?
3. Level 3 – Source selection: Within that domain, OKF, RAG, or both?

The LinkedIn CAPT framework is a real production implementation of this principle: structured playbooks for deterministic high-stakes operations, probabilistic retrieval for exploratory queries, all routed through MCP. Results confirmed: over 1,000 engineers using it, with a reported 70% reduction in issue triage time.

The “Retrieve When You Should Compile” Failure Mode

There’s a dangerous anti-pattern in systems without a proper router. The agent retrieves a plausible-sounding paragraph when it should be computing a defensible answer from structured data.

Ask “What was our revenue last quarter?” without routing, and the agent might search through PDF reports, retrieve a sentence mentioning a number, and present it confidently. That number might be a forecast. It might be for a different product line. It might be 18 months old.

With a router, that query routes directly to a specific OKF concept file for quarterly revenue, linked to the actual data source. The answer is computed from the record. Verifiable, not probable.

That distinction is the difference between an architecture that passes a financial audit and one that creates liability.

Router Decision Logic in Practice

The pseudocode below is simplified but represents how production routers actually work. The key insight: classification happens before any retrieval, and high-stakes paths are hard-coded, not inferred.

FUNCTION route_query(query):

    // Step 1: Check against known high-stakes keyword patterns
    IF query matches canonical_keywords[]:
        → Route to OKF (deterministic)
        → Return exact file content + source link + timestamp
        → STOP — do not continue to RAG

    // Step 2: Classify intent with semantic router
    intent = classify_intent(query)
    // STRUCTURED  = needs a specific fact, policy, or metric
    // EXPLORATORY = open-ended, contextual, multi-document
    // HYBRID      = needs both canonical facts + context

    // Step 3: Route based on classification
    IF intent == STRUCTURED:
        → Check OKF index for matching concept
        → If found: return OKF answer with citation
        → If not found: flag as "knowledge gap" for curation

    IF intent == EXPLORATORY:
        → Route to RAG pipeline
        → Apply reranking + freshness scoring
        → Return top-k chunks with source attribution

    IF intent == HYBRID:
        → Fetch OKF canonical facts first
        → Run RAG for contextual depth
        → Use OKF output to verify/correct RAG result
        → Merge into single grounded response

    // Fallback: when confidence is low
    IF confidence_score < threshold:
        → Return "I need more information" + suggest sources
        → Do NOT hallucinate a confident answer

Warning: The most dangerous router failure is a false EXPLORATORY classification on a high-stakes query. Always maintain an explicit hard-coded list of canonical query patterns that bypass classification entirely and go straight to OKF. Never rely solely on a learned classifier for patient safety, legal, or financial data.

Section summary: The router isn’t a nice-to-have optimization. It’s the component that makes OKF and RAG work as a system. Without it, you have two knowledge layers with no intelligence deciding which one answers what.

The Dual-Layer Architecture in Practice

With the router in place, the full system comes together. Two knowledge layers, one agent on top.

OKF is the spine: The curated 80%, the precise, compliance-relevant facts that the agent cannot get wrong. Refund windows. Pricing tables. Dosages. Regulatory requirements. SLA commitments. Version-controlled in git, with clear ownership and a traceable update history.

RAG is the reach: The long tail,  the sprawling, unpredictable 20% no team could curate by hand. Forty thousand support tickets. Years of transcripts. Archived research. Unstructured documentation from a dozen sources. Semantic search handles all of it without requiring anyone to write a concept file.

The router is the intelligence layer that reads each query, classifies it, and directs it where it will be answered most reliably.

How the Layers Make Each Other Better

This is the insight that gets lost when people frame OKF and RAG as competing solutions. They’re not. They improve each other.

OKF gives RAG ground truth. When a curated concept exists for a question, the agent uses it over anything retrieved from vector search. This single rule makes a measurable dent in hallucinations: the authoritative answer pre-empts the plausible-but-wrong one. OKF also gives the agent a map, the index.md hierarchy tells it what knowledge exists before any search runs, so it drills into RAG only where curated answers run out.

RAG gives OKF reach. No team can curate a decade of raw documents. RAG handles the edge cases, historical questions, and exploratory research that no knowledge base can anticipate.

Neither has to pretend it can do the other’s job.

The Self-Maintaining Knowledge Loop

Most discussions miss this angle entirely: OKF is designed to be maintained by AI agents, not just consumed by them.

Google’s reference implementation includes an enrichment agent, built on Agent Development Kit and Gemini, that walks a BigQuery dataset and automatically drafts OKF concept documents for every table and view. A second LLM pass enriches each concept with citations, schemas, and join paths. Humans review the drafts in a pull request, approve, merge. Going forward, the agent handles cross-reference updates and change detection.

This solves a problem that has killed every enterprise wiki initiative for thirty years: maintenance exhaustion. Wikis die because keeping them current is an endless, unrewarded cognitive burden. With OKF and an enrichment agent, the maintenance loop is automated. Humans govern. Agents maintain. The knowledge stays current without the burnout.

Verified Case Studies: Real Organizations, Real Results

Every case study here is sourced from publicly documented deployments. No projections, no “similar organizations.” Real results.

Important context: Not every organization below used OKF specifically, the format was published in June 2026. What these deployments share is the architectural pattern OKF now formalizes: deterministic, structured knowledge for high-stakes queries paired with probabilistic retrieval for exploratory ones, connected by an intelligent router. OKF standardizes this pattern into a portable, open specification.

Case Study 1: Google Knowledge Catalog + OKF (June 2026)

The challenge: Google’s internal teams had knowledge fragmented across metadata catalogs, wikis, code comments, shared drives, and individual engineers. Every new agent deployment required rebuilding the same context-assembly pipeline from scratch, for whichever proprietary system it happened to touch.

The solution: Google transformed its Dataplex product into Knowledge Catalog, which natively ingests OKF bundles and serves them to agents. They shipped three sample bundles immediately, GA4 e-commerce, Stack Overflow, and Bitcoin public datasets, alongside the enrichment agent and a static HTML visualizer. The official Google Cloud documentation confirms Knowledge Catalog “curates a context graph that links BigQuery datasets with business semantics, helping reduce AI hallucinations by ensuring models use enterprise-approved data.”

The outcome: Any agent framework, Claude, GPT-4, Gemini, or any future model, can now consume curated enterprise context from the same OKF bundle without translation layers. Knowledge becomes portable across a changing model landscape.

The key lesson: Vendor neutrality isn’t a philosophical preference. It’s the architectural property that makes a knowledge investment durable when models change, and they always change.

Case Study 2: LinkedIn CAPT — Structured Playbooks at 1,000+ Engineers

The challenge: LinkedIn’s AI coding agents were powerful in isolation but blind to organizational context: thousands of internal microservices, proprietary frameworks, specialized systems. Without that context, agents produced plausible-but-wrong suggestions for company-specific engineering tasks.

The solution: LinkedIn built CAPT (Contextual Agent Playbooks and Tools), a framework on the Model Context Protocol (MCP). CAPT encodes rules, procedures, and verification steps as structured, version-controlled playbooks, deterministic paths for high-stakes engineering operations. Probabilistic RAG handles exploratory queries. A router distinguishes between the two.

One specific playbook encoded the entire A/B test cleanup process, previously requiring deep expertise that bottlenecked around a few experts. Any engineer on any team can now run a safe, consistent cleanup without touching the experimentation stack.

The outcome: Over 1,000 LinkedIn engineers using CAPT daily by January 2026. The New Stack reported a confirmed 70% reduction in issue triage time across many areas, based on a direct quote from LinkedIn’s Prakash. Data analysis workflows ran 3x faster.

The key lesson: The most valuable organizational knowledge isn’t facts, it’s procedures. Encoding how to do things correctly, deterministically, is often worth more than encoding what is true.

Case Study 3: Infosys Topaz + Energy Supplier — 70% Autonomous, 60% Faster

The challenge: A large energy supplier ran ~5,000 field support calls per week. Meter technicians called from customer sites needing guidance on equipment issues. Average handling time for the top 10 issue categories exceeded 5 minutes. 60–70% of issues were repetitive.

The solution: Infosys Topaz built a hybrid system on Amazon Bedrock (Claude Sonnet) with Amazon OpenSearch Serverless. Structured equipment specs, safety protocols, and compliance checklists route deterministically. Historical incident reports and troubleshooting manuals go through RAG. The router distinguishes “what are the specs for this meter model?” (exact, deterministic) from “what did technicians do last time this error appeared?” (exploratory, semantic).

The outcome, documented on AWS’s official blog:

• AI now handles 70% of previously human-managed calls autonomously
• Average handling time dropped 60% (from over 5 minutes to under 2 minutes)
• Human intervention required dropped from 30–40% to 20% within 6 months
• Customer satisfaction scores increased 30%

The key lesson: In domains with physical safety stakes, the architecture of the knowledge system is a safety decision, not just a technical one. Equipment specs and compliance checklists never go through probabilistic retrieval.

Case Study 4: Healthcare Procedure Coding – 12 Minutes to Seconds

The challenge: A large public healthcare company specializing in radiology software needed to automate procedure code selection during patient scheduling. The manual process took 12–15 minutes per call. Procedure codes varied by clinic, state regulations, patient history, and dozens of other factors, creating explosive configuration complexity.

The solution: Hasura/PromptQL’s domain-specific LLM platform combined a deterministic code database, exact match on procedure codes, no guessing, with a contextual layer handling the patient history and regulatory context. The router distinguishes exact code lookup (never probabilistic) from contextual interpretation (semantic).

The outcome: Call times dropped from 12–15 minutes to seconds. The company’s stated projected business impact is $50–100 million from increased efficiency and reduced training costs. Hallucinated medical codes, which create billing liability and compliance risk, dropped to near-zero by removing probabilistic retrieval from the code selection path entirely.

The key lesson: Identifying which subset of queries cannot afford to be probabilistic, and routing them away from RAG entirely, is the highest-leverage architectural decision in any high-stakes AI deployment.

Case Study 5: Databricks Supervisor Agent – 20%+ on Hybrid Queries

The challenge: Complex enterprise queries routinely require joining structured data with unstructured content. “Find papers by a co-author with 115 publications about Rydberg atoms” needs structured author-count filtering and semantic topic matching, a single-turn RAG system cannot split that query, route each half correctly, and combine the results.

The solution: Databricks built the Supervisor Agent to orchestrate multi-step reasoning across structured and unstructured sources. Structured sub-tasks route to SQL-queryable data. Semantic sub-tasks route to vector search. A supervisor layer decomposes and recombines.

The outcome: VentureBeat’s coverage of Databricks’ research confirmed gains of 20%+ over state-of-the-art single-turn RAG baselines on Stanford’s STaRK benchmark. Critically, Databricks ran the same benchmark using a strongerfoundation model on single-turn RAG, it still lost to the multi-step agent by 21% on academic domains and 38% on biomedical domains. The performance gap is architectural, not a model quality problem.

The key lesson: You cannot solve an architecture problem by upgrading the model. Better routing and hybrid retrieval consistently outperform better models on single-retrieval baselines.

Before vs. After: What This Architecture Actually Changes

The clearest way to see the value of the hybrid stack is to walk the same two questions through a system without it, then with it.

The shift isn’t just technical. It’s organizational. When knowledge is version-controlled and owned, teams stop asking “is the agent reliable?” and start asking “is this specific concept file current?”, a much simpler, more actionable question.

Common Mistakes Teams Make Building This Architecture

Mistake 1: Assuming RAG Is a Solved Problem

The most expensive assumption in enterprise AI: adding RAG means hallucination is handled. It isn’t. Stanford’s RegLab study found legal RAG tools still hallucinating on 17–33% of production queries. “Better than a base LLM” and “reliable enough for legal work” are different standards.

The fix: identify which query categories can never afford probabilistic retrieval, and route them away from RAG entirely before deployment.

Mistake 2: Trying to Put Everything in OKF

OKF is a precision tool, not a general-purpose document store. Every concept requires human curation, that’s what makes it trustworthy, and also what constrains its scale. Teams that try to migrate their entire document archive into OKF files spend months building something already outdated.

OKF is for the knowledge you cannot afford to get wrong. RAG handles everything else.

Mistake 3: Making the Router Probabilistic

Some teams deploy both layers but let the model decide which source to query. This reintroduces probabilistic behavior at the routing level, the model might “choose” RAG for a medication dosage because the retrieved chunk looked more detailed. The router must be deterministic for high-stakes classifications. Not “usually routes to OKF.” Always.

Mistake 4: Ignoring Re-indexing Costs

When knowledge changes in a RAG system, the update isn’t free. Re-chunk the document, generate new embeddings, update the vector index, wait for propagation. At scale, this costs real compute and time, and outdated chunks sit live during the gap.

An OKF update is a git commit. One file. One line changed. Instantly visible to any agent reading the bundle.

For knowledge that changes frequently, pricing, policies, active promotions, the operational cost difference over a year is significant.

Mistake 5: Treating Governance as a Phase 2 Problem

McKinsey’s 2025 State of AI findings are consistent: only 6% of organizations qualify as AI high performers (5%+ EBIT impact). The distinguishing factor isn’t model capability, it’s governance infrastructure, data quality, and measurement. Organizations that bolt governance onto an existing RAG system after deployment discover they’ve built a reliable-looking interface on top of an ungoverned data layer. OKF + RAG builds governance in from day one.

Tip: The fastest way to identify your governance gaps: ask your current RAG system “when was the last time this information was verified?” If it can’t answer, your governance layer doesn’t exist yet.

The Hidden Economics of the Hybrid Stack

Here’s the counterintuitive finding: the hybrid architecture often costs less to operate than RAG alone, once you count the full picture.

The Re-Indexing Tax

Infrastructure costs account for 35–50% of total RAG deployment budgets, and a significant portion is ongoing re-embedding. Documents change. Policies update. Products get discontinued. At scale, that’s a continuous compute expense, and a continuous risk of stale knowledge.

OKF updates are git commits. No embedding pipeline. No compute. No propagation delay.

The 90/10 Infrastructure Rule

The most consistent finding from 2025 enterprise AI architecture research: production AI agent success follows roughly a 90% infrastructure / 10% model capability split. A better model does not fix a context failure. It amplifies it, producing more fluent, more confident, harder-to-detect wrong answers.

BCG’s 2025 study of 1,250 executives found that 74% of companies struggle to scale AI value. BCG’s finding: the gap is infrastructure and operating model, not model quality.

40–50% of Budget Is Content Preparation Anyway

Enterprise AI deployments consistently show that 40–50% of total project effort goes to content preparation and data architecture, not model fine-tuning, not UI, not deployment. Cleaning data, designing schemas, establishing governance.

That budget is being spent regardless. The question is whether it produces a re-indexable blob of document chunks, or a version-controlled, human-reviewable knowledge graph that any future agent can read without rebuilding anything.

OKF makes that investment durable. The markdown concept files don’t need to be rebuilt when you switch models. They don’t go stale when you upgrade your vector database. They’re just files.

Section summary: The hybrid architecture isn’t more expensive than RAG alone, it’s cheaper once you account for re-indexing costs, governance overhead, and the compounding value of a knowledge base that any future model can read without rebuilding.

Implementation Roadmap: From Zero to Hybrid

This doesn’t need to be a six-month project. Here’s a sequence that delivers value at each phase.

Phase 1: Knowledge Audit (Weeks 1–2)

Before building anything, classify what you know. Pull every knowledge asset: policies, runbooks, data dictionaries, pricing, product specs, compliance documents.

For each category, answer: How often does this change? What’s the cost if the agent gets it wrong?

High-stakes + needs exact answer = OKF candidate. High-volume + exploratory = RAG candidate. Most organizations discover that 60–70% of their highest-value queries come from a surprisingly small set of precisely defined facts.

Phase 2: Build Your First OKF Bundle (Weeks 3–6)

Start with five to ten concept files. Pick what your current agent gets wrong most often. Write one file per concept. Add type, a clear description, a resource link, and a precise markdown body.

Put the bundle in a git repository. Set up a PR review workflow for updates. That’s your foundation, a version-controlled knowledge base any agent can read, with a complete audit trail from day one.

Phase 3: Deploy RAG for the Long Tail (Weeks 4–8, parallel)

Standard RAG pipeline: document ingestion, chunking, embedding, vector store. The key addition in a hybrid system is tagging each source document with type and recency metadata, information the router will use when classifying queries.

Pinecone, Weaviate, and Chroma are production-proven for the vector database layer. LangChain and LlamaIndex have mature primitives for the ingestion pipeline.

Phase 4: Implement the Router (Weeks 6–10)

Start rule-based. Explicit keyword matching for known high-stakes query categories, fast to build, easy to audit, correctness is transparent. Upgrade to a semantic or LLM-completion router once you have enough query volume to evaluate empirically.

Minimum viable router: any query touching your curated canonical concepts routes to OKF first. Everything else goes to RAG.

Phase 5: Enable the Self-Maintaining Loop (Ongoing)

Point an enrichment agent at your OKF bundle to detect stale concept files and draft updates for human review. Start with highest-churn concepts, pricing, active policies, current promotions, and build the review workflow before expanding coverage.

This is the compounding phase. Each week of agent-assisted maintenance is a week the knowledge base stays current without human cognitive load.

Two Myths Worth Killing

Myth 1: “OKF Makes RAG Obsolete”

It doesn’t. The moment your corpus is too large or too unpredictable to curate, vector search wins, and it isn’t close. Fifty thousand support tickets cannot be curated into OKF files. Years of unstructured reports cannot either. RAG handles the scale that no human-curated system will ever match.

The hybrid architecture doesn’t diminish RAG’s role. It focuses RAG where it excels and stops asking it to do something it was never designed to: guarantee a single factually correct answer on a high-stakes question.

Myth 2: “Just Stuff Everything into the Context Window”

As context windows have grown to millions of tokens, some teams propose replacing retrieval with context injection,  just feed the agent everything it might need.

The data contradicts this. Irrelevant context actively degrades generation quality. Studies on long-context models consistently show that adding irrelevant documents lowers response accuracy even when the correct documents are also present. The model gets confused by the noise.

Selective, routed retrieval beats brute-force context injection regardless of context window size. Routing intelligence is not a workaround for limited context. It’s an architectural principle.

The Future: Determinism + Probability = Reliability

The convergence happening in production AI is clear: structured and unstructured retrieval are merging into a single stack.

Model Context Protocol (MCP), which reached 97 million monthly SDK downloads in 2025, is becoming the standard interface connecting agents to both OKF bundles and RAG indexes through a single API. An agent doesn’t need to know whether it’s querying a markdown file or a vector database. It asks for knowledge. The routing and plumbing stay hidden underneath.

Graph and vector approaches are converging. Combining knowledge graph queries, structured relationships, with vector store searches, semantic matching, gives agents knowledge that’s precise on facts and flexible on context. OKF’s markdown link graph is an early, simple implementation of this pattern, already formalized and open.

And the self-maintaining knowledge loop, agents maintaining OKF bundles that humans review via pull requests, represents a fundamental shift in how organizations manage knowledge. For three decades, the problem was that humans couldn’t keep knowledge systems current at scale. Agents can. The new model: agents maintain, humans govern.

As Gartner projects 80% of GenAI apps built on RAG by 2028, the organizations already building the governance and routing infrastructure to make that RAG reliable, pairing it with structured, version-controlled knowledge layers, will be the ones McKinsey identifies as true AI high performers.

Conclusion: It Was Never One or the Other

The question for enterprise AI in 2026 isn’t which knowledge system to choose. It’s how to architect both of them together.

OKF gives you precision where it matters most, the 20% of knowledge that generates 80% of your highest-stakes queries. Auditability. Version control. A traceable answer with a source link, not a confident guess from a semantic matching algorithm.

RAG gives you scale, the ability to search millions of documents, tickets, transcripts, and reports that no team could ever curate by hand. The long tail no knowledge base can anticipate.

The router makes the system intelligent, routing each query to whichever layer is best equipped to answer it reliably, without the agent needing to know the difference.

The Infosys case alone, a verified 70% autonomous resolution rate, 60% faster handling time, from a production deployment documented by AWS, shows what happens when this architecture is implemented correctly. The healthcare deployment shows what it means in domains where errors aren’t inconveniences, they’re liabilities.

Frequently Asked Questions

Founder & Lead Writer

Ahmed Gadallah

Computer Scientist & Data Architect specializing in software engineering, advanced data analytics, and AI. As the founder of DSN Daily, he delivers data-driven insights across science, technology, and business, turning complex knowledge into actionable strategies that help readers make smarter decisions and stay ahead of emerging trends.

LinkedIn Twitter / X Facebook YouTube