What Changes When You Build RAG on Cloudflare Instead of PostgreSQL?

Building RAG on Cloudflare instead of PostgreSQL changes how the system is divided. A conventional implementation tends to concentrate authorization, document metadata, vectors, sessions, and orchestration behind an application service and its databases. The Cloudflare implementation breaks those responsibilities into a managed ecosystem whose boundaries can align with agent concepts: a document can keep a stable identity, a search can keep state, and an agent harness can call a small set of document tools. The retrieval problem itself does not change.

The hypothesis is narrower than “Cloudflare makes RAG better.” A document-centric system may be easier to operate when each document has a stable logical identity, an independently versioned evidence release, and an explicit retrieval contract. In this implementation, vector search returns candidate locations; the system then reloads exact passages from immutable source objects and verifies their hashes before those passages can support an answer or citation. That discovery-versus-evidence rule is something we designed—it is not a benefit Cloudflare supplies automatically, and the same rule can be implemented with PostgreSQL and object storage. Cloudflare simply makes the separation visible because the responsibilities already live in different services.

The project began with the original book-to-skill idea: give an agent a compact skill that helps it navigate a book and answer questions about its contents. That is useful for fast orientation, but it is not always enough for scientific or software-engineering work. Sometimes I need the entire section verbatim, including the surrounding mathematical argument or probability distribution, rather than an abbreviated summary. We therefore built a new version of the workflow that treats the book—or any formal document—as a real retrieval problem: compile and preserve its hierarchy, publish immutable sections, index smaller retrieval units, and let a book-backed agent either answer from verified evidence or return the exact source sections without model interpretation. It is the first working form of a system I have wanted to build for years: a library of book-backed agents.

The implementation establishes that this alternative is feasible. One shared development runtime has exercised publication, exact section reads, and federated retrieval across multiple document agents. It does not prove that Cloudflare is categorically better than PostgreSQL. We have not run an apples-to-apples benchmark in which both architectures use the same corpus, retrieval policy, evaluation set, and production workload.

The useful comparison is therefore narrower and more practical: what moves when RAG is built on Cloudflare, what becomes easier, what becomes harder, and under which conditions an engineer might prefer that operating model to FastAPI, PostgreSQL, pgvector, and object storage.

The comparison starts above the infrastructure

Before comparing services, it helps to separate the logical RAG pipeline from its deployment topology. A credible system must do roughly the same work in either architecture:

The diagram compresses ten implementation concerns into six readable stages. “Hydrate evidence” means reloading the authoritative, source-ordered passages after retrieval has identified where to look. The final result includes citations, a machine-readable disposition, and enough timing and retrieval information to inspect what happened.

The cloud provider does not eliminate this work. It changes where each responsibility lives. More importantly, it does not guarantee answer quality. If the parser destroys a maintenance procedure's nested steps, if retrieval selects a troubleshooting anecdote instead of the manufacturer's diagnostic sequence, or if the model ignores the supplied evidence, the result will be poor on any platform.

The right comparison is therefore not “serverless AI versus traditional AI.” It is the same evidence discipline expressed through two different operating models.

Architecture A: document agents on Cloudflare

The Cloudflare design treats each governed document as a logical agent with a stable identity. That does not mean deploying one Worker for every manual, standard operating procedure, or formal governance document. A shared Worker hosts the runtime. The “agent” is the combination of a stable document identity, an active immutable release, a routing card, retrieval policy, exact-reading tools, and durable state where needed.

At the front door, Cloudflare Access authenticates machine-to-machine requests with a service token. An agent harness sends a client identifier and secret over HTTPS; Access validates that service identity before the Worker handles the request. This fit Hermes because it is an automated caller: it did not need an interactive sign-in flow, and it did not receive an account-wide administrative API token. The Worker exposes a small set of outcome-oriented tools through Model Context Protocol (MCP) rather than mirroring every internal API. Whichever agent harness you prefer—Claude Code, Codex, Hermes, OpenCode, or another MCP-capable runtime—can ask a document a question, request a citation, or read complete sections without model interpretation. Multi-document questions can open a federated search session instead. Cloudflare documents the service-token flow and its two authentication headers here.

The Cloudflare services are easiest to understand as a small ecosystem rather than a list of product names:

The result is a serverless architecture with very little standing infrastructure. The Worker receives MCP tool calls and coordinates policy and retrieval. Access protects that boundary. D1 records releases, routing cards, policies, inventory, and promotion state. R2 stores immutable source artifacts and the hash-verified sections used as evidence. Vectorize finds likely sections but does not serve as the citation source. Workers AI runs the embedding, reranking, and answer models. Durable Objects keep strongly consistent state for a document identity or federated search session.

That separation matters. A vector match is a clue, not a citation. The system will not allow discovery text alone to enter the answer prompt. After candidate selection and reranking, it fetches exact ranges from R2 and verifies their hashes. Only this canonical evidence can support a quotation or citation.

Cloudflare's documentation describes Agents as durable runtimes with identity, state, sessions, routing, scheduling, recovery, and observability; Durable Objects provide the stateful coordination primitive underneath that model. Those capabilities make the components fit together naturally, but the evidence boundary remains an application decision—not a default feature the platform provides for us.

Architecture B: FastAPI, PostgreSQL, and object storage

The conventional design puts a stateless API service at the centre. A mobile or web client reaches a load balancer or managed application platform, which distributes requests across at least two FastAPI instances when availability matters. PostgreSQL stores users, tenants, permissions, sessions, documents, versions, metadata, chunks, evaluation records, and operational state. The same database can store vectors through pgvector, or the application can use a separate managed vector database. Object storage holds source files and larger normalized artifacts. Hosted embedding, reranking, and generation APIs complete the pipeline.

This model is familiar for good reasons. PostgreSQL offers transactions, joins, mature indexing, row-level security, and an enormous operating ecosystem. A retrieval service can combine dense and sparse branches, reciprocal-rank fusion, metadata filters, relationship expansion, and final reranking. FastAPI gives the team an explicit application boundary for streaming, authorization, validation, prompt construction, retries, and provider adaptation.

The conventional architecture is also easier to move across infrastructure providers because the application owns more of the orchestration. If a company already operates PostgreSQL, containers, centralized observability, secrets management, and deployment pipelines, another API service may introduce less organizational novelty than adopting several Cloudflare-native primitives.

That control has a cost. Even a modest high-availability deployment has more continuously provisioned components, more connection and migration concerns, more backup and recovery decisions, and more failure domains that the application team must understand. The platform is not necessarily difficult; it is simply more visibly yours.

What actually changes between the two designs

The Cloudflare architecture trades familiar database-centric composition for managed, purpose-specific services. It reduces idle infrastructure and places ingress close to the user. Durable Objects remove much of the work involved in coordinating a stateful search session across requests. MCP gives an agent host a standard tool boundary. The trade-off is a stronger commitment to Cloudflare's execution and storage semantics, along with the need to reason carefully about which data belongs in D1, R2, Vectorize, or a Durable Object.

That commitment was part of the attraction. Cloudflare already operates a large delivery and security network close to users, and it is deliberately extending that network into an AI application layer. Its recent platform work positions Workers AI and AI Gateway as a unified inference path for hosted and third-party models, while its own inference-engine work argues that a distributed network needs a different cost and scheduling model from centralized multi-GPU clusters. That is a coherent strategic direction for agent workloads: request handling, state, retrieval, evidence storage, inference, and observability can run within one operating surface. It is not proof that Cloudflare is universally better than Azure. Azure has broader enterprise services and mature relational options; Cloudflare's advantage for this experiment was a smaller serverless footprint and fewer continuously provisioned components. That CDN-to-AI progression is why I was interested in testing the design. See Cloudflare's AI platform announcement and its account of the inference engine built for its distributed network.

The traditional architecture concentrates more of the system in application code and PostgreSQL. This makes transactional workflows, complex administrative queries, analytics joins, and row-level-security policies more direct. It also leaves model selection and vector infrastructure relatively portable. The trade-off is that availability, scaling, connection management, patching, deployment, and recovery have a larger operational footprint.

There is no inherent answer-quality advantage in either topology. If both use the same normalized source, query expansion, embeddings, filters, reranker, prompt, and generation model, their answer quality should be comparable. Architecture changes the cost and difficulty of delivering those components reliably; it does not absolve the team from building them.

Data ownership complicates the decision. A vendor-hosted multi-tenant RAG service can be economical, but it also makes the vendor the custodian of every customer's operational documents. For a small provider that does not want that responsibility, a customer-owned deployment is attractive. The Cloudflare stack can be provisioned in the customer's account, while the traditional stack can be delivered as customer-owned containers, databases, and object storage. In either case, authorization must be enforced before retrieval and preserved in caches. A cache key is not a security boundary.

Latency is the adoption gate

Edge ingress helps, but “edge” is not a synonym for “fast answer.” In our tests, model inference dominated. The observed live long-form book query used 8,394 input tokens and produced 880 output tokens through @cf/mistralai/mistral-small-3.1-24b-instruct. It took 38.748 seconds end to end. An earlier trial of the same model completed in 21.5 seconds. Both are too slow for an interactive field workflow.

The Cloudflare development deployment was built and exercised using resources available on Cloudflare's free tier. Its observed performance and answer quality therefore reflect the model choices, quotas, and operating constraints of that tier—not the best possible outcome under a paid production configuration.

Those measurements are still useful because they locate the problem. The system could retrieve, rerank, restore chunks to source order, verify evidence, and produce a grounded answer. The remaining adoption risk was not whether Cloudflare could run the architecture. It was whether the generation path could deliver the first useful information quickly enough.

A useful field tool should show a relevant, cited passage before it finishes composing a longer answer. For this project, the working target—not a current measurement—is p95 first useful evidence under five seconds and p99 complete answer under ten seconds from the technician's device. Exact section reads should bypass the language model entirely. Frequently required safety procedures should be available offline. Predictive caching can precompute likely evidence bundles from tomorrow's work orders, but it should cache authorized evidence rather than stale final answers.

This is a more honest performance strategy than waiting for a large model to finish and calling the result “edge AI.”

One HVAC question through both systems

Consider this illustrative request:

“I’m at a rooftop unit showing repeated flame failure after ignition. What should I check first, and which company procedure applies?”

The request arrives with authenticated organization and technician identifiers, a work-order reference, equipment metadata, and the user's permissions. Those contextual fields matter: they prevent one customer's manuals from leaking into another customer's answer and give retrieval a way to prefer the correct equipment family and current procedure revision.

The Cloudflare path

Access validates the service identity, the Worker handles the request, and a Durable Object stores the revisioned state of the federated search session. Inside that request, the router is ordinary application logic: it compares the question and equipment context with compact routing cards and selects a bounded set of document agents. A routing card is document-level metadata, not evidence and not a prompt asking a model to answer. A simplified card looks like this:

{
  "document": "equipment-service-manual",
  "topics": ["ignition", "flame sensing", "lockout"],
  "covers": ["diagnostics", "service procedure"],
  "excludes": ["building design"],
  "effective_release": "current"
}

The router might activate three logical agents: the specific rooftop-unit manual, the company's ignition-failure SOP, and the combustion-safety procedure. It does not query every document in the library.

If the user already knows the exact document, federation has no latency advantage. A conventional stack can filter one hybrid database query by document ID and will probably be faster because it performs fewer routing and coordination steps. The Cloudflare path can also bypass federation and query that document agent directly, but the gap remains an empirical question rather than a claimed win. Federation is for the harder cases: identifying which documents apply or combining evidence across several of them.

The implemented router is also not fanning out a collection of Mistral conversations. It ranks compact routing cards with lexical and vector signals, applies the BGE reranker, and selects a small document set. Those selected agents then run retrieval-only work concurrently: lexical and semantic candidate generation, active-release and metadata filters, reranking, source-order restoration, and hash-verified passage loading. They return evidence bundles, not prose answers. The system globally reranks and hydrates that combined evidence, then makes one Mistral synthesis call. If the evidence remains ambiguous, the same final stage asks a clarification tied to the documents already considered—for example, whether the flame signal drops before or after the gas valve closes.

This detail matters to the architecture comparison because the router is not allowed to change retrieval behavior ad hoc. Each request pins a versioned policy that defines candidate counts, thresholds, reranking depth, evidence limits, and maximum document fan-out. Experiments can evaluate bounded alternatives, but a reviewed activation is required before the active policy changes. A PostgreSQL implementation should enforce the same separation; otherwise we would be comparing two topologies that quietly use different retrieval behavior.

The traditional path

The identity layer validates the same request and FastAPI builds a typed pipeline context. A routing stage queries document summaries or catalogue metadata to identify the same three document families. PostgreSQL applies tenant and version filters; pgvector or a managed vector service returns semantic candidates; a lexical branch returns exact terminology matches. The retrieval service combines and reranks them, hydrates canonical passages from PostgreSQL or object storage, and constructs the same evidence-bound prompt.

The application records the request, selected documents, retrieval scores, citations, token usage, provider response, and latency. If the evidence is ambiguous, the API returns the same kind of typed clarification instead of manufacturing an answer.

The response contract

Both systems should return a machine-readable contract shaped like this:

{
  "status": "supported",
  "first_action": "Apply lockout and combustion-safety procedure before repeating ignition tests.",
  "checks": [
    "Inspect the flame sensor and connection.",
    "Confirm the ignition sequence reaches stable flame.",
    "Follow the model-specific pressure and valve checks."
  ],
  "reason": "Current source evidence connects post-ignition flame loss with flame-sensing and fuel-delivery checks.",
  "sources": [
    { "document": "Rooftop Unit Manual", "section": "Ignition and Flame Failure", "release": "current" },
    { "document": "Company SOP", "section": "Repeated Ignition Failure", "range": "steps 2-6" },
    { "document": "Combustion Safety Procedure", "section": "Lockout Before Inspection" }
  ],
  "next_action": "open_exact_sections"
}

That answer is illustrative, not a deployed HVAC result or maintenance instruction. Its purpose is to show the contract: useful first action, bounded explanation, exact sources, current revision, and a no-generation path to the original text. A valid empty result would say that the bounded search did not find enough evidence and suggest a targeted clarification. A provider outage would return a typed service error. Those states must never collapse into the same empty string.

Cost and operating comparison

For a concrete planning model, assume a 25-person HVAC company with 10 technicians making 100 team-wide RAG requests per operating day, 26 days per month. That is 2,600 monthly questions. Assume an average request sends 6,000 input tokens and receives 700 output tokens, while the reranker examines 32 candidates of roughly 600 tokens each.

When I ran the experiment, the September 2026 pricing snapshot used in my internal model put the Cloudflare path at approximately $0.00255 in gross inference per request: about $0.00249 for generation, roughly $0.000060 for reranking, and a negligible query-embedding amount. After the modeled daily Workers AI allowance, the 2,600 monthly questions contribute roughly $3.77 in billed AI usage. Storage and query services bring the modeled 2,500-document deployment to approximately $8.79 per month.

The conventional reference architecture uses managed application compute, managed PostgreSQL with pgvector, object storage, and the same hosted-model assumption. A lean single-region configuration is modeled at approximately $36.64 per month. An availability-oriented configuration with redundant application capacity and a stronger database posture is modeled at approximately $91.64 per month. Both figures exclude engineering labour, support, mobile application work, enterprise identity, and production observability.

These totals are estimates, not vendor quotes. The base-platform column below holds usage constant at 2,600 questions per month so the corpus-storage effect remains visible. The final column adds a separate adoption scenario in which a larger library serves more users making more requests throughout the day; it is illustrative, not a claim that document count itself causes usage.

The flat-looking base totals do not mean scale is free; they isolate corpus growth while holding traffic constant. The usage examples show the larger economic lever. At 10,000 questions per month, the planned 6,000-input-token and 700-output-token response shape produces about $24.93 in gross generation cost before reranking, daily free allocation, or the Workers plan minimum. More users asking more questions—especially in bursts—move inference cost much faster than this corpus moves storage cost. Free-tier thresholds, index limits, per-operation charges, and current model pricing must still be checked before any production commitment. The estimates also assume one active vector representation per document per environment. Accidentally retaining multiple historical vector sets would change the capacity picture.

A second projection uses the implementation's observed long-book answer rather than the shorter field-answer assumption above. That answer used 8,394 input tokens and 880 output tokens, for a conservative envelope of roughly 321 Workers AI neurons and $0.00354 per interpreted answer before free allocation. At 500 evenly distributed questions per month, the model remains within the daily free neuron allowance and the current 25-request-per-day application guard. At 2,500 questions per month, it exceeds both the free allowance and the development guard; sustained service requires Workers Paid and an approved cap increase. The planning estimate is approximately $10.53 per month, including the $5 plan minimum and about $5.53 in AI overage. Vectorize, R2, D1, Worker requests, and Durable Object calls remain within their stated included allocations at that volume.

These two projections answer different questions. The approximately $8.79 HVAC estimate uses a planned 6,000-input-token, 700-output-token field response across a modeled 2,500-document corpus. The $10.53 implementation projection holds the current small index constant and scales the larger observed book-answer shape. Neither is an invoice or a load test, and bursty traffic can change the result because the Workers AI allowance resets daily rather than accumulating monthly.

The maintenance-hour estimates are planning assumptions, not public benchmarks. The latency row is intentionally asymmetric because inventing a traditional-stack number would make the table look complete while making it less true.

Which architecture should an SMB choose?

The Cloudflare design is compelling when a small organization wants a customer-owned deployment, global ingress, stateful agent sessions, low idle cost, and fewer conventional infrastructure duties. It is especially attractive when the product naturally decomposes into immutable documents, bounded document agents, and a router that coordinates them.

The traditional design is stronger when the organization already runs a platform team, needs complex cross-entity transactions and analytics, relies heavily on PostgreSQL row-level security, or requires infrastructure portability and explicit control over every retrieval stage. It is also easier to justify when RAG is one feature inside a broader operational application whose authoritative data already lives in PostgreSQL.

For a 25-person HVAC business without a platform team, I would begin with the customer-owned Cloudflare deployment—but only after treating latency as a release gate. I would make exact source retrieval a first-class experience, stream cited evidence early, keep safety-critical material available offline, and reserve long synthesis for questions that genuinely need it. Low cost is useful; fast, trusted answers are the product.

Part 2, Building Versioned Document Agents and Federated Retrieval on Cloudflare, explains how we built that system: how a PDF becomes a signed release, why Vectorize never becomes the citation source, how Hermes talks to document agents through MCP, and how a federated router chooses which agents are worth asking.

Sources and verification notes

Platform descriptions were checked against current first-party documentation on September 8, 2026: Cloudflare Agents, Durable Objects, Vectorize, R2, D1, Workers AI, MCP on Cloudflare, and Cloudflare Access service tokens.

Architecture, latency, token, and model-test details come from the local technical guide listed in the source metadata. Workload and cost comparisons come from the planning report Edge RAG for HVAC Field Technicians. Predictive caching is a proposed feature documented in the local source material; it is not live.