A fluent answer with half the facts is dangerous when nobody can tell it is partial. Silence at least looks like a gap. A polished paragraph does not.
That is the judgment in AWS’s sample, “Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK.” Classic retrieval-augmented generation (embed the question, fetch the nearest passages from one index, write once) still fits a large class of questions. It fails quietly when the answer has to cross sources and the reader has no way to see what was left out.
A question that has to cross the hall
Picture a question a finance lead actually asks: what did last quarter’s filing language promise, and does the operations report show we did it? The words live in two libraries. A system that always opens the same cabinet, then writes the memo anyway, will sound finished. AI automation of that habit does not make it safer. It makes the miss faster, and harder to catch in the meeting.
The sample names three ordinary breaks. The answer spans more than one body of documents. The correct source depends on what was asked. Or the first look is close but not enough, and someone has to look again before writing. Its own line is worth keeping in view:
Ask a question whose answer spans two corpora, and a single retrieval returns half of what the model needs, which is worse than returning nothing, because the answer comes back fluent and incomplete.
That “worse” holds when the incompleteness is invisible. If the system shows its sources and its gaps, a partial answer can still be useful. If it does not, fluency is a liability. The line is a design judgment from the sample, not a measured result against the older system.
For AI for business, sort the problem as one source of truth or a question that has to cross sources. Calling that “RAG versus agents” names the wrong split. A support policy in a single store can stay on one search and one draft. The sample treats that path as cheaper per query and simpler. The same split shows up in AI for sales when a quote needs both the rate card and the implementation notes, and neither file is allowed to stand in for the other.
What the agent is actually doing
An AI agent, in this design, is a short reason-and-act loop with named tools. It decides whether to retrieve, which knowledge base matches the question, and whether a second look is warranted, then composes a cited answer. It is not a chatbot with a longer prompt.
Inside a knowledge base, the sample describes a Bedrock API, AgenticRetrieveStream, that plans across turns, breaks the question down, retrieves more than once, and streams a synthesized answer with citations. Amazon Bedrock AgentCore hosts the agent and exposes each knowledge base as a tool through a Gateway. The Gateway uses MCP, the Model Context Protocol, so each library has a name the trace can show. Calls run under the agent’s own IAM role (Identity and Access Management) rather than a separate login pool.
The sample states that this retrieval API, and the Gateway connector built around it, are available only on a Managed Knowledge Base, not on one where you operate the vector store yourself. That exclusivity is a claim of the walkthrough. Nothing published with the sample is a benchmark against single-shot RAG on latency, accuracy, or dollars.
What the sample actually stands up is easy to say in one breath. One infrastructure-as-code app creates two managed knowledge bases, a tool per base, a hosted agent, and dashboards. The demo libraries are deliberately unrelated: a synthetic 10-K for a fictional company, Octank, marked as sample data rather than real financials, and a public-domain Congressional Research Service report on tornadoes. Unrelated topics make a wrong tool obvious. That is a teaching trick, not a dataset you should trust.
Managed does not mean unmanned
Choosing the managed type means Amazon Bedrock operates the vector store. You still supply a name and an IAM role. Service-managed embeddings are a documented option. You can also bring your own embedding model. The sample further says built-in reranking comes with the managed base, and that those built-in models carry no extra charge. Treat that cost line as the sample’s architectural claim. It is not a priced study.
Set this beside the August 2024 CDK post, “Build an end-to-end RAG solution using Knowledge Bases for Amazon Bedrock and the AWS CDK.” That design had you deploy an IAM role, an Amazon OpenSearch Serverless collection and index, and a knowledge base over documents in S3. Three stacks, by hand, in order. You operated the index, and a capacity line item belonged to that shape. This design removes that collection. The sample’s comparison says the idle floor is storage, not OpenSearch Compute Units. You still own identity, the document bucket, the runtime, and the cost of judging answers.
Customer-managed stores still earn their place when you already run OpenSearch Serverless, or when data residency, tuning, or a compliance control requires the index to stay in your hands. Coverage for the managed base and AgentCore is not every AWS Region. The sample points readers to current regional documentation rather than a fixed list.
The cabinet it should not open
Separate libraries are a permissions decision, not only a tracing trick. Two named tools make routing visible and keep quality from being averaged across unrelated domains. A blended index hides a wrong-cabinet choice inside a rank score.
Named tools do not decide who may grant the agent which cabinet. If the agent’s role can open contracts, payroll, and a sales knowledge base, a question from a sales rep can come back with a fluent citation from the wrong store. The trace will show that it happened. It will not stop it, and it will not satisfy a residency rule. A pricing sheet and an implementation note are a clean demo. A customer contract and an HR policy are not.
Grant the smallest set of knowledge bases the job requires, on the role that runs the agent. Treat a cross-domain retrieve as a data-access event, not as a clever ranking choice.
If you cannot see the path, you do not have a system
Each question can take a different path. Same prompt, different tool, different number of looks, different bill. The sample’s point is that by default you can see neither the path nor whether the answer was any good. On an agentic system, that visibility is the product, not garnish.
Two empty states get confused, and they should not. The sample’s dashboards stay blank until something asks a question. Evaluation is a second wait. Judges score only after a session closes, and the results log can stay quiet for about the session idle timeout, roughly 30 minutes in the setup the sample describes. No traffic, no graphs. Traffic, and the scores still wait until the room goes quiet.
The trap executives will misuse is token math. Nested traces paint one model call onto several rows, so an unfiltered total triple-counts the bill. Filter to the client leaf (the span that actually represents the call) before anyone reports spend. In one illustrated run in that sample, not a benchmark, the top three rows repeat a single call of 7, 299 input tokens and 268 output tokens. The filtered view of that same run is 14 model calls, 60, 472 input tokens, and 2, 709 output tokens. Skip the filter and both the call count and the tokens triple. Unfiltered spans invent spend.
A cousin of that trap is a blank chart that means nothing. AWS’s native knowledge-base metrics expect an ID prefixed with knowledge-base/. The sample’s own namespaces use the bare ID. Copy the wrong dimension and the graph stays empty while the system is fine.
The sample’s dashboard offers a rule of thumb for its own shape, not a law of AI for business: for N queries, expect about N agent invocations, about 2N retrievals, about 3N model calls, and about 5N gateway operations. Path length is not fixed. A cross-source question costs whatever the agent decides to do.
A perfect score on a tidy demo
The sample’s built-in judges score Correctness, Faithfulness, Tool-Selection Accuracy, and Response Relevance, with a model acting as the judge. On the illustrated continuous-evaluation run, all four scored 1.00, five traces each, with sampling at 100 percent. A perfect score on two small, clean documents does not say how those judges behave on a messy corpus, a hostile question, or a partial citation.
That 100 percent sampling is labeled an experiment in the sample, not a production setting. Evaluation cost scales with sampling and with traffic. Turn it down before the pattern meets a real user base.
The sample also plots reference-free signals per knowledge base (whether retrieved material was used, how much of the answer was grounded, how much was duplicate) without a ground-truth set and without an extra judge. It does not publish formulas you could reimplement from the description alone. Use them as a smell test until you have defined them on your own corpora.
Citations, faithfulness, and tool-selection accuracy are what make the pattern auditable. A fluent paragraph is not evidence. A green dashboard is not a correct answer.
Questions worth asking before you copy the pattern
- When is single-shot RAG still the better design? When one corpus holds the answer, when you cannot observe the path, and when latency and cost have to be predictable. The sample treats the single-shot path as cheaper per query and simpler. An agent with a variable number of looks fails the predictability test even when it is more complete.
- What does a managed knowledge base actually remove? It removes the customer-operated vector store, including the OpenSearch Serverless collection in the 2024 pattern, and it can use a service-managed embedding model. It does not remove IAM, the document store, a runtime, or the cost of judging answers.
- Why not merge every document into one index? Separate tools make a wrong cabinet visible in the trace and keep quality from being averaged across unrelated domains. A blended index hides the mistake inside a rank score. Separation still does not grant the right permissions.
- What can the agent retrieve that the person asking should not see? Whatever knowledge bases its role can open. A named tool shows which store was used. It does not check that the person in the chat is allowed to see that store, and a citation from the wrong cabinet can still read as authoritative. Treat each grant as an access decision, especially where contracts, HR, or residency lines are involved.
- How do you know the agent worked, rather than merely replied? You can name the tool it picked, token totals were filtered to the client leaf, and faithfulness and tool-selection scores agree with the citations. If you cannot say which cabinet opened, you do not have agentic retrieval.
- What does a cross-source question cost relative to one index? More, and not by a fixed multiple. The sample’s own expectation is several retrievals and model calls per question, and the path changes with the question. One-index RAG has a cost you can budget. This design has a cost you have to measure, including evaluation if you sample live sessions.
- Does a 1.00 on the demo mean the system is ready? No. That score is five traces per evaluator on two small, clean documents in one illustrated run. It does not measure messy enterprise corpora or adversarial questions.
Name the tool, or don’t call it agentic
Swap demo files for your own corpora only after you have decided which questions are allowed to cross a boundary, and which ones should still hit a single index and stop. If you cannot explain which tool the agent picked, you do not yet have agentic retrieval. You have a longer prompt and a smoother voice.