AWS: AI Agents Are Only as Reliable as the Information Behind Them

Recently, AWS has been unusually direct about what will make enterprise AI agents succeed, or fail in quiet and expensive ways. As it pushes Amazon Bedrock toward production-grade agent deployments, the spotlight has moved away from model benchmarks and toward something more familiar to information professionals. It is the information layer that the agent pulls from when it answers questions and takes action.

Amazon Bedrock Knowledge Bases is a good illustration of the shift. It lets organizations connect agents to internal repositories like SharePoint, Google Drive, OneDrive, Amazon S3, and other content sources. Instead of relying only on what a large language model “knows,” an agent can retrieve your organization’s policies, contracts, procedures, and customer content, then use that material to complete tasks.

That retrieval layer is also where things go wrong.

AWS has been clear about the failure modes.

A knowledge base that is not refreshed becomes outdated by default. How documents are divided into retrievable “chunks” can determine whether the agent finds the right passage or misses it. Repositories packed with redundant or irrelevant content can crowd out what matters when the agent is working with limited context.

This is not just an AWS concern. Gartner has reported that 63% of organizations either do not have, or are not sure they have, the right data management practices for AI. Deloitte has found a similar dynamic from another angle, reporting that data-related issues caused 55% of surveyed organizations to avoid certain generative AI use cases.

AWS’s recommendations will sound like engineering to some readers, but they map closely to Information Governance. Automatically synchronize sources. Monitor content for staleness. Retrieve current information from authoritative systems when needed. Filter redundant content. Maintain traceability from source to response. Apply access controls during retrieval. Anyone who has worked in IG will recognize the underlying problems. Organizations have been wrestling with authoritative sources, version confusion, permissions drift, inconsistent classification, and defensible disposition for years.

The difference now is the impact.

When AI only drafts or summarizes, a quality problem often stays on the page as a wrong answer. Once an agent can locate information and initiate a process, that same quality problem can show up as a wrong action. That is where reliability becomes a business risk instead of an annoyance. It also shows up in project outcomes. In a 2026 Gartner survey, 38% of infrastructure and operations leaders who had experienced AI setbacks identified poor data quality or limited data availability as a direct cause of AI project failure.

Contract management is an easy example because the stakes are obvious and the documents are messy in predictable ways. Picture an agent that finds an agreement, identifies the renewal date, checks notice requirements, and alerts the business owner. In theory, you reduce missed renewals and last-minute scrambles, and you bring some discipline to contract operations.

Now bring in reality.

The repository contains the executed agreement, several drafts, an email attachment someone saved as “final,” and an amendment that changed termination provisions.

The agent retrieves an earlier draft instead of the signed version. Or it finds the signed agreement but misses the amendment. The technology did not necessarily fail. The agent simply relied on the wrong information. That is why authoritative sources, version control, and defensible elimination of redundant, obsolete, and trivial (ROT) information become critical to AI deployments.

Giving an agent access to more information does not automatically make it better at its job.

If several versions of the same document are available and their status is unclear, more content simply creates more opportunities to get the answer wrong. In an agent context, getting the answer wrong can mean triggering the wrong workflow, escalating to the wrong owner, or generating a notice that should not be sent.

Even when an agent finds the “right” document, it still may not have enough context to use it correctly.

Metadata and classification tell the system what a document represents. Who owns it. Its effective date and status. Its sensitivity. Which business unit it applies to. How it relates to other records. Those distinctions are not academic. An employment policy may have been superseded. A contract may apply to one subsidiary but not another. A customer file may contain personal information that cannot be used for every purpose. An agent can retrieve something that matches the search perfectly while still being inappropriate for the task it is performing. Good metadata helps the system distinguish between a document that is relevant and a document that should actually be relied upon.

Permissions is where this gets risky fast, because agents can search across systems that most employees never query directly.

An agent helping an employee with a routine task should not suddenly surface confidential HR information, privileged legal material, trade secrets, or restricted customer information simply because those records exist somewhere in a connected repository. Access controls need to apply at the point of retrieval. Organizations should be able to answer three questions without hand-waving. What can the agent reach, why does it need that access, and how do permissions change as the agent’s responsibilities expand? Being technically capable of finding information still does not create a legitimate business need to use it.

Finally, if something goes wrong, you need to be able to reconstruct what happened. AWS’s agentic retrieval capabilities can preserve retrieval results, citations, and trace information showing how an agent searched its knowledge sources.

That becomes essential when AI outputs touch financial transactions, employment decisions, contract administration, regulatory processes, or other consequential activities. When an agent produces the wrong result, the organization should be able to determine what information it relied on, where it came from, whether it was current, and what actions followed.

AI agents could take enterprise AI well beyond drafting, summarization, and other productivity uses.

They may increasingly perform parts of business processes with limited human intervention. That makes the condition of the underlying information more consequential, not less.

Organizations preparing for agentic AI should spend at least as much time examining the information environment as the model itself.

They need reliable authoritative sources, current information, appropriate classification and permissions, and enough traceability to understand what the agent relied upon if something goes wrong.

The agent may be doing the work on its own.

You still need a human in the loop to make sure the information behind it is current, correctly classified, properly secured, and traceable.