Context engineering for the enterprise: what it takes beyond prompt engineering

Context engineering is the discipline of assembling the right, verified information for a model or agent at the right time. Prompt engineering tunes instructions; context engineering governs everything else the model sees. For an enterprise, that means engineering the data itself: schema, versioning, confidence, review, and provenance, before a single token reaches the window.

What is context engineering, and how is it different from prompt engineering?

We build bem, and we came to this term as skeptics. The ecosystem converging on context engineering is right about the problem. Most of it is still solving the easier half.

The term is in live use across the agent ecosystem, and the definitions largely agree. Anthropic's engineering team defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference," and frames the goal as finding the smallest set of high-signal tokens that produce the behavior you want. LangChain's post on the subject quotes Andrej Karpathy calling it the "delicate art and science of filling the context window with just the right information for the next step," and organizes the work into four moves: write context down, select what is relevant, compress what accumulates, and isolate contexts across agents.

Prompt engineering, in this framing, is a subset. Instructions are one input among many. Context engineering also covers tool definitions, message history, memory, and retrieved data, everything the model attends to when it decides what to do next.

This is real progress and we will not pretend otherwise. Attention is finite, agents drown in their own transcripts, and write, select, compress, isolate is the correct treatment for that disease. If you build agents, learn it.

But read those definitions again and notice the shared assumption. Both are about selection: the right information already exists somewhere, and the craft is getting it into the window at the right moment. The discipline, as commonly practiced, optimizes which tokens reach the model. It says much less about whether those tokens were ever verified.

Why do AI agents hallucinate on company data?

Usually because the context they received was never validated, and the model has no way to know that. Unvalidated context in means hallucinated actions out. An agent given a stale claim status does not malfunction when it acts on it. It performs exactly as designed, on an input nobody signed off on.

The enterprise version of this problem is structural. Your business context is not a pile of documents. It is entities and their histories: this shipment, this claim, this policy, each described by a dozen documents that disagree in small ways. The bill of lading says one weight, the carrier invoice another. The policy was amended twice. Which version does the agent see?

Retrieval, the first answer most teams reach for, returns passages that mention the entity. The agent still has to reconcile them, re-extract the fields, and decide what is current, on every request, with no memory that it decided differently yesterday. Retrieval is a fine answer for prose questions over manuals and policies. It was never designed to be a system of record.

The math is unkind. Extraction at 98 percent accuracy per field compounds to roughly 82 percent across a ten-field document. Feed that into an agent that acts, and nearly one action in five rests on at least one wrong fact, and nothing in the window tells the model which one. No amount of compressing, selecting, or isolating fixes an input that was wrong before it arrived.

So here is the enterprise twist the frameworks do not cover: context is not documents. It is determinations. A determination is a specific, validated claim about an entity, the invoice total on record, which process extracted it, from which page, at what confidence, reviewed by whom. Documents are evidence. Determinations are context.

What is the best way to provide business context to an AI agent?

Make the context exist before the agent asks for it. Most teams bolt an agent onto raw data, then engineer the window harder when it misbehaves. Invert that. Engineer the data first, and the window problem shrinks to something the standard playbook handles well.

In practice that means providing operational context as validated records rather than raw files. Extract each document once, against a schema that rejects malformed output instead of accepting whatever the model produced. Score every field, so the agent, or the human reviewing its work, knows which values are load-bearing and which are guesses. Route low-confidence fields to a person before they enter the record, not after they have driven a decision. Version the record and the process that produced it, so the same input yields the same output next month. Keep provenance, so any value can be traced back to the page it came from.

Then give the agent the record. This is what the operational side of a context layer for AI agents actually is: not a smarter window, but agent-readable records with the proof attached. An agent consuming determinations does not re-parse twelve PDFs to answer a status question, and it does not get twelve slightly different answers depending on chunk order.

The distinction to hold onto: for knowledge questions, retrieval over prose is a reasonable default. For actions on business entities, booking the shipment, paying the invoice, settling the claim, the agent needs records it can act on without re-litigating them each time.

What does engineering enterprise context actually require?

Less than a platform pitch implies, and more than a parsing library delivers. Six properties keep coming up when we work through this with operations and data teams in finance, healthcare, industrial and manufacturing, insurance, and logistics.

Requirement	What it gives the agent	What its absence looks like
Schema enforcement	Typed fields it can act on without guessing at structure	Free-form text that every consumer re-interprets differently
Versioning	The same input produces the same record, release over release	Silent behavior changes nobody can bisect
Per-field confidence	A machine-readable signal for what to trust	One blended accuracy number that hides which field is wrong
Human review routing	Doubt resolved by a person before the record ships	Errors discovered downstream, as incidents
Event log	The history of every change, with source and timestamp	An agent acting on stale state with no way to detect it
Entity registry	One canonical record per shipment, claim, or policy	Twelve documents about one entity treated as twelve facts

None of these is exotic. Data engineering solved most of them for structured pipelines years ago. What is new is applying them to unstructured inputs, the PDFs, emails, and attachments where operational context actually lives, and doing it rigorously enough that an agent can act on the result. That combination, schema plus confidence plus provenance over unstructured sources, is the part of context engineering the window-management playbook leaves as an exercise for the reader. It is also the part a security or procurement review will ask about, because it is the part that determines whether an agent's mistake surfaces as a review task or as an incident.

How does bem approach context engineering for the enterprise?

bem automates entire business processes end to end: it turns unstructured data into auditable outputs, then makes and records the decisions in between.

We built bem for the data half of the discipline. bem's functions (Classify, Split, Extract, Enrich, Payload Shaping, Send, Parse, Join, Render, Evaluate, and Decide) chain into workflows behind one V3 API, with four SDKs, Terraform, and a CLI. Every function returns strict-typed JSON validated against your schema, with a per-field confidence score, and low-confidence fields route to human review automatically. Workflows are deterministic and versioned, same input, same output until you publish a new version, with regression testing and drift detection on every function version, and every record carries a full audit trail. Agents consume those records through the V3 API, the File System API, or bem's MCP server. bem is not a retrieval system and does not rank passages; it is the layer that makes the records worth retrieving exist in the first place.

On top of that foundation sits bem's entity plane: an entity registry that resolves every document about a shipment or claim to one canonical record, lifecycle state that tells an agent which actions are valid right now, and an event log that records every change with its source and version. The functions produce determinations; the entity plane is where those determinations accumulate into the durable context an agent can trust.

One proof point: Fleetio cut invoice processing time 65 percent, with 100 percent accuracy on invoice totals, by extracting once and letting systems consume the validated record.

We call ourselves AI skeptics building the next generation of AI, and this is what that means in practice. We do not ask anyone to trust a model. We make every determination carry the proof behind it, so trusting the agent becomes a matter of reading the record. Engineer the prompt and you get a better demo. Engineer the context and you get something you can deploy.

Sources

anthropic.com: Effective context engineering for AI agents (Anthropic's definition of context engineering)
langchain.com: Context engineering for agents (LangChain's write/select/compress/isolate framing and Karpathy quote)