By the Time the Model Sees Your Prompt, More Than a Dozen Systems Have Already Touched It

Part 1 of The AI Field Guide. Five cards on the assembly line between hitting send and seeing the answer.

TL;DR. A four-week series. Last week was the thesis and three preview cards. This week: five cards on the runtime of a single request. June 23: Where the Answers Come From. June 30: Why You Can Trust Some Systems and Not Others. Free PDF after Part 3.

---

Most people picture an AI request the way they picture a Google search: query in, answer out. Click the box, get the result.

The actual shape is an assembly line. By the time the model sees your prompt, more than a dozen systems have already touched it. By the time you see the answer, several more have shaped it.

Last week I said the bird is gone in three seconds and the fix is to know the marks ahead of time. This week we open the box on those three seconds and label the assembly stops.

Five cards. In order.

Card 1: An LLM Request, End to End

The first card is the map: eight steps from user prompt to operational logging, with a parallel column showing what changes when the request’s “use my data to train future models” toggle is off versus on. This matters more than people think. The toggle does not change what the model can do for you in this request. It changes whether anything you sent might be used to improve a future version of the model.

Two things to notice on this card before anything else.

The first is that training and inference are different operations on different timelines. Training is the offline process that turned a pile of data into the frozen weights you are talking to now. Inference is what runs every time you hit send. Conflating the two is the most common source of confused AI policy. Lawyers reviewing AI vendor contracts in 2026 still routinely treat live requests as training events. They are not.

The second is that everything between step 3 (safety checks) and step 6 (post-processing) is where the assembly line does its real work. The model itself, in step 5, is brief. It takes the packed prompt, generates tokens, and stops. The expensive, error-prone work is the wrapping.

This is the card I would print and hand to anyone in your organization who is asking what does the AI do with my data? The honest answer is on the right-hand column, in plain English, with the toggle states named.

Card 2: Caching, Cost, and Latency

The same prompt can cost a tenth of a cent or ten cents to answer. It can return in 300 milliseconds or 20 seconds. The difference is almost never the model. It is the cache hierarchy, the context length, the tool calls, and the output size.

Caching is the part of enterprise AI that nobody puts on a slide deck and that nobody can afford to ignore. Anthropic launched prompt caching as a first-class feature in mid-2024 and discounted cached input tokens to roughly 10 percent of fresh ones. OpenAI followed with automatic caching in late 2024 at 50 percent off. The economic effect is straightforward: a system that asks the same questions with the same system instructions a thousand times a day, and caches well, runs at a fraction of the cost of a system that does not.

The operational effect is less obvious. Caches change what users experience. A cached response feels instant. A cache miss with a long retrieval feels slow. Users adapt to the median latency of your system, and a system that mixes 300ms and 20s responses feels less reliable than one that consistently delivers three seconds. This card lays out the seven knobs that determine which side of that experience your users get.

The trade-off in the bottom right is the one to study. More caching helps latency and cost, but it can hurt freshness and quality if you are not careful about what gets cached and for how long. Operational AI is mostly the discipline of running this trade-off well.

Card 3: How Tools Fit Into an LLM Request

Until late 2023, “AI” meant a model returns text. Now AI means a model returns text, or a function call, or both, and the function might call your CRM or your calendar or a Python interpreter. The model itself has not changed in this respect. The system around the model has. This is what “tool use” is.

The card walks the eight-step request flow again, but highlights two specific moments. Step 4 is where the system decides which tools to expose to this request. Step 6 is where the tool results get integrated back into the response. Most enterprise AI engineering work is in those two boxes.

This is where skills are the new software. A skill is the packaging unit that says for this kind of request, expose these specific tools, with these specific permissions, under this specific persona. Anthropic formalized the Agent Skills spec in October 2025. Google mirrored the concept inside Workspace at Cloud NEXT in April 2026. The skill is what an analyst clicks. The tool is what the skill calls. The model is what makes the tool call decision.

The five tool families on the card (skills, retrieval, plugins, MCP connectors, agents) are not interchangeable. They sit at different distances from the model and carry different policy properties. Most enterprise AI deployments today have at most two of the five.

Card 4: Context Assembly, End to End

This is the card I find myself reaching for in vendor conversations. The funnel is the part most vendors will not draw.

The card is a funnel. At the top, the universe of things the system could theoretically see: files, conversation history, documents, memory, tools, profile data, policy overlays. The next band is the permitted universe, narrowed by identity, role, region, and policy. Below that is the retrieved universe, narrowed by what RAG and tool calls actually fetched in the time available. Below that is the selected context, ranked and filtered. Below that is the packed prompt, fitting inside the model’s context window. And at the bottom, finally, is what the model actually sees and responds to.

Most “the AI got it wrong” complaints in 2026 are context assembly bugs. The right source existed, but it was not retrieved. The right document was retrieved, but it was not selected. The right context was selected, but it was buried under fresher but less-relevant material. The right facts were present, but stale ones were too, and the model picked the wrong one.

Five common failure modes are on the card, with corrective questions next to each. Why did the model give a bad answer? is the wrong question. Where in the funnel did the right context fall out? is the right one.

This card is where knowledge work is code cashes in. The funnel is the runtime. The skills, retrieval rules, ranking logic, and policy overlays are the source code. Knowledge work is code because the funnel is the program that decides what your knowledge worker (or your AI) gets to see.

Card 5: The Context Window Is Not Memory

I previewed this card in last week’s manifesto. Here is the full version.

Four kinds of memory-like state exist in any AI system:

Model weights: frozen from training, the same for every user, updated only when a new model version ships.

Context window: temporary working set for this request, thrown away when the chat closes.

External memory or retrieval stores: saved notes, user profiles, retrieved documents, enterprise knowledge bases. Persistent. Outside the model.

Logs and application state: operational records, audit trails, thread metadata. Persistent. Often not user-visible.

People conflate these constantly. The model remembered almost always means the application retrieved or the platform persisted. For trust, privacy, and policy decisions, which one of these four held it is not an optional question.

This card has a six-row matrix at the bottom of common confusions, with the actually-correct answer next to each. Print it. Pin it in your policy team’s room.

What this card cluster argues

When something goes wrong in an AI system, it is almost never the model. The model is doing roughly the same thing it did six months ago, give or take a release. What changes around it is what was retrieved, how it was packed, which tools were exposed, what got cached, and what survived the context-assembly funnel.

The model is a frozen mathematical object. The assembly line around it is where your organization’s choices live. It is where your data lives, where your policies sit, where your costs accumulate, and where your skills are registered.

If your AI strategy is which model should we buy?, you are answering the wrong question. The model choice is the smallest decision on this list. The assembly line is the strategy.

What’s next

Next Tuesday: Where the Answers Come From — Part 2 of The AI Field Guide. Five more cards on the training run, what knowledge cutoff actually means, how retrieval gets the model past the cutoff, and where the answer is physically running. June 30: Why You Can Trust Some Systems and Not Others, the closer of the series, with the synthesis card.

PDF after Part 3.

Receipts

• Anthropic — Prompt caching documentation. Launched mid-2024 with cached input pricing at roughly 10% of fresh tokens.

• OpenAI — Prompt caching documentation. Automatic prompt caching launched late 2024 with cached input at 50% off.

• Anthropic — Agent Skills overview and spec, October 2025.

• Model Context Protocol — MCP specification. The open standard for tool and resource integration with LLMs.

• Beyond Reason, prior pieces — The manifesto, Skills Are the New Software, Knowledge Work Is Code.

The bird is still moving. By now you can name what it is doing in the air. See you next Tuesday.