The LLM Loop: A Mental Model for Better AI Work

Most people who are new to LLMs treat one part of the work as the whole thing. They polish the prompt and wonder why the answer is still thin. Or they paste in more and more context without ever changing what they asked for. Or they switch to a bigger model before they have decided what a good answer would even look like. Each of those moves can help. None of them is the whole picture.

The whole picture is a loop, and it fits in one diagram. The answer you get from an LLM is the product of three inputs working together, plus the iteration that ties them back to the goal. I have found this to be the single most effective way to explain how to actually work with these tools, so I want to share it more widely.

I am currently training about a hundred data engineers at a client. The workshop is on getting up to speed on an unfamiliar codebase faster using LLM-assisted documentation. This loop turned out to be the part people needed first: the mental model that makes everything else click into place. This is not a new architecture. It is the picture I reach for before the rest of the material has anywhere to land.

Newcomers usually arrive already holding some of the pieces. They know prompts matter, though they often can’t say what separates a good prompt from a vague one. A workable prompt usually has four parts: the task (what you want done), the context (what the model needs to know to do it), an example or format (what a good answer looks like), and the constraints (what to avoid or stay within). Some know context matters. Model choice is usually a mystery. And almost no one expects iteration to be a normal, central part of the process rather than a sign they got the first prompt wrong.

That gap is exactly what the loop is for.

The answer you get from an LLM is not just the result of a prompt. It is the result of three inputs working together:

• the instructions, which describe the outcome you want
• the context, which gives the model what it needs to know
• the model, which determines the reasoning, speed, and effort available for the task

These three are genuinely distinct levers. Instructions can be perfect while the model is starved of the context it needs. The context can be rich while the ask is vague. And the same instructions and context will land differently on a fast, cheap model than on a slower reasoning one. You can’t collapse them into “the prompt,” because when an answer disappoints, the fix depends on which of them is the weak link.

Then comes the part that makes it a loop instead of a checklist: iteration. You refine the instructions, add or remove context, push back on weak answers, and sometimes reach for a different model. The skill is not knowing one lever. The skill is reading the answer you got and knowing which lever to pull next.

That is why this model has become one of the most important parts of the workshop. Once students see the loop, the rest of the material has somewhere to go. Model selection is no longer an isolated topic: it is one of the inputs. Context engineering is no longer a bag of tricks: it is another input. Evaluating an answer is no longer a vibe check: it is asking whether the current outcome is moving toward the goal.

Why Documentation Is The Right Example

Documentation is a prime use case for LLMs because it asks them to do something they are naturally useful for: distill a large body of information into something a person can understand and interrogate.

This matters a lot for consultants and contractors, who are often dropped into unfamiliar projects and expected to become useful quickly. But it also applies to anyone working in a codebase they do not know well. You are not just asking the model to summarize files. You are using it as a way to build context faster.

That makes documentation a good teaching environment for the loop. The goal is clear: get oriented in the codebase. The inputs are visible: what you ask, what context you provide, and which model or tool you use. The feedback is immediate: either the answer helps you understand the system, or it does not.

The Loop In Action

Here is the loop running on a real workshop task: a data engineer, new to a project, needs to understand how a nightly pipeline actually works.

First pass. They open a fresh chat and type:

Explain this pipeline.

The model has no files, so it answers with a generic description of what a “nightly pipeline” usually does: schedulers, extract steps, loads. Technically correct, useless for this codebase. The instinct at this point is to blame the model and ask the question again, louder. That doesn’t help, because the question wasn’t the weak link.

Adjust the context. They paste in the actual DAG definition, the config file, and the two transform modules it imports:

Here are the files that define our daily_sales pipeline. Walk me through what runs, in order, and where the data comes from and goes.

Now the answer is specific and grounded. It names the real tasks and tables. Same model, far better outcome, because the missing lever was context, not instructions.

Adjust the instructions. The walkthrough is accurate but it is a flat list of steps. What the engineer actually needs is the why, and a sense of what’s fragile:

Good. Now rewrite this as onboarding docs for a new engineer. Call out the non-obvious parts and anything that looks fragile or likely to break.

The output sharpens into something a teammate could actually read, and it flags a retry step with no error handling.

Push back, and reach for a different model if needed. The engineer suspects that retry claim is wrong:

You said the retry step swallows errors silently. Show me the exact lines, and if you can’t point to them, say so.

A fast model may waver here; this is the kind of careful, cite-your-evidence reasoning worth spending a stronger model on. The answer either points to the real lines, a genuine finding for the docs, or admits it overreached, which is itself useful.

Four turns, and every lever moved at least once: context, then instructions, then a challenge, with model choice held in reserve for the step that needed it. No single one of those moves would have gotten there alone.

What The Loop Gives You

The value of the model is diagnostic.

When an answer is weak, the question is no longer “how do I write a better prompt?” The question becomes more useful: which part of the loop needs to change?

Do I need to clarify the outcome?

Do I need to add or remove context?

Do I need a different model?

Do I need to push back on the answer I got?

That is the shift I saw in the workshop. Once students had the loop, the rest of the material had a place to attach. They were no longer collecting tips. They were learning how to steer.