Most people who are new to LLMs treat one part of the work as the whole thing. They polish the prompt and wonder why the answer is still thin. Or they paste in more and more context without ever changing what they asked for. Or they switch to a bigger model before they have decided what a good answer would even look like. Each of those moves can help. None of them is the whole picture.
I use one diagram to put the pieces together. An LLM answer depends on three inputs, followed by the iteration that brings the result closer to the goal. This has been the clearest way for me to explain how the work fits together.
I am currently training about a hundred data engineers at a client. The workshop uses LLM-assisted documentation to help engineers get oriented in an unfamiliar codebase. I expected the practical exercises to carry the course, but people first needed a way to diagnose why an answer was weak. This loop gave us that vocabulary.
Newcomers usually arrive already holding some of the pieces. They know prompts matter, though they often can’t say what separates a good prompt from a vague one. A workable prompt usually has four parts: the task (what you want done), the context (what the model needs to know to do it), an example or format (what a good answer looks like), and the constraints (what to avoid or stay within). Some know context matters. Model choice is usually a mystery. And almost no one expects iteration to be a normal, central part of the process rather than a sign they got the first prompt wrong.
The loop connects those pieces.
The loop has three inputs:
- the instructions, which describe the outcome you want
- the context, which gives the model what it needs to know
- the model, which determines the reasoning, speed, and effort available for the task
If the four-part prompt from earlier is in your head, notice the mapping: task, example, and constraints are the instructions lever, and context is promoted to its own. These three are genuinely distinct levers. Instructions can be perfect while the model is starved of the context it needs. The context can be rich while the ask is vague. And the same instructions and context will land differently on a fast, cheap model than on a slower reasoning model. You can’t collapse them into “the prompt,” because when an answer disappoints, the fix depends on which of them is the weak link.
Then comes the part that makes it a loop instead of a checklist: iteration. You refine the instructions, add or remove context, push back on weak answers, and sometimes reach for a different model. The skill is not knowing one lever. The skill is reading the answer you got and knowing which lever to pull next.
The model now sits near the start of the workshop. Model selection becomes one input rather than an isolated topic. Context engineering becomes a deliberate choice about what the model needs. Evaluating an answer means checking whether the result is moving toward the goal and deciding what to change next.
Why documentation works as the example
Documentation is a prime use case for LLMs because it asks them to do something they are naturally useful for: distill a large body of information into something a person can understand and interrogate.
This matters a lot for consultants and contractors, who are often dropped into unfamiliar projects and expected to become useful quickly. But it also applies to anyone working in a codebase they do not know well. You are not just asking the model to summarize files. You are using it as a way to build context faster.
That makes documentation a good teaching environment for the loop. The goal is clear: get oriented in the codebase. The inputs are visible: what you ask, what context you provide, and which model or tool you use. The feedback is immediate: either the answer helps you understand the system, or it does not.
The loop in action
Here is the loop running on a real workshop task: a data engineer, new to a project, needs to understand how a nightly pipeline actually works.
First pass. They open a fresh chat and type:
Explain this pipeline.
The model has no files, so it answers with a generic description of what a “nightly pipeline” usually does: schedulers, extract steps, loads. Technically correct, useless for this codebase. The instinct at this point is to blame the model and ask the question again, louder. That doesn’t help, because the question wasn’t the weak link.
Adjust the context. They paste in the actual DAG definition, the config file, and the two transform modules it imports:
Here are the files that define our
daily_salespipeline. Walk me through what runs, in order, and where the data comes from and goes.
Now the answer is specific and grounded. It names the real tasks and tables. Same model, far better outcome, because the missing lever was context, not instructions.
Adjust the instructions. The walkthrough is accurate but it is a flat list of steps. What the engineer actually needs is the why, and a sense of what’s fragile:
Good. Now rewrite this as onboarding docs for a new engineer. Call out the non-obvious parts and anything that looks fragile or likely to break.
The output sharpens into something a teammate could actually read, and it flags a retry step that swallows errors silently.
Push back, and reach for a different model if needed. The engineer suspects that retry claim is wrong:
You said the retry step swallows errors silently. Show me the exact lines, and if you can’t point to them, say so.
A fast model may waver here; this is the kind of careful, cite-your-evidence reasoning worth spending a stronger model on. The answer either points to the real lines, a genuine finding for the docs, or admits it overreached, which is itself useful.
Four turns, and every lever moved at least once: context, then instructions, then a challenge, with model choice held in reserve for the step that needed it. No single one of those moves would have gotten there alone.
What the loop gives you
The value of the loop is diagnostic.
When an answer is weak, the question is no longer “how do I write a better prompt?” The question becomes more useful: which part of the loop needs to change?
Do I need to clarify the outcome?
Do I need to add or remove context?
Do I need a different model?
Do I need to push back on the answer I got?
In the workshop, students stopped treating each weak answer as a prompting failure. They started asking which input needed to change and what evidence in the answer pointed them there.