How an LLM works: tokens and context
An LLM (Large Language Model) splits text into tokens and, at each step, computes the probability of the next token. Saying it "does not think" is an oversimplification, but technically that is what it does: pick the most fitting continuation given the context.
This has several practical consequences:
- Everything the model knows comes either from training data or from the text you put in the context. There is no third source.
- Your project's code is not in the training data — the agent learns it only by reading files.
- Whatever you put in the context determines the quality of the answer. A "weak answer" usually means "weak context".
| Term | What it means | Why it matters |
|---|---|---|
| Token | The unit of text — roughly a word fragment | Pricing and limits are counted in tokens |
| Context window | The maximum number of tokens the model can see in one request | Anything that does not fit is truncated or compacted |
| System prompt | The opening instruction that sets the model's role and rules | This is where the agent's behavioural frame is set |
| Context filling up | As the conversation grows, older parts get summarised | Details can be lost — keep important decisions in CLAUDE.md |
LLMs are non-deterministic: ask the same question twice and you may get two different (usually equivalent) answers. That is why the agent's output must be checked by tests, compilation and review — "it worked once" does not mean "it always will".
📚 Sources and documentation
- How Claude Code worksofficialcode.claude.com
The agent loop and what happens when the context fills up.
- Context windowofficialcode.claude.com