Three completely different numbers people conflate when talking about LLM "size" — and mixing them up leads to genuinely wrong predictions about what a model can do.
"How big is this model?" sounds like one question. It isn't. A model's parameter count, its context window, and the token count of whatever you happen to be sending it right now are three unrelated numbers that live on three different layers of the system — one fixed at training time, one fixed by the model's architecture, and one that changes with literally every request. Treating them as interchangeable is how people end up assuming a "bigger" model can automatically read a longer document, or that a huge input means the model just got smarter. Neither follows.
Tokensare the actual units of text the model reads and writes — roughly sub-word chunks, not whole words. A common short word like "the" is usually one token; a rarer or longer word like "engineering" often gets split into several sub-word tokens. Every input you send and every output the model generates is measured in tokens, and API pricing is almost always billed per token — so how much text you're sending or receiving right now is a tokens question, full stop. It has nothing directly to do with how capable the model is or how much text it could theoretically handle.
Parametersare the actual learned numerical weights inside the model — the individual numbers, tuned during training, that make up its neural network. A model with "70 billion parameters" genuinely has 70 billion individual learned values. Parameter count is a rough proxy for the model's overall capacity, complexity, and knowledge — but it's fixed the moment training finishes. It does not change based on how long or short your prompt is, and it does not tell you anything about how much text you can send the model in one go.
Context windowis the maximum number of tokens — input and output combined, for a given request or conversation — that the model can consider at once. It's a fixed architectural limit of that specific model: "128K context window" means up to roughly 128,000 tokens total across the whole exchange. Go over it, and earlier content typically has to be dropped or truncated, because the model simply cannot see beyond that window — no matter how many parameters it has.
A model's designers decide its parameter count and its context window separately, based on separate tradeoffs — parameter count trades training and inference compute for raw capability and knowledge; context window trades memory and compute for how much text the model can attend to at once. Neither one is derived from the other, which is exactly why a huge, highly capable model can ship with a comparatively modest context window, and a much smaller model can ship with an enormous one. Tokens sit on a completely different axis: they're not a fixed property of the model at all, just a count of however much text a specific request happens to contain. "How many tokens am I using right now" changes every time you type; "how many parameters does this model have" and "what's its context window" never change for a given model, no matter what you send it.
False, and it's an easy assumption to make because both numbers get marketed together as "how big" a model is. Parameter count is the model's learned capacity — the number of weights it holds after training, fixed and unchanging. Context window is the maximum text length, in tokens, it can process in a single request — a separate architectural limit set by its designers. A model can be scaled up in parameters without touching its context window at all, or the reverse, because the two are decided independently based on entirely different engineering tradeoffs. Knowing a model has "X billion parameters" tells you nothing about how long a document or conversation it can handle in one request. That specific number is the context window, published separately, not derived from the parameter count.
Explains the three genuinely independent numbers people conflate when discussing LLM size: tokens (the actual units of input/output text, billed per-request), parameters (the model's fixed, learned numerical weights, set at training time), and context window (a fixed architectural limit on how many tokens the model can process in a single request). Includes a three-box diagram separating the three properties and a side-by-side comparison of two models where parameter count and context window vary in opposite directions.
All three numbers get thrown around in the same sentence when people describe a model — "it's a 70 billion parameter model with a 128K context window, and my prompt was about 2,000 tokens" — which makes them sound like they sit on one continuous scale of "bigger." They don't. Parameters describe what the model learned during training and never change afterward. Context window describes a hard architectural ceiling on how much text a single request can contain, also fixed for a given model. Tokens describe how much text is actually present in one specific request or response, and that number is different every single time. Two of the three are fixed properties of the model; one is a live measurement of your current input and output.
A token is the model's actual unit of text — produced by a tokenizer that breaks text into sub-word pieces, not whole words. Short, common words are often a single token; longer or rarer words, and most non-English text, frequently split into multiple tokens. Every input token and every output token counts, and per-token pricing is the standard billing model for LLM APIs — so the cost and time of a given call is fundamentally a tokens question.
A parameter is one learned numerical weight inside the model's neural network. A "70B" model has 70 billion such weights, adjusted during training to encode everything the model picked up from its training data. Parameter count is a rough, imperfect proxy for overall model capability and knowledge, but it is entirely fixed post-training — it has no relationship to the length of any particular prompt.
The context window is the maximum number of tokens — input plus output combined — a specific model can process for one request or conversation, a limit baked into the model's architecture (e.g. how it handles positional information) and its training. Exceed it, and the system generally has to drop or truncate earlier content, because content beyond the window is architecturally invisible to the model regardless of how many parameters it has.
Choosing a model for a task that involves long documents or long conversation history is a context-window decision, not a parameter-count decision — a highly capable model with a small context window may be the wrong pick even if it would otherwise be the "smartest" available, because it simply cannot ingest the material in one pass. Conversely, estimating API cost or whether a request will fit at all is a tokens question, independent of both the model's parameter count and (as long as you're under the limit) its context window. Mixing these up leads to real mistakes: assuming a "bigger" model can automatically handle a longer document, underestimating API costs by counting words instead of tokens, or being surprised when a long-running conversation suddenly starts "forgetting" earlier messages once the context window fills up.
No. Tokens are typically sub-word chunks produced by a tokenizer, not whole words. Common short words are often a single token, while longer, rarer, or non-English words frequently split into multiple tokens. A rough rule of thumb in English is around 3/4 of a word per token, but it varies by tokenizer and by the specific text.
No. Parameter count and context window are set independently by the model's architecture and training choices. A model can be scaled up in parameters without any change to its context window, and a smaller model can ship with a very large context window. Neither number determines the other.
The model cannot process tokens beyond its context window, so earlier content in the conversation typically has to be dropped or truncated to fit within the limit before the model can continue — regardless of how many parameters the model has. Some applications manage this with summarization or sliding-window strategies, but the underlying constraint is the same: content outside the window is simply not seen.
No. Parameter count is fixed once training finishes and has nothing to do with any individual request. Sending a short prompt or a very long one uses the same fixed set of weights either way — only the token count of that specific request changes.
Tokens are the actual unit the model computes on, so token count tracks compute cost far more directly than word or character count does. Words and characters don't map consistently to tokens across languages or vocabulary, which is why providers meter and price by token rather than by a more human-intuitive unit.
Yes. Context window size depends on architectural and training decisions — such as how positional information is encoded and what sequence lengths the model was trained and evaluated on — that are independent of parameter count. Two similarly-sized models can ship with very different context window limits.
Try our AI Engineering Studio
More calculators, simulators, and guides for this discipline.