LLM Parameters: Model Weights and Generation Settings

A 70B large language model (LLM) has roughly 70 billion learned parameters; setting temperature to 0.2 changes how a supported API samples its answer, not how many weights the model has. LLM parameters therefore describe two different decisions: which model to run, and how to request its output. Choose the model for the job, then configure only the controls its exact API and version support.

Verified against the providers' live documentation on 25 October 2026. The distinction matters more than it used to: some current APIs reject sampling controls, while others accept them and ignore them. A copied settings preset can therefore fail outright or have no effect.

LLM Parameters Have Two Different Jobs

Model parameters belong to the trained model. Generation parameters belong to the request you send it. Changing the latter does not retrain the former.

Think of a building. Its load-bearing structure determines what it can support; adjustable louvers change how you use the space. Turning a louver does not add another floor. Likewise, changing temperature does not turn a smaller model into a larger one.

What Are Parameters in LLM?

In the model-size sense, parameters are learned numerical values, principally weights, that shape the model's calculations. Training adjusts those values using data. In API conversations, people also use “parameters” for settings that influence output generation. IBM's explanation distinguishes learned values from external configuration.

A supplier saying “our model has more parameters” is discussing the model. A developer saying “change the parameters for invoice extraction” probably means the request settings. Ask which meaning they intend before treating either statement as a performance claim.

What Is a Parameter in LLM?

An individual learned parameter is a number used inside the network, such as a weight that changes how strongly one internal signal contributes to another. It is not a neatly labeled fact, a document, or a word stored in its own drawer.

There is also a separate category: training hyperparameters, settings such as learning rate that govern how training proceeds. A business application calling an existing model normally controls neither these training settings nor its learned weights.

Tokens are different again. They are the units into which input and output are broken, often pieces of words. More tokens in a prompt do not mean more model parameters. The guide to tokens in AI explains that unit; the system prompt guide explains how instructions steer behavior without changing trained weights.

Part One: Model Parameters Tell You About Scale

Parameter count helps estimate storage and computation, but it cannot select the best model for your business task. Start with it when planning infrastructure. Finish with task-specific evaluation.

What Are the Billions of Parameters in LLM?

The “B” means billion learned values. A model described as 70B has approximately 70 billion parameters; “T” means trillion. Published names can be rounded, so an exact deployment estimate should use the actual checkpoint and its storage format.

Two models with the same count can have different architectures, training data, training methods and specialization. Even the same model can occupy very different amounts of memory depending on the precision used to store each weight.

For a business buyer, the useful question is whether the model meets the acceptance criteria: does it extract the correct invoice total, preserve the customer identifier, or produce a patch that passes the required checks? The larger number does not answer those questions.

A 70B Model's Weight Memory, Calculated

Raw weight storage is parameter count multiplied by bytes per parameter. Using a hypothetical model with exactly 70 billion weights gives the following arithmetic, not a measured deployment result.

These are decimal gigabytes. The 140 GB figure is about 130.4 GiB, the binary unit some memory tools display. The conversion changes the unit, not the model.

Hugging Face documents how quantization, storing weights at lower precision, reduces their memory requirement while trying to preserve accuracy. Our LLM quantization guide covers the formats and tradeoffs.

Longer prompts and concurrent requests also change the serving workload. Hugging Face's model memory estimator documentation explicitly separates loading the model from running inference, the process of generating answers.

What the Count Says About Cost and Quality

More weights generally mean more resources to store and operate, all else equal. But “all else equal” is a large qualification. Precision, architecture, hardware, batching and the amount of text processed affect the result.

For self-hosting, count helps you shortlist hardware. For a hosted API, it does not calculate your bill: the provider's usage rules and your workload determine that. A model that needs repeated retries or extensive human correction can also be a poor operational choice despite its apparent efficiency.

For quality, treat size as a clue about capacity, not a score. A procurement document should ask for results on your representative inputs, the failure cases and the evaluation conditions. It should not accept an unpublished estimate of a closed model's parameter count as evidence.

Total vs Active Parameters in MoE Models

Total parameters describe the whole model; active parameters describe the reported computation path for a token. Both numbers matter in a mixture-of-experts model, usually shortened to MoE.

An MoE contains expert subnetworks and a router that selects which ones process each token. These “experts” are mathematical components, not named specialists you can assign to accounting or legal work. A smaller active subset can reduce computation relative to activating the whole model, while the other weights still have to exist somewhere. PyTorch's MoE explanation covers routing and the communication involved.

The phrase “104B active” has a concrete maker-published example: Moonshot AI's Kimi K3.1 Pro announcement gives 2.8T total and 104B active parameters. Converting the total to 2,800B gives 104 ÷ 2,800, or about 3.7% active. That percentage is our arithmetic on the announced counts, not a speedup measurement or a quality ranking.

A second example, Qwen's Qwen3-235B-A22B model card, states 235B total and 22B activated parameters. The illustration keeps the inactive rooms visible because the storage obligation does not disappear when a particular token takes another path.

For deployment, ask how the whole model is stored and distributed, how experts are placed across hardware, and what performance looks like at your actual concurrency. An active count is not a specification for graphics processor (GPU) memory. The mixture-of-experts explainer takes that infrastructure question further.

Part Two: Generation Parameters Steer the Response

Generation settings change selection and stopping behavior within the model's capabilities. An API is the interface your application calls to request a response. Its settings cannot supply missing source material, repair an ambiguous instruction or guarantee a correct answer.

Temperature: How Concentrated the Choices Are

Temperature changes the probability distribution used to select the next token. Lower settings favor the most likely choices more strongly; higher settings allow more variation. This is useful for choosing between consistent phrasing and a broader range of drafts.

For an invoice workflow, variation is rarely the objective. For alternative campaign headlines, it may be useful. But low temperature does not make a fabricated invoice number true, and high temperature does not add knowledge.

Even a zero-temperature request is not a universal reproducibility guarantee. Anthropic's glossary explicitly warns that identical inputs can still produce different results at temperature zero. Reproducibility is a property to evaluate across the whole serving system.

Top-p: How Much of the Candidate Pool Remains

Top-p, or nucleus sampling, limits the candidate set by cumulative probability. It is a probability threshold, not the number of candidates and not a percentage of the vocabulary.

For an illustrative next-token distribution of 0.60, 0.25, 0.10 and 0.05, a top-p threshold of 0.9 needs the first three candidates: the first two sum to 0.85, while the first three reach 0.95. Those numbers are a teaching example, not observed model output.

Temperature reshapes the probabilities; top-p cuts the eligible set. For an initial business configuration, change one sampling control at a time. That also follows OpenAI's recommendation in its Chat Completions reference.

LLM Token Limit: A Ceiling, Not a Requested Length

The generation token limit caps the response. It does not ask the model to use exactly that many tokens, and it does not increase the model's context window, its capacity to handle the conversation.

A limit that is too small can interrupt a JSON object, a sentence or a code patch. A large ceiling does not automatically make the answer useful. Give a length instruction in the prompt, then leave enough headroom under the cap for a complete result.

Reasoning requires another distinction. OpenAI's Responses reference says its output ceiling includes both visible output and reasoning tokens. Anthropic's Messages reference likewise counts thinking tokens toward max_tokens. Do not budget a reasoning request as though every allowed token will appear in the final answer.

Stop Sequences: End at a Specific Boundary

A stop sequence ends generation when a designated string occurs. Use one only when the application has a reliable boundary, such as an explicit end marker that cannot legitimately appear in the content.

A newline is a bad general-purpose stop for a multi-line report. A closing brace is a bad general-purpose stop for nested JSON: the first matching brace can end the response too early. For structured extraction, prefer the provider's supported schema mechanism and validate the returned fields.

Leaving custom stops out is usually the sensible starting point. An output-token ceiling and validation still matter.

Penalties: Discourage Repetition With Care

A presence penalty responds to whether a token has appeared. A frequency penalty responds to how often it has appeared. Positive penalties discourage reuse; negative penalties can encourage it, where the endpoint supports those values.

Repeated text is not always a defect. A product code, customer name, JSON key or programming identifier may need to recur exactly. Penalizing it can work against the task.

For extraction and code, begin without repetition penalties. For drafting, first fix an instruction that encourages repetitive structure. Only then consider a small supported penalty, and check whether it improves the draft without changing required terminology. Google's generation reference also warns that negative frequency penalties can drive repeated-token loops.

API Ranges and Defaults: OpenAI, Anthropic and Google

The endpoint, model and API version determine which settings you can send. A shared SDK field name is not evidence that every provider implements the same control.

The table scopes OpenAI to Chat Completions, Anthropic to Messages API version 2026-10-21, and Google to the Gemini Developer API generateContent. It does not transfer settings from Google's Vertex AI or Interactions API.

NS means the current cited field documentation does not state a numeric range or default. It does not mean zero, unlimited or unsupported. “Omit” below is application advice, not an invented provider default.

Sources: OpenAI request schema, Anthropic request schema, Anthropic version migration, Google generation schema, Google model metadata and Google's top-p range.

Google's general troubleshooting table gives a narrower temperature range than its generation-field reference. Use the field reference together with the selected model's maxTemperature; use Model.temperature, Model.topP and Model.outputTokenLimit for model-specific values. Do not promote a sample request into a documented default.

OpenAI: Reasoning Compatibility Comes First

OpenAI's current GPT-6 guide instructs you to remove temperature and top_p when reasoning effort is not none. GPT-6 Astra and GPT-6.1 Sol do not support none; GPT-6 Sol and GPT-6 Luna do. This is a support restriction, not documentation of a numeric fixed temperature.

The endpoint also matters. Responses uses max_output_tokens, whose current schema minimum is 16; its numeric default and a universal maximum are not stated there. Its create schema does not expose the Chat Completions stop, presence_penalty or frequency_penalty fields. In Chat Completions, the reference specifically marks stop unsupported for o3 and o4-mini and deprecates max_tokens in favor of max_completion_tokens.

The practical rule is to build a request for the chosen endpoint, then apply the model's compatibility rules. Do not copy the entire table into one request.

Anthropic: The API Version Can Remove the Knob

Anthropic's October 21 migration guide says sending temperature, top_p or top_k with API version 2026-10-21 produces a 400 error, even at a default value. The omission behavior includes temperature 1.0.

Older API versions do not automatically restore flexibility on every model. Anthropic's thinking documentation says models including Claude Sonnet 5.5 reject non-default sampling values regardless of whether thinking is enabled. It states a top-p default of 0.99 for those models and explicitly says 1 is rejected. An older generic range cannot override that model rule.

For actual generation, supply a positive max_tokens budget within the model's limit. The documented minimum of zero serves cache prewarming without producing an answer; it is not a useful extraction preset.

Google: An Accepted Setting Can Have No Effect

Google's Gemini 3.6 Flash and 3.5 Flash-Lite migration guide says those models ignore deprecated temperature, top-p and top-k settings. It instructs developers to remove them and says future model generations will reject them with a 400 error.

This creates a different failure mode from a rejected request: your application can appear to accept a temperature change while the model behaves exactly as before. Check the selected model's documentation before interpreting a successful response as proof that a control worked.

Sensible Settings for Business Tasks

Begin with the provider-supported behavior and a measurable definition of success. The numbers below are our starting suggestions for adjustable endpoints, not provider defaults, measured optima or portable presets. The compatibility rules above take priority.

Extraction: Protect Exact Values

For invoice fields, support-ticket classification or contract metadata, try temperature 0 or 0.2 only where supported. Leave top-p alone. Omit custom stops and repetition penalties; on a supporting OpenAI Chat Completions model, explicitly setting both penalties to 0 is another way to define a no-penalty baseline.

On current Anthropic requests and the named Gemini Flash models, omit sampling controls. Use explicit field definitions, allowed categories, examples of missing data and the provider's supported structured-output feature. Validate the result independently of its formatting.

For a short extracted record, an initial ceiling of 1,000 output tokens can be a starting experiment on a compatible non-reasoning request. It is not a universal limit: inspect truncation and actual usage, and budget reasoning separately where applicable. Ask for a missing-value marker rather than an invented invoice number.

Acceptance rule: every required field is present, values match the source, and absent information stays absent. Passing JSON validation alone is insufficient.

Drafting: Vary the Wording, Preserve the Facts

For an outreach email or product description, start with the model's supported default behavior. On an adjustable endpoint, compare it with temperature 0.7, keeping top-p unchanged. Specify audience, length, tone, required facts and prohibited claims in the prompt.

Leave penalties out initially. If a supporting OpenAI Chat Completions model repeatedly reuses unwanted phrases, a frequency penalty of 0.2 is a modest experiment within its documented range. Keep it only if review shows an improvement; required product names must remain intact.

For models that remove sampling controls, request contrasting approaches explicitly, such as a concise factual draft and a narrative draft. Judge the differences in the resulting work.

Acceptance rule: useful variation without new factual claims. “More surprising” is not the same as “better business copy.”

Code: Budget for Completion and Check Behavior

For a focused patch on an adjustable endpoint, temperature 0 or 0.2 is a reasonable comparison point. For a reasoning model, follow its supported configuration and omit unsupported sampling fields. Leave repetition penalties and custom stops out unless a tested protocol requires them.

Provide the relevant code, constraints and expected behavior. Give enough output room for a complete patch. If the response hits its ceiling, treat the result as incomplete instead of assuming the visible portion is safe to apply.

Acceptance rule: the patch meets the specification and passes appropriate checks. Repeated identifiers are normal code, and a plausible-looking answer is not evidence that it runs.

What Parameter Counts and Knobs Cannot Promise

The overhyped claim is that a single number can stand in for an evaluation. It cannot, whether that number is a weight count or a temperature.

A larger model is not automatically better at your narrow workflow. A smaller active MoE count does not mean the whole model fits in that much memory. A low temperature does not guarantee truth. A high output ceiling does not guarantee a complete argument, and a successful API call does not prove every supplied setting was applied.

The useful evidence is a representative input, a defined success condition and an inspected result. For a purchase, ask the supplier to demonstrate the workflow under the configuration you will actually use. For a build, record the model, endpoint, API version, prompt and settings so you can understand what changed when behavior changes.

What Buyers, Operators and Builders Should Do Next

Buyers should compare outcomes; operators should define failure; builders should remove unsupported configuration. Those responsibilities meet in the same small evaluation set.

A buyer choosing a hosted service can proceed without a parameter count if the service demonstrates the required results. Wait on a self-hosting hardware purchase until the full checkpoint, precision and workload are sized. An operator with a working workflow does not need to replace it because another model advertises a larger number.

Builders changing models or API versions should act now on compatibility checks. Operators using a managed interface with no exposed generation controls are largely unaffected by the request-field syntax, but still need to validate the output.

1. 1

   Collect the work that must succeed

   Next Monday, gather 20 representative examples from one workflow, including incomplete, ambiguous and unusually long inputs. Write the expected result or review criteria before comparing models.

2. 2

   Make the request configuration explicit

   Record the endpoint, model and API version. Check which controls are supported, remove the others, and choose an output budget that permits a complete result. Keep required terminology and missing-data behavior in the prompt.

3. 3

   Change one thing and judge the result

   Compare the baseline with one supported adjustment. For extraction, inspect source agreement; for drafts, inspect factual accuracy and usefulness; for code, inspect behavior and checks. Keep the simpler configuration unless the change earns its place.

Frequently Asked Questions

How does the temperature parameter work in LLM?

It changes how concentrated the next-token probability distribution is. Lower values favor likely choices; higher values permit more variety. It changes sampling, not learned weights, and some current APIs reject or ignore the setting.

What is the approximate size of a 70B model?

For exactly 70 billion weights, raw storage is about 140 GB at 16-bit, 70 GB at 8-bit or an ideal 35 GB at 4-bit. These decimal figures exclude runtime memory and quantization overhead. The actual checkpoint and workload determine deployment requirements.

What are the four types of parameters?

There is no universal four-type classification for LLMs. A useful distinction is learned model parameters, architecture settings, training settings and generation settings. These are categories for understanding the system, not a claim that every API has four controls.

What is the difference between GPT and LLM?

LLM means large language model, the broad category. GPT means generative pretrained transformer and names a particular model family and approach within that category. Other language-model families are also LLMs.

What are the four types of LLM?

There is no single standard list of four. Dense versus mixture-of-experts describes architecture; open versus closed weights describes availability. Those distinctions answer different questions and should not be combined into a supposed quality ranking.

Is ChatGPT an LLM or AI?

ChatGPT is an AI application that uses models. Distinguish the application, the model selected within it and the API settings available to developers; they are different layers.

What AI is not LLM?

AI is broader than language models. A fraud classifier or a computer-vision system can use machine-learning methods without using an LLM. Parameter count is not exclusive to language models either.

What is the smartest AI LLM?

There is no task-independent answer that parameter count can establish. Compare models on representative examples of the work, with the same acceptance criteria and documented configuration. A result on one benchmark does not settle every business task.

What branch of AI are LLMs?

LLMs are deep-learning models used for language processing and generation. Their learned parameters belong to the model; temperature and similar request controls belong to how the application uses it.

For practical explanations of AI systems and the decisions behind them, join the newsletter.