Best Local LLMs in 2026: Qwen3.6, Gemma 4, gpt-oss, and Phi-4 (Compared)
Six current local LLMs ranked by memory fit, workload, and runner, with 4-bit sizes, 2026 pricing, setup steps, and models to skip.
The best local LLM for a 32 GB machine is Qwen3.6-35B-A3B: its current 4-bit Ollama build is 24 GB, leaving 8 GB before the runtime, operating system, and context cache take their share. For 8 GB, choose Qwen3.5-9B; for 16 GB, choose Gemma 4 12B for multimodal work or gpt-oss-20b for reasoning.
The short answer: the best local LLM for each machine
A local model has to fit before any benchmark matters. That sounds obvious, yet it is the mistake behind most bad recommendations: a sparse model may activate only a few billion parameters for each token while still requiring every weight to sit in memory.
This ranking starts with the current downloadable artifact, adds working headroom, and only then asks what the model is good at. These are comparisons from current model cards and runner pages, not performance claims from a test that did not happen.
The single best default is Qwen3.6-35B-A3B on a machine with roughly 32 GB of available system or unified memory. It combines open weights, multimodal input, and agentic coding in a current 24 GB Q4_K_M package. If that file does not fit with room to work, stepping down to Qwen3.5-9B is a better decision than forcing the larger model into swap.
The 16 GB tier has two winners because the workload decides it. Gemma 4 12B is the more useful fit when an assistant needs to inspect images, video, or audio. gpt-oss-20b is the reasoning and tool-use pick, but its 14 GB package leaves only 2 GB before the rest of the computer asks for memory.
How much memory a local LLM really needs
Four quantities get blurred together in local-model discussions: parameter count, active parameter count, quantized file size, and working memory. They answer different questions.
Total parameters describe all the learned weights in the model. Active parameters describe the subset a mixture-of-experts model uses for one token. The second number helps explain compute efficiency, but the first still matters for storage and memory residency. Google makes the distinction explicit for Gemma 4 26B A4B: only 4 billion parameters are active, yet all 26 billion must be loaded into memory.
Quantization stores those weights at lower numerical precision. A 4-bit build is much smaller than full precision, with a possible capability tradeoff. It is closer to compressing a detailed map into a pocket edition than deleting three quarters of the roads: the structure remains, but some precision is lost.
Working memory is the model file plus everything required to use it. The runner needs memory. The operating system needs memory. The KV cache, a fast scratchpad for the prompt and generated conversation, grows with context. Google's own Gemma table warns that its static-weight estimates exclude supporting software and the context window.
That is why a model advertised with a 256K context limit can fit in memory while the full 256K context does not. A context limit is a maximum supported by the architecture, not a promise about your laptop. Both Qwen3.5-9B and Qwen3-Coder-Next tell users to reduce context after an out-of-memory error.
Start with the memory tier, then choose the model
Use these as deployment floors, not guarantees for every context length:
- 4 GB dedicated VRAM: Phi-4-mini's 2.5 GB Q4_K_M file is the credible small-model choice when the machine also has normal system RAM. For CPU-only use, 8 GB of system RAM is a more sensible floor.
- 8 GB dedicated VRAM: Qwen3.5-9B's 6.6 GB package fits if context stays modest. A machine with 12 GB or more total system or unified memory is safer.
- 16 GB available memory: Gemma 4 12B has the cleaner fit. gpt-oss-20b reaches OpenAI's official 16 GB floor, but it is the tight option.
- 32 GB available memory: Qwen3.6-35B-A3B is the strongest all-round recommendation here because its 24 GB package leaves 8 GB before runtime and context use.
- 64 GB or more: Qwen3-Coder-Next becomes viable. Its 52 GB package still leaves only 12 GB before the rest of the stack.
The headroom calculation exposes how different these fits are. Qwen3.5-9B leaves 1.4 GB in an 8 GB allocation, or 17.5%. gpt-oss-20b leaves 2 GB in 16 GB, or 12.5%. Qwen3.6 leaves 8 GB in 32 GB, or 25%. Qwen3-Coder-Next leaves 12 GB in 64 GB, or 18.75%.
Those percentages are not available memory after launch. They are the upper bound before the runner, operating system, and KV cache take their share. The gpt-oss pairing is therefore a technical floor, while the Qwen3.6 pairing has the healthiest margin of the four.
1. Qwen3.6-35B-A3B: best overall local LLM for 32 GB
Qwen3.6-35B-A3B is the best overall local LLM when 32 GB of memory is genuinely available. Qwen describes it as a 35 billion parameter mixture-of-experts model with 3 billion active parameters, open weights, multimodal thinking, and a focus on agentic coding. The current Ollama package is 24 GB at Q4_K_M, which is the number that decides whether it belongs on your machine.
The model fits a founder or senior builder who wants one private assistant for repository questions, implementation planning, image inspection, and longer coding work. It is a better default than a coding-only model when the same local assistant also has to read screenshots, diagrams, or documents. The 3 billion active figure helps execution efficiency, but it does not shrink the package to 3 GB: the installed 4-bit artifact is still 24 GB.
The practical wall is context. The file consumes three quarters of a 32 GB allocation before a conversation begins. Long repositories, many images, and large histories all expand the cache, so a shorter useful context is preferable to an advertised maximum that pushes the machine into swap.
Best for: A 32 GB machine that needs one capable local model for coding, reasoning, and images.
Standout: 35 billion total parameters, 3 billion active, multimodal input, and a 24 GB Q4_K_M build.
Pricing: Open weights under the Apache License 2.0 attached to the current Ollama artifact; local hardware is the operating cost.
Free trial: Not applicable for downloadable weights.
- Best balance of current capability and practical 32 GB fit in this list.
- Multimodal input avoids maintaining a separate vision model.
- Agentic coding is a first-party release priority, not an incidental use case.
- Exact current Ollama command and artifact are easy to verify.
- The 24 GB file is too large for a 24 GB machine in normal use.
- Large contexts can consume the 8 GB headroom quickly.
- Smaller models will respond faster on modest hardware.
Run the top pick with Ollama
Check the memory you can dedicate
Use 32 GB as the working floor for this 24 GB artifact. On unified-memory systems, subtract what the operating system and other applications already consume.
Install Ollama
Download the current desktop or command-line build from Ollama. The Free tier is enough for unlimited use on your own hardware.
Launch the exact model tag
Run
ollama run qwen3.6:35b-a3b. That tag resolves to the current 24 GB Q4_K_M artifact documented on Ollama's model page.Begin with a short working context
Start with the repository folder or documents needed for the task, not an entire archive by default. If the machine swaps or the runner reports an out-of-memory error, reduce context before choosing a lower-precision artifact.
2. Qwen3.5-9B: best local LLM for an 8 GB GPU
Qwen3.5-9B is the best compact local LLM here for an 8 GB dedicated GPU. The official model card describes a 9 billion parameter causal language model with a vision encoder, and the current Ollama package is 6.6 GB with text and image input. That makes it the smallest recommendation in this list that can serve as a credible multimodal daily assistant.
The concrete use case is a desktop coding companion that can also read an error screenshot, a UI mockup, or a diagram. A solo builder with an 8 GB GPU can use it for code explanation, short refactors, and visual questions without allocating the 24 GB required by Qwen3.6. It is also the sensible fallback when the larger model technically runs but spends its time paging memory.
The 256K context label needs discipline. Qwen's card gives a native limit of 262,144 tokens and explicitly advises reducing context after an out-of-memory error. The 6.6 GB file leaves just 1.4 GB in an 8 GB VRAM budget before cache and runner overhead, so the full context belongs on much larger hardware.
Best for: An 8 GB dedicated GPU, compact multimodal chat, and everyday coding help.
Standout: A 6.6 GB artifact with text and image input.
Pricing: Downloadable model weights with no model subscription listed on the source page; hardware and any hosted runner are separate.
Free trial: Not applicable for downloadable weights.
- Fits a common 8 GB GPU with a modest context.
- Accepts images as well as text.
- Much easier to place than a 20 GB to 24 GB artifact.
- Current family, rather than an older Qwen2.5 default.
- Only 1.4 GB of nominal VRAM remains in the 8 GB pairing.
- The maximum context is not realistic at the minimum hardware tier.
- A 9 billion parameter model has less room for difficult reasoning than the larger picks.
3. Gemma 4 12B: best multimodal local LLM for 16 GB
Gemma 4 12B is the best 16 GB choice when multimodal input is central to the job. Google's current family processes text, images, and video, while the 12B size also accepts audio. Open weights permit responsible commercial use under the Gemma terms, which makes it useful for a private document or media assistant when those terms fit the project.
The model is a strong fit for a product operator who needs to compare screenshots, summarize a recorded call, inspect a short video, and reason over the accompanying text on the same machine. That is a wider input mix than gpt-oss-20b, which is text only. It also has a cleaner memory position: Ollama's current gemma4:12b artifact is 7.6 GB, leaving substantial room in a 16 GB allocation.
Google's own Q4_0 estimate is 6.7 GB, while Ollama's current artifact is 7.6 GB. Those numbers are not contradictory because the build and quantization package differ. The useful lesson sits in Google's caveat: static model memory excludes supporting software and the context window, so neither number should be treated as the entire working footprint.
The 12B model belongs to Gemma's medium group with a 256K context limit. As with Qwen, that architectural ceiling is not a sensible default on 16 GB. Start with the documents or media needed for the task, then grow context only while the machine remains responsive.
Best for: A 16 GB multimodal assistant for text, images, video, and audio.
Standout: Broad input support in a current 7.6 GB Ollama artifact.
Pricing: Open weights under the Gemma terms; no model subscription is required to download the weights.
Free trial: Not applicable for downloadable weights.
- Best input coverage at the 16 GB tier.
- The 7.6 GB artifact leaves more working room than gpt-oss-20b.
- Google publishes unusually clear model-memory guidance.
- Responsible commercial use is explicitly permitted under the Gemma terms.
- Gemma terms require a separate license review for commercial deployment.
- The full 256K context is not a practical target on the minimum machine.
- The 12B pick is not the family's smallest or largest option, so choosing by name alone is easy to get wrong.
4. gpt-oss-20b: best local reasoning model at the 16 GB floor
gpt-oss-20b is the reasoning-first local model for a machine that can meet a strict 16 GB memory floor. OpenAI lists 21 billion total parameters, 3.6 billion active parameters, up to 128K context, an Apache 2.0 license, and support for tool use, function calling, Structured Outputs, and low, medium, or high reasoning effort. The current Ollama package is 14 GB and accepts text only.
This is the choice for a local automation that has to reason through a structured task and emit a predictable schema, such as classifying support requests before a human reviews them. It is also useful for a developer who values adjustable reasoning effort more than image input. The model was trained on a mostly English, text-only mix focused on STEM, coding, and general knowledge, so it should not be mistaken for a vision model.
OpenAI says the 20B model can run with 16 GB of memory. Ollama's 14 GB artifact explains the floor, but the remaining 2 GB is only 12.5% of that allocation before the runner, operating system, and context cache. A 24 GB system is the more comfortable recommendation if this model will do sustained work rather than answer short prompts.
Best for: Text reasoning, tools, structured output, and agent workflows.
Standout: 3.6 billion active parameters, adjustable reasoning effort, and an official 16 GB floor.
Pricing: Open weights under Apache 2.0; there is no model subscription for local use.
Free trial: Not applicable for downloadable weights.
- Strong first-party support for reasoning and tool-oriented workloads.
- Apache 2.0 is straightforward for many commercial projects.
- Structured Outputs and adjustable reasoning effort help application design.
- The 14 GB package is smaller than Qwen3.6's 24 GB package.
- Text only, so it cannot replace a multimodal assistant.
- The official 16 GB floor leaves very little working headroom.
- A long 128K conversation requires more memory than the base file suggests.
5. Phi-4-mini-instruct: best small local LLM for CPU and 4 GB VRAM
Phi-4-mini-instruct is the best small local LLM in this list for constrained hardware. Microsoft's model card specifies 3.8 billion parameters, a 128K context limit, text-only input, support for 24 languages, and an MIT license. Ollama's Q4_K_M artifact is 2.5 GB, which gives it a credible place on a 4 GB dedicated GPU or a CPU-only machine with at least 8 GB of system RAM.
The useful scenario is a narrow local utility: rewrite a support reply, extract fields from a short document, classify notes, or help with a small Python script without sending the text to a hosted model. Microsoft explicitly positions it for memory-constrained, compute-constrained, and latency-bound environments, as well as reasoning and math. That is a better reason to pick it than pretending it competes with a 24 GB model on every task.
Its main weakness is unusually clear. The static training cutoff is June 2024, and Microsoft warns that the small model cannot store much factual knowledge, so users may encounter factual errors. The model card points to search or retrieval-augmented generation as a possible mitigation. In plain terms, give it the source material for factual work instead of asking it to remember current facts.
Best for: CPU-friendly utilities, older hardware, short text tasks, and a 4 GB GPU tier.
Standout: A 2.5 GB Q4_K_M artifact under the MIT license.
Pricing: Downloadable MIT-licensed weights; local hardware remains the cost.
Free trial: Not applicable for downloadable weights.
- Smallest credible package in the main ranking.
- Clear MIT license.
- Officially designed for constrained memory and latency.
- Tool-enabled function calling is documented in the model card.
- Text only.
- June 2024 knowledge cutoff.
- Microsoft warns that limited factual storage can produce incorrect answers.
- Long conversations can drift even though the architecture supports 128K context.
6. Qwen3-Coder-Next: best local coding LLM for 64 GB or more
Qwen3-Coder-Next is the best specialist local coding LLM here for a 64 GB or larger machine. Qwen designed it for coding agents and local development, with 80 billion total parameters, 3 billion active parameters, long-horizon reasoning, complex tool use, recovery from execution failures, and a native 262,144-token context. The current Ollama Q4_K_M artifact is 52 GB.
This is the model for a senior builder who wants a local agent to navigate a large repository, call tools, edit files, and recover after a failed command. It is not the default for general chat, image work, or a 32 GB laptop. The official card says it uses non-thinking mode only, which means it does not produce the model's visible thinking blocks.
The name's 3 billion active figure causes the same trap as other sparse models. Ollama's artifact is 52 GB because all 80 billion total weights still need representation in the package. Loading that into 64 GB leaves 12 GB, or 18.75%, before runtime and cache use, so 64 GB is a floor and more memory is valuable for repository-scale context.
Qwen explicitly advises reducing context to 32,768 after an out-of-memory error. That is the correct first response on a 64 GB machine. A shorter context that keeps the agent responsive is more useful than forcing the full 262,144-token ceiling.
Best for: Local coding agents, repository work, and tool-rich development on 64 GB or more.
Standout: 80 billion total parameters, 3 billion active, a 52 GB Q4_K_M package, and coding-agent specialization.
Pricing: Open weights under the Apache License 2.0 attached to the current Ollama artifact; hardware is separate.
Free trial: Not applicable for downloadable weights.
- Purpose-built for coding agents and local development.
- Tool use and failure recovery are part of the official positioning.
- Large native context for machines that can afford the cache.
- Current Ollama package has a verified launch command.
- The 52 GB package excludes almost every mainstream laptop.
- Coding specialization makes it a poor universal default.
- Non-thinking mode only.
- Full context can still trigger an out-of-memory error at the minimum tier.
Ollama vs LM Studio vs llama.cpp
The model decides the ceiling, but the runner decides how much friction sits between a download and a useful local endpoint. Ollama is the automation choice, LM Studio is the graphical choice, and llama.cpp is the control choice. All three can be valid on the same machine.
Ollama: best for the shortest path to a CLI and API
Ollama is the easiest recommendation when a local model needs to become a command, script, or API endpoint. Its current Free plan includes local models on your hardware, a CLI, API, and desktop apps. Models on your own hardware are always unlimited, which is the only pricing fact that matters for a purely local build.
Ollama's paid tiers add cloud capacity rather than charging for local inference. Prices and limits below were verified on August 5, 2026.
Individual cloud session limits reset every 5 hours, and weekly limits reset every 7 days. Those resets do not apply to models running on your own hardware. For the top pick, the entire local setup can be one command after installation: ollama run qwen3.6:35b-a3b.
Best for: Builders who want a clean local CLI, API, desktop app, and predictable model tags.
Standout: Free, unlimited local hardware use plus optional cloud tiers.
Pricing: Free $0; Pro $20 monthly or $200 yearly; Max $100 monthly with new sign-ups paused; Team $25 per seat monthly with a 5-seat minimum; Enterprise custom.
Free trial: The Free tier is ongoing, not a timed trial.
LM Studio: best graphical local-model experience
LM Studio is the better runner when downloading, loading, and chatting with a model should happen through a graphical interface. Its official guide takes the user through a Discover tab, a model loader, and a Chat tab. The Free plan runs local LLMs and voice transcription on the user's machine, and the pricing page says no data leaves the device for that local use.
LM Studio also exposes cloud inference, so its pricing needs a clean separation from local use. Prices were verified on August 5, 2026:
- Free, $0: local LLMs, Bionic Agent, llama.cpp and MLX runtimes, offline voice transcription, a limited zero-data-retention web-search tool when logged in, and LM Link for up to 5 devices.
- Pay as you go: cloud credits billed per 1 million tokens. DeepSeek V4 Flash is $0.13 input, $0.028 cached input, and $0.26 output. DeepSeek V4 Pro is $1.74 input, $0.15 cached input, and $3.48 output.
- Pay as you go, continued: GLM-5.2 is $1.50 input, $0.30 cached input, and $4.50 output. Kimi K2.6 is $0.95 input, $0.16 cached input, and $4.00 output. Kimi-K2.7-Code is also $0.95 input, $0.16 cached input, and $4.00 output. Kimi K3 is $3.00 input, $0.30 cached input, and $15.00 output.
- Bionic Pass: pricing and plan details are coming soon.
For a founder evaluating models on one machine, the Free graphical flow is the reason to choose LM Studio. For an application that needs a repeatable service under version control, Ollama or llama.cpp gives a cleaner operational path.
Best for: Visual model discovery, loading, and local chat.
Standout: A $0 local desktop experience with llama.cpp and MLX runtimes.
Pricing: Free $0; cloud inference is pay as you go at the model rates above; Bionic Pass is not yet priced.
Free trial: The Free tier is ongoing, not a timed trial.
llama.cpp: best for hardware control and hybrid inference
llama.cpp is the control choice when the machine, build flags, and layer placement matter. The current MIT-licensed project supports Apple silicon through Metal, NVIDIA GPUs through CUDA, AMD GPUs through HIP, and CPU plus GPU hybrid inference for a model larger than available VRAM. It provides a CLI, a server, and a built-in web interface.
The concrete use case is a custom workstation where part of a model sits on the GPU and the remainder spills into system memory, or where a builder needs a specific backend. Running llama serve launches the project server. That control is valuable, but it comes with more setup and more responsibility for choosing the right GGUF file and runtime flags.
llama.cpp has no commercial tier table to decode. It is an MIT-licensed open-source project. The cost is the hardware and the builder time required to configure it.
Best for: Custom hardware, CPU and GPU hybrid inference, and low-level deployment control.
Standout: Broad backend support and a local server without a paid product tier.
Pricing: MIT-licensed open source; no paid tier is listed by the project.
Free trial: Not applicable.
- Ollama gives the shortest path to a reproducible command and API.
- LM Studio gives the most approachable graphical model workflow.
- llama.cpp gives the deepest hardware and backend control.
- Ollama's cloud pricing can distract from the fact that local use is unlimited.
- LM Studio's GUI is less natural for scripted deployment.
- llama.cpp asks the user to make more model-format and runtime decisions.
How these local LLMs were picked
The cutoff for this comparison was August 5, 2026. A model needed current first-party documentation, downloadable weights, a useful consumer or prosumer workload, and a runnable artifact whose size could be checked against an official runner page.
The ranking used six criteria:
- Practical 4-bit footprint: the current model package had to fit a named memory tier with some room left.
- Useful workload: each model needed a reason to exist in the list, such as multimodal input, reasoning tools, low-memory use, or coding-agent specialization.
- Current family: older defaults were removed when a current successor family was already documented and runnable.
- First-party limits: context, modality, licensing, and known weaknesses came from vendor model cards or official project pages.
- Runner availability: a current Ollama artifact or clear support in a current runner had to exist.
- Candid exclusion: a strong server model did not become a consumer recommendation merely because its active-parameter count looked small.
No cross-vendor benchmark score was used as a universal ranking because the vendors do not publish one shared evaluation setup here. The verdict turns on deployment fit and workload, which are the decisions a local-model buyer can verify before a large download.
This is also why the list contains six models rather than padding to match a longer incumbent roundup. Qwen3.6 and Gemma 4 replace several older family recommendations, while the memory tiers prevent two nearly identical model sizes from becoming separate filler entries.
The ones to avoid on ordinary consumer hardware
Avoid does not mean bad. It means the model is a poor answer to a consumer-hardware query.
Mistral Small 4 is a current, capable model with text and image input, configurable reasoning, a 256K context window, and an Apache 2.0 license. It also has 119 billion total parameters and 6 billion active per token. That total-weight class makes it a workstation or server candidate, not the default answer for a normal laptop.
Llama 4 Scout has 109 billion total parameters and 17 billion active. Meta says the Int4 model fits a single NVIDIA H100 GPU. Llama 4 Maverick has 400 billion total parameters and 17 billion active, and Meta describes a single H100 host deployment. Those are useful deployment facts, but they do not describe ordinary consumer hardware.
DeepSeek-V4-Flash sounds like the smaller option, yet it has 284 billion total parameters and 13 billion active. DeepSeek-V4-Pro rises to 1.6 trillion total and 49 billion active. Both support a 1 million token context in official DeepSeek services, which is a hosted-service advantage, not evidence that the open weights belong on a mainstream desktop.
A new 2026 shortlist should not begin with Llama 3.1 8B, Mistral 7B, Phi-3.5 Mini, Gemma 2 9B, or Qwen2.5 7B merely because an older roundup still names them. They may remain useful for an existing deployment, but the current Qwen3.5, Qwen3.6, Gemma 4, and Phi-4 families deserve the first evaluation for a new build.
The distinction matters for migration cost. A stable existing application does not need a model upgrade for fashion. A new application should not start with an older default before checking whether the current family fits the same memory and license constraints.
Final decision guide
At 4 GB of dedicated VRAM, use Phi-4-mini-instruct for narrow text work and supply evidence for factual tasks. At 8 GB of dedicated VRAM, use Qwen3.5-9B with a modest context for the best compact mix of text, images, and coding.
At 16 GB, use Gemma 4 12B when images, audio, or video matter. Choose gpt-oss-20b when reasoning, tools, and structured text output matter more, and prefer 24 GB total system memory if the computer will run other development tools beside it.
At 32 GB, choose Qwen3.6-35B-A3B. It is the best all-round answer in this comparison because its current 24 GB package leaves enough nominal headroom to be useful while preserving multimodal and agentic-coding capability.
At 64 GB or more, choose Qwen3-Coder-Next only when coding agents are the main workload. Its 52 GB package and coding specialization are excessive for general chat, but appropriate for repository-scale local development.
For a broader look at open-weight models that are not limited to consumer hardware, use the best open-source LLMs comparison. If the model choice is settled but Ollama is not, compare the current Ollama alternatives.
The runner choice is simpler: Ollama for repeatable commands and an API, LM Studio for a graphical workflow, llama.cpp for hardware control. The best local LLM is the one that fits the machine, serves the workload, and leaves enough memory to finish the prompt.
Frequently asked questions
What is the best local LLM for coding in 2026?
Qwen3-Coder-Next is the specialist choice on a machine with 64 GB or more because its current Q4_K_M artifact is 52 GB and the model is designed for coding agents. Qwen3.6-35B-A3B is the better all-round coding choice at 32 GB.
What is the best local LLM for 16 GB of RAM?
Gemma 4 12B is the safer multimodal fit because its current Ollama artifact is 7.6 GB. gpt-oss-20b reaches OpenAI's official 16 GB floor for reasoning, but its 14 GB artifact leaves only 2 GB before runtime and context memory.
Is Ollama better than LM Studio for local LLMs?
Ollama is better for a CLI, scripts, and an API. LM Studio is better when a graphical model browser, loader, and chat interface matter more. Both have a $0 local-use path.
How much RAM or VRAM do I need for a local LLM?
Start with the quantized model file, then leave room for the runner, operating system, and KV cache. Practical options in this comparison begin with a 2.5 GB file for constrained machines and rise to a 52 GB file that needs 64 GB or more.
Are local LLMs free?
The models in this list have downloadable open weights, so there is no per-prompt model fee for local inference. The machine, storage, memory, electricity, setup time, and any optional cloud service remain costs.
Get the AI business workflow audit checklist for a practical way to decide which local or hosted workflow is worth building next.
Aug 5, 2026
