Best Local LLMs in 2026: Qwen3.6, Gemma 4, gpt-oss, and Phi-4 (Compared)

Run quantized Gemma 4 12B on a 16 GB laptop, gpt-oss-20b on a 24 GB GPU, and gpt-oss-120b on a workstation with an 80 GB GPU. These seven best local LLMs in 2026 are ranked by hardware fit and workload, with the maker's memory guidance separated from a model's maximum context window.

Best Local LLMs in 2026: The Short Answer for Each Machine

Gemma 4 12B is the laptop default; gpt-oss is the text-reasoning choice when you have a dedicated GPU. Google's Gemma launch page explicitly names 16 GB of VRAM or unified memory. OpenAI's gpt-oss-20b card says its MXFP4 model runs within 16 GB, making it my recommendation for the 24 GB GPU in this comparison. Its larger sibling has a documented deployment on a single GPU with 80 GB.

That last recommendation is for a workstation equipped with the stated accelerator, not every computer sold as a workstation. If yours has a smaller GPU, start with the smaller model. A model has to fit before any benchmark matters.

Licenses, parameters, context limits, hardware statements and release dates were checked against the makers' pages on October 16, 2026. This is a documentation-based comparison; no local inference benchmark was performed.

Since October 15, 2026, Meta’s Muse Spark Open weights are available for businesses that want a private multimodal model on their own machines. Meta names MLX on two M5 Ultra Mac Studios, alongside llama.cpp and Unsloth support. This adds a high-memory option; smaller offices should still start with the laptop or single-GPU picks above.

Since August 13, 2026, DeepSeek-V4-Pro-0813 has had an official public release, with MIT-licensed weights on DeepSeek's own Hugging Face organization. JetBrains released Mellum2.1 on October 8, 2026, with Apache 2.0 weights. The practical split is hardware and purpose: DeepSeek's card documents a server deployment, while Mellum2.1 belongs in the local AI models for coding comparison.

How much memory a local LLM really needs

Parameter count, active parameters, loading memory and working memory describe different things. Mixing them is how a plausible download becomes an unusable local assistant.

Total parameters describe the learned weights. In a mixture-of-experts model, only a subset is active for each generated token. That smaller active count describes computation; it does not establish how much memory the complete model needs.

Quantization stores weights at reduced numerical precision. A compressed weight format can make a model practical on a smaller machine, but its name does not specify the entire memory budget. The runtime also needs working buffers and a KV cache, the scratch space that holds information from the conversation.

Google gives a useful concrete example. Its Gemma memory table estimates 26.7 GB for BF16, 13.4 GB for SFP8 and 6.7 GB for Q4_0 for Gemma 4 12B. These are approximate loading estimates. The same page warns that supporting software and context need additional memory. Google's laptop recommendation therefore does not mean the full-precision model fits into the same allocation as its quantized version.

Start with the memory tier, then choose the model

For a 16 GB laptop, begin with Gemma 4 12B through a supported quantized runtime. For a 24 GB GPU, begin with gpt-oss-20b for text reasoning. For a workstation with an 80 GB GPU, consider gpt-oss-120b. Muse Spark Open sits above these tiers: its MLX deployment spans two Mac Studios, while Unsloth’s GGUF memory guidance still runs into hundreds of GB. These recommendations follow the published guidance; they are not measurements of your machine.

For Qwen3.5-9B, Qwen3.6-35B-A3B and Phi-4-mini-instruct, the cards do not supply the consumer memory floors previously used here. Choose an exact artifact and check its documented requirements rather than treating the parameter count as a RAM specification.

The context window is another separate limit. It describes the amount of input and output the architecture supports, not what fits beside the weights on your laptop. Both Qwen cards explicitly tell users to reduce context after an out-of-memory error.

1. Gemma 4 12B: best local LLM for a laptop

Gemma 4 12B is the first pick for a laptop assistant that needs to read more than text. Google's instruction-tuned model accepts text, images, video and audio, then generates text. A product operator can use that input mix to ask about a screenshot alongside meeting notes without selecting a separate model for each medium. The limit is the actual runtime: check that it supports the modalities you intend to use.

The card specifies 11.95 billion total parameters, a 256K-token context and Apache 2.0 licensing. The old description of this model as restricted to Gemma's custom terms was incorrect. Its unified architecture feeds media into the language model without separate vision and audio encoders.

Google released it on June 3, 2026 and explicitly describes local use with 16 GB of VRAM or unified memory. That is the reason it leads the laptop recommendation. It does not establish full-context fit or promise that every desktop app exposes audio and video.

Best for: A laptop assistant for documents, screenshots and media.
License: Apache 2.0; public weights in Google's Hugging Face organization.
Parameters and context: 11.95B total; 256K tokens.
Maker's hardware guidance: 16 GB VRAM or unified memory, with an appropriate local deployment.
Release: June 3, 2026.

Set up the laptop pick

1. 1

   Choose the instruction-tuned model

   Start from Google's gemma-4-12B-it card. The instruction-tuned checkpoint is the assistant variant; keep its identity and license attached to whichever supported local format you select.

2. 2

   Choose a supported local runner

   Google's launch page names LM Studio and Ollama. In a graphical runner, inspect the publisher, format and quantization before downloading. A community conversion is a separate artifact from Google's original checkpoint.

3. 3

   Load only the context the task needs

   Begin with a document or screenshot you can check yourself. Leave room for the runtime and response instead of setting the model's maximum context by default.

4. 4

   Check the output against the source

   Ask for a summary and the supporting passage or visual detail. A local answer still needs checking; keeping data on the device does not make the answer correct.

2. gpt-oss-20b: best local LLM for a 24 GB GPU

gpt-oss-20b is the text-reasoning pick for a 24 GB GPU when the runtime supports its quantized format. OpenAI documents 21 billion total parameters, 3.6 billion active parameters, Apache 2.0 licensing and an MXFP4 deployment that runs within 16 GB of memory. The recommendation for the larger GPU budget is an editorial hardware choice, not an OpenAI claim that this is the minimum.

A useful workload is sorting support requests into a structured schema for human review. The card documents function calling, structured outputs and adjustable reasoning effort. Your application still supplies the tools; downloading the weights does not create a working automation by itself.

OpenAI's model reference gives a 131,072-token context and text-only input and output. Use Gemma when the task requires images or audio. The gpt-oss card also requires the Harmony response format, so a compatible runtime matters as much as downloading the correct weights.

Best for: Text reasoning, structured responses and tool-connected local applications.
License: Apache 2.0; public weights in OpenAI's Hugging Face organization.
Parameters and context: 21B total, 3.6B active; 131,072 tokens.
Maker's hardware guidance: Runs within 16 GB using MXFP4.
Release: August 5, 2025.

For the maker-documented Ollama path, install the runner and use ollama run gpt-oss:20b. Select local execution; no hosted-model account is needed for that path.

3. Qwen3.6-35B-A3B: a larger multimodal alternative

Qwen3.6-35B-A3B belongs on the shortlist when you can validate a larger deployment and want text, image and video input. Qwen describes 35 billion language-model parameters with 3 billion active, plus a vision encoder. Its retained-reasoning option can help an application carry a line of reasoning through successive requests. That makes it relevant to document and diagram workflows as well as development work.

The card gives 262,144 tokens natively, extensible to 1,010,000 with configuration changes. Its deployment examples use 8 GPUs, but that is an example configuration, not a minimum. Crucially, the card does not certify the old article's consumer-memory recommendation.

Do not choose it merely because a reduced-precision package fits on disk. Qwen advises reducing context after an out-of-memory error while recommending at least 128K for complex-task thinking. That tension is the decision: if your hardware cannot sustain the context your work needs, select a smaller model.

Best for: A larger multimodal deployment with a verified artifact and memory budget.
License: Apache 2.0, stated in the official card metadata.
Parameters and context: 35B language-model parameters, 3B active; 262,144 native tokens.
Maker's hardware guidance: Multi-GPU serving examples; no numeric consumer RAM minimum.
Release: April 14, 2026, the date on Qwen's announcement.

4. Qwen3.5-9B: the smaller Qwen vision option

Qwen3.5-9B is the smaller alternative when you want Qwen's text and vision capabilities. Its card specifies 9 billion language-model parameters, a vision encoder and 262,144 native context tokens, extensible to 1,010,000. It is a candidate for asking questions about a screenshot or a document alongside ordinary text. The card does not establish a consumer GPU memory minimum.

That last point changes the buying advice. A runner's quantized download size cannot certify the memory required by Qwen's original checkpoint. Confirm the chosen conversion, its vision support and its runtime before assigning it to your GPU. A smaller language model is a useful starting point for memory planning, not a complete deployment specification.

Best for: A smaller Qwen assistant for text, images and video.
License: Apache 2.0, stated in the official card metadata.
Parameters and context: 9B language-model parameters plus vision encoder; 262,144 native tokens.
Maker's hardware guidance: No numeric consumer RAM or VRAM minimum; reduce context if memory is exhausted.
Release: March 2, 2026, explicitly listed in the maker's release history.

5. Phi-4-mini-instruct: best for small text utilities

Phi-4-mini-instruct is the focused choice for constrained text tasks. Microsoft's card specifies 3.8 billion parameters, 128K context, text input and output, and an MIT license. Its intended uses include memory-constrained, compute-constrained and latency-bound applications. For a local utility, extracting fields from supplied notes is a more defensible workload than asking it to recall current events.

Microsoft warns that the small model has limited capacity to store factual knowledge. Its public-data cutoff is June 2024, while the card dates its release to February 2025. Supply the relevant document for factual questions and check the answer against it.

The hardware section lists NVIDIA A100, A6000 and H100 as tested GPUs. That list is not a minimum requirement. It does not support the previous promise that a particular low-memory GPU or CPU-only machine will run every deployment format.

Best for: Small text utilities that work from supplied evidence.
License: MIT; public weights in Microsoft's Hugging Face organization.
Parameters and context: 3.8B; 128K tokens.
Maker's hardware guidance: Tested GPU families are named; no numeric consumer memory minimum.
Release: February 2025, as stated on the card.

6. gpt-oss-120b: the high-memory workstation pick

gpt-oss-120b is the workstation choice when you have an appropriate 80 GB GPU and need a larger text-reasoning model. OpenAI specifies 117 billion total parameters, 5.1 billion active parameters and a deployment on a single GPU with 80 GB with MXFP4. Its weights are public in OpenAI's own repository. This is a concrete hardware tier, not a recommendation for an ordinary desktop merely labeled a workstation.

A technical operator might use it as a shared local reasoning endpoint for document-based analysis or structured tool workflows. It retains the smaller model's reasoning controls and Harmony-format requirement. The tradeoff is straightforward: choose it when the larger deployment serves a defined workload, not because its model name contains a larger number.

The official model reference specifies a 131,072-token context and text-only input and output. If your workstation lacks the documented GPU capacity, gpt-oss-20b remains the more defensible starting point.

Best for: Larger local text-reasoning deployments on suitably equipped workstations or servers.
License: Apache 2.0.
Parameters and context: 117B total, 5.1B active; 131,072 tokens.
Maker's hardware guidance: Single 80 GB GPU using MXFP4.
Release: August 5, 2025.

7. Muse Spark Open: for a high-memory private deployment

Muse Spark Open is worth evaluating when your business needs a shared private model for multimodal reasoning and tool-driven work, and can support the hardware. Meta documents text, image, video and audio inputs with text output. An internal assistant that reads diagrams and meeting clips before using local tools is a plausible workload. A smaller office doing document summaries should keep the Gemma or gpt-oss recommendation rather than buy machines simply to fit this model.

Meta’s Apple-silicon deployment uses two M5 Ultra Mac Studios with MLX, describing up to 512 GB of unified memory per machine. Its model card separately gives 700 GB of VRAM for NVFP4 and 2 TB for BF16. Those are different deployment paths from the smaller GGUF conversions below. The 42B active parameters are not a memory requirement for the full 816B-parameter model.

Unsloth’s local guide says its 2-bit UD-IQ2_XXS build fits 256 GB. Its planning table gives 250-316 GB for 2-bit options and 512 GB for 4-bit, measured as total RAM plus VRAM, or unified memory. Longer context needs more memory. The GGUF repository lists UD-IQ2_S at 275 GB as a download, not a complete working-memory budget.

The distinction matters for business automation: Unsloth recommends UD-IQ2_S or larger for agentic tasks, reporting degradation on general tasks below that quantization. Treat the smallest build’s fit claim as a starting point for evaluation, not proof that it is the right assistant for your workflow. Follow the guide’s Muse-capable runtime instructions; an existing llama.cpp installation may need updating. Keep connected tools and document processing local too if the workflow must stay on your machines.

Best for: A shared private multimodal endpoint with a dedicated high-memory deployment.
License: Muse Model License, with its own usage and attribution conditions.
Parameters and context: 816B total, 42B active; 1M tokens.
Maker’s hardware guidance: MLX on two M5 Ultra Mac Studios; separate NVFP4 and BF16 requirements above.
Release: October 15, 2026.

Ollama vs LM Studio vs llama.cpp

Choose Ollama for a simple command and API, LM Studio for graphical discovery and chat, and llama.cpp for hardware control. Each is a runner, the software that loads and executes a model. Runner support must match the exact model and artifact you choose.

Ollama: a direct path to a local CLI and API

Ollama documents a local path that skips the cloud API key. Use that path when keeping inference on your own hardware is the point. Its commands and API make it a useful starting point for a script or local application; selecting a cloud model changes where inference happens.

The current Ollama pricing page lists Free at $0, Pro at $20 monthly or $200 annually, Max at $100 monthly, and Team at $500 monthly, with Enterprise priced by agreement. Pro includes $60 in monthly usage credits, Max $300, and Team $1,000 shared across unlimited users. Included usage now resets monthly. Models running on your own hardware remain unlimited; these cloud allowances do not establish whether a local model fits your machine.

LM Studio: graphical local-model discovery and chat

LM Studio provides model search and download, a chat interface and local serving endpoints. Its documentation covers GGUF models through llama.cpp and MLX on Apple silicon, as well as document chat performed offline. This suits an operator comparing local assistants through a graphical workflow.

The current LM Studio pricing page lists Free at $0 with local model support, Bionic+ at $20 monthly, and Pro at $100 monthly. The paid plans add cloud features and usage. The old unpriced Bionic Pass description and per-token cloud menu no longer match the page.

LM Studio also documents a headless runtime and developer APIs. Choose it for the workflow you prefer; a graphical interface does not mean it is restricted to manual use.

llama.cpp: hardware control and hybrid inference

llama.cpp supports CPU and GPU hybrid inference, plus backends for Apple silicon, NVIDIA and AMD hardware. It offers a CLI, server and web interface under the MIT license. The benefit is control over deployment; the cost is responsibility for choosing a compatible model file and configuration.

Hybrid inference can make a larger model usable across available memory, but it does not turn system RAM into GPU memory or establish a speed guarantee. Decide whether the resulting response time is useful for your workload.

How these local LLMs were picked

The ranking favors documented hardware fit and distinct everyday workloads. Every ranked model has public weights in its maker's own Hugging Face organization. Licensing, parameter count, context, hardware statements and release dates were checked separately; a training cutoff or recent repository edit was not treated as a release date.

Gemma leads because the maker explicitly targets laptops and its input support suits a broad assistant. gpt-oss-20b supplies the clearest text-reasoning recommendation for the requested GPU tier. The Qwen and Phi entries offer different size and modality tradeoffs, while gpt-oss-120b provides a documented single-GPU endpoint. Muse Spark Open extends the shortlist to much larger private multimodal deployments, with its conversion requirements kept separate from Meta’s original formats.

No cross-vendor benchmark score is used to manufacture a universal league table. Where a maker supplies no numeric consumer hardware requirement, the article says so. An appealing community conversion is not evidence that the original checkpoint has the same footprint.

The seven-model count excludes coding-only recommendations. For those, use the best local AI models for coding, including JetBrains Mellum2.1 Thinking and Qwen3-Coder-Next. Mellum2.1's own card confirms public Apache 2.0 weights; its release does not create a maker-backed laptop memory minimum.

The ones to avoid on ordinary consumer hardware

DeepSeek-V4-Pro-0813 has public weights, but its card is not a laptop deployment guide. The official model card states that both the repository and weights use the MIT license and that this release supersedes Pro Preview. It gives a vLLM example on a node with 4×GB300 accelerators. That is an example configuration, not a claim that no other configuration can run it.

The useful consequence is that organizations can evaluate self-hosting the released model. The release does not establish fit on the laptop or consumer GPU discussed here. Do not carry the preview's parameter counts, hosted-service limits or a smaller active-parameter number into a hardware recommendation for the new checkpoint.

The other choice to avoid is buying hardware from a model name alone. A model can have public weights and still require a deployment you cannot maintain. Start from a maker-supported configuration or a separately verified artifact, then decide whether the workload justifies the machine.

Final decision guide

Laptop: Start with Gemma 4 12B when your machine meets Google's documented memory guidance and the chosen runtime supports your media inputs. Choose Phi-4-mini-instruct for a narrow text utility after checking its exact deployment format.

24 GB GPU: Start with gpt-oss-20b for text reasoning. Choose a multimodal model instead when images or audio are essential; hardware capacity does not change a text-only model's input support.

Workstation: Use gpt-oss-120b when you have the documented accelerator capacity. Consider Qwen3.6-35B-A3B for a larger multimodal setup only after checking the artifact and context budget. Coding-agent selection belongs in the dedicated comparison linked above.

High-memory private deployment: Evaluate Muse Spark Open if multimodal tool workflows justify the larger machines. Choose the exact MLX or GGUF path first, then budget for its weights, context and runtime. An office with a laptop or consumer GPU should keep the smaller picks.

For models beyond these local hardware tiers, use the best open-source LLMs comparison. If the model choice is settled but the runner is not, compare the Ollama alternatives.

The next move is a bounded local task on hardware you already own: a document summary, structured extraction or screenshot question with an answer you can check. Verify the result and memory behavior before expanding the workload or buying a larger machine.

Frequently asked questions

Which local LLM should I run on a laptop?

Start with Gemma 4 12B. Google explicitly names 16 GB of VRAM or unified memory for local deployment. Select a supported quantized format and leave room for the runtime and context; that recommendation does not apply to every full-precision artifact.

Which local LLM should I choose for a 24 GB GPU?

gpt-oss-20b is the text-reasoning recommendation here. OpenAI documents an MXFP4 deployment within 16 GB of memory. The larger GPU budget is the scenario being recommended for, not a new minimum or a guarantee of maximum-context fit.

Which model should I run on a workstation?

gpt-oss-120b has a documented deployment on a single GPU with 80 GB. Muse Spark Open belongs in a substantially higher-memory deployment, using Meta’s two-Studio MLX path or a separately verified GGUF configuration. If your workstation has less accelerator memory, begin with gpt-oss-20b or verify another artifact against its own requirements. The workstation label alone does not establish compatibility.

Where are the local coding-model recommendations?

Use the dedicated local AI models for coding comparison, which includes Mellum2.1. This list focuses on general local workloads and hardware fit.

Get the AI business workflow audit checklist for a practical way to decide which local or hosted workflow is worth building next.