On Device AI: What Can Run on Staff Laptops and Phones

In 2026, on device AI can keep short summaries, draft edits and speech processing on staff hardware, but the app still controls whether data leaves. Move a bounded workflow first, make any cloud handoff explicit, and judge the result on your own documents before expanding it.

For an operator, the useful question is which part of a job can stay on a laptop or phone. A field worker turning a note into a handover summary is a better starting point than an assistant expected to research, reason and act across the whole business.

Verified against the makers' live documentation on 23 October 2026. The task matrix below is an analysis of documented capabilities and requirements, not a performance benchmark.

What Is On Device AI?

On-device AI runs the model on the device's own processor. That can be its central processing unit (CPU), graphics processing unit (GPU), or neural processing unit (NPU), a chip designed for supported AI calculations. Running a model to produce an answer is called inference. An NPU needs software designed to use it; its presence does not automatically accelerate every AI app. Microsoft's NPU developer guide

Think of a translator sitting beside you with a document, rather than receiving it by courier at another office. Local inference keeps that processing beside you. The app can still send a copy elsewhere, just as someone can email the translated document afterward.

Nothing needs to leave the device for the local model call. But the surrounding app may upload prompts, sync documents, call an online tool, or choose a cloud model. Apple's framework, for example, can give a model tools that query either local or online databases. The processing location and the app's sharing behavior are separate decisions. Apple's Foundation Models documentation

An offline feature also needs preparation. Model files must already be present, and the app must have access to its input without a network connection. A local model cannot summarize a document that exists only in an unreachable remote system.

Which On-Device AI Models Can a Business Use Today?

Start with the software your workflow can use, then check the hardware. Apple and Google's frameworks are building blocks for apps; Windows includes specific ready-made features; desktop model runners let staff or developers choose their own models. These are different adoption paths.

Apple Foundation Models: Small Features Inside Your App

Apple's Foundation Models framework is an interface for building model-powered features in Swift, Apple's programming language. Its on-device model supports tasks such as summarising supplied text, extracting details and refining a draft. Guided generation can return a defined Swift data structure, useful when an app needs fields rather than a paragraph. Apple's framework overview

For a product lead, a sensible feature is turning a technician's note into a structured handover for human review. The app supplies the source material and defines the output. That is a much narrower job than asking the model to decide how the equipment should be repaired.

Apple's current generation guide lists code creation, basic mathematics and logical reasoning among tasks to avoid for its on-device model. Use ordinary application code for calculations and validation, and evaluate any consequential generated content. Apple's capability guidance

The framework starts with the iOS 26, iPadOS 26 and macOS 26 generation of operating systems; newer APIs have their own availability requirements. Hardware must support Apple Intelligence: examples include iPhone 15 Pro, iPhone 16 and later models, and M1-or-later Macs. Check the actual model's availability at runtime, including language, region and download readiness. Framework availability, Apple's device requirements

Specify the on-device model in the implementation. The framework now also supports Private Cloud Compute and other server providers. An app saying it uses Foundation Models no longer tells you where a particular request runs. Apple's model support

Google On-Device AI: Android and Chrome

Google Gemini Nano is the local model behind Android's AICore system service. Developers access it through ML Kit, which provides summarisation, proofreading, rewriting and other APIs. For a field-service app, shortening a worker's supplied note is a reasonable pilot. Google's Android overview

Device support depends on the API and model version. Use Google's supported-device lists and an availability check; a generic Android version or RAM figure does not establish eligibility. There is also an operational constraint: ML Kit GenAI inference is permitted only while the app is the top foreground application, and request and battery-use quotas apply. That makes it a poor foundation for an unattended background document queue. ML Kit requirements and limits

The more flexible Prompt API remains beta without a service-level agreement or deprecation policy, so Google can make changes that break compatibility. Assess that dependency before making it essential to staff work. Google's Prompt API status

Google Chrome exposes Gemini Nano through built-in browser APIs on supported desktops and laptops. A web product can offer a local summary of text it already has; the Summarizer API is available in stable Chrome, subject to feature and model availability checks. Chrome Summarizer documentation

The CPU path requires at least 16 GB RAM and 4 CPU cores; the GPU path requires strictly more than 4 GB of VRAM, the GPU's memory. Google also requires 22 GB of free disk space on the Chrome profile volume. That is a free-space requirement, not a claim that the model itself occupies that much space. These foundation-model APIs do not currently work in Chrome on Android or iOS. Chrome hardware requirements

For an internal web tool, this means checking the staff browser fleet before promising universal availability. Keep the original text and a manual workflow usable when the local model is unavailable.

Copilot+ PCs: Useful Windows Features, With Specific Boundaries

Microsoft Copilot+ PCs provide hardware for Windows features such as Live Captions translation, Windows Studio Effects and improved Windows search. These features have their own device, language and rollout conditions. Microsoft's business feature guide

The class requires a compatible 40+ TOPS NPU, 16 GB RAM and 256 GB storage, alongside the Windows requirements. TOPS means trillions of operations per second; it describes compute capacity, not how well a model will answer your business question. Microsoft's hardware specifications

Live Captions is a useful example of a local feature: Microsoft says audio processing and caption generation stay on the device. Standard captions already exist in Windows; Copilot+ adds translation. However, generated captions are not saved, so this is not a meeting archive or a replacement for a workflow that needs retained transcripts. Microsoft's Live Captions documentation

A buyer should identify the feature that justifies the hardware. Buying an NPU does not move Microsoft Copilot's cloud experiences, or an arbitrary desktop model, onto that chip. Windows AI specifications, NPU software requirements

Ollama and LM Studio: Choose the Model Yourself

Ollama is a local model runner with an API that applications can call. Its documentation distinguishes local model execution from its optional cloud features. It suits a technical builder who wants to connect a local model to an existing workflow and inspect whether it is using CPU or GPU resources. Ollama's documentation

For a restricted local workflow, Ollama documents disable_ollama_cloud: true in its server configuration, or the OLLAMA_NO_CLOUD=1 environment setting followed by a restart. This disables Ollama's cloud models and web search. It does not control the network behavior of another app connected to Ollama. Ollama's local-only configuration

LM Studio provides a desktop interface for downloading, loading and chatting with models, including families such as Qwen, Mistral and Gemma. These are open-weight models when their maker makes the trained model files available; their licenses still differ. LM Studio's getting-started guide

Once the files are downloaded, LM Studio supports offline chat, local document processing and a local server. Its current privacy policy separately describes optional cloud models and web search. Choose the local model path deliberately. Offline operation, LM Studio's current privacy policy

LM Studio recommends at least 16 GB RAM, with additional platform requirements; that is a starting point, not a promise that every model fits. Intel Macs are not supported, while Apple Silicon is. On Windows, its x64 build requires AVX2 CPU support. LM Studio system requirements

Use the best local LLMs guide to choose a model for the machine, and Ollama vs LM Studio to compare runners, deployment and work-use licensing. The runner and the model are separate choices.

Which Tasks Belong on Staff Devices?

Local processing is most attractive when the input is already present, the task is bounded and a person can check the output. The cloud remains a useful option when the job exceeds the device or requires a centrally operated service. The final column is an adoption judgment, not a measured quality ranking.

For transcription, separate speech recognition from summarisation. A language model that rewrites text is not automatically an audio transcriber. Google's Basic speech mode uses a traditional on-device speech model; its Advanced mode uses a generative model. The application must then decide whether and where the resulting text is saved. Google's Speech Recognition guide

For cost, compare the paid cloud API call with a free local run that incurs no per-call cloud inference charge, while still accounting for hardware, electricity and administration. Google's local API cost statement

The Limits That Matter More Than a Chip Badge

The first limit is often how much work fits, not whether a model launches. A context window is the model's working space for instructions, source material, conversation history and its answer.

Apple's current guide documents a 4,096-token context window for its default on-device model. Tokens are pieces of text, and the input and output share that budget. The guide also provides a way to query the actual model's capacity. A long document cannot be treated as though the whole window were available just for its source text. Apple's context-window guide

Splitting a document into chunks can help it fit, but your application must preserve relationships across those chunks. If a deadline appears in one section and its exception appears elsewhere, a collection of isolated summaries may miss the relationship. Evaluate the finished task, not merely whether every chunk produced text.

Desktop models also need memory for their weights and working state. Longer context and simultaneous requests can increase memory use. A model that works for a person drafting a reply may be unsuitable for a background batch while other applications are open. Ollama's memory and concurrency guidance

That is why the practical test belongs on representative staff hardware with ordinary applications running. Record completion time, failed requests, correction effort and battery impact for your own workload. None of the hardware thresholds above establishes a universal speed or quality result.

Privacy and Compliance Depend on the Whole App

On-device processing can remove a cloud transfer from a workflow; it cannot certify the workflow's compliance. Ask what the chosen feature sends, stores and exposes, then have the relevant security and compliance owners assess that configuration.

The makers' statements have specific boundaries:

• Apple: its on-device model can work without connectivity, but Foundation Models also supports cloud providers and tools that fetch online data. Record the model and tools selected by the app. Apple's model support
• Google on Android: ML Kit says input data and outputs are not sent to Google. It also sends performance and usage metrics and contacts servers for updates. Local content processing does not mean no network activity. ML Kit privacy terms
• Google in Chrome: Google says local model use sends no data to Google or third parties, with the network needed for the initial model download. That statement does not prevent the website using the API from sending data through its own code. Chrome's model requirements
• Microsoft: Live Captions keeps its processing local, but other features have different boundaries. Paint Cocreator generates images on the NPU while using cloud safety services; Microsoft says it collects prompts and identifiers for abuse prevention. A locally generated result is not proof of a fully local feature. Cocreator's privacy explanation
• Desktop runners: Ollama's local privacy statement applies when running locally. LM Studio's current policy distinguishes local processing from optional cloud services. A cloud provider's promise not to retain a request still means the request left the device. Ollama, LM Studio

Before staff use sensitive material, document the source location, selected model, outbound requests, local history, backups and deletion behavior. Include extensions and connected tools in that review. Device encryption and access control matter because keeping content local also means keeping copies on employee hardware.

What Is Overhyped?

The claim that a laptop can replace a cloud AI service confuses a computation with an operating model. You may move the inference while still needing model updates, support, permissions, evaluation and a way to handle failed tasks.

An NPU is useful for compatible workloads. It is not a general guarantee of model compatibility. An offline answer removes the need for a network round trip during inference; it does not prove the answer arrives sooner. And a model installed locally can still invent details or produce unsuitable output.

The opposite claim is unhelpful too: local models do not need to replace your most capable cloud model to be useful. A dependable draft edit or short summary can justify adoption on its own. The decision is whether that specific feature clears your acceptance bar on the devices people have.

Who Should Put AI on Device Now?

Operators should pilot bounded, reviewable work now; buyers should wait for a demonstrated hardware need. Different roles have different next steps.

For an operator, start with a recurring task such as shortening internal handover notes. Choose inputs with a known expected result and require a person to approve the output. Keep the original source available for comparison.

For a builder or product lead, use the platform API when it matches your deployed devices and feature. Implement model availability, input limits and a manual fallback as part of the product. A mixed phone fleet needs a working experience for people whose devices cannot run the feature.

For a buyer or CTO, inventory existing machines before ordering replacements. Separate the case for a particular Windows NPU feature from the case for a desktop model runner. Approve hardware spending after the chosen workflow demonstrates a useful result on representative machines.

Wait if the intended job is unattended bulk processing on an API that forbids background inference, or if the local model fails your required quality. Keep the current setup if an approved cloud workflow already meets the need and local operation adds no useful privacy, connectivity or operational advantage.

Monday's Move: Pilot One Workflow

1. 1

   Choose the job and its data boundary

   Pick a specific input and output, such as a handover note turned into a shorter draft. Decide whether any of that content may leave the device before configuring a fallback.

2. 2

   Check actual devices and model readiness

   Inventory the relevant operating system, chip, memory and free storage. Download the required model, confirm availability and keep a manual route for unsupported machines.

3. 3

   Evaluate output and network behavior separately

   Use approved sample material. Compare the result with the source, inspect outbound traffic, and repeat the prepared workflow offline. Working offline is useful evidence about that path, but does not prove the connected app never uploads data.

4. 4

   Make the expansion decision

   Record failures, correction effort and support burden alongside completion time. Expand only if the task is useful and its data handling matches the intended boundary. Keep cloud access explicit wherever it is allowed.

For more practical AI adoption decisions, join the newsletter.