Local AI — for individuals and organizations

Local AI for individuals

Local AI for individuals — I tried it out.

It works. The results are perfectly fine. Watch the video, where I compare Claude with Qwen.

Until now, this wasn't something for the average Joe. Without engineering skills, I wouldn't have been able to pull this off. But the landscape is shifting — Nvidia and Microsoft have announced RTX Spark, a PC platform designed to run AI agents locally. The first laptops and mini desktops from Dell, HP, Lenovo, ASUS, MSI, and Microsoft Surface are arriving in fall 2026.

Here is my architecture setup and a few screenshots.

I use a MacBook, Qwen as the LLM, and the Hermes agent (https://hermes-agent.nousresearch.com/) from Nous Research as the harness.

You can absolutely run local AI with a local AI agent as an individual. But the barrier is high.

The hardware is expensive, and installing and configuring it is not for people who get overwhelmed by terminal commands.

That will certainly change in the future. At some point there may be hardware with local AI preinstalled.

In my view, local AI is worth it for individuals in three cases:

• New hardware is due anyway — if you can afford it and you were planning to buy a MacBook or a new hardware regardless.
• Privacy matters a lot to you — if you care deeply about your data never leaving the house.
• Teenagers in the house — if young people should learn how to handle technology.

I am a mother myself, and I believe teenagers should learn to work with technology — in a protected space. With local AI, kids are not tracked during their experiments and no profiles are built about them. Anyone who can afford the hardware should consider it.

Local LLMs for SMBs and small organizations

Between personal use and enterprise deployment, there is an important middle ground: small and medium-sized businesses (SMBs).

These include, for example:

• law firms
• tax advisors
• medical practices
• engineering offices
• consultancies
• agencies
• smaller industrial companies

Different criteria apply here than at large companies.

An SMB usually does not have enough AI usage to economically justify its own GPU server. Token volumes are typically far below the scale at which self-hosting pays off on cost grounds.

Case 1: SMBs without special data protection requirements

When there are no special requirements, a hosted API is often the best solution.

Examples:

• writing marketing copy
• summarizing internal documents
• developing ideas
• general research
• creating presentations

Advantages:

• no hardware of your own required
• access to the best available models
• no technical operations required
• low entry costs

For most small companies, this is the most economically sensible approach.

Case 2: SMBs with high data protection requirements

In certain industries, the decision changes.

Examples:

• law firms with client data
• tax advisors with financial data
• physicians with patient data
• companies with confidential development data

Here, local AI can make sense even if it is not economically cheaper.

The motivation is then not: "We save costs." — but: "Sensitive data should not leave the company."

Local desktop LLM

Every employee runs their own model on their workstation.

Examples:

• MacBook Pro
• Mac Studio
• a powerful Windows PC

Suitable for:

• document drafts
• summaries
• internal knowledge work
• analyzing your own files

Advantages:

• maximum data control
• no central infrastructure required
• no GPU server needed

Disadvantages:

• every device has to be maintained
• smaller models than with cloud providers
• setup and updates have to be organized

Small central AI server

A company can also operate its own AI machine.

This becomes interesting when:

• several employees should use the same models,
• a shared body of knowledge is needed,
• internal documents should be searchable centrally,
• automated workflows emerge.

Examples:

• internal document search
• knowledge base
• contract analysis
• automated document processing

This is where the transition to the classic enterprise scenario begins.

The actual decision for SMBs

Local AI does not automatically mean your own server

At SMBs, "local AI" is often misunderstood.

There is a big difference between:

Local AI on the workstation

• MacBook
• Mac Studio
• local machine

Self-hosting a company AI

• GPU server
• infrastructure
• monitoring
• updates
• MLOps

The first variant can already make sense for small teams. The second only becomes relevant for larger organizations or special requirements.

Local LLMs for enterprises

For enterprises, the questions are entirely different. Large organizations have use cases with far higher volumes.

In general, there are the following options:

The three operating models

1. Hosted API

An external provider (e.g. OpenAI or Anthropic) runs the infrastructure. The user pays per processed token.

Advantages

• No hardware required
• Ready to use quickly
• Access to the most capable ("frontier") models
• You only pay for actual usage

Disadvantages

• Data leaves your organization
• Costs scale proportionally with usage
• Dependence on the provider's pricing and limits

Suitable for:

• small to medium usage
• fluctuating load
• applications with the highest quality requirements.

2. Managed Deployment

The hardware is rented and managed by the provider, but runs inside your own cloud environment.

Advantages

• Data stays inside your own cloud tenant
• No in-house MLOps team required
• Easy switch thanks to API compatibility
• Fine-tuning often integrated

Disadvantages

• Ongoing rental costs
• Less control
• Only open-weight models available

Suitable for:

• regulated industries (health, finance, legal)
• companies with high data protection requirements.

3. Full Self-Hosting

The organization operates models, hardware, and infrastructure entirely on its own.

Advantages

• Maximum data sovereignty
• Very low cost per request at high utilization
• Own fine-tuning options
• One infrastructure can serve several applications

Disadvantages

• High capital expenditure
• Power consumption
• Complex operations
• Requires specialized MLOps staff
• Powerful hardware necessary

Suitable for:

• sustained high usage
• sensitive data
• regular fine-tuning.

Economic considerations

The article names three important thresholds:

An often underestimated cost factor is personnel: an MLOps engineer causes significantly higher annual costs than the actual GPU hardware.

Decision logic

The author recommends the following order for decision-making:

1. Check data protection
   • Must data absolutely not leave the company?
   • If yes: API is off the table.
2. Assess available expertise
   • No operations team → managed deployment.
   • Own MLOps team → self-hosting is possible.
3. Analyze usage volume
   • Under approx. 2M tokens/day → API is usually cheaper.
   • Above that → self-hosting is economically interesting.
4. Consider required model quality
   • For the most advanced closed models, often only an API remains usable.

The hybrid architecture

The article sees a hybrid solution as the most practical approach for many companies:

Local models handle:

• document processing
• classification
• extraction of structured data
• privacy-critical applications

External APIs handle:

• complex reasoning tasks
• rare, particularly high-quality requests

According to the article, companies report 40–70% lower costs compared to a pure API solution. At the same time, access to powerful frontier models is preserved.