Introduction
Most AI assistants share the same shape: a chat box backed by a model in someone else's cloud. That's fine for drafting an email — and a real problem the moment the work involves regulated data, proprietary source code, or a customer's private records that contractually cannot leave the building.
@RitS, we built a different shape: a local-first AI coworker — an agent that runs on your own machine, uses your real files, terminal, and 25+ connectors, and returns finished deliverables. Critically, it's provider-agnostic: the same agent can call a frontier cloud model when that's the right tool, or route every token to a self-hosted model when the data can't leave your infrastructure. This post walks through the architecture that makes that possible.
Local-First, Not Cloud-Only
'Local-first' means the agent, its session state, and its access to your tools all live on your machine — not on a remote server you're renting time on. The model doing the reasoning is a pluggable choice, not a fixed dependency.
That single design decision changes the risk calculus. Your files are read locally. Your terminal runs locally. And when you point the agent at a self-hosted model, the entire loop — prompt, context, tool calls, and response — stays inside your own network perimeter.
Provider-Agnostic by Design
The heart of the system is a provider router. Every model has an identifier, and the router dispatches based on its prefix: a bare id like a frontier chat model goes to the cloud provider; a prefixed id routes to a self-hosted, OpenAI-compatible endpoint. Switching a workload from a public cloud model to a private on-premises one is a configuration choice, not a rewrite.
- Cloud models (OpenAI, Anthropic, Google, and others) for the general case.
- Self-hosted models via an OpenAI-compatible local endpoint for sensitive or offline work.
- One agent, one set of connectors, one interface — the model underneath is swappable per task.
- No pipeline is welded to a single vendor, so switching providers never means rebuilding the app.
Self-Hosted Models: When Data Can't Leave
For work that can't touch a public API, the coworker connects to a self-hosted model host — a private inference server running open models on your own hardware (the GPU in the picture above, not a rented cloud instance). Curated, capability-checked models are exposed in the picker; embedding and reranker models are kept out of it so users only ever pick something that can actually hold a conversation.
A lightweight heartbeat polls the host every 30 seconds and surfaces a live status indicator — how many models are present, how many are actually usable — so the person using the agent always knows whether their private model backend is online before they rely on it. The design is deliberately defensive: a partial or slow response from the host degrades the indicator, it doesn't crash the app.
One Codebase, Three Surfaces
The same application renders as a native macOS desktop app, an iOS build, and a browser dev environment — one React single-page app behind a native shell. On the desktop, the shell supervises a bundled local backend on a private port; on mobile it points at a reachable host. The user-facing experience is identical; only how the app reaches its backend differs per surface.
Security: Token-Scoped and Zero-Trust by Default
Local-first is not the same as unguarded. Every request to the backend is authenticated against a per-launch token — the desktop shell mints a fresh token each time it starts and injects it into both the backend and the UI, so nothing on the machine can talk to the agent without it. Public health checks and OAuth callbacks are the only tokenless paths.
This is the same zero-trust, least-privilege posture we apply to any production system, brought down to a single-user desktop app: verify every call, scope every credential, and assume nothing is trusted just because it's running on the same machine.
Built and Proven the @RitS Way
The whole thing is delivered under the same discipline we apply everywhere: a hermetic test suite where every backend request and event stream is mocked, plus a set of acceptance-test-driven Given/When/Then scenarios that verify the real behaviors — branding, the self-hosted-model heartbeat online and offline, the responsive mobile layout, and the command palette — as executable checks rather than assertions.
Nothing ships as 'done' because someone said so; it ships because a failing acceptance test became a passing one.
How @RitS Applies This
A local-first, provider-agnostic coworker is the natural endpoint of several themes we build on across our work: Agentic AI that plans and acts, zero-trust and least-privilege security at every boundary, and an open-source, self-hostable stack that keeps clients free of vendor lock-in. For organizations in regulated industries — banking, public sector, healthcare — the ability to run a capable AI agent entirely on owned infrastructure isn't a nice-to-have; it's the difference between adopting AI and being unable to.
@RitS, we help clients stand up exactly this kind of capability: agents that are useful enough to reach for daily, and private enough to trust with the work that matters most.
Want to explore what this could do for your business?
Talk to us