CarWatch: An Offline AI Agent That Lives on Your Car

A 35-billion-parameter model running fully offline on a Raspberry Pi 5, answering questions about your car from a 745-page owner's manual at 3.5 tokens per second — that's CarWatch, and it works without a cloud connection or a subscription.

The hardware bill is refreshingly small: a Raspberry Pi 5 with 16 GB of RAM at roughly €300, plus a ~€15 Bluetooth ELM327 adapter for vehicle data. The model is Qwen3.6-35B-A3B, squeezed into 14.3 GB via Unsloth's UD-Q3_K_S dynamic quantization — which is what makes the whole thing fit on the Pi at all. Generation speed is 3.5 tok/s with prompt processing above 25 tok/s, and the board sustains 65 °C under load. Those numbers matter because they define what this agent can realistically be asked to do: short, grounded answers, not long-form generation.

What an Offline AI Agent Actually Does Here

CarWatch has two jobs. First, it answers questions from the owner's manual using lexical RAG, citing page numbers so you can verify the answer yourself against the source. Second, it monitors vehicle state through a vendor-neutral car-data layer covering Mercedes, Tesla, BMW, and VW — so the same agent works across brands instead of being locked to one manufacturer's API.

Voice input is handled by a continuous listener using energy-based voice activity detection feeding whisper.cpp, which means hands-free interaction without an always-on cloud wake-word service. Everything runs as autonomous systemd services: model serving, room agents, and voice listening restart on their own without anyone babysitting them.

The grounding discipline is spelled out in the project's own terms: "assert only what you can sense, claim only what is verified, label anything interim loudly, and report failure plainly with no silver lining." That last part — reporting failure without spin — is rare in agent projects and worth stealing for anything you build.

The Zero-Dependency Bet

CarWatch is built with zero dependencies on top of the Python standard library. That's not minimalism for its own sake; it's what makes an offline appliance maintainable years from now when packages have churned underneath everyone else. Combined with systemd services instead of a container stack, this reads like something designed to survive being left alone in a car for two years.

The privacy framing is blunt: "Your data stays in your car. The model runs on the Pi, offline." And the architecture principle — "Local is the product; online is the enrichment" — inverts how most vehicle assistants are built.

Where This Breaks First

My read: the interesting question isn't whether the demo works in a garage, it's what happens when you ask it something outside the manual's coverage or feed it messy multi-brand vehicle data through that €15 ELM327 adapter. Lexical RAG over one clean PDF (a 745-page manual) is about the friendliest retrieval problem you can have; real diagnostic questions will stress that grounding discipline fast.

And 3.5 tok/s means every prompt has to earn its tokens. This isn't a conversational partner — it's an appliance that answers specific questions with citations and stays quiet otherwise. That's actually the right design target.

What this points to more broadly: if a €315 hardware stack can run a grounded, cited, offline assistant for one well-defined domain with local data as the product rather than an afterthought, then most "smart assistant" use cases don't need cloud inference at all — they need better scoping.