AIDBUDDY — Stanley Lin

Platform	Android — model and vector database entirely on-device
Model	Google Gemma 3 (1B parameters) via MediaPipe LLM inference
Base	Built on Google's MediaPipe RAG Android sample
Retrieval	Hierarchical parent-child chunking — 1000-char parents, five overlapping ~250-char children
Design	Dual-mode — deterministic keyword search for medics, RAG chat for Medical Officers
Context	Republic of Singapore Navy Medical Service — demoed to DSTA engineers (MedXplore)

1.0 PROBLEM

Medics and Medical Officers on naval vessels operate without internet connection, so they rely on hardcopy notes to find the right protocol. In an emergency, flipping through a manual is time you don’t have. Cloud-based clinical AI is unusable by definition in this environment — anything useful has to run entirely on the device.

2.0 APPROACH / ARCHITECTURE

The prototype started from Google’s MediaPipe RAG pipeline sample for Android, adapted to the naval medical use case. The language model is Google’s Gemma 3 at one billion parameters; the model and the vector database sit entirely on the phone, with a Retrieval-Augmented Generation pipeline on top. A Medical Officer can ask a question and get an answer drawn only from the approved manuals.

RAG architecture diagram: documents are chunked, embedded, and indexed into a vector store; a user query is embedded, matched against the store, and the retrieved contexts augment the prompt sent to the LLM, which generates the response — Figure 1. Illustration of a RAG architecture. From Oche, Folashade, Ghosal & Biswas, *"A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions"* (arXiv:2507.18910), licensed under CC BY 4.0. Unmodified.

The hard part was chunking the medical documents. Paragraph-level chunking retrieved poorly: long chunks produced embeddings too unspecific to match well, while short chunks didn’t carry enough context for the model to actually use. The fix was hierarchical chunking — a 1000-character paragraph becomes a parent chunk, broken into five overlapping child chunks of roughly 250 characters each. Retrieval matches against the specific child chunks, but the linked parent chunk is what gets fed to the model as context. Precise matching and sufficient context, at the same time.

3.0 THE REDESIGN — LESS AI, ON PURPOSE

A demo to DSTA engineers changed the shape of the product. Their point: even a well-guarded model can still hallucinate — and for medics, who are trained to follow strict protocols with no deviation, a confident wrong answer isn’t a small bug. It’s dangerous.

So the design split in two. For medics, the AI was removed completely and replaced with something deterministic: fast keyword search over protocols broken into single steps. No generation, no room for the system to be creatively wrong. For Medical Officers, who are trained to exercise judgment, the RAG chatbot stayed as a fast way to retrieve information.

4.0 RESULTS

The system operates with no internet connection at any stage — the demo below runs with airplane mode on throughout. On a representative set of clinical decision queries, the RAG pipeline reached roughly 70% baseline accuracy after the move to hierarchical chunking, which produced the single largest retrieval improvement in the project.

Figure 2. Demo — the different operating modes, with airplane mode on throughout (status bar shows no connectivity)

5.0 ERRATA & REVISIONS

The honest lesson from this project: I’d been measuring the wrong thing. I was optimising for how clever the system was, when the real constraint was how much the environment could tolerate being wrong. The hard part wasn’t the model — it was understanding the people using it and the cost of failure for them. The best version of the product was the one where I had the discipline to use less technology, not more.

If continuing this today, I would evaluate against a formally held-out test set rather than iterating informally on a development set — the 70% figure is a baseline from development, not a validated ceiling — and add an offline re-ranking step after retrieval for the Medical Officer mode.