There's a moment I keep coming back to.
I was three days into building a maternal health assistant, and I had just enabled input
moderation — the safety layer that screens what users send before it reaches your agent.
Sensible. Standard. Then I looked at the categories it screens for. self-harm.
violence/graphic.
And I thought about a pregnant woman typing "I have heavy bleeding."
Or "my baby has stopped moving."
If the moderation layer flagged either of those, my agent would never see the message. The symptom screener would never run. Nobody would be told. And from the outside, everything would look like it was working perfectly — no error, no crash, just silence where an escalation should have been.
The safety feature would have been the thing that made it unsafe.
That was the whole project, really. Not the agent. The moment-by-moment work of figuring out where the model was allowed to be involved, and where it absolutely wasn't.
The setup
IDEALIZE 2026, run by AIESEC in University of Moratuwa, had a mini-competition alongside the main ideathon: build something real on Agent Kernel, an open-source agentic framework from Yaala Labs, and submit it to the project repository. It had to address at least one UN Sustainable Development Goal.
I entered solo with five days.
I picked maternal health. Sri Lanka's Public Health Midwife system is genuinely one of the best in South Asia, and it runs on paper — handwritten registers for the midwife, a physical booklet for the mother. Between clinic visits there is no channel. A worrying symptom at 2am means either travelling to a facility or waiting it out. Both sides already use WhatsApp every day. Nothing runs on it.
So: a WhatsApp agent where mothers register, ask when their next clinic visit is, and report symptoms — and where anything matching a danger sign gets escalated to their assigned midwife automatically.
Simple to describe. The hard part started immediately.
The rule I set on day one
The model never makes a clinical judgement.
Not "the model is instructed not to." Not "the prompt says it shouldn't." The model is architecturally incapable of it, because the decisions live somewhere it can't reach.
That sounds like a limitation. It turned out to be the design.
Here's what it actually meant in code.
1. Identity comes from the channel, not the conversation
The obvious way to build this is a tool like get_mother_profile(phone), where the model
passes in whose record to fetch.
Think about what that allows. The model can hallucinate a number. Worse — a user can just type someone else's number and the model will helpfully look it up. Medical records, handed over because someone asked nicely.
Agent Kernel sets the session ID from the WhatsApp sender's phone number. So every tool resolves identity from the session context instead, and no tool accepts a phone number identifying the sender at all. Not as an optional parameter. Not with validation. The parameter doesn't exist.
Identity is bound by the channel. It cannot be asserted in conversation.
The same logic decides who is a midwife. Role is a database lookup against the caseload, not something the model infers from how someone talks about themselves.
2. Severity is decided in Python
When a mother reports a symptom, the model's entire job is to pass her raw text to a function. That function matches against a version-controlled table of danger signs and returns a severity. The model receives that severity and cannot override it, reinterpret it, or run the check again hoping for a different answer.
Every clinical threshold lives in a data file. Not in code, and definitely not in a prompt.
3. Escalation isn't something the model can forget to do
My first design had escalate_to_phm as a tool the model could call when severity came
back red.
Read that again. The safety-critical action depends on the model choosing to call it. And the scenario where it forgets is the scenario where forgetting matters most.
So escalation stopped being a tool. The screening function escalates itself, as a side effect, in the same call that determines severity. There is no model-callable escalation tool. The model cannot fail to escalate, because escalating was never its decision.
4. Fail toward escalation, always
Every ambiguous path resolves the same direction:
- Reference data not yet verified? Treat as red. Escalate.
- Symptom reported but nothing matched? Amber, never green.
- Exception thrown anywhere in matching? Red. Escalate.
- Green only when no symptom was reported at all.
The system is allowed to be annoying. It is not allowed to be quietly reassuring.
Same logic applies to delivery. A midwife's WhatsApp messaging window can be closed, which means escalation can fail to send. So when it fails, the escalation is stored as undelivered and the mother is told, in that same reply, to contact her midwife or nearest hospital directly. Nothing in the system is ever allowed to imply that help is on the way when it isn't.
Silent failure would have looked exactly like success.
5. The safety layer that almost broke safety
Back to that moment at the start.
The fix was to narrow input moderation to categories that cannot plausibly fire on a symptom report, and write down why. It's a smaller net. I bought that deliberately, in exchange for never silencing a mother describing what's happening to her.
I also had to disable the built-in PII detection, because it flags phone numbers — and a mother legitimately types her midwife's number during registration. A PII filter that breaks the escalation path in the name of protecting privacy is not protecting anything. So PII redaction happens in my logging layer instead, and only there.
There's a general lesson in both. The question isn't does this safety feature work. It's what does it break when it fires on the wrong thing.
The data problem, which I did not solve
Here's the part I'm least comfortable with, so I'll be direct about it.
A system like this needs three sets of real clinical reference data: the antenatal visit schedule, the childhood immunisation schedule, and the danger-sign table. All of it exists — Sri Lanka's Family Health Bureau has a public document library, the Epidemiology Unit publishes the immunisation schedule, and every parent holds the physical Child Health Development Record.
With one day left, I couldn't verify all of it to the standard the rest of the system demanded. And at one point I nearly took a nicely formatted schedule from an AI chat, citations and all — until I read the citations. One of the cited documents openly says it doesn't contain the schedule. The others were a nutrition strategy and a performance report. It looked sourced. It wasn't.
Which is a slightly embarrassing thing to nearly do, in a project whose entire argument is that it doesn't fabricate clinical data.
So I built provenance into the data instead. Every reference file carries a header — the
source, publisher, document date, URL, retrieval date, and whether a clinician has
reviewed it (none has). Every file carries a status. And that status propagates all the
way through the tools into what the mother is actually told. When the data isn't
verified, the agent says so, in the reply, and points her to her midwife rather than
reading out dates it can't stand behind.
An agent that knows its own data isn't trustworthy, and says so instead of guessing, is more useful than one that sounds confident. That ended up being my favourite thing in the build.
Write the limitations down
My README has nine known limitations. Narrowed moderation, disabled PII detection, free-text midwife assignment, unverified reference data, no clinician review, in-memory sessions, and more.
Nine looks like a lot. I was tempted to trim it. I'm glad I didn't, because every one of them is a real constraint I hit and reasoned about — and a reader finding a limitation you already named reads completely differently from one finding a limitation you hid.
The limitations list is the most credible section in the whole document.
What I'd tell myself on day one
Constraints made the system better, not smaller. Every time I took a decision away from the model, the design got clearer and the failure modes got easier to reason about.
Ask what happens when the safety feature is wrong. Not whether it works — what it breaks when it fires on the wrong thing.
"Sourced" and "looks sourced" are different. Read what the citations actually say.
Fail loudly. In a system where nobody is watching the logs, a silent failure is indistinguishable from success, and that's the worst bug you can ship.
There's a second story from this build — I hit a bug that turned out to be in the framework's own documentation, and fixing it became my first open-source contribution. That one's its own post.
Built for the IDEALIZE 2026 mini-competition on Agent Kernel. It is a prototype, not a clinical tool, and no clinical value in it has been reviewed by a clinician.