The first mental health tools to use AI were looking for red flags -- specific words that signaled crisis. "Suicide." "Hopeless." "Can't go on." These keyword systems were easy to build and easy to explain. They were also inadequate for the actual problem of recognizing urgency in how people describe distress.
Real urgency doesn't announce itself. The person who types "I'm fine, just tired" and the person who genuinely is fine and tired often sound identical to a keyword scanner. The difference is in what surrounds those words: the frequency of check-ins over time, the shift in how someone describes their day compared to three weeks ago, the specific way someone qualifies a feeling before immediately walking it back.
What keyword flags miss
Keyword detection fails in two directions. It misses urgency that expresses itself in ordinary language, and it over-flags expressions that are idiomatic rather than clinical. "I could kill for a coffee right now" is not the same signal as describing feeling trapped in a situation with no exit. A system that treats both as equivalent isn't doing triage. It's generating noise.
Clinical urgency has texture. It appears in how someone answers, not just what they answer. Does the language become flatter over successive check-ins? Does someone who normally elaborates suddenly become monosyllabic? Is there a gap between reported mood and the emotional register of the writing itself? These are the signals that matter, and none of them are capturable by a word list.
The false-negative problem is the more dangerous failure mode. A system that misses urgency has worse consequences than a system that over-flags. But over-flagging has its own cost: if the system generates too many escalations, the clinical resources that respond to escalations get stretched thin, and the ones that actually matter get lost in the queue. Getting false negatives down requires understanding the language of minimization, which is far more common than the language of explicit crisis.
The language of minimization
People in distress rarely describe their situation in terms of its full severity. There are social costs to disclosure, especially through an employer-adjacent platform, and most people have a strong instinct to present their situation as more manageable than it is. This is especially true in the first session, before someone has decided whether this is a system worth being honest with.
The language of minimization has specific features. "It's not that bad" and "I'm probably overreacting" are almost never said by someone who genuinely believes those things. They're said by someone who is providing cover for disclosing something that feels risky to disclose. A system calibrated to treat minimizing language as a signal of mild presentations will systematically under-detect severity. A system that treats it as a signal of concern, potentially warranting a follow-up question, will catch more of what matters.
What the model needs to learn
Building an AI system that can recognize urgency requires training on the kind of language that doesn't look urgent on the surface. That's a different dataset problem than most NLP tasks. You're not looking for the presence of specific tokens. You're looking for the relationship between what's present, what's absent, and how that pattern has shifted.
Lullgrove's triage model was built with clinical psychologists reviewing output at every stage -- not to label data, but to interrogate the model's reasoning. When it flagged something, was the reasoning coherent? When it didn't flag something, did the clinical reviewer agree? The goal was a model that could explain its assessment in terms a clinician would recognize, even when the underlying computation wasn't transparent in the usual sense.
What became clear was that the most reliable signals weren't in individual responses. They were in patterns: the shape of how someone described the same domain (work, sleep, relationships) over multiple sessions; the specific hedging language that precedes a difficult disclosure; the way that someone who's been minimizing something for weeks finally stops minimizing it. Language as trajectory, not language as snapshot.
That's what we mean when we say AI can learn to hear urgency. Not that it listens for the word. That it learns to recognize what urgency sounds like in the register of ordinary, qualified, deflected self-disclosure -- which is almost always how it actually shows up. The clinical sophistication isn't in flagging the obvious cases. It's in surfacing the ones that don't look like cases at all until you see the pattern.
What this requires from the product
A model trained to recognize urgency through pattern and context rather than keywords requires more sustained engagement from the user. A single check-in doesn't produce a trajectory. Two or three do. That creates a design tension: the product needs enough engagement to build a picture, but the check-in experience needs to be low-friction enough that employees come back, especially the ones who are struggling and for whom the effort of engagement is already higher than it would be otherwise.
Resolving that tension means keeping individual check-ins brief and open-ended rather than comprehensive. The goal of any single session isn't to capture everything -- it's to capture something honest, and to do so in a way that makes the next check-in slightly easier because the system has started building context. Over time, the cumulative picture from a series of short honest check-ins is more clinically useful than a single long structured assessment. That's a different design philosophy than most mental health products operate from, and it reflects a different understanding of where the signal actually lives.
The practical consequence for urgency detection is that the model gets better over time. The first check-in for a new user is a weak signal. The third, in combination with the first two, is a much stronger one. A triage system designed around trajectory rather than snapshot catches the person whose urgency has been building for weeks before it reaches the point where they would use a keyword that a simpler system would flag. That's the difference that matters for outcomes -- getting to someone earlier, with information that supports a considered routing decision rather than a reactive one.