I've been away for a couple of months. Not because I went to the woods, a retreat, or a digital detox—though sometimes it sounds tempting😌.

I just finally dove into what I've long wanted to explore deeper: building new AI products, understanding the architecture of language models, and periodically feeling like someone who opened a closet and all the boxes fell on them at once.

And the other day I saw a post: UOM—the university where I studied nutrition and which I trust a lot—officially launched the program "Neural Networks for Helping Professionals".

And I was genuinely happy. Because what I've been doing for a year has stopped being a strange hobby of someone who talks too much to ChatGPT and has become a recognized direction even in serious educational projects.

This is no longer "someday later." It's happening now.
Meanwhile, Perplexity and Alisa already offer analysis interpretation and recommendations, and influencers are vying to advise: "Collect your medical data and give it to AI."

Sounds modern. Convenient. Almost as if the future has finally brought slippers and tea☕️.

And in a way, it has.
But every time I hear such advice, a little alarm light goes off inside me. Because I know what's usually not visible from the outside.

There's this thing called language model hallucinations.
Sounds almost romantic, but in reality it's more prosaic: the model can confidently state something that wasn't in your data.
Not because it's malicious, cunning, or decided to ruin your evening. But because it's designed that way: it generates a statistically probable answer, not a verified fact from a safe labeled "Truth."

And in a medical context, this is no longer just a funny bug.

In one study of medical case summaries, the hallucination rate without special mitigation measures reached up to 64.1%. And the MedHallu benchmark on 10,000 questions from medical literature showed that even strong models, including GPT-4o and Llama-3.1, struggle to recognize complex medical hallucinations: the best result in the hardest category was F1-score 0.625.

And no, this doesn't mean AI should be wrapped in foil, put in a drawer, and never taken out.
I use it constantly myself. My trusted NutriAgent reduced analyzing a client case from several days to several hours.
It really works great where the task is algorithmic: it summarizes indicators, compares them with optimal values, highlights deviations, helps build hypotheses. In seconds, it does what used to take an hour of manual work, and it doesn't forget to look where the human eye, after the third table, no longer wants to look.

But.
Sometimes you catch it red-handed. It confidently describes details that weren't in the source materials.
That is, it found the main thing. And then decided to write a little novel✍🏻.

And here's the real problem with AI in health: not that it makes mistakes. But that you don't always understand where exactly it started making things up.
When data is insufficient or ambiguous, the model doesn't always say: "I don't know." Sometimes it fills in the picture with what "usually happens" in similar cases. Confidently, calmly, with the look of an honor student at the blackboard.

And without professional context, it's almost impossible to distinguish such an answer from a conclusion based on your specific indicators.

🤫Yesterday I talked to a craftsman about the casing for a bathhouse.
Casing is a special structure in the opening, without which a wooden log house can crush doors and windows during shrinkage. So it's an important thing. Especially if you, like me, thought before the conversation that it was just a word from the category "something funny."

At some point, I realized I didn't understand almost anything from the craftsman's explanations. I honestly admitted it.
He thought and said:
"Well, either you'll have to study everything yourself, or trust someone who knows."

And I was like: there it is. The whole conversation about AI, tests, and health in one phrase.
Either you dive deep enough to see the errors, or you turn to someone who has already walked that path.

AI is not on that list yet.
It's not a doctor. Not a nutritionist. Not a specialist who bears responsibility for conclusions.
It's a tool in the hands of an expert.
Very powerful, fast, useful. But still a tool.
Kind of like a screwdriver: a great thing until you ask it to build a bathhouse on its own, calculate log shrinkage, and pick out windows for you.

Have you already tried uploading your tests or symptoms to a neural network? If so, how much do you trust what you got?

@nastroi_zdorovie 🎧