The AI Was Trained on People Who Aren’t Me

A friend told me her new symptom checker kept missing things that were obvious to her. I wasn’t surprised. I asked her what she thought “clean data” actually meant to the people who built it. She hadn’t thought about it that way before, and honestly, most people haven’t.

Clean data usually means the mess has been taken out. Missing entries deleted, outliers dropped, anything that doesn’t fit the pattern smoothed over so the model performs better. In medicine, a lot of that mess is disabled people. Rare presentations, atypical symptoms, bodies that don’t behave the way the textbook says they should. Clean that out and you haven’t removed noise, you’ve removed us.

I’ve had specialists look at my chart and hesitate, because what’s in front of them doesn’t match what they were trained to expect from a person my age with my history. Now picture that same hesitation baked into software, trained on a data set that already decided people like me were outliers before I ever walked into the room.

Nobody sat down and decided to leave disabled people out of medical AI on purpose. It happened quietly, as a technical decision made by someone who never had to live with what it cost.

Has a system, medical or otherwise, ever struggled to place you because you didn’t fit its idea of normal?

If you’ve ever had to explain your own body to a system that wasn’t built for it, Self Advocacy Unleashed gives you the language to do it with confidence. Join the waitlist, link in bio.

Madeleine x