Ishtiaque Ahmed (photo: supplied)
In Bangladesh, a community mental health worker might sit with a client who cannot sleep, cannot stop worrying and describe the feeling with the Bangla word, manoshik chap, which translates as “tension.”
The word carries many meanings — economic, social and spiritual — and almost none fit into the categories that today's most powerful artificial intelligence systems were trained to recognize. A large language model that flattens all these meanings into “anxiety” or “depression” loses nuance and can misinterpret a situation entirely.
A new Department of Computer Science research project is trying to fill that gap between how the world's leading AI models understand distress and how people actually experience it.
“In mental health, the stakes are higher because what is being translated is a person's own account of their suffering,” says Syed Ishtiaque Ahmed, an associate professor in the Department of Computer Science and a Schwartz Reisman Institute faculty affiliate.
Ahmed and his team were awarded a $1.8 million grant by the Wellcome Trust to build and test a culturally aligned generative AI assistant for para-counselors — a type of frontline community health worker — who are often the only source of mental health support in low-resource settings throughout Bangladesh.
The Wellcome Trust is a global charitable foundation that funds research around the world to improve health and advance equitable solutions to urgent health challenges. Through its Mental Health program, Wellcome supports research aimed at improving understanding of anxiety, depression and psychosis and developing more effective ways to intervene early.
During the two-year project, Ahmed's Third Space research group at U of T will work with two Bangladeshi partners — the non-profit SAJIDA Foundation and the for-profit social enterprise mPower Social Enterprises Limited — and a network of people with lived experience managing anxiety, depression and psychosis.
The counselors will use the U of T team’s AI assistant to prepare for sessions, make sense of what they heard from patients and think through next steps. The AI system never speaks directly to patients, and all its suggestions are reviewed, edited or discarded by the counselor with the goal of strengthening their judgment.
It is poised to become one of the first AI mental health interventions designed specifically for communities in the Global South.
“Cultural alignment should not be a finishing touch,” Ahmed says. “It’s the whole problem we are trying to solve. I have spent 15 years watching technologies designed elsewhere arrive in Bangladesh assuming the local context is a rounding error. It never is.”
In Bangladesh, there is roughly one therapist for every 85,000 people.
“Mental health is often neglected across the Global South, and tools that do exist to help alleviate the crisis don’t always work outside of the Western bubble where they were created,” Ahmed says.
Ahmed and his team believe AI models have the potential to help, but only when implemented in a safe, conscientious manner that emphasizes the humans involved in the process.
“A person in distress opens an app and talks directly to a model that has no supervision, no accountability and no idea who is on the other end,” Ahmed says. “We would never accept that from a human counselor. We should not accept it from a machine either.”
Ahmed’s group approaches the problem differently. The AI model helps para-counselors prepare for sessions, make sense of what they have heard and think through next steps. The goal is to strengthen human judgment, not replace it.
He adds: “The counselor keeps the relationship, the accountability and the final word. You get the benefits of the technology without outsourcing the conscience.”
The project treats ethics and data sovereignty as central design questions. People with lived experience are involved throughout as co-designers, reviewers and safety monitors, and are fairly compensated for their expertise.
The underlying models and methods of the project will be released openly so they can be adapted by other communities.
“I would rather build something that a community can trust and control than something that merely happens to them,” Ahmed says.
Ahmed is supported by a team of graduate and undergraduate students who will gain crucially important hands-on training in building culturally grounded machine learning and language systems in the real world.
Arid Hasan is a PhD student in computer science, supervised by Ahmed. For seven months, he’s been leading the research team on the project.
“It’s been interesting and challenging at the same time,” says Hasan, who’s been working on Bangla natural language processing since 2018. “You have to provide very specialized interventions when you’re working with people who are going through different mental and physical health issues.”
Over the next two years, Ahmed and his team will assemble an ethically sourced body of data, build and test the AI, co-design the tool with the counselors who will use it and run a six-month field pilot across five regions of Bangladesh.
If successful, the results will travel well beyond Bangladesh, to other languages and places such as West Bengal, Nepal and East Africa.
“If we get this right, the contribution is not only a tool,” Ahmed says. “It’s a demonstration that the Global South can be where responsible AI is built, not just where it is deployed after the fact.”
Original story by Coby Zucker for A&S News.