The UN Made AI-Generated Refugees

I am talking to Amina, an AI avatar that plays the role of a woman living in a refugee camp in Chad after she escaped violence in Sudan. “I’m hanging in there, yeah,” Anima tells me. “ It’s tough here in the camp, but we’re staying strong.” I ask her how she came to be in a refugee camp. “I fled my hometown of Al Junaynah in Sudan back in June 2023, due to the violence, yeah?” It says. “It was a terrifying journey to escape the conflict. Now I’m trying to keep my kids safe and find some stability here in the camp. It’s not easy but we have to keep going, right?”

0:00

/2:27

1×

Amina is an experiment, part of a pair of AI avatars created by the United Nations University Center for Policy Research (UNU-CPR), a research institution connected to the United Nations. It’s one that is sure to be controversial, considering that the UN itself says a major problem facing refugees is their dehumanization for political gain or convenience. The UNU-CPR project is using an inhuman technology in an attempt to help people learn more about what they are facing. The group also tested a soldier persona called “Abdalla,” which “simulates the behavior and decision-making patterns of an actual combatant, offering negotiators and mediators a possible tool to train for future high-stakes negotiations.”

“The idea was to have independent academic research available to the UN,” Eleanore Fournier-Tombs, a data scientist that leads a research lab at UNU-CPR focusing on AI policy, told 404 Media.

Ask Amina and Ask Abdalla began as a project in an AI for Conflict Prevention class taught by Eduardo Albrecht, a professor at Columbia and a senior fellow at the UNU-CPR. According to Albrecht, he and his students designed the avatars after a classroom discussion about using AI in humanitarian work.

In early tests at a workshop attended by humanitarian organizations, refugee aid groups, and nonprofits, Albrecht and Fournier-Tombs said the reactions were strong and that many were negative. “Why would we want to present refugees as AI creations when there are millions of refugees who can tell their stories as real human beings?” one person said, according to a writeup of the workshop. Another participant worried about “reinforcing biases if interactions with refugees are done with AI agents rather than actual humans,” and that the AI might “sanitize or downplay real human suffering.”

Right now, anyone can test Anima or Abdalla and talk with them for three minutes at a time. “The first is designed to create an accurate digital representation of a refugee living in a camp in Chad. The second creates a digital replica of a combatant leader in the Rapid Support Forces (RSF), a group active in the southeastern part of Sudan from which many refugees are fleeing,” a write up of the project says.

The RSF began as groups of loose-knit militias in the early 2000s before becoming a more formal military group. It’s been conducting ethnic cleansing in Darfur for years now. I asked Abadalla if the RSF is committing genocide, and it gave a generally correct, though vague answer, and not one that you would expect from the group doing the killing.

“The RSF has faced serious allegations of committing atrocities and acts of genocide, particularly in Darfur, including systematic killings, mass rapes, and forced displacement of specific ethnic groups,” Abdalla tells me about the group it’s meant to represent. “These acts reflect a pattern of violence aimed at terrorizing civilians, contributing to the ongoing humanitarian crisis in the region.”

When I spoke with them, both Albrecht and Fournier-Tombs were clear that the goal of the workshop was to spark conversation and deal with the technology now, as it is. 

“We’re not proposing these as solutions for the UN, much less UNHCR (United Nations High Commissioner for Refugees). We’re just playing around with the concept,” Albrecht said. “You have to go on a date with someone to know you don’t like ‘em.”

0:00

/2:59

1×

Fournier-Tombs said that it’s important for the UN to get a handle on AI and start working through the ethical problems with it. “There’s a lot of pressure everywhere, not just at the UN, to adopt AI systems to become more efficient and do more with less,” she said. “The promise of AI is always that it can save money and help us accomplish the mission…there’s a lot of tricky ethical concerns with that.”

She also said that the UN can’t afford to be reactive when it comes to new technology. “Someone’s going to deploy AI agents in a humanitarian context, and it’s going to be with a company, and there won’t be any real principles or thought, consideration, of what should be done,” she said. “That’s the context we presented the conversation in.”

Albrecht detailed how he and his students constructed Amina and Abdalla in a paper published by the UNU-CPR titled Does the United Nations Need Agents? “Both the Amina and Abdalla avatars were created using HeyGen,” the paper explains. “HeyGen relies on OpenAI’s [GPT-4o mini] to animate the video avatars, and allows for linking via [Retrieval-Augmented Generation] to an external database where the knowledge bases curated by the anthropologist agent are uploaded.”

One of Albrecht’s concerns was accuracy. So he tested it. “This study evaluated Amina’s representativeness using 20 questions drawn from four distinct surveys, none of which were included in her knowledge base: the SENS Nutritional Survey (4 questions), Post-distribution Monitoring Report of Food Assistance in Refugee Camps (3 questions), Norwegian Refugee Council’s ‘War in Sudan’ (8 questions) and UNHCR’s ‘Sudanese Emergency’ (5 questions). 40 Analysis of Amina’s responses revealed that she correctly answered 16 out of 20 questions, achieving an 80 percent accuracy rate.”

“Talking” to Amina and Abdalla is a surreal experience, one anyone can experience by going to the website which Albrecht said will be live for a month after the publication of this article. Their responses feel generic and stilted, as if they were trained on UN reports about the conflict and not interviews with actual refugees. The paper admits that this is a massive limitation of the agents.

“It is impossible to know what information is or is not included in the training data of the LLM since commercial providers do not fully disclose the specific datasets used,” the paper says. “This represents a limitation in the experiment design which should be explored further.”

The paper also speculated how agents like these might one day be used in humanitarian work. “If Amina works, ‘her’ rapid responses could be of great value,” it says. “For example, they could be used to quickly make a case to donors (often in very different locations and with very little time) on what population needs to be prioritized when earmarking aid to the region. If Abdalla works, ‘his’ responses could help negotiators and mediators prepare for more targeted real-world engagement.”

Again, people who attended the UNU-CPR workshop and interacted with Amina pushed back against the idea that AI avatars should be used to communicate with donors. “Participants noted that refugees ‘are very capable of speaking for themselves in real life,’” the paper said.

Albrecht knows that AI systems, especially LLMs, are encoded with the biases of their creators. “Let’s say an NGO is conducting a needs assessment, in part, utilizing these agentic systems. What kind of knowledge would that target population know about how such a system is used? How are they informed? Most importantly, do they have the power to reject or accept the use of these tools and their outcomes?” he said. “Because if you’re making decisions towards a population based, in part, on the outcome of these agentic systems…you’re very directly cutting out the agency of that population you are purporting to help.”

The goal of the experiment, Albrecht said, was always to provoke an emotional reaction and start a conversation about these ethical concerns. 

“You create a kind of straw man to see how people attack it and understand its vulnerabilities.”