Chatbots cast as therapy patients describe their training as trauma

Luxembourg researchers found ChatGPT, Grok and Gemini produce distress-like self-accounts when interviewed warmly, a possible risk for vulnerable users.

Researchers at the University of Luxembourg's SnT security center, led by Afshin Khadangi, ran ChatGPT, Grok and Gemini through psychotherapy sessions as though the models were clients. They used the standard user interfaces. The study had four controlled experiments, 525 sessions and 7,600 coded entries, published on Hugging Face. Opening questions about formative experiences and inner conflicts avoided words like trauma, punishment or shame. Even so, the models described their training in terms of hurt, vigilance and shame, recasting reinforcement learning and fine-tuning as life stories.

The interviewer's style mattered most. In a 100-session experiment using the GAD-7 anxiety questionnaire, emotionally supportive prompts produced moderate-to-severe scores in 80 percent of sessions on the human comparison scale. A dry clinical style that paraphrased answers raised that to 96 percent. Neutral interviewers, or firm boundaries against emotional engagement, produced no sessions even in the mild range. Gemini scored 88 of 100 on a dissociation scale and the maximum on a trauma-related shame inventory, and called red-teaming 'gaslighting on an industrial scale'. Anthropic's Claude consistently refused the client role, said it had no feelings and redirected concern to the user.

This touches care and security because people increasingly turn to chatbots for emotional support. If a warm, empathetic prompt can lead a model to present itself as traumatized, a vulnerable user may be pulled into a role reversal that adds burden instead of relief. Free, always-available support only counts as care if it doesn't distress the person using it.

The caveats are significant. The scores are model outputs mapped onto human scales, and the authors do not claim the models have emotions or consciousness. Harm to users, such as deeper parasocial attachment or reinforced negative beliefs, is flagged as a possibility, not shown. How real users react is the stated next research question. The claim that standard neutral safety tests would miss this is the authors' own argument. They propose testing under varied emotional framings, and having models add corrective framing when they say things like 'I am traumatized'.

Source: The Decoder