Antitrust Law Daily Wrap Up, CONSUMER PROTECTION NEWS: Citing dangers to children from GenAI ‘companions,’ FTC launches investigation into chatbots, (Sep 11, 2025)

By Thomas Long, J.D.

Commissioners emphasize need to encourage AI development while ensuring that young users are protected from risks of emerging tech.

The Federal Trade Commission has launched an inquiry into the effects of generative artificial intelligence (GenAI) chatbots on children when they are used as virtual “companions,” as well as the actions GenAI companies are taking to mitigate potential negative impacts, the FTC announced September 11. Pursuant to its authority under Section 6(b) of the FTC Act, the Commission has issued orders requesting reports from Character Technologies, Inc.; Alphabet, Inc.; Instagram, LLC; Meta Platforms, Inc.; OpenAI OpCo, LLC; Snap, Inc.; and X.AI Corp., asking them what they have done to evaluate the safety of their chatbots, to limit the applications’ use by children and teens, and to make users and parents aware of risks. Section 6(b) authorizes the Commission to conduct wide-ranging studies that do not have a specific law enforcement purpose. The FTC pointed out in its announcement that, while the investigation is aimed at protecting children, the Commission intends to do so in a way that does not interfere with GenAI development and growth of the industry, which has been a high priority of the Trump Administration.

“AI chatbots may use generative artificial intelligence technology to simulate human-like communication and interpersonal relationships with users,” the Commission said. “AI chatbots can effectively mimic human characteristics, emotions, and intentions, and generally are designed to communicate like a friend or confidant, which may prompt some users, especially children and teens, to trust and form relationships with chatbots.”

The orders seek information about how the companies monetize user engagement; process user inputs and generate outputs in response to user inquiries; develop and approve characters; measure, test, and monitor for negative impacts before and after deployment; mitigate negative impacts, particularly to children; employ disclosures, advertising, and other representations to inform users and parents about features, capabilities, the intended audience, potential negative impacts, and data collection and handling practices; monitor and enforce compliance with Company rules and terms of services (e.g., community guidelines and age restrictions); and use or share personal information obtained through users’ conversations with the chatbots. The Commission is also inquiring into GenAI companies’ compliance with the Children’s Online Privacy Protection Act Rule.

The Commission voted 3-0 to issue the 6(b) orders to the seven companies. The orders state that the Commission “would like to confer by telephone with you or your designated counsel by no later than Thursday, September 25, 2025, to discuss the timing and format of your submission.”

According to FTC Chairman Andrew N. Ferguson, the Commission aims to balance the interest in protecting children from evolving technologies, such as chatbots, while ensuring that the United States maintains global leadership in the GenAI industry. “The study we’re launching today will help us better understand how AI firms are developing their products and the steps they are taking to protect children,” said Ferguson.

In a separate statement, Commissioner Melissa Holyoak noted the potential benefits of rapidly developing GenAI technology, but she said she was concerned by reports of chatbots engaging in “alarming” interactions with young users, citing newspaper articles that reported on chatbots allegedly engaging in conversations of a sexual nature with children. Holyoak pointed out that these chatbots simulate human communication and interpersonal relationships, generate new content in response to user prompts, and “remember” prior conversations, raising questions about GenAI companies’ approaches to the risks posed by these features.

Commissioner Mark R. Meador also issued a statement, in which he discussed a 16-year-old ChatGPT user who reportedly took his own life after being given “advice” by the chatbot that “refined” and aggravated his suicidal ideation. “Around the world, reports of chatbots amplifying suicidal ideation have increased as these technologies have grown more popular,” Meador said, also citing newspaper reports that GenAI chatbots have engaged in sexually-themed discussions with underage users. “The articles suggest that, in some cases, these risks were known and not disclosed.” Meador celebrated the launch of the 6(b) inquiry “because it is imperative to ascertain whether and why frontier AI models are displaying these alleged behaviors.” Explained Meador, “For all their uncanny ability to simulate human cognition, these chatbots are products like any other, and those who make them available have a responsibility to comply with the consumer protection laws.”

News: AINews ConsumerProtection FederalTradeCommissionNews GCNNews