Roblox has released three of its AI safety tools to the open-source community

Roblox has decided to make its three AI safety models openly accessible through the Robust Open Online Safety Tools Model Community, also known as ROOST. This independent nonprofit, established in a venture with Google, OpenAI, and Discord in 2025, maintains open-source infrastructure focused on online safety.

The three models being contributed by Roblox include the PII Classifier, Roblox Sentinel, and the latest voice safety classifier. Already active on its own platform, these tools aim to bolster safety, with the PII Classifier identifying cases of sharing or requesting personal information by detecting various disguises and code words.

In addition, Roblox offers an evaluation dataset showcasing synthetic chat samples designed to mimic methods used to bypass these filters. Roblox Sentinel is designed to spot early warning signs of potential threats to children, while the voice safety classifier handles real-time voice chat moderation, improved with feedback from the ROOST community.

This new evaluation dataset will assist companies in testing the efficiency of their safety solutions, with moderation support extended to 30 languages and eight types of violations. According to Naren Koneru, Roblox's vice president for trust and safety engineering, sharing these models provides a valuable foundation for other platforms to develop their moderation tools. It also enables mutual learning, as companies can share advancements back.

Koneru stresses that safety is a collective effort, and Roblox remains committed to advancing detection tools through collaboration within the ROOST community. In the context of this commitment, an investigation by the US Senate Judiciary Subcommittee on Crime and Counterterrorism was announced. It centers on claims that suggest Roblox prioritizes engagement and revenue over child safety.

The platform has been requested to submit detailed records on topics including child safety and cases of abuse by August 31, 2026. To address safety concerns, Roblox has launched measures such as restricting direct messaging for those under 13, placing age checks for chat features globally, initiating a "sensitive issues" tag for content, and bolstering moderation systems and parental controls.