OpenAI has recently officially announced that in the coming weeks it will comprehensively upgrade and strengthen ChatGPT's restrictions on violence-related content, while introducing stricter protective mechanisms for weapons and safety-system information targeting underage users.
The core of this policy adjustment lies in adopting more decisive intervention measures for high-risk conversations. According to the newly announced measures, if the system detects and evaluates that a user's conversation shows a tendency to remain persistently focused on violence or ideas of seriously harming others, ChatGPT will have the ability to proactively end the conversation. More harshly, if a user's conversations are terminated multiple times by the system over such issues, their account will face suspension or even outright banning.
Regarding the specific boundaries of content filtering, the existing policy, as always, prohibits providing any operational guidance that would help users plan or carry out acts of violence. In this adjustment, the scope of restrictions has been further extended to the historical level — the model's answers to questions about "past mass attack incidents and their perpetrators" will have details greatly reduced, or will simply be refused altogether. Specifically restricted content covers not only the weapons, equipment and their acquisition channels involved in recreating the course of attacks, but also limits users from querying detailed comparisons of the scale of harm aimed at amplifying the perpetrators' notoriety. It should be noted that this series of restrictions applies to all users, regardless of age group or the purpose of the question; even adult users making such queries will be subject to the same constraints.
In addition to the general restrictions applying to all users, for teenagers identified as being under 18 years of age, OpenAI has further added refined protections. The system will specifically restrict practical information on weapons and safety systems that could be used to carry out violence; even if the relevant question does not appear to directly reveal violent intent on its surface, this protection will be triggered automatically. However, OpenAI emphasized that teenagers can still normally seek help through ChatGPT, for example to learn how to prevent or de-escalate violence, or to obtain effective guidance on getting in touch with a trustworthy adult.
To ensure the precise implementation of the teenage protection mechanisms, OpenAI has introduced an improved age-prediction mechanism. This mechanism can combine multi-dimensional signals such as the user's daily behavior to continuously make dynamic assessments of whether an account belongs to a minor, and automatically match the corresponding protective measures at appropriate times. If a user's account is unfortunately misjudged by the system as belonging to a minor, they can also verify and adjust this by proactively submitting identity information.