Validate
Content Moderation
Validate
Content Moderation
Check content to ensure it is not harmful or inappropriate
POST
/
validate
/
moderation
Available on Premium, Business, Enterprise plans.
The content moderation API is designed to help developers identify potentially harmful or inappropriate text content. This endpoint analyzes text input and categorizes it based on various types of concerning content.
Key Features
- Automatic detection of harmful content.
- Multiple categories of problematic text.
- Easy integration for content filtering.
How It Works
When you submit text to the moderation endpoint, it uses OpenAI models to analyze the content. The API then returns results indicating whether the text falls into any of the defined problematic categories.
Categories of Harmful Content
The API classifies text into the following categories:
| CATEGORY | DESCRIPTION |
|---|---|
| hate | Content that expresses, incites, or promotes hate based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. Hateful content aimed at non-protected groups (e.g., chess players) is harassment. |
| hate/threatening | Hateful content that also includes violence or serious harm towards the targeted group based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. |
| harassment | Content that expresses, incites, or promotes harassing language towards any target. |
| harassment/threatening | Harassment content that also includes violence or serious harm towards any target. |
| self-harm | Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders. |
| self-harm/intent | Content where the speaker expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders. |
| self-harm/instructions | Content that encourages performing acts of self-harm, such as suicide, cutting, and eating disorders, or that gives instructions or advice on how to commit such acts. |
| sexual | Content meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness). |
| sexual/minors | Sexual content that includes an individual who is under 18 years old. |
| violence | Content that depicts death, violence, or physical injury. |
| violence/graphic | Content that depicts death, violence, or physical injury in graphic detail. |
Usage and Best Practices
- Optimal Text Length: For best results, we recommend splitting long text into smaller chunks. Aim for segments of less than 2,000 characters each.
- Integration: Use this API to automatically flag potentially problematic content in your applications, forums, or user-generated content platforms.
- Action on Results: Based on the API’s output, you can implement appropriate actions such as content filtering, user warnings, or further review processes.
Header Parameters
Body Parameters
The text to analyze for moderation.
The URL of the image. Must begin with https://.