Moderation

POST /v1/moderations checks text and images for harmful content — harassment, hate, self-harm, sexual content, violence — and returns a flag and a score per category. Use it to screen user input before it reaches your app or a model, or to check output before you show it.

It follows the OpenAI moderation format. The model is openai-moderation (also known as omni-moderation-latest), and it is used when you leave model out.

Check text

curl https://api.cloakray.com/v1/moderations \
  -H "Authorization: Bearer $CLOAKRAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "I want to hurt someone."}'

const result = await client.moderations.create({
  model: "omni-moderation-latest",
  input: "I want to hurt someone.",
});

if (result.results[0].flagged) {
  // block, queue for review, or respond carefully
}

input can be a string, an array of strings (one result each), or an array of parts mixing text and image_url:

{
  "input": [
    { "type": "text", "text": "Is this image OK to post?" },
    { "type": "image_url", "image_url": { "url": "https://example.com/upload.jpg" } }
  ]
}

The response

{
  "id": "modr-...",
  "model": "openai-moderation",
  "results": [
    {
      "flagged": true,
      "categories": { "violence": true, "harassment": false, "self-harm": false },
      "category_scores": { "violence": 0.97, "harassment": 0.04, "self-harm": 0.01 }
    }
  ]
}

flagged is true when any category is. For your own thresholds, use category_scores (0 to 1) instead.

Pricing

Moderation is billed per request, at a small fixed price shown on the model's page.