Introduction

Quality Evaluator API Reference 1.0.0

The Quality Evaluator API uses LLMs to automatically assess translation quality against requirements you define, in plain language rather than fixed rule sets. It scales the kind of review a human linguist would do: check tone, terminology, formatting, and custom requirements across any volume of segments. Quality requirements are managed on a Content Group. Link evaluation to the Content Group and the API resolves the checks that apply to it automatically, so there is nothing separate to create or keep in sync.

How evaluation works

When you evaluate segments against a Content Group, the API resolves its AI Checks: Style Guide Rules attached to the Content Group, written in natural language (e.g., “Translation must not use Yoda-style speech”). Only Rules with AI Check enabled that apply to the target locale are evaluated. Each segment is judged by an LLM against every resolved Rule. The response returns one result per Rule for each segment, with a verdict and an explanation.

Key Features

Content Group integration: Quality requirements stay in sync with the Content Group without manual maintenance
AI Checks: Style Guide Rules resolved into natural-language checks and evaluated by an LLM
Analytics: Track evaluation metrics and AI unit consumption over time

Base URLs

The Quality Evaluator API is available in multiple regions:

Region	Base URL
EU	`https://eu.phrase.com/quality-evaluator`
US	`https://us.phrase.com/quality-evaluator`

Quick Start

Attach Style Guide Rules to your Content Group (with AI Check enabled)
Evaluate segments by referencing the Content Group. Checks are resolved automatically

# Example: Evaluate segments against a Content Group's resolved AI Checks
curl -X POST "https://eu.phrase.com/quality-evaluator/v3/evaluation" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "contentGroupId": "cg-abc123",
    "segments": [
      {
        "id": "seg-1",
        "source": "Hello, world!",
        "target": "Hallo, Welt!"
      }
    ],
    "sourceLocaleCode": "en_us",
    "targetLocaleCode": "de_de"
  }'

Each request accepts up to 200 segments. A segment id is optional; when provided, it must be unique within the request and is echoed back in the response. The response lists every segment with the results of the checks that ran on it:

{
  "contentGroupId": "cg-abc123",
  "segments": [
    {
      "id": "seg-1",
      "source": "Hello, world!",
      "target": "Hallo, Welt!",
      "results": [
        {
          "type": "AI_CHECK",
          "ruleUid": "rule-abc123",
          "evaluation": {
            "isValid": true,
            "explanation": "The translation follows the requirement."
          }
        }
      ]
    }
  ]
}

If the Content Group has no applicable checks, every segment is returned with an empty results array. No error is raised. Segments with an empty or blank target are not evaluated and also return an empty results array.

Inspect the checks for a Content Group

To see which Rules apply to a Content Group before you evaluate, call GET /v3/qualityProfiles/{contentGroupId}. You can filter the result by locale, enabled state, and requirement text.

Overview

API Documentation

Quality Evaluator API Reference 1.0.0

How evaluation works

Key Features

Base URLs

Quick Start

Inspect the checks for a Content Group

​Quality Evaluator API Reference 1.0.0

​How evaluation works

​Key Features

​Base URLs

​Quick Start

​Inspect the checks for a Content Group

Quality Evaluator API Reference 1.0.0

How evaluation works

Key Features

Base URLs

Quick Start

Inspect the checks for a Content Group