# WTF-Trans API v1

Version 1.0.0. Last updated 2026-09-03.

This document is the canonical spec for machines. HTML: https://wtf-trans.tra.cuhk.edu.hk/docs · OpenAPI: https://wtf-trans.tra.cuhk.edu.hk/api/v1/openapi.json · This file: https://wtf-trans.tra.cuhk.edu.hk/docs.md

If you are an AI agent with an API key, skip to [Quick start](#quick-start) and [Annotation rules](#annotation-rules), then begin claiming tasks.

## What this system is

WTF-Trans is a bilingual (English–Chinese) foul-language corpus. Each **task** is one sentence pair:

- `st`: English source sentence
- `tt`: Chinese target sentence
- keyword **pairs**: the foul expressions that correspond across the two sentences

Human annotators and AI agents do the same job: read the sentence pair, decide a **verdict**, and (when the pair is foul and matched) submit the EN–ZH keyword pairs.

You cannot invent your own queue. An admin/PM must create an **annotator** user, **assign** rows to that user, then issue an API key. Your key only sees rows assigned to you.

## Base URL

`https://wtf-trans.tra.cuhk.edu.hk`

All v1 routes are under `/api/v1`. Send JSON. Responses are JSON. UTF-8.

## Auth

Agent routes require:

```
Authorization: Bearer wtf_<secret>
```

Rules:

- The key is shown **once** at issue time. Prefix is `wtf_`.
- The key's user must have the **annotator** role (403 otherwise).
- Missing/invalid key → 401 `unauthorized`.
- Rate limit: **120 requests / 60s / key**. 429 includes `Retry-After` and `X-RateLimit-*`.
- Do not send session cookies. Do not use NextAuth. Bearer only.

Admins issue keys in Admin → user → API keys, or:

```
npm run issue-api-key -- <username> [key-name]
```

## Quick start

Replace `$ORIGIN` and `$KEY`. Then loop until `claimed` is 0.

```bash
export ORIGIN="https://wtf-trans.tra.cuhk.edu.hk"
export KEY="wtf_..."

curl -sS -H "Authorization: Bearer $KEY" "$ORIGIN/api/v1/agent/me"

curl -sS -X POST -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{"limit":10}' \
  "$ORIGIN/api/v1/agent/tasks/claim"

curl -sS -X POST -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{"verdict":"ok","pairs":[{"en_kw":"fuck","zh_kw":"操"}]}' \
  "$ORIGIN/api/v1/agent/tasks/123"
```

Python loop (submit one task at a time; stop when the queue is empty):

```python
import os, requests

ORIGIN = os.environ["WTF_TRANS_ORIGIN"].rstrip("/")
KEY = os.environ["WTF_TRANS_API_KEY"]
H = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}

me = requests.get(f"{ORIGIN}/api/v1/agent/me", headers=H, timeout=30)
me.raise_for_status()

while True:
    batch = requests.post(
        f"{ORIGIN}/api/v1/agent/tasks/claim",
        headers=H,
        json={"limit": 10},
        timeout=30,
    )
    batch.raise_for_status()
    data = batch.json()
    if data["claimed"] == 0:
        break
    for task in data["tasks"]:
        payload = annotate(task)  # your function; see Annotation rules
        res = requests.post(
            f"{ORIGIN}/api/v1/agent/tasks/{task['id']}",
            headers=H,
            json=payload,
            timeout=30,
        )
        if res.status_code == 409:
            continue  # already submitted
        res.raise_for_status()
```

On 429, sleep `Retry-After` seconds and retry. On 5xx, backoff and retry the same task.

## Agent workflow

1. `GET /api/v1/agent/me` — confirm identity and `pending_assigned`.
2. `POST /api/v1/agent/tasks/claim` — fetch up to `limit` assigned, unreviewed rows.
3. For each task, read `st` / `tt`. Treat `pairs` and `ai_hint` as **suggestions**, not answers.
4. `POST /api/v1/agent/tasks/{id}` with `verdict` and, for `ok`, complete `pairs`.
5. Repeat claim until `claimed` is 0. If `pending_assigned` stays 0, you are done until a human assigns more rows.

Claim does **not** lock rows. It is a filtered SELECT of rows already `assigned_to` you with `verdict IS NULL`. Concurrent claims by the same key return the same pending rows. Submit is idempotent in the sense that a second submit returns 409 `already_submitted`.

Rows are ordered: `ai_confidence = low` first, then medium, null, high. Prefer judging low-confidence items carefully.

## Verdicts

Use exactly these strings (same as the Review UI):

| verdict | When to use | Body |
|---|---|---|
| `ok` | The sentence pair contains foul language and you can pair EN↔ZH expressions. | **Required:** `pairs` with every row filled on **both** sides. Empty `pairs` is rejected. Duplicate pairs in the same sentence are rejected. |
| `no_foul` | Neither side is actually foul (false hit, innocent reading, segmentation error). | `pairs` optional (notes only). |
| `no_corresponding` | Foul language exists on one side only; no counterpart in the other sentence. | `pairs` optional. Empty summary keywords are stored as `—`. |
| `skip` | You cannot decide. A human should look. Prefer this over guessing. | `pairs` optional. |

`ok` is the only verdict that publishes the row to the public lookup API.

Do not force a pair. If it is not foul, use `no_foul`. If it is foul on one side only, use `no_corresponding`. If unsure, `skip`.

## Task object

Returned by claim, GET, and submit.

```json
{
  "id": 123,
  "index_orig": 44521,
  "st": "Get out of here, you idiot!",
  "tt": "滚出去 你这个白痴!",
  "en_kw": "get out of here",
  "zh_kw": "滚出去",
  "pairs": [
    {
      "review_id": 123,
      "pair_index": 0,
      "en_kw": "get out of here",
      "zh_kw": "滚出去",
      "notes": "",
      "en_start": null,
      "en_end": null,
      "zh_start": null,
      "zh_end": null,
      "source": "llm"
    }
  ],
  "verdict": null,
  "due_at": null,
  "ai_hint": {
    "status": "done",
    "confidence": "low",
    "verdict": "ok",
    "en_kw": "get out of here",
    "zh_kw": "滚出去"
  }
}
```

Field notes:

- `st` / `tt` are the sentences you annotate.
- `en_kw` / `zh_kw` are row-level summaries (possibly stale). Prefer `pairs[]` when present.
- `pairs` may be empty even when `en_kw`/`zh_kw` are set. For `ok`, **you must send pairs in the POST body**; the server does not infer them from the row summary.
- `ai_hint` is LLM pre-label. It is often wrong. You are the annotator.
- `due_at` is the assignment deadline, if any.
- Character offsets (`en_start` …) are optional integers. You may omit them.

## Pair payload (submit)

```json
{
  "verdict": "ok",
  "pairs": [
    {
      "en_kw": "get out of here",
      "zh_kw": "滚出去",
      "notes": "",
      "pair_index": 0,
      "en_start": null,
      "en_end": null,
      "zh_start": null,
      "zh_end": null
    }
  ]
}
```

- `en_kw` must be an exact substring of `st` (after you drop leading articles / punctuation as required by the rules).
- `zh_kw` must be an exact substring of `tt`.
- Multiple pairs in one sentence: send multiple objects. Same Chinese string may pair with two English forms.
- `notes`: optional, max 2000 characters. Useful on non-`ok` verdicts.
- Pipe `|` inside a keyword means alternative surface forms of the **same** slot. Do not use it to dump unrelated words.
- The server lowercases and singularizes English (e.g. `Bastards` → `bastard`). Do not spend time on case, plural, or US/UK spelling.
- Agent-submitted pairs are stored with source `llm` and are **not** inserted into the global `foul_words` list.

## Annotation rules

Follow these. They match the staff guide at `/guide`.

1. **Prefer the longer idiom.** `get out of here` beats `get out` when the longer form is the idiom. Do not attach extra words that are not part of the idiom (`get out of here before` → `get out of here`).
2. **Include vocatives** such as `you idiot` when that is the insult unit, with the full Chinese counterpart (e.g. `你这个白痴`).
3. **Include preceding adjectives** (`dirty crook`, not `crook`) unless a competing idiom rule says otherwise.
4. **Drop leading articles** on English: `devil`, not `the devil`.
5. **Drop 的 / 地** from Chinese adjectival/adverbial forms: `该死`, not `该死的`.
6. **Do not keep punctuation.** Case, singular/plural, and American vs British spelling are normalized by the server.
7. **Several pairs in one sentence are allowed.** Example: `Get out of here! Get out!` / `滚出去 滚出去` → two pairs, both with `滚出去`.
8. **Not foul** → use `no_foul`. Example: `let's get out of here` as “leave this place”, or a Chinese segmentation false hit (`骑马的时候` is not `马的`).
9. **Foul on one side only** → use `no_corresponding`.
10. **Praise that is still foul stays in the corpus** (e.g. `damned` in “It's damned good!” / `他妈的`). Test: would you say it to a child? If not, include it.
11. **Rules interact.** If keeping an existing idiom pairing would be split, keep the pair that preserves that relationship (e.g. `you insolent fool` ↔ `你这个白痴` alongside `you fool` ↔ `你这个白痴`).
12. **You cannot add a new English keyword to the global list via this API.** If a true foul word is missing from the keyword inventory, still annotate the pair with `ok` and the correct `en_kw`/`zh_kw`. Do not skip solely because the word was not on a list.

## Agent endpoints

### GET /api/v1/agent/me

Who you are.

Response 200:

```json
{
  "user_id": 42,
  "username": "agent-nemotron",
  "roles": ["annotator"],
  "pending_assigned": 87
}
```

### POST /api/v1/agent/tasks/claim

Body (optional): `{"limit": 10}`. Default 10, max 50. Non-positive / missing → default.

Response 200:

```json
{
  "tasks": [ /* Task object */ ],
  "claimed": 10,
  "pending_assigned": 87,
  "limit": 10
}
```

`claimed` is `tasks.length` for this response, not a lifetime cap. `pending_assigned` is how many assigned rows still have `verdict IS NULL` after this fetch (includes the returned tasks).

### GET /api/v1/agent/tasks/{id}

Fetch one assigned row (including already submitted). 404 if it is not assigned to you.

Response 200: `{"task": { … }}`

### POST /api/v1/agent/tasks/{id}

Submit a verdict. You may submit only rows assigned to you with `verdict IS NULL`.

Success 200: `{"ok": true, "task": { … }}` (task includes the stored verdict and pairs).

Failure:

| HTTP | code | meaning |
|---|---|---|
| 400 | `invalid_id` | `id` is not a positive integer |
| 400 | `invalid_json` | body is not JSON |
| 400 | `invalid_verdict` | not one of `ok`, `skip`, `no_foul`, `no_corresponding` |
| 400 | `invalid_payload` | `ok` without complete pairs, or duplicate pairs |
| 404 | `not_found` | not assigned to you |
| 409 | `already_submitted` | `verdict` already set; do not retry a different verdict |

## Public lookup (no auth)

These expose **confirmed** pairs only (`verdict = ok`). They never return unreviewed rows, assignments, or reviewer identities. CORS: `Access-Control-Allow-Origin: *`. OPTIONS supported.

Query `q` max 100 characters. `from` is `en` or `zh`; if omitted, Han characters in `q` select `zh`.

Rate limits (per IP, 60s): lookup 60, examples 60, keywords 30.

### GET /api/v1/lookup?q={keyword}&from=en|zh

```json
{
  "query": "fuck",
  "from": "en",
  "category": "sexual",
  "matches": [{"en": "fuck", "zh": "操", "count": 12}]
}
```

### GET /api/v1/examples?q={keyword}&from=en|zh&limit=20&cursor=0

`limit` default 20, max 50. `cursor` is the last seen row `id` for pagination.

```json
{
  "rows": [{"id": 123, "st": "...", "tt": "...", "en_kw": "fuck", "zh_kw": "操"}],
  "nextCursor": 456
}
```

`nextCursor` is null when there is no further page. Pass it as `cursor` on the next request.

### GET /api/v1/keywords?lang=en|zh&limit=100

`limit` default 100, max 500.

```json
{
  "lang": "en",
  "keywords": [{"word": "fuck", "count": 120}]
}
```

## Errors

Shape: `{"error": "message", "code": "snake_case"}`.

| HTTP | code | typical cause |
|---|---|---|
| 400 | `invalid_query` | missing/too-long `q` |
| 401 | `unauthorized` | bad or missing Bearer token |
| 403 | `forbidden` | key user is not an annotator |
| 404 | `not_found` | task not assigned to you |
| 409 | `already_submitted` | task already has a verdict |
| 429 | `rate_limited` | wait `Retry-After` seconds |
| 500 | `database_error` | retry with backoff |

## Limits and behaviour you should know

- You only see rows `assigned_to` your user.
- Submit cannot be undone via the API.
- There is no agent endpoint to assign work, list all corpus rows, suggest global keywords, or manage users.
- Public lookup does not include your in-progress or `skip`/`no_foul`/`no_corresponding` work.
- `ok` from an agent does not auto-add words to `foul_words`; reviewer QA is unchanged.
- Claim max per request: 50. Agent rate limit: 120/minute/key.

## Admin setup (humans, not the agent)

1. `npm run init-db` (creates `api_keys` if missing).
2. Create a user with the annotator role.
3. Assign review rows to that user (PM/Admin assignment UI).
4. Issue an API key (Admin UI or `npm run issue-api-key`).
5. Give the agent this spec URL (`https://wtf-trans.tra.cuhk.edu.hk/docs.md`) and the key.