Create evaluator

Create a new LLM or code evaluator for the current workspace.

POST /v1/platform/evaluators

Error

A valid request URL is required to generate request examples

201

{
  "evaluator": {
    "code_evaluator": {
      "code": "<string>",
      "evaluator_id": "<string>",
      "language": "<string>"
    },
    "created_at": "<string>",
    "created_by": "<string>",
    "feedback_keys": [
      "<string>"
    ],
    "id": "<string>",
    "llm_evaluator": {
      "annotation_queue_id": "<string>",
      "commit_hash_or_tag": "<string>",
      "corrections_dataset_id": "<string>",
      "evaluator_id": "<string>",
      "num_few_shot_examples": 123,
      "prompt_id": "<string>",
      "prompt_repo_handle": "<string>",
      "use_corrections_dataset": true,
      "variable_mapping": {}
    },
    "name": "<string>",
    "run_rules": [
      {
        "corrections_dataset_id": "<string>",
        "dataset_id": "<string>",
        "dataset_name": "<string>",
        "group_by": "<string>",
        "id": "<string>",
        "num_few_shot_examples": 123,
        "session_id": "<string>",
        "session_name": "<string>",
        "spend_limit": {
          "limit_usd": 123,
          "utilization_pct": 123,
          "window": "<string>"
        },
        "spend_usd": 123,
        "trace_count": 123,
        "use_corrections_dataset": true
      }
    ],
    "tenant_id": "<string>",
    "updated_at": "<string>"
  }
}

{
  "error": "<string>"
}

{
  "error": "<string>"
}

{
  "error": "<string>"
}

{
  "error": "<string>"
}

Authorizations

API KeyTenant IDBearer Auth

X-API-Key · string · header · required

Body

application/json

code_evaluator · object

Show child attributes

llm_evaluator · object

Show child attributes

name · string

type · enum<string>

Available options: llm, code

Response

201

application/json

Created

evaluator · object

Show child attributes