REST API

The SupaEval REST API provides direct HTTP access to all platform features. Use this for custom integrations, languages without an official SDK, or webhook-based workflows.

Base URL

All API requests should be made to: https://api.supaeval.com

Authentication

Authenticate your API requests using Bearer token authentication. Include your API key in the Authorization header:

http

Authorization: Bearer sk_live_your_api_key_here

Rate Limits

API requests are rate limited to 1000 requests per minute. Contact support for higher limits.

Datasets

Create Dataset

Create a new evaluation dataset:

http

POST /api/v1/datasets
Content-Type: application/json

{
  "name": "my-evaluation-dataset",
  "description": "Test dataset for agent evaluation",
  "metadata": {
    "domain": "general",
    "language": "en"
  }
}

Response:

json

{
  "id": "dataset_abc123",
  "name": "my-evaluation-dataset",
  "description": "Test dataset for agent evaluation",
  "created_at": "2024-01-30T10:00:00Z",
  "item_count": 0,
  "metadata": {
    "domain": "general",
    "language": "en"
  }
}

Add Items to Dataset

Add test cases to an existing dataset:

http

POST /api/v1/datasets/{dataset_id}/items
Content-Type: application/json

{
  "items": [
    {
      "input": "What is the capital of France?",
      "expected_output": "Paris",
      "metadata": {"difficulty": "easy"}
    },
    {
      "input": "Explain quantum computing",
      "expected_output": "Quantum computing uses quantum bits...",
      "metadata": {"difficulty": "hard"}
    }
  ]
}

Evaluations

Create Evaluation

Start a new evaluation run:

http

POST /api/v1/evaluations
Content-Type: application/json

{
  "dataset_id": "dataset_abc123",
  "agent_endpoint": "https://your-agent.api/chat",
  "metrics": ["accuracy", "relevance", "faithfulness"],
  "config": {
    "timeout": 30,
    "retry_count": 3
  }
}

Response:

json

{
  "id": "eval_xyz789",
  "dataset_id": "dataset_abc123",
  "status": "running",
  "created_at": "2024-01-30T10:05:00Z",
  "progress": 0.0,
  "estimated_completion": "2024-01-30T10:10:00Z"
}

Get Results

Retrieve evaluation results:

http

GET /api/v1/evaluations/{evaluation_id}/results

Response:

json

{
  "evaluation_id": "eval_xyz789",
  "status": "completed",
  "overall_score": 0.87,
  "pass_rate": 85.5,
  "metrics": {
    "accuracy": 0.90,
    "relevance": 0.88,
    "faithfulness": 0.83
  },
  "completed_at": "2024-01-30T10:09:30Z",
  "total_items": 100,
  "passed_items": 85,
  "failed_items": 15
}

Error Handling

The API uses standard HTTP status codes. Errors return JSON with details:

json

{
  "error": {
    "code": "invalid_request",
    "message": "Dataset not found",
    "param": "dataset_id",
    "type": "invalid_request_error"
  }
}

Common Status Codes

200 - Success
201 - Resource created
400 - Bad request (invalid parameters)
401 - Unauthorized (invalid API key)
404 - Resource not found
429 - Rate limit exceeded
500 - Internal server error

Pagination

List endpoints support pagination using limit and offset parameters:

http

GET /api/v1/datasets?limit=20&offset=0

Next Steps

Use the Python SDK for easier integration
Use the JavaScript SDK for Node.js apps
Learn about evaluation metrics