Run an offline experiment against an on-demand agent
Runs a Mastra dataset experiment against an agent built on demand from public.agent_versions. Send agentId (agents.id or agent_versions.id), datasetId, and optional scorers (catalog scorer_type_id values, e.g. “tone”, “bias” — code scorers and per-org LLM judges both supported). Set async: true for fire-and-forget (returns an experimentId to poll GET /api/datasets//experiments/); omit it to block and receive the full summary. Requires X-Organization-Id and X-User-Id. NOTE: for memory-enabled agents maxConcurrency defaults to 1 (concurrent items would race on a shared thread); for per-item memory isolation give each dataset item its own requestContext.threadId. See src/mastra/routes/README.md (“/v1/experiments”) for details.
Authorizations
Org API key (scopes the request to its organization) or the shared trusted key (requires X-Organization-Id on every request).
Headers
Organization that owns the agent and Requesty key.
User id — used for Requesty key resolution and score provenance. May also be sent in the request body or as a userId query param.
Body
Catalog scorer_type_id values (e.g. "tone", "bias"). Code scorers and per-org LLM judges both supported.
Response
Experiment completed (sync) or started (async)