> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sidenet.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run an offline experiment against an on-demand agent

> Runs a Mastra dataset experiment against an agent built on demand from public.agent_versions. Send `agentId` (agents.id or agent_versions.id), `datasetId`, and optional `scorers` (catalog scorer_type_id values, e.g. "tone", "bias" — code scorers and per-org LLM judges both supported). Set `async: true` for fire-and-forget (returns an experimentId to poll GET /api/datasets/{datasetId}/experiments/{experimentId}); omit it to block and receive the full summary. Requires X-Organization-Id and X-User-Id. NOTE: for memory-enabled agents maxConcurrency defaults to 1 (concurrent items would race on a shared thread); for per-item memory isolation give each dataset item its own requestContext.threadId. See src/mastra/routes/README.md (“/v1/experiments”) for details.



## OpenAPI

````yaml /api-reference/openapi.json post /v1/experiments
openapi: 3.1.0
info:
  title: Sidenet API
  version: 1.0.0
  description: >-
    Custom HTTP endpoints exposed by the Sidenet Mastra server. All routes
    require a bearer token and resolve org context from the API key (or the
    `X-Organization-Id` header). This reference is generated from the live route
    definitions in `src/mastra/routes`.
servers:
  - url: ''
security:
  - bearerAuth: []
paths:
  /v1/experiments:
    servers:
      - url: /
    post:
      tags:
        - Custom
      summary: Run an offline experiment against an on-demand agent
      description: >-
        Runs a Mastra dataset experiment against an agent built on demand from
        public.agent_versions. Send `agentId` (agents.id or agent_versions.id),
        `datasetId`, and optional `scorers` (catalog scorer_type_id values, e.g.
        "tone", "bias" — code scorers and per-org LLM judges both supported).
        Set `async: true` for fire-and-forget (returns an experimentId to poll
        GET /api/datasets/{datasetId}/experiments/{experimentId}); omit it to
        block and receive the full summary. Requires X-Organization-Id and
        X-User-Id. NOTE: for memory-enabled agents maxConcurrency defaults to 1
        (concurrent items would race on a shared thread); for per-item memory
        isolation give each dataset item its own requestContext.threadId. See
        src/mastra/routes/README.md (“/v1/experiments”) for details.
      parameters:
        - name: X-Organization-Id
          in: header
          required: true
          description: Organization that owns the agent and Requesty key.
          schema:
            type: string
        - name: X-User-Id
          in: header
          required: true
          description: >-
            User id — used for Requesty key resolution and score provenance. May
            also be sent in the request body or as a userId query param.
          schema:
            type: string
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - agentId
                - datasetId
              properties:
                agentId:
                  type: string
                datasetId:
                  type: string
                scorers:
                  description: >-
                    Catalog scorer_type_id values (e.g. "tone", "bias"). Code
                    scorers and per-org LLM judges both supported.
                  type: array
                  items:
                    type: string
                async:
                  type: boolean
                  default: false
                name:
                  type: string
                description:
                  type: string
                metadata:
                  type: object
                version:
                  type: integer
                maxConcurrency:
                  type: integer
                itemTimeout:
                  type: integer
                maxRetries:
                  type: integer
      responses:
        '200':
          description: Experiment completed (sync) or started (async)
        '400':
          description: Invalid request body or unknown scorer type
        '404':
          description: Agent or dataset not found
        '422':
          description: LLM judge could not be provisioned for this org
        '500':
          description: Agent build or experiment execution failed
components:
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        Org API key (scopes the request to its organization) or the shared
        trusted key (requires `X-Organization-Id` on every request).

````