Back to search
100
MCP live MCP 2025-06-18 streamable-http

OpenMark

ai.openmark/openmark

Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.

Website ↗
Watch this dependency Share evidence report
Uptime
100.0%
4 direct probes · 30d
Response
533ms
last probe
Tools
10
callable
Resources
0
readable
Prompts
0
available
Primary URL
https://openmark.ai/mcp
Score: 100/100
Handshake verified by our own probe.
pass Answers the MCP handshake 50/50
The handshake and the list calls succeeded on the primary URL.
pass Lists tools, resources or prompts 30/30
10 tools, 0 resources, 0 prompts returned.
pass 3 or more catalogue items 10/10
10 items in total; 3 or more earns these points.
pass Describes itself 10/10
96 characters; more than 30 earns these points.

Tools · 10

get_task_format

How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scor…

validate_task

Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a loo…

list_models

Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reason…

prepare_attachments

Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachmen…

estimate_cost

Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run sett…

run_benchmark

Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's c…

get_results

Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and q…

export_openclaw_csv

A completed job's results as the CSV the OpenMark Router plugin for OpenClaw reads (the same file as the app's Export > OpenClaw). Save it in the plugin's benchmarks folder, by default ~/.openclaw/wor…

get_account

The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.

watch_task

Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's cre…

How to use

Claude Desktop or claude.ai: add a custom connector (Customize → Connectors; on Team and Enterprise plans an owner adds it under Organization settings → Connectors) with this server URL: https://openmark.ai/mcp

Cursor or Cline: add to your MCP config:

{
  "mcpServers": {
    "openmark": {
      "url": "https://openmark.ai/mcp",
      "transport": "streamable-http"
    }
  }
}