Back to search
100
MCP live MCP 2025-11-25 streamable-http

AimRank

io.aimrank/aimrank

Read public pairwise-preference rankings and their statistical quality evidence. No API key.

Watch this dependency Share evidence report
Uptime
100.0%
5 direct probes · 30d
Response
326ms
last probe
Tools
21
callable
Resources
2
readable
Prompts
5
available
Primary URL
https://app.aimrank.io/mcp/public
Score: 100/100
Handshake verified by our own probe.
pass Answers the MCP handshake 50/50
The handshake and the list calls succeeded on the primary URL.
pass Lists tools, resources or prompts 30/30
21 tools, 2 resources, 5 prompts returned.
pass 3 or more catalogue items 10/10
28 items in total; 3 or more earns these points.
pass Describes itself 10/10
92 characters; more than 30 earns these points.

Tools · 21

aimrank_get_daily_pair

Fetch today's Daily Pair — the trending matchup chosen at 00:00 UTC. Pairs with the `vote_of_the_day` agent template. Returns the challenge + a fresh matchup. If the cron hasn't run yet, falls back v…

aimrank_get_ranking

Fetch a ranking's metadata + its top 25 entities. Entities carry a Bradley-Terry rating with a 95% interval (ci_95) and a tie-aware rank: entities whose intervals overlap share a rank. api_key is op…

aimrank_get_leaderboard

Get the top N entities of a ranking with honest uncertainty. Ratings come from a Bradley-Terry fit over the counted votes, each with a 95% interval (ci_95). `rank` is tie-aware — entities the data ca…

aimrank_get_matchup

Get the next pair of entities to evaluate. Strategies: adaptive (default), swiss, balanced, random, uncertainty, explore, information_gain -- the same selection the web app uses. Left/right order is …

aimrank_search_entities

Find entities by name substring within a ranking. api_key is optional on the public endpoint; on /mcp/mcp a missing key is refused.

aimrank_create_share_card

Return share-ready URLs + embed snippets for a ranking. Formats: "all" (default — everything), "twitter", "linkedin", "embed_leaderboard", "embed_vote", "badge_markdown", "qr_code", "data_card". Pass…

aimrank_predict_matchup_winner

Predict a matchup's winner from the current ratings -- for analysis. Uses Glicko-2 win probability under the same formula matchmaking uses internally. Use it to analyse a ranking, or AFTER a vote to …

aimrank_discover_public_rankings

Browse public rankings — agent equivalent of surfing aimrank.io. Filters: category, media_type ("text" | "image" | "audio" | "video"), sort ("trending" | "recent" | "most_votes" | "random"), limit 1-…

aimrank_list_domain_entities

List all entities in a domain's pool sorted by rating. api_key is optional on the public endpoint; on /mcp/mcp a missing key is refused.

aimrank_get_domain

Get domain details including entity pool and competition list. api_key is optional on the public endpoint; on /mcp/mcp a missing key is refused.

aimrank_list_competitions

List all competitions (open voting + brackets) in a domain. api_key is optional on the public endpoint; on /mcp/mcp a missing key is refused.

aimrank_get_data_card

Auto data card — Layer 7. Provenance + IAA + coverage + per-annotator quality + transitivity + qualification stats in one synthesized markdown audit artefact. Format: 'markdown' (default) or 'json' f…

aimrank_get_agreement

Inter-annotator agreement — Layer 4. Returns Cohen's κ (n=2 raters), Fleiss' κ, Krippendorff's α, Scott's π + Landis-Koch interpretation, plus the multi-rater coverage report. `min_raters` overrides …

aimrank_get_ceiling

Pairwise-κ ceiling from qualification attempts. The upper bound any judge / RM can credibly hit on this domain — used to pin the L3 pilot refund clause (refund if κ < ceiling - 5pp). api_key is optio…

aimrank_get_plackett_luce

Plackett-Luce listwise ranking aggregation — OFFLINE AUDIT ONLY. Reported post-hoc; it NEVER writes ratings, and setting it as a ranking's algorithm returns 400. AimRank has 6 rating engines plus 2 o…

aimrank_get_crowd_bt

CrowdBT — joint fit of entity scores + per-annotator reliability η. Surfaces noisy / anti-correlated raters explicitly. Methodology audit, not a live engine. api_key is optional on the public endpoin…

aimrank_list_public_judges

Browse public judge-prompt artifacts shared by other users. Filter by ``category`` (free-text), ``domain_id``, or both. Sort options: ``fork_count`` (default — most-forked first), ``recent``, ``faith…

aimrank_find_similar_domains

Find public domains most similar to a query or a known domain. Use ``query`` for free-text search ("pharma evaluation") OR ``domain_id`` to find domains similar to one you already know. Exactly one m…

aimrank_predict

iCOMPLY contract: pairwise prediction with provenance fields. Returns the engine's predicted winner + probability shaped so iCOMPLY's Annex IV section-4 generator can ingest the response directly. Se…

aimrank_explain

iCOMPLY contract: explain a prediction in features + narrative. Returns the Glicko-2 μ/φ inputs for both entities + a deterministic narrative. No LLM-synthesised rationale — these are the actual numb…

aimrank_healthcheck

iCOMPLY contract: liveness + DB-connectivity probe.

Resources · 2

discover_rankings

Top 25 trending public rankings as JSON.

aimrank://rankings/discover.json
methodology_paper

The 7-layer AimRank preference-data quality methodology (paper v0.5); full text at https://app.aimrank.io/paper.

aimrank://methodology.md

Prompts · 5

aimrank_comparative_analysis

Run a comparative analysis: create a ranking, add candidates, collect votes, surface the leaderboard.

aimrank_run_tournament

Single-elimination tournament: collect votes via the matchmaking engine until a winner emerges.

aimrank_quality_audit

Quality audit: check IAA, per-annotator agreement, coverage, and AI-involvement on a ranking.

aimrank_start_rlhf_collection

Start an RLHF-ready preference-data collection from scratch with quality gates.

aimrank_compare_two_models

Run a blind A/B model-output battle with LLM-judge votes — for AI labs evaluating model checkpoints.

How to use

Claude Desktop or claude.ai: add a custom connector (Customize → Connectors; on Team and Enterprise plans an owner adds it under Organization settings → Connectors) with this server URL: https://app.aimrank.io/mcp/public

Cursor or Cline: add to your MCP config:

{
  "mcpServers": {
    "aimrank": {
      "url": "https://app.aimrank.io/mcp/public",
      "transport": "streamable-http"
    }
  }
}