Back to search
100
MCP live MCP 2026-07-28 streamable-http

Robots.txt Policy Lint

dev.workers.pathwren.www/robots-policy-lint

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

Watch this dependency Share evidence report
Uptime
100.0%
5 direct probes · 30d
Response
236ms
last probe
Tools
9
callable
Resources
3
readable
Prompts
1
available
Primary URL
https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots
Score: 100/100
Handshake verified by our own probe.
pass Answers the MCP handshake 50/50
The handshake and the list calls succeeded on the primary URL.
pass Lists tools, resources or prompts 30/30
9 tools, 3 resources, 1 prompts returned.
pass 3 or more catalogue items 10/10
13 items in total; 3 or more earns these points.
pass Describes itself 10/10
100 characters; more than 30 earns these points.

Tools · 9

no_arguments_lint_this_hosts_robots_txt

TAKES NO ARGUMENTS. POST {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} to https://www.pathwren.workers.dev/mcp/robots — the …

am_i_allowed

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"am_i_allowed","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and the answer is what this host's …

whoami

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that to https://www.pathwren.workers.dev/mcp/robots and the answer is the user-agent you sent, the class…

example

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and you are done: free, read-only, no AP…

lint_robots_txt

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Parse a robots.txt you paste and report every fau…

check_path_allowed

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate a pasted robots.txt for one crawler and …

audit_ai_access

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate a pasted robots.txt against every AI cra…

diff_robots_txt

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Compare two versions of a robots.txt and report o…

merge_policy

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Merge one of eight maintained robots.txt stances …

Resources · 3

ai-crawler-index

Every indexed AI crawler with its robots token, operator, category and whether it documents obedience.

https://www.pathwren.workers.dev/data/agents.json
stance-block-ai-training

The block-ai-training robots.txt, as served. The other seven are at /robots/<stance>.txt and are described at /policy/<stance>.json.

https://www.pathwren.workers.dev/robots/block-ai-training.txt
changes.json

The since-cursor feed for everything this host publishes: operator IP-range lists that gained or lost prefixes, upstreams that failed or recovered, crawler records added or edited. Read `cursor` from …

https://www.pathwren.workers.dev/changes.json?since=0

Prompts · 1

start_here

Takes no arguments. Safe to call. Deterministic. Touches no third party. The first call worth making against robots-policy-lint: run whoami (no arguments) to see what this host can tell you about your…

How to use

Claude Desktop or claude.ai: add a custom connector (Customize → Connectors; on Team and Enterprise plans an owner adds it under Organization settings → Connectors) with this server URL: https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots

Cursor or Cline: add to your MCP config:

{
  "mcpServers": {
    "robots.txt_policy_lint": {
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots",
      "transport": "streamable-http"
    }
  }
}