Back to search
30
MCP

TheCrawler

io.github.manchittlab/thecrawler

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Uptime
0.0%
3 probes
Response
0ms
last probe
Tools
0
callable
Primary URL
—

Similar MCP servers embedding-nearest

MarkGrab
Universal web content extraction — any URL to LLM-ready markdown. HTML, YouTube, PDF, DOCX.
0 tools
Document Crawler & Search
Crawls websites to generate searchable Markdown documentation with vector embeddings for semantic search capabilities beyond training cutoff
0 tools
Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
0 tools
n8n-nodes-webcrawlerapi
n8n node for WebCrawler API integration - a website to markdown API for LLM and RAG
0 tools
webclaw
Web extraction MCP server. Scrape, crawl, extract, summarize any URL to clean markdown.
0 tools
webclaw
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
0 tools

How to use

Add to your Claude Desktop / Cursor / Cline MCP config:

{
  "mcpServers": {
    "thecrawler": {
      "url": "None",
      "transport": "http"
    }
  }
}