Back to search
30
MCP

TheCrawler

io.github.manchittlab/thecrawler

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Uptime
0.0%
34 probes
Response
0ms
last probe
Tools
0
callable
Primary URL
—

Similar MCP servers embedding-nearest

Web Scraper
Extract clean Markdown from any URL, removing boilerplate for use in RAG pipelines.
0 tools
MarkGrab
Universal web content extraction — any URL to LLM-ready markdown. HTML, YouTube, PDF, DOCX.
0 tools
MarkGrab
Universal web content extraction — any URL to LLM-ready markdown. HTML, YouTube, PDF, DOCX.
0 tools
Crawlberg
Scrape, crawl, and map websites to Markdown or JSON via local CLI.
0 tools
WebReaper
AI-native web scraper: scrape, crawl and map any site to clean markdown over stdio. MIT-licensed.
0 tools
Scraper
Extracts content from websites through four core tools for HTML scraping, markdown conversion, text extraction, and link discovery with batc
0 tools

How to use

Add to your Claude Desktop / Cursor / Cline MCP config:

{
  "mcpServers": {
    "thecrawler": {
      "url": "None",
      "transport": "http"
    }
  }
}