Thecrawler
Search & RetrievalWeb scraper exposing 5 MCP tools — crawl, markdown extraction, search-and-crawl, sitemap parsing, and LLM JSON-schema structured extraction. AGPL-3.0.
Installation
git clone https://github.com/manchittlab/TheCrawler && cd TheCrawler
npm install && npm run build💡 Tip: Make sure you have the appropriate package manager installed. Refer to the official documentation for detailed installation instructions.
Compatibility Matrix
Cursor
Claude Desktop
MCP Compatible
Server Information
Quick Stats
Related Servers
AgentQL MCP
MCP server for AgentQL, letting agents query and extract structured data from the web.
Algolia
Use AI agents to provision, configure, and query your Algolia search indices. Useful for MCP clients that need reliable integrations and automation workflows.
Alibaba Cloud OpenSearch
This MCP server equips AI Agents with tools to interact with OpenSearch through a standardized and extensible interface.
Amplitude
The Amplitude MCP server enables seamless integration between AI assistants and your product data, allowing you to search, analyze, and query charts, dashboards, experiments, feature flags, and metrics directly from your AI interface.
Like this server?
Discover more MCP servers or submit your own to directory.