← Back to all tags

crawler Servers

10 servers tagged with "crawler"

Mcpdocsearch

This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.

Search & RetrievalMIT

Aio Mcp

πŸš€ All-in-one MCP server with AI search, RAG, and multi-service integrations (GitLab/Jira/Confluence/YouTube) for AI-enhanced development workflows. Folk from https://github.com/nguyenvanduocit/all-in-one-model-context-protocol

Ai Agentsunknown

Infomesh

πŸ•ΈοΈ Fully decentralized P2P search engine for LLMs β€” no API key, no billing, forever free. MCP-native web search via Kademlia DHT + libp2p + FTS5. Production stable βœ… β†’ pip install infomesh

Developer ToolsMIT

Mcp Server Requests

Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Headers, LLM-Accessible Response Headers, and more.

Developer ToolsMIT

Mcp Leetcode Crawler

mcp server for craw the interview questions in discussion

Search & Retrievalunknown

Better Fetch

A powerful Model Context Protocol (MCP) server that intelligently fetches and processes web content with nested URL crawling capabilities. Transform any documentation site or web resource into clean, structured markdown files perfect for AI consumption and analysis.

Search & Retrievalunknown

Web Scout Mcp Suebksnn

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Search & RetrievalMIT license

Crawl4ai Mcp

MCP (Model Context Protocol) server wrapping Crawl4AI library as functions using Python

Search & RetrievalMIT

Webpeel

Web data API that just works. Pass any URL, get clean markdown β€” Cloudflare, JavaScript, CAPTCHAs handled automatically. No config needed. 97.6% success rate across 500+ domains. Free tier included.

Search & Retrievalunknown

Mcp Fouradata

Fouradata Mcp is an MCP server on MCP.so β€” view details, tools, and install instructions.

Developer Toolsunknown