crawl
12 MCP servers and Agent skills related to crawl, each with install commands, source and popularity data, ready to paste into Cursor, Claude Code and other clients.
Firecrawl Search
v1.0.0
io.clawhub.ashwingupy/firecrawl-search
Web search and scraping via Firecrawl API. Use when you need to search the web, scrape websites (including JS-heavy pages), crawl entire sites, or extract structured data from web pages. Requires FIRECRAWL_API_KEY environment variable.
web-search-plus
v4.3.1
io.clawhub.robbyczgw-cla/web-search-plus
Search the web and extract URLs with routing across Serper, Brave, Tavily, Querit, Linkup, Exa, Firecrawl, You.com, SearXNG, SerpBase, and Keenable. Returns ranked sources and page text; supports freshness filters and research across providers.
Tavily Search Pro
v1.0.0
io.clawhub.shaharsha/tavily-search-pro
Tavily AI search platform with 5 modes: Search (web/news/finance), Extract (URL content), Crawl (website crawling), Map (sitemap discovery), and Research (deep research with citations). Use for: web search with LLM answers, content extraction, site crawling, deep research.
Web Search Pro
v2.1.4
io.clawhub.zjianru/web-search-pro
Agent-first web search and retrieval for live web search, news search, docs lookup, code lookup, company research, site crawl, site map.
Crawl4ai Skill
v1.0.10
io.clawhub.lancelin111/crawl4ai-skill
Web crawling and scraping tool with LLM-optimized output. 网页爬虫爬取工具 | Web crawler, web scraper, spider.
Technical Seo Checker
v19.0.0
io.clawhub.aaron-he-zhu/technical-seo-checker
Use when the user asks to "check technical SEO"; audits crawlability, indexing, Core Web Vitals, robots.txt, sitemaps, canonicals, redirects.
mineru document extractor
v0.1.30
io.clawhub.mineru-extract/mineru-document-extractor
MinerU document extraction — convert PDFs, scanned documents, images, Word (DOC/DOCX), PowerPoint (PPT/PPTX), Excel (XLS/XLSX).
Firecrawler
v1.0.0
io.clawhub.capt-marbles/firecrawler
Web scraping and crawling with Firecrawl API. Fetch webpage content as markdown, take screenshots, extract structured data, search the web, and crawl documentation sites. Use when the user needs to scrape a URL, get current web info, capture a screenshot, extract specific data from pages, or crawl docs for a framework/library.
Crawl4AI Web Scraper
v1.0.1
io.clawhub.angusthefuzz/crawl-for-ai
Full web page scraping with JavaScript rendering via local Crawl4AI instance, delivering clean markdown or detailed JSON including links and media.
AnyCrawl-API
v1.0.1
io.clawhub.techlaai/anycrawl
Perform high-performance web scraping, crawling, and Google search with multi-engine support and structured data extraction via AnyCrawl API.
Firecrawl
v1.2.7
io.clawhub.byungkyu/firecrawl-api
Firecrawl API integration with managed authentication. Scrape, crawl, map, and search web content. Use this skill when users want to extract content from websites, crawl entire sites, map URLs, or search the web. For other third party apps, use the api-gateway skill (https://clawhub.ai/byungkyu/api-gateway). Calls run through the `maton` CLI with OAuth login, or over raw HTTP with a Maton API key where the CLI cannot be installed. Every call is authenticated as the user's connection and reaches only what that connection's authorization allows, which the provider enforces on every request; the endpoints documented here are the ones this skill uses, and any other endpoint of this app needs the user to ask for it by name. Default to read and list calls, and confirm every write or new connection with the user. This file also documents the three constructs that turn a Firecrawl connection into automation, in the order they are used: the connection (the first step), a hosted function that runs a Firecrawl action th
Firecrawl CLI
v1.0.0
io.clawhub.yash-kavaiya/firecrawl-cli
Web scraping, crawling, searching, and browser automation via the Firecrawl CLI (firecrawl).