搜索「crawl scrape firecrawl fetch」的结果

Firecrawl

io.mcp-cn.firecrawl-mcp

基于Firecrawl的智能网页爬取和内容提取MCP工具

com.mcparmory/firecrawl

大规模抓取、爬取网页并提取结构化数据

io.github.fetchsandbox/mcp

面向 agent 的确定性验证引擎：为修复提供证明——旧代码上失败、新代码上通过。

io.github.imfurkana/unrendered-ai-crawler-checker

AI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.

io.github.mysleekdesigns/crawlforge-mcp-server

网页抓取、遍历、深度调研与自主信息抽取，31 个 MCP 工具，输出干净的 Markdown/JSON。

io.github.xberg-io/crawlberg

通过本地 CLI 抓取、爬取并映射网站，输出 Markdown 或 JSON。

io.github.nolindnaidoo/scrape-le

Analyse robots.txt content and report whether a path may be crawled.

io.github.firecrawl/firecrawl-mcp-server

Firecrawl 的 MCP server——网页搜索、网页抓取，以及生物医学/arXiv 论文检索。

io.github.maximilianfeix/proxy-scraper

Free proxies that actually work: verified HTTP/SOCKS proxies, and pages fetched through them

xyz.atlasfetch/atlasfetch

Reverse geocode a coordinate to country, region, municipality and street, plus your own geofences.

io.github.Mingye-Lu/acrawl

自主网页爬虫：17 个浏览器工具，外加目标驱动的 run_goal 智能体；单二进制、隐身模式。

io.github.fastcrw/crw

Open-source web scraper for AI agents with scrape, crawl, and map tools

com.scrapeunblocker/scrapeunblocker-mcp

通过 ScrapeUnblocker 的反爬 API 获取任意网页的 HTML 或 AI 解析后的 JSON。

io.github.Grubbomatic/crawl-readiness

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

io.github.schemacrawler/schemacrawler-ai

支持用自然语言查询数据库模式——快速浏览表、键、存储过程并获取 SQL 帮助。

io.github.scrape-badger/scrapebadger

面向 AI 智能体的 Twitter/X 抓取 API：获取个人资料、推文、热门趋势等。

io.github.YawLabs/fetch-mcp

HTTP 抓取 MCP 服务器：SSRF 防护、HTML 转 Markdown、阅读模式、元数据提取。

io.github.Ozymandias-Owens-2/scrapewright

Give it a URL, get structured rows. A model writes the parser once; replays are free.

io.github.nickjlamb/pubcrawl

PubMed 与 Europe PMC 文献检索、美英药品说明书查询与临床试验检索，供 AI 助手使用。

io.github.harshmaur/reddit-scraper-pro

Reddit posts, comments, search, users & subreddits as JSON via Apify. No Reddit API key. $1.50/1k

io.github.mrfentmen/commoncrawl-mcp

Use this MCP server to Common Crawl index discovery for historical web captures.

io.github.SanggonBoy/PyreCrawl

Web scraping + research toolkit for AI agents: scrape, crawl, monitor URLs with anti-bot bypass.

io.github.jmshinhwa/robots-txt-ai-crawler-audit

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

io.github.us/crw

面向 AI 智能体的开源网页抓取器，提供抓取、爬取与站点结构梳理工具。