ActableSite AI 抓取监控
v1.4.1
com.actablesite/readiness
检查 AI 爬虫的 robots.txt 策略,并监控公开站点的策略、sitemap 与 llms.txt 变化。
“Crawl” 共 131 个结果
v1.4.1
com.actablesite/readiness
检查 AI 爬虫的 robots.txt 策略,并监控公开站点的策略、sitemap 与 llms.txt 变化。
v1.0.0
io.github.im-ff/crawlinx-mcp
Free technical-SEO audit MCP: crawl a site, run checks, return an LLM-ready shareable report.
v0.1.0
app.crawlie/crawlie
技术 SEO 与 GEO(AI 搜索)站点审计:托管抓取、按优先级排序的修复项与报告对比。
v0.2.1
io.github.shulex-voc/smart-crawler
电商竞品情报、VOC 情感分析、Amazon ASIN 分析与 Reddit 增长方案。
v0.1.0
com.crawlconsole/crawlconsole
面向 AI 智能体的反向链接分析 MCP:权威度、引荐域名与竞品链接缺口。
v1.0.0
io.github.lazymac2x/ai-crawler-policy
部署在 Cloudflare Workers 上的 MCP 服务器:ai-crawler-policy。
v2.0.0-rc
io.github.DevsHero/shadow-crawl
隐蔽抓取与搜索,通过 Cyborg HITL 方式绕过 Cloudflare、DataDome 与 LinkedIn。
v1.0.0
io.github.Crawleo/Crawleo-MCP
托管版 Crawleo MCP(远程可流式 HTTP 端点)。
v2.4.0
dev.workers.b2bintel-edge.agent-web-x402-gateway/web-scrape-search-firecrawl-x402-mcp
Keyless Firecrawl/Exa alternative: URL to LLM Markdown, live web search & $4.90 30d pass.
v1.0.0
io.github.saulius876-lgtm/website-content-crawler
Web pages and whole sites to clean Markdown for RAG and AI agents. Crawls links and sitemaps.
v1.1.0
com.perfbee/mcp
Crawl your sites and read broken links, SEO and AI readiness issues with their HTML element.
v1.2.1
app.seobolt/seobolt
Crawl a site, audit Google and AI-search readiness, get a fix plan your coding agent applies.
v1.0.1
dev.datasetworks/website-seo-audit
Crawl a site for SEO issues per page and broken links. Respects robots.txt.
v0.18.0
io.github.dtkmn/mcp-zap-server
安全、自托管的 OWASP ZAP 操作器,用于引导式 AI 安全扫描与报告。
v1.0.0
io.github.lintlab/seo-site-qa
Crawl a site for broken links, redirects and missing SEO tags. Pay per page on Apify.
v1.1.0
io.github.pavel-tashev/broken-link-checker
Crawl a site for broken links, 404s, dead images, redirect chains and slow pages, with sources
v1.0.1
one.aisa/seo-onpage
Crawl a site: pages, resources, links, duplicates, tags, Lighthouse and raw HTML.
v1.0.0
cloud.spider/mcp
大规模爬取、抓取与搜索网页,并提供带反爬破解能力的浏览器自动化。
v1.0.0
io.github.abhibavishi/rankparse-mcp
SEO MCP 服务器——反链、域名权重、技术栈识别,基于 Common Crawl 的 18 个以上工具。
io.smithery.yakenator.web-inspector
分析单个网页乃至整站在 SEO、可访问性、性能与安全合规方面的表现。生成详细报告与可执行的问题清单,改善站点质量与用户体验。可跟踪历史审计数据,在多次抓取之间监测改进情况。
v0.4.3
io.github.atakanelik34/clawifi-mcp
Web search and page reading for AI agents: search, fetch, scrape, map and crawl, with sources.
v0.9.52
io.github.NovadaLabs/novada-mcp
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
v0.11.13
io.github.quantumproxies/quanticdata-mcp
Web data and proxies for agents: scrape to Markdown, SERP, crawl/map, unlocker, collectors, proxies.
v0.7.10
dev.mesharc/mesharc
Search the web, crawl and scrape websites to clean markdown, and track what changed between crawls.