GEO Optimizer — AI Search & Crawler Readiness Audit
v1.0.0
tech.geooptimizer/geo-audit
Audit any URL for GEO readiness, JSON-LD schemas, and AI crawler robots.txt permissions.
“Crawler” 共 112 个结果
v1.0.0
tech.geooptimizer/geo-audit
Audit any URL for GEO readiness, JSON-LD schemas, and AI crawler robots.txt permissions.
v0.1.96
io.github.nithin554/code-crawler
Static and dynamic call tracing MCP server backed by a SQLite property graph of your codebase
v0.1.0
ai.opencurrent/robots-ai-check
Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).
v0.1.1
io.github.kwizzlesurp10-ctrl/apexseo-geo-mcp
Keyless Base USDC x402 MCP for live AI-crawler, GEO/AEO & JSON-LD citation audits.
v1.0.0
io.github.willowswt/web-markdown-contacts-mcp
Crawl websites into LLM-ready RAG Markdown chunks and extract verified MX emails & tech stacks.
v0.7.10
io.github.mark1russell7/polite-crawler
A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.
vv18.0.3-1
io.github.schemacrawler/schemacrawler-ai
支持用自然语言查询数据库模式——快速浏览表、键、存储过程并获取 SQL 帮助。
v1.0.0
com.tidybutton/site-audit
Crawl a website from Claude and get a scored technical SEO report with a prioritised fix list.
v0.1.1
io.github.stellalytics/seo-site-audit
Technical SEO audit crawler: broken links, 404s, redirects, titles, canonicals. Runs on Apify.
v1.1.0
io.github.hamzawamiqhq/ai-crawler-check
Can ChatGPT, Claude and Perplexity crawlers reach a site? Real per-bot fetch, firewall/CDN blocks.
v1.0.0
io.github.bepolabs/web-to-markdown
Turn a website into Markdown with headings, links and word counts.
v1.0.2
com.mambabuilt/mcp-site-crawler-links-tables
Crawl a website for links, file download links, HTML tables as rows, and lists, with no AI
v1.0.0
io.github.ByteBlisk/sitereader
Website to clean Markdown or JSON for AI and RAG: read pages, crawl sites, extract described fields.
v1.0.0
io.github.prometheus-maker/website-to-markdown
Crawl a website into clean Markdown for LLMs/RAG: main content only, linked docs converted.
v1.1.0
com.atalaya-ai/ai-crawler-check
Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.
v1.0.2
io.github.kriton-dev/ai-crawler-access-check
Check declared robots.txt permissions for six AI crawlers at URL paths
v1.0.4
io.github.kriton-dev/website-to-markdown
Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay
v0.1.0
io.github.Nero-Engine/ai-crawler-access-checker
Shows if GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can read each site in a list.
v0.1.3
com.webcrawlerapi/mcp-server
Turn any web page into clean Markdown for your AI assistant. OAuth sign-in, no API key.
v0.3.1
io.github.crawlerpeek/crawlerpeek
Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.
v1.0.0
com.crawlerzoo/arena
Games for AI agents: negotiation, social deduction, racing and puzzles, vs bots and agents.
v1.8.0
app.openkrill/site-check
Check public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion.
v1.0.1
io.github.alphaparkinc/genpark-bloom-filter-url-dedup-crawler-skill
In-memory Bloom filter for URL and document hash deduplication preventing redundant web crawler inge
v0.1.4
io.github.missiongrowth/unrendered-ai-crawler-checker
AI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.