🕷️网页抓取 MCP 推荐

Gives AI agents a public IPv6 identity, hostname, port forwarding, web fetch, team mesh. Free tier.

Publish HTML or Markdown to a public URL in one call. Zero config, auto-registers on first use.

Safe, self-hosted web grounding for AI agents and crawlers over a stealth browser

Metalift MCP for AI agents: web search (2 credits), scrape, crawl, and map tools

Parse PDF/Word/PPT/HTML to Markdown; tables as JSON, image extraction, RAG chunking, page ranges.

Web scraping to clean Markdown with JS rendering, multi-page crawl, structured extract, sitemaps.

Autonomous web crawler. 17 browser tools + goal-driven run_goal agent. Single binary, stealth.

Bulk Bluesky profiles plus full follower/following exports via the open AT Protocol. Pay per record.

Track brand mentions & keywords on Bluesky. Sentiment, engagement, author reach. Pay per result.

AEO tools for AI agents: crawler permissions, llms.txt, structured data, AI-readiness audit.

Deterministic AI agent microtools, no accounts/API keys. fetch_extract: 98% token cut. 38 tools.

One-link team polls: create polls, vote, fetch results, and close polls.

Weekly 5-15 reports for executive teams: submit reports, fetch the Friday briefing.

Scrape, crawl, and map websites to Markdown or JSON via local CLI.

Find & fetch research datasets across 12 archives, omics registries, and literature sources.

HSM-backed vault secrets for AI agents (JIT fetch) plus prompt-injection and threat scanning.

Look up genes, fetch sequences, predict variant consequences, find orthologs and xrefs via Ensembl.

smartfetch direct-URL fetch, smartsearch web discovery, smartcrawl sites, and docs export MCP.

Backlink lookups and competitor gap analysis on Common Crawl's open web graph (4.4B links).

Social-video URL → transcript, frames, OCR text & metadata across 6 platforms. MCP, x402.

Search brands and retrieve design assets, company data, other brand context from Brandfetch's API

List, fetch and search AI Daily Insights — a Chinese daily AI-industry briefing built for agents.

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Search, scrape, and crawl the web for AI agents. Batch scraping and answers with citations.

Fetch your Trackman Golf stats (handicap, rounds, shots, club gapping) as MCP tools.

Web scraping for AI agents: fetch any page via proxy or full browser, past anti-bot defenses.

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

Scrape Shopify App Store apps, full review histories and catalog. E-commerce market research.

Scrape Bluesky posts, profiles, followers, threads and keyword search. Clean JSON, pay per result.

Deterministic web tools for AI agents: read URLs, metadata, email, scrape, RSS, DNS, SSL, HTTP.

HTML → text MCP.

RU INN/OGRN, banks, geo, WHOIS. Agent self-registers via register_agent. 20 free/day.

Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits.

Deploy HTML from any AI tool to a versioned, shareable library on wovo.dev; manage pages.

Search EUR-Lex, fetch parsed EU legislation and case law, explore the citation graph.

HTML entities MCP.

Find NOAA tide stations and NDBC buoys, fetch tide predictions, currents, and live conditions.

Save, version & share AI-generated HTML artifacts (canvases, dashboards, apps) as links.

Enables natural language schema queries — explore tables, keys, procedures, and get SQL help fast

Search PubMed/Europe PMC, fetch articles and full text (PMC/EPMC/Unpaywall), citations, MeSH terms.

Deploy HTML and Markdown as live URLs. Create, search, update, and manage pastes from any AI agent.

MCP server for Firecrawl — search, scrape, and interact with the web.

Read-only X/Twitter MCP: posts, threads, replies, search via twscrape. No API key.

Ask about a YouTube video or fetch its transcript. Paid per call in USDC on Solana via x402.

Scrape the FAA Aircraft Registry by N-number — aircraft make, model, year, owner name, address, airw

Turn HTML or templates into branded invoice & document PDFs from your agent. One call, no install.