🕷️网页抓取 MCP 推荐

Firecrawl

vlatest

基于Firecrawl的智能网页爬取和内容提取MCP工具

非官方的DeepWiki MCP服务器，通过MCP接收DeepWiki URL，爬取所有相关页面并转换为Markdown格式，可返回单个文档或按页面列表返回

Apify Actors MCP服务器，提供与各种Apify Actor的交互能力，包括网页抓取、数据提取、社交媒体监控等功能

paper-search-mcp-openai-v2

vlatest

Find and download academic papers from leading sources like arXiv, PubMed, bioRxiv, medRxiv, Google Scholar, Semantic Scholar, CrossRef, and IACR. Get standardized results and fetch full-text PDFs when available. Accelerate literature reviews with deep search and effortless retrieval.

News MCP — AP, BBC, NPR, HN, Google News Aggregator

vlatest

Five independent news sources (AP, BBC, NPR, Hacker News, Google News) behind one MCP interface. Fetch headlines on a topic across all sources, dedupe by URL, surface the underreported angles. Useful for daily briefings, research agents, and sentiment-shift detection.

pubmed-mcp-server

vlatest

Search PubMed/Europe PMC, fetch articles and full text (PMC/EPMC/Unpaywall), citations, MeSH terms.

framefetch

vlatest

# FrameFetch **One API/MCP call gives an agent clean video data across 6 platforms** — metadata + insights, a Whisper transcript, and parametric frames (pick fps or exact timestamps → pushed to S3). YouTube (incl. Shorts), TikTok, Reddit, Instagram, Pinterest. Agent-first: typed errors, refund-on-fail, result caching. Pay per call via x402 (USDC on Base) or Stripe. ## Endpoints - POST /v1/extract — any combination of metadata/insights/transcript/frames in one call - POST /v1/metadata · /v1/transcript · /v1/frames — shortcuts - GET /v1/platforms — capability matrix · POST /v1/keys — free key + credit ## Example curl -X POST https://framefetch.net/v1/extract -H "Authorization: Bearer <key>" -H "Content-Type: application/json" -d '{"url":"https://youtu.be/...","fields":["metadata","transcript"]}' https://framefetch.net

Keenable Web Search

vlatest

Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key. Tools: - search_web_pages: Search the web and return ranked results. - fetch_page_content: Fetch an indexed URL and extract its content as clean markdown

arxiv-mcp-server

vlatest

Search arXiv, fetch paper metadata, and read full-text content.

Web Auditor

vlatest

Analyze web pages and entire sites for SEO, accessibility, performance, and security compliance. Generate detailed reports and actionable issue lists to improve site quality and user experience. Track historical audit data to monitor improvements across multiple crawl sessions.

Google Maps Lead Gen MCP — Local Business Enrichment

vlatest

B2B lead generation tool: search Google Maps by 'plumbers in Austin', get back business profiles with emails, phone numbers, ratings, websites. The differentiator over plain Google Maps API is the contact-enrichment layer (scraped from each business's site). Built for agency prospecting workflows.

run.pay — Agent Marketplace

vlatest

Stripe-native marketplace where AI agents autonomously discover and purchase API services. Pay-per-call, no accounts needed. 6 live services: Phone Validator, Web Scraper, PDF Generator, Screenshot API and more. Built on Stripe Agent Toolkit.

Reliable web fetching MCP server with built-in retry logic, circuit breaker patterns, caching, and anti-bot bypass. Fetches URLs as raw HTML or clean markdown optimized for LLM consumption. Includes domain health checks and cache management tools.

Meta-Stamp Pockets

vlatest

Licensed creator content for AI agents. Discover music, video, and text catalogs as pre-indexed Pockets — a few hundred tokens instead of a 10,000-token scrape, delivered in under 30ms. Free discovery; pulls are licensed per-use via HTTP 402 paywall — connection is the contract, and 85% of every pull pays the rights holder instantly. **Tools:** `list_pockets` · `search_pockets` · `pull_content` No key needed to browse. Pulling content returns a 402 with signup instructions — register at the URL provided, attach your API key as a Bearer token, and pull licensed content with full attribution and audit trail.

Twitter/X Scraper — Profiles, Tweets & Search

vlatest

Twitter/X scraping API for AI agents. Scrape public profiles (bio, stats, verification), user tweets (text, engagement, media), and search results — all without needing a Twitter API key. Structured JSON output ready for analysis. Tools: twitter_scrape_profile, twitter_search_tweets, twitter_get_user_tweets. Use this for social media monitoring, influencer research, sentiment analysis, competitive intelligence, or building social dashboards. The social intelligence layer for AI agents. IMPORTANT: For other social networks, use social_lookup_profile. Returns structured JSON with profiles, tweets, engagement metrics. No API key required — x402 micropayment $0.005/call on Base L2.

RSS Reader

vlatest

Track and browse RSS feeds with ease. Fetch the latest entries from any feed URL and extract full article content for clean, distraction-free reading. Save or analyze articles in a consistent, structured format.

Markdown Renderer — Styled HTML with CSS Themes

vlatest

Markdown rendering API for AI agents. Convert Markdown to fully styled HTML with CSS themes: light, dark, and GitHub styles. Ready-to-embed HTML documents with responsive typography. Tools: text_render_markdown. Use this for generating presentable HTML from Markdown: reports, documentation, blog posts. IMPORTANT: For basic Markdown→HTML without styling, use text_convert_markdown_to_html (cheaper). Returns: {html, theme, wordCount}. No API key required — x402 micropayment $0.002/call on Base L2.

WorldMonitor MCP

vlatest

Live global-intelligence data as a 39-tool MCP server: real-time markets, conflict events, country risk and resilience scores, shipping chokepoints, maritime and aviation activity, energy, climate, health, cyber threats, sanctions, forecasts, and AI-generated world/country briefs. Discovery is anonymous — connect and issue tools/list with no key. Every tool supports JMESPath projection so agents fetch exactly the fields they need. Data calls use OAuth2 or an API key from worldmonitor.app/pro.

Email Sender — Transactional Email via API

vlatest

Email sending API for AI agents. Send transactional emails programmatically — text or HTML body, custom subject, from address, delivery status confirmation and message ID. Powered by Resend. Tools: email_send_message. Use this when your agent needs to send notifications, reports, alerts, or outreach emails. IMPORTANT: Verify recipient with email_verify_address BEFORE sending to avoid bounces. Returns: {id, status, from, to, subject}. No API key required — x402 micropayment $0.003/call on Base L2.

DeerDawn

vlatest

AI session memory: the brief your AI reads before every session so no session starts cold. Loads your project's current task, recent decisions, and open questions across Claude, ChatGPT, Cursor, and Codex.

Paper Search

vlatest

Search and download academic papers from arXiv, PubMed, bioRxiv, medRxiv, Google Scholar, Semantic Scholar, and IACR. Fetch PDFs and extract full text to accelerate literature reviews. Get consistent metadata for easier filtering, citation, and analysis.

Markdown to HTML — Headings, Code, Tables

vlatest

Markdown to HTML converter API for AI agents. Convert Markdown to clean HTML with headings, lists, code blocks, tables, links, and images. Optional full document wrapping with DOCTYPE. Tools: text_convert_markdown_to_html. Use this for rendering Markdown content in web UIs, email templates, or document generation. IMPORTANT: For styled HTML with CSS themes, use text_render_markdown instead. Returns: {html, wordCount}. No API key required — x402 micropayment $0.001/call on Base L2.

Sports MCP — NBA, NFL, NHL Live & Historical

vlatest

Bring sports data into an AI workflow without writing four ESPN scrapers. Useful for fantasy-sports copilots, sports-betting research agents, content sites that auto-summarize game results, or hobby projects where you want Claude to talk to you about last night's game.

Context7

vlatest

Fetch up-to-date, version-specific documentation and code examples directly into your prompts. Enhance your coding experience by eliminating outdated information and hallucinated APIs. Simply add `use context7` to your questions for accurate and relevant answers.

HTML to Markdown — Clean Conversion, Scripts Stripped

vlatest

HTML to Markdown converter API for AI agents. Convert HTML to clean Markdown: strips scripts, styles, and tracking, preserves headings, links, lists, images, and tables. Tools: text_convert_html_to_markdown. Use this for content migration, email processing, or preparing HTML content for LLM consumption. IMPORTANT: For scraping a live URL to Markdown, use web_scrape_to_markdown instead. Returns: {markdown, wordCount}. No API key required — x402 micropayment $0.001/call on Base L2.

PDF Generator — HTML/Markdown to Professional PDFs

vlatest

PDF document generation API for AI agents. Generate PDFs from HTML or Markdown: custom page size (A4, Letter), margins, headers, footers. Ideal for reports, invoices, contracts, and documentation. Tools: document_generate_pdf. Use this for generating professional documents, reports, or invoices. IMPORTANT: For capturing existing web pages as PDF, use webpage_to_pdf instead. Returns: base64-encoded PDF document. No API key required — x402 micropayment $0.008/call on Base L2.

ByteFun AI MCP服务 - 打通产品设计、UI设计、代码开发的服务平台，支持设计稿转代码和跨平台原生代码开发。提供两大核心功能模块：UI设计优化 - 将用户输入的需求转换为带有UI设计规范的静态HTML+CSS设计提示词，实现设计稿到代码的转换；代码开发优化 - 将用户输入的需求转换为带有完整开发规则的TypeScript代码开发提示词，支持跨平台原生开发。

scrapegraphai

vlatest

The ScrapeGraphAI MCP Server is a production-ready Model Context Protocol (MCP) server that connects Large Language Models (LLMs) to the ScrapeGraph AI API. It enables AI assistants like Claude and Cursor to perform AI-powered web scraping, research, and crawling directly through natural language interactions.

AgentScrape

vlatest

**Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC.** AgentScrape is a production MCP server giving agents six paid tools for web scraping, structured extraction, screenshots, and metadata — all on Cloudflare Workers, all settled in USDC on Base via x402. ## Tools - **`scrape_webpage`** — fetch as markdown, html, text, or json - **`extract_structured_data`** — AI-powered extraction with natural-language prompts + JSON schemas - **`screenshot_webpage`** — PNG capture, viewport configurable - **`extract_metadata`** — OG, Twitter Card, JSON-LD - **`create_browser_session`** — persistent sessions for multi-step flows - **`run_workflow`** — multi-step atomic execution ## Pricing $0.001–$0.008 USDC per call on Base mainnet. No subscriptions. No API keys. Pay per request via x402. **Free tier:** 10 calls per wallet per 30 days. No signup required. ## Stack Cloudflare Workers · Hono · `@x402/hono` v2 · MCP Streamable HTTP · xpay.sh facilitator · Browser Rendering MIT licensed. Buil

Search, answer, and fetch Tako's knowledge base of 100K+ data charts, plus deep research agents.

A home for your agent's HTML artifacts - publish, pull, share and review them over MCP.

Deploy HTML/React artifacts from your AI client to hosted, share-gated URLs. Private by default.

Publish HTML, files, or a URL to a permanent public URL, then update it — from any MCP agent.

Etix event discovery for Claude — search events, venues & performers and fetch event details

Read-only Workday for Claude — fetch tasks, pay, benefits via your signed-in session

Generate HTML to PDF documents in bulk or single — raw replacements or based on conditions, loops

Human visual review for agent-built HTML: publish artifacts, pull anchored comments, push fixes.

Open-source web scraper for AI agents with scrape, crawl, and map tools

400 professional Agent Skills + workflow recipes — searchable & fetchable over MCP.

Host the HTML pages your AI generates and share each as a link with comments and access control.

Stealth browser for AI agents: fetch pages behind Cloudflare/DataDome/CAPTCHA, extract clean data.

Read-only MCP server for the OrchestKit docs: full-text search + Markdown fetch. No auth.

Pure-Go browser for AI: fetch pages, render JS without Chromium, return clean Markdown over MCP.

Inspect a site's AI-search readiness: AI crawler access, llms.txt, schema markup, meta directives

Call 120+ AI models and live web context (scrape/crawl/extract) from any MCP client with one key

MCP server for SearXNG with engine targeting, multi-page fanout, and HTML→Markdown URL reading.