Firecrawl MCP Setup

What You’ll Learn

This guide shows you how to integrate Firecrawl with Junis using the Firecrawl MCP Server. Your agents will be able to:

Scrape single web pages or batch process multiple URLs
Crawl entire websites recursively
Search the web and extract content from results
Extract structured data using JSON schemas
Map website structures and discover all pages

Prerequisites:

Firecrawl account and API key (Get one here)
Admin role in Junis (for organization-level setup)
Basic understanding of web scraping concepts

Quick Setup (5 Minutes)

Get Firecrawl API Key

Visit Firecrawl Dashboard
Sign up or log in
Click “Create API Key”
Copy your API key (starts with fc-)

Add Firecrawl Platform to Junis

Navigate to Team > MCP in Junis and find the Firecrawl card.If Firecrawl is already configured (globe icon 🌍), skip to Step 3.Otherwise, click “Connect” and fill in:

Platform Name: Firecrawl
MCP Server URL: https://api.firecrawl.dev/mcp/
Transport Type: Streamable HTTP

Add Your Credentials

Click “Add Auth” on the Firecrawl card and paste your API key.Click “Test Connection” to verify it works.

Enable for Your Agents

Go to Admin > Agents, edit the agent you want to connect, and check “Firecrawl” in the MCP Platforms section.Save and test by asking: “Scrape the homepage of https://example.com”

Available Tools (8 Tools)

Firecrawl MCP provides 8 powerful web research tools:

Content Extraction

firecrawl_scrape

Single Page ScrapingExtract content from a single URL in markdown or HTML format.Example: “Scrape the content of https://blog.example.com/post”

firecrawl_batch_scrape

Batch ScrapingScrape multiple URLs simultaneously.Example: “Scrape these 5 product pages: URL1, URL2, …”

Site Discovery

firecrawl_map

Website MappingDiscover all indexed URLs on a website.Example: “Map all pages on example.com”

firecrawl_search

Web Search + ExtractSearch the web and extract content from top results.Example: “Search for ‘AI trends 2026’ and summarize top 5 results”

Advanced Operations

firecrawl_crawl

Recursive CrawlingCrawl an entire website recursively.Example: “Crawl all product pages on example.com”

firecrawl_extract

Structured Data ExtractionExtract data using JSON schemas for consistent formatting.Example: “Extract price, stock, and reviews from this product page”

Status Monitoring

firecrawl_check_batch_status

Batch StatusCheck progress of batch scraping jobs.

firecrawl_check_crawl_status

Crawl StatusMonitor progress of crawling operations.

Common Workflows

1. Single Page Analysis

Use Case: Analyze a blog post or article Workflow:

User: "Analyze the content of this blog post: [URL]"
Agent uses firecrawl_scrape to get the content
Agent summarizes and analyzes the text

Agent Instruction:

When asked to analyze a web page:
Use firecrawl_scrape to get the content
Focus on the main content (set onlyMainContent: true)
Provide a concise summary and key insights

2. Competitor Research

Use Case: Analyze multiple competitor websites Workflow:

User: "Research our top 5 competitors' pricing pages"
Agent uses firecrawl_batch_scrape with 5 URLs
Agent uses firecrawl_extract to get structured pricing data
Agent compares and creates a report

Agent Instruction:

For competitor research:
Use batch_scrape for multiple URLs
Extract key information like pricing, features, target audience
Create a comparison table
Highlight our advantages and areas for improvement

3. Content Discovery

Use Case: Find all blog posts on a website Workflow:

User: "Find all blog posts on example.com"
Agent uses firecrawl_map to discover all URLs
Agent filters URLs containing "/blog/"
Agent uses batch_scrape to get content
Agent categorizes and summarizes posts

Agent Instruction:

To discover content on a website:
Use firecrawl_map to get all URLs
Filter relevant URLs based on patterns
Use batch_scrape for multiple pages
Organize content by topic or date

4. Structured Data Collection

Use Case: Collect product information from e-commerce sites Workflow:

1. User: "Extract product details from these pages"
2. Agent uses firecrawl_extract with JSON schema:
   {
     "name": "string",
     "price": "number",
     "stock": "boolean",
     "rating": "number"
   }
3. Agent returns structured data

Agent Instruction:

For structured data extraction:
Define a clear JSON schema for the data
Use firecrawl_extract with the schema
Validate extracted data for completeness
Format results in a table or JSON

Tool Parameters Explained

firecrawl_scrape

Parameter	Type	Description	Example
`url`	string	Required. URL to scrape	`https://example.com`
`formats`	array	Output format(s)	`[{"type": "json", "prompt": "..."}]` (recommended)
`onlyMainContent`	boolean	Extract only main content	`true`
`includeTags`	array	HTML tags to include	`["article", "main"]`
`excludeTags`	array	HTML tags to exclude	`["nav", "footer"]`

firecrawl_batch_scrape

Parameter	Type	Description	Example
`urls`	array	Required. List of URLs	`["url1", "url2", "url3"]`
`options`	object	Scraping options	Same as `firecrawl_scrape`

firecrawl_search

Parameter	Type	Description	Example
`query`	string	Required. Search query	`"AI trends 2026"`
`limit`	number	Max results to return	`5`
`lang`	string	Language code	`"en"` or `"ko"`
`country`	string	Country code	`"US"` or `"KR"`

firecrawl_crawl

Parameter	Type	Description	Example
`url`	string	Required. Starting URL	`https://example.com`
`maxDepth`	number	Max crawl depth	`3`
`limit`	number	Max pages to crawl	`100`
`allowExternalLinks`	boolean	Follow external links	`false`
`includePaths`	array	URL patterns to include	`["/blog/*"]`
`excludePaths`	array	URL patterns to exclude	`["/admin/*"]`

firecrawl_extract

Parameter	Type	Description	Example
`urls`	array	Required. URLs to extract from	`["url1", "url2"]`
`prompt`	string	Extraction instructions	`"Extract product details"`
`schema`	object	JSON Schema for data	`{"name": "string", "price": "number"}`
`enableWebSearch`	boolean	Use web search	`false`

Example Use Cases

Web Research Agent

Agent Prompt:

You are a web research specialist. When users ask you to research a topic:
Use firecrawl_search to find relevant sources
Use firecrawl_scrape or batch_scrape to get full content
Analyze and synthesize information
Provide citations with URLs
Summarize key findings in bullet points

Connected MCP: Firecrawl + Notion (for saving research) Example Interaction:

User: "Research the latest developments in quantum computing"
Agent: [Uses firecrawl_search("quantum computing 2026", limit=10)]
       [Uses firecrawl_batch_scrape on top 5 results]
       "Here are the latest developments in quantum computing:
       - IBM announced a 1000+ qubit processor (Source: [URL])
       - Google achieved quantum advantage in optimization problems (Source: [URL])
       - New error correction methods show promise (Source: [URL])
       ..."

Content Monitoring Agent

Agent Prompt:

You monitor competitor websites and news sources for updates.
- Check specific URLs daily
- Compare new content with previous versions
- Highlight changes and new information
- Alert team if significant updates detected

Connected MCP: Firecrawl + Slack (for notifications) Example Interaction:

User: "Check if competitor X has updated their pricing page"
Agent: [Uses firecrawl_scrape on competitor pricing URL]
       [Compares with previously stored data]
       "Changes detected on competitor X pricing page:
       - Basic plan increased from $49 to $59/month
       - New Enterprise tier added at $299/month
       - Free trial extended from 7 to 14 days"

E-commerce Data Collector

Agent Prompt:

You collect product data from e-commerce websites.
- Extract product name, price, availability, ratings
- Use structured extraction (firecrawl_extract)
- Store data in a consistent format
- Track price changes over time

Connected MCP: Firecrawl + PostgreSQL (for storage) Example Interaction:

User: "Collect product data from these 20 URLs"
Agent: [Uses firecrawl_extract with schema:
        {
          "product_name": "string",
          "price": "number",
          "currency": "string",
          "in_stock": "boolean",
          "rating": "number",
          "review_count": "number"
        }]
       "Extracted data from 20 products:
       [Displays formatted table with all product information]"

Troubleshooting

Error: API Key Invalid

Symptom: Connection fails with authentication errorCause: Invalid or expired Firecrawl API keySolution:

Check your API key at https://www.firecrawl.dev/app/api-keys
Generate a new key if needed
Update credentials in Junis (Team > MCP > Firecrawl)

Error: Credit Limit Exceeded

Symptom: Scraping operations fail after many requestsCause: Firecrawl account has run out of creditsSolution:

Check your usage at https://www.firecrawl.dev/app
Purchase additional credits or upgrade plan
Implement rate limiting in your agent logic

Scraping Takes Too Long

Symptom: firecrawl_crawl or batch_scrape timeoutCause: Large websites or many URLsSolution:

Reduce maxDepth or limit parameters
Use includePaths to filter specific sections
Check status periodically with check_crawl_status
Break large jobs into smaller batches

Extracted Data is Incomplete

Symptom: firecrawl_extract misses some fieldsCause: JSON schema doesn’t match page structureSolution:

Test with firecrawl_scrape first to see raw content
Refine your JSON schema based on actual page structure
Use more descriptive prompt to guide extraction
Try extracting fewer fields per request

Tools Not Loading

Symptom: Agent connected but Firecrawl tools don’t appearSolution:

Verify connection: Team > MCP > Firecrawl > Test Connection
Check agent configuration: Admin > Agents > [Your Agent] > MCP Platforms
Restart agent (edit and save without changes)
Check logs: Admin > Dashboard > Recent Activity

Performance Tips

✅ Optimization Best Practices:

Use formats: [{"type": "json", "prompt": "..."}] for structured JSON output
End every prompt with “반드시 JSON 형식으로 반환해” for consistent formatting
Filter URLs with includePaths and excludePaths before crawling
Batch process multiple URLs instead of sequential single scrapes
Cache scraped content to avoid redundant API calls
Set reasonable limit and maxDepth for crawling operations
Use firecrawl_map first to plan your scraping strategy
Maximum 3 concurrent Firecrawl calls per agent turn to prevent context overflow

Rate Limits & Costs

Firecrawl Pricing Tiers

Tier	Credits/Month	Best For
Free	500 credits	Testing and small projects
Starter	10,000 credits	Regular scraping tasks
Growth	50,000 credits	Medium-scale operations
Enterprise	Custom	Large-scale data collection

Credit Usage

Scrape: 1 credit per page
Batch Scrape: 1 credit per page
Map: 5 credits per site
Search: 2 credits per query
Crawl: 1 credit per page discovered
Extract: 2 credits per page

Monitor your usage at https://www.firecrawl.dev/app to avoid unexpected credit depletion.

Advanced Configuration

Extract Mode Configuration (Recommended)

{
  "url": "https://example.com/page",
  "formats": [
    {
      "type": "json",
      "prompt": "Describe the data to collect in detail. 반드시 JSON 형식으로 반환해."
    }
  ],
  "onlyMainContent": false,
  "removeBase64Images": true,
  "blockAds": true,
  "waitFor": 3000,
  "timeout": 60000,
  "excludeTags": [
    "script", "style", "noscript",
    "nav", "header", "footer",
    "aside", ".sidebar", ".ad", ".advertisement"
  ]
}

Crawling Strategy

For large websites, use a phased approach: Phase 1: Discovery

Use firecrawl_map to get all URLs → Filter relevant sections

Phase 2: Selective Crawling

Use firecrawl_crawl with includePaths: ["/products/*", "/blog/*"]

Phase 3: Data Extraction

Use firecrawl_extract on discovered URLs with structured schema

What’s Next?

Custom MCP Servers

Build your own MCP integrations

GitHub MCP

Connect to GitHub repositories

MCP Overview

Back to MCP Integration overview

Additional Resources

Firecrawl Dashboard: https://www.firecrawl.dev/app
API Documentation: https://docs.firecrawl.dev/
MCP Server GitHub: https://github.com/firecrawl/firecrawl-mcp-server
MCP Protocol: https://modelcontextprotocol.io/

Pro Tip: Combine Firecrawl with Notion MCP to automatically save research findings, or with Slack MCP to get real-time alerts when monitored websites change.

User Guides

Documentation Index

​What You’ll Learn

​Quick Setup (5 Minutes)

​Available Tools (8 Tools)

​Content Extraction

firecrawl_scrape

firecrawl_batch_scrape

​Site Discovery

firecrawl_map

firecrawl_search

​Advanced Operations

firecrawl_crawl

firecrawl_extract

​Status Monitoring

firecrawl_check_batch_status

firecrawl_check_crawl_status

​Common Workflows

​1. Single Page Analysis

​2. Competitor Research

​3. Content Discovery

​4. Structured Data Collection

​Tool Parameters Explained

​firecrawl_scrape

​firecrawl_batch_scrape

​firecrawl_search

​firecrawl_crawl

​firecrawl_extract

​Example Use Cases

​Web Research Agent

​Content Monitoring Agent

​E-commerce Data Collector

​Troubleshooting

​Performance Tips

​Rate Limits & Costs

​Firecrawl Pricing Tiers

​Credit Usage

​Advanced Configuration

​Extract Mode Configuration (Recommended)

​Crawling Strategy

​What’s Next?

Custom MCP Servers

GitHub MCP

MCP Overview

​Additional Resources

What You’ll Learn

Quick Setup (5 Minutes)

Available Tools (8 Tools)

Content Extraction

Site Discovery

Advanced Operations

Status Monitoring

Common Workflows

1. Single Page Analysis

2. Competitor Research

3. Content Discovery

4. Structured Data Collection

Tool Parameters Explained

firecrawl_scrape

firecrawl_batch_scrape

firecrawl_search

firecrawl_crawl

firecrawl_extract

Example Use Cases

Web Research Agent

Content Monitoring Agent

E-commerce Data Collector

Troubleshooting

Performance Tips

Rate Limits & Costs

Firecrawl Pricing Tiers

Credit Usage

Advanced Configuration

Extract Mode Configuration (Recommended)

Crawling Strategy

What’s Next?

Additional Resources