> ## Documentation Index
> Fetch the complete documentation index at: https://docs.junis.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Firecrawl MCP Setup

> Enable web scraping, crawling, and structured data extraction for your agents

## What You'll Learn

This guide shows you how to integrate Firecrawl with Junis using the Firecrawl MCP Server. Your agents will be able to:

* Scrape single web pages or batch process multiple URLs
* Crawl entire websites recursively
* Search the web and extract content from results
* Extract structured data using JSON schemas
* Map website structures and discover all pages

<Info>
  **Prerequisites**:

  * Firecrawl account and API key ([Get one here](https://www.firecrawl.dev/app/api-keys))
  * Admin role in Junis (for organization-level setup)
  * Basic understanding of web scraping concepts
</Info>

***

## Quick Setup (5 Minutes)

<Steps>
  <Step title="Get Firecrawl API Key">
    1. Visit [Firecrawl Dashboard](https://www.firecrawl.dev/app/api-keys)
    2. Sign up or log in
    3. Click **"Create API Key"**
    4. Copy your API key (starts with `fc-`)
  </Step>

  <Step title="Add Firecrawl Platform to Junis">
    Navigate to **Team > MCP** in Junis and find the Firecrawl card.

    If Firecrawl is already configured (globe icon 🌍), skip to Step 3.

    Otherwise, click **"Connect"** and fill in:

    * **Platform Name**: `Firecrawl`
    * **MCP Server URL**: `https://api.firecrawl.dev/mcp/`
    * **Transport Type**: `Streamable HTTP`
  </Step>

  <Step title="Add Your Credentials">
    Click **"Add Auth"** on the Firecrawl card and paste your API key.

    Click **"Test Connection"** to verify it works.
  </Step>

  <Step title="Enable for Your Agents">
    Go to **Admin > Agents**, edit the agent you want to connect, and check **"Firecrawl"** in the MCP Platforms section.

    Save and test by asking: *"Scrape the homepage of [https://example.com](https://example.com)"*
  </Step>
</Steps>

***

## Available Tools (8 Tools)

Firecrawl MCP provides 8 powerful web research tools:

### Content Extraction

<CardGroup cols={2}>
  <Card title="firecrawl_scrape" icon="file">
    **Single Page Scraping**

    Extract content from a single URL in markdown or HTML format.

    *Example*: "Scrape the content of [https://blog.example.com/post](https://blog.example.com/post)"
  </Card>

  <Card title="firecrawl_batch_scrape" icon="files">
    **Batch Scraping**

    Scrape multiple URLs simultaneously.

    *Example*: "Scrape these 5 product pages: URL1, URL2, ..."
  </Card>
</CardGroup>

### Site Discovery

<CardGroup cols={2}>
  <Card title="firecrawl_map" icon="sitemap">
    **Website Mapping**

    Discover all indexed URLs on a website.

    *Example*: "Map all pages on example.com"
  </Card>

  <Card title="firecrawl_search" icon="magnifying-glass">
    **Web Search + Extract**

    Search the web and extract content from top results.

    *Example*: "Search for 'AI trends 2026' and summarize top 5 results"
  </Card>
</CardGroup>

### Advanced Operations

<CardGroup cols={2}>
  <Card title="firecrawl_crawl" icon="spider">
    **Recursive Crawling**

    Crawl an entire website recursively.

    *Example*: "Crawl all product pages on example.com"
  </Card>

  <Card title="firecrawl_extract" icon="code">
    **Structured Data Extraction**

    Extract data using JSON schemas for consistent formatting.

    *Example*: "Extract price, stock, and reviews from this product page"
  </Card>
</CardGroup>

### Status Monitoring

<CardGroup cols={2}>
  <Card title="firecrawl_check_batch_status" icon="clock">
    **Batch Status**

    Check progress of batch scraping jobs.
  </Card>

  <Card title="firecrawl_check_crawl_status" icon="chart-line">
    **Crawl Status**

    Monitor progress of crawling operations.
  </Card>
</CardGroup>

***

## Common Workflows

### 1. Single Page Analysis

**Use Case**: Analyze a blog post or article

**Workflow**:

```
1. User: "Analyze the content of this blog post: [URL]"
2. Agent uses firecrawl_scrape to get the content
3. Agent summarizes and analyzes the text
```

**Agent Instruction**:

```
When asked to analyze a web page:
1. Use firecrawl_scrape to get the content
2. Focus on the main content (set onlyMainContent: true)
3. Provide a concise summary and key insights
```

### 2. Competitor Research

**Use Case**: Analyze multiple competitor websites

**Workflow**:

```
1. User: "Research our top 5 competitors' pricing pages"
2. Agent uses firecrawl_batch_scrape with 5 URLs
3. Agent uses firecrawl_extract to get structured pricing data
4. Agent compares and creates a report
```

**Agent Instruction**:

```
For competitor research:
1. Use batch_scrape for multiple URLs
2. Extract key information like pricing, features, target audience
3. Create a comparison table
4. Highlight our advantages and areas for improvement
```

### 3. Content Discovery

**Use Case**: Find all blog posts on a website

**Workflow**:

```
1. User: "Find all blog posts on example.com"
2. Agent uses firecrawl_map to discover all URLs
3. Agent filters URLs containing "/blog/"
4. Agent uses batch_scrape to get content
5. Agent categorizes and summarizes posts
```

**Agent Instruction**:

```
To discover content on a website:
1. Use firecrawl_map to get all URLs
2. Filter relevant URLs based on patterns
3. Use batch_scrape for multiple pages
4. Organize content by topic or date
```

### 4. Structured Data Collection

**Use Case**: Collect product information from e-commerce sites

**Workflow**:

```
1. User: "Extract product details from these pages"
2. Agent uses firecrawl_extract with JSON schema:
   {
     "name": "string",
     "price": "number",
     "stock": "boolean",
     "rating": "number"
   }
3. Agent returns structured data
```

**Agent Instruction**:

```
For structured data extraction:
1. Define a clear JSON schema for the data
2. Use firecrawl_extract with the schema
3. Validate extracted data for completeness
4. Format results in a table or JSON
```

***

## Tool Parameters Explained

### firecrawl\_scrape

| Parameter         | Type    | Description                 | Example                                             |
| ----------------- | ------- | --------------------------- | --------------------------------------------------- |
| `url`             | string  | **Required**. URL to scrape | `https://example.com`                               |
| `formats`         | array   | Output format(s)            | `[{"type": "json", "prompt": "..."}]` (recommended) |
| `onlyMainContent` | boolean | Extract only main content   | `true`                                              |
| `includeTags`     | array   | HTML tags to include        | `["article", "main"]`                               |
| `excludeTags`     | array   | HTML tags to exclude        | `["nav", "footer"]`                                 |

### firecrawl\_batch\_scrape

| Parameter | Type   | Description                | Example                    |
| --------- | ------ | -------------------------- | -------------------------- |
| `urls`    | array  | **Required**. List of URLs | `["url1", "url2", "url3"]` |
| `options` | object | Scraping options           | Same as `firecrawl_scrape` |

### firecrawl\_search

| Parameter | Type   | Description                | Example            |
| --------- | ------ | -------------------------- | ------------------ |
| `query`   | string | **Required**. Search query | `"AI trends 2026"` |
| `limit`   | number | Max results to return      | `5`                |
| `lang`    | string | Language code              | `"en"` or `"ko"`   |
| `country` | string | Country code               | `"US"` or `"KR"`   |

### firecrawl\_crawl

| Parameter            | Type    | Description                | Example               |
| -------------------- | ------- | -------------------------- | --------------------- |
| `url`                | string  | **Required**. Starting URL | `https://example.com` |
| `maxDepth`           | number  | Max crawl depth            | `3`                   |
| `limit`              | number  | Max pages to crawl         | `100`                 |
| `allowExternalLinks` | boolean | Follow external links      | `false`               |
| `includePaths`       | array   | URL patterns to include    | `["/blog/*"]`         |
| `excludePaths`       | array   | URL patterns to exclude    | `["/admin/*"]`        |

### firecrawl\_extract

| Parameter         | Type    | Description                        | Example                                 |
| ----------------- | ------- | ---------------------------------- | --------------------------------------- |
| `urls`            | array   | **Required**. URLs to extract from | `["url1", "url2"]`                      |
| `prompt`          | string  | Extraction instructions            | `"Extract product details"`             |
| `schema`          | object  | JSON Schema for data               | `{"name": "string", "price": "number"}` |
| `enableWebSearch` | boolean | Use web search                     | `false`                                 |

***

## Example Use Cases

### Web Research Agent

**Agent Prompt**:

```
You are a web research specialist. When users ask you to research a topic:
1. Use firecrawl_search to find relevant sources
2. Use firecrawl_scrape or batch_scrape to get full content
3. Analyze and synthesize information
4. Provide citations with URLs
5. Summarize key findings in bullet points
```

**Connected MCP**: Firecrawl + Notion (for saving research)

**Example Interaction**:

```
User: "Research the latest developments in quantum computing"
Agent: [Uses firecrawl_search("quantum computing 2026", limit=10)]
       [Uses firecrawl_batch_scrape on top 5 results]
       "Here are the latest developments in quantum computing:
       - IBM announced a 1000+ qubit processor (Source: [URL])
       - Google achieved quantum advantage in optimization problems (Source: [URL])
       - New error correction methods show promise (Source: [URL])
       ..."
```

### Content Monitoring Agent

**Agent Prompt**:

```
You monitor competitor websites and news sources for updates.
- Check specific URLs daily
- Compare new content with previous versions
- Highlight changes and new information
- Alert team if significant updates detected
```

**Connected MCP**: Firecrawl + Slack (for notifications)

**Example Interaction**:

```
User: "Check if competitor X has updated their pricing page"
Agent: [Uses firecrawl_scrape on competitor pricing URL]
       [Compares with previously stored data]
       "Changes detected on competitor X pricing page:
       - Basic plan increased from $49 to $59/month
       - New Enterprise tier added at $299/month
       - Free trial extended from 7 to 14 days"
```

### E-commerce Data Collector

**Agent Prompt**:

```
You collect product data from e-commerce websites.
- Extract product name, price, availability, ratings
- Use structured extraction (firecrawl_extract)
- Store data in a consistent format
- Track price changes over time
```

**Connected MCP**: Firecrawl + PostgreSQL (for storage)

**Example Interaction**:

```
User: "Collect product data from these 20 URLs"
Agent: [Uses firecrawl_extract with schema:
        {
          "product_name": "string",
          "price": "number",
          "currency": "string",
          "in_stock": "boolean",
          "rating": "number",
          "review_count": "number"
        }]
       "Extracted data from 20 products:
       [Displays formatted table with all product information]"
```

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Error: API Key Invalid" icon="key">
    **Symptom**: Connection fails with authentication error

    **Cause**: Invalid or expired Firecrawl API key

    **Solution**:

    1. Check your API key at [https://www.firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys)
    2. Generate a new key if needed
    3. Update credentials in Junis (Team > MCP > Firecrawl)
  </Accordion>

  <Accordion title="Error: Credit Limit Exceeded" icon="money-bill">
    **Symptom**: Scraping operations fail after many requests

    **Cause**: Firecrawl account has run out of credits

    **Solution**:

    1. Check your usage at [https://www.firecrawl.dev/app](https://www.firecrawl.dev/app)
    2. Purchase additional credits or upgrade plan
    3. Implement rate limiting in your agent logic
  </Accordion>

  <Accordion title="Scraping Takes Too Long" icon="clock">
    **Symptom**: firecrawl\_crawl or batch\_scrape timeout

    **Cause**: Large websites or many URLs

    **Solution**:

    * Reduce `maxDepth` or `limit` parameters
    * Use `includePaths` to filter specific sections
    * Check status periodically with check\_crawl\_status
    * Break large jobs into smaller batches
  </Accordion>

  <Accordion title="Extracted Data is Incomplete" icon="database">
    **Symptom**: firecrawl\_extract misses some fields

    **Cause**: JSON schema doesn't match page structure

    **Solution**:

    1. Test with `firecrawl_scrape` first to see raw content
    2. Refine your JSON schema based on actual page structure
    3. Use more descriptive `prompt` to guide extraction
    4. Try extracting fewer fields per request
  </Accordion>

  <Accordion title="Tools Not Loading" icon="exclamation-triangle">
    **Symptom**: Agent connected but Firecrawl tools don't appear

    **Solution**:

    1. Verify connection: Team > MCP > Firecrawl > Test Connection
    2. Check agent configuration: Admin > Agents > \[Your Agent] > MCP Platforms
    3. Restart agent (edit and save without changes)
    4. Check logs: Admin > Dashboard > Recent Activity
  </Accordion>
</AccordionGroup>

***

## Performance Tips

<Check>
  **✅ Optimization Best Practices**:

  * Use `formats: [{"type": "json", "prompt": "..."}]` for structured JSON output
  * End every prompt with "반드시 JSON 형식으로 반환해" for consistent formatting
  * Filter URLs with `includePaths` and `excludePaths` before crawling
  * Batch process multiple URLs instead of sequential single scrapes
  * Cache scraped content to avoid redundant API calls
  * Set reasonable `limit` and `maxDepth` for crawling operations
  * Use `firecrawl_map` first to plan your scraping strategy
  * Maximum **3 concurrent Firecrawl calls** per agent turn to prevent context overflow
</Check>

***

## Rate Limits & Costs

### Firecrawl Pricing Tiers

| Tier           | Credits/Month  | Best For                    |
| -------------- | -------------- | --------------------------- |
| **Free**       | 500 credits    | Testing and small projects  |
| **Starter**    | 10,000 credits | Regular scraping tasks      |
| **Growth**     | 50,000 credits | Medium-scale operations     |
| **Enterprise** | Custom         | Large-scale data collection |

### Credit Usage

* **Scrape**: 1 credit per page
* **Batch Scrape**: 1 credit per page
* **Map**: 5 credits per site
* **Search**: 2 credits per query
* **Crawl**: 1 credit per page discovered
* **Extract**: 2 credits per page

<Warning>
  Monitor your usage at [https://www.firecrawl.dev/app](https://www.firecrawl.dev/app) to avoid unexpected credit depletion.
</Warning>

***

## Advanced Configuration

### Extract Mode Configuration (Recommended)

```json theme={null}
{
  "url": "https://example.com/page",
  "formats": [
    {
      "type": "json",
      "prompt": "Describe the data to collect in detail. 반드시 JSON 형식으로 반환해."
    }
  ],
  "onlyMainContent": false,
  "removeBase64Images": true,
  "blockAds": true,
  "waitFor": 3000,
  "timeout": 60000,
  "excludeTags": [
    "script", "style", "noscript",
    "nav", "header", "footer",
    "aside", ".sidebar", ".ad", ".advertisement"
  ]
}
```

### Crawling Strategy

For large websites, use a phased approach:

**Phase 1: Discovery**

```
Use firecrawl_map to get all URLs → Filter relevant sections
```

**Phase 2: Selective Crawling**

```
Use firecrawl_crawl with includePaths: ["/products/*", "/blog/*"]
```

**Phase 3: Data Extraction**

```
Use firecrawl_extract on discovered URLs with structured schema
```

***

## What's Next?

<CardGroup cols={2}>
  <Card title="Custom MCP Servers" icon="cog" href="/guides/mcp/custom">
    Build your own MCP integrations
  </Card>

  <Card title="GitHub MCP" icon="github" href="/guides/mcp/github">
    Connect to GitHub repositories
  </Card>

  <Card title="MCP Overview" icon="arrow-left" href="/guides/mcp/overview">
    Back to MCP Integration overview
  </Card>
</CardGroup>

***

## Additional Resources

* **Firecrawl Dashboard**: [https://www.firecrawl.dev/app](https://www.firecrawl.dev/app)
* **API Documentation**: [https://docs.firecrawl.dev/](https://docs.firecrawl.dev/)
* **MCP Server GitHub**: [https://github.com/firecrawl/firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server)
* **MCP Protocol**: [https://modelcontextprotocol.io/](https://modelcontextprotocol.io/)

<Note>
  **Pro Tip**: Combine Firecrawl with Notion MCP to automatically save research findings, or with Slack MCP to get real-time alerts when monitored websites change.
</Note>