> ## Documentation Index
> Fetch the complete documentation index at: https://zite.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape and crawl web pages with Firecrawl

> Extract clean data from websites and convert web pages to markdown in your Zite app.

## How it works

Connect your Zite app to Firecrawl to scrape web pages, extract structured data, and convert websites into clean markdown that's ready for AI processing or storage. By connecting Firecrawl, you can:

* Scrape single web pages and convert them to markdown
* Map and discover all URLs on a website
* Search the web and get clean, structured results
* Crawl entire websites to extract content from multiple pages
* Extract structured data from web pages using custom schemas
* Handle JavaScript-heavy pages and dynamic content automatically

<Note>
  Firecrawl requires your own API key from [firecrawl.dev](https://www.firecrawl.dev). Zite does not provide a shared Firecrawl account. Usage is billed directly through your Firecrawl account based on pages scraped.
</Note>

## How to integrate Firecrawl into your app

<Steps>
  <Step title="Get your Firecrawl API key">
    If you don't have a Firecrawl account yet, sign up at [firecrawl.dev](https://www.firecrawl.dev). Then, create an API key from your [API keys page](https://www.firecrawl.dev/app/api-keys).

    <Note>
      Firecrawl API keys start with `fc-`. Keep your key secure and never share it publicly.
    </Note>
  </Step>

  <Step title="Connect to Firecrawl">
    In the chat box, ask Zite to connect your app to Firecrawl or click the `plus icon` **+** to connect. Then, paste your Firecrawl API key when prompted.
  </Step>

  <Step title="Enter your use case">
    Tell Zite how you want to use Firecrawl in your app. Be specific about what you want to scrape or extract. For example:

    ```text theme={null}
    When a user enters a URL, scrape the page and show the main content as markdown in the app.
    ```

    Or for more advanced use cases:

    ```text theme={null}
    Let users enter a company website URL, then map all the pages on that site and show a list of discovered pages.
    ```
  </Step>

  <Step title="Test and deploy">
    Use your app to test the Firecrawl integration. Try scraping a few URLs to make sure everything works as expected.

    <Tip>
      Be mindful of credits. Firecrawl charges per page scraped. Start with small limits when crawling sites, especially during development.
    </Tip>

    Click `Publish` in the upper right corner to [**share**](/help/platform/access/share-to-web) or [**embed**](/help/platform/basics/embedding) your app.
  </Step>
</Steps>

## Common use cases

With a Firecrawl connection, you can create apps like:

* **Research Tools:** Let users paste URLs and extract clean content for analysis, or search the web and get AI-generated summaries
* **Content Aggregators:** Scrape articles from multiple sources and organize them in your app
* **SEO Analyzers:** Map websites, discover all pages, and analyze site structure
* **Lead Generation:** Extract company information and contact details from websites
* **Documentation Importers:** Archive entire documentation sites or pull content into your knowledge base
* **Price Monitoring:** Track product prices and extract structured pricing data from e-commerce sites
* **Market Research:** Harvest structured data from competitor sites for analysis
* **Content Audits:** Build URL inventories and analyze competitor websites

## Important notes

* **Credits and billing:** Firecrawl charges per page scraped. Monitor your usage to avoid unexpected costs.
* **Rate limits:** Respect rate limits when making multiple requests. For large crawls, use the async crawl endpoint.
* **Crawl limits:** Always set a page limit when crawling sites to control costs. The maximum is typically 50 pages per crawl.
* **JavaScript rendering:** Firecrawl handles JavaScript-heavy sites automatically, so modern SPAs work without extra configuration.
* **Clean output:** Use the `onlyMainContent` option to strip navigation and footer content, giving you just the main article or page content.

## Need help?

If you run into issues with Firecrawl:

* Check your API key is correct and active
* Verify your Firecrawl account has sufficient credits
* Ensure URLs are publicly accessible (Firecrawl cannot access pages behind authentication)
* Review Firecrawl's [documentation](https://docs.firecrawl.dev) for API details

<Info>
  For Zite-specific support, contact us at [support@zite.com](mailto:support@zite.com).
</Info>