https://github.com/alexfazio/firecrawl-quickstarts
A collection of cookbooks to help developers get started quickly with the Firecrawl API.
https://github.com/alexfazio/firecrawl-quickstarts
agentops anthropic firecrawl grok openai scraping
Last synced: 5 months ago
JSON representation
A collection of cookbooks to help developers get started quickly with the Firecrawl API.
- Host: GitHub
- URL: https://github.com/alexfazio/firecrawl-quickstarts
- Owner: alexfazio
- Created: 2024-10-26T09:56:26.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-12-31T07:31:46.000Z (5 months ago)
- Last Synced: 2024-12-31T15:12:21.801Z (5 months ago)
- Topics: agentops, anthropic, firecrawl, grok, openai, scraping
- Language: Jupyter Notebook
- Homepage:
- Size: 591 KB
- Stars: 26
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Firecrawl Quickstarts is an independent and unofficial collection of projects designed to help developers quickly get started with building applications using the Firecrawl API. Each quickstart provides a foundation that you can easily build upon and customize for your specific needs. This repository is not affiliated with, endorsed by, or officially supported by Firecrawl.
## Getting Started
To use these quickstarts, you'll need a Firecrawl API key. If you don't have one yet, you can sign up for free at [firecrawl.dev](https://firecrawl.dev).
## Available Quickstarts
### Firecrawl Web Crawling with OpenAI and Anthropic
This quickstart introduces how to integrate Firecrawl with OpenAI's Anthropic models to search and extract information based on specific user objectives. Learn to map a website, identify relevant pages, and retrieve content aligned with the objective. Ideal for targeted information gathering.
[Go to Firecrawl Web Crawling with OpenAI and Anthropic](./claude_researcher_with_map.ipynb)
### Integrating OpenAI o1 Models with Firecrawl
Explore how to enhance the Firecrawl web crawling process with OpenAI’s o1 reasoning models. This quickstart guides you in using these advanced models to generate search parameters, map sites, and validate extracted content, enhancing the precision and relevance of data extraction.
[Go to Integrating OpenAI o1 Models with Firecrawl](./crawl_and_extract_with_openai_o1.ipynb)
### Building a Web Crawler with Grok-2 and Firecrawl
Combine Grok-2’s AI-powered understanding with Firecrawl’s search to create an intelligent web crawler. This quickstart demonstrates building a targeted crawler that finds and processes structured data on web pages, with output in JSON format for seamless data handling.
[Go to Building a Web Crawler with Grok-2 and Firecrawl](./crawl_and_extract_with_xai_grok.ipynb)
### Firecrawl Map Endpoint Quickstart
Learn how to use Firecrawl's Map endpoint to create comprehensive sitemaps from single URLs. This quickstart is perfect for efficiently gathering website structures, enabling tasks such as content mapping, SEO analysis, and scalable web data extraction.
[Go to Firecrawl Map Endpoint Quickstart](./firecrawl_map_endpoint_tutorial.ipynb)
### Job Board Scraping with Firecrawl and OpenAI
Automate job listing extraction and analysis with Firecrawl and OpenAI’s Structured Outputs. This quickstart demonstrates scraping job boards, extracting structured job details, and matching listings to a user’s resume with schema-compliant outputs for reliable data processing.
[Go to Job Board Scraping with Firecrawl and OpenAI](./job_scraping_tutorial.ipynb)
### Firecrawl LLM Extract Tutorial
Learn how to use Firecrawl’s LLM-powered data extraction features. This quickstart covers extracting structured data from web pages, with options for schema-defined and prompt-only extraction, making it adaptable for diverse data formats and applications.
[Go to Firecrawl LLM Extract Tutorial](./llm_extract_tutorial.ipynb)
## General Usage
Each quickstart project is a Jupyter notebook designed to be easily opened and run on Google Colab. To get started, follow these steps:
1. **Open the Repository in Google Colab**
Each notebook has a link to open directly in Google Colab. Click on the link for the quickstart you want to explore.
2. **Set Up Your Firecrawl API Key**
Each notebook requires a Firecrawl API key. Once you've created your key (available [here](https://firecrawl.dev)), enter it in the notebook when prompted or set it as an environment variable as directed in the notebook.
3. **Run Each Notebook Cell Sequentially**
Follow the instructions within each notebook, running cells in order. The notebooks will guide you through each step, from setting up the environment to executing web scraping or extraction tasks.
4. **View Results and Experiment**
The notebooks are designed to be interactive. You can modify the code cells, adjust parameters, or try different objectives to explore Firecrawl’s capabilities further.
Each notebook includes explanations and usage examples to help you understand and customize your setup.
## Explore Further
To deepen your understanding of working with Firecrawl and its API, check out these resources:
- [**Firecrawl Documentation**](https://docs.firecrawl.dev) - Comprehensive guides and API references
- [**Firecrawl SDKs**](https://docs.firecrawl.dev/sdks/overview) - Explore our SDKs for [Python](https://docs.firecrawl.dev/sdks/python), [Node.js](https://docs.firecrawl.dev/sdks/node), [Go](https://docs.firecrawl.dev/sdks/go), and [Rust](https://docs.firecrawl.dev/sdks/rust)
- [**LLM Framework Integrations**](https://docs.firecrawl.dev/integrations/overview) - Learn how to use Firecrawl with frameworks like LangChain and Llama Index
- [**Firecrawl API Reference**](https://docs.firecrawl.dev/api-reference/introduction) - Detailed API endpoints and parameters
## Contributing
We welcome contributions to the Firecrawl Quickstarts repository! If you have ideas for new quickstart projects or improvements to existing ones, please open an issue or submit a pull request.
## Community and Support
- Join our [Firecrawl Discord community](https://discord.com/invite/gSmWdAkdwd) for discussions and support
- Follow us on [Twitter](https://twitter.com/firecrawl_dev) and [LinkedIn](https://www.linkedin.com/company/104100957) for updates
- Check out the [Firecrawl Support Documentation](https://docs.firecrawl.dev) for additional help
## License
This project is licensed under the [MIT](https://opensource.org/licenses/MIT) License - see the [LICENSE](LICENSE) file for details.
---
*It is the sole responsibility of the end users to respect websites' policies when scraping, searching, and crawling with Firecrawl. Users are advised to adhere to the applicable privacy policies and terms of use of the websites prior to initiating any scraping activities. By default, Firecrawl respects the directives specified in the websites' robots.txt files when crawling. By utilizing Firecrawl, you expressly agree to comply with these conditions.*
[↑ Back to Top ↑](#firecrawl-quickstarts)
Copyright (c) 2024-present, Alex Fazio