https://github.com/tobysolutions/plugin-firecrawl
This is the plugin for the firecrawl service with Eliza. This plugin is capable of data scraping and web search.
https://github.com/tobysolutions/plugin-firecrawl
Last synced: about 1 year ago
JSON representation
This is the plugin for the firecrawl service with Eliza. This plugin is capable of data scraping and web search.
- Host: GitHub
- URL: https://github.com/tobysolutions/plugin-firecrawl
- Owner: tobySolutions
- Created: 2025-02-18T09:19:48.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-18T09:20:52.000Z (over 1 year ago)
- Last Synced: 2025-04-05T08:53:31.951Z (about 1 year ago)
- Language: TypeScript
- Size: 19.5 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
Awesome Lists containing this project
README
# Firecrawl Plugin
We've created a service wrapper for the Firecrawl API that provides two main functionalities: web scraping and crawling. Here's a breakdown of how it works:
## Core Features
1. **Service Factory**
* Creates a service instance using an API key
* Returns an object with two methods: `getScrapeData` and `getCrawlData`
2. **Web Scraping (`getScrapeData`)**
* Fetches and extracts content from a single webpage
* Returns structured data including page content and metadata
* Endpoint: `https://api.firecrawl.dev/v1/scrape`
3. **Web Searching (`getSearchData`)**
* Searches for data based on conversations
* Endpoint: `https://api.firecrawl.dev/v1/search`
## Configuration
The plugin requires minimal configuration. In your character file, simply add:
```json
{
"FIRECRAWL_API_KEY": "your-api-key-here"
}
```
## Usage Examples
### Web Scraping
The plugin recognizes various ways users might request web scraping:
```typescript
// Single URL request
"Can you scrape the content from https://example.com?"
"Get the data from www.example.com/page"
// Two-step interaction
User: "I need to scrape some website data."
Agent: "I can help you scrape website data. Please share the URL you'd like me to process."
User: "example.com/products"
```
### Web Search
The plugin handles different crawling request patterns:
```typescript
// Direct search
"Find the latest news about SpaceX launches"
"Can you find details about the iPhone 16 release?"
```
## Response Handling
The plugin automatically:
- Validates URLs before processing
- Handles both direct and conversational requests
- Provides appropriate feedback during the scraping/crawling process
- Returns structured data from the target website
## Error Handling
The plugin includes built-in error handling for common scenarios:
- Invalid or missing URLs
- API authentication issues
- Network failures
- Malformed responses
## Actions
The plugin provides two main actions:
- `FIRECRAWL_GET_SCRAPED_DATA`: For single-page content extraction
- `WEB_SEARCH`: Web search for any data
## Security
- API keys should be kept secure and never shared
- All requests are made over HTTPS
- Input validation is performed on all URLs before processing