Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/steel-dev/steel-puppeteer-starter
A starter project for using Puppeteer with Steel on Node.js & TypeScript
https://github.com/steel-dev/steel-puppeteer-starter
browser-automation puppeteer web-scraping
Last synced: about 7 hours ago
JSON representation
A starter project for using Puppeteer with Steel on Node.js & TypeScript
- Host: GitHub
- URL: https://github.com/steel-dev/steel-puppeteer-starter
- Owner: steel-dev
- License: mit
- Created: 2024-11-13T20:22:59.000Z (2 days ago)
- Default Branch: main
- Last Pushed: 2024-11-14T18:48:17.000Z (1 day ago)
- Last Synced: 2024-11-14T19:38:52.242Z (1 day ago)
- Topics: browser-automation, puppeteer, web-scraping
- Language: TypeScript
- Homepage: https://docs.steel.dev
- Size: 25.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Steel + Puppeteer Starter
This template shows you how to use Steel with Puppeteer to run browser automations in the cloud. It includes session management, error handling, and a basic example you can customize.
[](https://stackblitz.com/edit/steel-puppeteer-starter?file=README.md)
[](https://replit.com/@steel-dev/steel-puppeteer-starter?v=1)
## Installation
Clone the repository and install dependencies:
```bash
git clone https://github.com/steel-dev/steel-puppeteer-starter
cd steel-puppeteer-starter
npm install
```
## Quick start
The example script in `index.ts` shows you how to:
- Create and manage a Steel browser session
- Connect Puppeteer to the session
- Navigate to a website (Hacker News in this example)
- Extract data from the page (top 5 stories)
- Handle errors and cleanup properly
- View your live session in Steel's session viewer
To run it:
1. Create a `.env` file in the root directory:
```bash
STEEL_API_KEY=your_api_key_here
```
2. Replace `your_api_key_here` with your Steel API key. Don't have one? Get a free key at [app.steel.dev/settings/api-keys](https://app.steel.dev/settings/api-keys)
3. Run the script:
```bash
npm start
```
## Writing your automation
Find this section in `index.ts`:
```typescript
// ============================================================
// Your Automations Go Here!
// ============================================================
// Example automation (you can delete this)
await page.goto('https://news.ycombinator.com');
// ... rest of example code
```
You can replace the code here with whatever automation scripts you want to run.
## Configuration
The template includes common Steel configurations you can enable:
```typescript
const session = await client.sessions.create({
useProxy: true, // Use Steel's proxy network
solveCaptcha: true, // Enable CAPTCHA solving
sessionTimeout: 1800000, // 30 minute timeout (default: 15 mins)
userAgent: 'custom-ua', // Custom User-Agent
});
```
## Error handling
The template includes error handling and cleanup:
```typescript
try {
// Your automation code
} finally {
// Cleanup runs even if there's an error
if (browser) await browser.close();
if (session) await client.sessions.release(session.id);
}
```
## Support
- [Steel Documentation](https://docs.steel.dev)
- [API Reference](https://docs.steel.dev/api-reference)
- [Discord Community](https://discord.gg/gPpvhNvc5R)