https://github.com/saifyxpro/headlessx
A lightweight, self-hosted headless browser automation platform. Designed as an alternative to Browserless, built for speed, privacy, and scalability.
https://github.com/saifyxpro/headlessx
automation automation-api automation-platform browser-automation browser-testing browserless chrome-headless chromedriver container-automation data-extraction headless headless-chrome headless-service playwright playwright-automation puppeteer scraping-service web-automation web-scraping
Last synced: 12 days ago
JSON representation
A lightweight, self-hosted headless browser automation platform. Designed as an alternative to Browserless, built for speed, privacy, and scalability.
- Host: GitHub
- URL: https://github.com/saifyxpro/headlessx
- Owner: saifyxpro
- License: mit
- Created: 2025-09-12T18:11:48.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-01-24T16:48:15.000Z (19 days ago)
- Last Synced: 2026-01-25T18:29:02.108Z (18 days ago)
- Topics: automation, automation-api, automation-platform, browser-automation, browser-testing, browserless, chrome-headless, chromedriver, container-automation, data-extraction, headless, headless-chrome, headless-service, playwright, playwright-automation, puppeteer, scraping-service, web-automation, web-scraping
- Language: JavaScript
- Homepage: https://headlessx.saify.me
- Size: 3.87 MB
- Stars: 1,603
- Watchers: 18
- Forks: 220
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Contributing: docs/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: docs/CODE_OF_CONDUCT.md
- Audit: audit-results.json
- Security: docs/SECURITY.md
Awesome Lists containing this project
README
### **The World's Most Advanced Anti-Detection Scraping Full Stack Web App**
[](https://github.com/)
[](https://github.com/)
[](https://github.com/)
[](https://github.com/)
[](https://nodejs.org/)
[](https://nextjs.org/)
[](https://github.com/AurelicButter/camoufox)
[](https://www.typescriptlang.org/)
[](https://playwright.dev/)
[](LICENSE)
[](CONTRIBUTING.md)
[](https://github.com/)
[](https://github.com/)
---
### 🎯 **0% Detection Rate** on all major anti-bot systems
*Powered by **Camoufox** — Firefox with C++ level fingerprint spoofing*
[📖 Documentation](#-api-endpoints) • [🚀 Quick Start](#-quick-start) • [🐛 Report Bug](https://github.com/) • [✨ Request Feature](https://github.com/)
---
## 🆕 What's New in V2.0
| 🦊 **Camoufox Engine** | 🛡️ **0% Detection** | ⚡ **3x Faster** |
| :---------------------------------------: | :------------------------------------------: | :-----------------------: |
| Firefox with binary-level stealth patches | Passes CreepJS, Sannysoft, & all major tests | Optimized context pooling |
### V2.0 Changelog
```diff
+ 🦊 NEW: Camoufox browser engine (replaces Chromium + stealth plugins)
+ 🛡️ NEW: 0% headless detection rate (was 67%+)
+ 🔒 NEW: Built-in WebRTC leak protection
+ 🎭 NEW: C++ level Canvas/WebGL/AudioContext fingerprint spoofing
+ ⚡ IMPROVED: Faster browser launches with persistent contexts
- 🧹 REMOVED: playwright-extra (no longer needed)
- 🧹 REMOVED: puppeteer-extra-plugin-stealth (no longer needed)
```
---
## 📊 Detection Benchmark Comparison
Tool
Headless Detection
Stealth Score
Engine
🦊 HeadlessX V2.0 (Camoufox)
✅ 0%
✅ 0%
Firefox (C++ patched)
HeadlessX V1 (Playwright Extra)
❌ 67%
⚠️ 50%
Chromium (JS patches)
Puppeteer Stealth
❌ 33%
❌ 80%
Chromium (JS patches)
Vanilla Playwright
❌ 100%
❌ 100%
Chromium
Vanilla Selenium
❌ 100%
❌ 100%
Chrome
*Lower scores = better (0% = undetectable)*
---
## 🤖 Scrapers
| 
**Google SERP** | 🌐 **Website Scraping** |
| :----------------------------------------------------------------------: | :---------------------------------------: |
| Extract search results with zero detection | Scrape any website with full JS rendering |
| ✅ **Live** | ✅ **Live** |
### Coming Soon
| 
**Amazon** | 
**LinkedIn** | 
**Twitter** | 
**Instagram** |
| :-----------------------------------------------------------------: | :---------------------------------------------------------------------: | :-------------------------------------------------------------------: | :-----------------------------------------------------------------------: |
| Product & Reviews | Job Listings & Profiles | Tweets & Trends | Posts & Stories |
---
## 🖥️ UI Screenshots
### Google SERP Scraper
### Website Scraper
---
## 📸 Proof of Undetectability
### BrowserScan (100% Passed)
| **Pixelscan (Human Detection)** | **Profile Tested with Proxy** |
| :--------------------------------------------: | :----------------------------------------: |
|  |  |
---
## 📋 Table of Contents
- [🆕 What's New in V2.0](#-whats-new-in-v20)
- [V2.0 Changelog](#v20-changelog)
- [📊 Detection Benchmark Comparison](#-detection-benchmark-comparison)
- [🤖 Scrapers](#-scrapers)
- [Coming Soon](#coming-soon)
- [🖥️ UI Screenshots](#️-ui-screenshots)
- [Google SERP Scraper](#google-serp-scraper)
- [Website Scraper](#website-scraper)
- [📸 Proof of Undetectability](#-proof-of-undetectability)
- [BrowserScan (100% Passed)](#browserscan-100-passed)
- [📋 Table of Contents](#-table-of-contents)
- [⚡ Quick Start](#-quick-start)
- [1️⃣ Clone \& Configure Environment](#1️⃣-clone--configure-environment)
- [2️⃣ Install Dependencies](#2️⃣-install-dependencies)
- [3️⃣ Install AI Models (CAPTCHA Solving)](#3️⃣-install-ai-models-captcha-solving)
- [4️⃣ Setup Database](#4️⃣-setup-database)
- [5️⃣ Start Development Server](#5️⃣-start-development-server)
- [6️⃣ Access the Application](#6️⃣-access-the-application)
- [Custom Ports](#custom-ports)
- [🔥 Key Features](#-key-features)
- [🦊 Camoufox Stealth Engine (V2.0)](#-camoufox-stealth-engine-v20)
- [🖥️ Modern Dashboard](#️-modern-dashboard)
- [🌐 API Endpoints](#-api-endpoints)
- [Website Scraping APIs](#website-scraping-apis)
- [Google SERP APIs](#google-serp-apis)
- [Example Request](#example-request)
- [Example Response](#example-response)
- [Configuration](#configuration)
- [Environment Variables](#environment-variables)
- [Dashboard Settings](#dashboard-settings)
- [General Configuration](#general-configuration)
- [Browser Engine (Camoufox)](#browser-engine-camoufox)
- [Proxies](#proxies)
- [🛠️ Manual Setup](#️-manual-setup)
- [Backend](#backend)
- [Frontend](#frontend)
- [🤝 Contributing](#-contributing)
- [📝 License](#-license)
- [🌟 Star History](#-star-history)
- [Built with ❤️ using cutting-edge technologies](#built-with-️-using-cutting-edge-technologies)
---
## ⚡ Quick Start
> **Prerequisites**:
> - [Node.js 18+](https://nodejs.org/) (v22+ recommended)
> - [pnpm 9+](https://pnpm.io/) (`npm install -g pnpm`)
> - **PostgreSQL Database** — either:
> - [Supabase](https://supabase.com/) (recommended, free tier available)
> - Self-hosted PostgreSQL 14+
### 1️⃣ Clone & Configure Environment
```bash
git clone https://github.com/your-repo/HeadlessX.git
cd HeadlessX
# Copy environment template
cp .env.example backend/.env
```
**Edit `backend/.env`** with your database connection:
```env
# REQUIRED: Your PostgreSQL connection string
DATABASE_URL="postgresql://user:password@host:5432/database"
# Optional: Server configuration
PORT=3001
NODE_ENV=development
```
> 💡 **Supabase Users**: Find your connection string at:
> Dashboard → Settings → Database → Connection String (URI)
### 2️⃣ Install Dependencies
```bash
# Install all packages (workspace: root, backend, frontend)
pnpm install
```
> ⚠️ If you see a Prisma warning during install, that's normal!
> It means you need to configure `DATABASE_URL` first, then run `pnpm db:push`.
### 3️⃣ Install AI Models (CAPTCHA Solving)
Windows
Linux / macOS
```powershell
install.bat
```
```bash
chmod +x install.sh
./install.sh
```
### 4️⃣ Setup Database
```bash
# Push Prisma schema to your database
pnpm db:push
# Or use migrations for production
pnpm db:migrate
```
### 5️⃣ Start Development Server
```bash
# Start both frontend and backend concurrently
pnpm dev
```
**Or start individually:**
Windows
Linux / macOS
```powershell
scripts\start.bat
```
```bash
./scripts/start.sh
```
### 6️⃣ Access the Application
| Service | URL | Notes |
| ----------------- | ---------------------------------------------- | ----------- |
| 🖥️ **Dashboard** | [http://localhost:3000](http://localhost:3000) | Frontend UI |
| 🔗 **Backend API** | [http://localhost:3001](http://localhost:3001) | REST API |
### Custom Ports
You can customize ports via environment variables:
```bash
# Backend (from root)
cross-env PORT=4001 pnpm dev:backend
# Frontend (Linux/macOS)
PORT=4000 pnpm --filter headlessx-frontend dev
```
---
## 🔥 Key Features
### 🦊 Camoufox Stealth Engine (V2.0)
| Feature | Description |
| -------------------------- | ----------------------------------- |
| 🔒 **Binary-Level Patches** | Firefox modified at C++ level |
| 🎭 **Fingerprint Spoofing** | Canvas, WebGL, AudioContext |
| 🌐 **WebRTC Protection** | No IP leaks in headless mode |
| 🖱️ **Human Behavior** | Natural mouse movements & scrolling |
| 📱 **Device Emulation** | Realistic viewport & user agents |
| 🚫 **Zero Detection** | Passes all anti-bot tests |
### 🖥️ Modern Dashboard
| Feature | Description |
| ------------------ | ------------------------------- |
| ⚛️ **Next.js 16** | React 19 with Turbopack |
| 🎨 **HeroUI** | Beautiful dark/light mode UI |
| ⚙️ **Live Config** | Change settings without restart |
| 📊 **Request Logs** | Full history with screenshots |
| 🧪 **Playground** | Test scraping in real-time |
| 🔑 **API Keys** | Secure key management |
---
## 🌐 API Endpoints
### Website Scraping APIs
| Endpoint | Method | Description |
| ------------------------- | ------ | ---------------------------- |
| `/api/website/html` | `POST` | Get raw HTML (fast) |
| `/api/website/html-js` | `POST` | Get HTML with JS rendering |
| `/api/website/content` | `POST` | Get Markdown content |
| `/api/website/screenshot` | `POST` | Capture full-page Screenshot |
| `/api/website/stream` | `POST` | Real-time SSE Stream |
### Google SERP APIs
| Endpoint | Method | Description |
| ------------------------- | ------ | ----------------------------- |
| `/api/google-serp/search` | `POST` | Extract Google search results |
| `/api/google-serp/stream` | `GET` | Real-time SSE Stream |
### Example Request
```bash
curl -X POST http://localhost:3001/api/website/html \
-H "Content-Type: application/json" \
-H "X-API-Key: your-api-key" \
-d '{"url": "https://example.com", "stealth": true}'
```
### Example Response
```json
{
"success": true,
"data": {
"url": "https://example.com",
"title": "Example Domain",
"html": "...",
"statusCode": 200
}
}
```
---
## Configuration
### Environment Variables
> ⚠️ **Note**: Most configuration (Browsers, Stealth, Proxy, Timeouts) is managed directly via the **Dashboard Settings**.
Only the following core variables are required in `.env`:
| Variable | Default | Description |
| --------------------- | ----------------------- | ----------------------------------------------- |
| `PORT` | `3001` | Backend API port |
| `DATABASE_URL` | - | PostgreSQL connection (Supabase or self-hosted) |
| `NEXT_PUBLIC_API_URL` | `http://localhost:3001` | Frontend API URL |
### Dashboard Settings
Configure these live at `/settings`:
#### General Configuration
- **Headless Mode**: Run browser without UI (faster execution)
- **Browser Timeout**: Max execution time per job (default: 60000ms)
- **Max Concurrent Jobs**: Concurrent scraping jobs (recommended: 3-8)
#### Browser Engine (Camoufox)
- **Block WebRTC**: Prevent IP leaks via WebRTC
- **Camoufox GeoIP**: Spoof location based on IP
- **Enable Cache**: Cache resources for speed
#### Proxies
- Manage proxy configurations and rotation settings
---
## 🛠️ Manual Setup
### Backend
```bash
cd backend
# 1. Configure environment
cp ../.env.example .env
# Edit .env with your DATABASE_URL
# 2. Install dependencies
pnpm install
# 3. Download Camoufox browser
npx camoufox-js fetch
# 4. Setup database
npx prisma db push
# 5. Start dev server
pnpm dev
```
### Frontend
```bash
cd frontend
pnpm install
pnpm dev
```
---
## 🤝 Contributing
Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
---
## 📝 License
This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.
---
## 🌟 Star History
If you find HeadlessX useful, please consider giving it a star! ⭐
---
### Built with ❤️ using cutting-edge technologies
[](https://nodejs.org/)
[](https://nextjs.org/)
[](https://github.com/)
[](https://playwright.dev/)
[](https://www.typescriptlang.org/)
[](https://react.dev/)
[](https://tailwindcss.com/)
[](https://www.prisma.io/)
**🦊 HeadlessX V2.0 — Undetectable by Design**
*© 2026 HeadlessX. All rights reserved.*