{"id":23516312,"url":"https://github.com/bernietv/workflows-app","last_synced_at":"2026-05-05T14:33:11.451Z","repository":{"id":269396452,"uuid":"907281366","full_name":"BernieTv/Workflows-App","owner":"BernieTv","description":"Create AI Powered Workflows to scrape the Web 🔁⚙️🔁🛠️","archived":false,"fork":false,"pushed_at":"2025-01-27T15:33:12.000Z","size":229,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-16T22:33:11.806Z","etag":null,"topics":["clerk","cron","nextjs14","prisma","puppeteer","reactflow","stripe","tailwindcss"],"latest_commit_sha":null,"homepage":"https://workflows-app-tan.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BernieTv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-23T08:31:46.000Z","updated_at":"2025-01-27T15:33:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"ed4de2fa-c066-472c-ba3a-88668a54f7c4","html_url":"https://github.com/BernieTv/Workflows-App","commit_stats":null,"previous_names":["bernietv/workflows-app"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernieTv%2FWorkflows-App","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernieTv%2FWorkflows-App/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernieTv%2FWorkflows-App/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernieTv%2FWorkflows-App/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BernieTv","download_url":"https://codeload.github.com/BernieTv/Workflows-App/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254042341,"owners_count":22004901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clerk","cron","nextjs14","prisma","puppeteer","reactflow","stripe","tailwindcss"],"created_at":"2024-12-25T15:12:37.714Z","updated_at":"2026-05-05T14:33:11.395Z","avatar_url":"https://github.com/BernieTv.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚀 AI-Powered Visual Web Scraper Builder 🔧\n\nThis Full Stack SaaS application allows users to visually build, manage, and schedule web scrapers using a workflow builder powered by AI. 🤖 Users can create, modify, and delete workflows with an intuitive drag-and-drop interface. The integration of AI simplifies the web scraping process, making it accessible to both technical and non-technical users. 🌐\n\n## 🔥 Features\n\n### 🌟 Key Features\n\n-   **🔍 Visual Workflow Builder**: Drag-and-drop interface to design scraping workflows effortlessly. 🔧\n-   **🤖 AI Assistance**: AI-powered suggestions for selectors, workflow optimization, and error handling. 🌐\n-   **🔑 Credential Management**: Securely manage login credentials for scraping protected websites. 🔒\n-   **⏳ Scheduling System**: Set up automatic scraping schedules for periodic data extraction. ⏰\n-   **🏛️ Workflow Management**: Create, modify, delete, and duplicate workflows with ease. 🔄\n-   **📄 Data Export**: Export scraped data in various formats (e.g., CSV, JSON). 📊\n\n### ⚡ Built with Next.js\n\n-   **🌐 Server-Side Rendering (SSR)** for optimized SEO and performance.\n-   **🌍 API Routes** to handle backend logic.\n-   **📁 Dynamic Routing** for user-specific workflows.\n-   **🔐 Built-in Authentication** using Clerk Authentication for secure user sessions.\n\n---\n\n### ⚙️ Prerequisites\n\n-   **💻 Node.js** (v16 or later)\n-   **📊 PostgreSQL** database\n-   **🔧 API Key for OpenAI**\n\n### 🔎 Steps\n\n1. **🔄 Clone the repository**\n\n    ```bash\n    git clone https://github.com/BernieTv/Workflows-App.git\n    cd Workflows-App\n    ```\n\n2. **📦 Install dependencies**\n\n    ```bash\n    npm install\n    ```\n\n3. **🔐 Set up environment variables**\n   Create a `.env` file in the root directory and add the following:\n\n    ```env\n    NEXT_PUBLIC_OPENAI_API_KEY=your_openai_api_key\n    DATABASE_URL=postgresql://username:password@localhost:5432/yourdb\n    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_secret\n    CLERK_SECRET_KEY=sk_\n    ```\n\n4. **🔧 Run database migrations**\n\n    ```bash\n    npx prisma migrate dev\n    ```\n\n5. **🚀 Start the development server**\n    ```bash\n    npm run dev\n    ```\n    The app will be available at `http://localhost:3000`. 🏠\n\n---\n\n## 🔧 Usage\n\n### 1. **🔐 Sign Up/Log In**\n\n-   Use Clerk Authentication to sign up or log in to your account. 🔑\n\n### 2. **🔧 Create a Workflow**\n\n-   Drag and drop nodes to define scraping tasks. 🌐\n-   Use AI suggestions for selector optimization. 🤖\n\n### 3. **🔑 Set Credentials**\n\n-   Securely store website login credentials if required. 🔒\n\n### 4. **⏳ Schedule Scraping**\n\n-   Use the scheduling feature to automate scraping tasks. 🕰️\n\n### 5. **📄 Export Data**\n\n-   Download scraped data in the desired format. 📊\n\n---\n\n## 💪 Development\n\n### ⚙️ Scripts\n\n-   **🚀 Start development server**: `npm run dev`\n-   **🌍 Build for production**: `npm run build`\n-   **🏠 Run production server**: `npm start`\n\n### 🔧 Linting and Formatting\n\n-   **🔢 Lint code**: `npm run lint`\n-   **🔄 Format code**: `npm run format`\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbernietv%2Fworkflows-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbernietv%2Fworkflows-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbernietv%2Fworkflows-app/lists"}