{"id":15062202,"url":"https://github.com/thomasjvu/entry-level-jobs-scraper","last_synced_at":"2026-01-02T19:02:30.408Z","repository":{"id":256559861,"uuid":"855533389","full_name":"thomasjvu/entry-level-jobs-scraper","owner":"thomasjvu","description":"Job scraper for entry-level roles for the Ground Zero project","archived":false,"fork":false,"pushed_at":"2024-09-13T16:59:46.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T02:36:16.412Z","etag":null,"topics":["puppeteer","supabase"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thomasjvu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-11T02:54:05.000Z","updated_at":"2025-01-17T01:51:27.000Z","dependencies_parsed_at":"2024-11-21T08:36:03.173Z","dependency_job_id":null,"html_url":"https://github.com/thomasjvu/entry-level-jobs-scraper","commit_stats":null,"previous_names":["thomasjvu/entry-level-jobs-scraper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjvu%2Fentry-level-jobs-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjvu%2Fentry-level-jobs-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjvu%2Fentry-level-jobs-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjvu%2Fentry-level-jobs-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thomasjvu","download_url":"https://codeload.github.com/thomasjvu/entry-level-jobs-scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243725633,"owners_count":20337670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["puppeteer","supabase"],"created_at":"2024-09-24T23:32:07.349Z","updated_at":"2026-01-02T19:02:30.302Z","avatar_url":"https://github.com/thomasjvu.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Game Jobs Scraper\n\nThis project is an example web scraper that automatically collects job listings from and stores them in a Supabase database. It's designed to run both locally and on Render.com.\n\n## Features\n\n- Scrapes job listings from GameJobs.co (as a learnable template)\n- Stores job data in a Supabase database\n- Avoids duplicate entries\n- Runs on a daily schedule using node-cron\n- Configurable for both local development and cloud deployment (Render.com)\n\n## Prerequisites\n\n- npm\n- Node.js\n- A Supabase account and project\n- (Optional) A Render.com account for deployment\n\n## Installation\n\n1. Clone the repository:\n   ```\n   git clone https://github.com/your-username/game-jobs-scraper.git\n   cd game-jobs-scraper\n   ```\n\n2. Install dependencies:\n   ```\n   npm install\n   ```\n\n3. Create a `.env` file in the root directory and add your Supabase credentials:\n   ```\n   SUPABASE_URL=your_supabase_project_url\n   SUPABASE_API_KEY=your_supabase_api_key\n   SCRAPER_EMAIL=your_scraper_user_email\n   SCRAPER_PASSWORD=your_scraper_user_password\n   ```\n\n## Usage\n\nTo run the scraper locally:\n\n```\nnpm start\n```\n\nThe scraper will run immediately and then on a schedule defined in `src/server.js`.\n\n## Deployment\n\nThis project is configured for deployment on Render.com.\n\n1. Push your code to a GitHub repository.\n2. In Render.com, create a new Web Service and connect it to your GitHub repo.\n3. Use the following settings:\n   - Environment: Node\n   - Build Command: `npm install`\n   - Start Command: `node src/server.js`\n4. Add the environment variables from your `.env` file in the Render dashboard.\n5. Deploy the service.\n\n## Project Structure\n\n- `src/server.js`: Main entry point, sets up the server and scheduling\n- `src/scrapers/gameJobsScraper.js`: Contains the scraping logic for GameJobs.co\n- `src/services/jobService.js`: Handles database operations\n- `src/utils/auth.js`: Manages authentication with Supabase\n- `src/db/supabase.js`: Sets up the Supabase client\n- `src/config.js`: Loads environment variables\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasjvu%2Fentry-level-jobs-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomasjvu%2Fentry-level-jobs-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasjvu%2Fentry-level-jobs-scraper/lists"}