{"id":26445173,"url":"https://github.com/srijitpatra99/puppeteer","last_synced_at":"2026-05-02T18:40:27.823Z","repository":{"id":128835150,"uuid":"461109964","full_name":"srijitpatra99/puppeteer","owner":"srijitpatra99","description":"A Node.js API that scrapes ICC Cricket Team Rankings and updates Firebase Realtime Database every 24 hours. 🏏📊","archived":false,"fork":false,"pushed_at":"2025-02-23T08:31:15.000Z","size":36,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-18T11:15:37.881Z","etag":null,"topics":["expressjs","firebase-realtime-database","node-cron","node-js","puppeteer"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/srijitpatra99.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-19T06:43:10.000Z","updated_at":"2025-02-23T08:32:59.000Z","dependencies_parsed_at":"2023-05-06T19:02:48.726Z","dependency_job_id":null,"html_url":"https://github.com/srijitpatra99/puppeteer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/srijitpatra99/puppeteer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srijitpatra99%2Fpuppeteer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srijitpatra99%2Fpuppeteer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srijitpatra99%2Fpuppeteer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srijitpatra99%2Fpuppeteer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/srijitpatra99","download_url":"https://codeload.github.com/srijitpatra99/puppeteer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srijitpatra99%2Fpuppeteer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263589498,"owners_count":23485124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expressjs","firebase-realtime-database","node-cron","node-js","puppeteer"],"created_at":"2025-03-18T11:15:41.346Z","updated_at":"2026-05-02T18:40:27.796Z","avatar_url":"https://github.com/srijitpatra99.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🏏 Web Scraping with Node.js  \n\nA **Node.js API** that scrapes the [Cricket Points Table](https://www.icc-cricket.com/rankings/mens/team-rankings/odi) and stores the **team rankings** and **last updated info** in a **Firebase Realtime Database**. This API continuously checks for changes **every midnight** and automatically updates the database to ensure real-time accuracy.  \n\n---\n\n## 🚀 Features  \n\n✅ Scrapes **ICC Cricket Team Rankings** (ODI) automatically  \n✅ Stores scraped data in **Firebase Realtime Database**  \n✅ Runs every **24 hours** to keep data updated  \n✅ Uses **Node.js, Cheerio, and Firebase Admin SDK**  \n\n---\n\n## 📂 Project Structure  \n\n```\n📦 Web-Scraping-NodeJS\n ┣ 📜 package.json\n ┣ 📜 app.js\n ┣ 📜 firebase-config.json  \u003c-- Your Firebase credentials file\n ┣ 📜 README.md\n ┗ 📂 node_modules\n```\n\n---\n\n## 🛠️ Setup \u0026 Installation  \n\n### 1️⃣ Prerequisites  \n\nEnsure you have the following installed:  \n- **Node.js** (v14+ recommended)  \n- **NPM** (Node Package Manager)  \n- A **Firebase Realtime Database** account  \n\n### 2️⃣ Clone the Repository  \n\n```sh\ngit clone https://github.com/yourusername/web-scraping-nodejs.git\ncd web-scraping-nodejs\n```\n\n### 3️⃣ Install Dependencies  \n\n```sh\nnpm install\n```\n\n### 4️⃣ Set Up Firebase  \n\n- Go to [Firebase Console](https://console.firebase.google.com/)  \n- Create a **Firebase Realtime Database**  \n- Generate your **Service Account Key** (JSON file)  \n- Place the JSON file in the root folder and rename it to **firebase-config.json**  \n\n### 5️⃣ Run the Scraper  \n\n```sh\nnode app.js\n```\n\n🎯 The API will now fetch and store data in **Firebase** every **24 hours** automatically.  \n\n---\n\n## 📌 Usage  \n\nOnce running, the scraper will:  \n1. Fetch the latest **Cricket Team Rankings** data  \n2. Store it in **Firebase Realtime Database**  \n3. Automatically update the database every **midnight**  \n\n---\n\n## 📷 Example Data  \n\n```json\n{\n  \"last_updated\": \"2025-02-22T00:00:00Z\",\n  \"rankings\": [\n    {\n      \"rank\": 1,\n      \"team\": \"India\",\n      \"matches\": 25,\n      \"points\": 3200,\n      \"rating\": 128\n    },\n    {\n      \"rank\": 2,\n      \"team\": \"Australia\",\n      \"matches\": 23,\n      \"points\": 2900,\n      \"rating\": 126\n    }\n  ]\n}\n```\n\n---\n\n## 🛠 Technologies Used  \n\n🔹 **Node.js** – Backend runtime environment  \n🔹 **Cheerio.js** – Scraping and parsing HTML  \n🔹 **Firebase Realtime Database** – Data storage  \n🔹 **Cron Jobs** – Automating scheduled tasks  \n\n---\n\n## 📜 License  \n\nThis project is **open-source** and available under the **MIT License**.  \n\n---\n\n🔥 **Enjoy Scraping!** 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrijitpatra99%2Fpuppeteer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrijitpatra99%2Fpuppeteer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrijitpatra99%2Fpuppeteer/lists"}