{"id":20596604,"url":"https://github.com/suhailroushan13/techcrunch-api","last_synced_at":"2025-08-11T00:34:00.619Z","repository":{"id":238173346,"uuid":"796027816","full_name":"suhailroushan13/techcrunch-api","owner":"suhailroushan13","description":"TechCrunch API is a Node.js package that allows you to scrape articles from TechCrunch based on categories or tags. This package is designed for systems using Ubuntu or other Debian-based distributions that support sudo commands, leveraging Puppeteer ","archived":false,"fork":false,"pushed_at":"2024-05-06T21:01:42.000Z","size":20,"stargazers_count":22,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-08-09T17:21:27.537Z","etag":null,"topics":["api","news","newsapi","nodejs","npm","npm-package","techcrunch"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/techcrunch-api","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/suhailroushan13.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-05-04T18:04:26.000Z","updated_at":"2025-07-10T19:23:27.000Z","dependencies_parsed_at":"2024-05-04T18:44:53.121Z","dependency_job_id":"4bc74087-673f-4658-8faa-e2d50a2c4ad0","html_url":"https://github.com/suhailroushan13/techcrunch-api","commit_stats":null,"previous_names":["suhailroushan13/techcrunch-api"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/suhailroushan13/techcrunch-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suhailroushan13%2Ftechcrunch-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suhailroushan13%2Ftechcrunch-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suhailroushan13%2Ftechcrunch-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suhailroushan13%2Ftechcrunch-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/suhailroushan13","download_url":"https://codeload.github.com/suhailroushan13/techcrunch-api/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suhailroushan13%2Ftechcrunch-api/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269808223,"owners_count":24478503,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-10T02:00:08.965Z","response_time":71,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","news","newsapi","nodejs","npm","npm-package","techcrunch"],"created_at":"2024-11-16T08:17:45.660Z","updated_at":"2025-08-11T00:34:00.596Z","avatar_url":"https://github.com/suhailroushan13.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TechCrunch API 🧑‍💻\n\nTechCrunch API is a Node.js package that allows you to scrape articles from TechCrunch based on categories or tags. This package is designed for systems using Ubuntu or other Debian-based distributions that support `sudo` commands, leveraging Puppeteer to navigate and scrape content from a headless Chromium environment. 🌐\n\n## Features 🚀\n\n- **Scrape by Category:** Automatically retrieve all articles under a specified category. 📂\n- **Scrape by Tag:** Collect articles that are tagged with a specific keyword. 🏷️\n- **Headless Browser Support:** Runs Chromium in headless mode to scrape dynamic content. 👻\n- **Optimized for Ubuntu:** Includes installation instructions specifically for Ubuntu, but compatible with other Linux distributions. 🐧\n\n## Prerequisites 📋\n\nBefore installing the TechCrunch Scraper, you need to ensure your system has the following dependencies installed:\n\n- Node.js (Version 14 or later recommended) 🟢\n- Puppeteer 🎭\n- Dependencies required for Puppeteer and headless Chromium 🔧\n\n## Installation\n\nFollow these steps to set up the TechCrunch Scraper package:\n\n### Step 1: Install System Dependencies\n\nOpen a terminal and execute the following commands to install necessary libraries:\n\n```bash\nnpm install puppeteer\nsudo apt-get update\nsudo apt-get install -y libgbm-dev xvfb chromium-browser libvpx7 libevent-2.1-7 libharfbuzz-icu0  libwebpdemux2 libenchant-2-2 libsecret-1-0  libmanette-0.2-0 libflite1  libgles2-mesa\nXvfb :99 -screen 0 1920x1080x24 \u0026\nexport DISPLAY=:99\n```\n\n### Step 2: Install TechCrunch API Package\n\nInstall the package via npm with the following command:\n\n```bash\nnpm install techcrunch-api\n```\n\n## Usage\n\n#### After installation, you can use the package in your Node.js scripts as follows:\n\n# ES6 Syntax\n\n```javascript\nimport { getByCategory, getByTag } from \"techcrunch-api\";\n\n// Fetch articles by category using async/await\n// Valid categories/tags for fetching articles (must be used in lowercase):\n// 1. media-entertainment\n// 2. transportation\n// 3. cryptocurrency\n// 4. security\n// 5. artificial-intelligence\n// 6. apps\n// 7. fintech\n// 8. startups\n// 9. venture\n// 10. hardware\n\nconst fetchArticles = async () =\u003e {\n  try {\n    const articles = await getByCategory(\"security\"); \n    console.log(articles);\n  } catch (error) {\n    console.error(\"Error fetching articles:\", error);\n  }\n};\n\nfetchArticles();\n\nconst fetchTag = async () =\u003e {\n  try {\n    const tags = await getByTag(\"apis\");\n    console.log(tags);\n  } catch (error) {\n    console.error(\"Error fetching tags:\", error);\n  }\n};\n\nfetchTag();\n```\n\n\n## Running the Scraper\n\n```bash\nnode app.js \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuhailroushan13%2Ftechcrunch-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuhailroushan13%2Ftechcrunch-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuhailroushan13%2Ftechcrunch-api/lists"}