{"id":22731555,"url":"https://github.com/fahimfba/web-scraper","last_synced_at":"2025-04-14T00:43:35.229Z","repository":{"id":117960713,"uuid":"410968043","full_name":"FahimFBA/Web-Scraper","owner":"FahimFBA","description":"Extract data from websites using the web-scrapper. Made with nodejs, ExpressJS, axios \u0026 cheerio.","archived":false,"fork":false,"pushed_at":"2024-09-13T09:44:57.000Z","size":1113,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T14:54:59.290Z","etag":null,"topics":["axios","cheerio","cheeriojs","javascript","js","npm","npm-package","webscrape","webscraping","webscraping-data","webscraping-search","webscrapper"],"latest_commit_sha":null,"homepage":"https://fahimfba.github.io/Web-Scraper/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FahimFBA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-27T16:51:08.000Z","updated_at":"2024-10-18T22:12:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"8382a18a-d527-4f0d-93e4-b4fd8e95db48","html_url":"https://github.com/FahimFBA/Web-Scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FahimFBA%2FWeb-Scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FahimFBA%2FWeb-Scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FahimFBA%2FWeb-Scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FahimFBA%2FWeb-Scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FahimFBA","download_url":"https://codeload.github.com/FahimFBA/Web-Scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804721,"owners_count":21164127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["axios","cheerio","cheeriojs","javascript","js","npm","npm-package","webscrape","webscraping","webscraping-data","webscraping-search","webscrapper"],"created_at":"2024-12-10T19:28:55.061Z","updated_at":"2025-04-14T00:43:35.209Z","avatar_url":"https://github.com/FahimFBA.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web Scraper\n\n\u003cdiv align=\"center\"\u003e\n\n⭐ the repo if you like this project 😀\n\n\u003cbr\u003e\n\nYou can check the live feed from [here](https://youtu.be/NvXpo41vNrQ) as well. 😀\n\n\u003c/div\u003e\n\u003cbr\u003e\n\nWhat is a Web-Scraper?\n\nAccording to [Wikipedia](https://en.wikipedia.org/wiki/Web_scraping), \" Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. The web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. \"\n\n\n# Used languages \u0026 framework:\n\n![JavaScript](https://badges.aleen42.com/src/javascript.svg)\n![nodejs](https://badges.aleen42.com/src/node.svg)\n![ExpressJS](https://img.shields.io/badge/-ExpressJS-yellow)\n![npm](https://badges.aleen42.com/src/npm.svg)\n![axios](https://img.shields.io/badge/-axios-lightgrey)\n![cheerio](https://img.shields.io/badge/-cheerio-lightgrey)\n\n\n\n# Run the scrapper\n\n- Clone the repository\n    - Using SSH \n\n      ```\n      git clone git@github.com:FahimFBA/Web-Scraper.git\n      ```\n    - Using HTTPS\n\n      ```\n      git clone https://github.com/FahimFBA/Web-Scraper.git\n      ```\n- Go to the Web-Scraper directory\n\n```\ncd Web-Scraper\n```\n\n- Run the project using the following command\n\n```\nnpm run start\n```\n\nBy default, it would scrap from The Guardian as I used [The Guardian](https://www.theguardian.com/international) to experiment with the web scrapper.\n\nTo experiment on different websites, change the url in the index.js and customize the class in the axios as well.\n\n\n##  Output (Using VS Code)\n\n\n![Output](img/output.png)\n\n\nSpecial thanks goes to [Ania Kubów](https://www.youtube.com/channel/UC5DNytAJ6_FISueUfzZCVsw)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffahimfba%2Fweb-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffahimfba%2Fweb-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffahimfba%2Fweb-scraper/lists"}