{"id":16236282,"url":"https://github.com/ganevdev/actor-webdesignernews-scraper","last_synced_at":"2025-10-30T20:53:40.473Z","repository":{"id":143754647,"uuid":"181773426","full_name":"ganevdev/actor-webdesignernews-scraper","owner":"ganevdev","description":"Scraper for www.webdesignernews.com, using Apify.","archived":false,"fork":false,"pushed_at":"2019-07-10T22:21:51.000Z","size":525,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-28T19:38:11.611Z","etag":null,"topics":["actor","apify","scrap","scraper","scraping","web-scraper"],"latest_commit_sha":null,"homepage":"https://apify.com/ganevru/webdesignernews-scraper","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ganevdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-16T21:56:23.000Z","updated_at":"2023-02-15T13:47:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"11d6ef35-3df6-4df7-9af6-a51a9b4e0f6d","html_url":"https://github.com/ganevdev/actor-webdesignernews-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ganevdev%2Factor-webdesignernews-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ganevdev%2Factor-webdesignernews-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ganevdev%2Factor-webdesignernews-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ganevdev%2Factor-webdesignernews-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ganevdev","download_url":"https://codeload.github.com/ganevdev/actor-webdesignernews-scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244002859,"owners_count":20382068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor","apify","scrap","scraper","scraping","web-scraper"],"created_at":"2024-10-10T13:29:52.716Z","updated_at":"2025-10-30T20:53:40.409Z","avatar_url":"https://github.com/ganevdev.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web Designer News - Scraper\n\nScraper for www.webdesignernews.com, using Apify.\n\n## Input settings\n\nOnly `startUrl` required, for example this could be the main page: `https://www.webdesignernews.com`, some of the pagination pages: `https://www.webdesignernews.com/page/2` or some specific section: `https://www.webdesignernews.com/category/apps` or `https://www.webdesignernews.com/category/apps/page/3`\n\n`wayToScrape` can be `old` or `new`, by default `old`. If, for example, the starting link is on the third page (`https://webdesignernews.com/page/3` for example), then scraper will scrape third page, fourth page, fifth and so on, until pagination or `maxRequestsPerCrawl` limit ends. But if the `wayToScrape` is `new` - third, second and first pages will be scraped. So you decide which way actor will work.\n\n## Apify use\n\nThis actor is published in Apify library: [https://apify.com/ganevru/webdesignernews-scraper](https://apify.com/ganevru/webdesignernews-scraper).\n\nHow to run actor in Apify: [how to run section](https://apify.com/ganevru/webdesignernews-scraper?section=how-to-run)\n\n## Local use\n\nIt is not necessary to use the [apify.com](https://apify.com) service to use actor. Copy this repository and install node dependencies, for example, this way:\n\n```\ngit clone git@github.com:Ganevru/actor-webdesignernews-scraper.git\ncd actor-webdesignernews-scraper\nnpm install\n```\n\nCreate in the root of project: `apify_storage/key_value_stores/default/INPUT.json`\n\nIn `INPUT.json`, write your settings, for example:\n\n```json\n{\n  \"startUrl\": \"https://webdesignernews.com/page/3\",\n  \"wayToScrape\": \"old\",\n  \"maxRequestsPerCrawl\": 3,\n  \"maxRequestRetries\": 3,\n  \"maxConcurrency\": 3,\n  \"liveView\": true,\n  \"proxyConfiguration\": {\n    \"useApifyProxy\": false\n  }\n}\n```\n\nStart scrapping with command:\n\n```\nnpm run local-start\n```\n\nOr, if you need a clean start:\n\n```\nnpm run local-start-fresh\n```\n\nThis will remove the results of all previous launches!\n\nThe result of the launch will be in `apify_storage/datasets/default`\n\nIt will be something like this:\n\n```json\n{\n  \"title\": \"AMP Pages Fundamentally Changing How the Web Works\",\n  \"vote\": 47,\n  \"thumb\": \"https://webdesignernews.s3.amazonaws.com/post_images/2657750/vlcsnap-2019-04-17-00h58m00s057-728x407-1d95-140x112.jpg\",\n  \"link\": \"https://www.webdesignernews.com/redirect/id/2657750\",\n  \"source\": \"androidpolice.com\",\n  \"date\": \"2 days ago\",\n  \"requestUrl\": \"https://webdesignernews.com/page/3\"\n}\n```\n\n```json\n{\n  \"title\": \"Managing Complexity\",\n  \"vote\": 71,\n  \"thumb\": \"https://webdesignernews.s3.amazonaws.com/post_images/2657412/1*RiHC2-4TI3SKYnw4TCSbrg-f33b-140x112.jpeg\",\n  \"link\": \"https://www.webdesignernews.com/redirect/id/2657412\",\n  \"source\": \"uxdesign.cc\",\n  \"date\": \"4 days ago\",\n  \"requestUrl\": \"https://webdesignernews.com/page/4\"\n}\n```\n\n```json\n{\n  \"title\": \"The Psychology of Pricing\",\n  \"vote\": 111,\n  \"thumb\": \"https://webdesignernews.s3.amazonaws.com/post_images/2659276/price-comparison-old-ug-e5f2-140x112.png\",\n  \"link\": \"https://www.webdesignernews.com/redirect/id/2659276\",\n  \"source\": \"quicksprout.com\",\n  \"date\": \"2 days ago\",\n  \"requestUrl\": \"https://webdesignernews.com/page/3\"\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fganevdev%2Factor-webdesignernews-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fganevdev%2Factor-webdesignernews-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fganevdev%2Factor-webdesignernews-scraper/lists"}