{"id":20671346,"url":"https://github.com/allnulled/fast-scrap","last_synced_at":"2026-04-17T16:31:02.558Z","repository":{"id":98546682,"uuid":"116406256","full_name":"allnulled/fast-scrap","owner":"allnulled","description":"Web scrapping made funny (based on ScrapperJS)","archived":false,"fork":false,"pushed_at":"2018-01-05T17:39:15.000Z","size":3,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-12T06:39:04.281Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allnulled.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-05T16:44:28.000Z","updated_at":"2021-06-25T11:15:36.000Z","dependencies_parsed_at":"2023-05-29T11:30:23.136Z","dependency_job_id":null,"html_url":"https://github.com/allnulled/fast-scrap","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/allnulled/fast-scrap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allnulled%2Ffast-scrap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allnulled%2Ffast-scrap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allnulled%2Ffast-scrap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allnulled%2Ffast-scrap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allnulled","download_url":"https://codeload.github.com/allnulled/fast-scrap/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allnulled%2Ffast-scrap/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31936487,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T12:37:54.787Z","status":"ssl_error","status_checked_at":"2026-04-17T12:37:25.095Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T20:26:27.997Z","updated_at":"2026-04-17T16:31:02.542Z","avatar_url":"https://github.com/allnulled.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fast Scrapper\n\n## 1. About\n\n**Fast Scrapper** is a small utility to ease the routine of scrapping data from external websites. It is based on ScrapperJS (which is based on CasperJS, and PhantomJS).\n\n## 2. Why?\n\nJust to reduce the effort of writing a new scrapper each time one has to get some data from other sites.\n\n## 3. Installation\n\nFirst, you need to download this project, and open a command-line inside.\n\nThen you must install the dependencies, so:\n\n    ~$ npm install\n\nIn order to use FastScrap, you only need to import the `fast-scrap.js` in your `*.js` file, like this:\n\n    var FastScrap = require(\"./fast-scrap.js\");\n\n\n## 4. Usage\n\nUse FastScrap calling its method `run`, and passing to it an object with all the required parameters for the scrapping.\n\nIn the `example.js` file you can find a real example, that will create a file, `dummyexample.html`, with the title of `http://www.example.com`. \n\nThis is how we can call the `FastScrap.run({...})` method (all the parameters are required):\n\n    FastScrap.run({\n\t    routes: [\"http://www.example.com\"],\n\t    urlPattern: \"*\",\n\t    mode: \"static\",\n\t    onStart: function() {\n            // Here, the initial operations\n            // Typically, start an empty file, or so\n\t    },\n\t    onLoad: function($) {\n\t        // Here, retrieve the data from page\n\t        // jQuery is available already. \n\t        // Example:\n\t    \treturn $(\"h1\").eq(0).html();\n\t    },\n\t    onFail: function(url) {\n            // Here, manage the failed scraps\n            // Typically, log some error\n\t    },\n\t    onStore: function(data, utils) {\n\t    \t// Here, store the scrapped data\n\t    \t// Typically, persisting the data\n\t    },\n\t    onFinish: function() {\n            // Here, the conclusive operations\n            // Typically, to close things\n\t    }\n    });\n\nTo run the scrap itself, and suposing that your file is called `example.js` (as in the example), you only need to:\n\n    ~$ node example.js\n\nAnd the scrap will start then.\n\n## 5. Notes\n\nYou can scrap multiple URLs effortlessly with FastScrap, passing all the routes you want to scrap the same way.\n\nTake into account that FastScrap will only allow you to run 1 instance at a time. On the other hand, it will also help to you to separate the different scrappings that you want to do, forcing either to put conditionals in the code, or simply separating the different scraps you want to perform in different files.\n\nIt was thought for simple small scraps, not for testing, neither for crawling. Who knows, maybe with time it's improved, but currently, this is how it comes.\n\n\n## 6. License\n\nScrap them all (for free, of course). \n\nFeel free to use and abuse it without feedback.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallnulled%2Ffast-scrap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallnulled%2Ffast-scrap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallnulled%2Ffast-scrap/lists"}