{"id":26216476,"url":"https://github.com/joseortuno/sentinel-scraper","last_synced_at":"2025-07-29T23:17:19.862Z","repository":{"id":40794545,"uuid":"237503257","full_name":"joseortuno/sentinel-scraper","owner":"joseortuno","description":"Scraper is a tool for scraping websites","archived":false,"fork":false,"pushed_at":"2023-03-04T05:52:29.000Z","size":4690,"stargazers_count":2,"open_issues_count":10,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-08T11:13:00.677Z","etag":null,"topics":["javascript","scraping","tool"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joseortuno.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-31T19:40:15.000Z","updated_at":"2023-03-08T11:13:00.677Z","dependencies_parsed_at":"2023-01-31T14:31:09.201Z","dependency_job_id":null,"html_url":"https://github.com/joseortuno/sentinel-scraper","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseortuno%2Fsentinel-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseortuno%2Fsentinel-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseortuno%2Fsentinel-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseortuno%2Fsentinel-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joseortuno","download_url":"https://codeload.github.com/joseortuno/sentinel-scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243213867,"owners_count":20254879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["javascript","scraping","tool"],"created_at":"2025-03-12T11:45:11.081Z","updated_at":"2025-03-12T11:45:11.722Z","avatar_url":"https://github.com/joseortuno.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scraper is a tool for scraping web througt of url and selectors\n\nv. 2.0.0\n\n## Usage\n\nCall to scraping tool:\n\n```javascript\nconst Scraper = require('sentinel-scraper');\n```\n\nCreate a instance for scraping an url:\n\n```javascript\nconst scraping = new Scraper('The url that do need scraping');\n```\n\n### Methods\n\n#### 1. SELECTOR\n\nTo scrape sections of a url through its selectors.\n\n```javascript\nscraping.select(selector, expression); // It is necesary insert the parameters.\n```\n\nParameters:\n\n1. selector: behaves as a '`.querySlectorAll()`'.\n2. expression (callback): currentValue, index (optional).\n\nRun a method for scraping a page:\n\n```javascript\nconst data = scraping.select('#selector', item =\u003e {\n  return item.children.item(0).href);\n})\n\n/* Output:\n  data = [\n    http//:www.example.com,\n    http//:www.example.com,\n    http//:www.example.com,\n    http//:www.example.com,\n    http//:www.example.com,\n    http//:www.example.com,\n  ]\n*/\n\n// Return an array with format you need. For example:\n\nconst data = scraping.select('#selector', item =\u003e {\n  return {\n    title: item.children.item(0).textContent,\n    image: item.children.item(0).src,\n    url: item.children.item(0).href,\n    });\n});\n\n/* Output:\n  data = [\n    {\n      title: 'lorem ipsum',\n      image: http//:www.example.com/image/image.png,\n      url: http//:www.example.com,\n    },\n    {\n      title: 'lorem ipsum',\n      image: http//:www.example.com/image/image.png,\n      url: http//:www.example.com,\n    },\n    {\n      title: 'lorem ipsum',\n      image: http//:www.example.com/image/image.png,\n      url: http//:www.example.com,\n    }\n  ]\n*/\n\n// Or create data in format you need without return nothing. For example:\n\nconst data = {};\nscraping.select('#selector', (item, index) =\u003e {\n  data[index] = [\n    item.children.item(0).textContent,\n    item.children.item(0).src,\n    item.children.item(0).href,\n  ];\n});\n\n/* Output:\n  data = {\n    1: [\n      'lorem ipsum',\n      http//:www.example.com/image/image.png,\n      http//:www.example.com,\n    ],\n    2: [\n      'lorem ipsum',\n      http//:www.example.com/image/image.png,\n      http//:www.example.com,\n    ],\n    3: [\n      'lorem ipsum',\n      http//:www.example.com/image/image.png,\n      http//:www.example.com,\n    ]\n  ]\n*/\n```\n\n#### 2. FOR\n\nIt is a static method for to scrape an array of urls. It is a factory of new Scraper();\n\n```javascript\nScraper.for(urls, expression); // It is necesary insert the parameters.\n```\n\nParameters:\n\n1. urls: array of urls.\n2. expression (callback): currentValue (instance of Scrape for url).\n\n```javascript\nconst urls = [\n  'http//:www.example.com/product/1',\n  'http//:www.example.com/product/2',\n  'http//:www.example.com/product/3',\n  'http//:www.example.com/product/4',\n  'http//:www.example.com/product/5'\n];\n\nScraper.for(urls, (scrape, index, url) =\u003e {\n  // the url return url parameter extracted\n  // for example: scrape.select();\n});\n````\n\nThe static method for we will use it when we want scrape different depths.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoseortuno%2Fsentinel-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoseortuno%2Fsentinel-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoseortuno%2Fsentinel-scraper/lists"}