{"id":20982988,"url":"https://github.com/disqada/scraper","last_synced_at":"2026-01-23T07:37:00.715Z","repository":{"id":216283788,"uuid":"740901741","full_name":"DisQada/Scraper","owner":"DisQada","description":"Very simple and easy to use web scrapper","archived":false,"fork":false,"pushed_at":"2024-10-19T01:44:51.000Z","size":270,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-06T01:03:35.632Z","etag":null,"topics":["nodejs","npm","npm-package","scraper","tool","web","webscraper","webscraping"],"latest_commit_sha":null,"homepage":"http://docs.disqada.org/Scraper/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DisQada.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"nabil-alsaiad","ko_fi":"nabil_alsaiad","custom":["https://wise.com/share/nabila567"]}},"created_at":"2024-01-09T09:55:29.000Z","updated_at":"2024-10-19T01:44:55.000Z","dependencies_parsed_at":"2024-10-24T10:52:01.297Z","dependency_job_id":"ae47f8dd-b2b7-4cd5-8c0d-f67922815185","html_url":"https://github.com/DisQada/Scraper","commit_stats":{"total_commits":36,"total_committers":3,"mean_commits":12.0,"dds":0.2222222222222222,"last_synced_commit":"bed890e10ee52660ec79eb03fb8ce3770b2853d5"},"previous_names":["disqada/scraper"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DisQada%2FScraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DisQada%2FScraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DisQada%2FScraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DisQada%2FScraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DisQada","download_url":"https://codeload.github.com/DisQada/Scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254183014,"owners_count":22028400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nodejs","npm","npm-package","scraper","tool","web","webscraper","webscraping"],"created_at":"2024-11-19T05:47:19.769Z","updated_at":"2026-01-23T07:37:00.669Z","avatar_url":"https://github.com/DisQada.png","language":"JavaScript","funding_links":["https://github.com/sponsors/nabil-alsaiad","https://ko-fi.com/nabil_alsaiad","https://wise.com/share/nabila567"],"categories":[],"sub_categories":[],"readme":"# Scraper\n\n## Badges\n\n[![github](https://img.shields.io/badge/DisQada/Scraper-000000?logo=github\u0026logoColor=white)](https://www.github.com/DisQada/Scraper)\n[![npm](https://img.shields.io/badge/@disqada/scraper-CB3837?logo=npm\u0026logoColor=white)](https://www.npmjs.com/package/@disqada/scraper)\n\n![version](https://img.shields.io/npm/v/@disqada/scraper.svg?label=latest\u0026logo=npm)\n![monthly downloads](https://img.shields.io/npm/dm/@disqada/scraper.svg?logo=npm)\n\n[![Test](https://github.com/DisQada/Scraper/actions/workflows/test.yml/badge.svg)](https://github.com/DisQada/Scraper/actions/workflows/test.yml)\n[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg?logo=semantic-release)](https://github.com/semantic-release/semantic-release)\n\n## Table of Contents\n\n- [Scraper](#scraper)\n  - [Badges](#badges)\n  - [Table of Contents](#table-of-contents)\n  - [About](#about)\n  - [License](#license)\n- [Getting Started](#getting-started)\n  - [Basic information](#basic-information)\n  - [Usage](#usage)\n    - [A full node](#a-full-node)\n    - [A text value](#a-text-value)\n      - [TextOptions](#textoptions)\n    - [An attribute value](#an-attribute-value)\n  - [CLI commands](#cli-commands)\n    - [download](#download)\n      - [Arguments](#arguments)\n      - [Examples](#examples)\n\n## About\n\nThis tool uses its own parsing of the HTML into JSON, then this package scrolls through it to find the desired information.\n\nRead [this](#basic-information) for more information. Or visit [this](https://disqada.github.io/Scraper) website for the API types\n\n## License\n\nCopyright \u0026copy; 2024 [DisQada](https://links.disqada.org)\n\nThis tool is licensed under the [Apache 2.0](https://apache.org/licenses/LICENSE-2.0) License.  \nSee the [LICENSE](LICENSE) file for more information.\n\n# Getting Started\n\n## Basic information\n\nThe package breaks down the HTML into JSON nodes, following the concept that an HTML tag is represented as a JSON object/node.\n\nEach node represents an HTML tag with it's tag name, attributes and children nodes. The children nodes may contain text as string values\n\n\u003e Note that the parser completely ignores HTML comments `(\u003c!--example--\u003e)`\n\n## Usage\n\nFirst and most importantly, we need to have our HTML string ready in a variable.\n\nIf you're going to use the same HTML string for multiple uses, it's better to parse it yourself and then pass the JSON output to the functions (so the HTML string will be parsed only once). The following example shows how to do this:\n\n```js\nimport { parse } from '@disqada/scraper'\n\n// Use another package to fetch the HTML from the web\nconst html = `\n  \u003chtml\u003e\n    \u003chead\u003e\n      \u003ctitle\u003eTest Page\u003c/title\u003e\n    \u003c/head\u003e\n    \u003cbody\u003e\n      \u003ch1 id=\"title\"\u003eHello, world!\u003c/h1\u003e\n      \u003cp class=\"content\"\u003eThis is a test paragraph.\u003c/p\u003e\n    \u003c/body\u003e\n  \u003c/html\u003e\n`\nconst nodes = parse(html)\n\n// Rest of the code ...\n```\n\n### A full node\n\n```js\nimport { findNode } from '@disqada/scraper'\n\nconst node = findNode(nodes, {\n  tag: 'h1',\n  attr: {\n    key: 'id',\n    value: 'title'\n  }\n})\n\n// node = {\n//   type: 'element',\n//   tagName: 'h1'\n//   attributes: [{key: 'id', value: 'title' }],\n//   children: []\n// }\n```\n\n### A text value\n\n```js\nimport { grabText } from '@disqada/scraper'\n\nconst text = grabText(nodes, {\n  tag: 'p'\n})\n\n// text = 'This is a test paragraph.'\n```\n\n#### TextOptions\n\nThe function [`grabText`](https://disqada.github.io/Scraper/functions/grabText) can be given a [`TextOptions`](https://disqada.github.io/Scraper/types/TextOptions) object that specifies some configurations for the search process. Note that it's optional.\n\n\u003e Click on the blue highlighted `TextOptions` to read more.\n\n### An attribute value\n\n```js\nimport { grabAttr } from '@disqada/scraper'\n\nconst attr = grabAttr(nodes, { tag: 'p' }, 'class')\n\n// attr = 'content'\n```\n\n## CLI commands\n\n### download\n\nYou can download an HTML file and its parsed JSON file under the `scrap` folder in the root path of your project outside runtime by calling the `download` command CLI.\n\n#### Arguments\n\n\u003e Note that either `--url` or `--path` must be given\n\n| Arg name | required | Description                                                                    |\n| -------- | -------- | ------------------------------------------------------------------------------ |\n| `--file` | true     | Name of the downloaded HTML and parsed JSON file                               |\n| `--url`  | false    | Link of the web page                                                           |\n| `--path` | false    | Path of a local HTML file (the HTML file will be copied to the `scrap` folder) |\n\n#### Examples\n\n```bash\nscraper --url='https://example.com/sample' --file='sample'\n```\n\n```bash\nscraper --path='./samples/v1/index.html' --file='sample1'\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdisqada%2Fscraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdisqada%2Fscraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdisqada%2Fscraper/lists"}