{"id":18711884,"url":"https://github.com/gahabeen/jsonframe-cheerio","last_synced_at":"2025-10-25T14:46:03.904Z","repository":{"id":57300197,"uuid":"80193033","full_name":"gahabeen/jsonframe-cheerio","owner":"gahabeen","description":"simple multi-level scraper json input/output for Cheerio","archived":false,"fork":false,"pushed_at":"2023-01-13T11:13:48.000Z","size":110,"stargazers_count":199,"open_issues_count":5,"forks_count":24,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-30T05:09:35.584Z","etag":null,"topics":["frame","json","parsed-data","scraper","scraping","selector"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gahabeen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-27T08:51:23.000Z","updated_at":"2024-11-13T15:47:57.000Z","dependencies_parsed_at":"2023-02-09T15:31:21.791Z","dependency_job_id":null,"html_url":"https://github.com/gahabeen/jsonframe-cheerio","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gahabeen%2Fjsonframe-cheerio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gahabeen%2Fjsonframe-cheerio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gahabeen%2Fjsonframe-cheerio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gahabeen%2Fjsonframe-cheerio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gahabeen","download_url":"https://codeload.github.com/gahabeen/jsonframe-cheerio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247445681,"owners_count":20939961,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["frame","json","parsed-data","scraper","scraping","selector"],"created_at":"2024-11-07T12:40:42.972Z","updated_at":"2025-10-25T14:45:58.860Z","avatar_url":"https://github.com/gahabeen.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# This repository is deprecated. Use it at your own risk!\n\n---\n\n[![NPM](https://nodei.co/npm/jsonframe-cheerio.png?downloads=true\u0026downloadRank=true\u0026stars=true)](https://nodei.co/npm/jsonframe-cheerio/)\n\n\u003ch2 align=\"center\"\u003e\u003ci\u003ejson\u003c/i\u003eframe\u003c/h2\u003e\n\n\u003ch4 align=\"center\"\u003e\n\t\u003cspan\u003esimple multi-level scraper json input/output\u003c/span\u003e\u003cbr\u003e\n\t\u003cbr\u003e\n\t\u003ca href=\"https://www.npmjs.com/package/jsonframe-cheerio\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/badge/npm-jsonframe--cheerio-green.svg\" alt=\"npm jsonframe-cheerio\"\u003e\n\t\u003c/a\u003e\n\t\u003ca href=\"https://travis-ci.org/gahabeen/jsonframe-cheerio\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/travis/gahabeen/jsonframe-cheerio.svg\" alt=\"\"\u003e\n\t\u003c/a\u003e\n\t\u003ca href=\"https://github.com/cheeriojs/cheerio#cheerio\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/badge/plugin-Cheerio-red.svg\" alt=\"a Cheerio Plugin\" /\u003e\n\t\u003c/a\u003e\n\u003c/h4\u003e\n\n## **2.0.5x** features\n\n😍 **JSON Syntax**: input json, output the same structured json including with scraped data\n\n🌈 **Simple patterns**: simple inline `selectors`, `extractors`, `filters` and `parser`.\n\n💪 **Reliable \u0026 fast**: used in production within crawlers\n\n[See the full changelog](#changelog)\n\n## Example\n\n```js\nlet cheerio = require('cheerio')\nlet $ = cheerio.load(`\n\t\u003cbody\u003e\n\t\t\u003ch1\u003eI love jsonframe!\u003c/h1\u003e\n\t\t\u003cspan itemprop=\"email\"\u003e Email: gabin@datascraper.pro  \u003c/span\u003e\n\t\u003cbody\u003e`)\n\nlet jsonframe = require('jsonframe-cheerio')\njsonframe($) // initializing the plugin\n\nlet frame = {\n\t\"title\": \"h1\", // this is an inline selector\n\t\"email\": \"span[itemprop=email] \u003c email\" // output an extracted email\n}\n\nconsole.log( $('body').scrape(frame, { string: true } ))\n/*=\u003e\n{\n\t\"title\": \"I love jsonframe!\",\n\t\"email\": \"gabin@datascraper.pro\"\n}\n/*\n```\n\n## Use\n\n**Install the plugin** to your Node.js app through **NPM**\n\n```\nnpm i jsonframe-cheerio --save\n```\n\n## API\n\n### Loading\nStart by `loading Cheerio`.\n\n```js\nlet cheerio = require('cheerio')\nlet $ = cheerio.load(\"HTML DOM to load\") // See Cheerio API\n```\n\nThen `load the jsonframe plugin`.\n\n```js\nlet jsonframe = require('jsonframe-cheerio') // require from npm package\njsonframe($) // apply the plugin to the current Cheerio instance\n```\n\n### Scraper\n\nOnce the plugin is loaded, you've first got to set the **frame** of your data.\n\nLet's take the following `HTML example`:\n\n```html\n\u003chtml\u003e\n\u003chead\u003e\u003c/head\u003e\n\u003cbody\u003e\n    \u003ch2\u003ePricing\u003c/h2\u003e\n\t\t\u003cimg class=\"picture\" src=\"somepath/to/image.png\"\u003e\n\t\t\u003ca class=\"mainLink\" href=\"some/url/to/somewhere\"\u003eA Link\u003c/a\u003e\n\t\t\u003cspan class=\"date\"\u003e We are the 04/02/2017\u003c/span\u003e\n\t\t\u003cdiv class=\"popup\"\u003e\u003cspan\u003eSome inner content\u003c/span\u003e\u003c/div\u003e\n    \u003cul id=\"pricing\" class=\"menu\"\u003e\n        \u003cli class=\"item\"\u003e\n            \u003cspan class=\"planName\"\u003eHacker\u003c/span\u003e\n            \u003cspan class=\"planPrice\" price=\"0\"\u003eFree\u003c/span\u003e\n            \u003ca href=\"/hacker\"\u003e \u003cimg src=\"./img/hacker.png\"\u003e \u003c/a\u003e\n        \u003c/div\u003e\n        \u003cli class=\"item\"\u003e\n            \u003cspan class=\"planName\"\u003ePro\u003c/span\u003e\n            \u003cspan class=\"planPrice\" price=\"39.00\"\u003e$39\u003c/span\u003e\n            \u003ca href=\"/pro\"\u003e \u003cimg src=\"./img/pro.png\"\u003e \u003c/a\u003e\n        \u003c/div\u003e\n    \u003c/ul\u003e\n\t\u003cdiv id=\"contact\"\u003e\n\t\t\u003cspan itemprop=\"usaphone\"\u003ePhone USA: (912) 148-456\u003c/div\u003e\n\t\t\u003cspan itemprop=\"frphone\"\u003ePhone FR: +332 38 30 37 90\u003c/div\u003e\n\t\t\u003cspan itemprop=\"email\"\u003eEmail: lspurcell@suddenlink.net\u003c/div\u003e\n\t\u003c/div\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n```\n\n#### $( selector ).scrape( frame , {options})\n\n`selector` is defined in [Cheerio's documentation](https://github.com/cheeriojs/cheerio#-selector-context-root-)\n\n`frame` is a JSON or Javascript Object\n\n`{options}` are detailed [later in its own section](#options)\n\n```js\nlet frame = {\n\t\"title\": \"h2\" // CSS selector\n}\n```\n\nWe then pass the frame to the function:\n\n```js\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n//=\u003e {\"title\": \"Pricing\"}\n```\n\n### Frame\n\n#### Inline Selector\nMost common selector, `inline line` by specifying nothing more than the data name property and the selector as its value.\n\n```js\n...\nlet frame = { \"title\": \"h2\" }\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{ \"title\": \"Pricing\" }\n*/\n...\n```\n\n#### New : Inline attribute / extractor / parser\nYou can now declare everything in line. You should just be careful to always use them in the following order when combining them : `@ (attribute), | (extractor), || (parse)`.\n\n_See examples for each of them above._\n\n#### Attribute\n`_a: \"attributeName\"` allows you to retrieve `any attribute data`  \n`@` inside the selector `_s` allows you to do it inline\n\n```js\n...\nlet frame = {\n\t\"proPrice\": \".planName:contains('Pro') + span@price\"\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{ \"proPrice\": \"39.00\" }\n*/\n...\n```\n\n\n#### Extractor\n`\u003c` inside the selector `_s` allows you to do it inline\n\nIt currently supports `email` (also `mail`), `telephone` (also `phone`), `date`, `fullName` (or `firstName`, `lastName`, `initials`, `suffix`, `salutation`) and `html` (to get the inner html) and by default (no declaration), we get the `inner text`.\n\n```js\n...\nlet frame = {\n\t\"email\": \"[itemprop=email] \u003c phone\",\n\t\"frphone\": \"[itemprop=frphone] \u003c phone\"\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"email\": \"example@google.net\",\n\t\t\"frphone\": \"33238303790\"\n\t}\n*/\n...\n```\n\n#### Filter\n`|` inside the selector `_s` allows you to do it inline\n\nIt currently supports `trim` (remove spaces at beginning and end), `lowercase or lcase`, `uppercase or ucase`, `capitalize or cap`, `words or w`, `noescapchar or nec`, `compact or cmp` and `number or nb`.\n\n```js\n...\nlet frame = {\n\t\"email1\": \"[itemprop=email] \u003c phone | uppercase\",\n\t\"email2\": \"[itemprop=email] \u003c phone | capitalize\"\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"email1\": \"EXAMPLE@GOOGLE.NET\",\n\t\t\"email2\": \"EXAMPLE GOOGLE NET\"\n\t}\n*/\n...\n```\n\n#### Parse / Regex\n`||` inside the selector `_s` allows you to use regexes in line\n`_p: /regex/` allows you to extract data based on **regular expressions**  \n\n\n```js\n...\nlet frame = {\n\t\"data\": \".date || \\\\d{1,2}/\\\\d{1,2}/\\\\d{2,4}\"\n}\n\n// or use the longer version for proper regex entry\n\nlet frame = {\n\t\"data\": {\n\t\t_s: \".date\",\n\t\t_p: /\\d{1,2}\\/\\d{1,2}\\/\\d{2,4}/ // n[n]/n[n]/nn[nn] format here\n\t}\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"date\": \"04/02/2017\"\n\t}\n*/\n...\n```\n\n#### List / Array\n`_d: [{ }]` allows you to get an `array / list of data`  \n`_d: [\"selector\"]` will retrieves a list based on the selector inbetween quotes.  \n`_d: [\"firstSelector\", \"secondSelector\"]` works too and merge the results into one array\n\nYou could even shorten it more by listing right from the selector as follows:\n`\"selectorName\": [\".selector\"]` which returns an array of strings\n\n```js\n...\nlet frame = {\n\t\"pricing\": {\n\t\t_s: \"#pricing .item\",\n\t\t_d: [{\n\t\t\t\"name\": \".planName\",\n\t\t\t\"price\": \".planPrice\"\n\t\t}]\n\t}\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"pricing\": [\n\t\t\t{\n\t\t\t\t\"name\": \"Hacker\",\n\t\t\t\t\"price\": \"Free\"\n\t\t\t},\n\t\t\t{\n\t\t\t\t\"name\": \"Pro\",\n\t\t\t\t\"price\": \"$39\"\n\t\t\t}\n\t\t]\n\t}\n*/\n\n// Or a shorter way which works for simple string arrays\n\nlet frame = {\n\t\"pricingNames\": [\"#pricing .item .planName\"]\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"pricingNames\": [\"Hacker\", \"Pro\"]\n\t}\n*/\n...\n```\n#### Grouped\n`\"_g\": { _s: \"\", _d: {} }` allows you to group some data selectors by a parent selector without naming the parent. You can also extends the group property to add some meaning or simply have several groups at the same level.  \nGroup property name must be `_g` or `_group` followed by `_` and whatever string you want.  \nex: `_g_head : {}` or `_g_body : {}`\n\n```js\n...\nlet frame = {\n\t_g: {\n\t\t_s: \"#pricing .item\",\n\t\t_d: {\n\t\t\t\"name\": \".planName\",\n\t\t\t\"price\": \".planPrice\"\n\t\t}\n\t},\n\t_g_second: {\n\t\t_s: \"#pricing .item\",\n\t\t_d: {\n\t\t\t\"secondName\": \".planName\",\n\t\t\t\"secondPrice\": \".planPrice\"\n\t\t}\n\t}\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"name\": \"Hacker\",\n\t\t\"price\": \"Free\",\n\t\t\"secondName\": \"Hacker\",\n\t\t\"secondPrice\": \"Free\"\n\t}\n*/\n...\n```\n\n\n#### Nested\n`\"parent\": { _s: \"parentSelector\", _d: {} }` allows you to segment your data by `setting a parent section` from which the child data will be scraped.  \n\nYou can also use `\"parent\": { }` when you only want to nest data into objects without setting a parent selector.\n\n```js\n...\nlet frame = {\n\t\"pricing\": {\n\t\t_s: \"#pricing .item\",\n\t\t_d: {\n\t\t\t\"name\": \".planName\",\n\t\t\t\"price\": \".planPrice\"\n\t\t}\n\t}\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"pricing\":{\n\t\t\t\"name\": \"Hacker\",\n\t\t\t\"price\": \"Free\"\n\t\t}\n\t}\n*/\n...\n```\n\n\u003e Note here that we get the first returned result (#pricing .item).\n\n#### Example\nSee how you can properly `structure your data`, ready for the output!\n\n```js\n...\nlet frame = {\n\t\"pricing\": {\n\t\t_s: \"#pricing .item\",\n\t\t_d: [{\n\t\t\t\"name\": \".planName\",\n\t\t\t\"price\": \".planPrice @ price\",\n\t\t\t\"image\": {\n\t\t\t\t\"url\": \"img @ src\",\n\t\t\t\t\"link\": \"a @ href\"\n\t\t\t}\n\t\t}]\n\t}\n}\n\nlet result = $('body').scrape(frame, { string: true })\nconsole.log( result )\n\n/* output =\u003e\n\t{\n\t\t\"pricing\":[\n\t\t\t{\n\t\t\t\t\"name\": \"Hacker\",\n\t\t\t\t\"price\": \"0\",\n\t\t\t\t\"image\": {\n\t\t\t\t\t\"url\": \"./img/hacker.png\",\n\t\t\t\t\t\"link\": \"/hacker\"\n\t\t\t\t}\n\t\t\t},\n\t\t\t{\n\t\t\t\t\"name\": \"Pro\",\n\t\t\t\t\"price\": \"39.00\",\n\t\t\t\t\"image\": {\n\t\t\t\t\t\"url\": \"./img/pro.png\",\n\t\t\t\t\t\"link\": \"/pro\"\n\t\t\t\t}\n\t\t\t}\n\t\t]\n\t}\n*/\n...\n```\n\n\u003e Note here that we get the first returned result (#pricing .item).\n\n\n### Options\n\n```js\n...\nlet frame = {\n\t\"proPrice\": {\n\t\t_s: \".planName:contains('Pro') + span\",\n\t\t_a: \"price\"\n\t}\n}\n\nlet result = $('body')\n\t.scrape(frame, {\n\t\t\ttimestats: true, // default: false\n\t\t\tstring: true // default: false\n\t\t})\nconsole.log(result)\n\n/* output =\u003e\n\t{\n\t\t\"proPrice\": {\n\t\t\t\"value\":\"39.00\",\n\t\t\t\"_timestats\": \"1\" // ms\n\t\t}\n\n\t}\n*/\n...\n```\n\n## Tests\n\nOne shot tests\n```bash\nnpm run test\n```\n\nWatching test on updates\n```bash\nnpm run test-watch\n```\n\n## Changelog\n\n⚠ Careful if you've been using **jsonframe** from the **version 1.x.x**, some things changed to make it more **flexible**, **faster to use (inline parameters)** and **more meaningful in the syntax**.\n\n**2.0.52** (28/02/2017)\n- Update the email regex\n- Update the website regex\n- Fix array into array results\n- Improving script efficiency getting data from node(s)\n- Fix date extractor when no date to extract\n\n**2.0.51** (27/02/2017)\n- Fix a fatal error (argh) which was just a typo about the new chained extractors\n\n**2.0.50** (27/02/2017)\n- Extractors chaining is now possible. For ex: `.selector \u003c html email` would work\n\n**2.0.49** (27/02/2017)\n- Fixing issue when attribute doesn't exists (@ attributeNmae)\n- Improving array of object management (need to find a way to avoid empty objects still)\n\n**2.0.48** (27/02/2017)\n- Add Filter `Split(char)` to split string based on character (default to whitespace)\n- Add Extractor `numbers or nb` (return potentially an array)\n- Update Filter `numbers or nb` (simply filter the string to output only numbers)\n- Add Filter `between(string1\u0026\u0026string2)` to filter data by starting and finishing string\n- Add Filter `before(string)` to get data before a string\n- Add Filter `after(string)` to get data after a string\n- Add array support to Filter `left(nb)` and `right(nb)` (slice the array elements)\n- Add Filter `fromto(startNb,endNb)` to either slice an array or a string from index to index\n- Add Filter `get(nb)` to extract either an array item or a character from a string\n\n**2.0.46** (26/02/2017)\n- Rebuild of the Unstructured scraper with breaks (_b) - Works like a charm now!\n\n**2.0.45** (25/02/2017)\n- Fix weird fullName parsing in some cases\n- Update Handle of Regex - Is now able to capture a group with a regex\n\n**2.0.44** (24/02/2017)\n- Inline array for extractors like `\"mails\": [\".parentSelector \u003c email\"]`\n- Adds french words: `prenom` and `nom` to humanname extractor\n- Add filters: `right(number)`, `left(number)`\n- Set a stricter regex for email extractor `/([a-zA-Z0-9._-]{0,30}@[a-zA-Z0-9._-]{0,15}\\.[a-zA-Z0-9._-]{0,15})/gmi`\n\n**2.0.3** (23/02/2017)\n- Possibility to scrape unstructured data with breaks (`_b`). More about this soooon in the readme.\n- New filters: `words or w`, `noescapchar or nec` and `compact or cmp`\n- Multi-filters is available now. Ex: `.selector | words compact`. Simply separated by spaces.\n- Disabling google libphonenumber for now\n\n**2.0.2** (15/02/2017)\n- String option to get a stringified output right away\n- Multi-groups possibility at same level (several _g wouldn't work as same property name) in frame like _g_head and _g_body for example\n- Joined arrays/lists with [\"firstlist.selector\", \"secondlist.selector\", \"...\"] when inline\n- Better handling of img node - automatic src attribute is output (if nothing else set)\n\n**2.0.1** (14/02/2017)\n- Fixed the non-passing tests and added all the new ones for 2.x.x updates\n- Refactoring the way data is processed for future multiple occurences\n\n**2.0.0** (12/02/2017)\n- ⚠ Changing  ~~`Type`~~ for `Extractor` with shortcode `\u003c` instead of `|`\n- ⚠ `filters` with the shortcode `|`\n- Inline parameters support for `\"attribute\"`, `\"extractor\"` and `\"parse\"`\n- Simple string arrays from inline selector\n- Group property to group data selectors whitout naming the group (childs take the place of the group property `\"_g\"` or `\"_group\"` )\n\n\n**1.1.1** (05/02/2017)\n- Short \u0026 functionnal parameters ( `_s`, `_t`, `_a`) instead of `\"selector\"`, `\"extractor\"`, `\"attr\"`. Idea behind being to easily differentiate **retrieved data name** to **functionnal data**.\n- Automatic handler for `img` selected element (automatically retrieve the img src link)\n- `_parent_` selector to target the **parent content**\n- A **regex parser** with the functionnal parameter **parse**: `_p` (`_parse` works too)\n- **Extractor** `_t: \"html\"` feature to get back **inner html of a selector**\n- Added **timestats** to measure time spent on each node via `.scrape(frame, {timestats: true})`\n- Refactorization of the whole code to make it evolutive (DRY)\n- Update of the tests cases accordingly\n\n\n**1.0.0** (27/01/2017)\n- Stable version release with basic features  \n\n## Contributing 🤝\n\u003e Feel free to follow the procedure to make it even more awesome!\n\n1. Create an `issue` so we `get the discussion started`\n2. Fork it!\n3. Create your feature branch: `git checkout -b my-new-feature`\n4. Commit your changes: `git commit -am 'Add some feature'`\n5. Push to the branch: `git push origin my-new-feature`\n6. Submit a pull request :D\n\n\n## License\n[Gabin Desserprit](mailto:gabin@datascraper.pro) - [datascraper.pro](datascraper.pro)  \nReleased under MIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgahabeen%2Fjsonframe-cheerio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgahabeen%2Fjsonframe-cheerio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgahabeen%2Fjsonframe-cheerio/lists"}