Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tautologistics/node-htmlparser

Forgiving HTML/XML/RSS Parser in JS for *both* Node and Browsers
https://github.com/tautologistics/node-htmlparser

Last synced: 28 days ago
JSON representation

Forgiving HTML/XML/RSS Parser in JS for *both* Node and Browsers

Awesome Lists containing this project

README 

        
#NodeHtmlParser

A forgiving HTML/XML/RSS parser written in JS for both the browser and NodeJS (yes, despite the name it works just fine in any modern browser). The parser can handle streams (chunked data) and supports custom handlers for writing custom DOMs/output.



##Installing



	npm install htmlparser



##Running Tests



###Run tests under node:

	node runtests.js



###Run tests in browser:

View runtests.html in any browser



##Usage In Node



```javascript

var htmlparser = require("htmlparser");

var rawHtml = "Xyz var foo = '<<bar>>';< /  script><!--<!-- Waah! -- -->";

var handler = new htmlparser.DefaultHandler(function (error, dom) {

	if (error)

		[...do something for errors...]

	else

		[...parsing done, do something...]

});

var parser = new htmlparser.Parser(handler);

parser.parseComplete(rawHtml);

sys.puts(sys.inspect(handler.dom, false, null));

```



##Usage In Browser



```javascript

var handler = new Tautologistics.NodeHtmlParser.DefaultHandler(function (error, dom) {

	if (error)

		[...do something for errors...]

	else

		[...parsing done, do something...]

});

var parser = new Tautologistics.NodeHtmlParser.Parser(handler);

parser.parseComplete(document.body.innerHTML);

alert(JSON.stringify(handler.dom, null, 2));

```



##Example output



```javascript

[ { raw: 'Xyz ', data: 'Xyz ', type: 'text' }

  , { raw: 'script language= javascript'

  , data: 'script language= javascript'

  , type: 'script'

  , name: 'script'

  , attribs: { language: 'javascript' }

  , children: 

     [ { raw: 'var foo = \'<bar>\';<'

       , data: 'var foo = \'<bar>\';<'

       , type: 'text'

       }

     ]

  }

, { raw: '<!-- Waah! -- '

  , data: '<!-- Waah! -- '

  , type: 'comment'

  }

]

```



##Streaming To Parser



```javascript

while (...) {

	...

	parser.parseChunk(chunk);

}

parser.done();	

```



##Streaming To Parser in Node



```javascript

fs.createReadStream('./path_to_file.html').pipe(parser);

```



##Parsing RSS/Atom Feeds



```javascript

new htmlparser.RssHandler(function (error, dom) {

	...

});

```



##DefaultHandler Options



###Usage



```javascript

var handler = new htmlparser.DefaultHandler(

	  function (error) { ... }

	, { verbose: false, ignoreWhitespace: true }

	);

```



###Option: ignoreWhitespace

Indicates whether the DOM should exclude text nodes that consists solely of whitespace. The default value is "false".



####Example: true



The following HTML:



```html

<font>

	<br>this is the text

<font>

```



becomes:



```javascript

[ { raw: 'font'

  , data: 'font'

  , type: 'tag'

  , name: 'font'

  , children: 

     [ { raw: 'br', data: 'br', type: 'tag', name: 'br' }

     , { raw: 'this is the text\n'

       , data: 'this is the text\n'

       , type: 'text'

       }

     , { raw: 'font', data: 'font', type: 'tag', name: 'font' }

     ]

  }

]

```



####Example: false



The following HTML:



```html

<font>

	<br>this is the text

<font>

```



becomes:



```javascript

[ { raw: 'font'

  , data: 'font'

  , type: 'tag'

  , name: 'font'

  , children: 

     [ { raw: '\n\t', data: '\n\t', type: 'text' }

     , { raw: 'br', data: 'br', type: 'tag', name: 'br' }

     , { raw: 'this is the text\n'

       , data: 'this is the text\n'

       , type: 'text'

       }

     , { raw: 'font', data: 'font', type: 'tag', name: 'font' }

     ]

  }

]

```



###Option: verbose

Indicates whether to include extra information on each node in the DOM. This information consists of the "raw" attribute (original, unparsed text found between "<" and ">") and the "data" attribute on "tag", "script", and "comment" nodes. The default value is "true". 



####Example: true

The following HTML:



```html

<a href="test.html">xxx</a>

```



becomes:



```javascript

[ { raw: 'a href="test.html"'

  , data: 'a href="test.html"'

  , type: 'tag'

  , name: 'a'

  , attribs: { href: 'test.html' }

  , children: [ { raw: 'xxx', data: 'xxx', type: 'text' } ]

  }

]

```



####Example: false

The following HTML:



```javascript

<a href="test.html">xxx</a>

```



becomes:



```javascript

[ { type: 'tag'

  , name: 'a'

  , attribs: { href: 'test.html' }

  , children: [ { data: 'xxx', type: 'text' } ]

  }

]

```



###Option: enforceEmptyTags

Indicates whether the DOM should prevent children on tags marked as empty in the HTML spec. Typically this should be set to "true" HTML parsing and "false" for XML parsing. The default value is "true".



####Example: true

The following HTML:



```html

<link>text</link>

```



becomes:



```javascript

[ { raw: 'link', data: 'link', type: 'tag', name: 'link' }

, { raw: 'text', data: 'text', type: 'text' }

]

```



####Example: false

The following HTML:



```html

<link>text</link>

```



becomes:



```javascript

[ { raw: 'link'

  , data: 'link'

  , type: 'tag'

  , name: 'link'

  , children: [ { raw: 'text', data: 'text', type: 'text' } ]

  }

]

```



##DomUtils



###TBD (see utils_example.js for now)



##Related Projects



Looking for CSS selectors to search the DOM? Try Node-SoupSelect, a port of SoupSelect to NodeJS: http://github.com/harryf/node-soupselect



There's also a port of hpricot to NodeJS that uses HtmlParser for HTML parsing: http://github.com/silentrob/Apricot