Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dharmafly/noodle
A node server and module which allows for cross-domain page scraping on web documents with JSONP or POST.
https://github.com/dharmafly/noodle
Last synced: about 1 month ago
JSON representation
A node server and module which allows for cross-domain page scraping on web documents with JSONP or POST.
- Host: GitHub
- URL: https://github.com/dharmafly/noodle
- Owner: dharmafly
- Created: 2012-06-29T10:00:46.000Z (over 12 years ago)
- Default Branch: master
- Last Pushed: 2024-05-14T14:04:11.000Z (7 months ago)
- Last Synced: 2024-10-30T07:59:26.244Z (about 1 month ago)
- Language: JavaScript
- Homepage: https://noodle.dharmafly.com/
- Size: 6.04 MB
- Stars: 747
- Watchers: 26
- Forks: 69
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-starred - dharmafly/noodle - A node server and module which allows for cross-domain page scraping on web documents with JSONP or POST. (others)
README
[noodle](https://noodle.dharmafly.com)
=============================
noodle is a Node.js server and module for querying and scraping data from web documents. It features:
```JSON
{
"url": "https://github.com/explore",
"selector": "ol.ranked-repositories h3 a",
"extract": "href"
}
```
Features
--------
- Cross domain document querying (html, json, xml, atom, rss feeds)
- Server supports querying via JSONP and JSON POST
- Multiple queries per request
- Access to queried server headers
- Allows for POSTing to web documents
- In memory caching for query results and web documents
Server quick start
------------------
Setup
$ npm install noodlejs
or
$ git clone [email protected]:dharmafly/noodle.git
$ cd noodle
$ npm install
Start the server by running the binary
$ bin/noodle-server
Noodle node server started
├ process title node-noodle
├ process pid 4739
└ server port 8888
You may specify a port number as an argument
$ bin/noodle-server 9090
Noodle node server started
├ process title node-noodle
├ process pid 4739
└ server port 9090
Noodle as a node module
-----------------------
If you are interested in the node module just run ```npm install noodlejs```,
require it and check out the [noodle api](https://noodle.dharmafly.com/reference/#Noodle-as-node-module)
```javascript
var noodle = require('noodlejs');
noodle.query({
url: 'https://github.com/explore',
selector: 'ol.ranked-repositories h3 a',
extract: 'href'
})
.then(function (results) {
console.log(results);
});
```
Tests
-----
The noodle tests create a temporary server on port `8889` which the automated
tests tell noodle to query against.
To run tests you can use the provided binary *from the noodle package
root directory*:
$ cd noodle
$ bin/tests
Contribute
----------
Contributors and suggestions welcomed.
- [https://noodle.dharmafly.com](https://noodle.dharmafly.com)
- [https://github.com/dharmafly/noodle](https://github.com/dharmafly/noodle)