https://github.com/redco/goose-phantom-environment

Environment for Goose parser which allows to run it in PhantomJS
https://github.com/redco/goose-phantom-environment

crawler environment goose goose-parser nodejs parse parser phantomjs scraper

Last synced: 1 day ago
JSON representation

Environment for Goose parser which allows to run it in PhantomJS

• Host: GitHub
• URL: https://github.com/redco/goose-phantom-environment
• Owner: redco
• License: mit
• Created: 2017-09-24T04:58:04.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2019-03-16T21:04:16.000Z (over 5 years ago)
• Last Synced: 2024-10-13T06:29:46.552Z (25 days ago)
• Topics: crawler, environment, goose, goose-parser, nodejs, parse, parser, phantomjs, scraper
• Language: JavaScript
• Size: 83 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Goose Phantom Environment

[![Build Status](https://img.shields.io/circleci/project/github/redco/goose-phantom-environment.svg?style=flat)](https://circleci.com/gh/redco/goose-phantom-environment)

[![Latest Stable Version](https://img.shields.io/npm/v/goose-phantom-environment.svg?style=flat)](https://www.npmjs.com/package/goose-phantom-environment)

[![Total Downloads](https://img.shields.io/npm/dt/goose-phantom-environment.svg?style=flat)](https://www.npmjs.com/package/goose-phantom-environment)

Environment for [Goose Parser](https://github.com/redco/goose-parser) which allows to run it in PhantomJS

## PhantomEnvironment

That environment is used for running Parser on node.

```JS

var env = new PhantomEnvironment({

    url: 'http://google.com',

});

```

The main and only required parameter is `url`. It contains an url address of the site, where Parser will start.

This environment allows to perform snapshots, use proxy lists, custom proxy rotator, white and black lists for loading resources and more sweet features. Find more info about options in [here](https://github.com/redco/goose-parser/blob/master/lib/PhantomEnvironment.js#L35).

## Tests

To run [tests](https://github.com/redco/goose-parser/blob/master/tests/phantom_parser_test.js) use command:

```bash

npm test

```