https://github.com/dheavy/scraper-cli
Deprecated CLI application for MyPleasu.re, powered by Node.js, that crawls and scrapes content from websites ("providers"). Run on a secure (HTTPS) environment.
https://github.com/dheavy/scraper-cli
Last synced: 10 months ago
JSON representation
Deprecated CLI application for MyPleasu.re, powered by Node.js, that crawls and scrapes content from websites ("providers"). Run on a secure (HTTPS) environment.
- Host: GitHub
- URL: https://github.com/dheavy/scraper-cli
- Owner: dheavy
- Created: 2015-05-12T06:49:25.000Z (about 11 years ago)
- Default Branch: master
- Last Pushed: 2015-05-12T06:51:01.000Z (about 11 years ago)
- Last Synced: 2025-04-02T21:15:38.140Z (over 1 year ago)
- Language: JavaScript
- Size: 117 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# [ DEPRECATED - FOR ILLUSTRATION/ARCHIVING PURPOSES ONLY ]
MyPleasu.re Scraper CLI
================================
A command line interface application, powered by Node.js, that crawls and scrapes content from websites ("providers"). Run on a secure (HTTPS) environment.
## Installation
#### Part 1. Node.js / Express
- Install [Node.js](http://nodejs.org/).
- Run `npm install` on the root directory of the project.
#### Part 2. SSL
The app runs on a secure HTTP environment. You'll need to do the following to set it up.
- Change into the **SSL** directory : `cd ssl`.
- Create the server key: `sudo openssl genrsa -des3 -out server.key 1024`. Don't forget the passphrase you're using.
- Create the certificate signing request: `sudo openssl req -new -key server.key -out server.csr`.
- Now remove the passphrase with the following commands: `sudo cp server.key server.key.org` and `sudo openssl rsa -in server.key.org -out server.key`.
- Sign your certificate with `sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt`.
- If your certificate/key are not in this dedicated folder, you'll need to update their path in the config file(s).
You can now run tests with `npm test` or start the app with `npm start` and checkout [https://localhost:4430](https://localhost:4430).