https://github.com/cfhull/igm-scrape

https://github.com/cfhull/igm-scrape

igmforum web-scraper

Last synced: 17 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/cfhull/igm-scrape
• Owner: cfhull
• License: mit
• Created: 2019-07-13T14:19:02.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2023-01-04T05:16:43.000Z (about 2 years ago)
• Last Synced: 2023-03-02T02:36:43.991Z (almost 2 years ago)
• Topics: igmforum, web-scraper
• Language: JavaScript
• Size: 1020 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 10
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# igm-scrape [![GitHub](https://img.shields.io/github/license/cfhull/igm-panel-scraper.svg?style=plastic)](https://github.com/cfhull/igm-panel-scraper/blob/master/LICENSE) [![CircleCI](https://img.shields.io/circleci/build/github/cfhull/igm-panel-scraper.svg?style=plastic)](https://circleci.com/gh/cfhull/igm-panel-scraper) 

igm-scrape is a web scraper that pulls data from the IGM Economic Experts Panel at `http://www.igmchicago.org/igm-economic-experts-panel`

Note: Currently does not support special surveys

## Prerequisites

`node.js`

`npm`

Cookie from `http://www.igmchicago.org/igm-economic-experts-panel`

### How to get cookie

1. Go to `http://www.igmchicago.org/igm-economic-experts-panel`

2. Opening your browser dev tools

3. Navigate to the `network` tab

4. As shown in the follow screenshot, select the `igm-economic-experts-panel` request, and copy the cookie string beginning with `incap_ses`



## Running

NPM

```console

$ npm i igm-scrape

$ igm-scrape 

```

Manual

```console

$ git clone 

$ cd 

$ node index.js 

```

File will be saved to the current directory as `igmdata.json`