https://github.com/ntoll/hscic_apiomatic

Scrapes HSCIC website and builds a list of JSON objects for each dataset and key indicator found therein.
https://github.com/ntoll/hscic_apiomatic

Last synced: 8 days ago
JSON representation

Scrapes HSCIC website and builds a list of JSON objects for each dataset and key indicator found therein.

• Host: GitHub
• URL: https://github.com/ntoll/hscic_apiomatic
• Owner: ntoll
• License: mit
• Created: 2014-10-06T11:48:42.000Z (about 10 years ago)
• Default Branch: master
• Last Pushed: 2014-11-10T11:51:45.000Z (about 10 years ago)
• Last Synced: 2024-05-01T22:58:02.263Z (7 months ago)
• Language: Python
• Homepage:
• Size: 1.97 MB
• Stars: 1
• Watchers: 3
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
hscic_apiomatic

===============

Scrapes HSCIC website and builds a list of JSON objects for each dataset or

key indicator found therein.

Requires beautifulsoup, requests and html2text::

    $ pip install -r requirements.txt

Usage

-----

To scrape the indicator portal::

    $ python grab_indicators.py

To scrape the current dataset catalogue::

    $ python grab_datasets.py

The results are two JSON files: indicators.json and datasets.json.

(The existing indicators.json and datasets.json files in the repos were

generated at the beginning of October 2014. These can probably be ignored but

are included for illustrative purposes.)

If anything goes wrong check out the indicator.log / datasets.log files. :-)

Feedback most welcome!

@ntoll