https://github.com/edsu/libweb

extract library homepage urls from LIBWEB
https://github.com/edsu/libweb

Last synced: about 1 year ago
JSON representation

extract library homepage urls from LIBWEB

• Host: GitHub
• URL: https://github.com/edsu/libweb
• Owner: edsu
• Created: 2011-08-20T22:05:02.000Z (almost 15 years ago)
• Default Branch: master
• Last Pushed: 2011-08-20T22:12:55.000Z (almost 15 years ago)
• Last Synced: 2025-05-12T13:13:20.518Z (about 1 year ago)
• Language: Python
• Homepage:
• Size: 304 KB
• Stars: 6
• Watchers: 2
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
Have you ever wanted a machine readable list of URLs for library websites 

around the world? It's pretty unlikely, but if you have, read on.

This screen scraper will crawl [LIBWEB](http://lists.webjunction.org/libweb/)

and extract information about library websites around the world and 

persist it as JSON. The libraries.json file that is committed here has name,

location and the URL for 7990 libraries, kinda like:

    [

      {

        "url": "http://libraries.adelphi.edu/", 

        "name": "Adelphi University", 

        "location": "Garden City NY"

      }, 

      {

        "url": "http://www.sunyacc.edu/Library/", 

        "name": "Adirondack Community College", 

        "location": "Queensbury NY"

      }, 

      {

        "url": "http://www.acp.edu/academics_library.html", 

        "name": "Albany College of Pharmacy", 

        "location": "Albany NY"

      }, 

      {

        "url": "http://www.albertus.edu/academic/library.shtml", 

        "name": "Albertus Magnus College", 

        "location": "New Haven CT"

      }, 

      {

        "url": "http://www.albright.edu/library/", 

        "name": "Albright College", 

        "location": "Reading PA"

      }, 

      {

        "url": "http://www.herr.alfred.edu/", 

        "name": "Alfred University", 

        "location": "Alfred NY"

      }, 

      ...

    ]

To re-crawl LIBWEB you will need to install [Python](http://python.org), 

[lxml](http://pypi.python.org/pypi/lxml) and then run libraries.py

License: [CCO](http://creativecommons.org/about/cc0)