https://github.com/abrie/atl-council-scraper

Scrape Atlanta City Council member information
https://github.com/abrie/atl-council-scraper

atlanta city-council webscraper

Last synced: 28 days ago
JSON representation

Scrape Atlanta City Council member information

• Host: GitHub
• URL: https://github.com/abrie/atl-council-scraper
• Owner: abrie
• Created: 2020-09-21T20:26:06.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2024-02-06T11:14:55.000Z (11 months ago)
• Last Synced: 2024-02-06T13:20:29.101Z (11 months ago)
• Topics: atlanta, city-council, webscraper
• Language: Python
• Homepage:
• Size: 477 KB
• Stars: 1
• Watchers: 2
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ATL City Council Scraper

https://citycouncil.atlantaga.gov/ is the authoritative source for Atlanta City Council information.

This repository scrapes council member info into JSON for consumption by apps and APIs.

Scraping is done at 11am daily via Github Actions using the script in [.github/workflows/run-scraper.yml](.github/workflows/run-scraper.yml). Neat technique.[1](https://simonwillison.net/2020/Oct/9/git-scraping/)

Get the most recent [scraped data here](scraped/atl-citycouncil.json). View [update history](https://github.com/abrie/atl-council-scraper/commits/master/scraped/atl-citycouncil.json).

For self-hosting you have two options:

## Run in a container

`docker run abriedev/atl-council-scraper scrape`

Save output to a .json file in your root folder

`docker run abriedev/atl-council-scraper scrape > name-of-output-file.json`

If you get "Cannot connect to the Docker daemon", 

install the Docker Toolbox, which includes the Docker Machine.

https://github.com/docker/machine/releases

## Build from source

__Requires Python 3.8__

Create a virtual environment (OSX / Linux / Windows):

`python3 -m venv venv`

OSX / Linux:

`source venv/bin/activate`

Windows:

`\venv\Scripts\activate.bat`

Running `make dependencies` will install BeautifulSoup4

1. `make dependencies`

2. `make test`

3. `python3 -m app scrape > scraped/atl-citycouncil.json`