Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/astrochun/voxcharta-my-voting-record
A Python tool to extract information from VoxCharta My Voting Records HTML
https://github.com/astrochun/voxcharta-my-voting-record
beautifulsoup4 csv json pandas python voxcharta webscraping
Last synced: about 11 hours ago
JSON representation
A Python tool to extract information from VoxCharta My Voting Records HTML
- Host: GitHub
- URL: https://github.com/astrochun/voxcharta-my-voting-record
- Owner: astrochun
- License: gpl-3.0
- Created: 2020-12-21T20:55:48.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2022-05-07T23:35:32.000Z (over 2 years ago)
- Last Synced: 2025-01-01T10:39:49.254Z (28 days ago)
- Topics: beautifulsoup4, csv, json, pandas, python, voxcharta, webscraping
- Language: Python
- Homepage:
- Size: 47.9 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# voxcharta-my-voting-record
[](https://github.com/astrochun/voxcharta-my-voting-record/actions/workflows/build.yml)
[](https://github.com/astrochun/voxcharta-my-voting-record/actions/workflows/linting.yml)
[](https://pypi.org/project/voxcharta-my-voting-record/)
[](https://github.com/astrochun/voxcharta-my-voting-record/blob/ci/28_build_test/LICENSE)
[](https://github.com/psf/black)
- [Overview](#overview)
- [Getting Started](#getting-started)
- [Requirements](#requirements)
- [Installation Instructions](#installation-instructions)
- [Vox Charta Data Export](#vox-charta-data-export)
- [Execution](#execution)
- [Versioning](#versioning)
- [Authors](#authors)
- [License](#license)
--------------
## Overview
This pure Python tool web scrapes data from [Vox Charta](https://voxcharta.org)'s
[My Voting Records page](https://voxcharta.org/tools/my-voting-record/).
It provides the data contents in two forms, a non-relational JSON file
and relational CSV file.
This software was driven by the need to archive such records since Vox Charta
will be sunsetted on December 31, 2020.
## Getting Started
These instructions will have the code running.
### Requirements
This software is tested with 3.10.2, the latest version available during recent development.
In principle it should work with >=3.7. It requires `BeautifulSoup4` and `pandas`.
### Installation Instructions
#### Python and setting up an environment
Using your preferred Python environment manager (`conda`, `virtualenv`, `venv`),
I recommend create a separate environment to avoid any possible conflicts with
existing software that you used. Below are `conda` instructions
```:
$ (sudo) conda create -n voxcharta python=3.10.2
```
Installation is straightforward:
```
$ conda activate voxcharta
$ (sudo) pip install voxcharta-my-voting-record
```
Or from source:
```
$ conda activate voxcharta
$ git clone https://github.com/astrochun/voxcharta-my-voting-record.git
$ cd voxcharta-my-voting-record
$ (sudo) pip install .
```
For editable/development installation, execute `(sudo) pip install -e .`
This is preferred over the `python setup.py` method as certain dependencies (`numpy`) will fail to install during the build process.
## Vox Charta Data Export
Before using this software, you will want to export your Vox Charta My Voting Records
page. Here are the steps. These are based on Chrome, so differences may occur for other browsers:
1. [Login to Vox Charta](https://voxcharta.org/wp-login.php) before you can't!
2. Click on "Tools > My Voting Records"
3. After the page fully loads (this can take some time), click "File > Save Page As"
4. Change filename to "myvotingrecords"
5. For format, select "Webpage, Complete"
## Execution
The primary script to execute is [`vox_run`](bin/vox_run)
Execution requires only one argument, which is the full path
to the HTML file.
```
$ vox_run /full/path/to/myvotingrecords.htm
```
This is all possible using [`Typer`](https://typer.tiangolo.com/).
The script provides rich-formatted log messages to the terminal using [`Rich`](https://rich.readthedocs.io/).
All contents are stored in `/full/path/to` unless `outfile_prefix` is specified
with a different path. Thus, without specifying `outfile_prefix` the above execution
will generate:
- JSON: `/full/path/to/myvotingrecords.json`
- CSV: `/full/path/to/myvotingrecords.csv`
A log file is constructed: `/full/path/to/vox_run.YYYY-MM-DD.log`
## Versioning
We use [SemVer](http://semver.org/) for versioning. For the versions available,
see the [releases on this repository](https://github.com/astrochun/voxcharta-my-voting-record/releases).
## Authors
* Chun Ly, Ph.D. ([@astrochun](http://www.github.com/astrochun))
## License
This project is licensed under the [GNU GPLv3 License](https://www.gnu.org/licenses/gpl-3.0.en.html) - see the [LICENSE](LICENSE) file for details.