Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/okfn-brasil/perfil-politico
A platform for profiling public figures in Brazilian politics
https://github.com/okfn-brasil/perfil-politico
django python
Last synced: 9 days ago
JSON representation
A platform for profiling public figures in Brazilian politics
- Host: GitHub
- URL: https://github.com/okfn-brasil/perfil-politico
- Owner: okfn-brasil
- License: gpl-3.0
- Created: 2018-04-18T22:32:22.000Z (over 6 years ago)
- Default Branch: main
- Last Pushed: 2022-09-21T15:45:11.000Z (over 2 years ago)
- Last Synced: 2024-07-05T15:17:27.864Z (6 months ago)
- Topics: django, python
- Language: Jupyter Notebook
- Homepage: https://perfilpolitico.serenata.ai/
- Size: 3.29 MB
- Stars: 161
- Watchers: 22
- Forks: 40
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-social-projects - Perfil Político
README
[](https://travis-ci.com/okfn-brasil/perfil-politico)
[](https://codecov.io/gh/okfn-brasil/perfil-politico)
[](https://codeclimate.com/github/okfn-brasil/perfil-politico)
[](https://apoia.se/serenata)
# Perfil Político
A platform for profiling candidates in Brazilian 2022 General Election, based
entirely on open data.
## Install
This project requires [Docker](https://docs.docker.com/install/) and
[Docker Compose](https://docs.docker.com/compose/install/).
### Settings
To run the API, you must copy the `.env.sample` to a `.env` file. You can edit
it accordingly if you want run in a production env.
#### Creating the container
You need to create the docker container:
```sh
$ docker-compose up -d
```
> Note: You can use `docker compose` instead of `docker-compose` in this project.
#### Database initial setup
You should create your database by applying migrations:
```sh
$ docker-compose run --rm django ./manage.py migrate
```
### Running
To run the project locally, you can simply use this command:
```sh
$ docker-compose up
```
The website and [API](#api) will be available at
[`localhost:8000`](http://localhost:8000) and the Jupyter at
[`localhost:8888`](http://localhost:8888).
### Bringing data into your database
Your local `data/` directory is mapped, inside the container, to `/mnt/data`.
Each command uses a CSV (compressed as `.xz` or not) from a public and
available source. Use `--help` for more info. Yet some extra data can be
generated with some Django custom commands.
Once you have download the datasets to `data/`, you can **create your own database from scratch**
running:
```sh
$ docker-compose run --rm django ./manage.py load_affiliations /mnt/data/filiacao.csv
$ docker-compose run --rm django ./manage.py load_candidates /mnt/data/candidatura.csv
$ docker-compose run --rm django ./manage.py link_affiliations_and_candidates
$ docker-compose run --rm django ./manage.py link_politicians_and_election_results
$ docker-compose run --rm django ./manage.py load_assets /mnt/data/bemdeclarado.csv
$ docker-compose run --rm django ./manage.py pre_calculate_stats
$ docker-compose run --rm django ./manage.py load_bills /mnt/data/senado.csv
$ docker-compose run --rm django ./manage.py load_bills /mnt/data/camara.csv
$ docker-compose run --rm django ./manage.py load_income_statements /mnt/data/receita.csv
# make sure to read the instructions on populate_company_info.sql before running the next command
$ docker-compose run --rm postgres psql -U perfilpolitico < populate_company_info.sql
```
> :warning: Note that it will change the primary keys for all candidates in the database!
> So be careful on running it for production environment because some endpoints as
> ` /api/candidate//` depends on this primary key to retrieve the correct data.
Or you can **update the data from your database** using the commands:
```sh
$ docker-compose run --rm django ./manage.py unlink_and_delete_politician_references
$ docker-compose run --rm django ./manage.py load_affiliations /mnt/data/filiacao.csv clean-previous-data
$ docker-compose run --rm django ./manage.py update_or_create_candidates /mnt/data/candidatura.csv
$ docker-compose run --rm django ./manage.py link_affiliations_and_candidates
$ docker-compose run --rm django ./manage.py link_politicians_and_election_results
$ docker-compose run --rm django ./manage.py load_assets /mnt/data/bemdeclarado.csv clean-previous-data
$ docker-compose run --rm django ./manage.py pre_calculate_stats
$ docker-compose run --rm django ./manage.py load_bills /mnt/data/senado.csv clean-previous-data
$ docker-compose run --rm django ./manage.py load_bills /mnt/data/camara.csv
```
> Note: The code only updates data coming from the csv's to the database.
It does not consider the possibility of changing data that is already in the
database but does not appear in the csv for some reason (in this case the data
in the database is kept untouched). Commands passing the `clean-previous-data-option` will
replace all the data for the respective csv, thus changing all primary keys.
### API
#### `GET /api/candidate////`
List all candidates from a certain state to a given post. For example:
`/api/candidate/2018/df/deputado-distrital/`
Post options for 2018 are:
* `1o-suplente`
* `2o-suplente`
* `deputado-distrital`
* `deputado-estadual`
* `deputado-federal`
* `governador`
* `presidente`
* `senador`
* `vice-governador`
* `vice-presidente`
State options are the abbreviation of the 27 Brazilian states, plus `br` for
national election posts.
#### `GET /api/candidate//`
Returns the details of a given candidate.
#### `GET /api/economic-bonds/candidate//`
Get electoral income history for a given candidate and companies that have a partnership.
Returns an object with the structure:
```
{
"companies_associated_with_politician": [
{
"cnpj": string,
"company_name": string,
"main_cnae": string,
"secondary_cnaes": string (cnaes separated by ','),
"uf": string,
"foundation_date": string (date format 'YYYY/MM/DD'),
"participation_start_date": string (date format 'YYYY/MM/DD')
}
// ... other companies in the same format as above ...
],
"election_income_history": [
{
"year": int,
"value": float,
"donor_name": string,
"donor_taxpayer_id": string
"donor_company_name": string
"donor_company_cnpj": string
"donor_economic_sector_code": string,
"donor_secondary_sector_codes": string
},
// ... other income statements in the same format as above ...
]
}
```
#### `GET /api/stats////`
Get national statistics for a given characteristic in a elected post.
Post options are:
* `deputado-distrital`
* `deputado-estadual`
* `deputado-federal`
* `governador`
* `prefeito`
* `presidente`
* `senador`
* `vereador`
Characteristic options are:
* `age`
* `education`
* `ethnicity`
* `gender`
* `marital_status`
* `occupation`
* `party`
#### `GET /api/stats/////`
Same as above but aggregated by state.
#### `GET /api/asset-stats/`
Returns an object with a key called `mediana_patrimonios` that is a list with
the median of elected people's asset value aggregated by year.
`optionally` you can add query parameters to filter the results by `state` or by
the `candidate post` (the valid posts are the same ones that are in the list above).
These parameters can support multiple values if you wish to filter by more than one thing.
Ex: `/api/asset-stats?state=MG&state=RJ&candidate_post=governador&candidate_post=prefeito`
## Tests
```sh
$ docker-compose run --rm django py.test
$ docker-compose run --rm django black . --check
```