{"id":20652318,"url":"https://github.com/openpolis/eu-state-aids","last_synced_at":"2026-04-24T07:32:41.916Z","repository":{"id":57427205,"uuid":"376159997","full_name":"openpolis/eu-state-aids","owner":"openpolis","description":"CLI to extract state aids data from public sources and produce CSV files","archived":false,"fork":false,"pushed_at":"2021-07-01T17:21:03.000Z","size":323,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-08-25T02:35:26.222Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openpolis.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-11T23:03:00.000Z","updated_at":"2021-07-01T19:12:20.000Z","dependencies_parsed_at":"2022-09-05T00:51:11.861Z","dependency_job_id":null,"html_url":"https://github.com/openpolis/eu-state-aids","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/openpolis/eu-state-aids","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openpolis%2Feu-state-aids","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openpolis%2Feu-state-aids/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openpolis%2Feu-state-aids/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openpolis%2Feu-state-aids/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openpolis","download_url":"https://codeload.github.com/openpolis/eu-state-aids/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openpolis%2Feu-state-aids/sbom","scorecard":{"id":709936,"data":{"date":"2025-08-11","repo":{"name":"github.com/openpolis/eu-state-aids","commit":"64aa1386e76d0b5cf650c90fbe1b36fe9664b912"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 0/23 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: MIT License: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"37 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2024-230 / GHSA-248v-346w-9cwc","Warn: Project is vulnerable to: PYSEC-2022-42986 / GHSA-43fp-rhv2-5gv8","Warn: Project is vulnerable to: PYSEC-2023-135 / GHSA-xqr8-7jwr-rhp7","Warn: Project is vulnerable to: PYSEC-2024-60 / GHSA-jjg7-2v4v-x38h","Warn: Project is vulnerable to: GHSA-cpwx-vrp4-4pq7","Warn: Project is vulnerable to: GHSA-gmj6-6f8f-6699","Warn: Project is vulnerable to: GHSA-h5c8-rqwp-cp95","Warn: Project is vulnerable to: GHSA-h75v-3vvj-5mfj","Warn: Project is vulnerable to: GHSA-q2x7-8rv6-6q7h","Warn: Project is vulnerable to: GHSA-6p56-wp2h-9hxr","Warn: Project is vulnerable to: GHSA-fpfv-jqm9-f5jm","Warn: Project is vulnerable to: GHSA-3f63-hfp8-52jq","Warn: Project is vulnerable to: GHSA-44wm-f244-xhp3","Warn: Project is vulnerable to: GHSA-4fx9-vc88-q2xc","Warn: Project is vulnerable to: PYSEC-2021-331 / GHSA-7534-mm45-c74v","Warn: Project is vulnerable to: PYSEC-2023-227 / GHSA-8ghj-p4vj-mr35","Warn: Project is vulnerable to: PYSEC-2022-10 / GHSA-8vj2-vxx3-667w","Warn: Project is vulnerable to: PYSEC-2021-317 / GHSA-98vv-pw6r-q6q4","Warn: Project is vulnerable to: PYSEC-2022-168 / GHSA-9j59-75qj-795w","Warn: Project is vulnerable to: GHSA-j7hp-h8jx-5ppr","Warn: Project is vulnerable to: PYSEC-2022-42979 / GHSA-m2vv-5vj5-2hm7","Warn: Project is vulnerable to: PYSEC-2022-8 / GHSA-pw3c-h7wp-cvhx","Warn: Project is vulnerable to: PYSEC-2022-9 / GHSA-xrcv-f9gm-v42c","Warn: Project is vulnerable to: PYSEC-2023-175","Warn: Project is vulnerable to: PYSEC-2022-42969","Warn: Project is vulnerable to: PYSEC-2023-238 / GHSA-5wvp-7f3h-6wmm","Warn: Project is vulnerable to: PYSEC-2024-161","Warn: Project is vulnerable to: PYSEC-2023-117 / GHSA-mrwq-x4v8-fh7p","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2023-74 / GHSA-j8r2-6x86-q33q","Warn: Project is vulnerable to: GHSA-34jh-p97f-mpxf","Warn: Project is vulnerable to: PYSEC-2023-212 / GHSA-g4mx-q9vg-27p4","Warn: Project is vulnerable to: GHSA-pq67-6m6q-mj2v","Warn: Project is vulnerable to: PYSEC-2023-192 / GHSA-v845-jxx5-vc9f","Warn: Project is vulnerable to: PYSEC-2024-187 / GHSA-rqc4-2hc7-8c8v","Warn: Project is vulnerable to: GHSA-jfmj-5v4g-7637"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-22T07:49:56.634Z","repository_id":57427205,"created_at":"2025-08-22T07:49:56.634Z","updated_at":"2025-08-22T07:49:56.634Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32214415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T03:15:14.334Z","status":"ssl_error","status_checked_at":"2026-04-24T03:15:11.608Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T17:33:52.925Z","updated_at":"2026-04-24T07:32:41.850Z","avatar_url":"https://github.com/openpolis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Description\n\n`eu-state-aids` is a package to import **state aids related data** from single countries sources\nand produce CSV files, according to a common data structure.\n\n[![TravisCI Badge](https://travis-ci.com/openpolis/eu-state-aids.svg?branch=master \"TravisCI building status\")](https://travis-ci.com/github/openpolis/eu-state-aids)\n[![PyPI version](https://badge.fury.io/py/eu-state-aids.svg)](https://badge.fury.io/py/eu-state-aids)\n![Tests Badge](https://op-badges.s3.eu-west-1.amazonaws.com/eu-state-aids/tests-badge.svg?2)\n![Coverage Badge](https://op-badges.s3.eu-west-1.amazonaws.com/eu-state-aids/coverage-badge.svg?2)\n![Flake8](https://op-badges.s3.eu-west-1.amazonaws.com/eu-state-aids/flake8-badge.svg?2)\n\nThe tool provides both a Command Line Interface (the `eu-state-aids` command), \nand an API. See the [Usage](#Usage) section.\n\nThe common CSV format used for the export:\n\n|Name|Type|Meaning|\n|----|----|-------|\n|Name of the beneficiary| String | The name of the aid's beneficiary|\n|ID of the beneficiary| Long Integer | The unique ID of the aid's beneficiary|\n|European operation program (ID)| String | The unique CCI code of the european program, see details [here](https://ec.europa.eu/sfc/sites/sfc2014/files/QG+pdf/CCI_0.pdf) |\n|Amounts (€)| Float with 2 digits precision | Total amount of the project (in Euro) |\n|Date| Date `YYYY[-MM-DD]` | Date of the beginning of the aid program (at least the year) |\n|State aid Scheme| String | The aid scheme code. The format is `SA.XXXXX`, wher the Xs are digits. |\n\n\n## Installation\n\nPython versions from 3.7 are supported.\n \nThe package depends on these python packages:\n* typer\n* openpyxl\n* pandas\n* requests\n* validators\n\nSo, it's better to create a *virtualenv* before installation.\n\nThe package is hosted on pypi, and can be installed, for example using pip:\n\n    pip install eu-state-aids \n\n\n## Usage\n\n### Command Line Interface\nThe `eu-state-aids` binary command will be available after installation. \nIt offers help with:\n\n    eu-state-aids --help\n\nThe `eu-state-aids` command can be used to extract the data from the official sources, \nand populate the CSV files.\n\nFor each country, data files will firstly be *fetched* and stored locally, \nand thereafter *used* in order to **export** CSV files.\n\nThis two-step procedure is useful, since it is not always possible to download source files (Excel, XML, ...) from \nBI systems of nation states, as it has been seen that they tend to time-out whenever the number of records is \nhigh enough.\n\nThe logic of these two phases can vary for each single european state, so each country will have a dedicated module,\nthat will be executable as a sub-command.\n\n\n### Bulgary\nTo retrieve data and produce a CSV file for Bulgary (bg), 2015:\n \n      eu-state-aids bg fetch 2015\n      eu-state-aids bg export 2015\n\nTo launch the scripts *for all years* for Bulgary (bg):\n\n    # download all years' excel files into local storage \n    for Y in $(seq 2014 2022)\n    do \n      eu-state-aids bg fetch $Y\n    done\n    \n    # process all years' excel files and export CSV records into local storage \n    #./data/bg/$Y.csv files\n    for Y in $(seq 2014 2022)\n    do\n      python  -m eu_state_aids bg export $Y\n    done\n\n### Italy\nItaly needs a slightly different procedure, as before invoking the fetch/export commands,\na `misure.csv` file needs to be generated, so that all aids records found in XML files can be\ncompared with found CE_CODE and filtered.\n\n      eu-state-aids bg generate_measures\n\nTo retrieve data and produce a CSV file for Italy (it), 2015, there is actually no need to fetch the file,\nas files have been copied on a reliable source.\n \n      eu-state-aids bg export 2015 --delete-processed\n\nThis will generate a loop over all months of 2015, fetch the files, if they're not already fetched, \nextract, transform and filter the records for each month and emit a CSV file with all the records found.\nThe amount of money is summed for each beneficiary (over all records in that year). The fetched file will be deleted\nafter the procedure, if required through the `--delete-processed` option.\n\nTo launch the scripts *for all years* for Italy (it):\n\n    # download all years' excel files into local storage \n    for Y in $(seq 2014 2022)\n    do \n      eu-state-aids it export $Y --delete-processed\n    done\n\n\n### API\nThe fetch and export logics can be used from within a python program, \nimporting the packages. All options values must be explicited in API calls.\n\n    from eu_state_aids import bg\n\n    for year in ['2015', '2016', '2017']:\n      bg.fetch(year, local_path='./data/bg')\n      bg.export(\n        year, local_path='./data/bg', \n        stateaid_url=\"https://stateaid.minfin.bg/document/860\", \n        program_start_year=\"2014\"\n      )\n  \n\n### Note on italian data\n\nItalian government sources suffer from two issues.\n1. XML files are not automatically downloadable from single dedicated URLS, but must be downloaded manually,\nas the softare solution adopted for the open data section of the web site does not allow such individual downloads.\nThey have been mirrored on a [public AWS resource](http://eu-state-aids.s3-website-eu-west-1.amazonaws.com/it/rna_mirror/), \nand will be fetched from there.\n2. XML files have not been compressed and the `OpenData_Aiuto_*.xml` files are huge (~1GB). Once compressed, \ntheir size reduce to 1/25th of the original size. So they will be stored on the AWS mirror in zipped format.\n \n## Support\n\nThere is no guaranteed support available, but authors will try to keep up with issues \nand merge proposed solutions into the code base.\n\n## Project Status\nThis project is funded by the European Commission and is currently (2021) under active developement.\n\n## Contributing\nIn order to contribute to this project:\n* verify that python 3.7+ is being used (or use [pyenv](https://github.com/pyenv/pyenv))\n* verify or install [poetry](https://python-poetry.org/), to handle packages and dependencies in a leaner way, \n  with respect to pip and requirements\n* clone the project `git clone git@github.com:openpolis/eu-state-aids.git` \n* install the dependencies in the virtualenv, with `poetry install`,\n  this will also install the dev dependencies\n* develop wildly, running tests and coverage with `coverage run -m pytest`\n* create a [pull request](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests)\n* wait for the maintainers to review and eventually merge your pull request into the main repository\n\n### Testing\nTests are under the tests folder. [requests-mock](https://requests-mock.readthedocs.io/en/latest/index.html)\nis used to mock requests to remote data files, in order to avoid slow remote connections during tests.\n\n## Authors\nGuglielmo Celata - guglielmo@openpolis.it\n\n## Licensing\nThis package is released under an MIT License, see details in the LICENSE.txt file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenpolis%2Feu-state-aids","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenpolis%2Feu-state-aids","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenpolis%2Feu-state-aids/lists"}