{"id":18610148,"url":"https://github.com/creativecommons/quantifying","last_synced_at":"2025-08-20T18:32:01.855Z","repository":{"id":60258766,"uuid":"533860990","full_name":"creativecommons/quantifying","owner":"creativecommons","description":"quantify the size and diversity of the commons--the collection of works that are openly licensed or in the public domain","archived":false,"fork":false,"pushed_at":"2024-12-13T20:42:13.000Z","size":83890,"stargazers_count":24,"open_issues_count":9,"forks_count":33,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-12-13T21:26:55.071Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/creativecommons.png","metadata":{"funding":{"github":["creativecommons"],"custom":["https://creativecommons.org/donate"]},"files":{"readme":"README.md","changelog":"history.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-07T17:02:39.000Z","updated_at":"2024-12-11T17:55:26.000Z","dependencies_parsed_at":"2023-02-18T10:31:00.377Z","dependency_job_id":"c5f04edb-71c7-4772-9166-d5bbe224f834","html_url":"https://github.com/creativecommons/quantifying","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creativecommons%2Fquantifying","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creativecommons%2Fquantifying/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creativecommons%2Fquantifying/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creativecommons%2Fquantifying/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/creativecommons","download_url":"https://codeload.github.com/creativecommons/quantifying/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230445926,"owners_count":18227060,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T03:08:43.940Z","updated_at":"2024-12-19T14:06:33.377Z","avatar_url":"https://github.com/creativecommons.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/creativecommons","https://creativecommons.org/donate"],"categories":[],"sub_categories":[],"readme":"# quantifying\n\nQuantifying the Commons\n\n\n## Overview\n\nThis project seeks to quantify the size and diversity of the commons--the\ncollection of works that are openly licensed or in the public domain.\n\n\n## Code of conduct\n\n[`CODE_OF_CONDUCT.md`][org-coc]:\n\u003e The Creative Commons team is committed to fostering a welcoming community.\n\u003e This project and all other Creative Commons open source projects are governed\n\u003e by our [Code of Conduct][code_of_conduct]. Please report unacceptable\n\u003e behavior to [conduct@creativecommons.org](mailto:conduct@creativecommons.org)\n\u003e per our [reporting guidelines][reporting_guide].\n\n[org-coc]: https://github.com/creativecommons/.github/blob/main/CODE_OF_CONDUCT.md\n[code_of_conduct]: https://opensource.creativecommons.org/community/code-of-conduct/\n[reporting_guide]: https://opensource.creativecommons.org/community/code-of-conduct/enforcement/\n\n\n## Contributing\n\nSee [`CONTRIBUTING.md`][org-contrib].\n\n[org-contrib]: https://github.com/creativecommons/.github/blob/main/CONTRIBUTING.md\n\n\n### Project structure\n\nPlease note that in the directory tree below, all instances of `fetch`,\n`process`, and `report` are referring to the three phases of data gathering,\nprocessing, and report generation.\n\n```\nQuantifying/\n├── .github/\n│   ├── workflows/\n│   │   ├── fetch.yml\n│   │   ├── process.yml\n│   │   ├── report.yml\n│   │   └── static_analysis.yml\n├── data/  # Data generated by script runs\n│   ├── 20XXQX/\n│   │   ├── 1-fetch/\n│   │   ├── 2-process/\n│   │   ├── 3-report/\n│   │   │   └── README.md  # All generated reports are displayed in the README\n│   └── ...\n├── dev/\n├── pre-automation/  # All Quantifying work prior to adding automation system\n├── scripts/  # Run scripts for all phases\n│   ├── 1-fetch/\n│   ├── 2-process/\n│   ├── 3-report/\n│   └── shared.py\n├── .cc-metadata.yml\n├── .flake8  # Python tool configuration\n├── .gitignore\n├── .pre-commit-config.yaml  # Static analysis configuration\n├── LICENSE\n├── Pipfile  # Specifies the project's dependencies and Python version\n├── Pipfile.lock\n├── README.md\n├── env.example\n├── history.md\n├── pyproject.toml  # Python tools configuration\n└── sources.md\n```\n\n\n## Development\n\n\n### Prerequisites\n\nFor information on learning and installing the prerequisite technologies for\nthis project, please see [Foundational technologies — Creative Commons Open\nSource][found-tech].\n\nThis repository uses [pipenv][pipenvdocs] to manage the required Python\nmodules:\n1. Install `pipenv`:\n   - Linux: [Installing Pipenv][pipenvinstall]\n   - macOS:\n     1. Install [Homebrew][homebrew]\n     2. Install pipenv:\n        ```shell\n        brew install pipenv\n        ```\n   - Windows: [Installing Pipenv][pipenvinstall]\n2. Create the Python virtual environment and install prerequisites using\n   `pipenv`:\n    ```shell\n    pipenv sync --dev\n    ```\n\n[found-tech]: https://opensource.creativecommons.org/contributing-code/foundational-tech/\n[pipenvdocs]: https://pipenv.pypa.io/en/latest/\n[pipenvinstall]: https://pipenv.pypa.io/en/latest/installation/\n[homebrew]: https://brew.sh/\n\n\n### Running scripts that require client credentials\n\nTo successfully run scripts that require client credentials, you will need to\nfollow these steps:\n1. Copy the contents of the `env.example` file in the script's directory to\n   `.env`:\n    ```shell\n    cp env.example .env\n    ```\n2. Uncomment the variables in the `.env` file and assign values as needed. See\n   [`sources.md`](sources.md) on how to get credentials:\n    ```\n    GCS_DEVELOPER_KEY = your_api_key\n    GCS_CX = your_pse_id\n    ```\n3. Save the changes to the `.env` file.\n4. You should now be able to run scripts that require client credentials\n   without any issues.\n\n\n### Static analysis\n\nStatic analysis tools ensure the codebase adheres to consistent formatting and\nstyle guidelines, enhancing readability and maintainability. Also see GitHub\nActions, below.\n\n\n#### Using [`pre-commit`][pre-commit]\n\nPre-commit allows for static analysis tools (`black`, `flake8`, `isort`, etc.)\nto be run manually or with every commit:\n\n1. (Pre-commit is installed by completing Create the Python virtual environment\n   and install prerequisites, above)\n2. Install or run manually\n   - Install the git hook scripts to enable automatic execution on every commit\n       ```shell\n       pipenv run pre-commit install\n       ```\n   - Run manually using helper dev script:\n       ```shell\n       ./dev/check.sh [FILE]\n       ```\n     If no file(s) are specified, then it runs against all files:\n       ```shell\n       ./dev/check.sh\n       ```\n3. _(Optional)_ review the configuration file:\n   [`.pre-commit-config.yaml`](.pre-commit-config.yaml)\n\n[pre-commit]: https://pre-commit.com/\n\n\n### Resources\n\n- **[Python Guidelines — Creative Commons Open Source][ccospyguide]**\n- [Black][black]: _the uncompromising Python code formatter_\n- [flake8][flake8]: _a python tool that glues together pep8, pyflakes, mccabe,\n  and third-party plugins to check the style and quality of some python code._\n- [isort][isort]: _A Python utility / library to sort imports_\n  - (It doesn't import any libraries, it only sorts and formats them.)\n- [ppypa/pipenv][pipenv]: _Python Development Workflow for Humans._\n- [pre-commit][pre-commit]: _A framework for managing and maintaining\n  multi-language pre-commit hooks._\n- [Logging][logging]: _Utilize the built-in Python logging module to implement a flexible logging system from a shared module._\n\n[ccospyguide]: https://opensource.creativecommons.org/contributing-code/python-guidelines/\n[black]: https://github.com/psf/black\n[flake8]: https://github.com/PyCQA/flake8\n[isort]: https://pycqa.github.io/isort/\n[pipenv]: https://github.com/pypa/pipenv\n[pre-commit]: https://pre-commit.com/\n[logging]: https://docs.python.org/3/library/logging.html\n\n\n### GitHub Actions\n\nThe [`.github/workflows/python_static_analysis.yml`][workflow-static-analysis]\nGitHub Actions workflow performs static analysis (`black`, `flake8`, and\n`isort`) on committed changes. The workflow is triggered automatically when you\npush changes to the main branch or open a pull request.\n\n[workflow-static-analysis]: .github/workflows/static_analysis.yml\n\n\n## Data sources\n\nKindly visit the [`sources.md`](sources.md) file for it.\n\n\n## History\n\nFor information on past efforts, see [`history.md`](history.md).\n\n\n## Copying \u0026 license\n\n\n### Code\n\n[`LICENSE`](LICENSE): the code within this repository is licensed under the\nExpat/[MIT][mit] license.\n\n[mit]: http://www.opensource.org/licenses/MIT \"The MIT License | Open Source Initiative\"\n\n\n### Data\n\n[![CC0 1.0 Universal (CC0 1.0) Public Domain Dedication\nbutton][cc-zero-png]][cc-zero]\n\nThe data within this repository is dedicated to the public domain under the\n[CC0 1.0 Universal (CC0 1.0) Public Domain Dedication][cc-zero].\n\n[cc-zero-png]: https://licensebuttons.net/l/zero/1.0/88x31.png \"CC0 1.0 Universal (CC0 1.0) Public Domain Dedication button\"\n[cc-zero]: https://creativecommons.org/publicdomain/zero/1.0/\n\n### Documentation\n\n[![CC BY 4.0 license button][cc-by-png]][cc-by]\n\nThe documentation within the project is licensed under a [Creative Commons\nAttribution 4.0 International License][cc-by].\n\n[cc-by-png]: https://licensebuttons.net/l/by/4.0/88x31.png#floatleft \"CC BY 4.0 license button\"\n[cc-by]: https://creativecommons.org/licenses/by/4.0/ \"Creative Commons Attribution 4.0 International License\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreativecommons%2Fquantifying","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcreativecommons%2Fquantifying","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreativecommons%2Fquantifying/lists"}