{"id":21016482,"url":"https://github.com/gsa/inventory-app","last_synced_at":"2025-08-29T13:15:49.078Z","repository":{"id":38961863,"uuid":"148514378","full_name":"GSA/inventory-app","owner":"GSA","description":"Docker image for ckan app powering inventory.data.gov","archived":false,"fork":false,"pushed_at":"2025-05-07T19:14:09.000Z","size":44296,"stargazers_count":31,"open_issues_count":1,"forks_count":10,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-07T20:25:55.734Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GSA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-09-12T17:08:13.000Z","updated_at":"2025-05-07T19:14:11.000Z","dependencies_parsed_at":"2023-10-04T12:26:32.889Z","dependency_job_id":"adcab3f5-9712-446a-bf6a-f9d07ad847f0","html_url":"https://github.com/GSA/inventory-app","commit_stats":{"total_commits":947,"total_committers":27,"mean_commits":"35.074074074074076","dds":0.7307286166842661,"last_synced_commit":"c05363ab828d18282b64b8653e633adf954ac357"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GSA%2Finventory-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GSA%2Finventory-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GSA%2Finventory-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GSA%2Finventory-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GSA","download_url":"https://codeload.github.com/GSA/inventory-app/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254282588,"owners_count":22045127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T10:13:28.616Z","updated_at":"2025-05-15T05:32:46.089Z","avatar_url":"https://github.com/GSA.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# inventory-app\n\n[![CircleCI](https://circleci.com/gh/GSA/inventory-app.svg?style=svg)](https://circleci.com/gh/GSA/inventory-app)\n\nIs a [Docker](https://www.docker.com/)-based [CKAN](http://ckan.org) development environment for [inventory.data.gov](https://inventory.data.gov).  For details on the system architecture, please see our [data.gov systems list](https://github.com/GSA/data.gov/blob/master/SYSTEMS.md)!\n\n_Note: this is currently a work in progress. We're mainly using this to manage\nthe `requirements-freeze.txt` for production dependencies. Very little works beyond that._\n\n## Development\n\n### Prerequisites\n\n- [Docker and Docker Compose](https://docs.docker.com/compose/)\n- [Cloud Foundry](https://docs.cloudfoundry.org/cf-cli/install-go-cli.html) CLI v7\n\n### Getting started\n\nBuild and bring up the containers.\n\n    make up\n    make up-with-data [_Gives development environment basic user, organization, and dataset_]\n\nOpen CKAN to verify it's working\n\n    open http://localhost:5000\n\nIf you would like to seed data into the system, examine the test framework (`e2e/cypress/support/command.js`) for some examples of creating organizations and/or datasets with resources.\n\n### docker compose commands\n\nTo enter into the app container in interactive mode as root, you will need to run the following:\n\n    docker compose exec app /bin/bash\n\nTo run a one off command inside the container:\n\n    docker compose exec app {command}\n\n### Update dependencies\n\nUpdate the \"lock\" file for dependencies.\n\n    make build requirements\n\nThis freezes the requirements at `requirements.txt`. Run the tests with the\nupdated dependencies.\n\n    make test\n\n### Live Editing\n\nTo edit CKAN or extension code live, the local code needs to be attached via a volume.\nAdd a local extension folder path into the `docker-compose.yml` file that you would like to edit\n(see volume section for commented example).\nAfter editing the extension/ckan core (see below), run `make up` (the app will not start appropriately)\nthen run `make debug` to restart the application with an interactive console.\n\n_TODO: tested `--reload` functionality of gunicorn, but [does not work well with paster flag](https://docs.gunicorn.org/en/stable/settings.html#reload)._\n_Hopefully this option improves in the future._\n\n#### Debugger\n\nAdd `import ipdb; ipdb.set_trace()` as a new line where you would like to start debugging. Then run `make debug`, and a new instance of the ckan app will be started that has an interactive console. Once the debugger statement is triggered, then a command prompt should display in the console. See [documentation](https://docs.python.org/3/library/pdb.html#debugger-commands) for available commands. `ipdb` is preferred for styling/readability reasons, but `pdb` will work as well. `web-pdb` was tested, but has various timing complications of it's own that causes unnecessary complications and failures.\n\nThe flask debugger is also imported as a dev requirement and turned on by default in the\n`development.ini` file (`debug = true`), which gives some UI tools on the webpage to parse stack\ntraces and various other examination tools. The behavior is inconsistent, probably due to\nckan serving pages as pylons sometimes and flask at others.\n\n**Make sure you remove all pdb statements before commiting to any repository!**\n\n### Tests\n\n    make test\n\nThe tests utilize cypress. The above command runs in \"headless\" mode, and debugging capabilities are limited. To fully install and rapidly iterate on tests, install cypress locally with npm.\n\n    npm install cypress\n\nThen, you should be able to run `make cypress` to turn on cypress in interactive mode. If you are using WSL you may need additional setup. Start at [this walkthrough](https://nickymeuleman.netlify.app/blog/gui-on-wsl2-cypress), and consider asking a team member for setup help. You will also need to install npx to use the make command.\n\nPlease be aware that the tests attempt to clean themselves after each spec file by specifically removing datasets and organizations. If datasets are manually added into the test organization, the dataset and organization may not be removed and this may have unintended consequences. If the system is in a bad state, you should be able to run `make clean` to restart in a clean environment.\n\n#### Extension Tests\n\n_TODO: add details about running and editing extension tests here._\n\n## Deploying to cloud.gov\n\nCopy `vars.yml.template` to `vars.yml`, and customize the values in that file. Then, assuming you're logged in for the Cloud Foundry CLI:\n\nCreate the database used by datastore. `${app_name}` should be the same as what you have in vars.yml.\n\n    cf create-service aws-rds micro-psql ${app_name}-datastore --wait\n\nCreate the database used by CKAN itself. You have to wait a bit for the datastore DB to be available.\n\n    cf create-service aws-rds small-psql ${app_name}-db --wait\n\nCreate the s3 bucket for data storage.\n\n    cf create-service s3 basic-sandbox ${app_name}-s3 --wait\n\nCreate the Redis service for cache\n\n    cf create-service aws-elasticache-redis redis-dev ${app_name}-redis --wait\n\nDeploy the Solr instance.\n\n    cf push --vars-file vars.yml ${app_name}-solr --wait\n\nCreate the secrets service to store secret environment variables. See [Secrets](#secrets) below.\n\nDeploy the CKAN app.\n\n    cf push --vars-file vars.yml ${app_name}\n\nEnsure the inventory app can reach the Solr app.\n\n    cf add-network-policy ${app_name} ${app_name}-solr --protocol tcp --port 8983\n\nYou should now be able to visit `https://[ROUTE]`, where `[ROUTE]` is the route reported by `cf app ${app_name}`.\n\n### Secrets\n\nTips on managing\n[secrets](https://github.com/GSA/datagov-deploy/wiki/Cloud.gov-Cheat-Sheet#secrets-management).\nWhen creating the service for the first time, use `create-user-provided-service`\ninstead of update.\n\n    cf update-user-provided-service ${app_name}-secrets -p \"CKAN___BEAKER__SESSION__SECRET, CKAN___WTF_CSRF_SECRET_KEY, DS_RO_PASSWORD, NEW_RELIC_LICENSE_KEY, SAML2_PRIVATE_KEY\"\n\nName | Description | Where to find\n---- | ----------- | -------------\nCKAN___BEAKER__SESSION__SECRET | Session secret for encrypting CKAN sessions.  | `pwgen -s 32 1`\nCKAN___WTF_CSRF_SECRET_KEY | CSRF secret for generating CSRF tokens.  | `pwgen -s 32 1`\nDS_RO_PASSWORD | Read-only password to configure for the [datastore](https://docs.ckan.org/en/2.9/maintaining/datastore.html) user. | Initially randomly generated [#2839](https://github.com/GSA/datagov-deploy/issues/2839)\nNEW_RELIC_LICENSE_KEY | New Relic License key. | New Relic account settings.\nSAML2_PRIVATE_KEY | Base64 encoded SAML2 key matching the certificate configured for Login.gov | [Google Drive](https://drive.google.com/drive/u/0/folders/1VguFPRiRb1Ljnm_6UShryHWDofX0xBnU).\n\n### CI configuration\n\nCreate a GitHub environment for each application you're deploying. Each\nGH environment should be configured with secrets from a [ci-deployer service\naccount](https://github.com/GSA/datagov-deploy/wiki/Cloud.gov-Cheat-Sheet#space-organization).\n\nSecret name | Description\n----------- | -----------\nCF_SERVICE_AUTH | The service key password.\nCF_SERVICE_USER | The service key username.\n\n## Login.gov integration\n\nWe use Login.gov as our\n[SAML2](https://github.com/GSA/datagov-deploy/wiki/SAML2-authentication)\nIdentity Provider (IdP). Production apps use the production Login.gov instance\nwhile other apps use the Login.gov identity sandbox.\n\nEach year in March, Login.gov rotates their credentials. See our\n[wiki](https://github.com/GSA/datagov-deploy/wiki/SAML2-authentication#working-with-logingov)\nfor details.\n\nOur Service Provider (SP) certificate and key are provided in through\nenvironment variable and user-provided service.\n\nThe Login.gov IdP metadata is stored in file under `config/`.\n\n## Adding Organizations, Publishers, and Users\n\nSee [instructions here](https://github.com/GSA/data.gov/wiki/inventory.data.gov).\n\n## License and Contributing\n\nWe're so glad you're thinking about re-using and/or contributing to Data.gov!\n\nBefore contributing to Data.gov we encourage you to read our\n[CONTRIBUTING](CONTRIBUTING.md) guide, our [LICENSE](LICENSE.md), and our README\n(you are here), all of which should be in this repository. If you have any\nquestions, you can email the Data.gov team at\n[datagov@gsa.gov](mailto:datagov@gsa.gov).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsa%2Finventory-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgsa%2Finventory-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsa%2Finventory-app/lists"}