{"id":18798947,"url":"https://github.com/cncf/devstats-docker-lf","last_synced_at":"2025-04-13T17:22:36.801Z","repository":{"id":72000107,"uuid":"176920733","full_name":"cncf/devstats-docker-lf","owner":"cncf","description":"📈DevStats deployment using docker and managed AuroraDB and ElasticSearch. Optionally you can use vagrant.","archived":false,"fork":false,"pushed_at":"2020-09-18T13:59:58.000Z","size":50,"stargazers_count":6,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-27T08:11:35.120Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cncf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-21T10:17:59.000Z","updated_at":"2025-03-17T23:29:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"610839e7-b597-4b75-b7ac-d29865a0ecce","html_url":"https://github.com/cncf/devstats-docker-lf","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cncf%2Fdevstats-docker-lf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cncf%2Fdevstats-docker-lf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cncf%2Fdevstats-docker-lf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cncf%2Fdevstats-docker-lf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cncf","download_url":"https://codeload.github.com/cncf/devstats-docker-lf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248750869,"owners_count":21155797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T22:13:31.465Z","updated_at":"2025-04-13T17:22:36.794Z","avatar_url":"https://github.com/cncf.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# devstats-docker-lf\n\nDevStats deployment using docker and managed AuroraDB and ElasticSearch\n\n\n# Usage\n\n# Devstats image\n\n- Use `./docker/docker_build.sh` to build `devstats-lfda` docker image.\n- Use `./docker/docker_bash.sh` to bash into `devstats-lfda` docker container.\n- Use `./docker/docker_publish.sh` to publish `devstats-lfda` image to the dockerhub.\n- Use `./docker/docker_remove.sh` to remove `devstats-lfda` docker image.\n- Use `./docker/docker_cleanup.sh` to remove all containers and do `docker system prune`.\n- Use `./docker/docker_remove_mapped_data.sh` to remove all data from directories shared between the host and containers.\n\n# ElasticSearch\n\n- Use `./docker/docker_es.sh` to start dockerized ElasticSearch 6.5.1 instance.\n- Use `./docker/docker_es_bash.sh` to bash into the ElasticSearch container.\n- Use `./docker/docker_es_indexes.sh` to see ES indexes.\n- Use `./docker/docker_es_types.sh index` to see given index types.\n- Use `./docker/docker_es_values.sh index` to see given index types.\n- Use `./docker/docker_es_dump.sh indexname filename` to dump ES index into a file.\n- Use `./docker/docker_es_restore.sh indexname filename` to restore ES index from a file.\n- Use `./docker/docker_es_query.sh index type query` to query ES instance (allows quick lucene queries).\n- Use `./docker/docker_es_search.sh index type query` to search ES instance (allows full query search JSON syntax).\n- Use `./docker/docker_es_logs.sh` to see ES logs.\n- Use `./docker/docker_es_health.sh` to do a health check (after succesfull deployment). It will display some inmdex structures and do some basic queries.\n- Use `./docker/docker_remove_es.sh` to remove dockerized postgres instance.\n- Use `./docker/docker_es_wait.sh` to wait for the ES docker container become ready.\n\n\n# Postgres and deployment\n\n- Use `PG_PASS=... ./docker/docker_psql.sh` to start dockerized postgres:11 instance.\n- Use `PG_PASS=... ./docker/docker_psql_bash.sh` to bash into the Postgres container.\n- To deploy from the container use `` GHA2DB_GITHUB_OAUTH=\"`cat /etc/github/oauths`\" PG_PASS=... PG_PASS_RO=... PG_PASS_TEAM=... ./docker/docker_deploy_from_container.sh ``.\n- It will deploy few Linux Foundation Networking projects into the dockerized postgres.\n- It will use `./docker/docker_deploy_all.sh` script which currently uses `SKIPGRAFANA=1` and `NOBACKUP=1`.\n- Use `PG_PASS=... ./docker/docker_psql_into_logs_db.sh` to connect to dockerized postgres host post 65432 -\u003e container port 5432.\n- Use `PG_PASS=... ./docker/docker_display_logs.sh` to see deployment logs.\n- Use `PG_PASS=... ./docker/docker_health.sh` to do a health check (after succesfull deployment). It will display number of texts in the database.\n- Use `./docker/docker_remove_psql.sh` to remove dockerized postgres instance.\n- Use `PG_PASS=... ./docker/docker_psql_wait.sh` to wait for the postgres docker container become ready.\n\n\n# Devstats hourly sync\n\n- Use `` GHA2DB_GITHUB_OAUTH=\"`cat /etc/github/oauths`\" PG_PASS=... ./docker/docker_devstats.sh `` to do devstats sync. This should be run hourly, ideally when new GHA files are available which is about every hour:08.\n\n\n# DockerHub\n\n- Devstats image can be pulled from the [docker hub](https://hub.docker.com/r/lukaszgryglicki/devstats-lfda/).\n\n\n# One command test all\n\n- Use `PASS=... ./docker/docker_test_all.sh` to test full deployment from the container. It will try to use `/etc/github/oauths` or `/etc/github/oauth` as a GitHub OAuth token (if file is present, otherwise it will use public mode - very restricted).\n- Or specify GitHub OAuth token manually `GHA2DB_GITHUB_OAUTH=... PASS=... ./docker/docker_test_all.sh` to test full deployment from the container.\n- Final command on a fresh system should be `clear; GHA2DB_GITHUB_OAUTH=key PASS=test ./docker/docker_test_all.sh`.\n- To use aurora postgres database `clear; GHA2DB_GITHUB_OAUTH=key PASS=test AURORA=1 ./docker/docker_test_all.sh`.\n\n\n# Vagrant\n\nYou can use vagrant for deployment, use scripts from `./vagrant` directory.\n\nTo test everything use `./deploy.sh` script. Prepend with `VAGRANT=1` or `DOCKER=1`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcncf%2Fdevstats-docker-lf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcncf%2Fdevstats-docker-lf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcncf%2Fdevstats-docker-lf/lists"}