https://github.com/cncf/devstats-docker-lf

📈DevStats deployment using docker and managed AuroraDB and ElasticSearch. Optionally you can use vagrant.
https://github.com/cncf/devstats-docker-lf

Last synced: about 1 year ago
JSON representation

📈DevStats deployment using docker and managed AuroraDB and ElasticSearch. Optionally you can use vagrant.

• Host: GitHub
• URL: https://github.com/cncf/devstats-docker-lf
• Owner: cncf
• License: apache-2.0
• Created: 2019-03-21T10:17:59.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2020-09-18T13:59:58.000Z (almost 6 years ago)
• Last Synced: 2025-03-27T08:11:35.120Z (over 1 year ago)
• Language: Shell
• Homepage:
• Size: 48.8 KB
• Stars: 6
• Watchers: 4
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

          
# devstats-docker-lf

DevStats deployment using docker and managed AuroraDB and ElasticSearch

# Usage

# Devstats image

- Use `./docker/docker_build.sh` to build `devstats-lfda` docker image.

- Use `./docker/docker_bash.sh` to bash into `devstats-lfda` docker container.

- Use `./docker/docker_publish.sh` to publish `devstats-lfda` image to the dockerhub.

- Use `./docker/docker_remove.sh` to remove `devstats-lfda` docker image.

- Use `./docker/docker_cleanup.sh` to remove all containers and do `docker system prune`.

- Use `./docker/docker_remove_mapped_data.sh` to remove all data from directories shared between the host and containers.

# ElasticSearch

- Use `./docker/docker_es.sh` to start dockerized ElasticSearch 6.5.1 instance.

- Use `./docker/docker_es_bash.sh` to bash into the ElasticSearch container.

- Use `./docker/docker_es_indexes.sh` to see ES indexes.

- Use `./docker/docker_es_types.sh index` to see given index types.

- Use `./docker/docker_es_values.sh index` to see given index types.

- Use `./docker/docker_es_dump.sh indexname filename` to dump ES index into a file.

- Use `./docker/docker_es_restore.sh indexname filename` to restore ES index from a file.

- Use `./docker/docker_es_query.sh index type query` to query ES instance (allows quick lucene queries).

- Use `./docker/docker_es_search.sh index type query` to search ES instance (allows full query search JSON syntax).

- Use `./docker/docker_es_logs.sh` to see ES logs.

- Use `./docker/docker_es_health.sh` to do a health check (after succesfull deployment). It will display some inmdex structures and do some basic queries.

- Use `./docker/docker_remove_es.sh` to remove dockerized postgres instance.

- Use `./docker/docker_es_wait.sh` to wait for the ES docker container become ready.

# Postgres and deployment

- Use `PG_PASS=... ./docker/docker_psql.sh` to start dockerized postgres:11 instance.

- Use `PG_PASS=... ./docker/docker_psql_bash.sh` to bash into the Postgres container.

- To deploy from the container use `` GHA2DB_GITHUB_OAUTH="`cat /etc/github/oauths`" PG_PASS=... PG_PASS_RO=... PG_PASS_TEAM=... ./docker/docker_deploy_from_container.sh ``.

- It will deploy few Linux Foundation Networking projects into the dockerized postgres.

- It will use `./docker/docker_deploy_all.sh` script which currently uses `SKIPGRAFANA=1` and `NOBACKUP=1`.

- Use `PG_PASS=... ./docker/docker_psql_into_logs_db.sh` to connect to dockerized postgres host post 65432 -> container port 5432.

- Use `PG_PASS=... ./docker/docker_display_logs.sh` to see deployment logs.

- Use `PG_PASS=... ./docker/docker_health.sh` to do a health check (after succesfull deployment). It will display number of texts in the database.

- Use `./docker/docker_remove_psql.sh` to remove dockerized postgres instance.

- Use `PG_PASS=... ./docker/docker_psql_wait.sh` to wait for the postgres docker container become ready.

# Devstats hourly sync

- Use `` GHA2DB_GITHUB_OAUTH="`cat /etc/github/oauths`" PG_PASS=... ./docker/docker_devstats.sh `` to do devstats sync. This should be run hourly, ideally when new GHA files are available which is about every hour:08.

# DockerHub

- Devstats image can be pulled from the [docker hub](https://hub.docker.com/r/lukaszgryglicki/devstats-lfda/).

# One command test all

- Use `PASS=... ./docker/docker_test_all.sh` to test full deployment from the container. It will try to use `/etc/github/oauths` or `/etc/github/oauth` as a GitHub OAuth token (if file is present, otherwise it will use public mode - very restricted).

- Or specify GitHub OAuth token manually `GHA2DB_GITHUB_OAUTH=... PASS=... ./docker/docker_test_all.sh` to test full deployment from the container.

- Final command on a fresh system should be `clear; GHA2DB_GITHUB_OAUTH=key PASS=test ./docker/docker_test_all.sh`.

- To use aurora postgres database `clear; GHA2DB_GITHUB_OAUTH=key PASS=test AURORA=1 ./docker/docker_test_all.sh`.

# Vagrant

You can use vagrant for deployment, use scripts from `./vagrant` directory.

To test everything use `./deploy.sh` script. Prepend with `VAGRANT=1` or `DOCKER=1`.