{"id":20645000,"url":"https://github.com/romanow/openshift-homework2","last_synced_at":"2025-04-16T02:11:36.105Z","repository":{"id":43351857,"uuid":"462353615","full_name":"Romanow/openshift-homework2","owner":"Romanow","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-14T13:50:55.000Z","size":45,"stargazers_count":1,"open_issues_count":5,"forks_count":11,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-16T02:11:26.489Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Romanow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-22T15:15:58.000Z","updated_at":"2024-08-14T06:25:20.000Z","dependencies_parsed_at":"2024-11-16T16:18:49.298Z","dependency_job_id":"ab03f639-5f27-4a40-8be3-2ab57cdfc5a0","html_url":"https://github.com/Romanow/openshift-homework2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Romanow%2Fopenshift-homework2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Romanow%2Fopenshift-homework2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Romanow%2Fopenshift-homework2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Romanow%2Fopenshift-homework2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Romanow","download_url":"https://codeload.github.com/Romanow/openshift-homework2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249183106,"owners_count":21226142,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T16:18:23.443Z","updated_at":"2025-04-16T02:11:36.097Z","avatar_url":"https://github.com/Romanow.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Домашнее задание: Monitoring\n\n## Формулировка\n\n1. Grafana + Prometheus:\n    1. создать Dashboard с графиком суммарного потребления памяти java приложениями (Heap + Metaspace);\n    2. настроить Alerting в Telegram / Slack если потребление памяти больше некоторого порога (выбираете сами по\n       графику).\n2. ELK Stack:\n    1. в Kibana построить график количества сообщений в логах в разрезе Log Level (debug, info, warn, error);\n    2. в Kibana построить график количества уникальных запросов за минуту (уникальные TraceId).\n\n## Создание Telegram Bot\n\n1. Создать бота через `@BotFather`\n   ```\n   /newbot\n   name: k8s_monitoring_\u003cyour-name\u003e\n   id: k8s_monitoring_bot\n   ```\n2. Создать канал K8S Monitoring, добавить `@k8s_monitoring_\u003cyour-name\u003e` как администратора. Отправить **хотя бы одно\n   сообщение** в группу.\n3. После этого через Telegram API получить chart ID: `http://api.telegram.org/bot\u003ctoken\u003e/getUpdates`\n\n## Запуск minikube\n\n```shell\n$ minikube start --driver=virtualbox --cpus 4 --memory 10240\n😄  minikube v1.26.1 на Darwin 12.4\n✨  Используется драйвер virtualbox на основе конфига пользователя\n👍  Запускается control plane узел minikube в кластере minikube\n🔥  Creating virtualbox VM (CPUs=4, Memory=10240MB, Disk=20000MB) ...\n🐳  Подготавливается Kubernetes v1.24.3 на Docker 20.10.17 ...\n    ▪ Generating certificates and keys ...\n    ▪ Booting up control plane ...\n    ▪ Configuring RBAC rules ...\n    ▪ Используется образ gcr.io/k8s-minikube/storage-provisioner:v5\n🌟  Включенные дополнения: default-storageclass, storage-provisioner\n🔎  Компоненты Kubernetes проверяются ...\n🏄  Готово! kubectl настроен для использования кластера \"minikube\" и \"default\" пространства имён по умолчанию\n\n$ minikube addons enable ingress\n💡  ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.\nYou can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS\n    ▪ Используется образ k8s.gcr.io/ingress-nginx/controller:v1.2.1\n    ▪ Используется образ k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1\n    ▪ Используется образ k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1\n🔎  Verifying ingress addon...\n🌟  The 'ingress' addon is enabled\n\n$ sudo tee -a /etc/hosts \u003e /dev/null \u003c\u003cEOT\n$(minikube ip)  store-service.local\n$(minikube ip)  grafana.local\n$(minikube ip)  kibana.local\nEOT\n```\n\n## Установка сервисов в minikube\n\nДля установки сервисов будем использовать Helm:\n\n```shell\n$ cd helm/\n\n# Grafana \u0026 Prometheus\n$ helm install monitoring monitoring-chart/\n\n# ElasticSearch \u0026 Logstash \u0026 Kibana\n$ helm install logging logging-chart\n\n# Postgres \u0026 Services\n$ helm install services services-chart\n\n# проверить что все сервисы запущены\n$ oc get pods\nNAME                                           READY   STATUS    RESTARTS   AGE\nelasticsearch-58bfdcf6ff-xh27r                 1/1     Running   0          67m\ngrafana-55bd99f8db-p7wdl                       1/1     Running   0          23m\nkibana-68459f9f86-hrdxv                        1/1     Running   0          67m\nlogstash-77c4df78d8-hh9l5                      1/1     Running   0          67m\norder-deployment-86f497fcf4-jsb4n              3/3     Running   0          6m19s\npostgres-order-deployment-5c5c8894d7-hrcnz     1/1     Running   0          6m19s\npostgres-store-deployment-c67d56bc6-wj55c      1/1     Running   0          6m19s\npostgres-warehouse-deployment-dddc4c86-fkc87   1/1     Running   0          6m19s\npostgres-warranty-deployment-f5757b75c-9pm2c   1/1     Running   0          6m19s\nprometheus-dc7cc886c-wbxrp                     1/1     Running   0          23m\nstore-deployment-68b4d874c-5hwmv               3/3     Running   0          6m19s\nwarehouse-deployment-64579b8869-4s9mw          3/3     Running   0          6m19s\nwarranty-deployment-7cd8d95874-nvlk4           3/3     Running   0          6m19s\n\n# (optional) для запуска потребуется newman, его нужно установить\n# https://support.postman.com/hc/en-us/articles/115003703325-How-to-install-Newman\n$ npm install -g newman\n\n# заменить name-suffix в файле postman/environment.json\n$ cd ../scripts/\n$ ./make-requests.sh\nServices\n\n❏ Store service\n↳ [store] Health Check\n  GET http://store-service.local/manage/health [200 OK, 355B, 95ms]\n  ✓  Health check\n\n↳ [store] Purchase item\n  POST http://store-service.local/api/v1/store/6D2CB5A0-943C-4B96-9AA6-89EAC7BDFD2B/purchase [201 Created, 316B, 115ms]\n  ✓  Purchase item\n\n↳ [store] User order info\n  GET http://store-service.local/api/v1/store/6D2CB5A0-943C-4B96-9AA6-89EAC7BDFD2B/161bb6b0-d425-4505-9e76-5cc1c4d06593 [200 OK, 398B, 98ms]\n  ✓  User order info\n\n↳ [store] User orders\n  GET http://store-service.local/api/v1/store/6D2CB5A0-943C-4B96-9AA6-89EAC7BDFD2B/orders [200 OK, 594B, 94ms]\n  ✓  User orders\n\n↳ [store] Warranty request\n  POST http://store-service.local/api/v1/store/6D2CB5A0-943C-4B96-9AA6-89EAC7BDFD2B/161bb6b0-d425-4505-9e76-5cc1c4d06593/warranty [200 OK, 320B, 766ms]\n  ✓  Warranty request\n\n↳ [store] Return order\n  DELETE http://store-service.local/api/v1/store/6D2CB5A0-943C-4B96-9AA6-89EAC7BDFD2B/161bb6b0-d425-4505-9e76-5cc1c4d06593/refund [204 No Content, 153B, 125ms]\n  ✓  Return order\n\n┌─────────────────────────┬────────────────────┬────────────────────┐\n│                         │           executed │             failed │\n├─────────────────────────┼────────────────────┼────────────────────┤\n│              iterations │                  1 │                  0 │\n├─────────────────────────┼────────────────────┼────────────────────┤\n│                requests │                  6 │                  0 │\n├─────────────────────────┼────────────────────┼────────────────────┤\n│            test-scripts │                  6 │                  0 │\n├─────────────────────────┼────────────────────┼────────────────────┤\n│      prerequest-scripts │                  5 │                  0 │\n├─────────────────────────┼────────────────────┼────────────────────┤\n│              assertions │                  6 │                  0 │\n├─────────────────────────┴────────────────────┴────────────────────┤\n│ total run duration: 1446ms                                        │\n├───────────────────────────────────────────────────────────────────┤\n│ total data received: 936B (approx)                                │\n├───────────────────────────────────────────────────────────────────┤\n│ average response time: 215ms [min: 94ms, max: 766ms, s.d.: 246ms] │\n└───────────────────────────────────────────────────────────────────┘\n```\n\n### Конфигурация Kibana\n\nЛогин и пароль: `logging`/`qwerty`.\n\nСоздание индекса для логгирования: `Management` -\u003e `Stack Management` -\u003e `Kibana` -\u003e `Index Pattern` -\u003e Create index\npattern -\u003e Name: `logstash-*`, Timestamp field: `@timestamp`.\n\n### Конфигурация Grafana\n\nЛогин и пароль: `admin`/`admin`.\n\nGrafana Dashboard: 10280.\n\nМетрики Prometheus:\n\n```shell\ncurl \"http://localhost:8080/manage/prometheus --user management:passwd\" | jq\n```\n\n### Возможные ошибки\n\n##### Ошибка скачивания образов Error: ErrImagePull\n\n```\nFailed to pull image \"romanowalex/order-service:v1.0-lecture\": rpc error: code = Unknown desc = context deadline exceeded\n``` \n\nПричина: kubelet не успевает загрузить образ за 2 минуты, и надо его загрузить в minikube руками.\n\n```shell\n$ minikube image pull romanowalex/order-service:v1.0-lecture\n```\n\n##### Ошибка скачивания ElasticSearch, Logstash, Kibana\n\n```\nThe Amazon CloudFront distribution is configured to block access from your country.\nWe can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error.\nTry again later, or contact the app or website owner\n```\n\nДля скачивания образов ElasticSearch нужно включить VPN, т.к. docker.elastic.co отказывается скачивать с российских ip.\n\n##### Ошибка синхронизации времени внутри VM\n\nЕсли VM ушла в сон, то она некорректно синхронизирует время с host машиной. В случае возникновения ошибки:\n\n```\nError fetching server time: Detected 41629.25100016594 seconds time difference between your browser and the server.\n```\n\nРешение: нужно синхронизировать время:\n\n```shell\nminikube ssh -- docker run -it --rm --privileged --pid=host alpine nsenter -t 1 -m -u -n -i date -u $(date -u +%m%d%H%M%Y.%S)\n```\n\n## Ссылки\n\n1. [minikube start](https://minikube.sigs.k8s.io/docs/start/)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fromanow%2Fopenshift-homework2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fromanow%2Fopenshift-homework2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fromanow%2Fopenshift-homework2/lists"}