{"id":19670994,"url":"https://github.com/emc-mongoose/darzee","last_synced_at":"2025-04-29T00:32:19.014Z","repository":{"id":39546443,"uuid":"85964073","full_name":"emc-mongoose/darzee","owner":"emc-mongoose","description":"Web service that provides the UI for controlling and monitoring Mongoose runs.","archived":false,"fork":false,"pushed_at":"2023-01-07T03:23:34.000Z","size":43951,"stargazers_count":4,"open_issues_count":25,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-03-26T22:35:20.844Z","etag":null,"topics":["angular","angular-cli","docker","docker-compose","javascript","node-js","performance-monitoring","prometheus","typescript","webui"],"latest_commit_sha":null,"homepage":"https://github.com/emc-mongoose/mongoose-base","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emc-mongoose.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-23T15:11:39.000Z","updated_at":"2024-03-26T22:35:20.845Z","dependencies_parsed_at":"2023-02-06T11:01:34.943Z","dependency_job_id":null,"html_url":"https://github.com/emc-mongoose/darzee","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emc-mongoose%2Fdarzee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emc-mongoose%2Fdarzee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emc-mongoose%2Fdarzee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emc-mongoose%2Fdarzee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emc-mongoose","download_url":"https://codeload.github.com/emc-mongoose/darzee/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224142764,"owners_count":17262884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angular","angular-cli","docker","docker-compose","javascript","node-js","performance-monitoring","prometheus","typescript","webui"],"created_at":"2024-11-11T17:07:47.608Z","updated_at":"2024-11-11T17:07:48.134Z","avatar_url":"https://github.com/emc-mongoose.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Issue Tracker](https://img.shields.io/badge/Issue-Tracker-red.svg)](https://mongoose-issues.atlassian.net/projects/GUI)\n[![Docker Pulls](https://img.shields.io/docker/pulls/emcmongoose/darzee.svg)](https://hub.docker.com/r/emcmongoose/darzee/)\n[![CI status](https://gitlab.com/Andrey_Koltsov/darzee/badges/master/pipeline.svg)](https://gitlab.com/Andrey_Koltsov/darzee/badges/master)\n\n# Darzee\nProvides web interface for Mongoose - storage performance testing tool maintained by Dell EMC. \n\n# Contents \n\n1. [Overview](#1-overview)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;1.1 [Launch Mongoose run](#11-launch-mongoose-run)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.1.1 [Nodes selection](#111-nodes-selection)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.1.2 [Configuration](#112-configuration)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.1.3 [Scenario](#113-scenario)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;1.2 [Monitoring](#12-monitoring)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.2.1 [Mongoose runs table](#121-mongoose-runs-table)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.2.2 [Mongoose run statistics](#122-mongoose-run-statistics)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.2.2.1 [Mongoose logs](#1221-mongoose-logs)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;1.2.2.2 [Mongoose charts](#1222-mongoose-charts)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;1.3 [Terminate Mongoose run](#13-terminate-mongoose-run)\u003cbr/\u003e\n2. [Configuration](#2-configuration)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;2.1 [Deploying ports](#21-deploying-ports)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;2.2 [Image versions](#22-image-versions)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;2.3 [Container configuration](#23-container-configuration)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;2.3.1 [Prometheus](#231-prometheus)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;2.4 [Other parameters](#24-other-parameters) \u003cbr/\u003e\n3. [Build and run](#3-build-and-run)\u003cbr/\u003e\n4. [Deployment](#4-deployment)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;4.1 [Docker](#41-docker)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;4.2 [Kubernetes](#42-kubernetes)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; 4.2.1 [Test Darzee in Kubernetes using predefined Mongoose environment](#421-test-darzee-in-kubernetes-using-predefined-mongoose-environment)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; 4.2.1.1 [Mongoose base](#4211-mongoose-base)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; 4.2.1.2 [Mongoose Pravega driver](#4212-mongoose-pravega-driver)\u003cbr/\u003e\n5. [Troubleshooting](#5-troubleshooting)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;5.1 [Prometheus inactivity](#51-prometheus-inactivity)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;5.2 [Missing load step node address](#52-missing-load-step-node-address)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;5.3 [Chart.js error on start up](#53-chartjs-error-on-start-up)\u003cbr/\u003e\n\n6. [Open issues](#6-open-issues)\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;6.1 [Mongoose run logs unavailability](#61-mongoose-run-logs-unavailability)\u003cbr/\u003e\n\n\n# 1. Overview \n\nDarzee lets you configure, run and monitor Mongoose runs. \n![](screenshots/architecture_overview.png)\n\n\n## 1.1 Launch Mongoose run\n\nMongoose launching process is divided in 3 steps: \n![](screenshots/setup/set_up_overview.png)\n\n\n### 1.1.1 Nodes selection \n\n![](screenshots/setup/nodes/nodes-selection.png)\nMultiple Mongoose run nodes could be selected. \u003cb\u003eFirst\u003c/b\u003e selected node will be the entry one. \u003cbr/\u003e\n\n## 1.1.2 Configuration \n\n![](screenshots/setup/configuration/configuration.png)\n\nConfiguration is presented as an editable JSON tree. \u003cbr/\u003e\nInitially, the configuration is fetched from the entry node.\n\n## 1.1.3 Scenario \nMongoose's scenarios are written in JavaScript. It's possible to write JavaScript code in the UI using code editor. \n![](screenshots/setup/scenario/scenario_general.png)\n\n\n\n## 1.2 Monitoring \n\n### 1.2.1 Mongoose runs table\nDiscovered Mongoose's runs are displayed within the runs table. \n![](screenshots/runs-table/run_table_filled.png)\n\nThere's 3 possible status of Mongoose runs: \t \n\t \n* ![#29a329](https://placehold.it/15/29a329/000000?text=+) \u003cb\u003eFinished\u003c/b\u003e - means Mongoose run has finished and its logs are available; \u003cbr/\u003e\n* ![#ffd633](https://placehold.it/15/ffd633/000000?text=+) \u003cb\u003eRunning\u003c/b\u003e - means Mongoose is still performing the benchmark. Charts will be updated every time Darzee received a new data; \u003cbr/\u003e\n* ![#a6a6a6](https://placehold.it/15/a6a6a6/000000?text=+) \u003cb\u003eUnavailable\u003c/b\u003e - means info about Mongoose run has been found on the server, but the related data is lost or couldn't be loaded. \u003cbr/\u003e\n\nYou could see Mongoose run status or run details by pressing the status icon. \n\n![](screenshots/runs-table/run_status_overview.png)\n\n\nBy pressing run status icon, you'll be redirected to run statistics screen. See [Mongoose logs](#1421-mongoose-logs), [Mongoose charts](#1422-mongoose-charts).\u003cbr/\u003e\n\n### 1.2.2 Mongoose run statistics\n\n![](screenshots/monitoring/run_statistics_overview.png)\n If the run is still being processed, the data will be updating dynamically based on the Mongoose's exported metrics and generated logs. \n\n\n#### 1.2.2.1 Mongoose logs \n![](screenshots/monitoring/logs/logs_overview.png)\n\n#### 1.2.2.2 Mongoose charts\n\n![](screenshots/monitoring/charts/charts_overview.png)\n\nCharts are drawn based on Mongoose's exported metrics. There're 4 types of Mongoose charts available:\n\n* \u003cb\u003eduration\u003c/b\u003e - mean duration of Mongoose's performed operations;\n* \u003cb\u003ebandwidth\u003c/b\u003e - amount of processed bytes per second; \n* \u003cb\u003ethroughput\u003c/b\u003e - shows rate of successful and failed operations performed by Mongoose. It takes mean values for each metric; \n* \u003cb\u003elatency\u003c/b\u003e - shows minimal and maximal latency during Mongoose's run; \n\n## 1.3 Terminate Mongoose run\n\nProcessing Mongoose run could be terminated from the UI. \n![](screenshots/monitoring/termination_instruction.png)\n\n\n# 2. Configuration \n\nUI's configuration depends on the parameters listen within [.env](.env) file.\n\n## 2.1 Deploying ports \nThe following parameters are being used to specify deploying ports of the services: \n \n* DARZEE_PORT - for Darzee itself; \n* PROMETHEUS_PORT - for Prometheus; \n* MONGOOSE_PORT - for Mongoose; \n\n## 2.2 Image versions \nThe following parameters are being used to specify docker image version of a specific service: \n* DARZEE_VER - Darzee's image version; \n* PROMETHEUS_VER - Prometheus image version; \n\n\n## 2.3 Container configuration \n\n### 2.3.1 Prometheus \nPrometheus configuration is being rewritten in a runtime in order to add targets dynamically. \nTo implement this, we're using internal container volume. \n* PROMETHEUS_CONFIGURATION_PATH - full path for Prometheus configuration; \n* PROMETHEUS_CONFIGURATION_FOLDER_PATH - path to Prometheus folder; \n\n## 2.4 Other parameters \n\n* MONGOOSE_HOST - specifies host of an initially loaded Mongoose (optional); \n\n# 3. Build and run \n\n## 3.1 Build \n\n### 3.1.1 Build docker image \n\nDocker image is being builted via Gradle. To build Docker image, use: \n``` \n$ ./gradlew buildImage\n```\n\n### 3.1.2 Build project \nDarzee has been created with Angular 7.0. You could build it just like any Angular app. \n```\n$ npm install\n$ ng build \n```\n\n## 3.2 Run \n\n### 3.2.1 Run via docker\nDarzee is deploying via docker image. The image contains front-end files, nodeJS server to serving the app and Prometheus.\nIn order to have the access from outside, NodeJS server should be mapped to port 8080 and Prometheus should be mapped to port 9090:\n``` \n$ docker run --env-file .env -p 8080:8080 -p 9090:9090 emcmongoose/darzee\n```\nNote that you should use project's [.env](.env) file.\n\n### 3.2.2 Run in development mode \nDarzee has been created with Angular CLI. It could be ran in development mode using the appropriate command: \n```\n$ npm install\n$ ng serve \n```\n\n# 4. Deployment \n\n## 4.1 Docker\n\nDarzee image is being loaded into the [docker hub](https://hub.docker.com/r/emcmongoose/darzee).\n\n```\nbash\ndocker run \\\n    --env-file .env \\\n    -p {DARZEE_PORT}:{DARZEE_PORT} \\\n    -p {PROMETHEUS_PORT}:{PROMETHEUS_PORT} \\\n    emcmongoose/darzee[-\u003cVERSION\u003e] \n```\nNote that you should use project's [.env](.env) file.\n\n## 4.2 Kubernetes\n\nDarzee can be deployed in a [kubernetes](https://kubernetes.io/) cluster. Examples of kubernetes object discription files are in the directory `/console/kubernetes`. \u003cbr/\u003e\nIn order to connect with the outside network, Darzee uses Kubernetes service `/console/kubernetes/darzee-service.yml`\n\nRun Darzee as a deployment: \u003cbr/\u003e\n```\n$ bash kubectl apply -f /console/kubernetes/darzee-deployment.yaml\n$ bash kubectl apply -f /console/kubernetes/darzee-service.yml\n```\n### 4.2.1 Test Darzee in Kubernetes using predefined Mongoose environment\n\nThe project contains scripts for deploying predefined Mongoose environment in order to run the UI. \u003cbr/\u003e\nMongoose's environments are described via Kubernetes deplyoments. Note that there's 2 type of ports for Mongoose: \n* \u003cb\u003eRemote API port\u003c/b\u003e. You should specify remote API port when adding it into [nodes list](#111-nodes-selection).\n* \u003cb\u003eRMI port\u003c/b\u003e (Remote Method Invocation port). You should specify RMI port when adding additional Mongoose nodes in order to run it in distributed mode.\nRMI ports should be specified in [configuration](#112-configuration) (load-step-node-addrs).\n\n#### 4.2.1.1 Mongoose base \n```\n$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-base/tools\n$ bash chmod +x  create-mongoose-environment.sh\n$ bash ./create-mongoose-environment.sh \n```\nThis will deploy 2 pods, each of them will contain 5 Mongoose instances. \u003cb\u003eRemote API ports are 9991...9994, 9999, RMI ports are 1091...1094, 1099\u003c/b\u003e.\nEvery one of them are connected to the web outside of k8s cluster via the service. \nYou can get service's external IP via \n```\n$ bash kubectl get svc mongoose-base-svc\n$ bash kubectl get svc mongoose-base-svc-2\n```\nCombining external IP with Mongoose ports, you'll get an access to it from the network outside of cluster.\n\nIn order to delete the environment, use: \n```\n$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-base/tools\n$ bash chmod +x  delete-mongoose-environment.sh\n$ bash ./delete-mongoose-environment.sh \n```\n\n#### 4.2.1.2 Mongoose Pravega driver\n```\n$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-pravega/tools\n$ bash chmod +x  create-mongoose-pravega-environment.sh\n$ bash ./create-mongoose-pravega-environment.sh \n```\nThis will create a deployment and a service. \nThe deployment will contain instance of Mongoose Pravega driver, the service will connect it to the network outside of cluster via \u003cb\u003eremote API port 9999\u003c/b\u003e. \u003cbr/\u003e\nNote that in [kubernetes object description for Mongoose pravega driver](/console/src/assets/configuration-examples/kubernetes/mongoose-pravega/mongoose-pravega-service.yml), you should \nspecify a running Pravega controller IP (\u003cb\u003e\u003cPRAVEGA_CONTROLLER_IP\u003e\u003c/b\u003e), otherwise it won't work.\n```\nargs: [--run-node, --storage-namespace=mongoose-pravega-scope, --run-port=9999, --load-step-node-port=1099, \n          --storage-net-node-addrs=\u003cPRAVEGA_CONTROLLER_IP\u003e, --load-batch-size=100, --storage-driver-limit-queue-input=10000]\n```\n\nYou can get service's external IP via \n```\n$ bash kubectl get svc mongoose-pravega-svc\n```\n\nIn order to delete the environment, use: \n```\n$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-pravega/tools\n$ bash chmod +x  delete-mongoose-pravega-environment.sh\n$ bash ./delete-mongoose-pravega-environment.sh \n```\n\n# 5. Troubleshooting \n\n## 5.1 Prometheus inactivity\n\nIf Darzee's container Prometheus is not active, it's possible to use a custom one with some limitations. \u003cbr/\u003e\nIt's recommended to restart the image in case of Prometheus inactivity thought.\n![](screenshots/troubleshooting/prometheus-inactivity.png)\n\n## 5.2 Missing load step node address\n\nIn case data for the load step is fetched from the Prometheus, but its IP address hasn't been found, it's possible to write it from the UI.\n![](screenshots/troubleshooting/mongoose-node-inactivity.png)\n\n## 5.3 Chart.js error on start up\n\u003cb\u003eNOTE: Actual for 06.08.2019\u003c/b\u003e \u003c/br\u003e\n\nWith Angular builder version change, some errors are popping up. We've experienced issues with the [charts library we're using](https://github.com/valor-software/ng2-charts): after building Darzee docker image from Gitlab CI, the UI hasn't been starting up because of invalid linking of the charts library with the UI via Webpack. Althought, it has been working correctly with manual build. \u003c/br\u003e\nDifferent Angular CLI and Angular versions is the reason why the issue took a place. It was resolved [here](https://github.com/emc-mongoose/darzee/pull/132). \u003c/br\u003e\n\n\u003cb\u003eRelated issues: \u003c/b\u003e \n* [ERROR Error: Uncaught (in promise): TypeError: Object(...) is not a function #1117](https://github.com/valor-software/ng2-charts/issues/1117) - same issue; \u003c/br\u003e\n* [\"export 'ɵɵinject' was not found in '@angular/core' since v8.0.0-beta.12](https://github.com/angular/angular/issues/30413) - explains why some libraries are breaking because of different Angular versions usage; \u003c/br\u003e\n\n\n\n# 6. Open issues \nWe're constantly working on making the UI and Mongoose project overall better. \nIn this section, you'd find a list of a high-priority issues that would be fixed as soon as possible.\n\n## 6.1 Mongoose run logs unavailability \nMongoose run's table is constructed based on the retrieved data from Prometheus: run ID, additional nodes, load step ID, etc. The list doesn't contain entry node address. \u003cbr/\u003e\n\u003cb\u003eNow\u003c/b\u003e: Mongoose run's entry nodes addresses are stored within browser's local storage. It helps to retain the information about entry nodes as long as the local storage not getting cleaned up, yet it'd be saved even if the browser and/or tab has been closed.\u003cbr/\u003e\nWe need entry node's address in order to get logs via POST request with Mongoose Logs API to its entry node. \u003cbr/\u003e\n\u003cb\u003eProblem\u003c/b\u003e: while working outside of the browser from which Mongoose run has been launched (other browser, other computer, etc.), the data about run's entry node is missing, thus logs couldn't be gathered via the Logs API.\u003cbr/\u003e\n\u003cb\u003eTemporary solution\u003c/b\u003e: see [Missing load step node address](#52-missing-load-step-node-address).\n\nTrack progress on the issue [here](https://mongoose-issues.atlassian.net/projects/GUI/issues/GUI-137?filter=allissues).\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femc-mongoose%2Fdarzee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femc-mongoose%2Fdarzee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femc-mongoose%2Fdarzee/lists"}