{"id":26507487,"url":"https://github.com/milescb/traccc-aas-performance","last_synced_at":"2025-03-20T23:20:00.802Z","repository":{"id":251818467,"uuid":"838524944","full_name":"milescb/traccc-aaS-performance","owner":"milescb","description":"Performance studies for traccc as-a-service","archived":false,"fork":false,"pushed_at":"2025-02-26T21:07:54.000Z","size":9611,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-26T22:20:16.992Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/milescb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-05T20:24:12.000Z","updated_at":"2025-02-26T21:07:58.000Z","dependencies_parsed_at":"2024-08-09T02:23:56.139Z","dependency_job_id":"e1a679e8-2ec6-4da8-89f5-804b55b67421","html_url":"https://github.com/milescb/traccc-aaS-performance","commit_stats":null,"previous_names":["milescb/traccc-aas-performance"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS-performance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS-performance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS-performance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS-performance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/milescb","download_url":"https://codeload.github.com/milescb/traccc-aaS-performance/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244706541,"owners_count":20496571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-20T23:20:00.379Z","updated_at":"2025-03-20T23:20:00.793Z","avatar_url":"https://github.com/milescb.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `Traccc` as-a-service Performance Studies\n\nSet of scripts to analyze performance from the output of the `perf_analyzer` tool used to analyze performance of models using the NVIDIA triton server. These scripts are designed to analyze the performance of `traccc` as-a-service in particular. The backend for this and instructions about getting the model can be found [in this git repo](https://github.com/milescb/traccc-aaS). \n\n## Do everything in one go\n\nThe file `submit/run_eval.sh` runs performance studies and makes plots. A full command looks like\n\n```\n./submit/run_eval.sh \u003coutput_data_dir\u003e \u003cmodel_name\u003e \u003cbool_multi_gpu\u003e\n```\n\n## Create input for `perf_analyzer`\n\nCreate input json file with \n\n```\npython generate_json.py -i \u003cinput_file\u003e -o \u003coutput_file\u003e\n```\n\n## Performance Analyzer\n\nRun `perf_analyzer` with the following:\n\n```\nperf_analyzer -m traccc-gpu --input-data $DATADIR/../test_data/test_perf_data.json\n```\n\n## Performance Plots\n\n### Throughput as a function of concurrency\n\nRun `perf_analyzer` with the CPU and GPU configuration\n\n```\nperf_analyzer -m traccc-gpu --input-data $DATADIR/../test_data/test_perf_data.json \\\n    --concurrency-range \u003cstart\u003e:\u003cend\u003e:\u003cstep\u003e \\\n    --verbose-csv --collect-metrics \\\n    --measurement-interval 10000 \\\n    -f gpu_1instances.csv \n```\n\nThen, we can make plots with the provide python notebooks. \n\n### Compare CPU / GPU performance on `traccc` examples\n\nFirst, make `.log` files containing the output of running the `traccc` examples via (for instance)\n\n```\n$INSTALLDIR/bin/traccc_seq_example_cuda \\\n    --use-detray-detector \\\n    --detector-file=$DATADIR/tml_detector/trackml-detector.csv \\\n    --digitization-file=$DATADIR/tml_detector/default-geometric-config-generic.json \\\n    --input-directory=$DATADIR/tml_full/ttbar_mu100 \u0026\u003e data/logs/gpu_mu100.log\n```\n\nthen run the following to create the plot:\n\n```\npython compare_gpu_cpu_traccc.py\n```\n\n## Test k8 server performance\n\nTo run `perf_analyzer` on a k8 cluster such as nautilus, follow the instructions on setting up the server in [traccc-aas](https://github.com/milescb/traccc-aaS). Then, forward the gRPC port:\n\n```\nkubectl port-forward service/atlas-sonic 8001:8001 -n atlas-sonic\n```\n\nThis avoids ssl authentication errors interfering with `perf_analyzer`. Then, run `perf_analyzer` as usual: \n\n```\ndata/nrp_inference_multi_gpu/ # if not already created\nperf_analyzer -m traccc-gpu -i grpc \\\n    --input-data data/perf_data_odd_mu200.json --concurrency-range 1:20:1 \\\n    --measurement-interval 10000 -u localhost:8001 -r 30 \\\n    \u003e data/nrp_inference_multi_gpu/out4gpu_1inst_take1.txt\n```\n\nFinally, to analyze the results, use the `multi_gpu_nautilus.ipynb` notebook. GPU performance metrics can be obtained through `Prometheus` which, at NRP, is accessible at `https://prometheus.nrp-nautilus.io`. All GPU metrics are prefixed with `nv_`. A few good metrics to analyzer are GPU utilization (`nv_gpu_utilization`), GPU power utilization (`nv_gpu_power_usage / nv_gpu_power_limit`), CPU utilization (`nv_cpu_utilization`), and the queue latency (`nv_inference_queue_duration_us / nv_inference_exec_count`). \n\n### !! Important !!\n\nDo not forget to uninstall the remote server when done testing:\n\n```\nhelm uninstall atlas-sonic -n atlas-sonic\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilescb%2Ftraccc-aas-performance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmilescb%2Ftraccc-aas-performance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilescb%2Ftraccc-aas-performance/lists"}