{"id":26507486,"url":"https://github.com/milescb/traccc-aas","last_synced_at":"2026-04-12T17:49:54.465Z","repository":{"id":246215102,"uuid":"820337412","full_name":"milescb/traccc-aaS","owner":"milescb","description":"Custom backend to run traccc as-a-Service ","archived":false,"fork":false,"pushed_at":"2025-08-01T17:53:39.000Z","size":22613,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-01T19:34:58.333Z","etag":null,"topics":["as-a-service","tracking-algorithm"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/milescb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-06-26T09:16:33.000Z","updated_at":"2025-06-30T23:46:33.000Z","dependencies_parsed_at":"2024-10-25T00:39:25.892Z","dependency_job_id":"840901e0-ffa6-44b8-aa8e-45a0c37b62e9","html_url":"https://github.com/milescb/traccc-aaS","commit_stats":null,"previous_names":["milescb/traccc-aas"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/milescb/traccc-aaS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/milescb","download_url":"https://codeload.github.com/milescb/traccc-aaS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milescb%2Ftraccc-aaS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270796217,"owners_count":24647319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["as-a-service","tracking-algorithm"],"created_at":"2025-03-20T23:20:00.303Z","updated_at":"2026-04-12T17:49:54.439Z","avatar_url":"https://github.com/milescb.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Traccc as-a-Service\n\nMain objective of this repo: run [traccc](https://github.com/acts-project/traccc/tree/main) \nas-a-Service. Getting this working includes creating three main components:\n\n1. a shared library of `traccc` and writing a standalone version with the essential pieces of \n   the code included\n2. a custom backend using the standalone version above to launch the Triton server\n3. a client to send data to the server\n\nA minimal description of how to build a working version is detailed below. In each subdirectory of \nthis project, a README containing more information can be found. For instructions on running the \npipeline with an Athena client on `lxplus`, consult \n[this CodiMD page](https://codimd.web.cern.ch/1FcLmapORpeBtAVL_M6h4A?view). The last part of this \nREADME describes the necessary steps for maintaining and updating `traccc-aaS` when a new release \nof `traccc` is available. \n\n## Obtaining the `ITk` geometry files\n\nYou will need access to the `ITk` geometry files to run this repository, which can be found \nat `/eos/project/a/atlas-eftracking/GPU/ITk_data/ATLAS-P2-RUN4-03-00-01/` on CERN's `lxplus` \ncomputing cluster. Note that to access these, you must be a part of the e-group \n`atlas-tdaq-phase2-EFTracking-developers`. If you are outside of ATLAS, please refer to the \n`odd_legacy` branch to run the ODD version of the code.\n\n## Running out of the box\n\nThe easiest way to run `traccc` as-a-Service is with our container. Pull the image at \n`docker.io/milescb/traccc-aas:v1.4_traccc1.0.0` and run the image interactively. To do this, \nyou need access to the ITk geometry files, obtained by following the above instructions, and these \nneed to be mounted to `/traccc/itk-geometry`. \n\nThen, server can be launched with:\n\n```\ntritonserver --model-repository=$MODEL_REPO\n```\n\nTo test this with the client, open another window in the docker (using tmux, for instance), then run:\n\n```\ncd traccc-aas/client\npython TracccTritonClient.py\n```\n\n### Get the code\n\nSimply clone the repository with \n\n```\ngit clone --recurse-submodules git@github.com:milescb/traccc-aaS.git\n\n# checkout this branch\ncd traccc-aaS \u0026\u0026 git checkout itk-hits-to-tracks\n```\n\n### Docker\n\nThe easiest way to build the custom backend is with the docker at \n`docker.io/milescb/triton-server:25.02-py3_gcc13.3`. Run this interactively with\n\n```\nshifterimg pull milescb/triton-server:25.02-py3_gcc13.3\nshifter --module=gpu --image=milescb/triton-server:25.02-py3_gcc13.3\n```\n\nor use your favorite docker application and mount the appropriate directories. \n\n### Shared Library on `nersc`\n\nTo run out of the box at `nersc`, an installation of `traccc` and the the backend can be found at \n`/global/cfs/projectdirs/m3443/data/traccc-aaS/prod/ver_082625_g200/install`. To set up the \nenvironment, enter the image, then set the following environment variables\n\n```\nexport DATADIR=/global/cfs/projectdirs/m3443/data/traccc-aaS/data\nexport INSTALLDIR=/global/cfs/projectdirs/m3443/data/traccc-aaS/prod/ver_082625_g200/install\nexport PATH=$INSTALLDIR/bin:$PATH\nexport LD_LIBRARY_PATH=$INSTALLDIR/lib:$LD_LIBRARY_PATH\n```\n\nThen, the server can be launched with \n\n```\ntritonserver --model-repository=$INSTALLDIR/models\n```\n\nOnce the server is launched, run the model (on the same node to avoid networking problems) via:\n\n```\ncd client \u0026\u0026 python TracccTritionClient.py \n```\nMore info in the client directory. \n\n### Building from source\n\nIf you don't have access to `nersc`, you'll have to build `traccc` yourself. \nFollow the instructions on the [traccc page](https://github.com/acts-project/traccc/tree/main) to \nbuild or run this configure command:\n\n```\ncmake \u003cpath_to_cmake\u003e \\\n    -DCMAKE_BUILD_TYPE=Release \\\n    -DTRACCC_BUILD_CUDA=ON \\\n    -DTRACCC_BUILD_EXAMPLES=ON \\\n    -DTRACCC_USE_ROOT=FALSE \\\n    -DCMAKE_INSTALL_PREFIX=$INSTALLDIR\nmake -j20 install\n```\n\n## Building the backend\n\nFirst, enter the docker and set environment variables as documented above. Then run\n\n```\ncd backend/traccc-gpu \u0026\u0026 mkdir build install \u0026\u0026 cd build\ncmake -B . -S ../ \\\n    -DCMAKE_INSTALL_PREFIX=../install/ \\\n    -DCMAKE_BUILD_TYPE=Release\n\ncmake --build . --target install -- -j20\n```\n\nThe server can then be launched as above:\n\n```\ntritonserver --model-repository=../../models\n```\n\n## Deploy on K8s cluster using SuperSONIC\n\nFor server-side large-scale deployment we are using the \n[SuperSONIC](https://github.com/fastmachinelearning/SuperSONIC) framework. \n\n\n### To deploy the server on NRP Nautilus\n\n```\nsource deploy-nautilus-atlas.sh\n```\n\nThe settings are defined in `helm/values-nautilus-atlas.yaml` files. \nYou can update the setting simply by sourcing the deployment script again. \n \nYou can find the server URL in the same configs. It will take a few seconds to start a server, \ndepending on the specs of the GPUs requested.\n\n### Running the client\n\nIn order for the client to interface with the server, the location of the server needs to be \nspecified. First, ensure the server is running\n\n```\nkubectl get pods -n atlas-sonic\n```\nwhich has output something like:\n\n```\nNAME                            READY   STATUS    RESTARTS   AGE\nenvoy-atlas-7f6d99df88-667jd    1/1     Running   0          86m\ntriton-atlas-594f595dbf-n4sk7   1/1     Running   0          86m\n```\n\nor, use the [k9s](https://k9scli.io) tool to manage your pods. You can then check everything \nis healthy with\n\n```\ncurl -kv https://atlas.nrp-nautilus.io/v2/health/ready\n```\n\nwhich should produce somewhere in the output the lines:\n\n```\n\u003c HTTP/1.1 200 OK\n\u003c Content-Length: 0\n\u003c Content-Type: text/plain\n```\n\nThen, the client can be run with, for instance:\n\n```\npython TracccTritonClient.py -u atlas.nrp-nautilus.io --ssl\n```\n\nTo see what's going on from the server side, run\n\n```\nkubectl logs triton-atlas-594f595dbf-n4sk7\n```\n\nwhere `triton-atlas-594f595dbf-n4sk7` is the name of the server found when running the `get pods` \ncommand above. \n\n### !!! Important !!!\n\nMake sure to `uninstall` once the server is not needed anymore. \n\n```\nhelm uninstall atlas-sonic -n atlas-sonic\n```\n\nMake sure to read the [Policies](https://docs.nationalresearchplatform.org/userdocs/start/policies/) \nbefore using Nautilus. \n\n## Code Maintenance\n\nIdeally this repository should be up-to-date with the latest release of `traccc`. In a perfect \nworld where `traccc` has no major API changes, all that is required to keep this repository in sync \nis rebuilding `traccc` and then rebuilding the backend with the commands outlined above. However, \nwhen major API changes are introduced, this is not possible and parts of the code may need to be \nedited and refactored. \n\n### Updating `traccc`\n\nThe first step in updating this repository is to pull the most recent changes of `traccc`, and then \nbuild using the commands outlined above. From here, you should first attempt to build the standalone \nwrapper in the `standalone` repository. \n\n### Updating standalone wrapper\n\nIf this doesn't work, compare `standalone/src/TracccGpuStandalone.hpp` to the examples in \n`traccc/examples/run/cuda/`. In particular, `seq_example_cuda.cpp` is a good place to start for a \ntemplate of how running the current version of `traccc` looks like. For proper variable \ninitialization in the class `TracccGpuStandalone`, both `full_chain_algorithm.hpp` and the \ncorresponding implementation file `full_chain_algorithm.cpp` are quite helpful. This step can take \ntime and requires at least a cursory understanding of how `traccc` algorithms are scheduled. \n\nIn the worst case, the output tracking containers may be changed. In this case, a more thorough \ninvestigation of the code may be necessary. First, find out what new output container(s) look like \nby browsing `traccc/core/include/traccc/edm` for something like `track_state_collection` or \n`track_fit_collection`. Then, identify how to access all the parameters necessary to execute the \nprint statements in `standalone/src/TracccGpuStandalone.cpp`. \n\nOnce the standalone compiles and runs, you can move on to the next step!\n\n### Updating backend\n\nIf you did not have to make any changes to `standalone/src/TracccGpuStandalone.cpp` but only to the \nheader file, then you should be able to build as before. If you did find yourself in the worst-case \nscenario above, then you will need to make similar changes to `backend/traccc-gpu/traccc.cc:676-836` \nas before to properly deal with output parsing. \n\nWhen complete, test by building and running the simple python client as detailed above. \n\n### Building production image\n\nOnce all the above steps are completed, you can build a container with your new developments by \nfirst creating a merge request with `main` of this repository (or change the tag of tracc-aaS in \n`backend/Dockerfile`), and then build an image from the `backend` repository with:\n\n```\npodman-hpc build -t name:tag .\n```\nor use your favorite docker application. \n\nIf you would like to build a container without making a MR, edit `backend/Dockerfile:22` with your \nfork of this repository. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilescb%2Ftraccc-aas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmilescb%2Ftraccc-aas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilescb%2Ftraccc-aas/lists"}