{"id":22574892,"url":"https://github.com/thoth-station/amun-api","last_synced_at":"2025-08-22T11:11:10.576Z","repository":{"id":33270627,"uuid":"150013302","full_name":"thoth-station/amun-api","owner":"thoth-station","description":"Thoth's execution engine for inspecting quality, performance, and usability of software and software stacks in a cluster.","archived":false,"fork":false,"pushed_at":"2023-10-03T03:05:29.000Z","size":6307,"stargazers_count":8,"open_issues_count":7,"forks_count":13,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-15T03:52:36.303Z","etag":null,"topics":["amun","artificial-intelligence","hacktoberfest","machine-learning","sofware-stacks","thoth"],"latest_commit_sha":null,"homepage":"https://thoth-station.ninja/","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thoth-station.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-23T18:29:53.000Z","updated_at":"2022-07-20T07:54:01.000Z","dependencies_parsed_at":"2025-02-02T15:41:25.100Z","dependency_job_id":null,"html_url":"https://github.com/thoth-station/amun-api","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/thoth-station/amun-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Famun-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Famun-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Famun-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Famun-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thoth-station","download_url":"https://codeload.github.com/thoth-station/amun-api/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Famun-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271628166,"owners_count":24792821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amun","artificial-intelligence","hacktoberfest","machine-learning","sofware-stacks","thoth"],"created_at":"2024-12-08T03:08:21.533Z","updated_at":"2025-08-22T11:11:10.553Z","avatar_url":"https://github.com/thoth-station.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"Amun Service\n------------\n\n.. image:: https://img.shields.io/github/v/tag/thoth-station/amun-api?style=plastic\n  :target: https://github.com/thoth-station/amun-api/releases\n  :alt: GitHub tag (latest by date)\n\n.. image:: https://quay.io/repository/thoth-station/amun-api/status\n  :target: https://quay.io/repository/thoth-station/amun-api?tab=tags\n  :alt: Quay - Build\n\nSee `this blog post for a detailed walkthrough together with a video\ndemonstrating usage \u003chttps://dev.to/fridex/how-to-beat-python-s-pip-inspecting-the-quality-of-machine-learning-software-1pkp\u003e`_.\n\nAmun is a service that executes the given application stack in the requested\nenvironment - given the list of package that should be installed as well as\ngiven the hardware that is requested to run the application. Its primary\npurpose is to act as an execution engine for Thoth where applications are\nbuilt and tested (applications are automatically generated given the software\nrequirements). However, it can be used to verify and check application behavior\nin a cluster.\n\nThere are performed 2 core steps by Amun:\n\n1. Assemble/build the given software by installing requested native packages\n   and/or Python packages into a container image. OpenShift's ImageStreams and\n   builds are used under the hood.\n\n2. Execute the given application using a script that is provided by a user -\n   there are run \"inspection jobs\" that execute provided user script. All the\n   information related to the node where the inspection job was run are\n   aggregated.  Such information consists of hardware available (such as CPU,\n   CPU flags and features, and such; see `amun-hwinfo\n   \u003chttps://github.com/thoth-station/amun-hwinfo\u003e`__) as well as information\n   from the kernel's process control block (such as number of context switches\n   performed, time spent in user/kernel space and such).\n\nThe second step is performed if the build succeeded and a user provided a script\nto test the application with in the given build environment on the requested\nhardware (there are used node selectors in the cluster for this purpose).\n\nThe actual second step is used to gather information whether the application\nruns with packages being installed in the build step as well as information\nsuch as performance characteristics or any other runtime-related information of\nthe assembled application.\n\nAs Amun accepts purely JSON on its input, the inspection step requires a test\nfile that is written to disk with an execute flag and run.\n\nAll the relevant logs from build and inspection job runs are aggregated and\nstored on Ceph together with actual results of inspections.\n\nSee `thoth-station/performance\n\u003chttps://github.com/thoth-station/performance\u003e`__ repository for an example of\na script that can be executed on Amun.\n\nOne can see Amun as a CI running in a cluster.\n\nRelated references\n==================\n\n* `thoth-station/performance \u003chttps://github.com/thoth-station/performance\u003e`__\n* `Dependency Monkey ZOO \u003chttps://github.com/thoth-station/dependency-monkey-zoo\u003e`__\n* `thoth-station/datasets \u003chttps://github.com/thoth-station/datasets/\u003e`__\n* `thoth-station/performance \u003chttps://github.com/thoth-station/performance/\u003e`__\n* `Kaggle dataset \u003chttps://www.kaggle.com/thothstation\u003e`__\n\nA request to Amun API\n=====================\n\nA single request to API is composed of:\n\n* an identifer of the inspection\n* a base image itself (e.g. ubi8)\n* a list of native packages (RPM or Deb packages) that should be installed into\n  the requested base image\n* a list of Python packages that should be installed into the requested base\n  image in a form of ``Pipfile``/``Pipfile.lock``\n* a script (bash, Python or any other scripting language - if the given\n  environment knows how to execute the script; if it has required interpreter)\n* hardware requirements for pod placement performing builds of application\n  stack (installing necessary dependencies)\n* hardware requirements for pod placement performing actual application\n  execution - \"inspection jobs\"\n\nSee provided OpenAPI/Swagger specification available in this Git repository.\nbase image is required parameter.\n\n.. figure:: https://raw.githubusercontent.com/thoth-station/amun-api/master/fig/api.gif\n   :alt: Amun API exposed supporting OpenAPI.\n   :align: center\n\nMonitoring builds and inspections\n=================================\n\nUpon a successful request to Amun API, a user obtains an ``inspection_id``.\nThis identifier is used to reference the given request. On the build endpoints\nthere are leveraged information about build status and the actual build logs,\non the job endpoints, there are leveraged information about the actual\ninspection runs - logs and logs printed to standard output and standard error\nstream. These results are obtained on a successful inspection run.\n\n.. figure:: https://raw.githubusercontent.com/thoth-station/amun-api/master/fig/diagram.png\n   :alt: Amun service architecture.\n   :align: center\n\nGathering Hardware Configuration\n================================\n\nEach time there is created a request with a script run (so there is actually\nspawned job responsible for running the provided script), there is run an\ninit container that gathers information about hardware that is present on\nnode where the application is run. This information is available in a form of\nJSON and becomes part of the actual result of an inspection run.\n\nThe Python script that gathers information about hardware present can be\nfound in\n`amun-hwinfo repository \u003chttps://github.com/thoth-station/amun-hwinfo\u003e`__.\n\nAn example scenario\n===================\n\nI, as an Amun user, would like to test performance of optimized TensorFlow\nbuilds available on the\n`AICoE Python package index \u003chttps://tensorflow.pypi.thoth-station.ninja\u003e`__.\nI would like to use:\n\n* TensorFlow provided on AICoE index (provide a  ``Pipfile`` and\n  ``Pipfile.lock`` respecting Pipenv configuration to use different package\n  indexes)\n* Python3, CUDA in specific version, .. - installed as RPMs\n* use a cluster node that exposes a GPU with CUDA support\n* I would like to use ubi8 as a base image\n* I don't need a node with GPU support to assemble/build the TensorFlow\n  application\n* I provide a Python script that is a TensorFlow application run to gather\n  information about TensorFlow (the application can print a JSON with results,\n  but can also push data to a remote API stated in the Python script itself).\n\nAmun guarantees the application is built as requested and it is placed on the\ncorrect node inside the cluster given the application requirements (a GPU with\nCUDA support).\n\nIf the build part fails, the script cannot be run. The build failures can be\nobserved on exposed build endpoints and are available on Ceph.\n\nREST API client\n===============\n\nTo comunicate with Amun API, use the autogenerated Swagger client that\navailable in the\n`amun-client repository \u003chttps://github.com/thoth-station/amun-client\u003e`__.\n\nResults adapters\n================\n\nLibrary called `thoth-storages \u003chttps://github.com/thoth-station/storages\u003e`__\nimplements `adapters that offer Python interface for accessing inspection files\n\u003chttps://thoth-station.ninja/docs/developers/storages/thoth.storages.html#module-thoth.storages.inspections\u003e`__.\n\nAn example of an inspection\n===========================\n\nAn example of an Amun inspection request can be found in this repo in examples\ndirectory. The structure corresponds to one inspection placed on Ceph with all\nthe relevant bits:\n\n.. code-block:: console\n\n   ── inspection-rhtf-conv2d-0f845f38   # inspection id\n      ├── build\n      │   ├── Dockerfile\n      │   ├── log\n      │   └── specification\n      └── results\n          ├── 0\n          │   ├── hwinfo\n          │   ├── log\n          │   └── result\n          ├── 1\n          │   ├── hwinfo\n          │   ├── log\n          │   └── result\n          └── 2\n              ├── hwinfo\n              ├── log\n              └── result\n\n* ``\u003cinspection-id\u003e/build/Dockerfile`` - automatically\n  generated Dockerfile used to build the application, this Dockerfile is a\n  transcription of the JSON input to Amun API\n\n* ``\u003cinspection-id\u003e/build/log`` - build log produced during the application\n  assembling\n\n* ``\u003cinspection-id\u003e/build/specification`` - whole input (toghether with\n  defaults) sent to Amun API endpoints that forms Amun inspection request\n\nAll results are indexed, index corresponds to one of the items in the\ninspection batch (multiple inspection job runs can be performed with each\nrequest so the application is built just once).\n\n* ``\u003cinspection-id\u003e/results/\u003cnum\u003e/hwinfo`` - information about hardware on which the\n  inspection job was run (see `thoth-station/amun-hwinfo\n  \u003chttps://github.com/thoth-station/amun-hwinfo\u003e`__)\n\n* ``\u003cinspection-id\u003e/results/\u003cnum\u003e/log`` - inspection run log -- standard error and\n  standard output as produced by the inspection script (or any\n  library/subprocess it uses).\n\n* ``\u003cinspection-id\u003e/results/\u003cnum\u003e/result`` - the actual result of an inspection run\n  together with process information from the kernel.\n\nAll the results are available on `Ceph\n\u003chttps://ceph.io/ceph-storage/object-storage/\u003e`__ or any object storage\nproviding AWS S3 compatible interface.\n\n.. figure:: https://raw.githubusercontent.com/thoth-station/amun-api/master/fig/ceph.gif\n   :alt: AWS S3 compatible interface for storing objects.\n   :align: center\n\nArgo UI\n=======\n\nIt's possible to observe how inspections proceed using Argo UI. Argo UI is\nexposed on deployment.\n\n.. figure:: https://raw.githubusercontent.com/thoth-station/amun-api/master/fig/argo_ui.gif\n   :alt: Argo UI showing inspections.\n   :align: center\n\nDeploying the application\n=========================\n\nAll manifests required to deploy this application are available in\n`thoth-station/thoth-application\n\u003chttps://github.com/thoth-station/thoth-application/tree/master/amun\u003e`__.\n\nSee `this video with instructions on how to deploy Amun into a cluster using\nkustomize \u003chttps://www.youtube.com/watch?v=rJ6Px2JPMqI\u003e`__.\n\nLicense \u0026 Copying\n=================\n\nThis software is released under the terms of GNU General Public License in\nversion 3.\n\n© Red Hat; AICoE team - Project Thoth\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Famun-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthoth-station%2Famun-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Famun-api/lists"}