{"id":21343521,"url":"https://github.com/cloudmercato/ai-benchmark","last_synced_at":"2025-07-12T15:31:27.370Z","repository":{"id":116894605,"uuid":"318691814","full_name":"cloudmercato/ai-benchmark","owner":"cloudmercato","description":null,"archived":false,"fork":false,"pushed_at":"2023-11-27T19:57:42.000Z","size":20917,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2023-11-27T21:29:12.658Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudmercato.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-12-05T03:21:55.000Z","updated_at":"2023-11-24T22:18:42.000Z","dependencies_parsed_at":"2023-11-24T23:33:30.313Z","dependency_job_id":null,"html_url":"https://github.com/cloudmercato/ai-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fai-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fai-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fai-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fai-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudmercato","download_url":"https://codeload.github.com/cloudmercato/ai-benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225824685,"owners_count":17529906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T01:13:37.466Z","updated_at":"2024-11-22T01:13:38.061Z","avatar_url":"https://github.com/cloudmercato.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[AI Benchmark Alpha](http://ai-benchmark.com/alpha) is an open source python library for evaluating AI performance of various hardware platforms, including CPUs, GPUs and TPUs. The benchmark is relying on [TensorFlow](https://www.tensorflow.org) machine learning library, and is providing a lightweight and accurate solution for assessing inference and training speed for key Deep Learning models.\u003c/br\u003e\u003c/br\u003e\r\n\r\nIn total, AI Benchmark consists of \u003cb\u003e42 tests\u003c/b\u003e and \u003cb\u003e19 sections\u003c/b\u003e provided below:\u003c/br\u003e\r\n\r\n1. MobileNet-V2\u0026nbsp; `[classification]`\r\n2. Inception-V3\u0026nbsp; `[classification]`\r\n3. Inception-V4\u0026nbsp; `[classification]`\r\n4. Inception-ResNet-V2\u0026nbsp; `[classification]`\r\n5. ResNet-V2-50\u0026nbsp; `[classification]`\r\n6. ResNet-V2-152\u0026nbsp; `[classification]`\r\n7. VGG-16\u0026nbsp; `[classification]`\r\n8. SRCNN 9-5-5\u0026nbsp; `[image-to-image mapping]`\r\n9. VGG-19\u0026nbsp; `[image-to-image mapping]`\r\n10. ResNet-SRGAN\u0026nbsp; `[image-to-image mapping]`\r\n11. ResNet-DPED\u0026nbsp; `[image-to-image mapping]`\r\n12. U-Net\u0026nbsp; `[image-to-image mapping]`\r\n13. Nvidia-SPADE\u0026nbsp; `[image-to-image mapping]`\r\n14. ICNet\u0026nbsp; `[image segmentation]`\r\n15. PSPNet\u0026nbsp; `[image segmentation]`\r\n16. DeepLab\u0026nbsp; `[image segmentation]`\r\n17. Pixel-RNN\u0026nbsp; `[inpainting]`\r\n18. LSTM\u0026nbsp; `[sentence sentiment analysis]`\r\n19. GNMT\u0026nbsp; `[text translation]`\r\n\r\nFor more information and results, please visit the project website: [http://ai-benchmark.com/alpha](http://ai-benchmark.com/alpha)\u003c/br\u003e\u003c/br\u003e\r\n\r\n#### Installation Instructions \u003c/br\u003e\r\n\r\nThe benchmark requires TensorFlow machine learning library to be present in your system.\r\n\r\nOn systems that \u003cb\u003edo not have Nvidia GPUs\u003c/b\u003e, run the following commands to install AI Benchmark:\r\n\r\n```bash\r\npip install tensorflow\r\npip install ai-benchmark\r\n```\r\n\u003c/br\u003e\r\n\r\nIf you want to check the \u003cb\u003eperformance of Nvidia graphic cards\u003c/b\u003e, run the following commands:\r\n\r\n```bash\r\npip install tensorflow-gpu\r\npip install ai-benchmark\r\n```\r\n\r\n\u003cb\u003e`Note 1:`\u003c/b\u003e If Tensorflow is already installed in your system, you can skip the first command.\r\n\r\n\u003cb\u003e`Note 2:`\u003c/b\u003e For running the benchmark on Nvidia GPUs, \u003cb\u003e`NVIDIA CUDA`\u003c/b\u003e and \u003cb\u003e`cuDNN`\u003c/b\u003e libraries should be installed first. Please find detailed instructions [here](https://www.tensorflow.org/install/gpu). \u003c/br\u003e\u003c/br\u003e\r\n\r\n#### Getting Started \u003c/br\u003e\r\n\r\nTo run AI Benchmark, use the following code:\r\n\r\n```bash\r\nfrom ai_benchmark import AIBenchmark\r\nbenchmark = AIBenchmark()\r\nresults = benchmark.run()\r\n```\r\n\r\nAlternatively, on Linux systems you can type `ai-benchmark` in the command line to start the tests.\r\n\r\nTo run inference or training only, use `benchmark.run_inference()` or `benchmark.run_training()`. \u003c/br\u003e\u003c/br\u003e\r\n\r\n#### Advanced settings \u003c/br\u003e\r\n\r\n```bash\r\nAIBenchmark(use_CPU=None, verbose_level=1):\r\n```\r\n\u003e use_CPU=`{True, False, None}`:\u0026nbsp;\u0026nbsp; whether to run the tests on CPUs\u0026nbsp; (if tensorflow-gpu is installed)\r\n\r\n\u003e verbose_level=`{0, 1, 2, 3}`:\u0026nbsp;\u0026nbsp; run tests silently | with short summary | with information about each run | with TF logs\r\n\r\n```bash\r\nbenchmark.run(precision=\"normal\"):\r\n```\r\n\r\n\u003e precision=`{\"normal\", \"high\"}`:\u0026nbsp;\u0026nbsp; if `high` is selected, the benchmark will execute 10 times more runs for each test.\r\n\r\n\u003c/br\u003e\r\n\r\n### Additional Notes and Requirements \u003c/br\u003e\r\n\r\nGPU with at least 2GB of RAM is required for running inference tests / 4GB of RAM for training tests.\r\n\r\nThe benchmark is compatible with both `TensorFlow 1.x` and `2.x` versions. \u003c/br\u003e\u003c/br\u003e\r\n\r\n### Contacts \u003c/br\u003e\r\n\r\nPlease contact `andrey@vision.ee.ethz.ch` for any feedback or information.\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudmercato%2Fai-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudmercato%2Fai-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudmercato%2Fai-benchmark/lists"}