{"id":26151728,"url":"https://github.com/popey/measure-syft","last_synced_at":"2025-09-05T20:38:28.502Z","repository":{"id":276314659,"uuid":"928908840","full_name":"popey/measure-syft","owner":"popey","description":"A tool to measure and compare the performance of different versions of Syft.","archived":false,"fork":false,"pushed_at":"2025-02-07T15:24:32.000Z","size":18,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-10T16:19:43.799Z","etag":null,"topics":["performance-testing","python-script","sbom","syft"],"latest_commit_sha":null,"homepage":"https://github.com/anchore/syft","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/popey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-07T13:10:25.000Z","updated_at":"2025-03-06T02:54:05.000Z","dependencies_parsed_at":"2025-02-07T14:23:36.222Z","dependency_job_id":"12f3bfa1-bc11-4a17-bb4e-64ef6ca77d90","html_url":"https://github.com/popey/measure-syft","commit_stats":null,"previous_names":["popey/measure-syft"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popey%2Fmeasure-syft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popey%2Fmeasure-syft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popey%2Fmeasure-syft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popey%2Fmeasure-syft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/popey","download_url":"https://codeload.github.com/popey/measure-syft/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242987799,"owners_count":20217534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["performance-testing","python-script","sbom","syft"],"created_at":"2025-03-11T06:39:16.733Z","updated_at":"2025-03-11T06:39:17.321Z","avatar_url":"https://github.com/popey.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# measure-syft\n\nA tool to measure and compare the performance of different versions of [Syft](https://github.com/anchore/syft).\n\n## Introduction\n\nWhen making changes to Syft, particularly those that might affect performance, it's useful to measure the impact across different versions. This tool automates the process of:\n\n1. Fetching the latest release of Syft\n2. Building it from source\n3. Running performance tests\n4. Testing subsequent commits to main\n5. Comparing performance between branches (when testing PRs)\n\nThe tool generates a detailed Markdown report showing performance metrics for each tested version.\n\n## Installation\n\nmeasure-syft is written in Python and requires a few dependencies.\n\n### Pre-requisites\n\n* Python 3.x\n* Git\n* Go (for building Syft)\n* Docker or Podman (for running container tests)\n\nYou can use either [uv](https://github.com/astral-sh/uv) or Python's built-in venv to set up the environment:\n\nUsing uv:\n```shell\ngit clone https://github.com/popey/measure-syft\ncd measure-syft\nuv venv\nsource ./venv/bin/activate\nuv pip install requests\n```\n\nUsing venv:\n```shell\ngit clone https://github.com/popey/measure-syft\ncd measure-syft\npython -m venv venv\nsource ./venv/bin/activate\npip install requests\n```\n\n## Usage\n\nThere are two main ways to use measure-syft:\n\n### 1. Testing commits after latest release\n\nTo test all commits from the latest release to main:\n\n```shell\n./measure-syft.py\n```\n\nThis will:\n- Find the latest Syft release\n- Clone and build that version\n- Run performance tests\n- Test each subsequent commit up to main\n- Generate a report in the `results` directory\n\n### 2. Testing a specific PR\n\nTo compare main against a PR branch:\n\n```shell\n./measure-syft.py --pr feat/parallelize-file-hashing\n```\n\nThis will:\n- Build and test main\n- Build and test the specified PR branch\n- Generate a comparison report\n\n## Configuration\n\nThe script uses several configuration variables that can be modified in the source:\n\n* `test_container`: The container image to use for testing\n* `iterations`: Number of test runs per version (default: 5)\n* `build_dir`: Where to clone and build Syft\n* `results_dir`: Where to store test results\n* `platform`: Container platform to test against\n\n## Output\n\nThe script generates a Markdown report containing:\n\n* Test date and time\n* Container being tested\n* Environment variables\n* Table of results showing:\n  * Version/commit\n  * Minimum runtime\n  * Maximum runtime\n  * Average runtime\n\nExample output:\n```markdown\n# Syft Performance Test Results\n\nDate: 2024-02-07 10:00:00\nContainer: docker.io/huggingface/transformers-all-latest-torch-nightly-gpu:latest\nEnvironment Variables:\n- SYFT_PARALLELISM=48\n- SYFT_CHECK_FOR_APP_UPDATE=false\n\n## Results\n| Version/Description | Commit | Min (s) | Max (s) | Avg (s) |\n|-------------------|--------|---------|---------|---------|\n| v1.19.0 | - | 45.23 | 47.12 | 46.18 |\n| Add parallel... | [abc123](https://...) | 42.11 | 43.89 | 43.00 |\n```\n\n## Caveats\n\nThis tool is primarily designed for performance testing and comparison. The results can be affected by system load and other factors, so it's recommended to:\n\n* Run tests multiple times\n* Keep the test environment as consistent as possible\n* Consider the min/max/average values rather than individual run times\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpopey%2Fmeasure-syft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpopey%2Fmeasure-syft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpopey%2Fmeasure-syft/lists"}