{"id":19863619,"url":"https://github.com/sandialabs/vvtest","last_synced_at":"2025-08-12T16:38:15.514Z","repository":{"id":128628617,"uuid":"500963528","full_name":"sandialabs/vvtest","owner":"sandialabs","description":"A flexible test harness designed for scientific application development.","archived":false,"fork":false,"pushed_at":"2024-06-30T00:33:22.000Z","size":4415,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-02T04:35:10.228Z","etag":null,"topics":["hpc","python","scr-2277","snl-performance-workflow","testing-tools"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sandialabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-06-07T18:41:57.000Z","updated_at":"2024-10-02T17:37:41.000Z","dependencies_parsed_at":"2025-05-02T04:41:25.130Z","dependency_job_id":null,"html_url":"https://github.com/sandialabs/vvtest","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/sandialabs/vvtest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandialabs%2Fvvtest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandialabs%2Fvvtest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandialabs%2Fvvtest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandialabs%2Fvvtest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sandialabs","download_url":"https://codeload.github.com/sandialabs/vvtest/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sandialabs%2Fvvtest/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270098959,"owners_count":24527021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hpc","python","scr-2277","snl-performance-workflow","testing-tools"],"created_at":"2024-11-12T15:15:32.139Z","updated_at":"2025-08-12T16:38:15.475Z","avatar_url":"https://github.com/sandialabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vvtest\n\n**vvtest** is a test harness specialized for running regression, validation,\nand verification tests on high performance computing (HPC) platforms.\n\nAt a high level, **vvtest** just selects and runs scripts, called tests.\nIt is the test specification and management capabilities together with the\nplugin and control features that enable the execution of complex tests on\nworkstations up to large HPC platforms.\n\nNotable features include:\n* Pure Python, compatible with versions 2.7 and 3.x\n* Built-in CPU \u0026 GPU resource management and load balancing\n* Built-in support for HPC batch queuing systems, such as SLURM and LSF\n* Test parameterization, allowing a single script file to produce multiple\n  test instances with different parameter values\n* Inter-test dependencies - tests can depend on other tests. Dependencies\n  are run first and the results are available to the dependent test.\n* Test selection and execution control, such as by keyword, parameter,\n  platform, and a general option string\n* Tests can be any script language, but there is built-in support for Python\n  and (to a lesser extent) Bash\n* Configuration and plugins allow control over testing behavior\n* Tests are executed in a separate directory from the test scripts\n  (\"out of source\" execution)\n* Test result output formats include CDash, GitLab markdown, JUnit, and HTML.\n\n\n## Documentation\n\n* Run `vvtest -h` to show the options.\n* The RELEASE_NOTES file is kept up to date.\n* Additional documentation is available on the wiki.\n\n\n## Download and Install\n\nSince the only dependency is Python, just clone the\n**vvtest** repository and run the `vvtest` script. You can optionally run the\n`install_vvtest` script to install into a prefix of your choice.\n\n\n## An Example\n\nConsider a file called `mytest.vvt` with contents\n```\n# This is my cool test.\n#VVT: parameterize: np = 1 2\n\nimport vvtest_util as vvt\nimport shared_stuff as stuff\n\nprint ( 'Starting test {0} np={1}'.format( vvt.NAME, vvt.np ) )\n\nstuff.run_simulation( myoption='foo', numcpu=vvt.np )\nstuff.check_results()\n```\nThis test could be run as follows:\n```\n$ vvtest mytest.vvt\n==================================================\nTest list:\n    completed: 0\n    notrun: 2\n    total: 2\n\nPlatform ceelan, num procs = 16, max procs = 16\nStart time: Fri Jun  5 18:07:34 2020\nStarting: TestResults.ceelan/mytest.np=2\nStarting: TestResults.ceelan/mytest.np=1\nFinished: pass        0:01 06/05 18:07:34 TestResults.ceelan/mytest.np=2\nFinished: pass        0:01 06/05 18:07:34 TestResults.ceelan/mytest.np=1\nProgress: 2/2 = %100.0, time = 1s\n\n==================================================\nSummary:\n    completed: 2\n          2 pass\n    total: 2\n\nFinish date: Fri Jun  5 18:07:35 2020 (elapsed time 2s)\nTest directory: TestResults.ceelan\n```\nNote that the test specification file, `mytest.vvt`, resulted in two test\ninstances, `mytest.np=1` and `mytest.np=2`, which were run and both passed.\nA test can import the `vvtest_util.py` file, which is specific to each\ntest and contains information about the test instance and runtime parameters.\n\nThe test is run in a subdirectory to keep the output and artifacts separate.\n```\n$ ls -F TestResults.ceelan/mytest.np=2\nexecute.log  mytest.vvt@  vvtest_util.py  vvtest_util.pyc  vvtest_util.sh\n```\nThe output from the test script (both stdout and stderr) is contained in\nthe file `execute.log`.\n\n\n## History\n\n**vvtest** evolved from Mike Wong's test infrastructure in the late 1990s,\nthrough a python rewrite in the mid 2000s, to a refactoring in 2016 to\nmake it a project independent utility.\n\n\n## Copyright\n\nCopyright 2018 National Technology \u0026 Engineering Solutions of Sandia, LLC\n(NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the U.S.\nGovernment retains certain rights in this software. \n\n1. Redistributions of source code must retain the above copyright notice, this\n   list of conditions and the following disclaimer. \n\n2. Redistributions in binary form must reproduce the above copyright notice,\n   this list of conditions and the following disclaimer in the documentation\n   and/or other materials provided with the distribution. \n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\nFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\nDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\nOR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsandialabs%2Fvvtest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsandialabs%2Fvvtest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsandialabs%2Fvvtest/lists"}