{"id":18654532,"url":"https://github.com/z-shell/vats","last_synced_at":"2026-03-04T08:31:10.228Z","repository":{"id":103554053,"uuid":"425919547","full_name":"z-shell/VATS","owner":"z-shell","description":"Valgrind automatic test suite (VATS) runs a sequence of tests, preceded by a Valgrind call, with a command and arguments taken from file vtest.conf","archived":false,"fork":false,"pushed_at":"2023-12-01T09:05:37.000Z","size":27,"stargazers_count":5,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-02T13:56:00.282Z","etag":null,"topics":["automatic-tests","valgrind","vats","z-shell"],"latest_commit_sha":null,"homepage":"https://z.digitalclouds.dev","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/z-shell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-11-08T16:50:51.000Z","updated_at":"2022-06-04T08:40:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"9a6e789a-3d56-4fb3-be98-1c1ad36bead2","html_url":"https://github.com/z-shell/VATS","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/z-shell/VATS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2FVATS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2FVATS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2FVATS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2FVATS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/z-shell","download_url":"https://codeload.github.com/z-shell/VATS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2FVATS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30076849,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T08:01:56.766Z","status":"ssl_error","status_checked_at":"2026-03-04T08:00:42.919Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-tests","valgrind","vats","z-shell"],"created_at":"2024-11-07T07:15:41.072Z","updated_at":"2026-03-04T08:31:10.186Z","avatar_url":"https://github.com/z-shell.png","language":"Shell","readme":"\u003ctable style=\"width:100%;height:auto\"\u003e\n\u003ctr\u003e\u003ctd align=\"center\"\u003e\n\u003ca title=\"ZI\" target=\"_self\" href=\"https://github.com/z-shell/zi/\"\u003e\n\u003ch2\u003e\u003cimg align=\"center\" style=\"width:60px;height:auto\" src=\"https://github.com/z-shell/zi/raw/main/docs/images/logo.svg\" alt=\"ZI Logo\" /\u003e\u003c/a\u003e\n❮ ZI ❯ - VATS \u003c/h2\u003e\u003ch3\u003e Valgrind automatic test suite \u003c/h3\u003e\n\n  \u003ca href=\"https://asciinema.org/a/488213\" target=\"_blank\"\u003e\u003cimg style=\"width:90%;height:auto\" src=\"https://asciinema.org/a/488213.svg\" /\u003e\u003c/a\u003e\n\n\u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n \nRuns sequence of tests, preceded by `valgrind` call, with a command and arguments taken from file `vtest.conf`.\nThe arguments are dynamic because the `vtest.conf` entry can look like this:\n\n```SystemVerilog\n# Arguments passed to $test_bin, evaluated at use (i.e. for each test separately)\ntest_bin_args='+Z -f $ZTST_srcdir/ztst.zsh $file'   # runs ztst.zsh on given $file\ntest_bin=\"local-zsh\"                                # expands to ../Src/zsh\n```\n\nThe variable `$file` is set to current test-file for each test-run. For Zsh-integrated valgrind tests, `$file` is just passed as argument to `ztst.zsh`. It comes from the outer loop in `vruntests.zsh`. So it is `A01grammar.ztst`, for example.\n\n### Error Definitions\n\nYou can define errors so that they are skipped from test result (i.e. from Valgrind output). This is\nthe main feature of VATS because it allows to quickly check if changes broke anything. Usage is very\nmuch like of unit tests – run `make TESTNUM=A01`, look for any red color, done. Zero parsing with\neyes. A typical definition for Zshell can look like this:\n\n```zsh\nerrors1+=( \"* / zsh_main / setupvals / gettimeofday / *\" )\n```\n\nand is placed in `__error1.def` or other such file with index.\n\n### Integrating With Project\n\nFollowing patch applied to the Test/Makefile solves the integration of VATS to Zsh:\n\n```diff\n──────────────────────────────────────────────────────────────────────────────────────────\nmodified: Test/Makefile.in\n──────────────────────────────────────────────────────────────────────────────────────────\n@@ -35,6 +35,7 @@ VPATH           = @srcdir@\n sdir            = @srcdir@\n sdir_top        = @top_srcdir@\n INSTALL         = @INSTALL@\n+VLGRND                 = $(VALGRIND:1=v)\n\n @DEFS_MK@\n\n@@ -49,7 +50,7 @@ check test:\n            do echo $$f; done`\" \\\n         ZTST_srcdir=\"$(sdir)\" \\\n         ZTST_exe=$(dir_top)/Src/zsh@EXEEXT@ \\\n-        $(dir_top)/Src/zsh@EXEEXT@ +Z -f $(sdir)/runtests.zsh; then \\\n+        $(dir_top)/Src/zsh@EXEEXT@ +Z -f $(sdir)/$(VLGRND)runtests.zsh; then \\\n         stat=0; \\\n        else \\\n         stat=1; \\\n```\n\n### Basic Test-Configuration\n\nThe configuration-file of tests is `vtest.conf`. It defines two settings:\n\n```zsh\ntest_bin=\"../Src/cgiturl\"   # Binary that runs any test (is the tested program itself)\nzsh_control_bin=\"zsh\"       # Binary used when scheduling tests \u0026 interpreting Valgrind output\n```\n\nVariable `zsh_control_bin` is used to implement special `#!` behavior: `runtests.zsh`\nstarts with `#!/bin/sh`, reads `vtest.conf`, and restarts with `$zsh_control_bin`. This way\nuser can define shebang interpreter via separate configuration file (`vtest.conf`).\n\n### Remaining Test-Configuration\n\nThe setting `tkind` is used to set a **test-kind**. These are modes of Valgrind operation.\nAllowed values are: `error` (only detect read/write errors), `leak` (also detect memory leaks),\n`nopossiblylost` (detect memory leaks, but not _possibly_ lost blocks).\n\n```zsh\ntest_kind=\"leak\"             # Test kind: error, (leak|full - the same meaning), nopossiblylost\n```\n\n`Valgrind` messages of type `summary` and `info` are muted via lines:\n\n```zsh\nsummaries=\"no\"               # show valgrind summaries?\ninfo=\"no\"                    # show info messages?\n```\n\n### The vtest.conf pasted\n\n```zsh# -*- Mode: sh; indent-tabs-mode: nil; -*-\n# vim:ft=zsh:sw=4:sts=4:et\n\n# Main\ntest_bin=\"local-zsh\"         # Binary that runs any test (local-zsh: ../Src/zsh)\nzsh_control_bin=\"zsh\"        # Binary used when scheduling tests \u0026 interpreting Valgrind output\nmain_operation_parsing=\"yes\" # The main feature, whether to use it (i.e. whether to parse valgrind's output)\nvalgrind_path=\"valgrind\"     # Find the valgrind binary to run through $PATH\n\n# Valgrind run modes\ntest_kind=\"leak\"             # Test kind: error, (leak|full - the same meaning), nopossiblylost\ntrack_origins=\"no\"           # Whether to pass --track-origns=yes to valgrind binary\n\n# Gating of Valgrind's and other output\nsummaries=\"no\"               # Show valgrind summaries?\ninfo=\"no\"                    # Show info messages?\ntest_desc=\"yes\"              # Whether to print Zsh test-description during operation\n\n# DEBUG, especially interested_in=\"a-function\" helps to check why e.g. your\n# error definition that targets `a-function' doesn't match\nmdebug=\"no\"                  # Match-debug, use to debug error definition matching\ninterested_in=\"\"             # As if mdebug=1 (i.e. debugging of why an error-def matches\n                             # or not), but only active if: stack trace and error definition\n                             # contain $interested_in\n#interested_in=\"*mk*temp\"    # Example value, will match error def:\n                             # \"* / (_|)mk(s|)temp / * / find_temp_path\"\n#interested_in=\"(*locale*|*fork*|*printf*)\" # Other example value - debugging multiple error-defs at once\n\n# Arguments passed to $test_bin (i.e. ../Src/zsh most of the time), evaluated\n# at use (i.e. getting the value of $file will be done at each loop cycle).\ntest_bin_args='+Z -f $ZTST_srcdir/ztst.zsh $file'\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fz-shell%2Fvats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fz-shell%2Fvats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fz-shell%2Fvats/lists"}