{"id":37167271,"url":"https://github.com/stratum/testvectors-runner","last_synced_at":"2026-01-14T19:48:07.467Z","repository":{"id":113137536,"uuid":"210697278","full_name":"stratum/testvectors-runner","owner":"stratum","description":"Runner for TestVectors test cases","archived":false,"fork":false,"pushed_at":"2020-09-22T00:44:05.000Z","size":8658,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":15,"default_branch":"master","last_synced_at":"2023-04-06T11:55:59.848Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stratum.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-24T21:03:52.000Z","updated_at":"2024-06-19T06:23:14.085Z","dependencies_parsed_at":"2023-06-03T21:00:46.347Z","dependency_job_id":null,"html_url":"https://github.com/stratum/testvectors-runner","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/stratum/testvectors-runner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stratum%2Ftestvectors-runner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stratum%2Ftestvectors-runner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stratum%2Ftestvectors-runner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stratum%2Ftestvectors-runner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stratum","download_url":"https://codeload.github.com/stratum/testvectors-runner/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stratum%2Ftestvectors-runner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28432941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T18:57:19.464Z","status":"ssl_error","status_checked_at":"2026-01-14T18:52:48.501Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T19:48:06.740Z","updated_at":"2026-01-14T19:48:07.451Z","avatar_url":"https://github.com/stratum.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Test Vectors Runner\n\nThis project is a reference implementation of a Test Vector runner which executes [Test Vectors](https://github.com/stratum/testvectors) based tests for black-box testing of Stratum enabled switches.\n\nBuild status (master): [![CircleCI](https://circleci.com/gh/stratum/testvectors-runner.svg?style=svg)](https://circleci.com/gh/stratum/testvectors-runner)\n\n`testvectors-runner` works with various switch targets that expose P4Runtime and gNMI, including [Stratum switches](https://github.com/stratum/stratum). To get started, you'll need Switch Under Test (SUT) and set of corresponding Test Vectors.\n\n## Start a Stratum Switch\n\n\nTo start Stratum's behavioral model software switch (`stratum-bmv2`) in a Docker container for testing, run:\n```bash\nmake bmv2\n```\n\n\u003e Note: The `bmv2` container runs on the `host` network and creates two veth pairs on host machine which are used for testing data plane scenarios. \n\nTo start Stratum on hardware switches, including devices with Barefoot Tofino and Broadcom Tomahawk, visit the [Stratum Project repo](https://github.com/stratum/stratum) for details of how to get Stratum running on supported devices.\n\n## Get Test Vectors\n\nDownload Test Vector files matching your SUT (tofino/bcm/bmv2) from [Test Vectors repo](https://github.com/stratum/testvectors) or create your own Test Vectors.\n\n\u003e Another options is to download `Test Vector templates` which work with any switch platform together with a `template configuration file` which is used for rendering the templates and producing Test Vector files that match your SUT. See more details in the [Run with Test Vector Templates](#run-with-test-vector-templates) section below.\n\nIn addition to Test Vector files, a `target.pb.txt` file and a `portmap.pb.txt` file are mandatory for starting testvectors-runner. `target.pb.txt` stores the IP and port that your SUT is using, and `portmap.pb.txt` stores information related to specific switch ports used in the Test Vectors. Check [examples](https://github.com/stratum/testvectors/tree/master/tofino) in Test Vectors repo as well as the [readme](https://github.com/stratum/testvectors/blob/master/README.md) for more details.\n\n## Test with testvectors-runner\n\nFor running with hardware switches, testvectors-runner could be deployed on a server which has both gRPC and data plane connections to the SUT. For running with `stratum-bmv2`, testvectors-runner needs to be deployed on the same network where the bmv2 container is deployed.\n\nWhen loopback mode is enabled on hardware switches, it's also supported to deploy testvectors-runner directly on the switch. See the [loopback](#loopback-mode) section below for more details.\n\n### Use existing testvectors-runner binary docker image\n```bash\n./tvrunner.sh --target \u003cTARGET_FILE\u003e --portmap \u003cPORT_MAP_FILE\u003e --tv-dir \u003cTESTVECTORS_DIR\u003e\n```\nAbove command uses [tvrunner](https://hub.docker.com/repository/docker/stratumproject/tvrunner/general) binary docker image, executes testvectors from `tv-dir` on switch running on `target`. In addition to `--tv-dir` argument, you can also use `--tv-name \u003cTEST_NAME_REGEX\u003e` to run tests matching provided regular expression from `tv-dir`.\n\nFor example, assuming bmv2 container is deployed by `make bmv2` command and Test Vectors repo is downloaded to `~/testvectors`, first push a pipeline configuration to the bmv2 switch before running any tests:\n```bash\n./tvrunner.sh --target ~/testvectors/bmv2/target.pb.txt --portmap ~/testvectors/bmv2/portmap.pb.txt --tv-dir ~/testvectors/bmv2 --tv-name PipelineConfig\n```\n\nAbove command finds and executes Test Vector with name `PipelineConfig.pb.txt` under `~/testvectors/bmv2`. Then run `p4runtime` test suite by:\n```bash\n./tvrunner.sh --target ~/testvectors/bmv2/target.pb.txt --portmap ~/testvectors/bmv2/portmap.pb.txt --tv-dir ~/testvectors/bmv2/p4runtime\n```\n\n### Build and use local testvectors-runner binary docker image\nBuild testvectors-runner binary image locally using below command:\n```bash\ndocker build -t \u003cIMAGE_NAME\u003e -f build/test/Dockerfile .\n```\nRun tests with below command:\n```bash\n./tvrunner.sh --target \u003cTARGET_FILE\u003e --portmap \u003cPORT_MAP_FILE\u003e --tv-dir \u003cTESTVECTORS_DIR\u003e --image \u003cIMAGE_NAME\u003e\n```\n\nIn both cases, `tvrunner.sh` runs docker container in `host` network. To run docker container in another container's network, use below command:\n```bash\n./tvrunner.sh --target \u003cTARGET_FILE\u003e --portmap \u003cPORT_MAP_FILE\u003e --tv-dir \u003cTESTVECTORS_DIR\u003e --network \u003cNETWORK\u003e\n```\n\n\u003eNote: For more optional arguments, run *./tvrunner.sh -h*\n\n### Use go run command to run tests\n```bash\ngo run cmd/main/testvectors-runner.go --target \u003cTARGET_FILE\u003e --portmap \u003cPORT_MAP_FILE\u003e --tv-dir \u003cTESTVECTORS_DIR\u003e\n```\n\n### Build go binary, run tests\nBuild testvectors-runner go binary using below command:\n```bash\nmake build\n```\n\nUse the executed binary to run tests\n```bash\n./tvrunner --target \u003cTARGET_FILE\u003e --portmap \u003cPORT_MAP_FILE\u003e --tv-dir \u003cTESTVECTORS_DIR\u003e\n```\n\u003eNote: For more optional arguments, run *go run cmd/main/testvectors-runner.go -h* or *./tvrunner -h*\n\n### Loopback mode\n\nTo run tests in loopback mode just add `--dp-mode loopback` to the commands. It applies to all the options above. Take a Tofino switch as an example. First push a pipeline configuration by:\n```bash\n./tvrunner.sh --target ~/testvectors/tofino/target.pb.txt --portmap ~/testvectors/tofino/portmap.pb.txt --tv-dir ~/testvectors/tofino --tv-name PipelineConfig --dp-mode loopback\n```\n\nAs part of loopback mode setup, extra `Insert*` Test Vectors need to be executed before running any tests (see more details [here](docs/loopback.md)).\n```bash\n./tvrunner.sh --target ~/testvectors/tofino/target.pb.txt --portmap ~/testvectors/tofino/portmap.pb.txt --tv-dir ~/testvectors/tofino --tv-name Insert.* --dp-mode loopback\n```\n\nThen run `p4runtime` test suite by:\n```bash\n./tvrunner.sh --target ~/testvectors/tofino/target.pb.txt --portmap ~/testvectors/tofino/portmap.pb.txt --tv-dir ~/testvectors/tofino/p4runtime --dp-mode loopback\n```\n\nAfter all tests are done, run the `Delete*` Test Vectors to clean up.\n```bash\n./tvrunner.sh --target ~/testvectors/tofino/target.pb.txt --portmap ~/testvectors/tofino/portmap.pb.txt --tv-dir ~/testvectors/tofino --tv-name Delete.* --dp-mode loopback\n```\n\n### Run with Test Vector Templates\n\nTest Vector templates are tokenized Test Vector files and were created with the goal of maintaining a single set of tests that works across multiple switch platforms. As an alternative way of running Test Vectors, now it is also supported to run Test Vector templates together with a template configuration file (get more details in [Test Vectors repo](https://github.com/stratum/testvectors)) by pointing `--tv-dir` and `--tv-name` to the template file and using `--template-config` argument to specify the template configuration file, and all the other options above still apply:\n\n```bash\n./tvrunner.sh --target ~/testvectors/tofino/target.pb.txt --portmap ~/testvectors/tofino/portmap.pb.txt --tv-dir ~/testvectors/templates/p4runtime --tv-name L3ForwardTest --template-config ~/testvectors/tofino/template_config.json\n```\n\n## Additional Documents\n* [Test Vectors Runner Architecture](docs/architecture.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstratum%2Ftestvectors-runner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstratum%2Ftestvectors-runner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstratum%2Ftestvectors-runner/lists"}