{"id":19132461,"url":"https://github.com/taichi-dev/taichi-release-tests","last_synced_at":"2025-05-06T18:12:13.096Z","repository":{"id":42133430,"uuid":"507867697","full_name":"taichi-dev/taichi-release-tests","owner":"taichi-dev","description":null,"archived":false,"fork":false,"pushed_at":"2023-06-21T02:45:41.000Z","size":20127,"stargazers_count":2,"open_issues_count":1,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-19T13:44:36.783Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taichi-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-06-27T10:44:21.000Z","updated_at":"2023-01-20T17:35:24.000Z","dependencies_parsed_at":"2024-11-09T06:28:30.371Z","dependency_job_id":null,"html_url":"https://github.com/taichi-dev/taichi-release-tests","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taichi-dev%2Ftaichi-release-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taichi-dev%2Ftaichi-release-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taichi-dev%2Ftaichi-release-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taichi-dev%2Ftaichi-release-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taichi-dev","download_url":"https://codeload.github.com/taichi-dev/taichi-release-tests/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252741458,"owners_count":21797027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T06:18:21.557Z","updated_at":"2025-05-06T18:12:13.054Z","avatar_url":"https://github.com/taichi-dev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# taichi-release-tests\n\nThis repo contains scripts for running examples in taichi main repo, DiffTaichi, QuanTaichi, and more.\nThis is part of the testing process to ensure real world applications behave as expected.\n\n\n## How to run\n\n1. Dependencies\n\nThe only dependency is `PyYAML`:\n\n```\npip install PyYAML\n```\n\n2. Clone / symlink relevant repos to the `repos` directory\n\n```\nln -sf /path/to/taichi repos/taichi\n```\n\n3. Run examples with configured timelines\n\n```\n# Run\npython3 run.py --log=DEBUG timelines/\n\n# Run 3 instances simultaneously\npython3 run.py --log=DEBUG --runners 3 timelines/\n\n# Regenerate captures\npython3 run.py --log=DEBUG --generate-captures timelines/\n```\n\n\n## Conventions\n\n1. One or multiple yaml file per example in `timelines` directory, but not one yaml file for multiple examples.\n2. Put repos being tested in `repos` directory`\n3. Put captures in `truths` directory. \n\n\n## How to add your own test\n\nUsing the `waterwave.py` in taichi main repo as an example:\n\n### Prerequisites\n\n1. Activate a virtualenv with a usable taichi installation.\n2. Ensure you have followed instruction 1 \u0026 2 in `How to run` section.\n\n### Generate interaction records\n\n```bash\n# Run waterwave.py, and write recorded timeline to waterwave.yaml\n$ python record.py repos/taichi/python/taichi/examples/simulation/waterwave.py waterwave.yaml\n... Click click ...\n\n$ cat waterwave.yaml\n```\n\n```yaml\n---\n- path: repos/taichi/python/taichi/examples/simulation/waterwave.py\n  args: []\n  steps:\n  - {frame: 1, action: move, position: [0.574, 0.568]}\n  - {frame: 0, action: move, position: [0.572, 0.574]}\n  - {frame: 0, action: mouse-down, key: LMB}\n  - {frame: 0, action: move, position: [0.568, 0.592]}\n  - {frame: 0, action: move, position: [0.566, 0.605]}\n  - {frame: 0, action: mouse-up, key: LMB}\n  - {frame: 0, action: move, position: [0.564, 0.609]}\n  - {frame: 1, action: move, position: [0.424, 0.646]}\n  - {frame: 0, action: mouse-down, key: LMB}\n  - {frame: 0, action: move, position: [0.406, 0.646]}\n  - {frame: 8, action: key-down, key: Super_L}\n  - {frame: 30, action: succeed}\n```\n\n### Manually tune generated yaml\n\n`record.py` can only record keyboard and mouse events, more advanced `steps` can be added manually.\n\n#### Step format\n\n````yaml\n- frame: 30     # Common and mandatory arg, frame delay since last action\n  action: move  # Common and mandatory arg, what to do when target frame is arrived.\n  position:     # Specific arg for action `move`. In this example, \n  - 0.424\n  - 0.646\n````\n\n#### Currently available actions\n\n##### succeed \u0026 fail\n\n````yaml\n- frame: 30\n  action: succeed\n````\n\n````yaml\n- frame: 30\n  action: fail\n````\n\nSelf explanatory\n\n##### (key|mouse)-(up|down) \u0026 key-press \u0026 mouse-click \u0026 move\n\nThese are keyboard \u0026 mouse events.\nGenerally you don't pay attention to them, these can be generated by `record.py` utility.\n\n\n##### capture-and-compare\n\n```yaml\n- frame: 5\n  action: capture-and-compare\n  compare: sum-difference\n  threshold: 1000\n  ground_truth: truths/taichi/simulation/fractal.png\n```\n\n\nCaptures image in GUI window, compare it to a specified ground truth.\n\n`compare` can have these methods for now:\n\n| compare             | Description                                                                      |\n| ------------------- | -------------------------------------------------------------------------------- |\n| sum-difference      | Calc `sum(abs(a[i] - b[i]) for i in \u003cevery-pixel\u003e)`                              |\n| blur-sum-difference | Apply gaussian blur to both capture and ground truth, calculate `sum-difference` |\n| pixel-count         | Count every different pixel                                                      |\n| rmse                | Calc `sqrt(sum((a[i] - b[i])**2 for i in \u003cevery-pixel\u003e) / \u003ctotal-pixel-count\u003e)`  |\n\n`threshold` can specify a percentage(as a string, like `\"0.01%\"`).\n`ground_truth` is a path to png file, resides in `truths` directory.\n\nExample: [fractal.yaml](timelines/taichi/simulation/fractal.yaml)\n\n\n##### poke\n```yaml\n- frame: 30\n  action: poke\n  function: main\n  code: |\n    # Python code\n```\n\nAt target (graphical) frame, run given python `code` in specified `function`'s (stack) frame.\nSpecified python code can freely access local \u0026 global variables in target `function`.\n\nIf you want to target module level frame, use `function: \"\u003cmodule\u003e\"`.\n\nThis is used for poking adjustable parameters of target program,\nsince this runner cannot interact with GUI components.\n\nExample: [diff_sph.yaml](timelines/taichi/autodiff/diff_sph/diff_sph.yaml)\n\n\n### Integration with `taichi` CI\n\nFor now `taichi` CI will run this test for every PR and master merge.\n\nIf you added a test for a different repo, you should also modify `taichi` CI script,\nclone your target repo to `repos` directory.\n\n[*nix Test Script](https://github.com/taichi-dev/taichi/blob/master/.github/workflows/scripts/unix_test.sh#L65)\n\n[Windows Test Script](https://github.com/taichi-dev/taichi/blob/master/.github/workflows/scripts/win_test.ps1#L30)\n\n\n### Known limitations\n\n1. This runner only applies to python programs.\n2. Simulated interactions cannot interact with GUI components (IMGUI), please use `poke` to change parameter value.\n3. A lot of programs are not numerically stable, so you can't naively capture-and-compare. If you insist, try using fp64 instead of fp32.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaichi-dev%2Ftaichi-release-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaichi-dev%2Ftaichi-release-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaichi-dev%2Ftaichi-release-tests/lists"}