{"id":20001165,"url":"https://github.com/apecloud/foxlake-tests","last_synced_at":"2026-03-19T13:02:02.588Z","repository":{"id":210860848,"uuid":"727618713","full_name":"apecloud/foxlake-tests","owner":"apecloud","description":"The test cases for FoxLake, run by sqllogictest.","archived":false,"fork":false,"pushed_at":"2024-03-14T10:37:36.000Z","size":210,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-01-30T17:44:46.097Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apecloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-05T08:25:38.000Z","updated_at":"2023-12-05T08:25:38.000Z","dependencies_parsed_at":"2024-03-14T07:34:07.695Z","dependency_job_id":"6071e0f3-e9ff-42cb-aef8-1c2f36bb3f2d","html_url":"https://github.com/apecloud/foxlake-tests","commit_stats":null,"previous_names":["apecloud/foxlake-tests"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/apecloud/foxlake-tests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apecloud%2Ffoxlake-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apecloud%2Ffoxlake-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apecloud%2Ffoxlake-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apecloud%2Ffoxlake-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apecloud","download_url":"https://codeload.github.com/apecloud/foxlake-tests/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apecloud%2Ffoxlake-tests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29198178,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T14:35:27.868Z","status":"ssl_error","status_checked_at":"2026-02-07T14:25:51.081Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T05:16:54.267Z","updated_at":"2026-02-07T15:32:16.007Z","avatar_url":"https://github.com/apecloud.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Test suite of this project\nThis test suite contains following cases:\n- SQL tests. These cases aim to test the SQL handling of the project. It contains DDL, DML, DQL, etc.\n- CDC tests. These cases test the feature 'Zero-ETL' of the project.\n- Performance tests. These cases are based on TPC-H.\n\n# How to run the test suite\nYou can either run the tests by the Go program on the source project or by the test tool docker image.\n\n## Prepare the data for testing\nThe data files used by the test cases lay in directory `tests/cicd`. Please copy them all to your bucket in cloud storage like S3 or local storage like MinIO. Do not try to modify the directory structure of the data files. Because the test cases are based on the directory structure.\n\nMoreover, if you want to test correctness of the TPC-H queries. Please generate the TPC-H data files by yourself. The instructions of generating TPC-H data files is showed [here](foxlake-tpch-test/README.md). After generating the data files, you can run the test cases in tests/foxlake-tpch-sf1-test/tpch/ directory. It's worth noting that the test results in the cases are based on TPC-H SF1.\n\n\n## Running Golang program directly\nIf you wish to debug this test tool, clone the tool project and run on `logictest/mysql/main/main.go`\n```bash\n...\nInstall Golang yourself\n...\n\ngit clone git@github.com:apecloud/sqllogictest.git\ncd go/\ngo run logictest/mysql/main/main.go \\\n  -dsn username:password@(localhost:7060)/foxlake \\\n  -kv STORAGE_PROVIDER=s3c \\\n  -kv BUCKET=foxlake \\\n  -kv ENDPOINT=127.0.0.1:19000 \\\n  -kv REGION=cn-northwest-1 \\\n  -kv STORAGE_PATH=e2e-testing/for-test/ \\\n  -secret ACCESS_KEY_ID=ABCDEFGHIJK \\\n  -secret SECRET_ACCESS_KEY=abcdefghijklmnopqrstuvwxyz \\\n  verify /Path/to/FoxLake/tests/foxlake-sql-test/DQL/show.test\n```\n\nTo get the usage information of this tool, run the `main.go` without any parameter.\n```\ngo run logictest/mysql/main/main.go\nUsage: ./main [options] verify|generate|filter|analyze \u003ctest files...\u003e\noptions:\n  -dsn string\n        The DSN for MySQL. (default \"root:password@tcp(127.0.0.1:3306)/testdb\")\n  -env-file string\n        Optional. Path to the env file. File content should be kv pairs like key=value.\n  -init-db\n        Optional. Initialize the database by dropping all tables and views. If not set, the harness will not drop tables or views.\n  -kv value\n        Optional. Environment variable key-value pair (e.g. -kv key=value). Thoes key names with format {{.KEY_NAME}} in test files will be set as environment variables.\n  -one-value-per-line\n        Optional. Whether to use the old fashioned result format. i.e. One value per line format. Default: false\n  -parallelism int\n        Optional. The number of tests to run in parallel. This parameter should be greater than 0. (default 1)\n  -secret value\n        Optional. Secret key-value pair. Like '-kv', those key names with format {{.SECRET_KEY_NAME}} in test files will be set as secret variables. But secrets will not be shown in the console output. (e.g. -secret ACCESS_KEY_ID=abc).\n```\n\n## Running on docker image\nFirstly, you should pull the image from Docker hub. This docker image is based on this project [apecloud/sqllogictest](https://github.com/apecloud/sqllogictest).\n```bash\ndocker pull apecloud/sqllogictest:main\n```\nThen, you can run this image by docker command with parameters like running the Go program.\n```bash\ndocker run --rm \\\n  -v /Path/to/FoxLake/tests/:/home/admin/tests apecloud/sqllogictest:main \\\n  '-dsn' 'username:password@(host.docker.internal:7060)/foxlake' \\\n  '-kv' STORAGE_PROVIDER=s3c \\\n  '-kv' BUCKET=foxlake \\\n  '-kv' ENDPOINT=127.0.0.1:19000 \\\n  '-kv' STORAGE_PATH=e2e-testing/for-test/ \\\n  '-kv' REGION=cn-northwest-1 \\\n  '-secret' ACCESS_KEY_ID=ABCDEFGHIJK \\\n  '-secret' SECRET_ACCESS_KEY=abcdefghijklmnopqrstuvwxyz \\\n  verify tests/foxlake-sql-test/DQL/show.test\n```\nAlso, you can run the image without any parameter following to get the usage information.\n```bash\ndocker run --rm apecloud/sqllogictest:main\nUsage: ./main [options] verify|generate|filter|analyze \u003ctest files...\u003e\noptions:\n  -dsn string\n        The DSN for MySQL. (default \"root:password@tcp(127.0.0.1:3306)/testdb\")\n  -env-file string\n        Optional. Path to the env file. File content should be kv pairs like key=value.\n  -init-db\n        Optional. Initialize the database by dropping all tables and views. If not set, the harness will not drop tables or views.\n  -kv value\n        Optional. Environment variable key-value pair (e.g. -kv key=value). Thoes key names with format {{.KEY_NAME}} in test files will be set as environment variables.\n  -one-value-per-line\n        Optional. Whether to use the old fashioned result format. i.e. One value per line format. Default: false\n  -parallelism int\n        Optional. The number of tests to run in parallel. This parameter should be greater than 0. (default 1)\n  -secret value\n        Optional. Secret key-value pair. Like '-kv', those key names with format {{.SECRET_KEY_NAME}} in test files will be set as secret variables. But secrets will not be shown in the console output. (e.g. -secret ACCESS_KEY_ID=abc).\n```\nPlease note that you should mount the tests directory to the container. The tests directory is the directory of this project. And the container will run the tests under this directory. The mount directory should be the absolute path, and the mount directory in the container should be `/home/admin/tests`. The `/home/admin/tests` is an empty directory in the container.\n\n\n## Running multiple test cases sequentially or in parallel\nThis tool can accept a list of test files or directories. If you specify a directory, then any file with suffix `.test` and `.test-parallelly` under this directory and the sub-directoies will be collected. e.g. If you write this command:\n```bash\ndocker run --rm \\\n  -v /Path/to/FoxLake/tests/:/home/admin/tests \\\n  apecloud/sqllogictest:main \\\n  '-dsn' 'username:password@(host.docker.internal:7060)/foxlake' \\\n  '-parallelism' 4 \\\n  '-kv' STORAGE_PROVIDER=s3c \\\n  '-kv' BUCKET=foxlake \\\n  '-kv' ENDPOINT=127.0.0.1:19000 \\\n  '-kv' STORAGE_PATH=e2e-testing/for-test/ \\\n  '-kv' REGION=cn-northwest-1 \\\n  '-secret' ACCESS_KEY_ID=ABCDEFGHIJK \\\n  '-secret' SECRET_ACCESS_KEY=abcdefghijklmnopqrstuvwxyz \\\n  verify tests/foxlake-sql-test/ tests/foxlake-cdc-test/\n```\nThen all the test files with suffix `.test` and `.test-parallelly` under the directories `tests/foxlake-sql-test/` and `tests/foxlake-cdc-test/`, and the sub-directoies of them will be run. Those test files with suffix `.test` will be run sequentially by a single goroutine, and those test files with suffix `.test-parallelly` will be run in parallel by multiple goroutines. The number of all goroutines is specified by the parameter `-parallelism`. If you don't specify the `-parallelism` parameter, the default value is 1, which means all files will be run sequentially.\n\n\n## Parameters of this tool\nThe description of options can be found above. And please note that the options must be specified before the `verify|generate|filter|analyze`.\nThe meaning of `verify|generate|filter|analyze`:\n- verify: Runs the test files given, outputting a pass / fail line to STDOUT for each test record. All arguments after\n  this are interpreted as test files or directories, which contain tests to be run. For directory arguments,\n  directories are descended recursively, and all files with the .test extension will be added to the list of tests.\n\n- generate: Runs tests as verify does, but also produces a new version of each test file, named $testfile.generated,\n  with the results of this test run.\n\n- filter: Runs the tests and produces a new version of each test file, just like generate, but any tests that\n  fail are filtered out and not included in the generated files. This mode is useful when validating a new batch of\n  fuzzed statements against a test oracle to filter out statements that don't execute correctly.\n\n- analyze: Analyzes all test statements in the specified test files and prints out a usage count for various statement\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapecloud%2Ffoxlake-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapecloud%2Ffoxlake-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapecloud%2Ffoxlake-tests/lists"}