{"id":14987992,"url":"https://github.com/apache/cassandra-dtest","last_synced_at":"2025-04-12T22:18:48.250Z","repository":{"id":22717973,"uuid":"95281387","full_name":"apache/cassandra-dtest","owner":"apache","description":"Distributed tests for Apache Cassandra®","archived":false,"fork":false,"pushed_at":"2025-04-10T17:28:14.000Z","size":20819,"stargazers_count":54,"open_issues_count":47,"forks_count":134,"subscribers_count":25,"default_branch":"trunk","last_synced_at":"2025-04-12T22:18:07.328Z","etag":null,"topics":["cassandra","database","java"],"latest_commit_sha":null,"homepage":"https://cassandra.apache.org/","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"auditlog_test.py","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-24T07:00:07.000Z","updated_at":"2025-04-10T17:28:20.000Z","dependencies_parsed_at":"2023-09-27T16:54:01.303Z","dependency_job_id":"d6fcb9bd-dd86-491d-a7d9-e12c36483482","html_url":"https://github.com/apache/cassandra-dtest","commit_stats":{"total_commits":4350,"total_committers":157,"mean_commits":27.70700636942675,"dds":0.8370114942528736,"last_synced_commit":"022398a9a7a1861f5948a024cf0ddf4cf7682aff"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcassandra-dtest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcassandra-dtest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcassandra-dtest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcassandra-dtest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/cassandra-dtest/tar.gz/refs/heads/trunk","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248637966,"owners_count":21137579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cassandra","database","java"],"created_at":"2024-09-24T14:15:55.271Z","updated_at":"2025-04-12T22:18:48.206Z","avatar_url":"https://github.com/apache.png","language":"Python","readme":"Cassandra Distributed Tests (DTests)\n====================================\n\nCassandra Distributed Tests (or better known as \"DTests\") are a set of Python-based \ntests for [Apache Cassandra](http://apache.cassandra.org) clusters. DTests aim to \ntest functionality that requires multiple Cassandra instances. Functionality that\nof code that can be tested in isolation should ideally be a unit test (which can be\nfound in the actual Cassandra repository). \n\nSetup and Prerequisites\n------------\n\nSome environmental setup is required before you can start running DTests.\n\n### Native Dependencies\nDTests requires the following native dependencies:\n * Python 3\n * PIP for Python 3 \n * libev\n * git\n * JDK 8 (Java)\n \n#### Linux\n1. ``apt-get install git-core python3 python3-pip python3-dev libev4 libev-dev``\n2. (Optional - solves warning: \"jemalloc shared library could not be preloaded to speed up memory allocations\"): \n``apt-get install -y --no-install-recommends libjemalloc1``\n\n#### Mac\nOn Mac, the easiest path is to install the latest [Xcode and Command Line Utilities](https://developer.apple.com) to \nbootstrap your development environment and then use [Homebrew](https://brew.sh)\n\n1. (Optional) Make sure brew is in a good state on your system ``brew doctor``\n2. ``brew install python3 libev``\n\n### Python Dependencies\nThere are multiple external Python dependencies required to run DTests. \nThe current Python dependency list is maintained in a file named\n[requirements.txt](https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt)\nin the root of the `cassandra-dtest` repository.\n\nThe easiest way to install these dependencies is with pip and virtualenv. \n\n**Note**: While virtualenv isn't strictly required, using virtualenv is almost always the quickest \npath to success as it provides common base setup across various configurations.\n\n1. Install virtualenv: ``pip install virtualenv``\n2. Create a new virtualenv: ``virtualenv --python=python3 --no-site-packages ~/dtest``\n3. Switch/Activate the new virtualenv: ``source ~/dtest/bin/activate``\n4. Install remaining DTest Python dependencies: ``pip install -r /path/to/cassandra-dtest/requirements.txt``\n\n\nUsage\n-----\n\nThe tests are executed by the pytest framework which includes a helpful [Usage and\nInvocations](https://docs.pytest.org/en/latest/usage.html) document which is a great place to start\nfor basic invocation options when using pytest.\n\nAt minimum, \n\n  The only thing the framework needs to know is\nthe location of the (compiled (hint: ``ant clean jar``)) sources for Cassandra. There are two options:\n\nUse existing sources:\n\n    pytest --cassandra-dir=~/path/to/cassandra\n\nUse ccm ability to download/compile released sources from archives.apache.org:\n\n    pytest --cassandra-version=1.0.0\n\nTo run a specific test file, class or individual test, you only have to \npass its path as an argument:\n\n    pytest --cassandra-dir=~/path/to/cassandra pending_range_test.py\n    pytest --cassandra-dir=~/path/to/cassandra pending_range_test.py::TestPendingRangeMovements\n    pytest --cassandra-dir=~/path/to/cassandra pending_range_test.py::TestPendingRangeMovements::test_pending_range\n    \nWhen adding a new test or modifying an existing one, it's always a good idea to\nrun it several times to make sure it is stable. This can be easily done with \nthe ``--count`` option. For example, to run a test class 10 times:\n\n    pytest --count=10 --cassandra-dir=~/path/to/cassandra pending_range_test.py\n\nExisting tests are probably the best place to start to look at how to write\ntests.\n\nEach test spawns a new fresh cluster and tears it down after the test. If a\ntest fails, the logs for the node are saved in a `logs/\u003ctimestamp\u003e` directory\nfor analysis (it's not perfect but has been good enough so far, I'm open to\nbetter suggestions).\n\nIn case you want to run tests using your own CCM branch, please, refer to the comments in requirements.txt for details how to do that.\n\nWriting Tests\n-------------\n\nIf you'd like to know what to expect during a code review, please see the included [CONTRIBUTING file](CONTRIBUTING.md).\n\nDebugging Tests\n-------------\nSome general tips for debugging dtest/pytest tests\n\n#### pytest.set_trace()\nIf there is an unexpected value being asserted on and you'd like to inspect the state of all the tests variables just before a paricular assert, add ``pytest.set_trace()`` right before the problematic code. The next time you execute the test, when that line of code is reached pytest will drop you into an interactive python debugger (pdb). From there you can use standard python options to inspect various methods and variables for debugging.\n\n#### Hung tests/hung pytest framework\nDebugging hung tests can be very difficult but thanks to improvements in Python 3 it's now pretty painless to get a python thread dump of all the threads currently running in the pytest process.\n\n```python\nimport faulthandler\nfaulthandler.enable()\n```\nAdding the above code will install a signal handler into your process. When the process recieves a *SIGABRT* signal, python will dump python thread dumps for all running threads in the process. DTests installs this by default with the install_debugging_signal_handler fixture.\n\nThe following is an example of what you might see if you send a *SIGABRT* signal to the pytest process while in a hung state during the test teardown phase after the successful completion of the actual dtest.\n\n```bash\n(env) cassandra-dtest vcooluser$ kill -SIGABRT 24142\n\nFatal Python error: Aborted\n\nThread 0x000070000f739000 (most recent call first):\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 295 in wait\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 551 in wait\n  File \"/Users/mkjellman/src/cassandra-dtest/tools/data.py\", line 31 in query_c1c2\n  File \"/Users/mkjellman/src/cassandra-dtest/bootstrap_test.py\", line 91 in \u003clambda\u003e\n  File \"/Users/mkjellman/src/cassandra-dtest/dtest.py\", line 245 in run\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 916 in _bootstrap_inner\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 884 in _bootstrap\n\nThread 0x000070000e32d000 (most recent call first):\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncore.py\", line 183 in poll2\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncore.py\", line 207 in loop\n  File \"/Users/mkjellman/env3/src/cassandra-driver/cassandra/io/asyncorereactor.py\", line 119 in loop\n  File \"/Users/mkjellman/env3/src/cassandra-driver/cassandra/io/asyncorereactor.py\", line 258 in _run_loop\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 864 in run\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 916 in _bootstrap_inner\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 884 in _bootstrap\n\nCurrent thread 0x00007fffa00dd340 (most recent call first):\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 1072 in _wait_for_tstate_lock\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py\", line 1056 in join\n  File \"/Users/mkjellman/src/cassandra-dtest/dtest.py\", line 253 in stop\n  File \"/Users/mkjellman/src/cassandra-dtest/dtest.py\", line 580 in tearDown\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py\", line 608 in run\n  File \"/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py\", line 653 in __call__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/unittest.py\", line 174 in runtest\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/runner.py\", line 107 in pytest_runtest_call\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/callers.py\", line 180 in _multicall\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 216 in \u003clambda\u003e\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 222 in _hookexec\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 617 in __call__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py\", line 273 in \u003clambda\u003e\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/runner.py\", line 191 in __init__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py\", line 274 in call_runtest_hook\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py\", line 118 in call_and_report\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/runner.py\", line 77 in runtestprotocol\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/runner.py\", line 63 in pytest_runtest_protocol\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py\", line 81 in pytest_runtest_protocol\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/callers.py\", line 180 in _multicall\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 216 in \u003clambda\u003e\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 222 in _hookexec\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 617 in __call__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/main.py\", line 164 in pytest_runtestloop\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/callers.py\", line 180 in _multicall\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 216 in \u003clambda\u003e\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 222 in _hookexec\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 617 in __call__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/main.py\", line 141 in _main\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/main.py\", line 103 in wrap_session\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/main.py\", line 134 in pytest_cmdline_main\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/callers.py\", line 180 in _multicall\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 216 in \u003clambda\u003e\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 222 in _hookexec\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/pluggy/__init__.py\", line 617 in __call__\n  File \"/Users/mkjellman/env3/lib/python3.6/site-packages/_pytest/config.py\", line 59 in main\n  File \"/Users/mkjellman/env3/bin/pytest\", line 11 in \u003cmodule\u003e\nAbort trap: 6\n```\n\n#### Debugging Issues with Fixtures and Test Setup/Teardown\npytest can appear to be doing \"magic\" more often than not. One place it may be hard to follow what actual code will get executed by normal code inspection alone is determining which fixtures will run for a given test and in what order. pytest provides a ``--setup-plan`` command line argument. When pytest is invoked with this argument it will print a execution plan including all fixtures and tests that actually running the test will invoke. The below is an example for the current execution plan pytest generates for dtest *auth_test.py::TestAuthRoles::test_create_drop_role*\n\n```bash\n(env3) Michaels-MacBook-Pro:cassandra-dtest mkjellman$ pytest --cassandra-dir=/Users/mkjellman/src/mkjellman-oss-github-cassandra-trunk auth_test.py::TestAuthRoles::test_create_drop_role --setup-plan\n====================================================================== test session starts ======================================================================\nplatform darwin -- Python 3.6.3, pytest-3.3.0, py-1.5.2, pluggy-0.6.0\nrootdir: /Users/mkjellman/src/cassandra-dtest, inifile: pytest.ini\nplugins: timeout-1.2.1, raisesregexp-2.1, nose2pytest-1.0.8, flaky-3.4.0\ncollected 1 item                                                                                                                                                \n\nauth_test.py \nSETUP    S install_debugging_signal_handler\n    SETUP    C fixture_logging_setup\n      SETUP    F fixture_dtest_setup_overrides\n      SETUP    F fixture_log_test_name_and_date\n      SETUP    F fixture_maybe_skip_tests_requiring_novnodes\n      SETUP    F parse_dtest_config\n      SETUP    F fixture_dtest_setup (fixtures used: fixture_dtest_setup_overrides, fixture_logging_setup, parse_dtest_config)\n      SETUP    F fixture_since (fixtures used: fixture_dtest_setup)\n      SETUP    F fixture_dtest_config (fixtures used: fixture_logging_setup)\n      SETUP    F set_dtest_setup_on_function (fixtures used: fixture_dtest_config, fixture_dtest_setup)\n        auth_test.py::TestAuthRoles::()::test_create_drop_role (fixtures used: fixture_dtest_config, fixture_dtest_setup, fixture_dtest_setup_overrides, fixture_log_test_name_and_date, fixture_logging_setup, fixture_maybe_skip_tests_requiring_novnodes, fixture_since, install_debugging_signal_handler, parse_dtest_config, set_dtest_setup_on_function)\n      TEARDOWN F set_dtest_setup_on_function\n      TEARDOWN F fixture_dtest_config\n      TEARDOWN F fixture_since\n      TEARDOWN F fixture_dtest_setup\n      TEARDOWN F parse_dtest_config\n      TEARDOWN F fixture_maybe_skip_tests_requiring_novnodes\n      TEARDOWN F fixture_log_test_name_and_date\n      TEARDOWN F fixture_dtest_setup_overrides\n    TEARDOWN C fixture_logging_setup\nTEARDOWN S install_debugging_signal_handler\n===Flaky Test Report===\n\n\n===End Flaky Test Report===\n\n====================================================================== 0 tests deselected =======================================================================\n================================================================= no tests ran in 0.12 seconds ==================================================================\n```\n\n#### Instances Failing to Start (Unclean Test Teardown)\nGetting into a state (especially while writing new tests or debugging problamatic ones) where pytest/dtest fails to fully tear-down all local C* instancse that were started. You can use this handy one liner to kill all C* instances in one go:\n\n```bash\nps aux | grep -ie CassandraDaemon | grep java | awk '{print $2}' | xargs kill\n```\n\nLinks\n-------\n * [ccm](https://github.com/apache/cassandra-ccm)\n * [pytest](https://docs.pytest.org/)\n * [Python Driver](http://datastax.github.io/python-driver/installation.html)\n * [CQL over Thrift Driver](http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fcassandra-dtest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Fcassandra-dtest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fcassandra-dtest/lists"}