{"id":32175245,"url":"https://github.com/openmdao/testflo","last_synced_at":"2025-10-21T19:31:19.227Z","repository":{"id":43874666,"uuid":"65558138","full_name":"OpenMDAO/testflo","owner":"OpenMDAO","description":"A simple python testing framework that can run unit tests under MPI (or not).","archived":false,"fork":true,"pushed_at":"2025-08-25T12:12:18.000Z","size":508,"stargazers_count":5,"open_issues_count":8,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-09-21T16:26:02.359Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"naylor-b/testflo","license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenMDAO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-08-12T14:11:05.000Z","updated_at":"2025-08-25T12:12:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"22109f46-21e5-4754-95aa-3265264e83f1","html_url":"https://github.com/OpenMDAO/testflo","commit_stats":{"total_commits":284,"total_committers":12,"mean_commits":"23.666666666666668","dds":"0.35915492957746475","last_synced_commit":"0de8948998b41c7a8152d783cf2d6b6137475696"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/OpenMDAO/testflo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMDAO%2Ftestflo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMDAO%2Ftestflo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMDAO%2Ftestflo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMDAO%2Ftestflo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenMDAO","download_url":"https://codeload.github.com/OpenMDAO/testflo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMDAO%2Ftestflo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280322624,"owners_count":26311103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-21T19:30:20.683Z","updated_at":"2025-10-21T19:31:19.218Z","avatar_url":"https://github.com/OpenMDAO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI version][1]][2]\n\ntestflo\n=======\n\ntestflo is a python testing framework that uses a pipeline of\niterators to process test specifications, run the tests, and process the\nresults.\n\nWhy write another testing framework?\n------------------------------------\n\ntestflo was written to support testing of the OpenMDAO framework.\nSome OpenMDAO features require execution under MPI while some others don't,\nso we wanted a testing framework that could run all of our tests in the same\nway and would allow us to build all of our tests using unittest.TestCase\nobjects that we were already familiar with.  The MPI testing functionality\nwas originally implemented using the nose testing framework.  It worked, but\nwas always buggy, and the size and complexity of the nose framework made it\ndifficult to know exactly what was going on.\n\nEnter testflo, an attempt to build a simpler testing framework that would have\nthe basic functionality of other test frameworks, with the additional\nability to run MPI unit tests that are very similar to regular unit tests.\n\n\nSome testflo features\n---------------------\n\n*    MPI unit testing\n*    *pre_announce* option to print test name before running in order to\n     quickly identify hanging MPI tests\n*    concurrent testing  (on by default, use '-n 1' to turn it off)\n*    test coverage\n*    flexible execution - can be given a directory, a file, a module path,\n     *file:testcase.method*, *module:testcase.method*, or a file containing\n     a list of any of the above. Has options to generate test list files\n     containing all failed tests or all tests that execute within a certain\n     time limit.\n*    end of testing summary\n\n\nUsage\n-----\n\nFor a full list of testflo options, execute the following:\n\n`testflo -h`\n\n\nNOTE: Because testflo runs tests concurrently by default, your tests must be\nwritten with concurrency in mind or they may fail.  For example, if multiple\ntests write output to a file with the same name, you have to make sure that those\ntests are executed in different directories to prevent that file from being\ncorrupted.  If your tests are not written to run concurrently, you can always\njust run them with `testflo -n 1` and run them in serial instead.\n\nThe following is an example of what an MPI unit test looks like.  To tell\ntestflo that a TestCase is an MPI TestCase, you add a class attribute\ncalled N_PROCS to it and set it to the number of MPI processes to use for the\ntest.  That's all there is to it. Of course, depending on what sort of MPI code\nyou're testing, it's up to you to potentially test for different things on\ndifferent ranks.\n\n\n```python\n\nclass MyMPI_TestCase(TestCase):\n\n    N_PROCS = 4  # this is how many MPI processes to use for this TestCase.\n\n    def test_foo(self):\n\n        # do your MPI testing here, e.g.,\n\n        if self.comm.rank == 0:\n            # some test only valid on rank 0...\n\n\n```\n\n\nHere's an example of testflo output for openmdao.core:\n\n\n```\n\nopenmdao$ testflo openmdao.core\n............................................................................\n............................................................................\n............................................................................\n..............................\n\nOK\n\nPassed:  258\nFailed:  0\nSkipped: 0\n\n\nRan 258 tests using 8 processes\nWall clock time:   00:00:1.82\n\n```\n\nRunning testflo in verbose mode on openmdao.core.test.test_problem is shown\nbelow. The verbose output contains the full test name as well as the elapsed\ntime and memory usage.\n\n\n```\n\nopenmdao$ testflo openmdao.core.test.test_problem -v\nopenmdao.core.test.test_problem:TestCheckSetup.test_pbo_messages ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_check_promotes ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_conflicting_connections ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_conflicting_promoted_state_vars ... OK (00:00:0.00, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_conflicting_promotions ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestCheckSetup.test_out_of_order ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_explicit_connection_errors ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_find_subsystem ... OK (00:00:0.00, 69 MB)\nopenmdao.core.test.test_problem:TestCheckSetup.test_cycle ... OK (00:00:0.06, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_input_input_explicit_conns_no_conn ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_illegal_desvar ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_input_input_explicit_conns_w_conn ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_check_connections ... OK (00:00:0.06, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_mode_auto ... OK (00:00:0.03, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_check_parallel_derivs ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_simplest_run ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_basic_run ... OK (00:00:0.03, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_change_solver_after_setup ... OK (00:00:0.04, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_no_vecs ... OK (00:00:0.08, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_src_idx_gt_src_size ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_src_idx_neg ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_simplest_run_w_promote ... OK (00:00:0.02, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_unconnected_param_access ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_variable_access_before_setup ... OK (00:00:0.00, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_scalar_sizes ... OK (00:00:0.07, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_byobj_run ... OK (00:00:0.01, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_error_change_after_setup ... OK (00:00:0.31, 70 MB)\nopenmdao.core.test.test_problem:TestProblem.test_unconnected_param_access_with_promotes ... OK (00:00:0.04, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_variable_access ... OK (00:00:0.06, 69 MB)\nopenmdao.core.test.test_problem:TestProblem.test_iprint ... OK (00:00:0.25, 73 MB)\n\n\nOK\n\nPassed:  30\nFailed:  0\nSkipped: 0\n\n\nRan 30 tests using 8 processes\nWall clock time:   00:00:1.17\n\n```\n\nOperating Systems and Python Versions\n-------------------------------------\n\ntestflo is used to test OpenMDAO as part of its CI process,\nso we run it nearly every day on linux, Windows and OS X. It requires\npython 3.5 or higher.\n\n\nYou can install testflo directly from github using the following command:\n\n`pip install git+https://github.com/OpenMDAO/testflo.git`\n\n\nor install from PYPI using:\n\n\n`pip install testflo`\n\n\n\nIf you try it out and find any problems, submit them as issues on github at\nhttps://github.com/OpenMDAO/testflo.\n\n[1]: https://badge.fury.io/py/testflo.svg \"PyPI Version\"\n[2]: https://badge.fury.io/py/testflo \"testflo @PyPI\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenmdao%2Ftestflo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenmdao%2Ftestflo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenmdao%2Ftestflo/lists"}