{"id":26481986,"url":"https://github.com/kiwitcms/open-source-test-management","last_synced_at":"2026-02-24T09:02:02.473Z","repository":{"id":140261229,"uuid":"291726818","full_name":"kiwitcms/open-source-test-management","owner":"kiwitcms","description":"Guidelines for organizing testing [with Kiwi TCMS] aimed at other open source projects!","archived":false,"fork":false,"pushed_at":"2020-08-31T14:56:09.000Z","size":347,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-17T06:56:58.914Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://kiwitcms.org","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiwitcms.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"custom":["https://kiwitcms.org/#subscriptions"],"github":["kiwitcms"],"patreon":null,"open_collective":"kiwitcms","ko_fi":null,"tidelift":"pypi/kiwitcms","community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null}},"created_at":"2020-08-31T13:44:50.000Z","updated_at":"2024-06-09T19:43:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"dc745ff7-ee68-461e-85d0-959301342ea3","html_url":"https://github.com/kiwitcms/open-source-test-management","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kiwitcms/open-source-test-management","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiwitcms%2Fopen-source-test-management","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiwitcms%2Fopen-source-test-management/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiwitcms%2Fopen-source-test-management/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiwitcms%2Fopen-source-test-management/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiwitcms","download_url":"https://codeload.github.com/kiwitcms/open-source-test-management/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiwitcms%2Fopen-source-test-management/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29777606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T04:54:30.205Z","status":"ssl_error","status_checked_at":"2026-02-24T04:53:58.628Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-20T03:36:34.850Z","updated_at":"2026-02-24T09:02:02.453Z","avatar_url":"https://github.com/kiwitcms.png","language":null,"funding_links":["https://kiwitcms.org/#subscriptions","https://github.com/sponsors/kiwitcms","https://opencollective.com/kiwitcms","https://tidelift.com/funding/github/pypi/kiwitcms"],"categories":[],"sub_categories":[],"readme":"# What is this\n\nMultiple open source projects have come to us and said\n**We like Kiwi TCMS and we need a system like that but we have no idea\nhow to get started**. The reality is that the majority of these projects\nrely on contributors and most of them would prefer to be developers and\nto contribute code! Testing regularly remains on the side.\n\nThis repository contains guidelines for organizing testing and is targeted\nprimarily towards other open source projects. We'll try to keep it brief\nand actionable so you can get started immediately.\n\n**Notes:**\n\n- The word \"testing\" is overloaded with meaning in our industry. Sometimes it\n  means checking something, other times if means to verify, yet other times\n  it means to experiment. Here it is used as a catch-all term so you will have\n  to be aware of the context in which you want to apply these guidelines!\n\n\n# Evaluate your risk areas\n\nTesting is a process for collecting information and filling a knowledge gap\nthat we have, see\n[this illustration](https://raw.githubusercontent.com/atodorov/qa-automation-ruby-101/master/module00/testing_knowledge_gap.png).\nIn the general case of an established software project we should start\nwith an evaluation and find out which areas pose more risk than others:\n\n- Could be UI\n- Could be translations\n- Could be installation\n- Could be specific set of functionality\n\nTo quickly start evaluating these areas:\n\n- Talk to your users \u0026 developers - they always know specific things\n  which tend not to work often\n- Read through your bug tracker, check all reports for the last few versions\n- Talk to support/forum admins - when there are problems\n  people tend to ask about them\n\nWhen you start digging in the above resources you will start noticing patterns.\nRegressions in bug fixes and functionality will be immediately obvious. Questions\naround a common area of misunderstanding or common root cause as well.\n\n**Notice:** Risk here is very loosely defined as\n**if it goes wrong it's bad for the project/community**! It is up to you,\nthe tester, to assign categories and priorities and evaluate how much the\nsoftware project can tolerate for every specific area. For example a cumbersome\ninstallation will be a deal breaker for a desktop software but for a web app\nwhich would mostly be consumed as-a-service may not be.\n\n\n# You need test cases\n\nArmed with your list of risky areas you need to understand why these things\nhappen which will hint at how they can be prevented in the future.\nMind that not everything needs an automated check or even a manual scenario. Some\nitems can be better resolved at the development process - e.g. more tooling, customized\nIDE/linter plugins, etc.\n\nFor the items that remain you need to describe them in details. Be specific.\nFrom a common user story there could be multiple scenarios, both positive and\nnegative ones. Use whatever language will work best for your community -\ncould be Given-When-Then, could be 1, 2, 3, could be example data, etc. The point\nis that whomever will be looking at this scenario (with the intention to later execute it)\nunderstands what the author means. Generally testers will tell you that ambiguity\nis not desired however variations in how different people understand and execute the\nsteps to reproduce will reveal different paths, e.g. more scenarios.\n\nUsually in testing \u0026 QA we organize the test cases that we have in test plans\nor test suites, e.g. \"Installation testing\", \"Localization testing\",\n\"Performance testing\", etc. The division isn't that important, the important\nthing is that you have a plan what needs to be done and where to focus your effort!\n\n\n# You need a process\n\nSoftware testing/Quality Assistance (it's not really assurance anymore these days)\nis equal parts tools \u0026 technology, documentation and process. Be simple:\n\n- When would certain testing activities be executed - daily, weekly, on every pull request\n  or you will work from pre release branches\n- How much time is needed for various testing/stabilization and other pre-release\n  activities\n- Are you going to gate changes going into certain branches (e.g. string freeze,\n  feature freeze, bug-fixes only) or not\n- Who will be responsible for all of the above and how will they communicate? You\n  don't want to turn this into blaming game but everyone needs to know where \u0026 how\n  testing fits\n- What about the community, how are they going to be engaged? Do you need full-time\n  contributors or part-time, etc. What level of access \u0026 knowledge is required from them\n\n\n# Engage your community\n\nYou want to have test executions against various builds and work towards\nan increasing number of PASSing results.\n\nA fairly tipical process would be to have a small dedicated team who is responsible\nfor evaluating risk, documenting test plans \u0026 test cases, overseeing the process(es)\nand a larger pool of volunteers who will help with test execution.\n\nOrganizing test days per area would be a good way to go. Call to action for verifying\nthat bugs have indeed been fixed before a release is also good. Some contributors\nwould be happy to explore the cutting edge nightly builds and give you regular feedback.\n\nThe frequency of test execution for each test plan varies:\n\n- Translation - probably once shortly before relase when you see you have 100% of\n  the desired languages. Then check for typos, language style, etc.\n- Main functionality - rather often, probably on nightly builds or weekly\n- Performance, specialized areas - one-two times per release probably, especially if\n  these aren't critical and/or do not change often\n\n\n# Using a TCMS\n\n- First you need to have a testing process and then infrastructure, not the other way\n  around\n- You don't need Kiwi TCMS or any other system to organize all of the above\n- A TCMS system is used just to keep track of what has been done so that in the future\n  you can go back to it, examine it and decide if you need to do something else\n\n\n# Rinse \u0026 repeat\n\nThis is an iterative process. Design it like that from the start.\nSet aside a regular period in which you will be evaluating your testing process(es)\nand tweaking them as needed.\n\n\n# References\n\n*Please consider contributing a file/URL describing the testing process within\nyour own ogranization!*\n\n\n# What about the name\n\nIt is a tribute to the *Open Source Test Management* stand at FOSDEM\nstarted by SystemTestPortal and Kiwi TCMS in 2019.\n\n![\"Picture from FOSDEM\"](https://raw.githubusercontent.com/kiwitcms/open-source-test-management/master/open-source-test-management-fosdem.jpg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiwitcms%2Fopen-source-test-management","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiwitcms%2Fopen-source-test-management","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiwitcms%2Fopen-source-test-management/lists"}