{"id":18666979,"url":"https://github.com/adoptium/aqa-tests","last_synced_at":"2025-11-06T14:30:29.214Z","repository":{"id":37056655,"uuid":"88991896","full_name":"adoptium/aqa-tests","owner":"adoptium","description":"Home of test infrastructure for Adoptium builds","archived":false,"fork":false,"pushed_at":"2025-02-07T14:57:56.000Z","size":80542,"stargazers_count":138,"open_issues_count":494,"forks_count":317,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-02-07T15:37:37.511Z","etag":null,"topics":["openjdk-tests","tests"],"latest_commit_sha":null,"homepage":"https://adoptium.net/aqavit","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adoptium.png","metadata":{"funding":{"github":"adoptium","custom":["eclipse.org/donate/adoptium"]},"files":{"readme":"README.md","changelog":null,"contributing":"Contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-21T14:24:21.000Z","updated_at":"2025-02-07T14:58:01.000Z","dependencies_parsed_at":"2023-10-10T19:02:21.320Z","dependency_job_id":"a1f3cc8e-d1bf-4c4b-b0ff-afb7d95eed56","html_url":"https://github.com/adoptium/aqa-tests","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adoptium%2Faqa-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adoptium%2Faqa-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adoptium%2Faqa-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adoptium%2Faqa-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adoptium","download_url":"https://codeload.github.com/adoptium/aqa-tests/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239493675,"owners_count":19647995,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["openjdk-tests","tests"],"created_at":"2024-11-07T08:35:12.824Z","updated_at":"2025-11-06T14:30:29.167Z","avatar_url":"https://github.com/adoptium.png","language":"HTML","funding_links":["https://github.com/sponsors/adoptium","eclipse.org/donate/adoptium"],"categories":["HTML"],"sub_categories":[],"readme":"\u003c!--\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n[1]https://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\n--\u003e\n\n\u003c!-- PROJECT LOGO --\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://adoptium.net/aqavit\"\u003e\n    \u003cimg src=\"https://adoptium.net/images/aqavit-light.png\" alt=\"Logo\" width=\"250\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cbr /\u003e\n\n[![License](https://img.shields.io/github/license/Adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/blob/master/LICENSE)\n[![contributors](https://img.shields.io/github/contributors/adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/graphs/contributors)\n[![commit-activity](https://img.shields.io/github/commit-activity/m/adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/commits/master)\n[![closed-issues](https://img.shields.io/github/issues-closed/adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/issues?q=is%3Aissue+is%3Aclosed)\n[![closed-pr](https://img.shields.io/github/issues-pr-closed/adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/pulls?q=is%3Apr+is%3Aclosed)\n[![release-date](https://img.shields.io/github/release-date/adoptium/aqa-tests)](https://github.com/adoptium/aqa-tests/releases)\n\u003cbr /\u003e\n\n[![OpenChain](https://tinyurl.com/3zdb5ptz)](https://www.eclipse.org/openchain/)\n[![slack](https://img.shields.io/badge/Slack-4A154B?logo=slack\u0026logoColor=white)](https://adoptium.net/slack/)\n[![Twitter](https://img.shields.io/twitter/follow/adoptium?style=social)](https://twitter.com/adoptium)\n\n# Adoptium Testing\n\n#### Guide to the Test Jobs at Adoptium\n\nFor nightly and release builds, there are test jobs running as part of the Adoptium continuous delivery pipelines.  There is a [blog post and brief presentation](https://blog.adoptopenjdk.net/2017/12/testing-java-help-count-ways) that explains what testing we run and how they fit into the overall delivery pipeline.  As the world of testing at Adoptium is evolving and improving quickly, some documentation may fall behind the march of progress.  Please let us know and help us keep it up-to-date, and ask questions at the [Adoptium testing Slack channel](https://adoptium.slack.com/archives/C5219G28G)!\n\n![CI pipeline view](doc/diagrams/ciPipeline.jpg)\n\n#### Test 'Inventory'\n\nThe directory structure in this aqa-tests repository is meant to reflect the different types of test we run (and pull from lots of other locations).  The diagrams below show the test make target for each of the types, along with in-plan, upcoming additions (denoted by dotted line grey boxes). The provided links jump to test jobs in Jenkins (ci.adoptium.net).\n\n```mermaid\ngraph TD\n    A[openjdk-tests] --\u003eB[make _perf]\n    A[openjdk-tests] --\u003eC[make _openjdk]\n    A[openjdk-tests] --\u003eD[make _system]\n    A[openjdk-tests] --\u003eE[make _functional]\n    A[openjdk-tests] --\u003eF[make _jck]\n    A[openjdk-tests] --\u003eG[make _external]\n    B[make _perf] --\u003e|perf|H[performance]\n    H[performance] --\u003e|_sanity.perf|I[.....]\n    H[performance] --\u003e|_extended.perf|J[..]\n    C[make _openjdk] --\u003e|openjdk|K[openjdk]\n    D[make _system] --\u003e|system|L[system]\n    E[make _functional] --\u003e|functional|M[functional]\n    F[make _jck] --\u003e|jck|N[jck]\n    G[make _external] --\u003e|external|O[external]\n    O[external] --\u003e|_sanity.external|P[...]\n    O[external] --\u003e|_extended.external|Q[....] \n    \n```\n\n--- \n\n##### [openjdk](https://ci.adoptium.net/view/Test_openjdk/) tests - OpenJDK regression tests \nTests from OpenJDK\n\n--- \n\n##### [system](https://ci.adoptium.net/view/Test_system/) tests - System and load tests \nTests from the adoptium/aqa-systemtest repo\n\n--- \n\n##### [external](https://ci.adoptium.net/view/Test_external/) tests - 3rd party application tests\nTest suites from a variety of applications, along with microprofile TCKs, run in Docker containers\n\n```mermaid\ngraph TD\n A[openjdk-tests] --\u003e|make _external| B[external]\n    B --\u003e C[derby]\n    B --\u003e D[elasticsearch]\n    B --\u003e E[example]\n    B --\u003e F[jenkins]\n    B --\u003e G[kafka]\n    B --\u003e H[lucene-solr]\n    B --\u003e|_sanity.external|I[scala]\n    B --\u003e J[tomcat]\n    B --\u003e K[wildfly]\n    B --\u003e L[openliberty]\n    B --\u003e M[geode]\n    B --\u003e N[hbase]\n    B --\u003e O[akka]\n    B --\u003e P[logstash]\n    B --\u003e Q[openliberty-mp-tck]\n    B --\u003e|_extended.external|R[payara-mp-tck]\n    B --\u003e S[thorntail-mp-tck]\n   \n```\n\n--- \n\n##### [perf](https://ci.adoptium.net/view/Test_perf/) tests - Performance benchmark suites \nPerformance benchmark tests (both full suites and microbenches) from different open-source projects such as Acme-Air and adoptium/bumblebench\n\n```mermaid\ngraph TD\n A[openjdk-tests] --\u003e|make _perf| B[performance]\n    B --\u003e|_sanity.perf| C[bbench]\n    B --\u003e D[idle_micro]\n    B --\u003e E[odm]\n    B --\u003e|_extended.perf| F[liberty_dt]\n    B --\u003e G[acme_air]\n   ```\n\n--- \n\n##### [functional](https://ci.adoptium.net/view/Test_functional/) tests - Unit and functional tests\nFunctional tests not originating from the openjdk regression suite, that include locale/language tests and a subset of implementation agnostic tests from the openj9 project.\n\n--- \n\n##### jck tests - Compliance tests\nTCK tests (under the OpenJDK Community TCK License Agreement), in compliance with the license agreement.  While this test material is currently not run at the Adoptium project (see the [support statement](https://adoptopenjdk.net/support.html#jck) for details), those with their own OCTLA agreements may use the Adoptium test automation infrastructure to execute their TCK test material in their own private Jenkins servers.\n\n--- \n\n#### Guide to Running the Tests Yourself\nFor more details on how to run the same tests that we run at Adoptium on your laptop or in your build farm, please consult our [User Guide](doc/userGuide.md) (work in progress).\n\n#### What is our motivation?\nWe want:\n- better, more flexible tests, with the ability to apply certain types of testing to different builds\n- a common way to easily add, edit, group, include, exclude and execute tests on adoptium builds\n- the latitude to use a variety of tests that use many different test frameworks\n- test results to have a common look \u0026 feel for easier viewing and comparison\n\nThere are a great number of tests available to test a JVM, starting with the OpenJDK regression tests.  In addition to running the OpenJDK regression tests, we will increase the amount of testing and coverage by pulling in other open tests.  These new tests are not necessarily written using the jtreg format.\n\nWhy the need for other testing?  The OpenJDK regression tests are a great start, but eventually you may want to be able to test how performant is your code, and whether some 3rd party applications still work.  We will begin to incorporate more types of testing, including:\n- additional API and functional tests\n- stress/load tests\n- system level tests such as 3rd party application tests\n- performance tests\n- TCK tests\n\nThe test infrastructure in this repository allows us to lightly yoke a great variety of tests together to be applied to testing the adoptium binaries.  By using an intentionally thin wrapper around a varied set of tests, we can more easily run all types of tests via make targets and as stages in our Jenkins CI pipeline builds.\n\n\n#### How can you help?\nYou can:\n- browse through the [aqa-tests issues list](https://github.com/adoptium/aqa-tests/issues), select one, add a comment to claim it and ask questions\n- browse through the [aqa-systemtest issues](https://github.com/adoptium/aqa-systemtest/issues) or [stf issues](https://github.com/adoptium/stf/issues), claim one with a comment and dig right in\n- triage live test jobs at [ci.adoptium.net](https://ci.adoptium.net), check out the [triage doc](https://github.com/adoptium/aqa-tests/blob/master/doc/Triage.md) for guidance\n  - if you would like to regularly triage test jobs, you can optionally 'sign up for duty' via the [triage rotas](https://github.com/adoptium/aqa-tests/wiki/AdoptOpenJDK-Test-Triage-Rotas)\n- ask questions in the [#testing channel](https://adoptium.slack.com/archives/C5219G28G) \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadoptium%2Faqa-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadoptium%2Faqa-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadoptium%2Faqa-tests/lists"}