{"id":20306196,"url":"https://github.com/giis-uniovi/qacover","last_synced_at":"2025-04-16T01:12:47.009Z","repository":{"id":181358946,"uuid":"666643740","full_name":"giis-uniovi/qacover","owner":"giis-uniovi","description":"SQL Query Aware Test Data Coverage Evaluation for Java and .Net applications","archived":false,"fork":false,"pushed_at":"2025-04-14T04:13:45.000Z","size":1205,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-16T01:12:41.152Z","etag":null,"topics":["coverage","csharp","database","fpc","java","mcdc","mutation","net","sql","testing"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/giis-uniovi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-15T05:08:39.000Z","updated_at":"2025-04-05T07:18:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"f36f1e18-d9d2-4838-a83b-af66a005b630","html_url":"https://github.com/giis-uniovi/qacover","commit_stats":null,"previous_names":["giis-uniovi/qacover"],"tags_count":7,"template":false,"template_full_name":"giis-uniovi/samples-giis-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giis-uniovi%2Fqacover","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giis-uniovi%2Fqacover/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giis-uniovi%2Fqacover/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giis-uniovi%2Fqacover/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/giis-uniovi","download_url":"https://codeload.github.com/giis-uniovi/qacover/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249178218,"owners_count":21225350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coverage","csharp","database","fpc","java","mcdc","mutation","net","sql","testing"],"created_at":"2024-11-14T17:12:21.525Z","updated_at":"2025-04-16T01:12:47.000Z","avatar_url":"https://github.com/giis-uniovi.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Status](https://github.com/giis-uniovi/qacover/actions/workflows/test.yml/badge.svg)](https://github.com/giis-uniovi/qacover/actions)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=my%3Aqacover\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=my%3Aqacover)\n[![Test Reports](https://img.shields.io/badge/%20-Test_Reports-orange)](https://giis-uniovi.github.io/qacover/junit-noframes/junit-noframes.html)\n[![Test Reports (frames)](https://img.shields.io/badge/%20-Test_Reports_(frames)-orange)](https://giis-uniovi.github.io/qacover/junit-frames/index.html)\n[![Maven Central (core)](https://img.shields.io/maven-central/v/io.github.giis-uniovi/qacover-core)](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-core)\n[![Nuget](https://img.shields.io/nuget/v/QACover)](https://www.nuget.org/packages/QACover/)\n\n# QACover - SQL Query Aware Test Data Coverage Evaluation for Java and .Net applications\n\n*QACover* is a component to evaluate the test data coverage in relation to the\nSQL queries that are executed in a Java or .NET application.\nThe coverage is measured according to the *SQL Full Predicate Coverage* (SQLFpc) criterion,\na variant of MCDC tailored to SQL queries.\nThe criterion determines the situations of interest (test coverage items) \nto test a SQL query against the test database. \nThese situations are represented as a set of *Coverage Rules*.\nThere is also an option to measure the coverage of *Mutants for SQL queries* (SQLMutation criterion).\n\nEach time that the application executes a query, QACover intercepts the query execution, \ngenerates and evaluates the coverage rules, and stores the results in the local development environment.\n\nAt the end of the tests you can get the summary and detail reports of test data coverage.\nThis is an example of the summary report of a test session:\n\n![Example of a summary report](docs/image-index.png)\n\n## Contents\n  \n  - [Quick Start](#quick-start)\n  - [Example scenario](#example-scenario)\n  - [QACover Components](#qacover-components)\n    - [Java Dependencies in Maven Central](#java-dependencies-in-maven-central)\n    - [.NET Packages in NuGet](#net-packages-in-nuget)\n  - [Configuration](#configuration)\n    - [QACover configuration file](#qacover-configuration-file)\n    - [P6Spy configuration file](#p6spy-configuration-file)\n    - [Configuration for .NET](#configuration-for-net)\n  - [Logging](#logging)\n  - [Reporting](#reporting)\n    - [Report generation](#report-generation)\n    - [Content of the reports](#content-of-the-reports)\n    - [Include the source code](#include-the-source-code)\n  - [Contributing and Architecture](#contributing-and-architecture)\n\n## Quick Start\n\nExample for Java:\n\n- Add the dependency \n  [`qacover-core`](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-core) to your pom.xml\n- Copy the files  \n  [`qacover.properties`](qacover-core/qacover.properties) and \n  [`spy.properties`](qacover-core/spy.properties) from the `qacover-core` folder to the root of your project.\n- Edit the connection string of your application and insert `:p6spy` afer `jdbc`\n  (eg. if your connection string is `jdbc:sqlite:./target/TestDB.db` it must become `jdbc:p6spy:sqlite:./target/TestDB.db`).\n- Run your tests and wait to finish.\n\nThis creates the folder `target/qacover/rules` that contains the internal data about the coverage evaluation.\nTo generate an html report:\n\n- Download the *the standalone reporter jar file*\n  [`qacover-model-\u003cVERSION\u003e-report.jar`](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-model)\n  from Maven Central (go to Versions and then Browse the selected version to download).\n- Run this command from the root of your workspace:\n  ```bash\n  java -jar qacover-model-\u003cVERSION\u003e-report.jar  target/qacover/rules  target/qacover/reports\n  ``` \n- Open the `index.html` that you will found in the `target/qacover/reports` folder.\n\nIf you find that the class names are not the ones at the interaction point that executes the query, \nyou will need to tweak the configuration to include some exclusions for their packages\n(see later), remove the `target/qacover` folder and repeat again.\n\n## Example scenario\n\nFolder with the test package [qacoversample](qacover-core/src/test/java/test4giis/qacoversample)\ncontains an example of how to use the coverage information to improve the test data and test cases\nto reveal hidden bugs. It contains three sequential scenarios:\n\n1. Execute test cases and let QACover to evaluate the test coverage. \n   Test data was designed to cover a number of test situations that were manually determined. All tests pass.\n2. Use the test coverage information to automatically determine uncovered situations\n   to complete the previous test data and the test cases. This allows revealing two hidden faults.\n3. Final debug and fix.\n\n## QACover Components\n\n### Java Dependencies in Maven Central\n\nReleases of the java artifacts (java 8 or higher) are published in Maven Central under the group id `io.github.giis-uniovi`.\nThere are two different artifacts:\n\n- [`qacover-core`](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-core):\n  The main artifact to use as a a dependency in your application  (as shown in the Quick Start).\n- [`qacover-model`](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-model):\n  It only includes the model and classes to do reporting and to inspect the coverage rules.\n  Use it if you only need access to previously generated coverage rules (e.g. to generate reports from a program).\n\nEach of them has another downloadable jar that includes additional qualifier:\n\n- [`qacover-core` uber jar](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-core).\n  It includes all needed dependencies (excluding `slf4j`) and they are *shaded*\n  (i.e. renamed in a different namespace to avoid dependency conflicts):\n  - Download the artifact with the `-uber` qualifier if for any reason you cannot use it as a dependency in your application \n    (e.g. to deploy in an application server).\n    You simply need to put the jar in your server library and set the configuration to use QACover.\n  - Use the uber jar as a dependency declared in your pom.xml if you experiment conflicts with versions:\n    Add `\u003cqualifier\u003euber\u003c/qualifier\u003e` to the dependency declaration.\n- [`qacover-model` standalone reporter](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-model):\n  Download the artifact with the `-report` qualifier to generate the reports from the command line as shown in the [quick start](#quick-start).\n\n### .NET Packages in NuGet\n\nReleases for .NET platform are published in NuGet. \nMost of the code is converted automatically from Java to .NET using [JavaToCsharp](https://github.com/paulirwin/JavaToCSharp).\nThe same as for Java, there are two different packages:\n\n- [`QACover`](https://www.nuget.org/packages/QACover/):\n  The main package (netstandard2.9) to include as a package reference in your project configuration (e.g. the .csproj file if you are using C#).\n- [`QACoverReport`](https://www.nuget.org/packages/QACoverReport/):\n  A dotnet tool (netcore2.0) to generate the reports from the command line:\n  Install the tool with `dotnet tool install QACoverReport`\n  and execute it as a command `QACoverReport \u003crules folder\u003e \u003creports folder\u003e`.\n\n## Configuration\n\nOn Java, you need to have two configuration files to evaluate the coverage: \n[`qacover.properties`](qacover-core/qacover.properties) and \n[`spy.properties`](qacover-core/spy.properties) and to customize the JDBC Driver.\nOn .NET you only need the first one along with some additional code to intercept the queries.\n\n### QACover configuration file\n\nQACover looks for the `qacover.properties` in this order:\n- System properties.\n- The application classpath.\n- The default path where the application or the tests are executed.\n\nThe [`qacover.properties`](qacover-core/qacover.properties) available in the `qacover-core` module of this\nrepo contains a general configuration suitable for common scenarios, but sometimes it must be customized.\nSee the file for details on each configuration parameter. Next, we highlight the most important ones that are the\ninclusion and exclusion criteria.\n\nWhen a line of a method in your application executes a SQL query (*interaction point*),\na chain of calls to methods of your framework\nis executed until reaching the driver method that actually executes the query. \nHere is the point in which the actual execution of the query is detected,\nbut what we want is to determine the interaction point in the application.\nTo achieve this, QACover checks the call stack at the point of the actual execution\nand successively excludes every call made in any framework package until it locates the point of the \ndatabase interaction in your method.\n\nQACover excludes the system packages like the java, System, P6Spy or the QACover packages, but depending on the framework\nyou must configure additional exclusions by setting the `qacover.stack.exclusions` property in the file [`qacover.properties`](qacover-core/qacover.properties).\n\n**Example**: Folder `it/spring-petclinic-main` contains a typical sample from Spring Boot.\nThe exclusion is declared as:\n```\nqacover.stack.exclusions=org.springframework.,org.hibernate.,com.zaxxer.hikari.,com.sun.,sun.reflect.\n```\nthat removes the framework classes that we want to skip to locate the interaction point that is at the\n`org.springframework.samples.petclinic.PetclinicIntegrationTests` class.\n\nHowever, in this particular case, the interaction point is under `org.springframework`.\nWe must add the inclusions parameter to ensure that `org.springframework.samples.` is not excluded:\n```\nqacover.stack.inclusions=org.springframework.samples.\n```\n\nThere are other parameters to configure inclusion criteria for packages, \nand exclusion criteria for class names or table names.\nSee [`qacover.properties`](qacover-core/qacover.properties) for more details.\n\n### Configuration for Java: P6Spy configuration file\n\nThe [`spy.properties`](qacover-core/spy.properties) available in the `qacover-core` folder of this\nrepo contains the minimal configuration required by P6Spy:\n- `modulelist=giis.qacover.driver.InterceptorFactory` must always be present to indicate the point in which\n  P6Spy passes the control to QACover\n- Also, you may need to configure the formats for boolean, dates an times.\n\nSee the [`spy.properties`](qacover-core/spy.properties) file\nor the [`P6Spy documentation`](https://p6spy.readthedocs.io/en/latest/configandusage.html) for more details.\n\n### Configuration for .NET\n\nConfiguration for .NET project use the same \n[`qacover.properties`](qacover-core/qacover.properties) than Java,\nbut does not use `spy.properties`. Instead, it requires some coding:\n\n**On ADO.NET**: Use a connection wrapper. There is a default implmentation at\n  [SampleDbConnectionWrapper.cs](net/QACover/Giis.Qacover.Driver/SampleDbConnectionWrapper.N.cs)\n\nBecause there is no `spy.properties` to activate the query interception, you have to either define the environment variable\n```csharp\nQACOVER_LISTENER_CLASS=Giis.Qacover.Driver.EventListener\n``` \n\nor make a direct call to:\n```csharp\nGiis.Qacover.Driver.EventTrigger.SetListenerClassName(\"Giis.Qacover.Driver.EventListener\");\n```\n\n**On Entity Framework**: Use a custom context that inherits from `DbContext`, see e.g.\n  [Ef2InterceptorContext.cs](net/QaCoverEf2spy/Giis.QACover.Ef2driver/Ef2InterceptorContext.N.cs)\n\n## Logging\n\nLogging can be configured for packages starting with `giis.qacover.`:\n- INFO level is suitable in most cases: logs the queries, parameters and a short summary of the\nevaluation results. \n- DEBUG level displays details on how the configuration files are read and the query interception.\n\nIn addition to standard logs, other folders `log-*` are created in the `rules` folder\nto display additional debug information about the queries that are evaluated, the database schema,\nand the coverage rules.\n\n## Reporting\n\nThe report generation creates a set of static html files in the designated folder,\nto easily inspect summary and details of the coverage data. \n\n### Report generation\n\nTo generate reports you have three options:\n\n- From the command line: Download the\n  [`qacover-model` standalone reporter](https://central.sonatype.com/artifact/io.github.giis-uniovi/qacover-model)\n  as shown in the quick start and execute:\n```bash\n        java -jar qacover-model-\u003cVERSION\u003e-report.jar  target/qacover/rules  target/qacover/reports\n```\n- From a program or test that includes `qacover-model` in the classpath:\n```java\n        new giis.qacover.report.ReportManager().run(\"target/qacover/rules\", \"target/qacover/rules\");\n```\n- From the maven lifecycle: If `qacover-model` is declared as a dependency, \n  execute the `ReportMain` method using the `exec-maven-plugin`:\n```xml\n        \u003cplugin\u003e\n            \u003cgroupId\u003eorg.codehaus.mojo\u003c/groupId\u003e\n            \u003cartifactId\u003eexec-maven-plugin\u003c/artifactId\u003e\n            \u003cversion\u003e1.6.0\u003c/version\u003e\n            \u003cexecutions\u003e\n                \u003cexecution\u003e\n                    \u003cid\u003eqacover-report\u003c/id\u003e\n                    \u003cphase\u003epost-integration-test\u003c/phase\u003e\n                    \u003cgoals\u003e\n                        \u003cgoal\u003ejava\u003c/goal\u003e\n                    \u003c/goals\u003e\n                    \u003cconfiguration\u003e\n                        \u003cclasspathScope\u003etest\u003c/classpathScope\u003e\n                        \u003cclasspath/\u003e\n                        \u003cmainClass\u003egiis.qacover.report.ReportMain\u003c/mainClass\u003e\n                        \u003carguments\u003e\n                            \u003cargument\u003etarget/qacover/rules\u003c/argument\u003e\n                            \u003cargument\u003etarget/qacover/reports\u003c/argument\u003e\n                        \u003c/arguments\u003e\n                    \u003c/configuration\u003e\n                \u003c/execution\u003e\n            \u003c/executions\u003e\n       \u003c/plugin\u003e\n```\n\n### Content of the reports\n\nThe `index.html` file contains the summary of test data coverage for each class:\n\n![Example of a summary report](docs/image-index.png)\nwhere:\n- %: Total percent of coverage (number of coverage rules covered divided by total number of coverage rules generated).\n- qrun: total number of query evaluations.\n- qcount: number of different queries that have been evaluated.\n- qerror: number of queries that have not been evaluated because some error.\n- dead: number of coverage rules covered.\n- count: number of coverage rules generated.\n- error: number of coverage rules that have not been evaluated because some error.\n\nEach class name is clickable to display a report that contains the details for queries that have been evaluated.\nThe report for a class looks like:\n\n![Example of a summary report](docs/image-class-1.png)\n...\n![Example of a summary report](docs/image-class-2.png)\n\nClicking the down arrow near the percent coverage at the query evaluated\nexpands the details of each coverage rule (covered in green, uncovered in yellow):\n- A textual message that explains the test situation that the coverage rule represents.\n  If a coverage rule is not covered, a test and/or the appropriate test data may be added in order to cover it.\n- The SQL representation of the coverage rule.\n- Additional indicators\n  - Sequential ID.\n  - dead: number of times that the coverage rule has been covered.\n  - count: number of times that the coverage rule has been executed.\n  - category, type, subtype, location: A classification about where the coverage rule comes from.\n\n### Include the source code\n\nThe general syntax of the report generator has four parameters (only the two first ones are required\nif you do not include the source code of the classes under test):\n```\n\u003crules-folder\u003e \u003creports-folder\u003e [\u003csource-folders\u003e [\u003cproject-folder\u003e]]\n```\n\nOn Java, if you want to include the source code in the reports, you have to set a value for the third parameter \n`\u003csource-folders\u003e` to include a comma-separated list of the path(s) to locate the sources. For example:\n- If executing the reports from the root of a maven Java project, \n  set `src/main/java`.\n- If executing the reports form the root of a multimodule Java project (parent project) with two modules,\n  set `module1/src/main/java,module2/src/main/java`.\n\nOn .NET, you have to set a value for both the third and fourth parameters: \n`\u003csource-folders\u003e` and `\u003cproject-folder\u003e`.\nThe reason is that the location of .NET source files does not exactly match the namespaces, so that,\nthe FPC coverage rules store the absolute path of the class source files that has to be\nresolved to a relative path before report generation. For example:\n- If executing the reports from a solution folder that contains a project, set both parameters to `.`\n- If executing the reports from the unit tests in a solution that contains a project, \n  set both parameters to `../../../..` (because the default directory where the tests run\n  is four levels down the solution folder)\n- If executing the reports from a solution folder that contains a project, \n  but the reports where generated inside a container that runs a server application under the `/app` folder,\n  set the parameters as `.` `/app` (The `/app` value allows to resolve the relative path of each source file \n  from the project folder). \n\n\n## Contributing and Architecture\n\nSee the general contribution policies and guidelines for *giis-uniovi* at \n[CONTRIBUTING.md](https://github.com/giis-uniovi/.github/blob/main/profile/CONTRIBUTING.md).\n\nNow we include some additional background technical information:\n\nQACover makes use of \n[p6spy](https://github.com/p6spy/p6spy) to intercept the jdbc calls,\n[TdRules](https://github.com/giis-uniovi/tdrules) to get the database schema and invoke the\n[SQLRules Service](https://in2test.lsi.uniovi.es/sqlrules/) to generate the coverage rules.\nThe execution of everything is made in local against the database configured in the connection string.\n\nThe internal structure of the main QACover packages (prefix `giis.qacover.`) is shown below (the prefixes are omitted for simplicity):\n- **`core` module**: Contains the `driver`, `core` and `core.sevices` packages.\n- **`model` module**: Contains the `model`, `storage`, `reader` and `report` packages.\n\nThese are the dependencies between packages:\n\n```mermaid\nflowchart TD\n  driver --\u003e core\n  core --\u003e services(core.services)\n  services --\u003e storage\n  storage --\u003e model\n  core --\u003e model\n  services --\u003e model\n  report --\u003e reader\n  report --\u003e model\n  reader --\u003e model\n  reader --\u003e storage\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiis-uniovi%2Fqacover","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgiis-uniovi%2Fqacover","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiis-uniovi%2Fqacover/lists"}