https://github.com/timdegroot1996/robotframework-dashboard
Robot Framework Dashboard and Result Database command line tool
https://github.com/timdegroot1996/robotframework-dashboard
Last synced: about 2 months ago
JSON representation
Robot Framework Dashboard and Result Database command line tool
- Host: GitHub
- URL: https://github.com/timdegroot1996/robotframework-dashboard
- Owner: timdegroot1996
- License: mit
- Created: 2024-10-13T19:58:38.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2025-03-07T17:13:48.000Z (about 2 months ago)
- Last Synced: 2025-03-07T17:17:35.756Z (about 2 months ago)
- Language: HTML
- Size: 12.1 MB
- Stars: 11
- Watchers: 2
- Forks: 3
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-robotframework - robotframework-dashboard - Dashboard for insights into test results across multiple runs. (Tools / Reporting Tools)
README
# Robot Framework Dashboard
### Table of Contents
- [Overview](#Overview)
- [Installation](#Installation)
- [Command Line Features](#Command-Line-Features)
- [Dashboard HTML Features](#Dashboard-HTML-Features)
- [Version Details](#Version-Details)
- [Usage](#Usage)
- [Examples](#Examples)
- [Custom Database Class](#Custom-Database-Class)
- [Contributing](#Contributing)
- [License](#License)
The Robot Framework Dashboard is a tool for [Robot Framework](https://robotframework.org/) that provides insight of your test results across multiple runs. The tool makes use of the built in Robot Framework [Result Visitor API](https://robot-framework.readthedocs.io/en/stable/_modules/robot/result/visitor.html) to analyse output.xml files, stores these in a simple sqlite3 database and finally creates a HTML dashboard that makes use of [Chart.js](https://www.chartjs.org/docs/latest/) and [Datatables](https://datatables.net/) for the graphs and tables and makes use of [Bootstrap](https://getbootstrap.com/) for styling.
If you need help, have suggestions or want to discuss anything, feel free to contact through the [Slack Channel](https://robotframework.slack.com/archives/C07SPR6N9AB) or through a [GitHub issue](https://github.com/timdegroot1996/robotframework-dashboard/issues)
1. Install Robot Framework 6.0 or higher (if not already installed):
2. Install Robot Framework Dashboard
```
pip install robotframework-dashboard
```
## Command Line Features
When running the tool the following steps will be executed
1. Database preparation creates robot_results.db or specified database if it does not yet exist and connects to it
2. Optionally reads test execution result from 1 or more output XML file(s) and uploads to the database
3. Optionally lists all runs currently available in the database, ordered by the time they were entered
4. Optionally removes run(s) specified by run_date or index of being added
5. Optionally generates a dashboard HTML file that can be used to look at the stored results
The Robot Framework Dashboard HTML will provide information across your runs across all levels.
Currently available features:
- Global filters: run, run tag, run date
- Runs
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Overview: table overview of runs
- Suites
- - Filter: suite
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Most failed: top 10 failing suites
- Overview: table overview of suites
- Tests
- Filter: suite, test
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Most failed: top 10 failing tests
- Fail messages: top 10 fail messages
- Overview: table overview of tests
- Keywords
- Filter: keywords
- Keyword statistics: pass, fail, skip
- Keyword times run
- Keyword total duration
- Keyword average duration
- Keyword min duration
- Keyword max duration
- Overview: table overview of keywords
## Version Details
- Tested robot versions: 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 7.0, 7.0.1, 7.1, 7.1.1
- When running robotdashboard with robot 6.x installed, output files that are generated by robot 7.x will have no duration fields when processed, this is caused by changes in the Result Visitor model
- When running robotdashboard with robot 7.x installed every output generated by any version above should be processed correctly
Run robotdashboard through the command line to initialize your database and generate the report:
```
robotdashboard -o output.xml
```
For detailed command line options take a look at:
```
robotdashboard -h
```
or
```
robotdashboard --help
```
Advanced examples:
- Generate the report without adding new outputs
```
robotdashboard
```
- Add tags to your output.xmls
```
robotdashboard -o output1.xml:tag1 -o reports/output2.xml:tag1:tag2
```
- Add all \*output\*.xml in an entire folder tree to the database and provide all these runs with tags
```
robotdashboard -f ./reports
robotdashboard -f ../../some_folder/sub_folder/logs
robotdashboard -f C:/nightly_runs:tag1:tag2:tag3
```
- Do not list runs and/or do not create the dashboard HTML
```
robotdashboard -l false -g false
```
- Custom path/file name for your dashboard HTML
```
robotdashboard -n results/result_robot_dashboard.html
```
- Custom database path/file location
```
robotdashboard -d result_data/robot_result_database.db
```
- Remove data from the database, can be based on run number (index) or run start time
```
robotdashboard -r 0
robotdashboard -r -1 -r "2024-10-17 15:05:04.563559"
```
- Add a custom dashboard html title
```
robotdashboard -t "My Cool Title"
```
- Show milliseconds in the graphs in the dashboard html (setting excludemilliseconds to False as it defaults to True)
```
robotdashboard -e False
```
- Make use of a custom DatabaseProcessor class see also [Custom Database Class](#Custom-Database-Class) for examples and more details of the requirements.
```
robotdashboard -c ./path/to/custom_class.py
robotdashboard --databaseclass mysql.py
```
## Examples
Here are some examples of generated files/output:
- [Full Dashboard](./example/robot_dashboard.html) -> Download and open in a browser to use it!
- [Database](./example/robot_results.db) -> Download and use any tool to check the tables or use it as a base for using the "robotdashboard" command line interface
- Example Command Line Output (below)
This is an example after running robotdashboard for the first time. 3 outputs are added, stored and processed in the dashboard HTML.
Input:
```
robotdashboard -o output1.xml -o output2.xml -o output3.xml
```
Results:
```
======================================================================================
____ ___ ____ ___ _____ ____ _ ____ _ _ ____ ___ _ ____ ____
| _ \ / _ \| __ ) / _ |_ _| _ \ / \ / ___|| | | | __ ) / _ \ / \ | _ \| _ \
| |_) | | | | _ \| | | || | | | | |/ _ \ \___ \| |_| | _ \| | | |/ _ \ | |_) | | | |
| _ <| |_| | |_) | |_| || | | |_| / ___ \ ___) | _ | |_) | |_| / ___ \| _ <| |_| |
|_| \_\\___/|____/ \___/ |_| |____/_/ \_|____/|_| |_|____/ \___/_/ \_|_| \_|____/
======================================================================================
1. Database preparation
created database connection: 'robot_results.db'
======================================================================================
2. Processing output XML(s)
Processing output XML 'output1.xml'
Processed output XML 'output1.xml' in 0.04 seconds
Processing output XML 'output2.xml'
Processed output XML 'output2.xml' in 0.05 seconds
Processing output XML 'output3.xml'
Processed output XML 'output3.xml' in 0.05 seconds
======================================================================================
3. Listing all available runs in the database
Run 0 | 2024-10-17 15:05:04.563559 | RobotFramework-Project1
Run 1 | 2024-10-18 16:43:12.772757 | Robotframework-Project2
Run 2 | 2024-10-21 10:54:25.663094 | Robotframework-Project1
======================================================================================
4. Removing runs from the database
skipping step
======================================================================================
5. Creating dashboard HTML
created dashboard 'robot_dashboard_20241021-150726.html' in 0.01 seconds
```
## Custom Database Class
See the [Example Folder](./example) for some examples of custom database class implementations.
### Available examples
Currently available database type examples:
- [template.py](./example/template.py) (only requirements are in the template file)
- [mysql.py](./example/mysql.py) (for a mysql database connection)
- ...
If you have made an implementation that is not yet an example please feel free to add it through a pull request or submit it in an issue. This way you can help other people implement their own solutions!!
### Custom Class Requirements
- File: The filename can be anything as long as it is a **python file**
- Class: The you should name the **class DatabaseProcessor**
- Methods:
- **\_\_init\_\_(self, database_path: Path):**, should handle the connection to the database and possible creation of the tables if necessary
- **close_database(self):**, should close the database connection
- **insert_output_data(self, output_data: dict, tags: list):**, should be able to handle the output_data dict and the run tags that are provided. Look at the example implementations for the content of output_data and tags.
- **get_data(self):**, should retrieve all data (runs/suites/tests/keywords) from all tables in the form of a dictionary containing the 4 data types. In which each data type is a list of dicts with entries. Example:
{'runs': [{"run_start": "2024-10-13 22:33:19", "full_name": "Robotframework-Dashboard", "name": "Robotframework-Dashboard", "total": 6, "passed": 4, "failed": 1, "skipped": 1, "elapsed_s": "6.313", "start_time": "2024-10-13 22:33:19.673821", "tags": ""}, {"run_start"...}]}
- **list_runs(self):**, should print out the runs in the database with useful identiefiers. This function can be empty as long as it exists. It is purely for the command line overview.
- **remove_runs(self, remove_runs):**, should be able to handle either indexes to be removed, or run_starts that are provided which can then be used to delete the data in the database.
- Do not use relative imports! This will not work on runtime!
## Contributing
Contributions are welcome! If you encounter any issues, have suggestions for improvements, or would like to add new features, feel free to open an issue or submit a pull request.
## License
This project is licensed under the MIT License.
Note: This project is not officially affiliated with or endorsed by robotframework.