{"id":31816182,"url":"https://github.com/scipopt/ipet","last_synced_at":"2026-05-20T01:33:01.209Z","repository":{"id":315580976,"uuid":"1059477006","full_name":"scipopt/ipet","owner":"scipopt","description":"Interactive Performance Evaluation Tools for Optimization Software","archived":false,"fork":false,"pushed_at":"2025-09-19T11:56:04.000Z","size":4376,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-19T13:17:35.004Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scipopt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-18T13:51:52.000Z","updated_at":"2025-09-18T13:55:39.000Z","dependencies_parsed_at":"2025-09-19T13:17:38.663Z","dependency_job_id":"afa989e7-d84d-4da7-87dc-0e9f20eef9c3","html_url":"https://github.com/scipopt/ipet","commit_stats":null,"previous_names":["scipopt/ipet"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/scipopt/ipet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipopt%2Fipet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipopt%2Fipet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipopt%2Fipet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipopt%2Fipet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scipopt","download_url":"https://codeload.github.com/scipopt/ipet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipopt%2Fipet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006758,"owners_count":26084184,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-11T09:44:22.903Z","updated_at":"2025-10-11T09:44:28.431Z","avatar_url":"https://github.com/scipopt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IPET (Interactive Performance Evaluation Tools) \n\nIPET is a toolbox that allows to easily create customized benchmark tables from\nraw solver log files, written in Python 3.\n\nIt is aimed to develop evaluations through a\ngrapical interface that can later be loaded and reused on other\ndata. Additional readers and evaluations are stored in\nxml-format which can be plugged into the evaluation method.\n\nIf you find IPET useful for your work, it would be nice if you\ncite it using the following bibtex item:\n\n```\n@Misc{ipet,\n  Title                    = {{IPET} Interactive Performance Evaluation Tools},\n  Author                   = {Gregor Hendel},\n  HowPublished             = {\\url{https://github.com/GregorCH/ipet}}\n}\n```\n\n# Table of contents\n\n- [How to get IPET](#how-to-get-ipet)\n- [Installation and prerequisites](#installation-and-prerequisites)\n  * [Linux installation inside a virtual environment](#linux-installation-inside-a-virtual-environment)\n  * [Linux installation into your global environment](#linux-installation-into-your-global-environment)\n  * [Testing your installation](#testing-your-installation)\n- [Usage and concept](#usage-and-concept)\n  * [Overview](#overview)\n  * [Basic usage on the command line](#basic-usage-on-the-command-line)\n  * [Tutorial for parsing results obtained with SCIP](#tutorial-for-parsing-results-obtained-with-scip)\n  * [Configuration](#configuration)\n  * [Starting the graphical user interface](#starting-the-graphical-user-interface)\n- [Building the documentation](#building-the-documentation)\n\n# How to get IPET\n\nIn the following installation instructions, we will assume that you clone the IPET repository by\n\n    git clone https://github.com/scipopt/ipet.git\n\nor download and unzip one of the available zipped archieves into a local directory called \"ipet\".\n\nIt is planned to make ipet available via the Python package indexer PyPi.\n\n# Installation and prerequisites\n\nIPET was originally written in Python2.7 and recently converted into Python3.\nWe give installation instructions for the Linux operating system. Since the code is\nwritten in python only, it should also work on other platforms. We assume that you either\nhave sudo-privileges on your target architecture, or are able to create a python3\nvirtual environment. You can test if virtual environments are available by running the command\n\n    virtualenv --python python3 myvenv\n    rm -r myvenv # remove the created directory\n\nIf your target system does not provide virtual environments, please contact your system administrator\nbefore you continue.\n\n## Linux installation inside a virtual environment\n\n*This is the recommended kind of installation.*\n\nUsing a virtual environment, you can safely install the required packages together without\nmessing with your globally installed system libraries. This is certainly the recommended way\nto install *IPET*. In the following, we assume you create the root directory of your virtual\nenvironment directly in the IPET root directory and call it \"venv\". You may choose\nto use a differently named virtual environment somewhere else, of course.\n\n1. Create and activate the virtual environment (a new directory called \"venv\" gets created):\n\n        cd ipet\n        virtualenv --python python3 venv\n        source venv/bin/activate\n        \n   Note that you may deactivate the virtual environment at any time by calling\n\n        deactivate\n\n2. Execute the command (note that it does no longer require to mention python3 because we are running inside the virtual environment)\n\n        pip install .\n\n3. As a developer, it might be useful to call the following command instead:\n\n        pip install -e .\n        \n   This creates symlinks to the IPET source files in the site-packages of the virtual environment library,\n   and allows for more rapid testing and development.\n\n## Linux installation into your global environment\n\n[We highly recommend to install IPET into a virtual environment.](#linux-installation-inside-a-virtual-environment)\n\nIf you would like to install *IPET* systemwide, all you need to do is\n\n    cd ipet\n    python3 -m pip install . --user\n\nor\n\n    cd ipet\n    sudo python3 -m pip install .\n\nThis step makes imports available systemwide such as\n\n    python3 -c \"from ipet import *\"\n\n## Testing your installation\n\nRun the command\n\n    python -m unittest test\n    \nif the output says OK, all tests were passed.\n\n# Usage and concept\n\n**under construction**\n\n## Overview \n \nIPET takes a logfile and some optional additional files like an error-, set- and metafile, extracts information and aggregates this data in a compact table.\nThe Logfiles need to have an `.out` extension, errorfiles need to have `.err`, setfiles are `.set` and metafiles are `.meta`\n\nThe process is divided in two stages:\n\n1) `ipet-parse`, which performs the parsing, where the standard data is extracted and stored in a .trn file.\n2) `ipet-evaluate`, where the data is aggregated and displayed in two tables by user-defined rules. The *aggregated table* displays the information that was condensed from the *long table*.\n\nFor more information, please refer to the [Basic usage](#basic-usage-on-the-command-line) section and the help pages, that can be displayed with `ipet-parse --help` and `ipet-evaluate --help`.\nIt is possible to configure ipet for your own solver and testset, please check the [Configuration](#configuration) section.\n\n## Basic usage on the command line\n\nIPET is easily used on the command line. Assume you have a logfile `testrun.out` that contains the output of running the solver on a list of instances. \nThe output of each run is preceded by a line indicating the instance and preceded by a line indicating correct shutdown of the solver. \nIn other words your format is the following:\n\n        @01 /path/to/my/first_instance.clp\n        \u003cOutput of solver for aforementioned instance\u003e\n        =ready=\n        @01 /path/to/my/second_instance.mpz\n        \u003cOutput of solver for aforementioned instance\u003e\n        =ready=\n        ...\n        =ready=\n        @01 /path/to/my/last_instance.clp\n        \u003cOutput of solver for aforementioned instance\u003e\n        =ready=\n        \nNow you call the parsing command with the logfile, which will create a `testrun.trn` file storing the parsed data.\n\n        ipet-parse -l testrun.out\n        \nIn the second step you call the evaluation command with this `testrun.trn` file and an evaluation file that encodes the datakey and aggregation functions for your table. \nThere is an example in `scripts/evaluation.xml`.\nCalling `ipet-evaluate` will display the aggregated table only:\n\n        ipet-evaluate -e scripts/evaluation.xml -t testrun.trn\n\nIf you are interesting in the considered values that result in this table you can have a look at the long table with the `--long` or `-l` option:\n\n        ipet-evaluate -e scripts/evaluation.xml -t testrun.trn --long\n\n## Tutorial for parsing results obtained with SCIP\n\nSay that you used SCIP to solve a number of instances and would like to get the solving time, number of nodes used and the status of the solver for each instance. This short tutorial will show you how to do that with ipet. As an input, it is assumed that you have the output of scip in a separate file for each instance in the folder `SCIP_OUT_FILES`.\n\n### Step 1\nA helper script scripts/concat.sh is available that can preprocess single output files and create a concatenated logfile in the format that is needed by ipet. As input it takes the folder containing your logfiles:\n```\n$ cd scripts\n$ ./concat.sh -f SCIP_OUT_FILES\nConcatenated logs from folder '../SCIP_OUT_FILES' into '../SCIP_OUT_FILES/concatenated.out'.\n```\n\n### Step 2\nWe can now parse the results by calling the ipet-parse command with the concatenated logfile:\n```\n$ ipet-parse -l SCIP_OUT_FILES/concatenated.out\n2021-07-01 14:49:50,530 -    INFO - root - Start parsing process using 8 threads\n  0%|                                                                                    | 0/1 [00:00\u003c?, ?it/s]2021-07-01 14:49:50,532 -    INFO - root - Start parsing process of outfile(s) SCIP_OUT_FILES/concatenated.out\n2021-07-01 14:49:50,983 -    INFO - root - converted /home/boro/repo/ipet/SCIP_OUT_FILES/concatenated.out --\u003e /home/boro/repo/ipet/SCIP_OUT_FILES/concatenated.trn\n100%|████████████████████████████████████████████████████████████████████████████| 1/1 [00:00\u003c00:00,  2.21it/s]\n```\n\n### Step 3\nCalling ipet-evaluate now with the resulting .trn file and the example evaluation file scipts/evaluation.xml will display our data in a table:\n```\n$ ipet-evaluate -t SCIP_OUT_FILES/concatenated.trn -e scripts/evaluation.xml --long\n2021-07-01 14:52:32,903 -    INFO - root - No external data file\n2021-07-01 14:52:32,949 -    INFO - ipet.evaluation.IPETEvalTable - Automatically set index to (['ProblemName'], [])\n2021-07-01 14:52:32,949 -    INFO - ipet.evaluation.IPETEvalTable - No validation information specified\n2021-07-01 14:52:32,949 -    INFO - ipet.validation.Validation - Validating with a (gap) tolerance of 0.0001 and a feasibility tolerance of 1e-06.\n2021-07-01 14:52:32,982 -    INFO - ipet.evaluation.IPETEvalTable - Validation resulted in the following status codes: [ fail_inconsistent: 59 | ok: 2 ]\n2021-07-01 14:52:33,149 -    INFO - ipet.evaluation.IPETFilter - Computing rows for intersection groups\n2021-07-01 14:52:33,213 - WARNING - ipet.evaluation.IPETEvalTable - Filtergroup diff-timeouts is empty and has been deactived.\nInstancewise Results:\n                 Time  Nodes             Status\nProblemName                                    \n31966239     36004.22      1  fail_inconsistent\n31966240     36007.75      1  fail_inconsistent\n31966241     36012.76      1  fail_inconsistent\n31966242     36001.43      1  fail_inconsistent\n31966243     36008.84      1  fail_inconsistent\n31966244     36029.03      1  fail_inconsistent\n31966245     36002.47      1  fail_inconsistent\n31966246     36015.92      1  fail_inconsistent\n31966247     36005.92      1  fail_inconsistent\n31966248     36013.47      1  fail_inconsistent\n31966249     36004.72      1  fail_inconsistent\n31966250     36010.12      1  fail_inconsistent\n31966251     36016.95      1  fail_inconsistent\n31966252     36000.58      1  fail_inconsistent\n31966253     36021.44      1  fail_inconsistent\n31966254     36010.16      1  fail_inconsistent\n31966255     36015.02      1  fail_inconsistent\n31966256     36000.52      1  fail_inconsistent\n31966257     36025.40      1  fail_inconsistent\n31966258     36012.98      1  fail_inconsistent\n31966259     36002.78      1  fail_inconsistent\n31966260     36007.17      1  fail_inconsistent\n31966261     36000.00      1  fail_inconsistent\n31966262     36000.67      1  fail_inconsistent\n31966263     36002.23      1  fail_inconsistent\n31966264     36011.77      1  fail_inconsistent\n31966265     36001.57      1  fail_inconsistent\n31966266     36002.57      1  fail_inconsistent\n31966267     36000.01      1  fail_inconsistent\n31966268     36000.20      1  fail_inconsistent\n31966269     36000.01      1  fail_inconsistent\n31966270     36000.01      1  fail_inconsistent\n31966271     36000.01      1  fail_inconsistent\n31966272     36000.00      1  fail_inconsistent\n31966273     36000.01      1  fail_inconsistent\n31966274     36000.27      1  fail_inconsistent\n31966275     36000.01      1  fail_inconsistent\n31966276     36000.01      1  fail_inconsistent\n31966277     36000.01      1  fail_inconsistent\n31966278     36000.01      1  fail_inconsistent\n31966279     36000.04      1  fail_inconsistent\n31966280     36000.01      1  fail_inconsistent\n31966281     36000.01      1  fail_inconsistent\n31966282     36017.63      1  fail_inconsistent\n31966283     36000.01      1  fail_inconsistent\n31966284     36011.14      1  fail_inconsistent\n31966285     36000.01      1  fail_inconsistent\n31966286     36005.53      1  fail_inconsistent\n31966287     36000.01      1  fail_inconsistent\n31966288     36012.31      1  fail_inconsistent\n31966289     36008.58      1  fail_inconsistent\n31966290     36013.95      1  fail_inconsistent\n31966291     36009.99      1  fail_inconsistent\n31966292     36001.28      1  fail_inconsistent\n31966293     36000.01      1  fail_inconsistent\n31966294     36013.66      1  fail_inconsistent\n31966295     36000.02      1  fail_inconsistent\n31966296     36010.50      1  fail_inconsistent\n31966297     36000.02      1  fail_inconsistent\n31966298         2.53      1                 ok\n31966299         2.54      1                 ok\nAggregated Results:\n              _time_ _limit_ _primfail_ _dualfail_ _fail_ _abort_ _solved_ _unkn_ _count_ _miss_  Time_shmean(1.0)  Nodes_shmean(100.0)\nGroup                                                                                                                                  \nall                0       0          0          0     59       0        2      0      61      0      26604.848145                  1.0\nalloptimal         0       0          0          0      0       0        2      0       2      0          2.534996                  1.0\neasyinstances      0       0          0          0      0       0        2      0       2      0          2.534996                  1.0\n```\n\n## Configuration\n\nUser-defined configuration files are stored in the `~/.ipet` folder, that is considered in the parsing process.\nThis includes\n\n- `~/.ipet/solvers`, where the user can define their own solver model. For an example check the `SCIPSolver` class in `ipet/parsing/Solver.py`\nAdditionally you need to place a `__init__.py` file in the `~/.ipet/solvers/` folder that loads the python modules (for example you have to classes `MySolver` and `OtherSolver` in files `~/.ipet/solvers/MySolver.py` and `~/.ipet/solvers/OtherSolver.py`):\n\n        __all__ = []\n        from .MySolver import MySolver\n        from .OtherSolver import OtherySolver\n\n- `~/.ipet/readers`, where the user can define their own rules for extracting data by giving a line, position and format of the number or string they want to parse from the logfile(s). For an example check `scripts/readers-example.xml`.\n- `~/.ipet/solufiles`, that contains solution files with information about the correct solution of instance files or their (inf)feasibility status. For an example check `test/data/short.solu`\n- \n## Starting the graphical user interface\n\nIPET has a subdirectory called \"scripts\" with scripts to invoke log file parsing, test run evaluating, and starting\nthe graphical user interface.\n\n# Building the documentation\n\nIn your virtual environment type:\n\n    pip install sphinx\n    cd doc\n    make html\n\nyour documentation will then be located in doc/build/html/index.html.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscipopt%2Fipet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscipopt%2Fipet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscipopt%2Fipet/lists"}