{"id":20744995,"url":"https://github.com/incquerylabs/magicdraw-viatra-benchmark","last_synced_at":"2025-06-16T11:07:29.242Z","repository":{"id":56339985,"uuid":"98527786","full_name":"IncQueryLabs/magicdraw-viatra-benchmark","owner":"IncQueryLabs","description":null,"archived":false,"fork":false,"pushed_at":"2020-11-13T14:31:09.000Z","size":22289,"stargazers_count":0,"open_issues_count":7,"forks_count":2,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-01-18T02:40:52.008Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IncQueryLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-27T11:23:33.000Z","updated_at":"2020-11-13T12:50:01.000Z","dependencies_parsed_at":"2022-08-15T17:00:14.142Z","dependency_job_id":null,"html_url":"https://github.com/IncQueryLabs/magicdraw-viatra-benchmark","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IncQueryLabs%2Fmagicdraw-viatra-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IncQueryLabs%2Fmagicdraw-viatra-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IncQueryLabs%2Fmagicdraw-viatra-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IncQueryLabs%2Fmagicdraw-viatra-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IncQueryLabs","download_url":"https://codeload.github.com/IncQueryLabs/magicdraw-viatra-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243039610,"owners_count":20226131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T07:17:57.042Z","updated_at":"2025-03-11T12:45:49.086Z","avatar_url":"https://github.com/IncQueryLabs.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The MagicDraw VIATRA Query performance benchmark\n\nThis benchmark is intended to showcase the performance of the [VIATRA](http://eclipse.org/viatra) [Query](https://wiki.eclipse.org/VIATRA/Query) engine when run directly on MagicDraw SysML models, through the EMF interface.\n\n## Getting Started\n\n### Cloning this repository\n\nUse the green \"Clone or download\" link on the GitHub web to clone the repository and get the source.\n\nFrom now on, the folder that contains this README.md file on your computer is called `WORKSPACE_BENCHMARK`\n\n```bash\nexport WORKSPACE_BENCHMARK`=c:/git/magicdraw-viatra-benchmark\n```\n\n### Get the instance models\n\n\n\nYou need to get a set of instance models (`mdzip` files) that the benchmark runs on.\n\nFrom now on, the folder that contains models on your computer is called `MODEL_LOCATION` \n\n```bash\nexport MODEL_LOCATION`=c:/models-tmt/\n```\n\nYou can download the models from this link: http://static.incquerylabs.com/projects/magicdraw/benchmark/models/models-tmt.zip\n\n### Running the benchmark\n\nAfter you clone this repository, you can run the benchmark by executing the following `./benchmark.sh` in the `WORKSPACE_BENCHMARK` folder:\n\nIn addition, you need to have Gradle and Java installed to run the build.\nThe results of the benchmark will be available in the `com.incquerylabs.benchmark.test/results` folder.\n\n### Create diagrams from results\n\nReporting is done using the [MONDO-SAM](https://github.com/FTSRG/mondo-sam/) tool, we are using the 0.1-maintenance branch.\nMONDO-SAM requires [R](https://www.r-project.org/) and [Python 3](https://www.python.org/) to be installed and available in your PATH.\nYou can find information on how to setup MONDO-SAM on their repository.\n\nThe following script downloads MONDO-SAM, converts the results from `com.incquerylabs.benchmark.test/results/\u003cquery\u003e` to `benchmark/results` then generates the diagrams to `benchmark/diagrams`:\n\nYou need to set the `WORKSPACE_BENCHMARK` environment variable to the repository root.\n\n### Configuring the benchmark\n\nBy default, the benchmark runs on a very small subset of queries, with the smallest model and only once per measurement.\nThis behaviour is configured by four variables and affects how `com.incquerylabs.benchmark.test/run.sh` behaves:\n\nThe name of the variables and their possible values are the following:\n* `BENCHMARK_ENGINES`: Which query engine implementations to measure\n  * Default (all possible values): `RETE, LOCAL_SEARCH, LOCAL_SEARCH_HINTS-CONDITION_FIRST, LOCAL_SEARCH_HINTS-TC_FIRST, HYBRID`\n* `BENCHMARK_QUERIES`: Which queries to measure - `all` is a special value to run the queries at once in the same MagicDraw instance\n  * Default: `transitiveSubstatesWithCheck3`\n  * Possible values: `all, blocksOrRequirementsOrConstraints, alphabeticalDependencies, circularDependencies, loopTransitionWithTriggerEffectEventNoGuard, stateWithMostSubstates, transitiveSubstatesWithCheck3, allBenchMarkedQueries`\n* `BENCHMARK_QUERIES_EXCLUDE`: Which queries to exclude from measure\n  * Default: ``\n  * Possible values: `blocksOrRequirementsOrConstraints, alphabeticalDependencies, circularDependencies, loopTransitionWithTriggerEffectEventNoGuard, stateWithMostSubstates, transitiveSubstatesWithCheck3, allBenchMarkedQueries`\n* `BENCHMARK_SIZES`: Which models to measure\n  * Default: `5000`\n  * Possible values: `300000, 540000, 780000, 1040000, 1200000`\n* `BENCHMARK_RUNS`: How many times the measurements should be run for each configuration\n  * Default: `1`\n  \nYou can either set this in environment variables before running the benchmark or modify the `run.sh` script directly.\n\n\u003c!-- end of Getting started (only edit outside of section, please) --\u003e\n\n## Repository structure\n* [Benchmark plugin](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/tree/master/com.incquerylabs.magicdraw.benchmark/src/com/incquerylabs/magicdraw/benchmark)\n  * [Benchmark queries](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/tree/master/com.incquerylabs.magicdraw.benchmark/src/com/incquerylabs/magicdraw/benchmark/queries) \n  * [Benchmark plugin entry point](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/blob/master/com.incquerylabs.magicdraw.benchmark/src/com/incquerylabs/magicdraw/benchmark/MagicDrawPerformancePlugin.java)\n  * [Benchmark execution script](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/blob/master/com.incquerylabs.magicdraw.benchmark/run.sh)\n* [Report generation](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/tree/master/benchmark)\n\n## [Benchmark details and results wiki page](https://github.com/IncQueryLabs/magicdraw-viatra-benchmark/wiki/MagicDraw-VIATRA-benchmark-results)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fincquerylabs%2Fmagicdraw-viatra-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fincquerylabs%2Fmagicdraw-viatra-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fincquerylabs%2Fmagicdraw-viatra-benchmark/lists"}