{"id":18422514,"url":"https://github.com/sri-csl/occam-benchmarks","last_synced_at":"2025-10-13T11:06:02.090Z","repository":{"id":48939726,"uuid":"298387849","full_name":"SRI-CSL/OCCAM-Benchmarks","owner":"SRI-CSL","description":"Set of benchmarks used by the OCCAM tool.","archived":false,"fork":false,"pushed_at":"2022-05-26T02:18:04.000Z","size":2808,"stargazers_count":4,"open_issues_count":3,"forks_count":4,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-04-07T15:44:16.371Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SRI-CSL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-24T20:27:45.000Z","updated_at":"2023-12-12T07:57:50.000Z","dependencies_parsed_at":"2022-09-24T00:22:29.933Z","dependency_job_id":null,"html_url":"https://github.com/SRI-CSL/OCCAM-Benchmarks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SRI-CSL/OCCAM-Benchmarks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRI-CSL%2FOCCAM-Benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRI-CSL%2FOCCAM-Benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRI-CSL%2FOCCAM-Benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRI-CSL%2FOCCAM-Benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SRI-CSL","download_url":"https://codeload.github.com/SRI-CSL/OCCAM-Benchmarks/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRI-CSL%2FOCCAM-Benchmarks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279014751,"owners_count":26085593,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T04:30:37.496Z","updated_at":"2025-10-13T11:06:02.073Z","avatar_url":"https://github.com/SRI-CSL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reproducibility #\n\n1. OCCAM needs to generate bitcode from the source code.\nThis might not be easy and different platforms might require different workarounds. \n\n2. OCCAM links the debloated bitcode with other debloated libraries and extra native libraries indicated in the `native_libs` field from the Manifest.\nMissing a native library or adding a native library that is not installed will make the whole process to fail. \n\nThe directory `portfolio` and `trimmer` are mostly tested on an Ubuntu 18.04 machine. \nBut you need to make sure that all libraries passed in the Manifest as `native_libs` are installed in your machine.\n\n# Benchmark runner for [OCCAM](https://github.com/SRI-CSL/OCCAM) #\n\nTo run OCCAM on a set of benchmarks and show metrics, type:\n\n\trunbench.py --sets=\"portfolio.set\"\n\nIf you want to run more than one benchmark set then add more .set\nfiles in the option `--sets` separated by comma:\n\n\trunbench.py --sets=\"portfolio.set,spec2006.set\"\n\nIf you want to pass extra options to `slash` then type:\n\n\trunbench.py --sets=\"portfolio.set\" --slash-opts=\"--disable-inlining\"\n\nIf you want to pass multiple options by separating them by comma:\n\n\trunbench.py --sets=\"portfolio.set\" --slash-opts=\"--disable-inlining,--enable-config-prime\"\n\nIf you want to add a timeout or memory limit then type:\n\n\trunbench.py --sets=\"portfolio.set\" --cpu=60 --mem=4096\n\t\nBy default, `runbench.py` runs OCCAM on the set of benchmarks selected\nby option `--sets` and displays the number of functions, number of\ninstructions, etc. before and after the specialization takes\nplace. The option `--sets` is mandatory and expects a list of .set\nfiles separated by comma.\n\nThe options `--cpu` and `--mem` set limits on CPU (in seconds) and\nmemory (in MB) for running OCCAM. The compilation of the programs is\nunconstrained.\n\n`runbench.py` reads from the `*.set` files to select which benchmarks\nto run.  Each `*.set` file is in JSON format. For each benchmark,\nthere are at three fields: `dirname`, a relative path wrt environment\n`OCCAM_HOME` variable, `execname`, the name of the executable, and\n`enabled` whether the benchmark is enabled or not. That directory must\ncontain `Makefile` and `build.sh`. The makefile generates the bitcode\nfor the benchmark, and `build.sh` runs OCCAM on it. The assumption is\nthat after running `build.sh` two executables are generated: one with\nsuffix `_orig` and the other with suffix `_slashed`. The executables\nare used by option `--rop` so it is important to follow this\nconvention.\n\nFor instance, the output of `runbench.py` might look like this:\n\n```\nProgram Reduction: (B:before and A:after OCCAM)\n\nProgram    B Fun   A Fun   % Fun Red   B Ins    A Ins   % Ins Red   B Mem Ins   A Mem Ins   % Mem Ins Red\ntree         106      93          12    7409     8921         -20        1615        1186              26\nreadelf      384     281          26   83390   111227         -33        6117        7714             -26\nbzip2         92      41          55   22047    19055          13        4761        4303               9\nmcf           43      22          48    2592     2388           7         654         488              25\n```\n\n## Benchmark categories ## \n\n- `portfolio`: some real applications \n- `spec2006`: SPEC 2006 benchmarks\n- `coreutils`: coreutils benchmarks\n- `trimmer`: [Trimmer](https://github.com/ashish-gehani/trimmer) evaluation applications\n\n## Deprecated options which are not maintained anymore ##\n\nThe option `--rop` shows also the number of ROP, JOP, and SYS gadgets,\nbefore and after specialization.\n\n```\nGadget Reduction: (B:before and A:after OCCAM)\n\nProgram    B ROP   A ROP   % ROP Red   B SYS   A SYS   % SYS Red   B JOP   A JOP   % JOP Red\ntree         313     256          18       0       0           0      64      39          39\nreadelf     4007    3868           3       1       3        -200    2276     873          61\nbzip2        479     971        -102       0       0           0      60     140        -133\nmcf          168     115          31       0       0           0       0       0           0\n\n```\n\n`Red` means reduction. If the percentage is negative then it means\nthat there was an increase.\n\n\n \n---\n\nThis material is based upon work supported by the National Science Foundation under Grant [ACI-1440800](http://www.nsf.gov/awardsearch/showAward?AWD_ID=1440800). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsri-csl%2Foccam-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsri-csl%2Foccam-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsri-csl%2Foccam-benchmarks/lists"}