{"id":30254115,"url":"https://github.com/binaryeq/daleq","last_synced_at":"2025-08-15T14:02:59.794Z","repository":{"id":308059014,"uuid":"901601922","full_name":"binaryeq/daleq","owner":"binaryeq","description":"datalog-based binary equivalence","archived":false,"fork":false,"pushed_at":"2025-08-10T21:36:09.000Z","size":1394,"stargazers_count":10,"open_issues_count":9,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-10T23:35:03.654Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/binaryeq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-11T01:03:07.000Z","updated_at":"2025-08-10T21:36:12.000Z","dependencies_parsed_at":"2025-08-03T23:20:59.745Z","dependency_job_id":null,"html_url":"https://github.com/binaryeq/daleq","commit_stats":null,"previous_names":["binaryeq/daleq"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/binaryeq/daleq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binaryeq%2Fdaleq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binaryeq%2Fdaleq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binaryeq%2Fdaleq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binaryeq%2Fdaleq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/binaryeq","download_url":"https://codeload.github.com/binaryeq/daleq/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binaryeq%2Fdaleq/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270579613,"owners_count":24610044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-15T14:01:23.505Z","updated_at":"2025-08-15T14:02:59.774Z","avatar_url":"https://github.com/binaryeq.png","language":"HTML","funding_links":[],"categories":["HTML"],"sub_categories":[],"readme":"# DALEQ - Datalog-based Binary Equivalence\n\n## Publications\n\n1. [Jens Dietrich, Behnaz Hassanshahi: DALEQ - Explainable Equivalence for Java Bytecode (preprint)](https://arxiv.org/abs/2508.01530)\n2. [Jens Dietrich, Tim White, Behnaz Hassanshahi, Paddy Krishnan: Levels of Binary Equivalence for the Comparison of Binaries from Alternative Builds. ICSME'25](https://arxiv.org/abs/2410.08427)\n3. [Jens Dietrich, Tim White, Valerio Terragni, Behnaz Hassanshahi: Towards Cross-Build Differential Testing. ICST'25](https://www.researchgate.net/publication/391937833_Towards_Cross-Build_Differential_Testing)\n\n## Build and Use CLI\n\nDALEQ has been developed and tested with Java 17.\nDALEQ can be built with Maven as follows:\n\n`mvn clean package dependency:copy-dependencies`\n\nThen run DALEQ as follows (for windows: change the classpath separator accordingly):\n\n```\njava -cp \"target/classes:target/dependency/*\" io.github.bineq.daleq.cli.Main \\\n\n -j1,--jar1 \u003carg\u003e   the first jar file to compare (required)\n -j2,--jar2 \u003carg\u003e   the second jar file to compare (required)\n -o,--out \u003carg\u003e     the output folder where the report will be generated (required)\n -s1,--src1 \u003carg\u003e   the first jar file with source code to compare (optional)\n -s2,--src2 \u003carg\u003e   the second jar file with source code to compare (optional)\n -a,--autoopen      if set, the generated html report will be opened automatically (optional, don't use this for CI integration)\n -dq,--daleq        one of {sound,soundy,both} (optional, default is soundy)\n```\n\nThe jar built can also be used directly as an executable superjar.\n\n```\njava -jar target/daleq-\u003cdaleq-version\u003e.jar  \u003coptions-as-before\u003e\n```\n\nRunning the program will create a report `report.html` in the specified output folder.\n\nThe program returns with the following exit codes:\n\n```\n0 - all classes and resources are equal, source codes (if analysed) are equivalent\n1 - some classes are not equal but still equivalent, resources are equal, source-codes (if analysed) are equivalent\n2 - some classes are not equivalent (wrt daleq) or some resources (e.g. metadata) are not equal or some sources (if analysed) are not equivalent\n```\n\nThis can be used to integrate the tool into CI processes.\n\n### Locating Souffle\n\n*Daleq* will try to find the *souffle* executable in the path.\nThis can be overridden by passing it as a JVM argument, i.e. `-DSOUFFLE=\u003cpath-to-souffle\u003e`.\n\n## Overview\n\n*Daleq* takes Java bytecode as input and produces a relational database.\nThis is done in two steps.\n\n### Step 1 - EDB Extraction\n\n```mermaid\nflowchart LR\n    bytecode[\"bytecode (.class)\"] --\u003e asm((\"asm\")) --\u003e edb[\"EDB\"] --\u003e souffle((\"souffle\")) --\u003e idb[\"IDB\"]\n```\n\nA database representing the bytecode is extracted. An [asm-based](https://asm.ow2.io/) static analysis is used for this purpose.\nThe result is the EDB (extensional database), a folder with tsv files, each tsv file corresponding to a predicate.\nThose facts represent the bytecode, including class name, bytecode version, super types and interfaces,\naccess flags for method, fields and classes, bytecode instructions in methods, etc.\n\nThe representation is low-level, however, there is one significant abstraction (provided by _asm_): constant pool references are\nresolved.\n\nEach fact has a unique generated id that is used to provide provenance.\n\n### Step 2 - IDB Computation\n\nThe IDB computation applies rules to compute a second set of facts (IDB files) that normalises\nthe EBD to establish whether two bytecode compared are equivalent or not.\n\nRules that are applied for this purpose are defined in\n`src/main/resources/rules/`, `advanced.souffle` is the default set.\n\nThe API to generate the IDB is provided by `io.github.bineq.daleq.Souffle::createIDB`.\n\n### Step 3 - Comparing Bytecode\n\nWhen trying to establish equivalence between two (compiled) classes, IDBs cannot be compared directly for the following reasons:\n\n1. The provenance (id) terms -- this reflects the process of normalisation that might be different for each of the classes compared\n2. the IDB contains additional facts for predicates to keep track of facts that have been removed (i.e. EDB facts that have no counterpart in the IDB), or moved.\n3. facts corresponding to bytecode instruction have an `instructioncounter` slot to define the order of instructions within a method. Those might change in the IDB as normalisation may move/remove/insert facts corresponding to instructions.\n\n#### Projection\n\nThe purpose of *projection* is to remove the aforementioned parts (facts and slots) from the database.\nAt the moment normalisation is implemented in Java.\n\n#### Workflow to Compare two Binaries\n\nThe classes `io.github.bineq.daleq.idb.IDBReader` and `io.github.bineq.daleq.idb.IDBPrinter` are used to read an IDB,\nand convert it into a textual representation suitable for comparison and diffing (using a standard text fiff tool such as _diff_ or _kdiff3_). This also supports projection.\n\n```mermaid\nflowchart LR\n    bytecode1[\"class1\"] --\u003e asm1((\"asm\")) --\u003e edb1[\"EDB1\"] --\u003e souffle1((\"souffle\")) --\u003e idb1[\"IDB1\"] --\u003e IDBReader1((\"project\"))  --\u003e file1 --\u003e diff --\u003e result\n    bytecode2[\"class2\"] --\u003e asm2((\"asm\")) --\u003e edb2[\"EDB2\"] --\u003e souffle2((\"souffle\")) --\u003e idb2[\"IDB2\"] --\u003e IDBReader2((\"project\"))  --\u003e file2 --\u003e diff\n\n```\n\n## Normalisation Rules\n\nThe datalog rules being used can be found in `src/main/resources/rules/`.\nThe structure of the rules is as follows:  \n\n- `core.souffle` - mostly generated standard rules mapping EDB facts to IDB facts, plus some additional guard predicates that can be used to override default rules\n- `commons/access.souffle` - rules to derive facts for access flags such as *final* and *public* from access flags using bit encoding, for improved usability\n- `normalisations/sound` - normalisations that have no impact on program behaviour\n- `normalisations/soundy` - normalisations that are [soundy](http://soundiness.org/) in the sense that they might have some impact on program behaviour if the program uses reflection or similar dynamic programming techniques to inspect its own code. Those have been vetted for possible exploitability. For a conservative approach, those rules can be excluded from the analysis by using the _sound_ CLI option.\n\n## Provenance\n\nThe first slot (term) for each fact is an *id*.\nFor EDB facts, those ids are generated during the bytecode analysis / extracyion phase.\n\nExample (from `AALOAD.facts`):\n\n```csv\nF1428629\torg/apache/hadoop/hdfs/HAUtil::getConfForOtherNodes(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/List;\t3200\n```\n\nWhen the IDB is created, each IDB fact also has this slot.\nThe  values encode a derivation tree. Example:\n\n```csv\nR_AALOAD[F1428629]  ...\n```\n\nThis fact has been computed by applying rule `R_AALOAD` to fact `F1428629`.\nI.e. those ids encode a derivation tree:\n\n```mermaid\ngraph TD;\nR_AALOAD --\u003e F1428629;\n```\n\nA less trivial example is `R_42[F1,R_43[F2,F3]] ` , this encodes the following derivation tree:\n\n```mermaid\ngraph TD;\n    R_42 --\u003e F1;\n    R_42 --\u003e R_43;\n    R_43 --\u003e F2;\n    R_43 --\u003e F3;\n```\n\nThe project contains a parser to transform those strings into\ntrees (`io.github.bineq.daleq.souffle.provenance.ProvenanceParser`).\n\nThe grammar of the encoded proofs is defined here:\n`src/main/antlr4/io/github/bineq/daleq/souffle/provenance/Proof.g4`.\n\n\n\n\n## The Generated Report\n\nThis is an example of a generated report.  Here, we have compared the jar file for the artifact\n_javax.transaction:jta:1.1_ from Maven Central with the corresponding artifact rebuilt by Google's assured open source.\nDaleq employs various analysers to compare classes, meta-data and resources within the jars.\nThe results are displayed in a table, the rows correspond to files within the jar(s), and the columns correspond to the various analysers.\nAnalysers compare the content of files, specialised analysers are used to compare source code and bytecode. The actual daleq analyser results are displayed in the last column. Please have a look at our paper for more details.\n\nPossible analysis result states are:\n\n- **PASS** the comparison yields true\n- **FAIL** the comparison yields false\n- **N/A** the comparison cannot be applied (e.g., a source Code comparison cannot be used to compare bytecode)\n- **ERROR** the comparison has resulted in an error\n\n\u003cimg width=\"1200\" height=\"1584\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f840fb43-f55a-4dd2-a926-6bb4615296a0\" /\u003e\n\nThe markers next to each result are hyperlinks to generated pages with additional provenance.\nFor non-equivalence statements, those pages are usually diff reports rendered in HTML.\nFor daleq equivalence statements, advanced diff reports are generated based on the derivations recorded when datalog rules are applied.\nThis is an example:\n\n\u003cimg width=\"600\" alt=\"image\" src=\"https://github.com/user-attachments/assets/d1266e1f-858a-4546-a289-4c7b5fe90edc\" /\u003e\n\n\n## Constructing the EDB Only\n\n\n```\nusage: java -cp \u003cpath-to-built-jar\u003e io.github.bineq.daleq.edb.FactExtractor\n -cl \u003cclasses\u003e   the location of compiled classes, a jar file or folder\n -f \u003cfacts\u003e      a folder where to create the extension database (input\n                 .facts files), the folder will be created if it does not\n                 exist\n -s \u003csouffle\u003e    a file where to create the souffle program (.souffle\n                 file) containing imports and input predicate declarations\n -v              whether to verify the datalog facts against the schema,\n                 must be true or false, default is true\n```\n\n## Notes\n\n### Testing during Builds\n\nNote that tests requires souffle to be set up (see also **IDB Computation** section).\nThis has to be done locally at the moment, see also [issue 11](https://github.com/binaryeq/daleq/issues/11).\nTests that require souffle will not fail but will be skipped if souffle is not available.\n\n### Performance and Issues with Souffle\n\nSouffle [occasionally fails](https://github.com/binaryeq/daleq/issues/22).\nWhen souffle is used, a `Thread::sleep` instruction is used to pause DALEQ.\nThis reduces the number of errors (perhaps a souffle race condition?), but makes DALEQ slower.\n\nSouffle issues will be reported by the appplication, and flagged as __error__ .\n\n### Adding Your Own Analyser\n\nAdditional analysers can be easily added by following those steps:\n\n1. Write an analyser (say `MyAnalyser`) implementing `io.github.bineq.daleq.cli.Analyser`\n2. In your project, add a file `src/main/resources/META-INF/services/io.github.bineq.daleq.cli.Analyser`\n3. Add a line with the fully qualified name of `MyAnalyser` to this file\n4. Build a jar file\n5. When running DALEQ, add this jar file to the classpath\n6. Note that the order in which columns are displayed in reports is determined by the property `io.github.bineq.daleq.cli.Analyser::positionHint`, a values between 0 and 100.\n\n\n## Contributing\n\nTo contribute to this project, please follow the steps below to set up your development environment.\n\nFirst, create a virtual environment to isolate dependencies:\n\n```bash\npython3 -m venv .venv\n```\n\nActivate the virtual environment and install the `pre-commit` hooks.\n\n```bash\nsource .venv/bin/activate\npip install -r requirements.txt\npre-commit install\n```\n\nNow, the hooks will automatically run on each commit to check for any issues in the code. You can manually run the pre-commit hooks on all files at any time with:\n\n```bash\npre-commit run --all-files\n```\n\n## Sponsors\n\n\u003cimg width=\"200\" height=\"200\" alt=\"image\" src=\"https://github.com/user-attachments/assets/5a51f47a-d715-4a74-9346-09281e380202\" /\u003e\n\n[Oracle Labs Australia](https://labs.oracle.com/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinaryeq%2Fdaleq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbinaryeq%2Fdaleq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinaryeq%2Fdaleq/lists"}