{"id":34100112,"url":"https://github.com/instamatic-dev/edtools","last_synced_at":"2025-12-14T16:40:50.911Z","repository":{"id":37452870,"uuid":"155187970","full_name":"instamatic-dev/edtools","owner":"instamatic-dev","description":"Collection of tools for automated processing and clustering of electron diffraction data","archived":false,"fork":false,"pushed_at":"2025-05-22T15:11:19.000Z","size":472,"stargazers_count":12,"open_issues_count":0,"forks_count":10,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-09-04T19:53:43.357Z","etag":null,"topics":["cluster-analysis","electron-diffraction","xds"],"latest_commit_sha":null,"homepage":"https://edtools.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/instamatic-dev.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"citation.CFF","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-10-29T09:49:12.000Z","updated_at":"2025-07-21T08:08:21.000Z","dependencies_parsed_at":"2023-01-21T12:00:12.427Z","dependency_job_id":"d269056f-dd67-4c2f-a453-3d16278ea80d","html_url":"https://github.com/instamatic-dev/edtools","commit_stats":{"total_commits":142,"total_committers":9,"mean_commits":"15.777777777777779","dds":0.7183098591549295,"last_synced_commit":"4e906bebe7c3b83c45d56f92f4e04a857c1ef010"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/instamatic-dev/edtools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instamatic-dev%2Fedtools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instamatic-dev%2Fedtools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instamatic-dev%2Fedtools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instamatic-dev%2Fedtools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/instamatic-dev","download_url":"https://codeload.github.com/instamatic-dev/edtools/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instamatic-dev%2Fedtools/sbom","scorecard":{"id":489998,"data":{"date":"2025-08-11","repo":{"name":"github.com/instamatic-dev/edtools","commit":"0b9e3fa9707ec9554123b0e865f462d56c0f1517"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.1,"checks":[{"name":"Maintained","score":2,"reason":"2 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 1/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/publish.yml:1","Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish.yml:54: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish.yml:58: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/publish.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish.yml:69: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/test.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/instamatic-dev/edtools/test.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/publish.yml:19","Warn: pipCommand not pinned by hash: .github/workflows/publish.yml:34","Warn: pipCommand not pinned by hash: .github/workflows/publish.yml:64","Warn: pipCommand not pinned by hash: .github/workflows/publish.yml:65","Info:   0 out of   6 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   4 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENCE:0","Info: FSF or OSI recognized license: BSD 3-Clause \"New\" or \"Revised\" License: LICENCE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Vulnerabilities","score":0,"reason":"15 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2018-34 / GHSA-2fc2-6r4j-p65h","Warn: Project is vulnerable to: PYSEC-2021-856 / GHSA-5545-2q6w-2gh6","Warn: Project is vulnerable to: PYSEC-2019-108 / GHSA-9fq2-x9r6-wfmf","Warn: Project is vulnerable to: PYSEC-2018-33 / GHSA-cw6w-4rcx-xphc","Warn: Project is vulnerable to: PYSEC-2021-857 / GHSA-f7c7-j99h-c22f","Warn: Project is vulnerable to: GHSA-fpfv-jqm9-f5jm","Warn: Project is vulnerable to: PYSEC-2017-1 / GHSA-frgw-fgh6-9g52","Warn: Project is vulnerable to: PYSEC-2020-73","Warn: Project is vulnerable to: PYSEC-2021-142 / GHSA-8q59-q68h-6hv4","Warn: Project is vulnerable to: PYSEC-2018-49 / GHSA-rprw-h62v-c2w7","Warn: Project is vulnerable to: GHSA-xgfm-fjx6-62mj","Warn: Project is vulnerable to: PYSEC-2019-156 / GHSA-xp76-357g-9wqq","Warn: Project is vulnerable to: PYSEC-2023-102","Warn: Project is vulnerable to: PYSEC-2023-114","Warn: Project is vulnerable to: GHSA-6p56-wp2h-9hxr"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/publish.yml:45"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 6 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-19T18:50:39.286Z","repository_id":37452870,"created_at":"2025-08-19T18:50:39.286Z","updated_at":"2025-08-19T18:50:39.286Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27731469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-14T02:00:11.348Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-analysis","electron-diffraction","xds"],"created_at":"2025-12-14T16:40:50.327Z","updated_at":"2025-12-14T16:40:50.905Z","avatar_url":"https://github.com/instamatic-dev.png","language":"Python","readme":"[![build](https://github.com/instamatic-dev/edtools/actions/workflows/test.yml/badge.svg)](https://github.com/instamatic-dev/edtools/actions/workflows/test.yml)\r\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/edtools)](https://pypi.org/project/edtools/)\r\n[![PyPI](https://img.shields.io/pypi/v/edtools.svg?style=flat)](https://pypi.org/project/edtools/)\r\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/edtools)](https://pypi.org/project/edtools/)\r\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5727188.svg)](https://doi.org/10.5281/zenodo.5727188)\r\n\r\n# edtools\r\n\r\nCollection of tools for automated processing and clustering of batch 3-dimensional electron diffraction (3D ED) datasets.\r\n\r\n[The source for this project is available here][src].\r\n\r\n[src]: https://github.com/instamatic-dev/edtools\r\n\r\n## Installation\r\n\r\nInstall using `pip install edtools`. Installation should take less than 20 seconds on a normal desktop.\r\n\r\nFind the latest [releases](https://github.com/instamatic-dev/edtools/releases) for the versions that have been tested on.\r\n\r\n## OS Requirement\r\n\r\nWindows 10 or newer.\r\n\r\n## Software Requirements\r\n\r\n- Python 3.6+ including `numpy`, `scipy`, `matplotlib`, and `pandas` libraries\r\n- [`sginfo`](https://github.com/rwgk/sginfo) or [`cctbx.python`](https://cctbx.github.io/installation.html#installation) must be available on the system path for `edtools.make_shelx`\r\n- Access to [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux)\r\n- XDS package must be installed properly under WSL\r\n\r\n## Package dependencies\r\n\r\nCheck [pyproject.toml](pyproject.toml) for the full dependency list and versions.\r\n\r\n## Documentation\r\n\r\nSee the documentation at https://edtools.readthedocs.io.\r\n\r\n## Pipeline tools\r\n\r\nAt any step, run *edtools.xxx -h* for help with possible arguments.\r\n\r\n### autoindex.py\r\n\r\nLooks for files matching `XDS.INP` in all subdirectories and runs them using `XDS`.\r\n\r\n\tIn:  XDS.INP\r\n\tOut: XDS data processing on all files\r\n\r\nUsage:\r\n\r\n```\r\nedtools.autoindex\r\n```\r\n\r\n### extract_xds_info.py\r\n\r\nLooks files matching `CORRECT.LP` in all subdirectories and extracts unit cell/integration info. Summarizes the unit cells in the excel file `cells.xlsx` and `cells.yaml`. XDS_ASCII.HKL files matching the completeness / CC(1/2) criteria are listed in `filelist.txt`. Optionally, gathers the corresponding `XDS_ASCII.HKL` files in the local directory. The `cells.yaml` file can be used as input for further processing.\r\n\r\n\tIn:  CORRECT.LP\r\n\tOut: cells.yaml\r\n\t     cells.xlsx\r\n\t     filelist.txt\r\n\r\nUsage:\r\n\r\n```\r\nedtools.extract_xds_info\r\n```\r\n\r\n### find_cell.py\r\n\r\nThis program a cells.yaml file and shows histogram plots with the unit cell parameters. This program mimicks [`CELLPARM`](http://xds.mpimf-heidelberg.mpg.de/html_doc/cellparm_program.html) and calculates the weighted mean lattice parameters, where the weight is typically the number of observed reflections (defaults to 1.0). For each lattice parameter, the mean is calculated in a given range (default range = median+-2). The range can be changed by dragging the cursor on the histogram plots.\r\n\r\nAlternatively, the unit cells can be clustered by giving the `--cluster` command, in which a dendrogram is shown. The cluster cutoff can be selected by clicking in the dendrogram. The clusters will be written to `cells_cluster_#.yaml`.\r\n\r\n\tIn:  cells.yaml\r\n\tOut: mean cell parameters\r\n\t     cells_*.yaml (clustering only)\r\n\r\nUsage:\r\n\r\n```\r\nedtools.find_cell cells.yaml --cluster\r\n```\r\n\r\n### make_xscale.py\r\n\r\nPrepares an input file `XSCALE.INP` for `XSCALE` and corresponding `XDSCONV.INP` for `XDSCONV`. Takes a `cells.yaml` file or a series of `XDS_ASCII.HKL` files as input, and uses those to generate the `XSCALE.INP` file.\r\n\r\n\tIn:  cells.yaml / XDS_ASCII.HKL\r\n\tOut: XSCALE.INP\r\n\r\nUsage:\r\n\r\n```\r\nedtools.make_xscale cells.yaml -c 10.0 20.0 30.0 90.0 90.0 90.0 -s Cmmm\r\n```\r\n\r\n### cluster.py\r\n\r\nParses the `XSCALE.LP` file for the correlation coefficients between reflection files to perform hierarchical cluster analysis (Giordano et al., Acta Cryst. (2012). D68, 649–658). The cutoff threshold can be selected by clicking in the dendrogram window. The program will write new `XSCALE.LP` files to subdirectories `cluster_#`, and run `XSCALE` on them, and (if available), pointless.\r\n\r\n\tIn:  XSCALE.LP\r\n\tOut: cluster_n/\r\n\t\tfilelist.txt\r\n\t\t*_XDS_ASCII.HKL\r\n\t\tXSCALE processing\r\n\t\tPointless processing\r\n\t\tshelx.hkl\r\n\t\tshelx.ins (optional)\r\n\r\nUsage:\r\n\r\n```\r\nedtools.cluster\r\n```\r\n\r\n\r\n## Helper tools\r\n\r\n### make_shelx.py\r\n\r\nCreates a shelx input file. Requires `sginfo` to be available on the system path to generate the SYMM/LATT cards.\r\n\r\n\tIn:  cell, space group, composition\r\n\tOut: shelx.ins\r\n\r\nUsage:\r\n\r\n```\r\nedtools.make_shelx -c 10.0 20.0 30.0 90.0 90.0 90.0 -s Cmmm -m Si180 O360\r\n```\r\n\r\n### run_pointless.py\r\n\r\nLooks for XDS_ASCII.HKL files specified in the cells.yaml, or on the command line and runs Pointless on them.\r\n\r\n\tIn:  cells.yaml / XDS_ASCII.HKL\r\n\tOut: Pointless processing\r\n\r\n### update_xds.py\r\n\r\nLooks files matching `CORRECT.LP` in all subdirectories, and updates the cell parameters / space group as specified.\r\n\r\n\tIn:  XDS.INP\r\n\tOut: XDS.INP\r\n\r\nUsage:\r\n\r\n```\r\nedtools.update_xds -c 10.0 20.0 30.0 90.0 90.0 90.0 -s Cmmm\r\n```\r\n\r\n### find_rotation_axis.py\r\n\r\nFinds the rotation axis and prints out the inputs for several programs (XDS, PETS, DIALS, Instamatic, and RED). Implements the algorithm from Gorelik et al. (Introduction to ADT/ADT3D. In Uniting Electron Crystallography and Powder Diffraction (2012), 337-347). The program reads `XDS.INP` to get information about the wavelength, pixelsize, oscillation angle, and beam center, and `SPOT.XDS` (generated by COLSPOT) for the peak positions. If the `XDS.INP` file is not specified, the program will try to look for it in the current directory.\r\n\r\n\tIn:  XDS.INP, SPOT.XDS\r\n\tOut: Rotation axis\r\n\r\nUsage:\r\n\r\n```\r\nedtools.find_rotation_axis [XDS.INP]\r\n```\r\n\r\n## Demo of using edtools to process batch 3D electron diffraction datasets\r\n\r\nSee the demo at https://edtools.readthedocs.io/en/latest/examples/edtools_demo.html.\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstamatic-dev%2Fedtools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finstamatic-dev%2Fedtools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstamatic-dev%2Fedtools/lists"}