{"id":22901260,"url":"https://github.com/mooerslab/aimless-summary-table-extraction","last_synced_at":"2025-04-01T05:41:55.691Z","repository":{"id":263858406,"uuid":"891611192","full_name":"MooersLab/aimless-summary-table-extraction","owner":"MooersLab","description":"X-ray diffraction data processing: extract summary table in aimless log file","archived":false,"fork":false,"pushed_at":"2024-11-20T18:31:19.000Z","size":25,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-07T03:41:57.055Z","etag":null,"topics":["agile-biomolecular-crystallography","agile-crystallographic-data-processing","agile-diffraction-image-data-processing","aimless-summary","aimless-summary-table","autoxds-post-processing","bash-function","crystallographic-computing","x-ray-diffraction-data-processing"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MooersLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-20T16:30:27.000Z","updated_at":"2024-11-20T18:31:23.000Z","dependencies_parsed_at":"2024-11-20T17:36:48.345Z","dependency_job_id":"ad4dbcb8-b5ee-43ef-a5c4-1a988131c9ee","html_url":"https://github.com/MooersLab/aimless-summary-table-extraction","commit_stats":null,"previous_names":["mooerslab/aimless-summary-table-extraction"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2Faimless-summary-table-extraction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2Faimless-summary-table-extraction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2Faimless-summary-table-extraction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2Faimless-summary-table-extraction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MooersLab","download_url":"https://codeload.github.com/MooersLab/aimless-summary-table-extraction/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246591782,"owners_count":20801984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agile-biomolecular-crystallography","agile-crystallographic-data-processing","agile-diffraction-image-data-processing","aimless-summary","aimless-summary-table","autoxds-post-processing","bash-function","crystallographic-computing","x-ray-diffraction-data-processing"],"created_at":"2024-12-14T01:32:34.193Z","updated_at":"2025-04-01T05:41:55.646Z","avatar_url":"https://github.com/MooersLab.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Version](https://img.shields.io/static/v1?label=aimless-summary-table-extraction\u0026message=0.0\u0026color=brightcolor)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\n# Extract summary table from aimless.log file\n\n## What is this?\n\nA bash function that eases access to the summary statistics in the aimless.log file after processing with `autoxds` on the Structural Molecular Biology (SMB) server at the Stanford Synchrotron Radiation LightSource (SSRL).\nThe script `autoxds` by Dr. Ana Gonzalez is only available at SSRL.\nThe script wraps around the programs `XDS`, `pointless`, `aimless`, and `ctruncate` to ease the processing of diffraction images.\n\nThe bash function takes the diffraction image file stem and the run number as command line arguments.\nThe output files are assumed to be in a subfolder.\nThe function summary() prints the summary table to the terminal and a file for further formatting.\n\n## The problem\n\nThe summary table in the aimless.log file from the scaling and merging of X-ray diffraction images provides an overview from which to decide whether to reprocess the diffraction images with a different high-resolution limit.\nAccessing this summary table is simple using the search facility in Vim.\nIt is more laborious to use other text editors.\n\nHowever, loading the file into Vim and entering the command takes time.\nI sought a faster approach using the command line when processing many datasets.\nFor example, when testing different resolution limits, the up arrow key can rerun the bash function.\nThe command line dominates this kind of iterative work.\n\n## The solution\n\nA bash function that is sourced on log in to a bash shell and thus always ready.\n\n## Installation\n\n1. Copy the contents of summary.sh into a .bashFunctions file in your home directory.\n2. Source .bashFunctions from .bash_profile file.\n3. Invoke the bash shell by entering `bash`, if not already in bash. This function works in zsh shell.\n\n## Usage\n1. Navigate to the folder with the images and the `autoxds` output subfolders.\n1. Enter `summary \u003cimage filestem\u003e \u003crun number\u003e`. The output files are assumed to be in a subfolder with the following syntax for the file name `\u003cfilestem\u003e_\u003crun-number\u003e_xds`\n2. The `more` file viewer displays the extracted table in the terminal. Enter `q` to escape this viewer. Enter the spacebar to advance. Enter `b` to go in reverse.\n\n### Example of use\n\nThe image file stem is `3161`.\nThe aimless log file resides in the subfolder `3161_1_xds`.\nThe aimless log file name is `3161_1_aimless.log`.\n\n```bash\nsummary 3161 1\n```\n\nEasy-peasy!\n\n## Example of extracted table\n\n\n\n```bash\n\u003c!--SUMMARY_BEGIN--\u003e $TEXT:Result: $$ $$\nSummary data for        Project: XDSproject Crystal: XDScrystal Dataset: XDSdataset\n\n                                           Overall  InnerShell  OuterShell\nLow resolution limit                       32.87     32.87      1.36\nHigh resolution limit                       1.34      7.34      1.34\n\nRmerge  (within I+/I-)                     0.028     0.023     1.064\nRmerge  (all I+ and I-)                    0.029     0.023     1.124\nRmeas (within I+/I-)                       0.031     0.025     1.190\nRmeas (all I+ \u0026 I-)                        0.030     0.025     1.191\nRpim (within I+/I-)                        0.013     0.011     0.526\nRpim (all I+ \u0026 I-)                         0.010     0.010     0.386\nRmerge in top intensity bin                0.022        -         - \nTotal number of observations               96519       613      4140\nTotal number unique                         9795        80       461\nMean((I)/sd(I))                             24.2      66.0       1.7\nMn(I) half-set correlation CC(1/2)         1.000     0.999     0.765\nCompleteness                                99.7      99.4      97.9\nMultiplicity                                 9.9       7.7       9.0\nMean(Chi^2)                                 0.89      0.82      0.77\n\nAnomalous completeness                      99.6     100.0      94.5\nAnomalous multiplicity                       5.2       5.6       4.8\nDelAnom correlation between half-sets     -0.184     0.200    -0.001\nMid-Slope of Anom Normal Probability       0.914       -         -  \n\nThe anomalous signal appears to be weak so anomalous flag was left OFF\n\nEstimates of resolution limits: overall\n   from half-dataset correlation CC(1/2) \u003e  0.30: limit =  1.34A  == maximum resolution\n   from Mn(I/sd) \u003e  1.50:                         limit =  1.34A  == maximum resolution\n   from Mn(I/sd) \u003e  2.00:                         limit =  1.38A \n\nEstimates of resolution limits in reciprocal lattice directions:\n  Along h k plane\n   from half-dataset correlation CC(1/2) \u003e  0.30: limit =  1.34A  == maximum resolution\n   from Mn(I/sd) \u003e  1.50:                         limit =  1.34A  == maximum resolution\n  Along l axis\n   from half-dataset correlation CC(1/2) \u003e  0.30: limit =  1.34A  == maximum resolution\n   from Mn(I/sd) \u003e  1.50:                         limit =  1.37A \n\nAnisotropic deltaB (i.e. range of principal components), A^2:  1.94\n\nAverage unit cell:    38.99   38.99  143.00   90.00   90.00  120.00\nSpace group: H 3 2\nAverage mosaicity:   0.20\n\nMinimum and maximum SD correction factors: Fulls   0.27   2.82 Partials   0.00   0.00\n\n$$ \u003c!--SUMMARY_END--\u003e\n```\n\nLooks like I need to reprocess with a dmin of 1.333 Angstroms to get a  Mn(I/sd) ~  1.5.\n\n## Update history\n\n|Version      | Changes                                                                                                                                                                         | Date                 |\n|:-----------|:------------------------------------------------------------------------------------------------------------------------------------------|:--------------------|\n| Version 0.1 |   Added badges, funding, and update table.  Initial commit.                                                                                                                | 2024 November 20  |\n\n## Sources of funding\n\n- NIH: R01 CA242845\n- NIH: R01 AI088011\n- NIH: P30 CA225520 (PI: R. Mannel)\n- NIH: P20 GM103640 and P30 GM145423 (PI: A. West)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmooerslab%2Faimless-summary-table-extraction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmooerslab%2Faimless-summary-table-extraction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmooerslab%2Faimless-summary-table-extraction/lists"}