{"id":25895499,"url":"https://github.com/diamondlightsource/nxstacker","last_synced_at":"2025-10-30T05:48:19.843Z","repository":{"id":242522298,"uuid":"804810737","full_name":"DiamondLightSource/nxstacker","owner":"DiamondLightSource","description":"An utility to produce NeXus-compliance file from a stack","archived":false,"fork":false,"pushed_at":"2024-11-14T17:17:09.000Z","size":167,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-06T22:42:27.150Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DiamondLightSource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-23T10:10:38.000Z","updated_at":"2024-11-14T17:08:54.000Z","dependencies_parsed_at":"2024-06-03T14:54:55.447Z","dependency_job_id":"2f51facf-30ee-4fcc-ae9d-0d163e3cda7c","html_url":"https://github.com/DiamondLightSource/nxstacker","commit_stats":null,"previous_names":["diamondlightsource/nxstacker"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fnxstacker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fnxstacker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fnxstacker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fnxstacker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DiamondLightSource","download_url":"https://codeload.github.com/DiamondLightSource/nxstacker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241582515,"owners_count":19985845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-02T22:31:38.684Z","updated_at":"2025-10-30T05:48:19.837Z","avatar_url":"https://github.com/DiamondLightSource.png","language":"Python","readme":"# nxstacker\n\n*nxstacker* is an utility to stack projections from different types of\nexperiments to produce NeXus-compliance file(s), such as\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html) and\n[NXstxm](https://manual.nexusformat.org/classes/applications/NXstxm.html). It\ncurrently supports the beamlines i08-1, i13-1 and i14 in Diamond Light Source.\n\n## Installation\n\nAfter cloning the repository, you can install with pip:\n\n```console\npython -m pip install .\n```\n\n## Usage\n\nThe key function you would be interacting with is *tomojoin*. A command-line\ninterface (CLI) is also provided.\n\n### Get help\n\nFor ptycho-tomography,\n\n```console\ntomojoin ptycho --help\n```\n\nFor xrf-tomography,\n\n```console\ntomojoin xrf --help\n```\n\n### Get version\n\n```console\ntomojoin --version\n```\n\n### Get NXtomo file from ptycho-tomography experiment\n\n```python\nfrom nxstacker.tomojoin import tomojoin\n\nnxtomo_files = tomojoin(\n    \"ptychography\",\n    proj_dir=\"/i14/dir/projections/stored\",\n    nxtomo_dir=\"/tmp/\",\n    from_scan=\"275019-275199\",\n    save_phase=True,\n    save_modulus=True,\n    save_complex=True,\n    median_norm=True,\n    unwrap_phase=True,\n    remove_ramp=True,\n)\n```\n\nor via the CLI:\n\n```console\ntomojoin ptycho --proj-dir \"/i14/dir/projections/stored\" --nxtomo-dir \"/tmp\"\n--from-scan 275019-275199 --save-phase --save-modulus --save-complex\n--remove-ramp --median-norm --unwrap-phase\n```\n\nThis produces 3\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html) files\nfrom a ptycho-tomography experiment in *i14*: the complex representation, its\nmodulus and its phase. The successfully saved\nfiles are returned and stored in the `nxtomo_files` variable.\n\n#### Projection files following a particular naming pattern\n\nIf the projection files follow a particular naming pattern, e.g.\n\"/i14/dir/projections/stored/proj\\_275019.hdf5\",\n\"/i14/dir/projections/stored/proj\\_275020.hdf5\" and\n\"/i14/dir/projections/stored/proj\\_275021.hdf5\" etc., you can use the parameter\n`proj_file` with the placeholder `%(scan)`. This avoids going through every file\nin `proj_dir` with a lot of files.\n\n```python\nfrom nxstacker.tomojoin import tomojoin\n\nnxtomo_files = tomojoin(\n    \"ptychography\",\n    proj_file=\"/i14/dir/projections/stored/proj_%(scan).hdf5\",\n    ...\n)\n```\n\nor via the CLI (the double quotation marks are essential):\n\n```console\ntomojoin ptycho --proj-file \"/i14/dir/projections/stored/proj_%(scan).hdf5\" ...\n```\n\n#### Identifier specification\n\nThe `from_scan` string is of the format \\\u003cSTART\\\u003e[-\\\u003cEND\\\u003e[:\\\u003cSTEP\\\u003e]]:\n\n- \"100-105\" means 100, 101, 102, 103, 104, 105,\n\n- \"100-105:2\" means 100, 102, 104,\n\n- they can be chained, \"100-103,110,120-122\" means 100, 101, 102, 103, 110,\n120, 121, 122.\n\nSimilarly, you can use `from_proj` for projection numbers and `from_angle` for\nrotation angles.\n\nAlternatively, you can provide the scan numbers from a single-columned text\nfile and use the parameter `scan_list`, which is the path of the text file.\nSimilarly, you can use `proj_list` for projection numbers and `angle_list` for\nrotation angles.\n\nTo exclude certain scan numbers, you can use `exclude_scan`, it follows the\nformat of \\\u003cSTART\\\u003e[-\\\u003cEND\\\u003e[:\\\u003cSTEP\\\u003e]]. Similarly, you can use `exclude_proj`\nfor projection numbers and `exclude_angle` for rotation angles.\n\n#### Ignore metadata in raw files\n\nIf the raw files are not available (e.g. have been archived) and you are able\nto provide the scans/projections/rotation angles via `scan_list`, `proj_list`\nor `angle_list`, respectively, you can use the parameter\n`ignore_metadata_from_raw` (or `--ignore-raw` from the CLI) to avoid fetching\nmetadata from the raw files.\n\n```python\nfrom nxstacker.tomojoin import tomojoin\n\nnxtomo_files = tomojoin(\n    \"ptychography\",\n    scan_list=\"/i08-1/dir/scan_list.txt\",\n    angle_list=\"/i08-1/dir/angle_list.txt\",\n    ignore_metadata_from_raw=True,\n    ...\n)\n```\n\nor via the CLI:\n\n```console\ntomojoin ptycho --scan-list \"/i08-1/dir/scan_list.txt\"\n--angle-list=\"/i08-1/dir/angle_list.txt\" --ignore-raw ...\n```\n\nSee [API](#api) for more information about the parameters.\n\n### Get a NXtomo file from XRF-tomography experiment\n\n```python\nfrom nxstacker.tomojoin import tomojoin\n\nnxtomo_files = tomojoin(\n    \"xrf\",\n    proj_dir=\"/i14/dir/projections/stored\",\n    nxtomo_dir=\"/tmp/\",\n    from_scan=\"275019-275199\",\n    transition=\"V-Ka,Pt-La\",\n)\n```\n\nor via the CLI:\n\n```console\ntomojoin xrf --proj-dir \"/i14/dir/projections/stored\" --nxtomo-dir\n\"/tmp\" --from-scan 275019-275199 --transition V-Ka,Pt-La\n```\n\n`transition` is a comma-delimited list of transitions. The specified\ntransition must be present in the projection files.\n\nSee [API](#api) for more information about the parameters.\n\n## Tip to speed up\n\n### Skip projection file validation\n\nYou can set the parameter `skip_proj_file_check` to `True` to skip the\nvalidation of projection files if you know *all* the HDF5 files in `proj_dir`\nare from a particular source. The default is `False` for safety reason. Setting\nthis to `True` can potentially speed up the time in finding the projection\nfiles, especially if you have a lot of HDF5 files there. The same can be\nachieved via the CLI by using `--skip-check`.\n\n## API\n\n### nxstacker.tomojoin\n\n#### `tomojoin`\n\nThere are 22 parameters for this function, excluding parameters specific to a\nparticular type of experiment.\n\n##### Common parameters\n\n- *experiment_type*\n\nthe type of experiment, such as \"ptycho\" or \"xrf\". This must be specified.\n\n- *facility*\n\nthe facility, such as \"i08-1\", \"i13-1\" or \"i14\".\nIf it is `None`, it will be dedcued. The utility usually does a good job in\nguessing the facility, but if it struggles, please kindly provide this.\n\n- *proj_dir*\n\nthe directory where the projections are stored. If it is `None`,\nthe current working directory is used.\n\n- *proj_file*\n\nthe projection file with placeholder `%(scan)` deduced from *from_scan*,\n*scan_list* and *exclude_scan*, and `%(proj)` deduced from *from_proj*,\n*proj_list* and *exclude_proj*. If it is `None`, it will search in\n*proj_dir*.  If the name of the projection files follow a pattern, it is\nrecommended to use this as it avoids going through files and directories when\nsearching in *proj_dir*.\n\n- *nxtomo_dir*\n\nthe directory where the\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html)\nfile(s) are saved. If it is `None`, the current working directory is used.\n\n- *from_scan*\n\nthe string specification of scan identifier with the format\n\\\u003cSTART\\\u003e[-\\\u003cEND\\\u003e[:\\\u003cSTEP\\\u003e]]. If it is `None`, it is empty.\n\n- *scan_list*\n\nthe text file with a single-column scan identifier to be included.\nIf it is `None`, it is empty.\n\n- *exclude_scan*\n\nthe scan to be excluded. If it is `None`, nothing is excluded.\n\n- *from_proj*\n\nthe projection number string specification, see `from_scan`.\n\n- *proj_list*\n\nthe text file with a single-column projection numbers to be included.\nIf it is `None`, it is empty.\n\n- *exclude_proj*\n\nthe projection to be excluded. If it is `None`, nothing is excluded.\n\n- *from_angle*\n\nthe rotation angle string specification, see `from_scan`.\n\n- *angle_list*\n\nthe text file with a single-column rotation angle to be included.\nIf it is `None`, it is empty.\n\n- *exclude_angle*\n\nthe rotation angle to be excluded. If it is `None`, nothing is excluded.\n\n- *raw_dir*\n\nthe directory where the raw data are stored. For most of the time this can be\nleft as `None` as the raw directory is inferred from the projection files, but\nit is useful when the original raw directory is invalid or the raw data is in a\nnon-standard directory.\n\n- *sort_by_angle*\n\nwhether to sort the projections by their rotation angles. Default to False.\n\n- *pad_to_max*\n\nwhether to pad the individual projection if it is not at the maximum size of\nthe stack. Default to True. If it is False and the size is inconsistent,\nRuntimeError is raised.\n\n- *compress*\n\nwhether to apply compression (Blosc) to the\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html)\nfile(s). Default to False.\n\n- *quiet*\n\nwhether to suppress log message. Default to False.\n\n- *dry_run*\n\nwhether to perform a dry-run. Default to False.\n\n- *skip_proj_file_check*\n\nwhether to skip the file check when adding an hdf5 to the list of projection\nfiles. Usually this is true when you are doing a typical stacking and sure no\nother hdf5 files are present in `proj_dir`. Default to False.\n\n- *ignore_metadata_from_raw*\n\nwhether to ignore metadata obtained from the raw files because of their\nunavailability or speed. If this is True, `scan_list`/`proj_list` and\n`angle_list` must be provided as those information are no longer obtained from\nraw files. Default to False.\n\n##### Specific to ptychography\n\nThere are 7 parameters specific to `ptycho`/`ptychography`.\n\n- *save_complex*\n\nwhether to save the complex number representation of the reconstruction to a\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html) file.\nDefault to False.\n\n- *save_modulus*\n\nwhether to save the modulus of the reconstruction to a\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html) file.\nDefault to False.\n\n- *save_phase*\n\nwhether to save the phase of the reconstruction to a\n[NXtomo](https://manual.nexusformat.org/classes/applications/NXtomo.html) file.\nDefault to True.\n\n- *remove_ramp*\n\nwhether to remove phase ramp. *This has not been implemented*. Default to\nFalse.\n\n- *median_norm*\n\nwhether to shift the phase by its median. Default to False.\n\n- *unwrap_phase*\n\nwhether to unwrap phase. Default to False.\n\n- *rescale*\n\nwhether to rescale the reconstruction. *This has not been implemented*.\nDefault to False.\n\n##### Specific to XRF-tomography\n\nThere are 1 parameter specific to `xrf`.\n\n- *transition*\n\na comma-delimited list of transition to be saved. The transition must be\npresent in the projection file. E.g. \"W-La,Ca-Ka\"\n\n## Contributing\n\nContribution is very welcomed. Please use the [issue\npage](https://github.com/DiamondLightSource/nxstacker/issues) to report any\nbug and missing feature.\n\n## Acknowledgements\n\nThe motivation of this utility is to unify the efforts of various tools\ndeveloped in different beamlines at Diamond Light Source. Benedikt Daurer's\n[PtychographyTools](https://github.com/DiamondLightSource/PtychographyTools)\nand Yousef Moazzam's\n[scripts for i14](https://github.com/yousefmoazzam/tomo-utils/) provide a lot\nof inspirations to this utility.\n\n## Licence\n\nMIT\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Fnxstacker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiamondlightsource%2Fnxstacker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Fnxstacker/lists"}