{"id":45979784,"url":"https://github.com/datalab-org/datalab-app-plugin-insitu","last_synced_at":"2026-03-11T01:04:35.768Z","repository":{"id":284596515,"uuid":"881954767","full_name":"datalab-org/datalab-app-plugin-insitu","owner":"datalab-org","description":"datalab block plugin for in situ measurements (NMR, UV-Vis, XRD)","archived":false,"fork":false,"pushed_at":"2026-02-26T22:49:48.000Z","size":152507,"stargazers_count":4,"open_issues_count":22,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-28T18:58:50.262Z","etag":null,"topics":["chemistry","data-management","datalab-plugin","experimental-data","materials-science"],"latest_commit_sha":null,"homepage":"https://datalab-app-plugin-insitu.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datalab-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-01T15:11:24.000Z","updated_at":"2026-02-28T04:22:49.000Z","dependencies_parsed_at":"2025-09-10T16:27:11.800Z","dependency_job_id":"c99614f6-d613-4265-b409-3e8aff1f013d","html_url":"https://github.com/datalab-org/datalab-app-plugin-insitu","commit_stats":null,"previous_names":["datalab-org/datalab-app-plugin-insitu"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/datalab-org/datalab-app-plugin-insitu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalab-org%2Fdatalab-app-plugin-insitu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalab-org%2Fdatalab-app-plugin-insitu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalab-org%2Fdatalab-app-plugin-insitu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalab-org%2Fdatalab-app-plugin-insitu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datalab-org","download_url":"https://codeload.github.com/datalab-org/datalab-app-plugin-insitu/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalab-org%2Fdatalab-app-plugin-insitu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30364739,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"ssl_error","status_checked_at":"2026-03-10T21:40:59.357Z","response_time":106,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chemistry","data-management","datalab-plugin","experimental-data","materials-science"],"created_at":"2026-02-28T17:32:24.999Z","updated_at":"2026-03-11T01:04:35.712Z","avatar_url":"https://github.com/datalab-org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cdiv align=\"center\"\u003e\u003ci\u003edatalab *in situ* plugin\u003c/i\u003e\u003c/div\u003e\n\n\u003cdiv align=\"center\" style=\"padding-bottom: 5px\"\u003e\n\u003ca href=\"https://demo.datalab-org.io\"\u003e\u003cimg src=\"https://img.shields.io/badge/try_it_out!-public_demo_server-orange?logo=firefox\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://github.com/datalab-org/datalab-app-plugin-insitu/releases\"\u003e\u003cimg src=\"https://badgen.net/github/release/datalab-org/datalab-app-plugin-insitu?icon=github\u0026color=blue\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/datalab-org/datalab-app-plugin-insitu\"\u003e\u003cimg src=\"https://badgen.net/github/license/datalab-org/datalab-app-plugin-insitu?icon=license\u0026color=purple\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://datalab-app-plugin-insitu.readthedocs.io/en/latest/?badge=latest\"\u003e\u003cimg src=\"https://img.shields.io/readthedocs/datalab-app-plugin-insitu?logo=readthedocs\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g\"\u003e\u003cimg src=\"https://img.shields.io/badge/Slack-chat_with_us-yellow?logo=slack\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n\nA Python plugin for processing and visualizing *in situ* (NMR, UV-Vis, XRD) data within [_datalab_](https://github.com/datalab-org/datalab) instances.\n\nThe project was originally developed in the [Grey Group](https://www.ch.cam.ac.uk/group/grey/) in the Department of Chemistry at the University of Cambridge, with support and ongoing maintenance from [*datalab industries ltd.*](https://datalab.industries).\n\n## Features\n\n- Support for NMR, XRD and UV-Vis data, recorded alongside electrochemical experiments, temperature ramps or other time series data (support varies across data types)\n- Integration with electrochemical data formats for combined analysis using [navani](https://github.com/be-smith/navani)\n- Interactive visualization using Bokeh, with heatmaps and spectra selection\n- Support for both local files, as well as running as a plugin directly on a *datalab* instance\n\n\u003cdiv align=\"center\"\u003e\n   \u003cimg src=\"./static/datalab_plugin_bokeh_plot.png\" width=600rem\u003e\n\u003c/div\u003e\n\n## Roadmap\n\nThere are still several planned features and improvements for this plugin,\nincluding:\n\n- Supporting all formats for electrochemistry, NMR, XRD and UV-Vis that are supported by the corresponding non-*in situ* blocks available in *datalab*.\n- More flexible specification of other time series data across all data types, e.g., temperature or pressure ramps.\n- Improved performance and caching of data, especially for large datasets.\n\n# Installation\n\nThe `datalab-app-plugin-insitu` package is currently a \"core plugin\" of *datalab* (since *datalab* v0.5.3), i.e., it will be installed on *datalab* instances by default.\n\n## Development installation\n\nWe recommend you use [`uv`](https://astral.sh/uv) for managing virtual environments and Python versions.\n\nOnce you have `uv` installed, you can clone this repository and install the package in a fresh virtual environment with:\n\n```shell\ngit clone git@github.com:datalab-org/datalab-app-plugin-insitu\ncd datalab-app-plugin-insitu\nuv sync --all-extras --dev\n```\n\nYou can activate `pre-commit` in your cloned repository with `uv run pre-commit install`.\n\nYou can run the tests using `pytest`, optionaly with the marker `show_plots` to display the plots generated by the tests:\n\n```shell\nuv run pytest -m show_plots\n```\n\n## Usage\n\nFull reference API documentation can be found on [ReadTheDocs](https://datalab-app-plugin-insitu.readthedocs.io/).\n\n### Data Structure Requirements\n\n\n#### NMR\n\nYour *in situ* NMR data should be organized as follows:\n\n```\ndata_folder.zip/\n├── \u003cnmr_folder\u003e/\n│   ├── 1/\n│   │   ├── acqus\n│   │   └── pdata/\n│   │       └── 1/\n│   │           └── ascii-spec.txt\n│   ├── 2/\n│   │   └── ...\n│   └── ...\n└── \u003cechem_folder\u003e/\n    └── *.MPR\n```\n\nwith the `\u003cnmr_folder\u003e` and `\u003cechem_folder\u003e` names specified at runtime (or via the *datalab* UI).\n\n#### UV-Vis\n\n```\ndata_folder.zip/\n├── \u003cuv-vis folder\u003e/\n│   ├── scan_0000.txt\n│   ├── scan_0001.txt\n│   ├── scan_0002.txt\n│   ├── scan_0003.txt\n│   ├── scan_0004.txt\n│   ├── scan_0005.txt\n│   ├── ...\n│   └── scan_\u003cN\u003e.txt\n├── \u003cuv-vis reference folder\u003e/\n│   └── reference.txt\n└── \u003cechem_folder\u003e/\n    ├── \u003cechem_file_1\u003e\n    ├── \u003cechem_file_2\u003e\n    └── ...\n```\n\nwhere the `\u003cuv-vis folder\u003e` contains the UV-Vis scans (named as `*_\u003cn\u003e.txt`, which will be used for sorting), the `\u003cuv-vis reference folder\u003e` contains the reference spectrum, and the `\u003cechem_folder\u003e` contains one or more electrochemical data files loadable by [navani](https://github.com/be-smith/navani). Note if multiple files are present then navani will attempt to stitch them together, so do not upload duplicates (e.g processed and unprocessed versions of the same file).\n\n#### XRD\n\nThere are two XRD plotting modes, against Temperature and against Electrochemistry, both pathways require a folder containing the xrd data and the log data as shown below, with the electrochemistry pathway requiring a third directory for the electrochemistry data.\n\n```\ndata_folder.zip/\n├── \u003cxrd folder\u003e/\n│   ├── 0000-scan.dat\n│   ├── 0001-scan.dat\n│   ├── 0002-scan.dat\n│   ├── 0003-scan.dat\n│   ├── 0004-scan.dat\n│   ├── 0005-scan.dat\n│   ├── ...\n│   └── \u003cN\u003e-scan.dat\n└── \u003clog folder\u003e/\n    └── time_series_log.csv\n└── \u003cechem folder\u003e/ (optional)\n    ├── \u003cechem_file_1\u003e\n    ├── \u003cechem_file_2\u003e\n    └── ...\n\n```\n\nwhere the `\u003cxrd folder\u003e` contains the XRD scans (named as `0000-scan.dat`, `0001-scan.dat`, etc.), and the `\u003clog folder\u003e` contains a time series log file in CSV format, mapping scan number to the time series data, e.g. for the Temperature mode:\n\n```csv\nscan_number,Temp\n0000, 25.0\n0001, 30.0\n0002, 35.0\n0003, 40.0\n```\n\nAnd for the electrochemistry mode:\n\n```csv\nscan_number,start_time,end_time\n0000, 2025-07-02 19:05:59.614000, 2025-07-02 19:06:56.167000\n0001, 2025-07-02 19:15:03.582000, 2025-07-02 19:15:59.280000\n0002, 2025-07-02 19:24:07.586000, 2025-07-02 19:25:04.074000\n0003, 2025-07-02 19:33:12.198000, 2025-07-02 19:34:08.917000\n```\n\nThe `\u003cechem folder\u003e` contains one or more electrochemistry files readable by [navani](https://github.com/be-smith/navani), Note if multiple files are present then navani will attempt to stitch them together, so do not upload duplicates (e.g processed and unprocessed versions of the same file). For the XRD usecase it's currently expected that the echem file has a `Timestamp` column containing date-time information for matching to the log file (e.g Neware files).\n\n### Front-end Usage\n\nThe following configuration parameters are available in the *datalab* UI:\n\n#### Required Parameters\n\n- **`scan_time`** (UV-Vis only): The time in seconds between each scan, including any rest time. This is used to calculate the time of each scan when no log file is available.\n\n#### Optional Parameters\n\n##### Performance Optimization\n\n- **`data_granularity`**: Controls sampling frequency along the x-axis (wavelength/2θ) of the heatmap to reduce rendering time. Default: automatically calculated to target ~1000 data points.\n  - XRD: Uses max-pooling interpolation\n  - UV-Vis: Uses linear interpolation\n\n- **`sample_granularity`**: Controls sampling frequency along the y-axis (time/experiment number) of the heatmap. Uses linear interpolation. Default: automatically calculated to target ~1000 samples.\n\n##### File Selection\n\n- **`glob_str`** (XRD only): A pattern to match specific files in the XRD data folder. Examples:\n  - `\"summed\"` → matches files containing \"summed\" (automatically converted to `*summed*`)\n  - `\"*summed*\"` → matches files containing \"summed\" (explicit wildcards)\n  - `\"*.xy\"` → matches all `.xy` files\n  - Leave empty to process all files in the folder (default behavior)\n\n## License\n\nThis project is released under the conditions of the MIT license. Please see [LICENSE](https://github.com/datalab-org/datalab-app-plugin-insitu/blob/main/LICENSE) for the full text of the license.\n\n## Contact\n\nFor questions and support, please [open an issue](https://github.com/datalab-org/datalab-app-plugin-insitu/issues) on the GitHub repository or join the [public datalab Slack workspace](https://join.slack.com/t/datalab-world/shared_invite/zt-2h58ev3pc-VV496~5je~QoT2TgFIwn4g).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatalab-org%2Fdatalab-app-plugin-insitu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatalab-org%2Fdatalab-app-plugin-insitu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatalab-org%2Fdatalab-app-plugin-insitu/lists"}