{"id":43678508,"url":"https://github.com/mousebrains/q2netcdf","last_synced_at":"2026-04-02T13:37:49.302Z","repository":{"id":261179626,"uuid":"883511717","full_name":"mousebrains/q2netcdf","owner":"mousebrains","description":"Convert Rockland Q-files into NetCDF","archived":false,"fork":false,"pushed_at":"2026-03-22T02:06:37.000Z","size":470,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-22T11:06:57.389Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mousebrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-05T05:01:49.000Z","updated_at":"2026-03-22T02:06:40.000Z","dependencies_parsed_at":"2024-11-05T06:17:46.581Z","dependency_job_id":"70d7da7e-c65e-4859-8dc0-38688aae6f69","html_url":"https://github.com/mousebrains/q2netcdf","commit_stats":null,"previous_names":["mousebrains/q2netcdf"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/mousebrains/q2netcdf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mousebrains%2Fq2netcdf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mousebrains%2Fq2netcdf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mousebrains%2Fq2netcdf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mousebrains%2Fq2netcdf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mousebrains","download_url":"https://codeload.github.com/mousebrains/q2netcdf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mousebrains%2Fq2netcdf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31307177,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-05T01:21:02.171Z","updated_at":"2026-04-02T13:37:49.292Z","avatar_url":"https://github.com/mousebrains.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# q2netcdf\n\n[![Python Version](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![CI](https://github.com/mousebrains/q2netcdf/actions/workflows/ci.yml/badge.svg)](https://github.com/mousebrains/q2netcdf/actions/workflows/ci.yml)\n[![Coverage](https://codecov.io/gh/mousebrains/q2netcdf/branch/main/graph/badge.svg)](https://codecov.io/gh/mousebrains/q2netcdf)\n[![PyPI](https://img.shields.io/pypi/v/q2netcdf)](https://pypi.org/project/q2netcdf/)\n\n\u003e **Note**: The package requires Python 3.10+, but the standalone `mergeqfiles` script is compatible with Python 3.7+ for deployment on legacy systems (e.g., MicroRider instruments).\n\nPython tools for manipulating [Rockland Scientific's](https://rocklandscientific.com) Q-files as generated by the [ISDP data logger](https://rocklandscientific.com/news/rockland-data-logger/).\n\n## Features\n\n- **Q-file to NetCDF Conversion**: Convert binary Q-files to CF-1.8 compliant NetCDF format\n- **Header Parsing**: Extract metadata, sensor configurations, and timing information\n- **Data Extraction**: Read scalar channels and frequency spectra from binary records\n- **File Manipulation**: Merge, reduce, and inspect Q-files\n- **Configuration Generation**: Create syntactically correct `isdp.cfg` files\n- **Format Support**: Handles Q-file versions 1.2 and 1.3\n- **Sensor Mapping**: Built-in support for 200+ sensor identifier codes\n\n## Command-Line Tools\n\n- **`q2netcdf`** - Translate Q-files into CF-1.8 compliant NetCDF files\n- **`QFile`** - Dump header and data records from Q-files\n- **`QHeader`** - Display header record information\n- **`QReduce`** - Reduce Q-file size by removing records\n- **`mergeqfiles`** - Merge multiple Q-files (designed for MicroRider/Slocum Glider integration)\n- **`mkISDPcfg`** - Generate syntactically correct `isdp.cfg` configuration files\n\n## Quick Start\n\n```bash\n# Display Q-file header information\nQHeader data.q\n\n# Convert Q-file to NetCDF\nq2netcdf data.q --nc output.nc\n\n# Dump all records from Q-file\nQFile data.q --logLevel INFO\n```\n\n## Installation\n\n### From PyPI\n\n```bash\npip install q2netcdf\n```\n\nOr using [pipx](https://pipx.pypa.io/stable/installation/) for the command-line tools:\n\n```bash\npipx install q2netcdf\n```\n\n### From Source\n\n```bash\ngit clone git@github.com:mousebrains/q2netcdf.git\ncd q2netcdf\npip install -e .\n```\n\n### For Development\n\n```bash\ngit clone git@github.com:mousebrains/q2netcdf.git\ncd q2netcdf\npip install -e \".[dev]\"\npytest  # Run tests\n```\n\n## Installation on the MicroRider\n\nFor deployment on Rockland Scientific MicroRider instruments:\n\n1. Transfer [src/q2netcdf/mergeqfiles.py](src/q2netcdf/mergeqfiles.py) and [src/q2netcdf/mergeqfiles.cfg](src/q2netcdf/mergeqfiles.cfg) to the MicroRider's data directory.\n2. Dismount the data directory on your host computer.\n3. Connect to the MicroRider via a serial connection using your favorite terminal emulator at 115,200 baud.\n4. `ls data` to verify the files are present. This will cause the MicroRider to unmount data from being read-only to the MicroRider and then remount it as read-write. This will typically cause the serial connection to drop then be reestablished. You might have to restart your terminal emulator's connection.\n5. `mv data/mergeqfiles.py /usr/local/bin/mergeqfiles` to install the script as mergeqfiles.\n6. Test with `mergeqfiles --help` to make sure the installation was successful.\n7. You can disconnect the MicroRider's serial connection.\n8. You should now run a simulation on the Slocum glider to verify Q-files are being uploaded to the science computer's storage as .mri files.\n9. Diagnostic information is written into the data/mergeqfiles.log file on the MicroRider.\n10. You can modify data/mergeqfiles.cfg to adjust the information included in the .mri file sent to the glider.\n\nThis is designed for integration with [TWR's Slocum Glider uRider proglet](https://www.teledynemarine.com/brands/webb-research/slocum-glider).\n\n## Python API\n\n```python\nfrom q2netcdf import QFile\n\n# Stream records from a Q-file\nwith QFile(\"data.q\") as qf:\n    header = qf.header()\n    print(header.channels(), header.spectra())\n    for record in qf.data():\n        print(record.stime, record.channels)\n\n# Convert a Q-file to an xarray Dataset\nfrom q2netcdf.q2netcdf import loadQfile\n\nds = loadQfile(\"data.q\")\nprint(ds)\nds.to_netcdf(\"output.nc\")\n```\n\n## Usage Examples\n\n### Converting Q-files to NetCDF\n\n```bash\nq2netcdf data.q --nc output.nc\n```\n\n### Inspecting Q-file Headers\n\n```bash\n# Show full header with channel and spectra names\nQHeader data.q\n\n# Show condensed header\nQHeader data.q --nothing\n\n# Show only configuration\nQHeader data.q --channels --spectra --frequencies\n```\n\n### Examining Q-file Data Records\n\n```bash\n# Dump all records\nQFile data.q\n\n# Limit number of records displayed\nQFile data.q --n 10\n```\n\n### Merging Q-files\n\n```bash\nmergeqfiles -o merged.q file1.q file2.q file3.q\n```\n\n### Generating ISDP Configuration Files\n\n```bash\nmkISDPcfg --help\n```\n\n## File Format\n\nQ-files are binary files containing oceanographic data from Rockland Scientific instruments. The format is documented in Rockland Technical Note TN-054.\n\n### Q-file Structure\n- **Header Record** (0x1729): File version, timestamp, channel/spectra identifiers, frequencies, configuration\n- **Data Records** (0x1657): Timestamped measurements with scalar channels and frequency spectra\n\n### Supported Versions\n- **v1.2**: Original format with full metadata in each data record\n- **v1.3**: Optimized format with reduced redundancy\n\n## Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](docs/CONTRIBUTING.md) for guidelines.\n\n### Development Setup\n\n```bash\ngit clone git@github.com:mousebrains/q2netcdf.git\ncd q2netcdf\npip install -e \".[dev]\"\npytest                    # Run tests\nruff check src/ tests/    # Lint\nmypy src/                 # Type check\n```\n\nTo run a module directly: `python3 -m q2netcdf.QHeader --help`\n\n## Additional Documentation\n\n- [CHANGELOG.md](docs/CHANGELOG.md) - Version history and changes\n- [CONTRIBUTING.md](docs/CONTRIBUTING.md) - Contributor guidelines and development setup\n- [SECURITY.md](docs/SECURITY.md) - Security policy and vulnerability reporting\n- [QFILE_FORMAT.md](docs/QFILE_FORMAT.md) - Q-file binary format specification\n\n## License\n\nThis project is licensed under the [GNU General Public License v3.0 or later](LICENSE).\n\n## Author\n\n**Pat Welch**\nEmail: pat@mousebrains.com\n\n## Acknowledgments\n\nDeveloped for use with [Rockland Scientific](https://rocklandscientific.com) oceanographic instruments and the ISDP data logger.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmousebrains%2Fq2netcdf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmousebrains%2Fq2netcdf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmousebrains%2Fq2netcdf/lists"}