{"id":13719778,"url":"https://github.com/equinor/seismic-zfp","last_synced_at":"2025-04-04T18:06:26.816Z","repository":{"id":37450694,"uuid":"216040753","full_name":"equinor/seismic-zfp","owner":"equinor","description":"Compress and decompress seismic data","archived":false,"fork":false,"pushed_at":"2024-10-29T10:03:29.000Z","size":1423,"stargazers_count":67,"open_issues_count":9,"forks_count":15,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-10-29T12:08:26.033Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/equinor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION","codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-18T14:19:43.000Z","updated_at":"2024-10-17T07:17:16.000Z","dependencies_parsed_at":"2024-01-14T17:04:10.388Z","dependency_job_id":"1e2987c8-73b3-4cab-93a8-1fa56e53f004","html_url":"https://github.com/equinor/seismic-zfp","commit_stats":{"total_commits":357,"total_committers":12,"mean_commits":29.75,"dds":"0.33613445378151263","last_synced_commit":"f8e2582136a866ca22717f4d4f9f1bd7742bbd1f"},"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/equinor%2Fseismic-zfp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/equinor%2Fseismic-zfp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/equinor%2Fseismic-zfp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/equinor%2Fseismic-zfp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/equinor","download_url":"https://codeload.github.com/equinor/seismic-zfp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247226213,"owners_count":20904465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T01:00:55.367Z","updated_at":"2025-04-04T18:06:26.796Z","avatar_url":"https://github.com/equinor.png","language":"Python","funding_links":[],"categories":["Software","Software and Tools"],"sub_categories":["Seismic and Seismology"],"readme":"# seismic-zfp #\n\n[![LGPLv3 License](https://img.shields.io/badge/License-LGPL%20v3-green.svg)](https://opensource.org/licenses/)\n[![Run Tests](https://github.com/equinor/seismic-zfp/actions/workflows/run_tests.yml/badge.svg)](https://github.com/equinor/seismic-zfp/actions/workflows/run_tests.yml)\n[![codecov](https://codecov.io/gh/equinor/seismic-zfp/branch/master/graph/badge.svg)](https://codecov.io/gh/equinor/seismic-zfp)\n[![SCM Compliance](https://scm-compliance-api.radix.equinor.com/repos/equinor/seismic-zfp/badge)](https://scm-compliance-api.radix.equinor.com/repos/equinor/seismic-zfp/badge)\n[![PyPi Version](https://img.shields.io/pypi/v/seismic-zfp.svg)](https://pypi.org/project/seismic-zfp/)\n\nPython library to convert SEG-Y files to compressed cubes and retrieve arbitrary sub-volumes from these, fast.\n\n## Motivation ##\n\nReading whole SEG-Y volumes to retrieve, for example, a single time-slice is wasteful.\n\nCopying whole SEG-Y files uncompressed over networks is also wasteful.\n\nThis library addresses both issues by implementing the [seismic-zfp (.SGZ) format](docs/file-specification.md).\nThis format is based on [ZFP compression](https://computing.llnl.gov/projects/floating-point-compression)\nfrom [Peter Lindstrom's paper](https://www.researchgate.net/publication/264417607_Fixed-Rate_Compressed_Floating-Point_Arrays)\nusing the official Python bindings, distributed as zfpy.\n\n\nZFP compression enables smoothly varying d-dimensional data in 4\u003csup\u003ed\u003c/sup\u003e subvolumes \nto be compressed at a fixed bitrate. The 32-bit floating point values in 4x4x4 units\nof a 3D post-stack SEG-Y file are well suited to this scheme. \n\nDecomposing an appropriately padded 3D seismic volume into groups of these units which \nexactly fill one 4KB disk block, compressing these groups, and writing them sequentially \nto disk yields a file with the following properties:\n- Compression ratio of 2\u003csup\u003en\u003c/sup\u003e:1 compression, \ntypically a bitrate of 4 works well, implying a ratio of 8:1\n- The location of any seismic sample is known\n- Arbitrary subvolumes can be read with *minimal* redundant I/O, for example:\n  - Padding IL/XL dimensions with 4, and the z-dimension depending on bitrate\n  - Padding IL/XL dimensions with 64 and the z-dimension with 4 (16:1 compression)\n#### Using IL/XL optimized layout ###\n- Groups of 4 inlines or crosslines can be read with **no** redundant I/O\n- A single inline can be read and with **no** additional I/O compared to the SEG-Y \nbest-case scenario (provided at least 4:1 compression ratio)\n- A z-slice can be read by accessing n_traces/16 disk blocks, \ncompared to n_traces disk blocks for SEG-Y\n#### Using z-slice optimized layout ####\n- A z-slice can be read by accessing **just** n_traces/4096 disk blocks, \ncompared to n_traces disk blocks for SEG-Y\n#### 2D SEG-Y Support ####\nAs of v0.2.4 support for compressing 2D SEG-Y files (INLINE_3D and CROSSLINE_3D always zero) is included.\nCompression and reading follows the same pattern as 3D files, but segyio emulation only provides the \nfollowing attributes: trace, header, samples, bin \u0026 text. However an additional funciton read_subplane() \nis available for extracting horizontally and vertically contrained data.\n\n\n#### Headers ####\nThe [seismic-zfp (.SGZ) format](docs/file-specification.md) also allows for preservation of information in \nSEG-Y file and trace headers, with compression code identifying constant \nand varying trace header values and storing these appropriately.\n\nFor further explanation of the design and implementation of seismic-zfp, please refer to [publications](#publications).\n\n#### NOTE: Previously the extension .sz was used for seismic-zfp, but has been replaced with .sgz to avoid confusion around the compression algorithm used.\n\n## Get seismic-zfp\n- Wheels from [PyPI](https://pypi.org/project/seismic-zfp/) with zgy and vds support: `pip install seismic-zfp[zgy,vds]`\n- Wheels from [PyPI](https://pypi.org/project/seismic-zfp/) without zgy or vds support: `pip install seismic-zfp`\n- Source from [Github](https://github.com/equinor/seismic-zfp): `git clone https://github.com/equinor/seismic-zfp.git`\n\n*Note that seismic-zfp depends on the Python package [ZFPY](https://pypi.org/project/zfpy/), which is a binary distribution on PyPI built for Linux and Windows.*\n\n*The optional dependency [pyvds](https://github.com/equinor/pyvds) - requires openvds package from Bluware which is **not** open-source*\n\n*The optional dependency of zgy2sgz has been replaced with [pyzgy](https://github.com/equinor/pyzgy) - a pure-Python alternative.*\n\n## Examples ##\n\nFull example code is provided [here](examples), but the following reference is useful:\n\n#### Create SGZ files from SEG-Y, ZGY or VDS ####\n\n```python\nfrom seismic_zfp.conversion import SegyConverter, ZgyConverter, VdsConverter\n\nwith SegyConverter(\"in.sgy\") as converter:\n    # Create a \"standard\" SGZ file with 8:1 compression, using in-memory method\n    converter.run(\"out_standard.sgz\", bits_per_voxel=4)\n    # Create a \"z-slice optimized\" SGZ file\n    converter.run(\"out_adv.sgz\", bits_per_voxel=2, blockshape=(64, 64, 4))\n                  \nwith ZgyConverter(\"in_8-int.zgy\") as converter:\n    # 8-bit integer ZGY and 1-bit SGZ have similar quality\n    converter.run(\"out_8bit.sgz\", bits_per_voxel=1)\n\nwith VdsConverter(\"in.vds\") as converter:\n    # VDS compression is superior, but doesn't permit non-cubic bricks...\n    converter.run(\"out.sgz\", bits_per_voxel=4, blockshape=(8, 8, 128))\n```\n\n#### Convert SGZ files to SEG-Y ####\n\n```python\n# Convert SGZ to SEG-Y\nwith SgzConverter(\"out_standard.sgz\") as converter:\n    converter.convert_to_segy(\"recovered.sgy\")\n```\n\n#### Read an SGZ file ####\n```python\nfrom seismic_zfp.read import SgzReader\nwith SgzReader(\"in.sgz\") as reader:\n    inline_slice = reader.read_inline(LINE_IDX)\n    crossline_slice = reader.read_crossline(LINE_IDX)\n    z_slice = reader.read_zslice(LINE_IDX)\n    sub_vol = reader.read_subvolume(min_il=min_il, max_il=max_il, \n                                    min_xl=min_xl, max_xl=max_xl, \n                                    min_z=min_z, max_z=max_z)\n```\n\n#### Use segyio-like interface to read SGZ files ####\n```python\nimport seismic_zfp\nwith seismic_zfp.open(\"in.sgz\")) as sgzfile:\n    il_slice = sgzfile.iline[sgzfile.ilines[LINE_NUMBER]]\n    xl_slices = [xl for xl in sgzfile.xline]\n    zslices = sgzfile.depth_slice[:5317]\n    trace = sgzfile.trace[TRACE_IDX]\n    trace_header = sgzfile.header[TRACE_IDX]\n    binary_file_header = sgzfile.bin\n    text_file_header = sgzfile.text[0]\n```\nIncluding equivalents to segyio.tools\n```python\nwith seismic_zfp.open(\"in.sgz\") as sgz_file:\n    dt_sgz = seismic_zfp.tools.dt(sgz_file)\n\ncube_sgz = seismic_zfp.tools.cube(\"in.sgz\")\n```\n\nPlus some extended utility functionality:\n```python\nwith seismic_zfp.open(\"in.sgz\")) as sgzfile:\n    subvolume = sgzfile.subvolume[IL_NO_START:IL_NO_STOP:IL_NO_STEP, \n                                  XL_NO_START:XL_NO_STOP:XL_NO_STEP,\n                                  SAMP_NO_START:SAMP_NO_STOP:SAMP_NO_STEP]\n    header_arr = sgzfile.get_tracefield_values(segyio.tracefield.TraceField.NStackedTraces)\n```\n\n## Command Line Interface\nA simple command line interface for converting from SEGY or ZGY to SGZ is also bundled with seismic-zfp. This is available using the command `seismic-zfp`.\n\n#### Converting from SEG-Y to SGZ and back\n```bash\nseismic-zfp sgy2sgz \u003cINPUT FILE PATH\u003e \u003cOUTPUT FILE PATH\u003e --bits-per-voxel 4\nseismic-zfp sgz2sgy \u003cINPUT FILE PATH\u003e \u003cOUTPUT FILE PATH\u003e\n```\nTo see all options run the command `seismic-zfp sgy2sgz --help`\n\n#### Converting from ZGY to SGZ\n```bash\nseismic-zfp zgy2sgz \u003cINPUT FILE PATH\u003e \u003cOUTPUT FILE PATH\u003e --bits-per-voxel 4\n```\nTo see all options run the command `seismic-zfp zgy2sgz --help`\n\n## Contributing\nContributions welcomed, whether you are reporting or fixing a bug, implementing or requesting a feature. Either make a github issue or fork the project and make a pull request. Please extend the unit tests with relevant passing/failing tests, run these as: `python -m pytest`\n\n## Publications\nWade, David (2020): _Seismic-ZFP: Fast and Efficient Compression and Decompression of Seismic Data_, First EAGE Digitalization Conference and Exhibition, Nov 2020, Volume 2020, p.1-5\n- [Full-Text PDF](docs/EAGE_extended-abstract.pdf) (without EAGE formatting)\n- [EarthDoc Link](https://www.earthdoc.org/content/papers/10.3997/2214-4609.202032080) (N.B. paywall protected)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fequinor%2Fseismic-zfp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fequinor%2Fseismic-zfp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fequinor%2Fseismic-zfp/lists"}