{"id":27127023,"url":"https://github.com/4d-star/opat-core","last_synced_at":"2025-10-24T13:44:24.848Z","repository":{"id":283993043,"uuid":"953497322","full_name":"4D-STAR/opat-core","owner":"4D-STAR","description":"Core libraries and modules for OPAT file format","archived":false,"fork":false,"pushed_at":"2025-04-01T16:17:52.000Z","size":2648,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-01T16:39:15.641Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/4D-STAR.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-23T14:06:41.000Z","updated_at":"2025-04-01T16:17:55.000Z","dependencies_parsed_at":"2025-03-23T15:35:27.927Z","dependency_job_id":null,"html_url":"https://github.com/4D-STAR/opat-core","commit_stats":null,"previous_names":["4d-star/opat-core"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4D-STAR%2Fopat-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4D-STAR%2Fopat-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4D-STAR%2Fopat-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4D-STAR%2Fopat-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/4D-STAR","download_url":"https://codeload.github.com/4D-STAR/opat-core/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247694889,"owners_count":20980731,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-07T16:54:11.693Z","updated_at":"2025-10-24T13:44:24.828Z","avatar_url":"https://github.com/4D-STAR.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo/opatCoreLogo.png\" width=\"300\" alt=\"OPAT Core Libraries Logo\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eOPAT Core Libraries\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/opatio/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/opatio\" alt=\"PyPI - Version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/4D-STAR/opat-core/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/4D-STAR/opat-core\" alt=\"GitHub License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://4d-star.github.io/opat-core/\"\u003e\u003cimg src=\"https://img.shields.io/badge/Documentation-View%20Here-blue\" alt=\"Documentation\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\nThis repository contains the core C++ library, python module, and file specification for the OPAT file format. \n\nOPAT is a structured binary file format developed by the 4D-STAR collaboration for efficient and standardized storage of set of tabular data indexed by some floating point vector (such as composition).\n\nThe general principle behind OPAT is that an arbitrary number of data cards are stored, indexed by some arbitrary-length composition vector. Each data card contains an arbitrary number of data tables indexed by some string tag.\n\nThe provided python module can be used to create and read OPAT files while the C++ module is intended to be used to interface OPAT tables with C++ code for reading (but not currently generation).\n\n## Installation\nThis repository provides both C++ and python bindings. The first thing to note is that these do not depend on each other at all. If you want to generate OPAT tables and/or use them in python code you will want the python module. If you want to use OPAT tables in C++ code you will want the C++ module. \n\nThere are more details on usage for each language in their respective folder; however, broad installation instructions are included here as well.\n\n### Dependencies Overview\n\nBelow is a summary of the key dependencies for each part of the project.\n\n#### Python Dependencies\nThese are managed by `pip` and are listed in `opatIO-py/pyproject.toml`.\n\n| Dependency  | Minimum Version | Usage                                  | Source/Authors                                                                 |\n|-------------|-----------------|----------------------------------------|--------------------------------------------------------------------------------|\n| `numpy`     | `\u003e= 1.21.1`     | Numerical operations, array handling   | [NumPy Developers](https://numpy.org/)                                         |\n| `xxhash`    | `\u003e= 3.5.0`      | Fast hashing algorithms (internal use) | [Yann Collet (Cyan4973)](https://github.com/Cyan4973/xxHash)                   |\n| `scipy`     | `\u003e= 1.15.0`     | Triangulation (for interpolation)      | [SciPy Developers](https://scipy.org/)                                         |\n\n#### C++ Dependencies\nMost C++ dependencies are handled via Meson's wrap system (built at compile time). Boost is an exception.\n\n| Dependency | Installation                                     | Usage                                                                 | Source/Authors                                                                                                                                                                     |\n|------------|--------------------------------------------------|-----------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Boost      | System install or via bundled script (prompts user) | Numerical linear algebra (`ublas`) in `TableLattice`                  | [Boost Community](https://www.boost.org/)                                                                                                                                          |\n| Qhull      | Meson wrap (built automatically)                 | N-dimensional Delaunay triangulation for `TableLattice`               | [Barber, C.B., Dobkin, D.P., and Huhdanpaa, H.T., \"The Quickhull algorithm for convex hulls,\" ACM Trans. on Mathematical Software, 22(4):469-483, Dec 1996](http://www.qhull.org/) |\n| xxHash     | Meson wrap (built automatically)                 | Fast hashing for `FloatIndexVector` lookups in `OPAT` class           | [Yann Collet (Cyan4973)](https://github.com/Cyan4973/xxHash) \u0026 [Stefan Brumme (stbrumme)](https://create.stephan-brumme.com/xxhash/)                                                                     \n| PicoSHA2   | Meson wrap (built automatically)                 | SHA-256 hashing for data integrity checks (`CardCatalogEntry`)        | [Shintarou Okada (okdshin)](https://github.com/okdshin/PicoSHA2)                                                                                                                             \n| cxxopts    | Meson wrap (built automatically)                 | Command-line option parsing for CLI tools (e.g., `opatHeader`)        | [Jarryd Beck (jarro2783)](https://github.com/jarro2783/cxxopts)                                                                                                                                \n\n### Python Installation\n```bash\npip install opatio\n```\n\n### C++ Installation\nYou will need `meson`, `cmake`, and `ninja` installed pre-installed. Note that cmake is needed in order to build subprojects which use CMake as their build system. *opat-core does not make use of CMake as a build system*. These can be installed with pip\n```bash\npip install \"meson\u003e=1.6.0\"\npip install cmake\npip install ninja\n```\nThen you can build and install opat-core\n```bash\ngit clone https://github.com/4D-STAR/opat-core\ncd opat-core\nmeson setup build --buildtype=release\nmeson compile -C build\n```\nIf you want to run tests you can use meson's build in test command. Note that due to small numerical differences between computers and compilers, some tests may fail. This is expected and can be ignored. Specifically, `TableLattice` tests rely on the ordering / adjacency of Delaunay triangulation which can vary between systems.\n```bash\nmeson test -C build\n```\n\n\nTo install headers, libraries, the pkg_config file (`opatIO.pc`), and the command line utilities\n```bash\nmeson install -C build\n```\n\n---\n\n# Usage\n\n## Python Usage\nSee the `opatIO-py` directory for detailed installation and usage instructions. The Python module allows for both creating and reading OPAT files. A comprehensive API manual can also be found in `opatIO-py/docs/build/latex/opatio.pdf`.\n\nThe general workflow involves creating an `OPAT` object, adding tables to it, and then saving it.\n\n### Creating and Writing an OPAT File\n```python\nfrom opatio import OPAT\n\n# Initialize an OPAT object\nopacityFile = OPAT()\nopacityFile.set_comment(\"This is a sample opacity file\")\nopacityFile.set_source(\"OPLIB\") # Optional: set a source identifier\n\n# Assume you have your data:\n# X, Z: components of the index vector (e.g., Hydrogen and Metal mass fractions)\n# logR: 1D array of log(R) values (e.g., density variable)\n# logT: 1D array of log(T) values (e.g., temperature variable)\n# logKappa: 2D array of log(opacity) values, with shape (len(logR), len(logT))\nX, Z = 0.7, 0.02\nlogR = [1.0, 2.0, 3.0] \nlogT = [4.0, 5.0, 6.0, 7.0]\nlogKappa = [[1.1, 1.2, 1.3, 1.4], [2.1, 2.2, 2.3, 2.4], [3.1, 3.2, 3.3, 3.4]]\n\n\n# Add a table to a new DataCard, indexed by (X, Z)\n# The tag \"data\" identifies this specific table within the DataCard\ncard = opacityFile.add_table(index_vector=(X, Z), \n                             tag=\"data\", \n                             row_values=logR, \n                             col_values=logT, \n                             table_data=logKappa, \n                             rowName=\"logR\", \n                             columnName=\"logT\")\n\n# You can add another table to the same DataCard\n# For example, interpolation coefficients\n# xOrder, yOrder, coeff = ... (your coefficient data)\n# opacityFile.add_table(index_vector=(X,Z), tag=\"interp_coeffs\", \n#                       row_values=x_coeffs_indices, col_values=y_coeffs_indices, table_data=coeffs,\n#                       card=card) # Pass the existing card object\n\n# Save the OPAT file\nopacityFile.save(\"opacity.opat\")\n\n# Optionally, save an ASCII representation (for debugging or human-readable inspection)\nopacityFile.save_as_ascii(\"opacity_ascii.txt\")\n\nprint(\"OPAT file created and saved.\")\n```\n\n### Reading an OPAT File\n```python\nfrom opatio import read_opat\n\n# Load an existing OPAT file\nopacityFile = read_opat(\"opacity.opat\")\n\n# Print the header information\nprint(opacityFile.header)\n\n# Access a specific DataCard by its index vector\n# Note: Index vectors are tuples. For floating point comparisons, \n# direct dictionary-like access might require exact matches.\n# It's often better to iterate or use methods that handle floating point precision if needed.\n# For this example, we assume the index (0.7, 0.02) exists.\ntarget_index = (0.7, 0.02)\nif target_index in opacityFile.cards:\n    data_card = opacityFile.cards[target_index]\n    \n    # Access a specific table within the DataCard by its tag\n    if \"data\" in data_card.tables:\n        table = data_card.tables[\"data\"]\n        print(f\"\\nTable 'data' for index {target_index}:\")\n        print(f\"Row Names (logR): {table.row_values}\")\n        print(f\"Column Names (logT): {table.col_values}\")\n        print(f\"Table Data (logKappa):\\n{table.table_data}\")\n    else:\n        print(f\"Table 'data' not found in card with index {target_index}\")\nelse:\n    print(f\"DataCard with index {target_index} not found.\")\n\n# Iterate through all cards and their tables\nfor index_vec, card in opacityFile.cards.items():\n    print(f\"\\nDataCard at index: {index_vec}\")\n    for tag, table in card.tables.items():\n        print(f\"  Table tag: {tag}, Shape: ({table.num_rows}, {table.num_cols})\")\n```\n\n### Converting OPAL Type I to OPAT\nA utility is provided to convert common OPAL Type I opacity files to the OPAT format.\n```python\nfrom opatio.convert import OPALI_2_OPAT\n\n# Assuming an OPAL Type I file named \"GS98hz\" is in the current directory\nOPALI_2_OPAT(\"GS98hz\", \"gs98hz.opat\")\nprint(\"Converted OPAL Type I file to gs98hz.opat\")\n```\n\n## C++ Library/Header Usage\nThe C++ library is primarily designed for reading and interpolating data from OPAT files. Below are some common usage examples. For a more detailed API manual, refer to the Doxygen documentation generated in `opatIO-cpp/docs/html/index.html` (or the PDF version in `opatIO-cpp/docs/latex/refman.pdf`).\n\n### Linking the C++ Library\n\nThe installation process detailed above generate a package config file `opatIO.pc` that can be used to link against the\nC++ library. You can use this in your `CMakeLists.txt`, `meson.build` files, or directly in your compiler commands. \n\nas a simple example lets say you make a file called `main.cpp` with the following contents:\n```c++\n#include \u003copatIO.h\u003e\n\nint main() {\n    opat::OPAT opat_file = opat::readOPAT(\"example.opat\");\n    std::cout \u003c\u003c \"Successfully read OPAT file: example.opat\" \u003c\u003c std::endl;\n}\n```\nif you were to just run\n```bash\ng++ main.cpp -o main\n```\nThis would fail because the compiler neither knows where to find the `opatIO.h` header file nor the `libopatio.a` library.\n\nHowever, if you use the `pkg-config` tool, you can easily compile and link against the OPAT library:\n```bash\ng++ main.cpp -o main $(pkg-config --cflags --libs opatIO) --std=c++23\n```\n\n\u003e ⚠️ The `--std=c++23` flag is required as the OPAT C++ library makes use of C++23 features. --std=c++17 is the minimum required version, but C++23 is recommended for full compatibility.\n\u003e \n\n### C++ API Usage Examples\n\n#### Accessing a Table by Index and Tag\n\n```c++\n#include \"opatIO.h\"\n#include \"indexVector.h\" // For FloatIndexVector\n#include \u003cstring\u003e\n#include \u003cvector\u003e\n#include \u003ciostream\u003e\n\nint main() {\n    std::string filename = \"example.opat\";\n    try {\n        opat::OPAT opat_file = opat::readOPAT(filename);\n        // Define the index vector for the DataCard you want to access\n        FloatIndexVector target_index({0.35, 0.004}); // Example index\n        // Define the tag for the table within the DataCard\n        std::string table_tag = \"data\"; // Example tag\n\n        const opat::DataCard\u0026 data_card = opat_file[target_index];\n        const opat::OPATTable\u0026 table = data_card[table_tag];\n        \n        std::cout \u003c\u003c \"Table '\" \u003c\u003c table_tag \u003c\u003c \"' at index \" \u003c\u003c target_index \u003c\u003c \":\" \u003c\u003c std::endl;\n        table.print(); // Print the table data\n    } catch (const std::exception\u0026 e) {\n        std::cerr \u003c\u003c \"Error accessing table: \" \u003c\u003c e.what() \u003c\u003c std::endl;\n        return 1;\n    }\n    return 0;\n}\n```\n\n#### Slicing a Table\n```c++\n#include \"opatIO.h\"\n#include \"indexVector.h\"\n#include \u003cstring\u003e\n#include \u003cvector\u003e\n#include \u003ciostream\u003e\n\nint main() {\n    std::string filename = \"example.opat\";\n    try {\n        opat::OPAT opat_file = opat::readOPAT(filename);\n        FloatIndexVector target_index({0.35, 0.004});\n        std::string table_tag = \"data\";\n\n        // Define slices for rows and columns\n        opat::Slice row_slice(0, 6);    // Rows 0 through 5\n        opat::Slice col_slice(25, 36);  // Columns 25 through 35\n\n        const opat::OPATTable\u0026 original_table = opat_file[target_index][table_tag];\n        opat::OPATTable sliced_table = original_table.slice(row_slice, col_slice);\n        \n        std::cout \u003c\u003c \"Sliced table '\" \u003c\u003c table_tag \u003c\u003c \"' at index \" \u003c\u003c target_index \u003c\u003c \":\" \u003c\u003c std::endl;\n        sliced_table.print(); // Print the sliced table\n    } catch (const std::exception\u0026 e) {\n        std::cerr \u003c\u003c \"Error slicing table: \" \u003c\u003c e.what() \u003c\u003c std::endl;\n        return 1;\n    }\n    return 0;\n}\n```\n\n#### Using `TableLattice` for Interpolation\nThe `TableLattice` class allows for N-dimensional linear interpolation of data within an OPAT file.\n\n```c++\n#include \"opatIO.h\"\n#include \"indexVector.h\"\n#include \"tableLattice.h\" // For TableLattice\n#include \u003cstring\u003e\n#include \u003cvector\u003e\n#include \u003ciostream\u003e\n\nint main() {\n    std::string opat_filename = \"your_data.opat\"; // Replace with your OPAT file\n    try {\n        opat::OPAT opat_data = opat::readOPAT(opat_filename);\n\n        // Initialize TableLattice with the loaded OPAT data\n        opat::lattice::TableLattice lattice(opat_data);\n        std::cout \u003c\u003c \"TableLattice initialized.\" \u003c\u003c std::endl;\n\n        // Define the index vector for which you want to interpolate data\n        // Ensure its dimension matches opat_data.header.numIndex\n        FloatIndexVector query_point({0.54, 0.07}); // Example for a 2D OPAT\n\n        // Get the interpolated DataCard\n        opat::DataCard interpolated_card = lattice.get(query_point);\n        std::cout \u003c\u003c \"Interpolated DataCard retrieved for \" \u003c\u003c query_point \u003c\u003c std::endl;\n\n        // Access tables from the interpolated_card as needed\n        // For example, if your OPAT files contain a table tagged \"density\":\n        // if (!interpolated_card.getKeys().empty()) {\n        //     const opat::OPATTable\u0026 density_table = interpolated_card[interpolated_card.getKeys()[0]];\n        //     density_table.print();\n        // }\n\n    } catch (const std::exception\u0026 e) {\n        std::cerr \u003c\u003c \"An error occurred: \" \u003c\u003c e.what() \u003c\u003c std::endl;\n        return 1;\n    }\n    return 0;\n}\n```\n\n## Command Line Utility Usage\nAfter running `meson install -C build`, three command-line utilities will be available in your system's path (you might need to resource your shell, e.g., `source ~/.bashrc` or `source ~/.zshrc`, or open a new terminal). These utilities are:\n\n1.  **`opatHeader`**: Prints out the main header information of an OPAT file.\n    ```bash\n    opatHeader --file path/to/your/file.opat\n    ```\n\n2.  **`opatInspect`**: Prints the main header and the card catalog (index of all data cards) of an OPAT file.\n    ```bash\n    opatInspect --file path/to/your/file.opat\n    ```\n\n3.  **`opatVerify`**: Verifies if the provided file is a valid OPAT file according to the specification. It checks the magic number and performs other integrity checks.\n    ```bash\n    opatVerify --file path/to/your/file.opat\n    ```\n\nNote the `--file` (or `-f`) flag is required before providing the path to the OPAT file for all tools.\n\n---\n\n## Fortran Usage\nA Fortran wrapper around the C++ `opatIO` module is provided. This interface allows Fortran applications to read data from OPAT files. Note that this is primarily for reading and may not receive the same level of feature development as the C++ and Python interfaces.\n\nThe Fortran module and object files are typically built into `build/opatIO-fortran/libopatio_f.a` (and associated `.mod` files in `build/opatIO-fortran/libopatio_f.a.p`). You will need to link against this library and potentially the C++ `opatIO` library as well.\n\nRefer to `opatIO-fortran/readme.md` and `opatIO-cpp/docs/static/fortran.md` for more details.\n\n### Example: Reading an OPAT File in Fortran\n\nThe core workflow involves:\n1. Loading the OPAT file using `load_opat_file`.\n2. Retrieving a specific table using `get_opat_table`.\n3. Checking the returned table structure for errors.\n4. Accessing table data.\n5. Freeing resources with `free_opat_file`.\n\n```fortran\nprogram opat_fortran_example\n    use opat_interface_module\n    use, intrinsic :: iso_c_binding\n    implicit none\n\n    character(len=256)      :: filename\n    real(c_double), target  :: index_vec(2) ! Assuming 2D index vector\n    character(len=32)       :: tag          ! Ensure tag length matches definition in module\n    type(opat_table_f_type) :: my_table\n    integer                 :: load_status\n    integer                 :: i, j\n\n    ! --- Configuration ---\n    filename = 'gs98hz.opat'  ! Ensure this file exists where the program is run\n                              ! A copy can be found in opatIO-cpp/examples\n    index_vec(1) = 0.9_c_double   ! Example X value\n    index_vec(2) = 0.08_c_double  ! Example Z value\n    tag = 'data'              ! Tag of the table to retrieve\n\n    ! --- Load OPAT File ---\n    print *, \"Attempting to load OPAT file: \", trim(filename)\n    load_status = load_opat_file(trim(filename))\n    if (load_status /= 0) then\n        print *, \"ERROR: Failed to load OPAT file. Status: \", load_status\n        stop 1\n    end if\n    print *, \"OPAT file loaded successfully.\"\n\n    ! --- Get Table ---\n    print *, \"Attempting to retrieve table with tag '\", trim(tag), \"' for index [\", index_vec(1), \",\", index_vec(2), \"]\"\n    call get_opat_table(index_vec, trim(tag), my_table)\n\n    ! --- Check for Errors and Process Table ---\n    if (my_table%error_code /= 0) then\n        print *, \"ERROR: Failed to retrieve table. Error code: \", my_table%error_code\n        if (my_table%error_code == 1) print *, \" (Likely cause: Index vector not found)\"\n        if (my_table%error_code == 2) print *, \" (Likely cause: Table tag not found for given index)\"\n    else\n        print *, \"Table retrieved successfully!\"\n        print *, \"Table dimensions: \", my_table%num_rows, \"rows, \", my_table%num_cols, \"cols\"\n\n        ! Check if pointers are associated (important!)\n        if (.not. c_associated(my_table%row_values)) then\n            print *, \"ERROR: Row values pointer not associated.\"\n        else if (.not. c_associated(my_table%col_values)) then\n            print *, \"ERROR: Column values pointer not associated.\"\n        else if (.not. c_associated(my_table%data)) then\n            print *, \"ERROR: Table data pointer not associated.\"\n        else\n            ! Access and print some data (example)\n            print *, \"First row value: \", my_table%row_values(1)\n            print *, \"First col value: \", my_table%col_values(1)\n            print *, \"Data at (1,1): \", my_table%data(1,1)\n\n            ! Example: Print a small part of the table\n            ! print *, \"Sample data:\"\n            ! do i = 1, min(my_table%num_rows, 3)\n            !    write(*,'(3(F8.3,X))') (my_table%data(i,j), j = 1, min(my_table%num_cols,3))\n            ! end do\n        end if\n    end if\n\n    ! --- Free OPAT File Resources ---\n    print *, \"Freeing OPAT file resources.\"\n    call free_opat_file()\n    print *, \"Program finished.\"\n\nend program opat_fortran_example\n```\n**Important Notes for Fortran Usage:**\n- Ensure the OPAT file (e.g., `gs98hz.opat`) is accessible in the directory where the Fortran executable is run.\n- Always check `my_table%error_code` after calling `get_opat_table`.\n- Crucially, call `free_opat_file()` when you are done with the OPAT file to prevent memory leaks.\n- The dimensions of `index_vec` must match the `numIndex` defined in the OPAT file's header.\n- The length of the `tag` character variable should be sufficient for the tags used in your OPAT files.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4d-star%2Fopat-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F4d-star%2Fopat-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4d-star%2Fopat-core/lists"}