{"id":22111382,"url":"https://github.com/hebirobotics/mfl","last_synced_at":"2026-01-11T17:40:42.027Z","repository":{"id":50537324,"uuid":"148813146","full_name":"HebiRobotics/MFL","owner":"HebiRobotics","description":"A Java library for reading and writing MATLAB's MAT File format","archived":false,"fork":false,"pushed_at":"2023-10-05T21:24:40.000Z","size":4982,"stargazers_count":65,"open_issues_count":0,"forks_count":12,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-07-14T18:04:38.021Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HebiRobotics.png","metadata":{"files":{"readme":"README.adoc","changelog":"CHANGELOG.adoc","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-09-14T16:11:46.000Z","updated_at":"2025-07-01T09:03:18.000Z","dependencies_parsed_at":"2023-01-30T18:31:20.142Z","dependency_job_id":null,"html_url":"https://github.com/HebiRobotics/MFL","commit_stats":null,"previous_names":["hebirobotics/matfileio","hebirobotics/matfilelib"],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/HebiRobotics/MFL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HebiRobotics%2FMFL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HebiRobotics%2FMFL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HebiRobotics%2FMFL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HebiRobotics%2FMFL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HebiRobotics","download_url":"https://codeload.github.com/HebiRobotics/MFL/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HebiRobotics%2FMFL/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266972894,"owners_count":24014597,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-01T10:40:20.585Z","updated_at":"2026-01-11T17:40:41.999Z","avatar_url":"https://github.com/HebiRobotics.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"image:https://img.shields.io/badge/License-Apache%202.0-blue.svg[link=https://opensource.org/licenses/Apache-2.0]\nimage:https://github.com/HebiRobotics/MFL/workflows/Maven%20Build/badge.svg[Maven Build]\n\n= MAT File Library\n\n== Introduction\n\nThe MAT File Library (MFL) is a Java library for reading and writing MAT Files that are compatible with MATLAB's MAT-File Format.\n\nIt's overall design goals are; 1) to provide a user-friendly API that adheres to MATLAB's semantic behavior, 2) to support working with large amounts of data in heap constrained or allocation limited environments, 3) to allow users to serialize custom data classes without having to convert to temporary objects.\n\nThis library currently supports the https://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf[Level 5 MAT-File Format] which has been the default for `.mat` and `.fig` files since https://en.wikipedia.org/wiki/MATLAB#Release_history[MATLAB 5.0 (R8)] in 1996. This includes `save` flags `-v6` and `-v7`, but not `-v4` or `-v7.3`. See MAT-File https://de.mathworks.com/help/matlab/import_export/mat-file-versions.html[Versions] for more info.\n\nThis library is free, written in 100% Java, and has been released under an Apache v2.0 license. It works with all Java versions from Java 6 to at least Java 21.\n\n== Maven Central\n\nMFL is in the Maven central repository and can easily be added to Maven, Gradle, and similar project managers.\n\n```XML\n\u003cdependency\u003e\n    \u003cgroupId\u003eus.hebi.matlab.mat\u003c/groupId\u003e\n    \u003cartifactId\u003emfl-core\u003c/artifactId\u003e\n    \u003cversion\u003e0.5.15\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nIn case you are working with link:http://ejml.org/[EJML] matrix types, you may also want to include the `mfl-ejml` extension.\n\n```XML\n\u003cdependency\u003e\n    \u003cgroupId\u003eus.hebi.matlab.mat\u003c/groupId\u003e\n    \u003cartifactId\u003emfl-ejml\u003c/artifactId\u003e\n    \u003cversion\u003e0.5.15\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n== Basic Usage\n\nThe **link:./mfl-core/src/main/java/us/hebi/matlab/mat/format/Mat5.java[Mat5]** class contains static factory methods that serve as the starting points for the public API. The basic types (e.g. `Struct`, `Cell`, `Sparse`, `Char`) follow their corresponding MATLAB semantics (e.g. a struct can't be logical) and provide a fluent API for the most common use cases. All of the numerical types (e.g. `double`, `uint8`, `int16`, `logical`, ...) are represented by the `Matrix` type which offers a unified interface for handling numeric conversions.\n\nSimilarly to MATLAB, all types are internally represented as a multi-dimensional `Array`. For example, a scalar struct is really a `1x1` array of structs. Most types offer convenience overloads for scalar, linear, 2-dimensional, and N-dimensional access.\n\nBelow are some example snippets on how to use the API. For more examples, please refer to **link:./mfl-core/src/test/java/us/hebi/matlab/mat/tests/Mat5Examples.java[Mat5Examples]** and the various unit tests.\n\n=== Creating and Writing MAT Files\n\nA `MatFile` is a data structure that contains a collection of named `Array` variables.\n\n```Java\n// Create MAT file with a scalar in a nested struct\nMatFile matFile = Mat5.newMatFile()\n    .addArray(\"var1\", Mat5.newString(\"Test\"))\n    .addArray(\"var2\", Mat5.newScalar(7))\n    .addArray(\"var3\", Mat5.newStruct().set(\"x\", Mat5.newScalar(42)));\n```\n\nThe simplest way to write a `MatFile` to disk is to use the built-in convenience functions.\n\n```Java\n// Serialize to disk using default configurations\nMat5.writeToFile(matFile, \"data.mat\");\n```\n\nMore complex use cases can leverage the `Sink` interface to write the `MatFile` to arbitrary data outputs. Sinks for various outputs (e.g. buffer, stream, or file) can be created via the `Sinks` factory or by extending `AbstractSink`.\n\n```Java\n// Serialize to a streaming file sink using the default configuration\ntry(Sink sink = Sinks.newStreamingFile(\"data.mat\")){\n    matFile.writeTo(sink);\n}\n```\n\nBuffers and other sinks that need to be pre-allocated ahead of time can use `MatFile::getUncompressedSerializedSize()` to calculate the maximum expected size beforehand. Note that `compressed` data should always be smaller, but there are some corner cases where very small entries such as scalars can actually increase the result by a few bytes.\n\nThe `Writer` API provides further degrees of customization for writing the data. For example, the `MatFile` can be written to a memory-mapped file using a custom `Deflate` (compression algorithm) level. We can initialize the file with the maximum expected size, and then automatically truncate it once the `Sink` is closed.\n\n```Java\n// Serialize to memory-mapped file w/ custom deflate level\nint safetySize = 256; // some (small) arrays may become larger when compressed\nlong maxExpectedSize = matFile.getUncompressedSerializedSize() + safetySize;\ntry(Sink sink = Sinks.newMappedFile(new File(\"data.mat\"), Casts.sint32(maxExpectedSize))){\n    Mat5.newWriter(sink)\n        .setDeflateLevel(Deflater.BEST_SPEED)\n        .writeMat(matFile);\n}\n```\n\n=== Reading MAT Files\n\nThere are convenience wrappers for reading from files as well. The read API is setup such that users can navigate through known file structures without requiring casts or temporary variables.\n\n```Java\n// Read scalar from nested struct\ndouble value = Mat5.readFromFile(\"data.mat\")\n    .getStruct(\"var3\")\n    .getMatrix(\"x\")\n    .getDouble(0);\n```\n\nA `MatFile` can also be read from a `Source`. Similar to `Sink`, a `Source` can represent various types of data inputs. `MatFile::getEntries()` can be used to iterate all variables inside a mat file.\n\n```Java\n// Iterate over all entries in the mat file\ntry(Source source = Sources.openFile(\"data.mat\")){\n    MatFile mat = Mat5.newReader(source).readMat();\n    for (MatFile.Entry entry : mat.getEntries()) {\n        System.out.println(entry);\n    }\n}\n```\n\n=== Advanced Filtering\n\nIn cases where users are only interested in reading a subset of entries, unwanted entries can be ignored by specifying an `EntryFilter`.\n\n```Java\n// Filter arrays that follow some criteria based on the name, type, dimension, or global/logical flags\ntry(Source source = Sources.openFile(\"data.mat\")){\n    MatFile mat = Mat5.newReader(source)\n        .setEntryFilter(header -\u003e header.isGlobal())\n        .readMat();\n}\n```\n\nThe filter gets applied only at the root level, so arrays inside a struct or cell array won't be filtered separately.\n\n=== Concurrent Compression\n\nAlmost all of the CPU time spent on reading or writing MAT files is related to compression. Fortunately, root entries are compressed independently from one another, so it's possible to do the work multi-threaded.\n\nUsers can enable concurrent reading by passing an executor service into the reader. In order to activate, the `Source` must also support sub-views (slices) on the underlying data (i.e. byte buffers or memory mapped files).\n\n```Java\n// Concurrent Decompression\nExecutorService executor = Executors.newFixedThreadPool(numThreads);\ntry(Source source = Sources.openFile(\"data.mat\")){\n    MatFile mat = Mat5.newReader(source)\n        .enableConcurrentDecompression(executor)\n        .readMat();\n} finally {\n    executor.shutdown();\n}\n```\n\nConcurrent writing unfortunately requires a temporary buffer for each root entry due to the size not being known ahead of time. The buffer allocation can be customized in case users want to use buffer-pools or memory-mapped buffers.\n\n```Java\n// Concurrent Compression\nExecutorService executor = Executors.newFixedThreadPool(numThreads);\ntry(Sink sink = Sinks.newStreamingFile(\"data.mat\")){\n    Mat5.newWriter(sink)\n        .enableConcurrentCompression(executor)\n        .setDeflateLevel(Deflater.BEST_SPEED)\n        .writeMat(mat);\n} finally {\n    executor.shutdown();\n}\n```\n\nThe table below shows a rough performance comparison of working with one of our production data logs.\n\n[width=\"100%\",options=\"header\",cols=\"a,a,a,a,a\"]\n|====================\n| Compression | Size | Threads | Write Time | Read Time\n| BEST_COMPRESSION | 144 MB | 1 | 280 sec | 3.5 sec\n| BEST_COMPRESSION | 144 MB | 8 | 47 sec | 0.8 sec\n| BEST_SPEED | 156 MB | 1 | 7.2 sec | 3.6 sec\n| BEST_SPEED | 156 MB | 8 | 1.5 sec | 0.8 sec\n| NO_COMPRESSION | 422 MB | 1 | 0.07 sec | 0.2 sec\n|====================\n\nThe data set was very multi-threading friendly (33x [95946x18] double matrices on the root level) and first loaded into memory to avoid disk access bottlenecks. The tests were done on a quad core with hyper-threading (Intel NUC6i7kyk).\n\n=== Serializing Custom Classes\n\nWe often encountered cases where we needed to serialize data from an existing math library. Rather than having to convert the data into an API class, we added the ability to create light-weight wrapper classes that serialize the desired data directly.\n\nIn order for a class to be serializable, it needs to implement the `Array` interface (easiest way is  to extend `AbstractArray`) as well as the `Mat5Serializable` interface. For examples, please take a look at the `mfl-ejml` module or the link(s) below:\n\n* link:./mfl-core/src/test/java/us/hebi/matlab/mat/tests/serialization/StreamingDoubleMatrix2D.java[StreamingDoubleMatrix2D] streams incoming row-major data into temporary files and combines them on serialization\n\n==== Efficient Java Matrix Library (EJML)\n\nlink:http://ejml.org/[EJML] is a popular linear algebra library for Java. The `mfl-ejml` module has preliminary support for converting between MAT files and EJML data types.\n\nThe serialization wrappers are very light and serialize the contained data into the `MAT File Format` directly without requiring additional memory for storing any intermediate data.\n\n```Java\n// Add single EJML matrix to root level\nMatFile mat = Mat5.newMatFile();\nmat.addArray(\"DMatrix\", Mat5Ejml.asArray(new DMatrixRMaj(rows, cols)));\n\n// Add multiple EJML matrices to sub-structure\nMatFile mat = Mat5.newMatFile().addArray(\"struct\", Mat5.newStruct()\n        .set(\"FMatrix\", Mat5Ejml.asArray(new FMatrixRMaj(rows, cols)))\n        .set(\"CMatrix\", Mat5Ejml.asArray(new CMatrixRMaj(rows, cols)))\n        .set(\"ZMatrix\", Mat5Ejml.asArray(new ZMatrixRMaj(rows, cols))));\n```\n\nAfter reading a `MAT File` the contained Matrix types can be converted to a user supplied EJML matrix via `output = Mat5Ejml.convert(matrix, output)`. The output matrix will be reshaped as needed.\n\n```Java\n// Convert Matrix to EJML Type\nMatFile mat = Mat5.newMatFile();\nDMatrixRMaj dMatrix = Mat5Ejml.convert(mat.getArray(\"DMatrix\"), new DMatrixRMaj(0, 0));\n```\n\n== General Notes\n\n=== Memory Efficient Serialization\n\nThe MAT 5 format stores all data fields with a header tag that contains the number of bytes and how they should be interpreted. Rather than writing into temporary buffers to determine the serialized size, we added ways to pre-compute all deterministic sizes beforehand.\n\nThe only non-deterministic case is compressing data at the root level, which we can work around by writing a dummy size and overwriting it once the final size is known. Thus, enabling compression requires the root level sink to support position seeking (i.e. in-memory buffers, memory mapped files, or random access files).\n\n=== Support for Undocumented Features\n\nUnfortunately, MAT 5 files have several features that aren't covered in the official documentation. This includes most of the recently added types (`table`, `timeseries`, `string`, ...), `handle` classes, `function handles`, `.fig` files, `Simulink` outputs, etc.\n\nOur current implementation supports reading all of the `.mat` and `.fig` files we were able to generate. It also supports editing and saving of the loaded MAT files, e.g., adding entries, changing matrices, or using a different compression level. However, changes to the undocumented parts, such as setting a property on a `handle` class, will not be saved.\n\n== Building Sources\n\nThe created sources include unit tests that make use of Java 7 and 8 syntax, so the project needs to be compiled with at least JDK 8.\n\n    mvn package\n\nFor more information, please check the CI build-script link:Jenkinsfile[]\n\n== Acknowledgements\n\nhttps://github.com/diffplug/matfilerw[MatFileRW] (active fork of https://github.com/gradusnikov/jmatio[JMatIO] maintained by link:http://diffplug.com/[DiffPlug]) served as an inspiration for parts of the implementation as well as a source for test data. We ended up porting and supporting all of their unit tests with the exception of `Base64 MDL` decoding (which we couldn't figure out the use case for).\n\nThe implementation for reading the undocumented `MCOS` (MATLAB Class Object System) data is based on https://github.com/mbauman[Matt Bauman]'s http://nbviewer.jupyter.org/gist/mbauman/9121961[reverse engineering efforts] as well as MatFileRW's implementation by https://github.com/MJDSys[Matthew Dawson].\n\n`Preconditions` was copied from link:https://github.com/google/guava[Guava].\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhebirobotics%2Fmfl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhebirobotics%2Fmfl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhebirobotics%2Fmfl/lists"}