https://github.com/dnbaker/diskmat
mmap-backed blaze matrices
https://github.com/dnbaker/diskmat
Last synced: about 1 month ago
JSON representation
mmap-backed blaze matrices
- Host: GitHub
- URL: https://github.com/dnbaker/diskmat
- Owner: dnbaker
- Created: 2020-05-05T18:02:25.000Z (about 6 years ago)
- Default Branch: main
- Last Pushed: 2020-09-18T20:26:02.000Z (over 5 years ago)
- Last Synced: 2025-11-28T03:48:24.706Z (7 months ago)
- Language: C++
- Size: 7.81 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
### diskmat
diskmat is a simple, header-only tool for providing a generic front-end to matrices that may not fit into memory.
It consists of two structures: `diskmat::DiskMat`, which uses mmap to store the matrix, and `diskmat::PolymorphicMat`,
which defaults to in-memory if the space consumed is below a threshold and falls back to disk-based management afterward.
### Example
```c++
#include
int main() {
diskmat::PolymorphicMat pmat(100000, 1000000);
// pmat is now a 4 terabyte matrix on disk
// Initializing
std::uniform_real_distribution urd;
for(size_t i = 0; i < (~pmat).rows(); ++i) {
auto r = row(~pmat, i);
r = blaze::generate(1000000, [&urd](auto x){std::mt19937_64 mt(x); return urd(mt);});
}
// Next, we can perform our math over the mmap'd data transparently
// This includes reductions:
blaze::DynamicVector meanvec = trans(blaze::mean(~pmat));
// Matrix multiplications, inc. via BLAS bindings
// blaze::DynamicVector mvp = (~pmat) * meanvec;
// and most other techniques
diskmat::PolymorphicMat smallermat(1000, 40000); // This is allocated on the heap
}
```
This can be simplified, as desired, by simply getting a reference to the base matrix (`auto &matr = ~pmat`).