https://github.com/ryan-williams/tdbs-dask

https://github.com/ryan-williams/tdbs-dask

Last synced: 20 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/ryan-williams/tdbs-dask
• Owner: ryan-williams
• Created: 2024-10-23T20:44:51.000Z (2 months ago)
• Default Branch: main
• Last Pushed: 2024-11-22T21:01:22.000Z (about 1 month ago)
• Last Synced: 2024-12-08T19:09:49.780Z (26 days ago)
• Language: Python
• Size: 97.7 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# tdbs-dask

Experiments reading TileDB-SOMA data using Dask.

- [TileDBSOMA@rw/ad-dask] supports exporting `Experiment`s and `ExperimentAxisQuery`s to AnnData with Dask-Array X 

- WIP code in this repo transpiles Dask DAGs to TileDB UDFs, e.g.:

  - [Random array + elementwise multiplication](tests/test_transpile_dask.py#L22)

  - [TDBS-Dask pbmc-small dataset](tests/test_transpile_dask.py#L54)

## Install 

```bash

git clone https://github.com/ryan-williams/tdbs-dask && cd tdbs-dask 

pip install -e .

```

## Appendix 

Extends work by [**@ivirshup**]:

- [access_tiledb_w_dask.ipynb]

- [map-blocks-approach-ipynb]

- [write-tiledb-from-dask.ipynb]

[TileDBSOMA@rw/ad-dask]: https://github.com/single-cell-data/TileDB-SOMA/compare/main...rw/ad-dask

[**@ivirshup**]: https://github.com/ivirshup

[access_tiledb_w_dask.ipynb]: https://gist.github.com/ivirshup/8500d9a874ea9313ca87c0d5e46886e9

[map-blocks-approach-ipynb]: https://gist.github.com/ivirshup/dc39029ad439cef4755e45582fc35541#file-map-blocks-approach-ipynb

[write-tiledb-from-dask.ipynb]: https://gist.github.com/ivirshup/018bb8ae1ea7746db768c3672b8a007b

[Dask]: https://dask.org/

[CELLxGENE Census]: https://chanzuckerberg.github.io/cellxgene-census/

[Anndata]: https://anndata.readthedocs.io/en/latest/