{"id":16977880,"url":"https://github.com/jcmgray/xyzpy","last_synced_at":"2026-04-30T01:01:37.961Z","repository":{"id":45063106,"uuid":"59599166","full_name":"jcmgray/xyzpy","owner":"jcmgray","description":"Efficiently generate and analyse high dimensional data.","archived":false,"fork":false,"pushed_at":"2024-09-05T17:58:17.000Z","size":24769,"stargazers_count":69,"open_issues_count":8,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-30T03:49:40.508Z","etag":null,"topics":["distributed","multidimensional-arrays","pandas","parallel","plot","xarray"],"latest_commit_sha":null,"homepage":"http://xyzpy.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jcmgray.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-05-24T18:41:21.000Z","updated_at":"2025-03-14T18:56:23.000Z","dependencies_parsed_at":"2023-01-20T16:00:46.806Z","dependency_job_id":"e743077e-edc1-4a8e-aa34-234bbb2bda54","html_url":"https://github.com/jcmgray/xyzpy","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jcmgray%2Fxyzpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jcmgray%2Fxyzpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jcmgray%2Fxyzpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jcmgray%2Fxyzpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jcmgray","download_url":"https://codeload.github.com/jcmgray/xyzpy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251638760,"owners_count":21619663,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed","multidimensional-arrays","pandas","parallel","plot","xarray"],"created_at":"2024-10-14T01:30:00.893Z","updated_at":"2026-04-30T01:01:37.945Z","avatar_url":"https://github.com/jcmgray.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://github.com/jcmgray/xyzpy/blob/main/docs/_static/xyzpy-logo-title.png?raw=true\" alt=\"symmray logo\" width=\"640\"/\u003e\n\n[![tests](https://github.com/jcmgray/xyzpy/actions/workflows/tests.yml/badge.svg)](https://github.com/jcmgray/xyzpy/actions/workflows/tests.yml)\n[![codecov](https://codecov.io/gh/jcmgray/xyzpy/branch/main/graph/badge.svg?token=Q5evNiuT9S)](https://codecov.io/gh/jcmgray/xyzpy)\n[![Docs](https://readthedocs.org/projects/xyzpy/badge/?version=latest)](https://xyzpy.readthedocs.io)\n[![PyPI](https://img.shields.io/pypi/v/xyzpy?color=teal)](https://pypi.org/project/xyzpy/)\n[![Anaconda-Server Badge](https://anaconda.org/conda-forge/xyzpy/badges/version.svg)](https://anaconda.org/conda-forge/xyzpy)\n[![Pixi Badge](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/prefix-dev/pixi/main/assets/badge/v0.json)](https://pixi.sh)\n\n-------------------------------------------------------------------------------\n\n[`xyzpy`](https://github.com/jcmgray/xyzpy) is python library for efficiently\ngenerating, manipulating and plotting data with a lot of dimensions, of the\ntype that often occurs in numerical simulations. It stands wholly atop the\nlabelled N-dimensional array library [`xarray`](http://xarray.pydata.org). The\nproject's documentation is hosted on [readthedocs](http://xyzpy.readthedocs.io).\n\nThe aim is to take the pain and errors out of generating and exploring data\nwith a high number of possible parameters. This means:\n\n- you don't have to write super nested for loops\n- you don't have to remember which arrays/dimensions belong to which variables/parameters\n- you don't have to parallelize over or distribute runs yourself\n- you don't have to worry about loading, saving and merging disjoint data\n- you don't have to guess when a set of runs is going to finish\n- you don't have to write batch submission scripts or leave the notebook to use SLURM, PBS or SGE\n- you don't have to lose progress if your run is interrupted\n- you don't have to fiddle with `CUDA_VISIBLE_DEVICES` or `taskset` to assign GPU devices or CPU cores to different runs\n\nTo this data generation functionality, ``xyzpy`` adds a simple plotting\ninterface accessed via [`ds.xyz.plot()`](https://xyzpy.readthedocs.io/en/latest/autoapi/xyzpy/plot/infiniplot/index.html#xyzpy.plot.infiniplot.infiniplot)\nthat automatically maps dataset dimensions to visual elements including color,\nmarker, marker size, line style, line width, subplot rows and columns, and text\nannotations. It also adds various other utilities for timing and tracking\nmemory usage, and for visualizing matrices and high dimensional tensors.\n\n## Quick-start\n\nHere's a simple example of generating and plotting a 5D function that uses the\nhigh level driver [`xyz.cultivate()`](https://xyzpy.readthedocs.io/en/latest/autoapi/xyzpy/index.html#xyzpy.cultivate)\nto handle a full cycle of data generation:\n\n```python\nimport xyzpy as xyz\n\ndef foo(x, delta, p, amp=1.0, C=0.0):\n    return {\"fx\": amp * (x - delta) ** p + C}\n\n# cultivate!\n# 0. annotate the function\n# 1. write missing parameters combinations to disk ('sow')\n# 2. compute those, with results stored persistenly to disk ('grow')\n# 3. load results into a xarray.Dataset, merging with existing ('reap')\nds = xyz.cultivate(\n    foo,\n    # this specifies we'll return a dict of named data_vars ourselves\n    var_names=None,\n    # this specifies we'll harvest results to the file \"foo.h5\"\n    data_name=\"foo.h5\",\n    # compute the outer product of these parameter combinations\n    combos=dict(\n        x=[-2 + i * 0.25 for i in range(17)],\n        p=[1, 2, 3],\n        delta=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0],\n        C=[-2.0, 1.0, 4.0],\n        amp=[-1.0, 1.0],\n    ),\n)\n\n# plot!\n# - we can map pretty much any coordinate to any visual property\n# - we can map to a palette (\"hue\") as well as position within that (\"color\")\nfig, axs = ds.xyz.plot(\n    x=\"x\",\n    y=\"fx\",\n    yscale=\"symlog\",\n    ylabel=\"$f(x)$\",\n    hue=\"C\",\n    markeredgecolor=\"C\",\n    color=\"delta\",\n    marker=\"delta\",\n    col=\"p\",\n    row=\"amp\",\n    markersize=3,\n)\n\n# clean up!\n# - if we didn't delete the dataset, next run will only compute missing data\n!rm foo.h5\n```\n\n![example](docs/ex_simple.png)\n\n\nPlease see the [docs](http://xyzpy.readthedocs.io) for more information.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjcmgray%2Fxyzpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjcmgray%2Fxyzpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjcmgray%2Fxyzpy/lists"}