{"id":15061950,"url":"https://github.com/stagedml/pylightnix","last_synced_at":"2025-04-10T09:45:55.484Z","repository":{"id":62581383,"uuid":"232791295","full_name":"stagedml/pylightnix","owner":"stagedml","description":"A Nix-style immutable data management library in Python","archived":false,"fork":false,"pushed_at":"2024-03-14T23:09:52.000Z","size":932,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-17T09:38:59.349Z","etag":null,"topics":["immutable-infrastructure","machine-learning","nix","package-management","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stagedml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-01-09T11:16:27.000Z","updated_at":"2024-09-17T12:47:46.000Z","dependencies_parsed_at":"2023-12-28T12:40:23.424Z","dependency_job_id":"64c44c0d-76a9-4464-b739-8c012daff825","html_url":"https://github.com/stagedml/pylightnix","commit_stats":{"total_commits":623,"total_committers":1,"mean_commits":623.0,"dds":0.0,"last_synced_commit":"a6e55186ad9c7f29ed6de7c0d5daf36320f0ddff"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stagedml%2Fpylightnix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stagedml%2Fpylightnix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stagedml%2Fpylightnix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stagedml%2Fpylightnix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stagedml","download_url":"https://codeload.github.com/stagedml/pylightnix/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248196319,"owners_count":21063392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["immutable-infrastructure","machine-learning","nix","package-management","python"],"created_at":"2024-09-24T23:27:56.375Z","updated_at":"2025-04-10T09:45:55.453Z","avatar_url":"https://github.com/stagedml.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![coverage](https://codecov.io/gh/stagedml/pylightnix/branch/master/graph/badge.svg)\n\nPylightnix\n==========\n\nPylightnix is a minimalistic Python library for managing filesystem-based\nimmutable data, inspired by [Purely Functional Software Deployment Model thesis\nby Eelco Dolstra](https://edolstra.github.io/pubs/phd-thesis.pdf) and the\n[Nix](https://nixos.org) package manager.\n\nThe library may be thought as of low-level API for creating caching wrappers for\na subset of Python functions. In particular, Pylightnix allows user to\n\n- Prepare (or, in our terms - **instantiate**) the computation plan aimed at\n  creating a tree of linked immutable stage objects, stored in the filesystem.\n- Implement (in our terms - **realize**) the prepared computation plan, access\n  the resulting artifacts. Pylightnix is able to handle **non-deterministic**\n  results of the computation.\n- Handle changes in the computation plan, re-use the existing artifacts whenever\n  possible.\n- Gain full control over the cached data by inspecting the storage\n\n``` python\nimport numpy as np\nfrom pylightnix import *\n\nfsinit('_pylightnix')\n\n@autostage\ndef stage_dataset(A=100, file=[selfref, 'dataset.npy']):\n  np.save(file, np.arange(0, A))\n\n@autostage(nouts=10)\ndef stage_train(rindex:int, dataset=stage_dataset, model=[selfref, 'model.npy']):\n  ds = np.load(dataset.file.val)\n  np.save(model, rindex)\n\nrrefs=realizeMany(instantiate(stage_train))\nassert len(rrefs)\u003e=10\n```\n\n## Documentation\n\nQuickStart\n[\\[PDF\\]](https://raw.github.com/stagedml/pylightnix-docs/master/Pylightnix-QuickStart-latest.pdf)\n\\| API Referece [\\[MD\\]](./docs/Reference.md)\n\nDemos:\n\n- [GNU Hello](./docs/demos/HELLO.md), turn Pylightnix into a toy\n  package manager to build the GNU Hello from sources.\n- [MDRUN](./docs/demos/MDRUN.py), evaluate code sections of a Markdown\n  document, cache the results between runs.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstagedml%2Fpylightnix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstagedml%2Fpylightnix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstagedml%2Fpylightnix/lists"}