{"id":47879402,"url":"https://github.com/cogent3/scinexus","last_synced_at":"2026-05-08T07:02:55.703Z","repository":{"id":348437180,"uuid":"1197775372","full_name":"cogent3/scinexus","owner":"cogent3","description":"composable apps for scientific programming","archived":false,"fork":false,"pushed_at":"2026-04-22T00:37:01.000Z","size":569,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-22T02:44:47.458Z","etag":null,"topics":["bioinformatics","scientific-computing","software-engineering","software-factory"],"latest_commit_sha":null,"homepage":"https://scinexus.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cogent3.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-31T21:55:03.000Z","updated_at":"2026-04-22T00:36:08.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/cogent3/scinexus","commit_stats":null,"previous_names":["cogent3/scinexus"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/cogent3/scinexus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogent3%2Fscinexus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogent3%2Fscinexus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogent3%2Fscinexus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogent3%2Fscinexus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cogent3","download_url":"https://codeload.github.com/cogent3/scinexus/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogent3%2Fscinexus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32770544,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T02:36:36.067Z","status":"ssl_error","status_checked_at":"2026-05-08T02:36:07.210Z","response_time":54,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","scientific-computing","software-engineering","software-factory"],"created_at":"2026-04-04T01:40:31.130Z","updated_at":"2026-05-08T07:02:53.601Z","avatar_url":"https://github.com/cogent3.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/cogent3/scinexus/refs/heads/main/docs/images/logo-text-bw.svg\" alt=\"scinexus\" width=\"300\"\u003e\n\u003c/p\u003e\n\n[![Coverage Status](https://coveralls.io/repos/github/cogent3/scinexus/badge.svg?branch=main)](https://coveralls.io/github/cogent3/scinexus?branch=main) [![Codacy Badge](https://app.codacy.com/project/badge/Grade/fd8810efd4f142069bd84144e14350b4)](https://app.codacy.com/gh/cogent3/scinexus/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![CI](https://github.com/cogent3/scinexus/actions/workflows/ci.yml/badge.svg)](https://github.com/cogent3/scinexus/actions/workflows/ci.yml)\n\n*`scinexus` is a framework for rapid development of data processing applications. It enables interoperability between objects through defined data types, allowing development of scientific domain app ecosystems. Just as `attrs` and `dataclasses` use type hints to simplify data type definition, `scinexus` uses them to simplify writing best-practice scientific algorithms.*\n\nMany scientific problems require repeating calculations across many files or database records. Such tasks suit data-level parallelism, but writing robust, maintainable code for them is often tedious and quickly becomes complex.\n\nAs the Unix philosophy articulates, writing algorithms that do one thing well and can be composed together through piping data of known type is a *Very Good Thing*™.\n\n**`scinexus` encourages this design pattern and eliminates the boilerplate.** We leverage the Python type annotation system to govern the compatibility (composability) of different applications. This enables in-process composition of your applications with validation of the consistency of the pipeline and the consistency of the data being run through it.\n\n**`scinexus` is designed for scientific reproducibility.** Scientific computations should record all conditions needed to reproduce an analysis. `scinexus` reduces the effort by intercepting all arguments (including defaults) used in app construction and logging the resulting app state.\n\n## Examples\n\nDevelopers can choose inheriting from a base class or use the `scinexus.define_app` decorator to make composable apps. The following examples show simple composition\n\n\u003cdetails\u003e\n\u003csummary\u003eLoading files so missing data does not cause a crash\u003c/summary\u003e\n\n```python\nfrom scinexus import define_app\n\n\n@define_app(app_type=\"loader\")\ndef read_json(path: str) -\u003e dict:\n    import json\n\n    with open(path) as f:\n        return json.load(f)\n\n\n@define_app\ndef validate(data: dict, required_field: str) -\u003e dict:\n    if required_field not in data:\n        # this becomes a NotCompleted sentinel object\n        # your run doesn't crash!\n        raise ValueError(f\"missing {required_field!r} field\")\n    return data\n\n\napp = read_json() + validate(required_field=\"name\")\n```\n\nYou can apply `app` to a single file path as `app(filepath)`, or operate in parallel (and show a progress bar) on a sequence of file paths as\n\n```python\nresults = list(app.as_completed([\"some_file_path.json\", \"some_other_file_path.json\"], parallel=True, show_progress=True)\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eA contrived numerical example\u003c/summary\u003e\n\n```python\nfrom scinexus import define_app\n\n\n@define_app\ndef normalise(values: list[float]) -\u003e list[float]:\n    lo, hi = min(values), max(values)\n    return [(v - lo) / (hi - lo) for v in values]\n\n\n@define_app\ndef threshold(values: list[float]) -\u003e list[bool]:\n    return [v \u003e 0.5 for v in values]\n\n\napp = normalise() + threshold()\napp([1.0, 5.0, 3.0, 9.0])\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eA configurable app\u003c/summary\u003e\n\n```python\nfrom scinexus import define_app\n\n\n@define_app(app_type=\"loader\")\ndef load_csv(path: str) -\u003e list[dict]:\n    import csv\n\n    with open(path) as f:\n        return list(csv.DictReader(f))\n\n\n@define_app\nclass summarise:\n    def __init__(self, column: str) -\u003e None:\n        \"\"\"column contains the values to produce summary stats for\"\"\"\n        self.column = column\n\n    def main(self, rows: list[dict]) -\u003e dict[str, float]:\n        vals = [float(r[self.column]) for r in rows]\n        return {\"mean\": sum(vals) / len(vals), \"min\": min(vals), \"max\": max(vals)}\n\n\napp = load_csv() + summarise(column=\"price\")\n```\n\n\u003c/details\u003e\n\n## Features\n\n- Type checking at composition time\n- Durable computing -- failures recorded as `NotCompleted` records, not exceptions\n- Data-level parallel execution with pluggable backends (stdlib, loky, MPI, or custom)\n- Progress bars (`tqdm` or `rich`)\n- Automated logging and citation tracking\n- Checkpointing via data stores (directory, SQLite)\n\n## Installation\n\n```bash\npip install scinexus\n```\n\n## History\n\nThe app framework and utility functions in `scinexus` incubated inside [cogent3](https://github.com/cogent3/cogent3) from March 2019, accumulating over seven years of development, testing, and real-world use in computational genomics before being extracted into a standalone package. The design is mature and has underpinned analyses in published studies.\n\nThe extraction into `scinexus` makes the infrastructure available to any scientific Python project, free of the `cogent3` dependency. See the [changelog](changelog.md) for a detailed list of changes from the cogent3 app infrastructure.\n\nWe acknowledge here that many members of the `cogent3` community contributed to the code that now lives here, including [@rmcar17](https://github.com/rmcar17), [@Nick-Foto](https://github.com/Nick-Foto), [@KatherineCaley](https://github.com/KatherineCaley), [@fredjaya](https://github.com/fredjaya), and [@khiron](https://github.com/khiron).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogent3%2Fscinexus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcogent3%2Fscinexus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogent3%2Fscinexus/lists"}