{"id":30133932,"url":"https://github.com/fsspec/projspec","last_synced_at":"2025-08-10T20:37:03.208Z","repository":{"id":292170857,"uuid":"979645709","full_name":"fsspec/projspec","owner":"fsspec","description":"A project about projects","archived":false,"fork":false,"pushed_at":"2025-08-06T20:03:28.000Z","size":524,"stargazers_count":6,"open_issues_count":5,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-10T00:39:10.268Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fsspec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-07T21:00:50.000Z","updated_at":"2025-08-08T20:42:30.000Z","dependencies_parsed_at":"2025-08-03T00:32:38.878Z","dependency_job_id":"92e8069a-2621-4f24-871b-6614e5d70976","html_url":"https://github.com/fsspec/projspec","commit_stats":null,"previous_names":["martindurant/projspec","fsspec/projspec"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fsspec/projspec","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fsspec%2Fprojspec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fsspec%2Fprojspec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fsspec%2Fprojspec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fsspec%2Fprojspec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fsspec","download_url":"https://codeload.github.com/fsspec/projspec/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fsspec%2Fprojspec/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269784746,"owners_count":24475455,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-10T02:00:08.965Z","response_time":71,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-10T20:37:01.903Z","updated_at":"2025-08-10T20:37:03.166Z","avatar_url":"https://github.com/fsspec.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ``projspec``\n\nA common interface to code projects.\n\n### What is a project?\n\nFrom the point of view of this library, any directory of stuff with metadata\ndescribing what that stuff is (contents), what to do with it (artifacts) is\na project. This includes things that might be called in other contexts\nan \"application\" or \"work-space.\"\n\nThis is implemented first in the context of the python-data ecosystem, so we\nwill be concerned with project types that are common in this field, but in\nprinciple a wide range of things.\n\n### Niche\n\nThere are a large number of project-oriented tools already in existence,\ndescribing a similarly large number of things about those projects. The\ntools have a lot of overlap with one-another but also unique use cases.\n\nThe following diagram shows an aspirational set of things we wish to\nconsider initially:\n\n![project diagram](https://raw.githubusercontent.com/martindurant/projspec/refs/heads/main/projspec.jpg)\n\nWhere we define:\n- project spec: a way to define a project type, often tied to a particular tool.\n- contents: the things that exist within the project, either as concrete files,\n as specs (in YAML, toml or other metadata) or links to other projects.\n- artifacts: the things a project makes, outputs or tasks that the project\n can execute.\n\n### Why\n\nThe following are the principal features we aim to provide, with the simplest\nfirst:\n\n##### Unified interface\n\nYou can interact with all project types the same way. If you only ever use one\nproject management tool, this is not so exciting. However, if you have multiple\nproject types, switching between them can be annoying, especially for rarely used\nones (helm is a good example of this in my personal experience).\n\nThis should integrate nicely with any project browsing IDE, where you don't\nnecessarily even know what project type a given directory is: no need any\nmore to trawl through README files to figure out how to execute a project.\n\n##### Programmatic introspection\n\nUnlike most, or maybe all, of the tools references by this library, we will\nprovide not just a CLI, but a python API. You can find all the information\nabout a project, make logical decisions and call the third-party tools\nautomatically.\n\nAlso, where a project is principally executed using a particular tool, it\nmight still wish to describe contents/artifacts that are not dealt with by\nthat tool. For instance, you might create environments using ``uv``, but\nalso want to declare data dependencies using ``intake``. The code within\nthe project can then find these assets by introspection.\n\n##### Index \u0026 search\n\nIf you have a lot of projects or interact with a project storage service,\nit can be a task just to figure out which is the right one to solve the task\nof the day. If we can index them (even remotely, without downloading),\nyou can rapidly query for particular project contents or outputs.\n\nNaturally, this becomes more powerful as more project types and artifacts\nbecome indexable, and more projects are stored/shared with you.\n\n### Layout of a Project\n\n(some technical details follow, which will migrate to documentation pages when they are\nprepared)\n\n#### Specs\n\nIn this library, a ``Project`` object contains various specifications that\nhave been parsed from the path given or its children. All of these specs are\nsubclasses of ``projspec.proj.base.ProjSpec`` and answer \"what kind of project is\nthis path.\" Of course, a given directory tree can be many different types of projects.\nFor instance, the existence of ``pixi`` metadata is totally independent of whether\nthe directory is a ``git`` repo or not or whether it contains dataset specifications.\nThe tools using these metadata do not directly interact with\none-another, but work quite happily alongside.\n\nEach given spec will have various descriptive metadata, and be associated with come\ncontents and artifacts, see below. ``projspec`` will attempt to match a directory\nwill _every known_ project type at instantiation.\n\nIt is a common pattern for a project to contain potentially several subprojects\nin nodes of the directory tree (e.g., \"monorepos\").\nBy default, ``projspec`` will only walk the tree\nif the top-level directory found no project spec hits, unless you pass ``walk=True``.\n\n#### Content\n\nThese are things that you can know about a project from its metadata files or file listings\nalone. They are inherent, integral parts of what the project is \"at rest.\"\n\nContents are essentially descriptive, and serve to define the project, so that you\ncan understand what it is and potentially find the right project among many when\nquerying. Contents do not support any actions, but may (and often do) associate with\nparticular artifacts.\n\nAll contents\ncan be inferred by reading (small) files directly from remote, without downloading the\nwhole project or running any external tool.\n\n#### Artifacts\n\nAn artifact in the context of this library is an action or output of a project. To actually\nexecute the action, the project must exist locally, and the appropriate tool be available in\nthe runtime.\n\nFor example, if a project is matched to be of type ``uv``, we can infer what environment(s)\nit might contain, but to build those environments the project must be copied to a local\nlocation, and ``uv``, the executable, be available to run.\n\nArtifacts will, in general, know whether they have been already run,\nand point to an output if it exists.\nIn some cases, we may be able to tell if the artifact has been produced already even\nin the remote version of the project. An instance of this would be a lockfile, which\nis the outcome of running an environment resolution on the project, but common is still\nstored alongside the code in the repo; as opposed to the environment's runtime, which\nwill not be stored and only exist locally.\n\nIt is possible for a single entity to be both a \"contents\" and \"artifact\" item. The\nexample of a lockfile, again, fits this description, since it may be in the repo and\nrepresent a constrained environment, but also it is the product of running an action\nagainst a looser environment specification in the project.\n\n## Support\n\nWork on this repository is supported in part by:\n\n\"Anaconda, Inc. - Advancing AI through open source.\"\n\n\u003ca href=\"https://anaconda.com/\"\u003e\u003cimg src=\"https://camo.githubusercontent.com/b8555ef2222598ed37ce38ac86955febbd25de7619931bb7dd3c58432181d3b6/68747470733a2f2f626565776172652e6f72672f636f6d6d756e6974792f6d656d626572732f616e61636f6e64612f616e61636f6e64612d6c617267652e706e67\" alt=\"anaconda logo\" width=\"40%\"/\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffsspec%2Fprojspec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffsspec%2Fprojspec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffsspec%2Fprojspec/lists"}