{"id":14156426,"url":"https://github.com/polydawn/timeless","last_synced_at":"2026-02-22T19:33:56.921Z","repository":{"id":73162091,"uuid":"96472792","full_name":"polydawn/timeless","owner":"polydawn","description":"The Timeless Stack -- docs and meta repo :book:","archived":false,"fork":false,"pushed_at":"2020-01-03T12:51:12.000Z","size":274,"stargazers_count":36,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-08-17T08:07:10.850Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/polydawn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-07-06T21:18:05.000Z","updated_at":"2023-11-29T08:31:40.000Z","dependencies_parsed_at":"2023-02-26T05:31:12.967Z","dependency_job_id":null,"html_url":"https://github.com/polydawn/timeless","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polydawn%2Ftimeless","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polydawn%2Ftimeless/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polydawn%2Ftimeless/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polydawn%2Ftimeless/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/polydawn","download_url":"https://codeload.github.com/polydawn/timeless/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228829171,"owners_count":17978160,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-17T08:05:27.988Z","updated_at":"2026-02-22T19:33:56.885Z","avatar_url":"https://github.com/polydawn.png","language":"Shell","funding_links":[],"categories":["others"],"sub_categories":[],"readme":"The Timeless Stack?\n===================\n\nThe Timeless Stack is a software ecosystem that's about *making computers reliable*.\n\n\nNow in html book form!\n----------------------\n\nThe latest docs are in the process of being ported to a html book format.\nYou can read that at https://repeatr.io/ !\n\n\nReleases\n--------\n\nBinary releases are available on the [github releases page](https://github.com/polydawn/timeless/releases)!\n\n\nMotivations\n-----------\n\n### What is \"reliable\"?\n\nReliable means the system changes when you want it to, and *stays the same* when not explicitly changed.\n\nChange needs to be easy.  Updates need to be simple.\nSimultaneously, control is paramount: *unexpected* change and *unwanted* updates need to be preventable.\n\nReliability also implies *decentralization*.\nRelying on a remote system is not reliable.\nManaging change control based on a network service is not reliable.\nChange control needs to be something that different systems and different people can own, individually, and needs to operate free-standing.\n\n### What is our edge?\n\nTo be reliable, the Timeless Stack builds everything around snapshots, immutable data,\nand reproducible transformations.\n\nTo do this, the Timeless Stack encompasses...\n\n- **handling snapshots of files**\n- **building software** in containers\n- **publishing releases** that associate names and metadata to filesets snapshots\n- and **distributing packages** of filesets that assemble into useful systems!\n\nIt's a lot of responsibilities.  We've broken them up into several tools.  Use the parts you need.  Ditch the ones you don't.  Each tool is API-driven and can be used in isolation, or glues together easily with the others.\n\nAt the foundation is Timeless Repeatr: a tool that defines a total sandbox for data processing and running programs, which works together with the concept of a Timeless Fileset in order to be reproducible anywhere, anywhen.\nRepeatr consumes a document we call a Formula: in keeping with the theme, it's all timeless.  No dates; just Filesets.\n\nTimeless Hitch records IDs for Timeless Filesets __together with__ the Timeless Repeatr Formulas that produced them.\nTimeless Hitch is both a release management system, an update tracking database, and *build instructions* for everything it tracks: with Timeless Hitch, we finally have all the advantages of a source distribution and a binary distribution *in the same system*.\n\nJump down to the 'Tools' heading for an overview of the other core tools; but we recommend scanning the 'Concepts' heading first, for vocab.\n\n\nConcepts\n--------\n\n|  Subject                           |  Concept          |  Meaning   |\n|-----------------------------------:|:-----------------:|:-----------|\n| Data/Files                         | Fileset           |  A set of files and directories, including standard posix metadata.  Nothing special :)\n| Data/Files                         | Ware              |  The \"packed\" form of a Fileset.  Tarballs, git commits; many formats are defined.  Wares are immutable and indexed by 'WareID'.\n| Data/Files                         | WareID            |  The __hash__ identifying a Ware.  Holding a WareID gives you an immutable reference to a Ware which you can unpack into a Fileset.\n| Builds/Compute\u003cbr\u003e/Transformation  | Formula           |  A complete definition of a computation.  Pins all input filesystems by WareID.  Runs in a sandbox.  Running produces a 'runrecord'.\n| Builds/Compute\u003cbr\u003e/Transformation  | Formula.Inputs    |  A group of Filesets and where to mount them in a directory tree when setting up the formula and preparing to exec.\n| Builds/Compute\u003cbr\u003e/Transformation  | Formula.Action    |  A script to run inside the container when evaluating a formula.\n| Builds/Compute\u003cbr\u003e/Transformation  | Formula.Output    |  A list of paths to save when the formula's action is complete.  Files under an output path will be exported as a Ware, and when the run is complete, the WareIDs are reported.\n| Builds/Compute\u003cbr\u003e/Transformation  | SetupHash         |  The hash of a 'formula'.  This is the unique name of a computation!  Use it as a primary key; memoize builds with it!\n| Builds/Compute\u003cbr\u003e/Transformation  | RunRecord         |  The result of running a 'formula'.  Includes a 'results' map, and also metadata like timestamp and UID (thus, runrecords are always unique).\n| Builds/Compute\u003cbr\u003e/Transformation  | Results           |  The group of WareIDs produced when a 'formula' is evaluated -- one WareID for every 'output' path!\n| Pipelines of Compute               | Module            |  A graph (a DAG) describing a series of proto-formulas: each \"step\" in the graph will be resolved to one formula, evaluated, and then its outputs can be used as inputs in other steps.\n\n\nTools\n-----\n\n|  Tool       |  Repo                                           |  Role      |\n|:-----------:|:-----------------------------------------------:|:-----------|\n| `repeatr`   | [github](https://github.com/polydawn/repeatr)   |  Repeatr evaluates formulas.  `formula` ---`repeatr run`---\u003e `runrecord`\n| `rio`       | [github](https://github.com/polydawn/rio)       |  R-I/O stands for Repeatable Input/Output: it's tooling for packing Filesets into Wares, mirroring Wares between storage systems, and unpacking Filesets fetched by WareID.  Glue for getting files to and from other systems, in other words.\n| `reach`     | [github](https://github.com/polydawn/reach)     |  Reach builds larger pipelines: use it to stich together several formulas by passing wares between them.  Entire pipelines quickly rebuild incrementally, automatically memoizing any steps that haven't changed.\n| `tlpkg`     | planned                                         |  The **T**ime**l**ess **p**ac**k**a**g**e manager: helps manage $PATH and environments to\n\nAll parts of this stack are loosely coupled:\n\n- `repeatr` works entirely with formulas.  You can template formulas with anything you like -- the API is simply json/yaml.\n- `reach` can be evalute modules to make declarative build pipelines work, and automatically invokes `repeatr` as needed to get it done;\n- ... or, `reach catalog` can be driven entirely by shell script to make releases or look up releases and metadata, without using the rest of the pipelining logic.\n- `rio` can pack and unpack filesystems on command, and scan remote internet resources to determine their hash -- or will do these same things automatically in service to `repeatr`.\n- `tlpkg` is built to handle packages that play well in the Timeless ecosystem -- but there's nothing to say you can't use `apt` or `yum` or `dnf` inside `repeatr` containers and `reach` pipelines.\n\nSee [the ecosystem map](ecosystem.png) for a big-picture layout of how the projects are connected.\n\n### Additional tools\n\n|  Tool       |  Repo                                           |  Role      |\n|:-----------:|:-----------------------------------------------:|:-----------|\n| `r2k8s`     | [github](https://github.com/polydawn/r2k8s)     |  `r2k8s` bridges the repeatr and kubernetes ecosystems: run formulas remotely in a kubernetes cluster, or template kubernetes podspecs to use formulas to launch commands.\n| `radix`     | [github](https://github.com/polydawn/radix)     |  A collection of repeatr formulas and reach modules.  You can use this as reference materials.  (All of our base image snapshots come from here!)\n\n\n\nHow do I get started doing something Fun?\n-----------------------------------------\n\nCheck out the [omnibus](./omnibus) dir for an example of using Repeatr and Rio to build... Rio, Repeatr, and the rest of the Timeless Stack!\n\nYou'll need to grab the [binary releases](https://github.com/polydawn/timeless/releases) in order to bootstrap\n(or, build the individual tools from source).\n\nThis [omnibus](./omnibus) example uses basic bash string templating to compose pipelines just to keep things simple\nand make it utterly clear that you can compose objects for this API any way you want.\nYou may want to use a more powerful language when building your own higher level pipelines.\nWe're explicitly committed to supporting that; the Repeatr JSON API *is meant* to be used freely.\n\n\n\nWhy hashes?  Why can't I just download \"somefoo-vbar.baz.tar.gz\"?\n-----------------------------------------------------------------\n\nRepeatr and the Timeless Stack uses hashes for WareIDs for the same reason stores use SKUs and part numbers: consistency and reliability.\n\nAs an individual, you can go to a hardware store and ask for a 5mm screw.  But you haven't really picked yet by saying that: the store staff will show you to an entire rack of screws.\n\nIf you're the manager of a factory, you order your screws by their part number, because you don't want to re-order screws halfway through a project and end up with a subtly different threading, or screws that are too *long* in some other dimension, and so forth.\nSimilarly, as a hardware store manager, your organize your stocks and reorder supplies by their SKUs, because *consistency is important*.\n\nStore bins may have labels of all sorts of descriptions in various precision to help you select which items you need, but the store itself is tracking inventory by SKUs, because a single, precise identifier is vastly simpler to work with.\n\nRepeatr is the same: we use hashes to identify files and data because it's *simple* and it's *consistent* and it's *immune to silent changes* over time.\nEnd-to-end use of hashes is part of what makes the Timeless Stack... *timeless*.\n\n\n\nKeeping in touch\n----------------\n\nIf you'd like to follow the Timeless project and get occasional email about releases and development milestones:\n\n- We have an (extremely) low-volume mailing list for announcements: [timeless-stack-announce@googlegroups.com](https://groups.google.com/forum/#!forum/timeless-stack-announce)\n\nIf you'd like to discuss development, ask questions, or just lurk the community:\n\n- Feel free to grab the email address from git commit messages and get in touch directly, or\n- We have a community mailing list that anyone may join: [timeless-stack@googlegroups.com](https://groups.google.com/forum/#!forum/timeless-stack)\n- If the mailing list is quiet, it's because most of the core team is on [Matrix](https://matrix.org); ask for more info.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolydawn%2Ftimeless","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpolydawn%2Ftimeless","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolydawn%2Ftimeless/lists"}