{"id":27185029,"url":"https://github.com/threefoldtech/0-flist","last_synced_at":"2026-02-15T01:33:46.776Z","repository":{"id":84223067,"uuid":"143600158","full_name":"threefoldtech/0-flist","owner":"threefoldtech","description":null,"archived":false,"fork":false,"pushed_at":"2025-02-24T11:25:42.000Z","size":638,"stargazers_count":1,"open_issues_count":7,"forks_count":1,"subscribers_count":18,"default_branch":"development-v2","last_synced_at":"2025-04-09T17:10:33.378Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"maxux/flister","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/threefoldtech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-05T09:35:48.000Z","updated_at":"2025-02-24T11:25:46.000Z","dependencies_parsed_at":"2023-09-26T17:44:30.945Z","dependency_job_id":"83d9ad68-a7ac-4c7a-965b-3c33f056bead","html_url":"https://github.com/threefoldtech/0-flist","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/threefoldtech/0-flist","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2F0-flist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2F0-flist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2F0-flist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2F0-flist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/threefoldtech","download_url":"https://codeload.github.com/threefoldtech/0-flist/tar.gz/refs/heads/development-v2","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2F0-flist/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29464220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-15T01:01:38.065Z","status":"ssl_error","status_checked_at":"2026-02-15T01:01:23.809Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-09T17:10:09.022Z","updated_at":"2026-02-15T01:33:46.762Z","avatar_url":"https://github.com/threefoldtech.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 0-flist\nShared library and standalone flist implementation.\n\nFor more information about `flist`, please check [flist documentation](doc/flist.md).\n\n# Components\nThis projects contains multiple components:\n- `libflist`: the official flist management library\n- `zflist`: a command-line program to manipulate flists (uses the libflist)\n- `pyflist`: python flist binding native extension\n\n## libflist\nLibrary don't have special compilation mode. Default build produce a shared library and a static archive\nto link `libflist` code. The code produced contains debug and production version (customized by a flag\non runtime to enable or not debug output).\n\nDevelopment note:\n- `flist_xxx` functions are internal functions\n- `libflist_xxx` functions are public functions\n\nMore information about how to use the library will be available soon.\n\n# zflist\nCommand line utility which use `libflist` to list, create and query a flist file.\n\nYou can easily ship a (mostly) static version of `zflist` by compiling it using `make embedded`. This build\nwill static link `libflist` and it's dependencies, but not produce a fully static binary. It will still\nbe linked to `glibc` in order to use any resolution and users functions correctly.\n\nThe `embedded` target produce a debug version, there is also a `production` target which disable any\ndebug output by default.\n\n# pyflist\nPython binding of the library. Work in progress.\n\n# Dependencies\nIn order to compile correctly `libflist`, you'll need theses libraries:\n- `sqlite3` (database, libflist)\n- `hiredis` (redis, libflist)\n- `libtar` (archive, libflist)\n- `libsnappy` (compression, libflist)\n- `c-capnp` (serialization, libflist)\n- `libb2` (hashing [blake2], libflist)\n- `zlib` (compression, libflist)\n\nTo compile `zflist`, you'll also need:\n- `jansson` (serialization, zflist)\n\nTo compile the python binding, you'll also need:\n- `python3` (obviously, extension, pyflist)\n\n## Ubuntu\n- Packages dependencies\n```\nbuild-essential libsnappy-dev libz-dev libtar-dev libb2-dev libjansson-dev libhiredis-dev libsqlite3-dev \n```\nYou will need to compile `c-capnp` yourself, see autobuild directory.\n\n# Notes\n## Merge priority\nWhen merging multiple flist together, the order of the merge is important.\nAll files are proceed one by one from first to last flist. When file collision occures\n(same filename is present in multiple flist), the first file found is the one used on the final archive.\n\nExample:\n```\nzflist init\nzflist merge first.flist\nzflist merge second.flist\nzflist commit merged.flist\n```\n\nIf `/bin/ls` is on both flist, the file from `first.flist` will be found in `merged.flist`.\n\n# Prefetch\n\nThe best part of flist system and concept is the fact that files are downloaded on-the-fly.\nThis is really useful for lot of use cases but sometime it's important performance wise to\nget files locally to not spend time downloading on runtime.\n\nWith the prefetch action, you can force the system to read all files in a directory and thus,\ndownloading all files not availables on the `0-fs` cache.\n\nThis action is only interresting when using on top of [`0-fs`](https://github.com/threefoldtech/0-fs).\n\n# Usage\n\n```\nUsage: ./zflist [temporary-point] open \u003cfilename\u003e\n       ./zflist [temporary-point] \u003caction\u003e \u003carguments\u003e\n       ./zflist [temporary-point] commit [optional-new-flist]\n\n  The temporary-point should be a directory where temporary files\n  will be used for keeping tracks of your changes.\n  You can also use ZFLIST_MNT environment variable to set your\n  temporary-point directory and not specify it on the command line.\n\n  By default, this tool will print error and information in text format,\n  you can get a json output by setting ZFLIST_JSON=1 environment variable.\n\n  First, you need to -open- an flist, then you can do some -edit-\n  and finally you can -commit- (close) your changes to a new flist.\n\n  If you want to upload chunks when inserting files, please set\n  environment variable ZFLIST_BACKEND to a json backend formatted string,\n  check backend documentation for more information\n\n  To use the hub subsystem, you need to specify at least a jwt token\n  via the environment variable ZFLIST_HUB_TOKEN, this jwt needs to be\n  valid for the hub. In addition, you can specify ZFLIST_HUB_USER if\n  your token contains multiple member-of users/organizations.\n\nAvailable actions:\n  open            open an flist to enable editing\n  init            initialize an empty flist to enable editing\n  ls              list the content of a directory\n  find            list full contents of files and directories\n  stat            dump inode full metadata\n  cat             print file contents (backend metadata required)\n  put             insert local file into the flist\n  putdir          insert local directory into the flist (recursively)\n  chmod           change mode of a file (like chmod command)\n  rm              remove a file (not a directory)\n  rmdir           remove a directory (recursively)\n  mkdir           create an empty directory (non-recursive)\n  metadata        get or set metadata\n  merge           merge another flist into the current one\n  hub             0-hub command line tools\n  commit          commit changes to a new flist\n  close           close mountpoint and discard files\n```\n\n# Example\n\nHow to create an flist, upload playloads and upload flist into playground hub. By default, `zflist`\nwill use production hub (`hub.grid.tf`) but you can override that by setting `ZFLIST_HUB` environment\nvariable, like below.\n\n```\nexport ZFLIST_MNT=/tmp/zflistmnt\nexport ZFLIST_HUB=\"https://playground.hub.grid.tf\"\nexport ZFLIST_BACKEND='{\"host\": \"playground.hub.grid.tf\", \"port\": 9910}'\nexport ZFLIST_HUB_TOKEN=kiUTd9jRjgt7QB6lRh2bcpNiC2UqvTLI\n\nzflist init\nzflist putdir /tmp/20191021_150741 /\nzflist commit /tmp/committed-flist.flist\nzflist hub upload /tmp/committed-flist.flist\n\n\n[+] response [200]: {\"payload\": {\"timing\": {}, \"name\": \"committed-flist.flist\", \"files\": {\"regular\": 890, \"directory\": 2, \"symlink\": 0, \"failure\": 0, \"special\": 0}}, \"status\": \"success\"}\n```\n\n# Repository Owner\n- [Maxime Daniel](https://github.com/maxux), Telegram: [@maxux](http://t.me/maxux)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2F0-flist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreefoldtech%2F0-flist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2F0-flist/lists"}