{"id":20499931,"url":"https://github.com/atlas-engineer/nfiles","last_synced_at":"2026-03-10T04:32:34.254Z","repository":{"id":42203044,"uuid":"452987681","full_name":"atlas-engineer/nfiles","owner":"atlas-engineer","description":"User configuration and data file management","archived":false,"fork":false,"pushed_at":"2025-01-03T23:33:30.000Z","size":208,"stargazers_count":22,"open_issues_count":6,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2026-03-02T22:57:17.484Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/atlas-engineer.png","metadata":{"files":{"readme":"readme.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-28T08:10:43.000Z","updated_at":"2025-10-04T16:57:28.000Z","dependencies_parsed_at":"2023-11-23T14:29:36.061Z","dependency_job_id":"5bf4be74-f216-4c38-ab37-37e781ded234","html_url":"https://github.com/atlas-engineer/nfiles","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/atlas-engineer/nfiles","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atlas-engineer%2Fnfiles","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atlas-engineer%2Fnfiles/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atlas-engineer%2Fnfiles/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atlas-engineer%2Fnfiles/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/atlas-engineer","download_url":"https://codeload.github.com/atlas-engineer/nfiles/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atlas-engineer%2Fnfiles/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30324424,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T01:36:58.598Z","status":"online","status_checked_at":"2026-03-10T02:00:06.579Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T18:19:07.695Z","updated_at":"2026-03-10T04:32:34.229Z","avatar_url":"https://github.com/atlas-engineer.png","language":"Common Lisp","readme":"#+TITLE: NFiles\n\nNFiles is a Common Lisp library that deals with customizable path resolution,\nfile persistence and loading.\n\nIts main use case is help manage user-centric files like configuration files.\n\nIn some aspects, it can be seen as \"Common Lisp 'logical pathnames' over CLOS\".\n\n** Goals\n\n- Performance ::\n  - Data is not persisted to disk if it does not need to.\n  - Files are read only once (unless modified externally).\n- Extensibility :: Persist any data structure the way you want it.\n- Reliability :: no corruption and no data loss should occur.\n\n** Features\n\n- Dynamic and customizable path expansion.\n- Extensible serialization and deserialization.\n- Cached reads and writes ::\n  When a =file= object expands to the same path as another one, a =read= or\n  =write= on it won't do anything in case there was no change since last write.\n- (*Experimental!*) On-the-fly PGP encryption.\n- Profile support.\n- On read errors, existing files are backed up.\n- On write errors, no file is written to disk, the existing file is preserved.\n- A =remote-file= can point to a URL, which is automatically downloaded if the\n  local file is not found.\n\n** Motivation\n\nThis package was developed after dealing with a problem when delivering Common\nLisp images: when an image is generated, path expansion may already be resolved\nand thus hard-coded within the image, which makes it unfit for delivery.\nConsider this:\n\n#+begin_src lisp\n\u003e (defvar *foo-path* (uiop:xdg-config-home))\n*FOO-PATH*\n\u003e *foo-path*\n#P\"/home/johndoe/.config/\"\n#+end_src\n\nNow if I ship this image to my friend Kaboom, =*foo-path*= will expand to\n\n#+begin_src lisp\n#P\"/home/johndoe/.config/\"\n#+end_src\n\non their machine instead of the expected\n\n#+begin_src lisp\n#P\"/home/kaboom/.config/\"\n#+end_src\n\n** Examples\n\nA basic session:\n\n#+begin_src lisp\n(defvar *config-file* (make-instance 'nfiles:config-file :base-path #p\"my-app/init.lisp\"))\n\n(nfiles:expand *config-file*)\n; =\u003e #P\"/home/johndoe/.config/my-app/init.lisp\"\n\n(setf (nfiles:content *config-file*) \"Hello file!\") ; The file is written to disk.\n\n(nfiles:content *config-file*)\n; =\u003e \"Hello file!\"\n#+end_src\n\nThe following convenience macro ensures the file is updated when done with the\nbody:\n\n#+begin_src lisp\n  (nfiles:with-file-content (content *config-file*)\n    (format t \"Length: ~a~%\" (length content))\n    (setf content (serapeum:string-replace \"file\" content \"config\")))\n#+end_src\n\nThe =with-paths= helper allows for let-style bindings of the expanded paths:\n\n#+begin_src lisp\n  (let ((file1 (make-instance 'nfiles:file))\n        (file2 (make-instance 'nfiles:file :base-path #p\"alt\")))\n    (nfiles:with-paths ((path1 file1)\n                        (path2 file2))\n      (list path1 path2)))\n#+end_src\n\nA =remote-file= works the same but needs some specialization:\n\n#+begin_src lisp\n  (defmethod nfiles:fetch ((profile nfiles:profile) (file remote-counter-file) \u0026key)\n    (dex:get (nfiles:url file)))\n\n  ;; Optional:\n  (defmethod nfiles:check ((profile nfiles:profile) (file remote-counter-file) content \u0026key)\n    (let ((path (nfiles:expand file)))\n      (ironclad:byte-array-to-hex-string\n       (ironclad:digest-file :sha3 path))))\n\n  (let ((file (make-instance 'nfiles:remote-file\n                             ;; The URL to download from if the file is not found on disk.\n                             :url (quri:uri \"https://example.org\")\n                             ;; Without base-path, the file won't be saved to disk.\n                             :base-path #p\"/tmp/index.html\"\n                             :checksum \"794df316afac91572b899b52b54f53f04ef71f275a01c44b776013573445868c95317fc4a173a973e90addec7792ff8b637bdd80b1a6c60b03814a6544652a90\")))\n    ;; On access, file is automatically downloaded if needed and the checksum is verified:\n    (nfiles:content file)\n    ;; ...\n    )\n#+end_src\n\nSee the [[file:package.lisp][package]] documentation for a usage guide and more examples.\n\n** Configuration\n\nNFiles was designed with configurability in mind.  All configuration happens through\nsubclassing of the =file= and =profile= classes together with method\nspecialization.\n\nAll configuration methods are specialized against =profile= and =file= so that\nthe user can easily *compose* the behaviour:\n- Profile-customization impacts all files using that profile;\n- File-customization impacts the files of that specific type (or subtype)\n  regardless of their profile.\n\nOf course you can specialize against both!\n\nThe specialization methods are divided into the following:\n\n- =resolve= :: This is where path resolution is done.  On call site, prefer the\n  =expand= convenience wrapper.\n\n- =deserialize= and =serialize= :: This is how the content is transformed\n  to the file on disk.  These functions are meant to be called by the\n  =read-file= and =write-file= methods.\n\n- =read-file= and =write-file= :: This is how the file is read and written to\n  disk.  These functions are responsible for calling the =deserialize= and\n  =serialize= methods.\n\n- =fetch= :: This generic function is only called for =remote-file= objects.  You\n  _must_ define its methods.  It does not have any method by default so as to\n  not burden NFiles with undesirable dependencies.\n\n- =check :=: Like =fetch=, this generic function is only called for =remote-file=\n  objects to test the integrity of the downloaded file.  You _must_ define its\n  methods.  It does not have any method by default so as to not burden NFiles\n  with undesirable dependencies.\n\n** Conditions and restarts\n\nSome NFiles-specific conditions are raised in case of exceptional situations to\nprovide for interactive and customizable behaviour:\n\n- =external-modification= :: The file was modified externally.  See the\n  =on-external-modification= slot to automate what to do in this case.\n\n- Read error restarts can also customized, see the =on-read-error= slot to\n  automate what to do in this case.\n\n- =process-error= :: This may be raised for instance when =gpg= fails to encrypt.\n  The =use-recipient= restart is provided to retry with the given recipient.\n\n** Shadowing\n\nNFiles 1 shadows =cl:delete=, thus you should not =:use= the package (as with\nany other library anyways).\n\n** Platform support\n\nIt's pure Common Lisp and all compilers plus all operating systems should be\nsupported.\n\nSome notes:\n\n- All compilers but SBCL depend on [[https://github.com/sionescu/iolib][IOlib]] to preserve file attributes.\n- Android devices also depend on [[https://github.com/sionescu/iolib][IOlib]] to preserve file attributes,\n  regardless of the compiler.\n- File attributes might not be preserved on Windows.\n\n** Roadmap\n\n- Improve PGP support.\n- Support OS-level locks (à-la Emacs / LibreOffice).\n- Improve portability.\n  - In particular, preservation of file attributes may not work on\n    Windows.\n- Compressing =write-file= and =read-file= (for instance with zstd / lz).\n  - But is it such a good idea?  Users should prefer compression at\n    the level of the file system.\n\n** Change log\n\n*** 1.1.4\n\n- Remove =NASDF= as a dependency.\n\n*** 1.1.3\n\n- Ensure a =cl:pathname= is returned from =resolve=.\n- Complete some missing documentation.\n\n*** 1.1.2\n\n- Add restarter functions for =ask=, =reload=, etc.\n  Mind that =cl:delete= is now shadowed (this was necessary to preserve backward\n  compatibility).\n  Do not =:use= the package!\n\n- Switch from =hu.dwim.defclass-star= to [[https://github.com/atlas-engineer/nclasses/][nclasses]].\n\n*** 1.1.1\n\n- Allow path expansion for =virtual-file= (as in 1.0.0).\n\n  This restores the usefulness of virtual-files, namely to handle the\n  path-expansion business while deferring the read/write business to a\n  third-party.\n\n  =virtual-profile= still nullifies path expansions (as in 1.1.0).\n\n*** 1.1.0\n\n- Add support for Android.\n- Nullify path expansion for =virtual-file= and =virtual-profile=.\n- Ensure that the =deserialize= method of =virtual-file= and =virtual-profile=\n  return nil.\n- Fix =basename= corner case.\n- Add report messages to all restarts.\n\n** History\n\nNFiles was originally developed for user file management in [[https://nyxt.atlas.engineer][Nyxt]], so the \"N\"\nmay stand for it, or \"New\", or whatever poetic meaning you may find behind it!\n","funding_links":[],"categories":["Online editors ##"],"sub_categories":["Third-party APIs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatlas-engineer%2Fnfiles","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fatlas-engineer%2Fnfiles","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatlas-engineer%2Fnfiles/lists"}