{"id":17113673,"url":"https://github.com/totakke/cavia","last_synced_at":"2025-04-13T02:32:08.988Z","repository":{"id":13609328,"uuid":"16302329","full_name":"totakke/cavia","owner":"totakke","description":"Remote resource management for Clojure projects","archived":false,"fork":false,"pushed_at":"2025-04-12T11:58:12.000Z","size":252,"stargazers_count":8,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-12T12:08:22.131Z","etag":null,"topics":["clojure","download-manager","downloader","testing"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/totakke.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-01-28T04:41:43.000Z","updated_at":"2025-04-12T11:58:16.000Z","dependencies_parsed_at":"2024-02-08T14:30:38.829Z","dependency_job_id":"9c4a0288-0da3-4d69-a358-d8ac1c59c427","html_url":"https://github.com/totakke/cavia","commit_stats":{"total_commits":212,"total_committers":5,"mean_commits":42.4,"dds":"0.30660377358490565","last_synced_commit":"ca2f07894a64f0076bcc7e42aced60325ec3b73b"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/totakke%2Fcavia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/totakke%2Fcavia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/totakke%2Fcavia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/totakke%2Fcavia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/totakke","download_url":"https://codeload.github.com/totakke/cavia/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248657793,"owners_count":21140842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","download-manager","downloader","testing"],"created_at":"2024-10-14T17:11:34.629Z","updated_at":"2025-04-13T02:32:08.981Z","avatar_url":"https://github.com/totakke.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cavia\n\n[![Clojars Project](https://img.shields.io/clojars/v/cavia.svg)](https://clojars.org/cavia)\n[![cljdoc badge](https://cljdoc.org/badge/cavia/cavia)](https://cljdoc.org/d/cavia/cavia)\n[![build](https://github.com/totakke/cavia/actions/workflows/build.yml/badge.svg)](https://github.com/totakke/cavia/actions/workflows/build.yml)\n\nRemote resource management for Clojure projects.\n\n## Rationale\n\nIn some cases, tests of a project require large-size files. Among other things,\ncodes for parsing and I/O should be tested by various kinds of files. But\ngenerally, SCM is not good for controlling such large test files. One of the\nsolutions is using other tools like git-annex or Git LFS. Some Clojurians,\nhowever, may think that they want to solve it in the Clojure ecosystem.\n\nCavia is useful for such developers. Cavia is written in Clojure so that it can\nbe directly used in a project and source codes. Cavia downloads test resources\nfrom remotes, checks their hash before tests, and provides convenience\nfunctions to access the resources.\n\n## NOTICE\n\nThe OSS license of Cavia was changed to the [MIT License](LICENSE) since v0.7.0.\n\n## Installation\n\nClojure CLI/deps.edn (as Git):\n\n```clojure\nio.github.totakke/cavia {:git/tag \"v0.7.2\" :git/sha \"0788cb4\"}\n```\n\nClojure CLI/deps.edn (as Maven):\n\n```clojure\ncavia/cavia {:mvn/version \"0.7.2\"}\n```\n\nLeiningen/Boot:\n\n```clojure\n[cavia \"0.7.2\"]\n```\n\n## Basic usage\n\n### Define resources profile\n\nFirst, load `cavia.core` and prepare resources' information with `defprofile`\nmacro.\n\n```clojure\n(require '[cavia.core :as cavia :refer [defprofile]])\n\n(defprofile prof\n  {:resources [;; Simple HTTP\n               {:id :resource1\n                :url \"http://example.com/resource1\"\n                :sha256 \"0123456789abcdef01234567890abcdef01234567890abcdef01234567890abc\"}\n\n               ;; Basic authorization\n               {:id :resource2\n                :url \"http://example.com/resource2\"\n                :sha1 \"123456789abcdef01234567890abcdef01234567\"\n                :auth {:type :basic, :user \"user\", :password \"password\"}}\n\n               ;; FTP\n               {:id :resource3\n                :url \"ftp://example.com/resource3\"\n                :sha256 \"23456789abcdef01234567890abcdef01234567890abcdef01234567890abcde\"\n                :auth {:user \"user\", :password \"password\"}}\n\n               ;; S3\n               {:id :resource4\n                :url \"https://bucket-name.s3.region.amazonaws.com/resource4\"\n                :sha1 \"3456789abcdef01234567890abcdef0123456789\"\n                :protocol :s3\n                :auth {:access-key-id \"accesskey\", :secret-access-key \"secretkey\"}}\n\n               ;; Compressed resource\n               {:id :resource5\n                :url \"http://example.com/resource5.gz\"\n                :sha1 \"456789abcdef01234567890abcdef0123456789a\"\n                :packed :gzip}]\n   :download-to \".cavia\"})\n```\n\nResources are defined in `:resources` as a vector including some maps. Each\nresource map must have `:id :url :md5/:sha1/:sha256` fields, which are\nmandatory. `:id` should be specified as keyword or string, which is used for\nresource access and downloading file name.\n\nMD5, SHA1, and SHA256 are supported as hash algorithms for verifying files. One\nalgorithm must be specified at least. If more than one algorithm are specified,\na stronger algorithm will be used: MD5 \u003c SHA1 \u003c SHA256.\n\n`:auth` field is optional. It can be used for password authentication.\nCavia is now supporting HTTP/HTTPS/FTP/FTPS/S3 protocols and Basic/Digest/OAuth2\nauthentications. A resource that `:packed` specified will be uncompressed after\ndownloading. Only gzip (`:gzip`) format is supported.\n\nCavia downloads resources to `:download-to` directory. The default location is\n`./.cavia`. Thus maybe you should add `/.cavia` to your SCM ignore list.\n\n### Resource management\n\nCavia provides some functions for managing resources.\n\n```clojure\n(cavia/get! prof)   ; downloads missing resources\n\n(cavia/verify prof) ; checks the downloaded resources' hash\n\n(cavia/clean! prof) ; removes the download directory\n```\n\nTo call Cavia functions without the profile specification, use `with-profile`\nmacro.\n\n```clojure\n(with-profile prof\n  (cavia/clean!)\n  (cavia/get!))\n```\n\n`get!` and other functions output logs and download progress to stdout. To call\nthe above functions quietly, use `with-verbosity` macro. For example, the\nfollowing code suppresses normal messages but displays download progress.\n\n```clojure\n(with-verbosity {:message false\n                 :download true}\n  (cavia/get! prof))\n```\n\n### Resource access\n\nYou do not need to remember the downloaded resources' paths any more. `resource`\nreturns the absolute path to the resource from the specified resource id. It\nreturns `nil` when the id is not defined.\n\n```clojure\n(cavia/resource prof :resource1)\n;;=\u003e \"/home/totakke/cavia-example/.cavia/resource1\"\n\n(cavia/resource prof :undefined)\n;;=\u003e nil\n```\n\n## Integration with test frameworks\n\nCavia is a library for management of test resources. It is good to use Cavia\nwith test frameworks like clojure.test,\n[Midje](https://github.com/marick/Midje), etc.\n\n### with clojure.test\n\n```clojure\n(ns foo.core-test\n  (:require [clojure.test :refer :all]\n            [cavia.core :as cavia :refer [defprofile]]))\n\n(defprofile prof\n  {:resources [{:id :resource1\n                :url \"http://example.com/resource1\"\n                :sha256 \"0123456789abcdef01234567890abcdef01234567890abcdef01234567890abc\"}]})\n\n(defn fixture-cavia [f]\n  (cavia/get! prof)\n  (f))\n\n(use-fixtures :once fixture-cavia)\n\n(deftest your-test\n  (testing \"tests with the cavia's resource\"\n    (is (= (slurp (cavia/resource prof :resource1)) \"resource1's content\")))\n```\n\n### with Midje\n\n```clojure\n(ns foo.t-core\n  (:require [midje.sweet :refer :all]\n            [cavia.core :as cavia :refer [defprofile with-profile]]))\n\n(defprofile prof\n  {:resources [{:id :resource1\n                :url \"http://example.com/resource1\"\n                :sha256 \"0123456789abcdef01234567890abcdef01234567890abcdef01234567890abc\"}]})\n\n(with-profile prof\n\n  (with-state-changes [(before :facts (cavia/get!))]\n    (fact \"tests for a large file\" :slow\n      (slurp (cavia/resource :resource1) =\u003e \"resource1's content\")))\n\n  )\n```\n\n## Use as a simple downloader\n\nCavia provides features of file downloading as independent functions.\n\n```clojure\n(require '[cavia.downloader as dl])\n\n(dl/http-download! \"http://example.com/foobar.txt\" \"path/to/foobar.txt\")\n```\n\nHTTP/HTTPS, FTP/FTPS, and S3 downloading functions support a resume option. If\nyou specify `:resume true`, the functions resume downloading a partially\ndownloaded file.\n\n```clojure\n(dl/s3-download! \"https://foo.s3.region.amazonaws.com/bar/foobar.txt\"\n                 \"path/to/foobar.txt\"\n                 {:access-key-id \"accesskey\", :secret-access-key \"secretkey\"}\n                 :resume true)\n```\n\n## Development\n\n### Integration test\n\nTo run integration tests testing FTP and S3 protocols, launch mock servers with\nDocker Compose first.\n\n```sh\ndocker compose up -d\nlein test :integration\n```\n\n## License\n\nCopyright 2014 [Toshiki Takeuchi](https://totakke.net/)\n\nDistributed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftotakke%2Fcavia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftotakke%2Fcavia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftotakke%2Fcavia/lists"}