{"id":21496524,"url":"https://github.com/mlibrary/bag-courier","last_synced_at":"2025-03-17T12:16:07.267Z","repository":{"id":207068627,"uuid":"718292114","full_name":"mlibrary/bag-courier","owner":"mlibrary","description":"Tools for packaging and sending BagIt bags","archived":false,"fork":false,"pushed_at":"2024-08-28T16:29:23.000Z","size":381,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-23T21:53:18.250Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlibrary.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-13T19:32:23.000Z","updated_at":"2024-08-06T20:46:02.000Z","dependencies_parsed_at":"2023-11-16T19:48:48.809Z","dependency_job_id":"82638990-1227-4477-b3ff-d77d2605cff5","html_url":"https://github.com/mlibrary/bag-courier","commit_stats":null,"previous_names":["ssciolla/bag-courier","mlibrary/bag-courier"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlibrary%2Fbag-courier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlibrary%2Fbag-courier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlibrary%2Fbag-courier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlibrary%2Fbag-courier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlibrary","download_url":"https://codeload.github.com/mlibrary/bag-courier/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244031154,"owners_count":20386534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T16:16:54.196Z","updated_at":"2025-03-17T12:16:07.236Z","avatar_url":"https://github.com/mlibrary.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bag-courier\n\nThis project includes a number of classes for packaging and sending BagIt bags to \nremote locations like APTrust. It also includes functionality for interacting with\nthe repository system Archivematica and one full integration process,\n`run_dark_blue.rb`.\n\nThe following sections cover how to configure, install, use, and test the project.\n\n## Prerequisite(s)\n\nDepending on the environment and configuration used with this project,\nyou may need to install one or more of the following:\n- [Ruby 3.2](https://www.ruby-lang.org/en/downloads/)\n- An SFTP client like OpenSSH (usually included with operating systems)\n- A `tar` executable (usually included, at least with Mac OS and Linux)\n- [Docker Desktop](https://www.docker.com/products/docker-desktop/)\n\nThe project may work with earlier Ruby versions, but it has not been tested.\n\nInteractions with others services such as Archivematica or an SFTP server\nmay be IP address restricted, so ensure you have access to the proper network or VPN.\n\n## Configuration\n\nIn all cases, you will need to set up a configuration file, which you can do by following these steps:\n\n1. Create a `.env` file based on the provided template, [`example.env`](/example.env).\n    ```sh\n    cp example.env .env\n    ```\n2. Open `.env` using your preferred text editor, and add values,\nusing the comments to guide you.\n\nFor local development, you may need to set up an SSH key and add it to your `ssh-agent`\nso the program can access remote file systems via an SFTP server without entering a password\nor passphrase.\n\n## Installation \u0026 Usage\n\nFor local development, you can set up the neccessary dependencies and\nexecute processes from this project either with Ruby directly on your machine\nor by using the provided Docker artifacts. The following sections cover each approach.\n\nNote that the project can use a MariaDB database to store records about its runs.\nThe recommended approach is to use the `database` service\n(defined in [`docker-compose.yml`](/docker-compose.yml) with the Docker approach described below.\nHowever, it is possible -- though not explicitly supported -- to use a MariaDB or MySQL\ndatabase from a server running on your local machine or elsewhere.\n\nDeployed versions of this application also use [Prometheus](https://prometheus.io/),\nand an associated Push Gateway, to publish and collect metrics about runs.\nDuring local development, you can use the `prometheus` and `pushgateway` Docker services\nto experiment with this functionality.\n\nThe following sections will assume you are not using a database -- or Prometheus --\nwith Ruby outside a container.\n\n### Ruby\n\n#### Installation\n\n```sh\ngem install bundler  # if you don't have it\nbundle install\n```\n\n#### Usage\n\n*Note*: The application expects to find necessary configuration in environment variables.\nTo assist with local development, it includes the [`dotenv`](https://github.com/bkeepers/dotenv) gem,\nwhich will populate the Ruby global `ENV` with key-value pairs from the prepared `.env` file.\nWhen using only Ruby to execute a file, if configuration is needed,\nprecede your `ruby script.rb` with `dotenv`, as shown below.\n\nThe primary job or process is `run_dark_blue.rb`.\n```sh\ndotenv ruby run_dark_blue.rb\n```\n\nIf you're working on a new job or just want to try out the classes,\nyou can copy `run_example.rb` and modify it as necessary.\n```sh\ncp run_example.rb run_test.rb\n# Tweak as necessary\ndotenv ruby run_test.rb\n```\n\n### Docker\n\n#### Installation\n\n*Note*: The provided `docker-compose.yml` file will detect your `.env` and create the appropriate\nenvironment variables.\n\nBuild the image for the `dark-blue` service.\n```sh\ndocker compose build dark-blue\n```\n\n#### Usage\n\nStart up the `database`, `prometheus`, and/or `pushgateway` services.\n```sh\ndocker compose up database prometheus pushgateway\n```\n\nRun the migrations, if you configured the application to use a database:\n```sh\ndocker compose run dark-blue rake db:migrate\n```\n\nRun the `dark-blue` service to start `run_dark_blue.rb`.\n```sh\ndocker compose up dark-blue\n```\n\nThe latest deposits to APTrust can be verified using `verify_aptrust.rb`.\nOverride the entry command for the `dark-blue` service with `run`.\n```sh\ndocker compose run dark-blue ruby verify_aptrust.rb\n```\n\nYou can re-deliver previously delivered packages by passing a `--packages` option to `run_dark_blue.rb`\nwith a comma-separated list of package identifiers.\n```sh\ndocker compose run dark-blue ruby run_dark_blue.rb --packages=some_id,some_other_id\n```\n\nTo execute a typical workflow (the first three steps above), you can simply run the following command:\n```sh\ndocker compose up\n```\n\n## Running tests\n\n[`minitest`](https://github.com/minitest/minitest) unit tests for classes are\nlocated in the `test` directory.\n\nTo run all tests in one or more test files, use `ruby -Ilib {file path} ...`, e.g.\n```sh\nruby -Ilib test/test_bag_adapter.rb test/test_bag_tag.rb\n```\nNote that some test files will throw an error if a database configuration has not been set up.\n\nWith Docker, you can run the above command by adding it after `docker compose run dark-blue`.\nYou can also use a provided `rake` task to run all unit tests\n(which, again, requires configuration of a database).\n```sh\ndocker compose run dark-blue rake test\n```\n\n## Resources\n- [BagIt specification](https://datatracker.ietf.org/doc/html/rfc8493)\n- [APTrust User Guide](https://aptrust.github.io/userguide/)\n- [Archivematica Storage Service REST API docs](https://www.archivematica.org/en/docs/archivematica-1.15/dev-manual/api/api-reference-storage-service/)\n- [aws-sdk-s3](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3.html)\n- [mlibrary/sftp](https://github.com/mlibrary/sftp)\n- [Sequel](http://sequel.jeremyevans.net/)\n- [Prometheus Client](https://github.com/prometheus/client_ruby/blob/main/README.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlibrary%2Fbag-courier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlibrary%2Fbag-courier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlibrary%2Fbag-courier/lists"}