{"id":19140789,"url":"https://github.com/codait/cos-utils","last_synced_at":"2025-09-13T07:15:51.527Z","repository":{"id":62564978,"uuid":"217178733","full_name":"CODAIT/cos-utils","owner":"CODAIT","description":"Cloud Object Storage utility","archived":false,"fork":false,"pushed_at":"2019-11-28T00:48:56.000Z","size":57,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-09-08T06:48:47.754Z","etag":null,"topics":["cloud-object-storage"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CODAIT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-24T00:36:06.000Z","updated_at":"2019-11-28T00:48:54.000Z","dependencies_parsed_at":"2022-11-03T16:00:56.204Z","dependency_job_id":null,"html_url":"https://github.com/CODAIT/cos-utils","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/CODAIT/cos-utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcos-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcos-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcos-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcos-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CODAIT","download_url":"https://codeload.github.com/CODAIT/cos-utils/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CODAIT%2Fcos-utils/sbom","scorecard":{"id":26209,"data":{"date":"2025-08-11","repo":{"name":"github.com/CODAIT/cos-utils","commit":"1f92d994a836ea71dece566b3670233b592f68a9"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.8,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/pythonpublish.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/CODAIT/cos-utils/pythonpublish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/pythonpublish.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/CODAIT/cos-utils/pythonpublish.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/pythonpublish.yml:18","Warn: pipCommand not pinned by hash: .github/workflows/pythonpublish.yml:19","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":0,"reason":"Found 0/11 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/pythonpublish.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 29 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-14T17:56:57.882Z","repository_id":62564978,"created_at":"2025-08-14T17:56:57.882Z","updated_at":"2025-08-14T17:56:57.882Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274932221,"owners_count":25376111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-13T02:00:10.085Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-object-storage"],"created_at":"2024-11-09T07:18:46.746Z","updated_at":"2025-09-13T07:15:51.479Z","avatar_url":"https://github.com/CODAIT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud Object Storage CLI\n\n[![Build Status](https://travis-ci.com/CODAIT/cos-utils.svg)](https://travis-ci.com/CODAIT/cos-utils) [![PyPI release](https://img.shields.io/pypi/v/cos-utils.svg)](https://pypi.org/project/cos-utils/) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/cos-utils)\n\nTable of content:\n- [Getting Started](#getting-started)\n- [Listing Cloud Object Storage buckets](#listing-cloud-object-storage-buckets)\n- [Listing the content of a Cloud Object Storage bucket](#listing-the-content-of-a-cloud-object-storage-bucket)\n- [Uploading files to a Cloud Object Storage bucket](#uploading-files-to-a-cloud-object-storage-bucket)\n- [Downloading files from a Cloud Object Storage bucket](#downloading-files-from-a-cloud-object-storage-bucket)\n- [Removing files from a Cloud Object Storage bucket](#removing-files-from-a-cloud-object-storage-bucket)\n\n---\n\n## Getting started\n\nThe utility requires Python 3.6 or above. \n\n### Installation\n\nYou can install the utility from [PyPI](https://pypi.org/project/cos-utils) or from the [source](#install-from-source).\n\n#### Install from pypi.org\n\n```\n$ pip install cos-utils --upgrade\n```\n\n#### Install from source code\n\n```\n$ git clone https://github.com/CODAIT/cos-utils.git\n$ cd cos-utils\n$ pip install .\n```\n\n#### Configuration\n\nSet the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`environment variables based on your [Cloud Object Storage HMAC credentials](https://cloud.ibm.com/docs/services/cloud-object-storage/iam?topic=cloud-object-storage-service-credentials).\n```\n$ export AWS_ACCESS_KEY_ID=...\n$ export AWS_SECRET_ACCESS_KEY=...\n```\n\n# Listing Cloud Object Storage buckets\n\nYou can run the list utility in a terminal window using the generated console script\n\n```\n$ list_buckets --help\n```\n\nor explicitly\n\n```\n$ python -m cos_utils.list_buckets --help\n```\n\nThe help lists required and optional parameters.\n\n```\nusage: list_buckets [-h] pattern\n\nList buckets in Cloud Object Storage instance.\n\npositional arguments:\n  pattern     Bucket name spec (supported wildcards: * and ?)\n\noptional arguments:\n  -h, --help  show this help message and exit\n\nEnvironment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY must be\ndefined to run the utility.\n```\n\n## List all buckets\n\n```\n$ list_buckets *\n```\n\n\u003e On Linux, Unix and MacOS wildcards need to be escaped to prevent shell expansion: `list_files \u003cbucket-name\u003e \\*`.\n\n## Apply a filter\n\nUse the `*` (any character) and `?` (one character) wildcards to define a filter condition.\n\nFor example, to limit output to buckets starting with `data-`:\n\n```\n$ list_buckets data-*\n```\n\n# Listing the content of a Cloud Object Storage bucket\n\nYou can run the list utility in a terminal window using the generated console script\n\n```\n$ list_files --help\n```\n\nor explicitly\n\n```\n$ python -m cos_utils.list_files --help\n```\n\nThe help lists required and optional parameters.\n\n```\nusage: list_files [-h] bucket pattern\n\nList the content of a Cloud Object Storage bucket.\n\npositional arguments:\n  bucket      Bucket name\n  pattern     Object key spec (supported wildcards: * and ?)\n\noptional arguments:\n  -h, --help  show this help message and exit\n\nEnvironment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY must be\ndefined to run the utility.\n```\n\n## List the content of `\u003cbucket-name\u003e`\n\n```\n$ list_files \u003cbucket-name\u003e *\n```\n\n\u003e On Linux, Unix and MacOS wildcards need to be escaped to prevent shell expansion: `list_files \u003cbucket-name\u003e \\*`.\n\n## Apply a filter\n\nUse the `*` (any character) and `?` (one character) wildcards to define a filter condition.\n\nFor example, to limit output to files ending in `.png`:\n\n```\n$ list_files \u003cbucket-name\u003e *.png\n```\n\n# Uploading files to a Cloud Object Storage bucket\n\nYou can run the upload utility in a terminal window using the generated console script\n\n```\n$ upload_files --help\n```\n\nor explicitly\n\n```\n$ python -m cos_utils.upload_files --help\n```\n\nThe help lists required and optional parameters. The examples listed below explain them in detail.\n\n```\nusage: upload_files [-h] [-p PREFIX] [-r] [-s] [-w] bucket pattern\n\nUpload files to a Cloud Object Storage bucket.\n\npositional arguments:\n  bucket                Bucket name\n  pattern               File or directory spec (supported wildcards: * and ?)\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -p PREFIX, --prefix PREFIX\n                        Key name prefix\n  -r, --recursive       Include files in subdirectories\n  -s, --squash          Exclude subdirectory name from key name\n  -w, --wipe            Clear bucket prior to upload\n\nEnvironment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY must be\ndefined to run the utility.\n```\n\n## Example scenario\n\nThe `\u003c/path/to/local/directory\u003e` contains the following directories and files:\n```\nfile1.png\nfile2.png\nfile3.jpg\nfile4.txt\ndir1/file5.gif\ndir1/file6.png\ndir1/dir2/file7.png\ndir1/dir3/file8.jpg\ndir1/dir3/file1.png\n```\n\nIn the examples given below `\u003cbucket-name\u003e` refers to an existing bucket in Cloud Object Storage.\n\n## Upload directories\n\nYou can upload the content of any directory.\n\n### Upload the content of `\u003c/path/to/local/directory\u003e` to bucket `\u003cbucket-name\u003e`\n\n```\n$ upload_files \u003cbucket-name\u003e \u003c/path/to/local/directory\u003e\n```\n\nBucket `\u003cbucket-name\u003e` contains the following objects:\n\n```\nfile1.png\nfile2.png\nfile3.jpg\nfile4.txt\n```\n\n### Same as before but clear the bucket first before uploading\n\nSpecify the optional `--wipe` parameter to clear the bucket before upload.\n\n```\n$ upload_files \u003cbucket-name\u003e \u003c/path/to/local/directory\u003e --wipe\n```\n\nBucket `\u003cbucket-name\u003e` contains the following objects:\n\n```\nfile1.png\nfile2.png\nfile3.jpg\nfile4.txt\n```\n\n### Same as before but include subdirectories\n\nSpecify the optional `--recursive` parameter include files in subdirectories.\n\n```\n$ upload_files \u003cbucket-name\u003e \u003c/path/to/local/directory\u003e --wipe --recursive\n```\n\nBucket `\u003cbucket-name\u003e` contains the following objects:\n\n```\nfile1.png\nfile2.png\nfile3.jpg\nfile4.txt\ndir1/file5.gif\ndir1/file6.png\ndir1/dir2/file7.png\ndir1/dir3/file8.jpg\ndir1/dir3/file1.png\n```\n\n### Same as before but don't use subdirectory names during object key generation\n\nSpecify the optional `--squash` parameter to ignore subdirectory names during object key generation.\n\n```\n$ upload_files \u003cbucket-name\u003e \u003c/path/to/local/directory\u003e --wipe --recursive --squash\n```\n\nBucket `\u003cbucket-name\u003e` contains the following objects. Note that `\u003c/path/to/local/directory\u003e` contains two files named `file1.png`. First `file1.png` is uploaded and later overwritten with the content of `dir1/dir3/file1.png`.\n\n```\nfile2.png\nfile3.jpg\nfile4.txt\nfile5.gif\nfile6.png\nfile7.png\nfile8.jpg\nfile1.png\n```\n\n### Same as before but include a static key name prefix\n\nSpecify the optional `--prefix \u003cprefix\u003e` parameter to add `\u003cprefix\u003e` to the object key for every file.\n\n```\n$ upload_files \u003cbucket-name\u003e \u003c/path/to/local/directory\u003e --wipe --recursive --squash --prefix data\n```\n\nBucket `\u003cbucket-name\u003e` contains the following objects:\n\n```\ndata/file2.png\ndata/file3.jpg\ndata/file4.txt\ndata/file5.gif\ndata/file6.png\ndata/file7.png\ndata/file8.jpg\ndata/file1.png\n```\n\n## Upload files\n\nYou can upload a single file by specifying `\u003c/path/to/local/directory/filename\u003e`.\n\n```\n$ upload_files \u003cbucket-name\u003e /path/to/local/directory/file1.png --wipe \n```\n\nBucket `\u003cbucket-name\u003e` contains the following object:\n\n```\nfile1.png\n```\n\nYou can upload multiple files by specifying a pattern `\u003c/path/to/local/directory/filename-pattern\u003e`\n\n```\n$ upload_files \u003cbucket-name\u003e /path/to/local/directory/*.png --wipe \n```\n\n\u003e On Linux, Unix and MacOS wildcards need to be escaped to prevent shell expansion: `/path/to/local/directory/\\*.png`.\n\nBucket `\u003cbucket-name\u003e` contains the following objects:\n\n```\nfile1.png\nfile2.png\n```\n\nUse the `--recursive` parameter to extend the search to subdirectories of `/path/to/local/directory/`.\n\n```\n$ upload_files \u003cbucket-name\u003e /path/to/local/directory/*.png --wipe --recursive\n```\n\n```\nfile1.png\nfile2.png\ndir1/file6.png\ndir1/dir2/file7.png\ndir1/dir3/file1.png\n```\n\n# Downloading files from a Cloud Object Storage bucket\n\nYou can run the download utility in a terminal window using the generated console script\n\n```\n$ download_files --help\n```\n\nor explicitly\n\n```\n$ python -m cos_utils.dowload_files --help\n```\n\nThe help lists required and optional parameters. The examples listed below explain them in detail.\n\n```\nusage: download_files [-h] [-d TARGET_DIR] bucket pattern\n\nDownload objects from a Cloud Object Storage bucket.\n\npositional arguments:\n  bucket                Bucket name\n  pattern               Object key spec (supported wildcards: * and ?)\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -d TARGET_DIR, --target_dir TARGET_DIR\n                        Local target directory. Defaults to the current\n                        directory.\n\nEnvironment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY must be\ndefined to run the utility.\n```\n\n### Download complete bucket content\n\nYou can download the complete content of a bucket to the current directory:\n\n```\n$ download_files \u003cbucket-name\u003e *\n```\n\n\u003e On Linux, Unix and MacOS wildcards need to be escaped to prevent shell expansion: `download_files \u003cbucket-name\u003e \\*`.\n\n### Same as before but specify a target directory\n\nUse the `--target_dir \u003c/path/to/local/dir\u003e` parameter to specify an existing directory where the downloaded files will be stored:\n\n```\n$ download_files \u003cbucket-name\u003e * --target_dir /tmp/downloads\n```\n\n### Use wildcards to selectively download files\n\nUse the `*` (any character) and `?` (one character) wildcards to define a filter condition.\n\n#### Download only png files\n\n```\n$ download_files \u003cbucket-name\u003e *.png\n```\n\n#### Download files that contain a certain string in their name\n\n```\n$ download_files \u003cbucket-name\u003e *fil*\n```\n\n# Removing files from a Cloud Object Storage bucket\n\nYou can run the remove utility in a terminal window using the generated console script\n\n```\n$ remove_files --help\n```\n\nor explicitly\n\n```\n$ python -m cos_utils.remove_files --help\n```\n\n### Remove all files from a bucket\n\n```\n$ remove_files \u003cbucket-name\u003e\n```\n\n# License\n\n[Apache-2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodait%2Fcos-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodait%2Fcos-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodait%2Fcos-utils/lists"}