{"id":31530191,"url":"https://github.com/frictionlessdata/dataflows-aws","last_synced_at":"2025-10-04T01:19:19.607Z","repository":{"id":57417903,"uuid":"203781338","full_name":"frictionlessdata/dataflows-aws","owner":"frictionlessdata","description":"AWS integration for dataflows","archived":false,"fork":false,"pushed_at":"2022-12-12T20:16:37.000Z","size":20,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-27T15:11:17.400Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frictionlessdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-22T11:29:48.000Z","updated_at":"2022-12-14T23:04:27.000Z","dependencies_parsed_at":"2023-01-28T00:45:50.093Z","dependency_job_id":null,"html_url":"https://github.com/frictionlessdata/dataflows-aws","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/frictionlessdata/dataflows-aws","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Fdataflows-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Fdataflows-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Fdataflows-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Fdataflows-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frictionlessdata","download_url":"https://codeload.github.com/frictionlessdata/dataflows-aws/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Fdataflows-aws/sbom","scorecard":{"id":411425,"data":{"date":"2025-08-11","repo":{"name":"github.com/frictionlessdata/dataflows-aws","commit":"9464a484be8d6bda6e49baf4ee949a3058bf3bf0"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/23 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.md:0","Info: FSF or OSI recognized license: MIT License: LICENSE.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 1 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T22:48:06.113Z","repository_id":57417903,"created_at":"2025-08-18T22:48:06.113Z","updated_at":"2025-08-18T22:48:06.113Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278252258,"owners_count":25956269,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-04T01:19:05.238Z","updated_at":"2025-10-04T01:19:19.600Z","avatar_url":"https://github.com/frictionlessdata.png","language":"Python","readme":"# dataflows-aws\n\n[![Travis](https://travis-ci.org/frictionlessdata/dataflows-aws.svg?branch=master)](https://travis-ci.org/frictionlessdata/dataflows-aws)\n[![Coveralls](http://img.shields.io/coveralls/frictionlessdata/dataflows-aws.svg?branch=master)](https://coveralls.io/r/frictionlessdata/dataflows-aws?branch=master)\n\nDataflows's processors to work with AWS\n\n## Features\n\n- `dump_to_s3` processor\n- `change_acl_on_s3` processor\n\n## Contents\n\n\u003c!--TOC--\u003e\n\n  - [Getting Started](#getting-started)\n    - [Installation](#installation)\n    - [Examples](#examples)\n  - [Documentation](#documentation)\n    - [dump_to_s3](#dump_to_s3)\n    - [change_acl_on_s3](#change_acl_on_s3)\n  - [Contributing](#contributing)\n  - [Changelog](#changelog)\n\n\u003c!--TOC--\u003e\n\n## Getting Started\n\n### Installation\n\nThe package use semantic versioning. It means that major versions  could include breaking changes. It's recommended to specify `package` version range in your `setup/requirements` file e.g. `package\u003e=1.0,\u003c2.0`.\n\n```bash\n$ pip install dataflows-aws\n```\n\n### Examples\n\nThese processors have to be used as a part of data flow. For example:\n\n```python\nflow = Flow(\n    load('data/data.csv'),\n    dump_to_s3(\n        bucket=bucket,\n        acl='private',\n        path='my/datapackage',\n        endpoint_url=os.environ['S3_ENDPOINT_URL'],\n    ),\n)\nflow.process()\n```\n\n## Documentation\n\n### dump_to_s3\n\nSaves the DataPackage to AWS S3.\n\n#### Parameters\n\n- `bucket` - Name of the bucket where DataPackage will be stored (should already be created!)\n- `acl` - ACL to provide the uploaded files. Default is 'public-read' (see [boto3 docs](http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.put_object) for more info).\n- `path` - Path (key/prefix) to the DataPackage. May contain format string available for `datapackage.json` Eg: `my/example/path/{owner}/{name}/{version}`\n- `content_type` - content type to use when storing files in S3. Defaults to text/plain (usual S3 default is binary/octet-stream but we prefer text/plain).\n- `endpoint_url` - api endpoint to allow using S3 compatible services (e.g. 'https://ams3.digitaloceanspaces.com')\n\n### change_acl_on_s3\n\nChanges ACL of object in given Bucket with given path aka prefix.\n\n#### Parameters\n\n- `bucket` - Name of the bucket where objects are stored\n- `acl` - Available options `'private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control'`\n- `path` - Path (key/prefix) to the DataPackage.\n- `endpoint_url` - api endpoint to allow using S3 compatible services (e.g. 'https://ams3.digitaloceanspaces.com')\n\n## Contributing\n\nThe project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).\n\nThe recommended way to get started is to create and activate a project virtual environment.\nTo install package and development dependencies into your active environment:\n\n```\n$ make install\n```\n\nTo run tests with linting and coverage:\n\n```bash\n$ make test\n```\n\nFor linting, `pylama` (configured in `pylama.ini`) is used. At this stage it's already\ninstalled into your environment and could be used separately with more fine-grained control\nas described in documentation - https://pylama.readthedocs.io/en/latest/.\n\nFor example to sort results by error type:\n\n```bash\n$ pylama --sort \u003cpath\u003e\n```\n\nFor testing, `tox` (configured in `tox.ini`) is used.\nIt's already installed into your environment and could be used separately with more fine-grained control as described in documentation - https://testrun.org/tox/latest/.\n\nFor example to check subset of tests against Python 2 environment with increased verbosity.\nAll positional arguments and options after `--` will be passed to `py.test`:\n\n```bash\ntox -e py37 -- -v tests/\u003cpath\u003e\n```\n\nUnder the hood `tox` uses `pytest` (configured in `pytest.ini`), `coverage`\nand `mock` packages. These packages are available only in tox envionments.\n\n## Changelog\n\nHere described only breaking and the most important changes. The full changelog and documentation for all released versions can be found in the nicely formatted [commit history](https://github.com/frictionlessdata/dataflows-aws/commits/master).\n\n#### v0.x\n\n- an initial processors implementation\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Fdataflows-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrictionlessdata%2Fdataflows-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Fdataflows-aws/lists"}