{"id":37403590,"url":"https://github.com/oslokommune/okdata-aws","last_synced_at":"2026-01-16T05:48:29.468Z","repository":{"id":39706939,"uuid":"310529566","full_name":"oslokommune/okdata-aws","owner":"oslokommune","description":"Collection of helpers for working with AWS Lambda","archived":false,"fork":false,"pushed_at":"2025-10-29T07:11:50.000Z","size":138,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-10-29T09:11:49.973Z","etag":null,"topics":["dataplatform"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oslokommune.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-11-06T07:56:14.000Z","updated_at":"2025-10-29T07:11:53.000Z","dependencies_parsed_at":"2023-10-03T13:17:49.534Z","dependency_job_id":"39638272-d819-4b31-b3fc-6640bc19dafb","html_url":"https://github.com/oslokommune/okdata-aws","commit_stats":{"total_commits":55,"total_committers":8,"mean_commits":6.875,"dds":0.5272727272727273,"last_synced_commit":"e5b11b6aad70a15c22f06a739183dc397cd02dec"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/oslokommune/okdata-aws","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oslokommune","download_url":"https://codeload.github.com/oslokommune/okdata-aws/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-aws/sbom","scorecard":{"id":713698,"data":{"date":"2025-08-11","repo":{"name":"github.com/oslokommune/okdata-aws","commit":"c9b88131c683272c9b2f80c4992dabb627b89e62"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.6,"checks":[{"name":"Maintained","score":7,"reason":"9 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 7","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":4,"reason":"Found 3/7 approved changesets -- score normalized to 4","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":8,"reason":"dependency not pinned by hash detected -- score normalized to 8","details":["Warn: pipCommand not pinned by hash: .github/workflows/run-tests.yml:21","Info:   6 out of   6 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql-analysis.yml:17","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql-analysis.yml:18","Info: jobLevel 'actions' permission set to 'read': .github/workflows/labeler.yml:9","Info: jobLevel 'contents' permission set to 'read': .github/workflows/labeler.yml:10","Warn: no topLevel permission defined: .github/workflows/codeql-analysis.yml:1","Warn: no topLevel permission defined: .github/workflows/labeler.yml:1","Warn: no topLevel permission defined: .github/workflows/run-tests.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":0,"reason":"license file not detected","details":["Warn: project does not have a license file"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":7,"reason":"SAST tool detected but not run on all commits","details":["Info: SAST configuration detected: CodeQL","Warn: 10 commits out of 27 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-22T08:53:07.788Z","repository_id":39706939,"created_at":"2025-08-22T08:53:07.788Z","updated_at":"2025-08-22T08:53:07.788Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28477421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T03:13:13.607Z","status":"ssl_error","status_checked_at":"2026-01-16T03:11:47.863Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataplatform"],"created_at":"2026-01-16T05:48:29.379Z","updated_at":"2026-01-16T05:48:29.456Z","avatar_url":"https://github.com/oslokommune.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# okdata-aws\n\nCollection of helpers for working with AWS.\n\n## Logging for Lambda\n\nBased on [Structlog](https://www.structlog.org/).\n\nStructured and enriched logging for AWS Lambda functions.\n\n### TL;DR:\n\n - Decorate handler with `logging_wrapper`\n - Encrich logs with key/value pairs using `log_add`\n - Time functions with `log_duration`\n - Log exceptions with `log_exception`\n\n### Usage\n\n#### Regular Lambda functions\n\nWrap your Lambda handler with `logging_wrapper`. Badabing badabom, you're good\nto go!\n\nYou can set the service name using the `logging.init` method, or configure it\nusing the `SERVICE_NAME` environment variable.\n\n```python\nfrom okdata.aws import logging\nfrom okdata.aws.logging import logging_wrapper\n\nlogging.init(\"my_fantastic_lambda\")\n\n@logging_wrapper\ndef handler(event, context):\n    if error:\n        return {\n            \"statusCode\": 500,\n            \"body\": \"Automatically logs bodies from error responses even!\",\n        }\n```\n\n#### FastAPI applications\n\nCall `add_fastapi_logging` with the FastAPI application as a parameter\nto add the logging middleware.\n\n```python\nfrom okdata.aws.logging import add_fastapi_logging\n\napp = FastAPI()\nadd_fastapi_logging(app)\n```\n\n#### Encriching logs\n\nBy automagic logs will be enriched with git revisions, cold start y/n call\nduration and much more, but to add even more magic you can use `log_add` and\n`log_duration`.\n\n```python\nfrom okdata.aws.logging import logging_wrapper, log_add, log_duration\n\n@logging_wrapper\ndef handler(event, context):\n    log_add(dataset=event[\"dataset\"], foo=context[\"foo\"])\n    log_duration(\n        lambda: slow_thinger(event[\"dataset\"]),\n        \"my_slow_thinger\"\n    )\n    ... and so on\n\ndef slow_thinger():\n    sleep(9999999999999999)\n```\n\n#### Exceptions\n\nStruct log can extract exception info if we log the exception to the special\n[`exc_info`](https://www.structlog.org/en/stable/api.html#structlog.processors.format_exc_info)\nkey.\n\nFor convenience we catch and log uncaught exceptions using this already.\n\nIf you need to process an exception you can use `log_exception` to log it to the\n`exc_info` key.\n\n```python\nfrom okdata.aws.logging import logging_wrapper, log_exception\n\n@logging_wrapper\ndef handler(event, context):\n    try:\n        thing()\n    except MyException as e:\n        log_exception(e)\n        return { ... }\n```\n\n\n## Status wrapper\n\nThe status wrapper logs details about a Lambda function execution and sends it\nto the status API.\n\nThe first component that touches the data (typically on upload) sets a \"trace\nID\", which is then inherited by the following processing steps. This allows the\nstatus API to track what has happened to the data, from upload through the\nvarious processing steps until the data is ready for consumption.\n\nFor pipeline components, the status wrapper picks up the trace ID from the\nLambda event automatically.\n\nThe status wrapper expects the `SERVICE_NAME` of the Lambda component to be set\nin an environment variable, along with `GIT_REV` and `GIT_BRANCH`.\n\n### Usage\n\nTag the Lambda handler function with `@status_wrapper`.\n\nThe handler function should set the `domain` and `domain_id` values using the\n`status_add` method:\n\n```python\nfrom okdata.aws.status import status_wrapper, status_add\n\n@status_wrapper()\ndef my_lambda_handler(event, context):\n    status_add(domain=\"dataset\", domain_id=f\"{dataset_id}/{version}\")\n\n    # Regular handler logic here ...\n\n    # The handler can also add a body object containing component-specific information\n    status_body = {\n        \"input\": \"/tmp/file.txt\",\n        \"output\": \"/tmp/file.csv\",\n        \"transformation\": \"text-to-csv\",\n    }\n    status_add(status_body=status_body)\n```\n\nBy default, this will send a status event with event status `OK` and trace\nstatus `CONTINUE`, meaning that the data pipeline is still running. If the\nhandler function fails, e.g. throws an exception, it will send event status\n`FAILED` and trace status `FINISHED`, in addition to the failure details\n(exception).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslokommune%2Fokdata-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foslokommune%2Fokdata-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslokommune%2Fokdata-aws/lists"}