{"id":20724507,"url":"https://github.com/laserdisc-io/tamer","last_synced_at":"2025-09-21T06:41:38.663Z","repository":{"id":36476247,"uuid":"225499179","full_name":"laserdisc-io/tamer","owner":"laserdisc-io","description":"Standalone alternatives to Kafka Connect Connectors","archived":false,"fork":false,"pushed_at":"2025-09-17T22:59:03.000Z","size":1754,"stargazers_count":45,"open_issues_count":9,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-09-18T00:26:18.523Z","etag":null,"topics":["avro","doobie","fp","fs2","functional-programming","jdbc","kafka","s3","scala","schema-registry","zio","zio-streams"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/laserdisc-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-12-03T01:00:07.000Z","updated_at":"2025-09-17T22:59:07.000Z","dependencies_parsed_at":"2023-10-17T06:45:34.189Z","dependency_job_id":"8b7668fb-cbcc-4cfa-a36f-62ee8ea7fd56","html_url":"https://github.com/laserdisc-io/tamer","commit_stats":{"total_commits":1405,"total_committers":10,"mean_commits":140.5,"dds":0.08825622775800712,"last_synced_commit":"0907a087dbb911927ada6fceebd4996b0f1b71b5"},"previous_names":[],"tags_count":49,"template":false,"template_full_name":null,"purl":"pkg:github/laserdisc-io/tamer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Ftamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Ftamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Ftamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Ftamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/laserdisc-io","download_url":"https://codeload.github.com/laserdisc-io/tamer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Ftamer/sbom","scorecard":{"id":579492,"data":{"date":"2025-08-11","repo":{"name":"github.com/laserdisc-io/tamer","commit":"a16db05eef5d90b9d04449c69e2f870308c712ff"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.9,"checks":[{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"30 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/clean.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:233: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:238: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:243: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:256: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:269: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:280: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:48: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:53: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:66: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:79: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:117: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:133: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:138: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:143: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:156: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:169: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:180: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:190: update your workflow using https://app.stepsecurity.io/secureworkflow/laserdisc-io/tamer/ci.yml/master?enable=pin","Info:   0 out of  15 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   4 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-20T18:48:29.245Z","repository_id":36476247,"created_at":"2025-08-20T18:48:29.245Z","updated_at":"2025-08-20T18:48:29.245Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276047233,"owners_count":25575881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-20T02:00:10.207Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","doobie","fp","fs2","functional-programming","jdbc","kafka","s3","scala","schema-registry","zio","zio-streams"],"created_at":"2024-11-17T04:15:02.447Z","updated_at":"2025-09-21T06:41:38.634Z","avatar_url":"https://github.com/laserdisc-io.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tamer\n[![Continuous Integration](https://github.com/laserdisc-io/tamer/actions/workflows/ci.yml/badge.svg)](https://github.com/laserdisc-io/tamer/actions/workflows/ci.yml)\n[![Known Vulnerabilities](https://snyk.io/test/github/laserdisc-io/tamer/badge.svg?targetFile=build.sbt)](https://snyk.io/test/github/laserdisc-io/tamer?targetFile=build.sbt)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.laserdisc/tamer-core_2.13/badge.svg?kill_cache=1\u0026color=orange)](https://search.maven.org/artifact/io.laserdisc/tamer-core_2.13/)\n[![Scala Steward badge](https://img.shields.io/badge/Scala_Steward-helping-blue.svg?style=flat\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAQCAMAAAARSr4IAAAAVFBMVEUAAACHjojlOy5NWlrKzcYRKjGFjIbp293YycuLa3pYY2LSqql4f3pCUFTgSjNodYRmcXUsPD/NTTbjRS+2jomhgnzNc223cGvZS0HaSD0XLjbaSjElhIr+AAAAAXRSTlMAQObYZgAAAHlJREFUCNdNyosOwyAIhWHAQS1Vt7a77/3fcxxdmv0xwmckutAR1nkm4ggbyEcg/wWmlGLDAA3oL50xi6fk5ffZ3E2E3QfZDCcCN2YtbEWZt+Drc6u6rlqv7Uk0LdKqqr5rk2UCRXOk0vmQKGfc94nOJyQjouF9H/wCc9gECEYfONoAAAAASUVORK5CYII=)](https://scala-steward.org)\n\nTamer is a domesticated Kafka source connector.\n\nIt puts the user of this library in complete control of how data is ingested and what state is preserved (in a compacted Kafka topic).\nAs an example, it allows for a JDBC source to pull a window of data (e.g. 5 minutes), starting from some time in the past, as fast as possible.\nAt every \"pull\", the user can decide what to do next (e.g. should the window be increased/decreased? Should the pull slow down?).\n\nTamer currently supports 4 possible source types:\n- any SQL data store Doobie can handle (e.g. Postgres, MySQL, etc)\n- any cloud storage that is AWS S3 compatible\n- OCI Object Storage\n- any type of REST API, with AuthN, pagination and HATEOAS support\n\nSome sensible defaults are provided out of the box but many customisations are possible.\n\n## Usage\n\nAdd one of Tamer's modules as a dependency to your project:\n\n```\n// check the current version on Maven Central (or use the badge above)\nlibraryDependencies += \"io.laserdisc\" %% \"tamer-db\"                % version\nlibraryDependencies += \"io.laserdisc\" %% \"tamer-oci-objectstorage\" % version\nlibraryDependencies += \"io.laserdisc\" %% \"tamer-rest\"              % version\nlibraryDependencies += \"io.laserdisc\" %% \"tamer-s3\"                % version\n```\n\nSee [here](example/src/main/scala/tamer/db/DatabaseSimple.scala) for a sample application that makes use of Tamer's Db module for ingesting data from a JDBC datasource.\n\n## End to end testing\n\n### Database module\n\nBasic manual testing is available for the code in the example module `tamer.db.DatabaseSimple` (and/or `tamer.db.DatabaseGeneralized`).\nThis code covers getting data from a synthetic Postgres database.\n\nMake sure you have docker installed before proceeding.\n\nFrom the `db/local` folder launch `docker compose up` (you can enter `docker compose down` if you want to start from scratch). After that you should be able to access the Kafka gui from [http://localhost:8000](http://localhost:8000).\n\nStart the `runDatabaseSimple.sh` program which contains some example environment variables.\nIf Tamer works you should see messages appearing in the Kafka gui.\n\n### S3 module\n\nBasic manual testing is available for the code in the example module `tamer.s3.S3Simple`.\nThis code covers getting data from a synthetic S3 bucket.\n\nMake sure you have docker installed before proceeding.\n\nFrom the `s3/local` folder launch `docker compose up` (you can enter `docker compose down` if you want to start from scratch). After that you should be able to access the Kafka gui from [http://localhost:8000](http://localhost:8000).\n\nStart the `runS3Simple.sh` program which contains some example environment variables.\nIf Tamer works you should see messages appearing in the Kafka gui.\n\n## License\n\nTamer is licensed under the **[MIT License](LICENSE)** (the \"License\"); you may not use this software except in compliance with the License.\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and limitations under the License.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaserdisc-io%2Ftamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flaserdisc-io%2Ftamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaserdisc-io%2Ftamer/lists"}