{"id":13484445,"url":"https://github.com/karafka/karafka","last_synced_at":"2025-12-28T17:54:51.243Z","repository":{"id":35672396,"uuid":"39947871","full_name":"karafka/karafka","owner":"karafka","description":"Ruby and Rails efficient Kafka processing framework","archived":false,"fork":false,"pushed_at":"2025-10-06T16:36:55.000Z","size":7269,"stargazers_count":2193,"open_issues_count":44,"forks_count":186,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-10-06T17:42:50.335Z","etag":null,"topics":["active-job","activejob-backend","apache-kafka","background-jobs","events","kafka","kafka-client","kafka-clients","kafka-consumer","kafka-manager","kafka-monitor","kafka-producer","kafka-ruby","karafka-framework","opensource","rails","ruby","ruby-library","ruby-on-rails","rubyonrails"],"latest_commit_sha":null,"homepage":"https://karafka.io","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karafka.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"custom":["https://karafka.io/#become-pro"]}},"created_at":"2015-07-30T11:49:27.000Z","updated_at":"2025-10-06T16:17:31.000Z","dependencies_parsed_at":"2023-09-22T23:17:56.450Z","dependency_job_id":"b4e10023-b654-443e-aa25-2020eef5f69d","html_url":"https://github.com/karafka/karafka","commit_stats":{"total_commits":2004,"total_committers":61,"mean_commits":32.85245901639344,"dds":"0.17465069860279436","last_synced_commit":"f1a1de76c50816377209714f7b4987b4d8783785"},"previous_names":[],"tags_count":149,"template":false,"template_full_name":null,"purl":"pkg:github/karafka/karafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karafka%2Fkarafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karafka%2Fkarafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karafka%2Fkarafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karafka%2Fkarafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karafka","download_url":"https://codeload.github.com/karafka/karafka/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karafka%2Fkarafka/sbom","scorecard":{"id":451852,"data":{"date":"2025-08-11","repo":{"name":"github.com/karafka/karafka","commit":"d4bb5ddac6e431917e25cb9261d234ca99f705a8"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":6.2,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Code-Review","score":0,"reason":"Found 0/20 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"30 commit(s) and 24 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: jobLevel 'contents' permission set to 'write': .github/workflows/push.yml:18","Info: topLevel 'contents' permission set to 'read': .github/workflows/ci.yml:16","Info: topLevel 'contents' permission set to 'read': .github/workflows/push.yml:9","Warn: no topLevel permission defined: .github/workflows/trigger-wiki-refresh.yml:1","Warn: no topLevel permission defined: .github/workflows/verify-action-pins.yml:1"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Pinned-Dependencies","score":10,"reason":"all dependencies are pinned","details":["Info:   8 out of   8 GitHub-owned GitHubAction dependencies pinned","Info:   7 out of   7 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 23 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-19T08:22:56.002Z","repository_id":35672396,"created_at":"2025-08-19T08:22:56.002Z","updated_at":"2025-08-19T08:22:56.002Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278854210,"owners_count":26057418,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["active-job","activejob-backend","apache-kafka","background-jobs","events","kafka","kafka-client","kafka-clients","kafka-consumer","kafka-manager","kafka-monitor","kafka-producer","kafka-ruby","karafka-framework","opensource","rails","ruby","ruby-library","ruby-on-rails","rubyonrails"],"created_at":"2024-07-31T17:01:24.552Z","updated_at":"2025-10-08T18:20:39.655Z","avatar_url":"https://github.com/karafka.png","language":"Ruby","readme":"![karafka logo](https://cdn.karafka.io/assets/misc/logo/karafka_logotype_transparent2.png)\n\n[![Build Status](https://github.com/karafka/karafka/actions/workflows/ci_linux_ubuntu_x86_64_gnu.yml/badge.svg)](https://github.com/karafka/karafka/actions/workflows/ci_linux_ubuntu_x86_64_gnu.yml)\n[![Gem Version](https://badge.fury.io/rb/karafka.svg)](http://badge.fury.io/rb/karafka)\n[![Join the chat at https://slack.karafka.io](https://raw.githubusercontent.com/karafka/misc/master/slack.svg)](https://slack.karafka.io)\n\n## About Karafka\n\nKarafka is a Ruby and Rails multi-threaded efficient Kafka processing framework that:\n\n- Has a built-in [Web UI](https://karafka.io/docs/Web-UI-Features/) providing a convenient way to monitor and manage Karafka-based applications.\n- Supports parallel processing in [multiple threads](https://karafka.io/docs/Concurrency-and-multithreading) (also for a [single topic partition](https://karafka.io/docs/Pro-Virtual-Partitions) work) and [processes](https://karafka.io/docs/Swarm-Multi-Process).\n- [Automatically integrates](https://karafka.io/docs/Integrating-with-Ruby-on-Rails-and-other-frameworks#integrating-with-ruby-on-rails) with Ruby on Rails\n- Has [ActiveJob backend](https://karafka.io/docs/Active-Job) support (including [ordered jobs](https://karafka.io/docs/Pro-Enhanced-Active-Job#ordered-jobs))\n- Has a seamless [Dead Letter Queue](https://karafka.io/docs/Dead-Letter-Queue/) functionality built-in\n- Supports in-development [code reloading](https://karafka.io/docs/Auto-reload-of-code-changes-in-development)\n- Is powered by [librdkafka](https://github.com/edenhill/librdkafka) (the Apache Kafka C/C++ client library)\n- Has an out-of the box [AppSignal](https://karafka.io/docs/Monitoring-and-Logging/#appsignal-metrics-and-error-tracking) and [StatsD/DataDog](https://karafka.io/docs/Monitoring-and-Logging/#datadog-and-statsd-integration) monitoring with dashboard templates.\n\n```ruby\n# Define what topics you want to consume with which consumers in karafka.rb\nKarafka::App.routes.draw do\n  topic 'system_events' do\n    consumer EventsConsumer\n  end\nend\n\n# And create your consumers, within which your messages will be processed\nclass EventsConsumer \u003c ApplicationConsumer\n  # Example that utilizes ActiveRecord#insert_all and Karafka batch processing\n  def consume\n    # Store all of the incoming Kafka events locally in an efficient way\n    Event.insert_all messages.payloads\n  end\nend\n```\n\nKarafka **uses** threads to handle many messages simultaneously in the same process. It does not require Rails but will integrate tightly with any Ruby on Rails applications to make event processing dead simple.\n\n## Getting started\n\n![karafka web ui](https://cdn.karafka.io/assets/misc/printscreens/web-ui.png)\n\nIf you're entirely new to the subject, you can start with our \"Kafka on Rails\" articles series, which will get you up and running with the terminology and basic ideas behind using Kafka:\n\n- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 1 – Kafka basics and its advantages](https://mensfeld.pl/2017/11/kafka-on-rails-using-kafka-with-ruby-on-rails-part-1-kafka-basics-and-its-advantages/)\n- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 2 – Getting started with Rails and Kafka](https://mensfeld.pl/2018/01/kafka-on-rails-using-kafka-with-ruby-on-rails-part-2-getting-started-with-ruby-and-kafka/)\n\nIf you want to get started with Kafka and Karafka as fast as possible, then the best idea is to visit our [Getting started](https://karafka.io/docs/Getting-Started) guides and the [example apps repository](https://github.com/karafka/example-apps).\n\nWe also maintain many [integration specs](https://github.com/karafka/karafka/tree/master/spec/integrations) illustrating various use-cases and features of the framework.\n\n### TL;DR (1 minute from setup to publishing and consuming messages)\n\n**Prerequisites**: Kafka running. You can start it by following instructions from [here](https://karafka.io/docs/Setting-up-Kafka).\n\n1. Add and install Karafka:\n\n```bash\n# Make sure to install Karafka 2.4\nbundle add karafka --version \"\u003e= 2.4.0\"\n\nbundle exec karafka install\n```\n\n2. Dispatch a message to the example topic using the Rails or Ruby console:\n\n```ruby\nKarafka.producer.produce_sync(topic: 'example', payload: { 'ping' =\u003e 'pong' }.to_json)\n```\n\n3. Run Karafka server and see the consumption magic happen:\n\n```bash\nbundle exec karafka server\n\n[86d47f0b92f7] Polled 1 message in 1000ms\n[3732873c8a74] Consume job for ExampleConsumer on example started\n{\"ping\"=\u003e\"pong\"}\n[3732873c8a74] Consume job for ExampleConsumer on example finished in 0ms\n```\n\n## Want to Upgrade? LGPL is not for you? Want to help?\n\nI also sell Karafka Pro subscriptions. It includes a commercial-friendly license, priority support, architecture consultations, enhanced Web UI and high throughput data processing-related features (virtual partitions, long-running jobs, and more).\n\nPart of the income is [distributed back](https://github.com/orgs/karafka/sponsoring) to other OSS projects that Karafka uses under the hood.\n\nHelp me provide high-quality open-source software. Please see the Karafka [homepage](https://karafka.io/#become-pro) for more details.\n\n## Support\n\nKarafka has [Wiki pages](https://karafka.io/docs) for almost everything and a pretty decent [FAQ](https://karafka.io/docs/FAQ). It covers the installation, setup, and deployment, along with other useful details on how to run Karafka.\n\nIf you have questions about using Karafka, feel free to join our [Slack](https://slack.karafka.io) channel.\n\nKarafka has [priority support](https://karafka.io/docs/Pro-Support) for technical and architectural questions that is part of the Karafka Pro subscription.\n","funding_links":["https://karafka.io/#become-pro"],"categories":["Happy Exploring 🤘","Ruby","Gems","Queues and Messaging","Web framework for Ruby","Development","Clients"],"sub_categories":["Background Jobs","Client libraries","Ruby"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarafka%2Fkarafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarafka%2Fkarafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarafka%2Fkarafka/lists"}