{"id":16410213,"url":"https://github.com/perkss/clojure-kafka-examples","last_synced_at":"2025-04-10T03:52:20.062Z","repository":{"id":45294020,"uuid":"124704977","full_name":"perkss/clojure-kafka-examples","owner":"perkss","description":"Clojure examples for Kafka and Kafka Streams (JAVA Interop)","archived":false,"fork":false,"pushed_at":"2024-09-21T06:59:46.000Z","size":118,"stargazers_count":83,"open_issues_count":0,"forks_count":11,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-10T03:51:59.875Z","etag":null,"topics":["avro-kafka","clojure","functional-programming","java","kafka","kafka-streams","kafka-topic","testcontainers"],"latest_commit_sha":null,"homepage":"https://perkss.github.io/#/clojure/KafkaClojure#text-body","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/perkss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-10T22:48:17.000Z","updated_at":"2025-03-13T15:28:07.000Z","dependencies_parsed_at":"2023-12-30T21:56:23.833Z","dependency_job_id":"c0d8ffd0-e903-42c4-bd3d-7cf57b4cae6b","html_url":"https://github.com/perkss/clojure-kafka-examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/perkss%2Fclojure-kafka-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/perkss%2Fclojure-kafka-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/perkss%2Fclojure-kafka-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/perkss%2Fclojure-kafka-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/perkss","download_url":"https://codeload.github.com/perkss/clojure-kafka-examples/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154998,"owners_count":21056542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro-kafka","clojure","functional-programming","java","kafka","kafka-streams","kafka-topic","testcontainers"],"created_at":"2024-10-11T06:23:31.592Z","updated_at":"2025-04-10T03:52:20.032Z","avatar_url":"https://github.com/perkss.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"## About\n\nA selection of examples using Kafka and Kafka Streams with Clojure. I wanted to learn the real Kafka API via Clojure not via a Clojure wrapper so here are examples of using the raw API, which is clean and means you do not need to wait for Clojure wrapper libraries to upgrade Kafka. \n\nA guide to the examples can be find in this post [here](https://perkss.github.io/#/clojure/KafkaClojure#text-body).\n\n## Prerequisites\nExpected to be able to set up Zookeeper and Kafka Broker to run the examples. Or use the Confluent platform. \nThe Kafka Stream Examples all have relevant tests rather than running a Jar. Please check individual module README to check how to run.\n\n## Unit Test\nCheck out the unit tests for each file to see it in action without the need for running with Zookeeper and Kafka.\n\n## Examples\n* Kafka Producer and Consumer in Clojure using Java interop.\n* Search a Topic and return records that match a Key from beginning to current point in time.\n* Kafka Streaming example to upper case of strings.\n* Kafka Streaming testing example.\n* Kafka Streaming join with two Streams (Left Join, Inner Join, Outer Join)\n* Kafka Streaming join with KTABLE.\n* Avro Kafka deserialization and serialization.\n* KStreamGroup Aggregate \n* TopologyTestDriver\n* More examples to come regularly watch this space ...\n\n## Integration Testcontainers\nTestcontainers provide the ability to test applications that have docker interactions simply by starting up your containers \nin our examples Kafka we can then start our applications up against this container and run the tests in a fully integrated \ntest with Kafka. Awesome!\n\n## Kafka and Zookeeper\n\nWe have provided our own docker-compose file that will start Kafka and Zookeeper via localhost, so other containers or local apps can access the broker.\n\nStart this using the command `docker-compose up -d`\n\n```yaml\nversion: '3'\nservices:\n  zookeeper:\n    image: confluentinc/cp-zookeeper:5.4.1\n    hostname: zookeeper\n    container_name: zookeeper\n    ports:\n      - \"2181:2181\"\n    environment:\n      ZOOKEEPER_CLIENT_PORT: 2181\n      ZOOKEEPER_TICK_TIME: 2000\n  broker:\n    image: confluentinc/cp-enterprise-kafka:5.4.1\n    hostname: broker\n    container_name: broker\n    depends_on:\n      - zookeeper\n    ports:\n      - \"29092:29092\"\n      - \"9092:9092\"\n    environment:\n      KAFKA_BROKER_ID: 1\n      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'\n      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT\n      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092\n      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1\n      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0\n```\n\n## Contributions\n\n* Please get involved and get recognition. If you want to add more examples then raise a PR. Or raise issues for ideas on examples that would be beneficial.  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperkss%2Fclojure-kafka-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fperkss%2Fclojure-kafka-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperkss%2Fclojure-kafka-examples/lists"}