{"id":14235670,"url":"https://github.com/mmolimar/kukulcan","last_synced_at":"2025-08-11T00:32:18.842Z","repository":{"id":54137206,"uuid":"264333067","full_name":"mmolimar/kukulcan","owner":"mmolimar","description":"A REPL for Apache Kafka","archived":false,"fork":false,"pushed_at":"2021-03-08T15:06:58.000Z","size":2878,"stargazers_count":115,"open_issues_count":0,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-05-15T13:07:15.790Z","etag":null,"topics":["ammonite","apache-kafka","java","kafka","kafka-admin","kafka-connect","kafka-consumer","kafka-producer","kafka-repl","kafka-shell","kafka-streams","python","repl","scala","shell"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mmolimar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-16T01:13:14.000Z","updated_at":"2024-03-21T17:31:23.000Z","dependencies_parsed_at":"2022-08-13T07:20:33.300Z","dependency_job_id":null,"html_url":"https://github.com/mmolimar/kukulcan","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmolimar%2Fkukulcan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmolimar%2Fkukulcan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmolimar%2Fkukulcan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmolimar%2Fkukulcan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mmolimar","download_url":"https://codeload.github.com/mmolimar/kukulcan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":216214763,"owners_count":15992198,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ammonite","apache-kafka","java","kafka","kafka-admin","kafka-connect","kafka-consumer","kafka-producer","kafka-repl","kafka-shell","kafka-streams","python","repl","scala","shell"],"created_at":"2024-08-20T21:02:15.876Z","updated_at":"2024-08-20T21:02:37.426Z","avatar_url":"https://github.com/mmolimar.png","language":"Scala","readme":"# kukulcan [![Build Status](https://circleci.com/gh/mmolimar/kukulcan.svg?style=shield)](https://circleci.com/gh/mmolimar/kukulcan)\n\n*K'uk'ulkan* (\"Feathered Serpent\") is the name of a deity which was worshipped by the Yucatec maya people. You can\nread a lot more in books or on the Internet about it and will see that, in someways, is related to the wind and water.\n\nBeyond the origin of this name I reused to name this project, Kukulcan provides an API and different\nsort of [REPLs](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) to interact with your streams \nor administer your [Apache Kafka](https://kafka.apache.org) deployment.\n\nIt supports POSIX and Windows OS and Scala, Java and Python programming languages.\n\n![](/docs/img/kukulcan.png)\n\n## Motivation\n\nAs Kafka users (developers, devops, etc.), we have to interact with it in different ways. Maybe using the\ncommand-line tools it provides, an external tool to look for a specific info, an IDE to develop a snippet\nin order to test something... but what if you could do all those things in the same interface with the\nflexibility of a REPL to do exactly what you need?\n\nThis is the aim of this project: enabling a simple interface to interact with your Kafka cluster taking advantage\nof its rich API plus some additional utils included and make your interaction better.\n\n## Getting started\n\n### Requirements\n\nBefore starting, you'll need to have installed JDK 11 and [SBT](https://www.scala-sbt.org/). Additionally,\nif you want to use the PyKukulcan REPL, you'll also need Python (tested 2.7, 3.4, 3.5, 3.6, and 3.7 versions)\nand [pip](https://pypi.org/project/pip) installed.\n\n### Building from source\n\nJust clone the ``kukulcan`` repo and *\"kukulcan-it\"*:\n\n``git clone https://github.com/mmolimar/kukulcan.git \u0026\u0026 cd kukulcan``\n\n``sbt kukulcan``\n\n### Configuration\n\nIn the ``config`` directory, you'll find some files with the configurations for each of the APIs Kukulcan provides.\nAll the possible configs for each API are in these files with their description so, if you need to set some\nspecific configs for your environment, you should set them there before starting.\n\nAlso, there is an important environment variable named **``KUKULCAN_HOME``**. If not set, its default value will be\nthe root folder of the cloned project. When starting a Kukulcan REPL, it will look for this environment variable\nand read the properties files contained in its ``config`` subdirectory.\n\n## Modules\n\nThe project contains three subprojects or modules described below but, if you generate the Scala docs executing\n``sbt doc`` in the command line, you'll be able to see a more detailed description.\n\n### kukulcan-api\n\nContains all the Scala and Java classes to interact with Kafka and extending the Kafka API to enable new\nfunctionalities. This API contains:\n\n* **KAdmin**: grouped utils for administrative operations for topics, configs, ACLs and metrics.\n* **KConnect**: methods to execute requests against Kafka Connect REST API.\n* **KConsumer** and **KProducer**: Kafka consumer/producer with some extra features.\n* **KStreams**: extends Kafka Streams to see how your topology is (printing it in a graph).\n* **KSchemaRegistry**: to interact and manage schemas in [Confluent Schema Registry](https://github.com/confluentinc/schema-registry).\n* **KKsql**: client for querying [Confluent KSQL](https://github.com/confluentinc/ksql) server and integrated with the KSQL-CLI.\n\n### kukulcan-repl\n\nEnables two sort of entry points for the REPLs: one based on the [Scala REPL](https://docs.scala-lang.org/overviews/repl/overview.html)\nand the other based on [JShell](https://docs.oracle.com/javase/9/jshell).\n\nAdditionally, it includes the logic to read and ``reload`` your configurations stored the ``config`` directory\ntransparently.\n\n### pykukulcan\n\nThis module includes the needed bindings to use Kukulcan with Python via [py4j](https://www.py4j.org).\nBy now, the only bindings available are for the REPL (in Java).\n\n## Running the REPL\n\nKukulcan takes advantage of multiple functionalities we already have in Java, Scala and other tools to build a\ncustom REPL. Actually, the project enables four type of REPLs with Ammonite, Scala, JShell and Python.\n\nAfter building the source, you'll be able to start any of the following REPL options. Here are a few examples\nyou can do with Kukulcan:\n\n- Graphical representation of a topology in Kafka Streams:\n\n  ![](/docs/img/kstreams.png)\n\n- Kafka Connect interaction:\n\n  ![](/docs/img/kconnect.png)\n\n- Managing schemas in Schema Registry:\n\n  ![](/docs/img/kschema-registry.png)\n\n- Interacting with KSQL server and using the KSQL-CLI:\n\n  ![](/docs/img/kksql.png)\n\n\u003e **NOTE**: The REPLs have already the Kukulcan imports: ``com.github.mmolimar.kukulcan`` in case of the\nScala and Ammonite REPLs and ``com.github.mmolimar.kukulcan.Kukulcan`` in case of the JShell REPL. So you just\nhave to start typing ``kukulcan.\u003coption\u003e.`` or ``Kukulcan.\u003coption\u003e.`` respectively.\n\n\n### Kukulcan Ammonite REPL\n\n[Ammonite](https://ammonite.io) is an improved Scala REPL with a lot of interesting features you can find very\nuseful to load scripts or even code in an easier way. Obviously, you must install it previously.\n\nIt already includes all dependencies, and the Kukulcan Scala API to interact with Kafka.\n\n\u003e **NOTE**: If you're going to use Kukulcan with Ammonite, you'll have to publish the project in local, executing\n  ``sbt publishLocal``.\n\nTo execute this REPL you just have to type:\n\n``./bin/kukulcan-amm``\n\nI do recommend using this REPL even though the other ones are fine as well. The only drawback here is that\nAmmonite-REPL does not support Windows systems.\n\n### Kukulcan Scala REPL\n\nA Scala REPL including all dependencies and the Kukulcan Scala API to interact with Kafka.\n\nTo execute this REPL you just have to type:\n\n``./bin/kukulcan``\n\nFor Windows OS:\n\n``bin\\kukulcan.cmd``\n\n### Kukulcan JShell REPL\n\nAn enriched Java JShell to interact with Kafka including all dependencies and imports from the Kukulcan Java API.\n\nTo execute this REPL you just have to type:\n\n``./bin/jkukulcan``\n\nFor Windows OS:\n\n``bin\\jkukulcan.cmd``\n\n### PyKukulcan REPL\n\nA Python shell including the needed bindings with Kukulcan Scala API via [Py4J](https://www.py4j.org).\n\n\u003e **NOTE**: If you're going to use the PyKukulcan REPL, you must have Python and pip installed. Then, install\n``pykukulcan`` in this way: ``pip install python/ -r python/requirements.txt``\n\nTo execute this REPL you just have to type:\n\n``./bin/pykukulcan``\n\nFor Windows OS:\n\n``bin\\pykukulcan.cmd``\n\n## TODOs\n\n- [ ] Tools in the KAdmin API.\n- [ ] Integration with REST Proxy.\n- [ ] API for Python.\n\n## Contribute\n\n- Source Code: https://github.com/mmolimar/kukulcan\n- Issue Tracker: https://github.com/mmolimar/kukulcan/issues\n\n## License\n\nReleased under the Apache License, version 2.0.\n","funding_links":[],"categories":["Scala"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmolimar%2Fkukulcan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmmolimar%2Fkukulcan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmolimar%2Fkukulcan/lists"}