{"id":22059216,"url":"https://github.com/pegasystems/charlatan","last_synced_at":"2025-05-12T19:43:41.320Z","repository":{"id":94490115,"uuid":"108577104","full_name":"pegasystems/charlatan","owner":"pegasystems","description":"Charlatan is a library created for Apache Kafka in order to substitute real Zookeeper with Zookeeper imitation. Charlatan was tested only with Apache Kafka version 0.11.0.1.","archived":false,"fork":false,"pushed_at":"2024-03-29T17:54:34.000Z","size":532,"stargazers_count":5,"open_issues_count":3,"forks_count":1,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-01T02:49:34.817Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pegasystems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-27T17:49:17.000Z","updated_at":"2023-09-08T13:50:32.000Z","dependencies_parsed_at":"2024-11-30T17:28:22.369Z","dependency_job_id":"7b6b4cd2-8363-4f19-b434-b9511378da49","html_url":"https://github.com/pegasystems/charlatan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasystems%2Fcharlatan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasystems%2Fcharlatan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasystems%2Fcharlatan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasystems%2Fcharlatan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pegasystems","download_url":"https://codeload.github.com/pegasystems/charlatan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253809695,"owners_count":21967775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-30T17:27:34.562Z","updated_at":"2025-05-12T19:43:41.297Z","avatar_url":"https://github.com/pegasystems.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Charlatan\n\nCharlatan is a library created for Apache Kafka in order to substitute real Zookeeper with Zookeeper imitation.\nCharlatan was tested only with Apache Kafka version 0.11.0.1. \n\n## Motivation\nKafka is a great product, but it requires running Zookeeper cluster. This means that to manage \nKafka cluster we also need to manage Zookeeper cluster.  \nThis complicates Kafka usage especially in case when we already have a service similar to Zookeeper. \nFor example etcd, consul, etc. or your own highly available service.\n\nCharlatan provides two options to use Kafka without Zookeeper:\n1. Instead of Zookeeper Kafka communicates with the different server using Zookeeper messaging \nprotocol. This option was implemented in [***Charlatan-server***](#charlatan-server) sub-project.\n2. Instead of Zookeeper driver Kafka uses Charlatan adapter that stores all Zookeeper related\ninformation in the relational database. In this case Kafka doesn't need other running service\nand database can play the role of the highly available service. This option was implemented in \n[***Charlatan-adapter***](#charlatan-adapter) sub-project.\n\n## Sub-projects \nCharlatan contains from few sub-projects\n\nSub-project              |Description\n-------------------------|------------ \n[***Charlatan-common***](#charlatan-common)    |Watch management, session management, common dao interfaces.\n[***Charlatan-server***](#charlatan-server)    |Netty based server implementation, Zookeeper messaging protocol implementation.\n[***Charlatan-adapter***](#charlatan-adapter)  |Fake Zookeeper driver, it can be used as a Kafka dependency instead of Zookeeper.jar.\n[***Charlatan-sqlite***](#charlatan-dao-sqlite)|Simple implementation of Charlatan DAO interfaces using sqlite. It is not  suitable for the production usage and it was build mainly for the demonstration and testing purposes.\n[***Charlatan-remote***](#charlatan-dao-remote)|Simple netty based implementation of Charlatan DAO interfaces that sends DAO requests to the remote server.\n\n### Charlatan-common\u003ca name=\"charlatan-common\"\u003e\u003c/a\u003e\nIn order to replace Zookeeper Charlatan library has to handle all the Zookeeper features. \n\nApache Kafka uses Zookeeper mainly to:\n* Keep specific Kafka related information: which topics exist, how many partitions topic has, \ntopic replication information, etc. This kind of information is stored in a node tree \n(check Zookeeper documentation for details).\n* Track status of Kafka brokers: who is the cluster leader, which broker is alive, \nwhich broker is leaving/joining.\n* Support watches. Zookeeper has concept of watches: a watch event is one-time trigger, sent to the \nclient that set the watch, which occurs when the data for which the watch was set changes. \nIn practice this means that if Kafka broker wants to receive an update in case specific node has changed\nor children node changes, it sets a watch for node in question.\n                  \nAll these features are implemented in the ***Charlatan-common***. \n* In order to keep Kafka related information Charlatan supports nodes CRUD operations by providing\nNodeDao interface that needs to be implemented for your database or other highly available service.\n* In order to track broker statuses  ***Charlatan-common*** provides its own implementation\n[SessionServiceImpl](charlatan-common/src/main/java/com/pega/charlatan/server/session/service/SessionServiceImpl.java) \nTo use custom implementation [SessionService](charlatan-common/src/main/java/com/pega/charlatan/server/session/service/SessionService.java) \ninterface must be implemented.\n* Watches support is implemented in [WatchServiceImpl](charlatan-common/src/main/java/com/pega/charlatan/watches/service/WatchServiceImpl.java).\nTo use custom implementation [WatchService](charlatan-common/src/main/java/com/pega/charlatan/watches/service/WatchService.java) \ninterface must be implemented.\n   \n### Charlatan-server\u003ca name=\"charlatan-server\"\u003e\u003c/a\u003e\nThis option implies that standard Kafka distribution is used and instead of real Zookeeper\nKafka is pointed to the Charlatan server. The schema could look like: \n![](images/CharlatanServer.png \"Charlatan server usage\")\nNote that as long as same database is used multiple Charlatan server can be running.\nOne or multiple Kafka instances can be connected to the Charlatan server. \nKafka broker isn't aware that it doesn't connect to a real Zookeeper. \nStandard Kafka setting ```Zookeeper.connect``` is used in order to point Kafka to Charlatan server.  \n\nIn this example we are starting Charlatan server with sqlite DAO implementations \nand default implementation of WatchService and SessionService.\n```java\nString host = \"localhost\";\nint port = 2181;\n\nCharlatanNettyServer server = new CharlatanServerBuilder()\n\t// Charlatan server host\n\t.setHost(host)\n\t// Charlatan server port\n\t.setPort(port)\n\t// Charlatan server id must be unique per Charlatan cluster\n\t.setId(\"server\"+port)\n\t.setWorkerCount(5)\n\t// Node DAO implementation\n\t.setNodeDao(new NodeDaoSqlite())\n\t// Watch service implementation\n\t.setWatchService(new WatchServiceImpl(NodeUpdateDaoSqlite(), new NamedThreadFactory(\"charlatan-watch-service\")))\n\t// Session service implementation\n\t.setSessionService(new SessionServiceImpl(new SessionDaoSqlite()))\n\t.setThreadFactory(new NamedThreadFactory(\"charlatan-service\"))\n\t.build();\n\nserver.start();\n```\n\n### Charlatan-adapter\u003ca name=\"charlatan-adapter\"\u003e\u003c/a\u003e\nThis option implies that in standard Kafka distribution Zookeeper.jar library will be replaced\nby the charlatan-adapter.jar and charlatan dao implementation will be provided. \nWhat do we need to make it work?\n1. Remove from Kafka distribution Zookeeper jar located in libs/Zookeeper-3.4.10.jar.\n2. Place charlatan-adapter.jar in libs/ folder.\n3. Place jar that contains your Charlatan DAO implementations in libs/ folder (charlatan-dao-sqlite.jar can be \nused for testing purposes).\n\u003cbr\u003eIn runtime ***Charlatan-adapter*** uses DAO interface implementations found in the classpath. \n***Charlatan-adapter*** fails if no DAO implementation was found or multiple DAO implementations were found.\n\nIn case Charlatan DAO interfaces were implemented for relational database the schema would look like:\n\n![](images/CharlatanAdapterDB.png \"Charlatan adapter with relational database\")\n\nIn a more general approach Charlatan DAO interfaces implementation is a client to your \nhighly available service.\n\n![](images/CharlatanAdapter.png \"Charlatan adapter general implementation\")\n\n### Charlatan-dao-sqlite\u003ca name=\"charlatan-dao-sqlite\"\u003e\u003c/a\u003e\nThis project was created for testing purposes and contains basic Charlatan DAO implementations for\nsqlite database.\n### Charlatan-dao-remote\u003ca name=\"charlatan-dao-remote\"\u003e\u003c/a\u003e\nThis project was created for demonstration purposes only and contains Charlatan DAO implementation \nthat acts as a client to the remote REST-ful service. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpegasystems%2Fcharlatan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpegasystems%2Fcharlatan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpegasystems%2Fcharlatan/lists"}