{"id":22574869,"url":"https://github.com/thoth-station/messaging","last_synced_at":"2025-09-08T16:42:25.419Z","repository":{"id":35522254,"uuid":"210524914","full_name":"thoth-station/messaging","owner":"thoth-station","description":"This is Thoth messaging, I will send my knowledge to the Universe!","archived":false,"fork":false,"pushed_at":"2023-10-18T00:43:55.000Z","size":966,"stargazers_count":3,"open_issues_count":3,"forks_count":10,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-24T13:51:25.198Z","etag":null,"topics":["aicoe","hacktoberfest","thoth"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thoth-station.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-24T06:09:44.000Z","updated_at":"2022-05-12T09:03:14.000Z","dependencies_parsed_at":"2023-01-15T22:40:45.966Z","dependency_job_id":null,"html_url":"https://github.com/thoth-station/messaging","commit_stats":null,"previous_names":[],"tags_count":54,"template":false,"template_full_name":"thoth-station/template-project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fmessaging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fmessaging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fmessaging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fmessaging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thoth-station","download_url":"https://codeload.github.com/thoth-station/messaging/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248251020,"owners_count":21072685,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aicoe","hacktoberfest","thoth"],"created_at":"2024-12-08T03:08:15.527Z","updated_at":"2025-04-10T16:11:25.458Z","avatar_url":"https://github.com/thoth-station.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Thoth Messaging\n---------------\n\n.. image:: https://img.shields.io/github/v/tag/thoth-station/messaging?style=plastic\n  :target: https://github.com/thoth-station/messaging/releases\n  :alt: GitHub tag (latest by date)\n\n.. image:: https://img.shields.io/pypi/v/thoth-messaging?style=plastic\n  :target: https://pypi.org/project/thoth-messaging\n  :alt: PyPI - Module Version\n\n.. image:: https://img.shields.io/pypi/l/thoth-messaging?style=plastic\n  :target: https://pypi.org/project/thoth-messaging\n  :alt: PyPI - License\n\n.. image:: https://img.shields.io/pypi/dm/thoth-messaging?style=plastic\n  :target: https://pypi.org/project/thoth-messaging\n  :alt: PyPI - Downloads\n\n.. image:: https://quay.io/repository/thoth-station/messaging/status\n  :target: https://quay.io/repository/thoth-station/messaging?tab=tags\n  :alt: Quay - Build\n\nThis provides a library called `thoth-messaging\n\u003chttps://pypi.org/project/thoth-messaging\u003e`_ used in project `Thoth\n\u003chttps://thoth-station.ninja\u003e`_.  It is a basic module to encapsule all messaging (here it is Kafka via Faust) primitives.\n\nType Hinting With MyPy\n######################\nThis module uses pydantic for type hinting and enforcing a regular schema in messaging.  If you are using mypy to check\nyour code please add the following to your mypy configuration file:\n\n.. code-block:: toml\n\n  [mypy]\n  plugins = pydantic.mypy\n\nIf you are creating an instance of a pydantic model in your own module you should directly use the `MessageContents`\nclass within the associate message file. Using ``foo_bar_message.model`` will only type hint for `BaseMessageContents`.\n\nDevelopment and Testing\n#######################\nFor development and testing it is very useful to have a local instance of Kafka running on your machine\n\nWe provide a ``docker-compose`` file to get you up and running quickly with a basic Kafka server; this file is based on\n`Single Zookeeper/Multiple Kafka \u003chttps://github.com/simplesteph/kafka-stack-docker-compose#single-zookeeper--multiple-kafka\u003e`__.\n\nIn order to start Zookeeper as well as the Kafka Servers simply run `$ podman-compose up` or `$ docker-compose up`,\nchoose the appropriate option based on the system which you are using.\n\nOnce you have Kafka up and running you should be ready to begin coding your own messaging producers and consumers.  The\ninterface between `Kafka` and `Python` is handled by a library called `Confluent Kafka \u003chttps://docs.confluent.io/current/clients/python.html\u003e`__.\nFaust's documentation will be extremely helpful to you when you are developing your own applications. If you would like\nexamples of producers and consumers from Team Thoth, look at the following two repositories,\n`investigator \u003chttps://github.com/thoth-station/investigator\u003e`__ and `package-update \u003chttps://github.com/thoth-station/package-update-job\u003e`__.\n\nYou may find it useful to use console producers and consumers while testing your, to create one simply attach a bash shell\nto one of your Kafka Servers by running: `$ podman exec -it messaging_kafka1_1 bash`, your container names should be\nthe same as given here, if not, run `$ podman ps` and choose the correct container.  These containers have all Kafka\nbinaries in appropriate places so you can simply run `$ kafka-console-consumer`, `$ kafka-console-producer`, or any other\nkafka command that you may find useful.\n\n*example:*\n\n.. code-block:: console\n\n  kafka-console-consumer --bootstrap-server localhost:9092 --topic test --from-beginning\n\nYou can test sending a message via the CLI using a file as -\n\n*example:*\n\n.. code-block:: console\n\n  pipenv shell\n  python cli.py --message-file messages_to_be_sent.json\n\n*Note*\nData is not persistent. Once pods are deleted so is the data associated with them.\n\nPitfalls\n########\n\nSome schemas in this library are defined as `Dict[str, Any]`.  This usually does not accurately reflect the actual\nschema required. These schemas can be purposefully vague because they are defined elsewhere and importing them from\nthe requisite libraries couples `thoth-messaging` version too closely to other components in the best case, and\nintroduces circular dependencies in the worst case.  In short, `thoth-messaging` is naive and can only enforce schemas\nwhich it has defined.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Fmessaging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthoth-station%2Fmessaging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Fmessaging/lists"}