{"id":17874475,"url":"https://github.com/marcosschroh/cookiecutter-faust","last_synced_at":"2025-03-21T22:31:55.049Z","repository":{"id":36703731,"uuid":"182124926","full_name":"marcosschroh/cookiecutter-faust","owner":"marcosschroh","description":"A Cookiecutter template for creating Faust projects quickly.","archived":false,"fork":false,"pushed_at":"2022-12-01T12:29:09.000Z","size":15783,"stargazers_count":70,"open_issues_count":4,"forks_count":17,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-18T05:43:59.947Z","etag":null,"topics":["coookiecutter","faust","python","python3","streaming"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcosschroh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":null},"created_at":"2019-04-18T16:51:26.000Z","updated_at":"2024-03-07T22:02:22.000Z","dependencies_parsed_at":"2023-01-17T04:15:40.515Z","dependency_job_id":null,"html_url":"https://github.com/marcosschroh/cookiecutter-faust","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcosschroh%2Fcookiecutter-faust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcosschroh%2Fcookiecutter-faust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcosschroh%2Fcookiecutter-faust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcosschroh%2Fcookiecutter-faust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcosschroh","download_url":"https://codeload.github.com/marcosschroh/cookiecutter-faust/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244880353,"owners_count":20525507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coookiecutter","faust","python","python3","streaming"],"created_at":"2024-10-28T11:09:01.491Z","updated_at":"2025-03-21T22:31:53.953Z","avatar_url":"https://github.com/marcosschroh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cookiecutter Faust\n\n[![Build Status](https://travis-ci.org/marcosschroh/cookiecutter-faust.svg?branch=master)](https://travis-ci.org/marcosschroh/cookiecutter-faust)\n[![GitHub license](https://img.shields.io/github/license/marcosschroh/cookiecutter-faust.svg)](https://github.com/marcosschroh/cookiecutter-faust/blob/feature/add-license-and-remove-network-after-clean/LICENSE)\n\n## Table of Contents\n\n  - [Features](#features)\n  - [Usage](#usage)\n  - [Useful Commands](#useful-commands)\n  - [Settings](#settings)\n  - [Docker and Docker Compose](#docker-and-docker-compose)\n  - [Development](#development)\n\n### Features\n\n- Python 3.7+\n- Docker-compose for local development that builds a `kafka` cluster\n- Useful commands included in Makefile\n- Kubernetes manifests included\n- CI generated (Gitlab/Travis)\n- Settings that include logging, minimal Faust and SSL configuration\n- Faust agent example with a test case\n- Option to use `rocksDB` as a Faust data store\n- Uses the active [faust-streaming](https://github.com/faust-streaming/faust) fork of the original [faust](https://github.com/robinhood/faust)\n\n### Usage\n\nLet's pretend you want to create a Faust project called `super faust`...\n\nFirst, install `Cookiecutter`:\n\n```bash\npip install \"cookiecutter\"\n```\n\nNow run it against this repo:\n\n```bash\ncookiecutter https://github.com/marcosschroh/cookiecutter-faust\n```\n\nYou'll be prompted for some values. Provide them, then a Faust project will be created for you\nbased on the convention of medium/large project mentioned above\n\nAnswer the prompts with your own desired options. For example:\n\n```bash\nproject_name [My Awesome Faust Project]: super faust\nproject_slug [super_faust]:\ndescription [My Awesome Faust Project!]:\nlong_description [My Awesome Faust Project!]:\nauthor_name [Marcos Schroh]:\nauthor_email [marcos-schroh@gmail.com]:\nversion [0.1.0]:\nSelect open_source_license:\n1 - MIT\n2 - BSD\n3 - GPLv3\n4 - Apache Software License 2.0\n5 - Not open source\nChoose from 1, 2, 3, 4, 5 (1, 2, 3, 4, 5) [1]:\nworker_port [6066]:\nSelect log_level:\n1 - CRITICAL\n2 - ERROR\n3 - WARNING\n4 - INFO\n5 - DEBUG\n6 - NOTSET\nChoose from 1, 2, 3, 4, 5, 6 (1, 2, 3, 4, 5, 6) [1]: 4\ninclude_schema_registry [y]:\ninclude_rocksdb [y]:\nSelect ci_provider:\n1 - travis\n2 - gitlab\n3 - none\nChoose from 1, 2 (1, 2) [1]: 2\n```\n\nEnter the project and take a look around:\n\n```bash\ncd super_faust/\nls\n\nCONTRIBUTORS.txt  Dockerfile  LICENSE  Makefile  README.md  docker-compose.yaml setup.cfg  pyproject.py  super_faust\n```\n\nNow time to run it. In a terminal located at the project root directory folder start the `kafka cluster`:\n\n```bash\nmake kafka-cluster\n```\n\nIn another terminal, run the `Faust application`:\n\n```bash\nmake install\nmake start-app\n```\n\nNow you will see the project starting. If you chose to have the [page view tutorial]((https://faust.readthedocs.io/en/latest/playbooks/pageviews.html)), you can send events to the `page_views` topic. In a different terminal also at the project root directory execute:\n\n```bash\nmake send-page-view-event payload='{\"id\": \"foo\", \"user\": \"bar\"}'\n```\n\nand in the first terminal you will see the logs of the event received.\n\n![Running Project](docs/img/cookiecutter-faust.gif)\n\nOptional, create a git repo and push it there:\n\n```bash\ngit init\ngit add .\ngit commit -m \"first awesome commit\"\ngit remote add origin git@github.com:marcosschroh/super-faust.git\ngit push -u origin master\n```\n\nNow take a look at your repo. Don't forget to carefully look at the generated README. Awesome, right?\n\n### Useful Commands\n\n|Command|Description| Default values|Example|\n|-------|------------|--------------|-------|\n| `make kafka-cluster`      | Run the kafka cluster          |      ---        | |\n| `make stop-kafka-cluster`      |  Stop kafka cluster, clean containers and network         |     ---         | |\n| `make install`      |  Install local requirements         |     ---         | |\n| `make start-app`      |  Start Faust application        |     ---         | |\n| `make bash service={the-service}`      |  Access to container         |   service=kafka           | `make bash` |\n| `make list-topics`      |      List topics     |    ---          | |\n| `make create-topic replication-factor={replication-factor} --partitions={number-of-partitions topic-name={your-topic-name}`      |  Create topic         |  replication-factor=1 partitions=1.           |  `make create-topic topic-name=test-topic`|\n| `make send-page-view-event payload='{a payload}'`| Send event to a page view application | -- | `make send-page-view-event payload='{\"id\": \"foo\", \"user\": \"bar\"}'` |\n| `make list-agents`| List faust agents| --- | |\n\n### Settings\n\nSettings are created based on [local-settings](https://github.com/drgarcia1986/simple-settings) package.\n\nThe only settings required if the `KAFKA SERVER` environment variable which by default is `KAFKA_BOOTSTRAP_SERVER`\n\n```python\nSIMPLE_SETTINGS = {\n    'OVERRIDE_BY_ENV': True,\n    'CONFIGURE_LOGGING': True,\n    'REQUIRED_SETTINGS': ('KAFKA_BOOTSTRAP_SERVER', 'STORE_URI'),\n}\n\n# The following variables can be ovirriden from ENV\nKAFKA_BOOTSTRAP_SERVER = \"kafka://localhost:29092\"\n\nTOPIC_ALLOW_DECLARE = True\nTOPIC_DISABLE_LEADER = False\n\nSSL_ENABLED = False\nSSL_CONTEXT = None\n\nif SSL_ENABLED:\n    # file in pem format containing the client certificate, as well as any ca certificates\n    # needed to establish the certificate’s authenticity\n    KAFKA_SSL_CERT = None\n\n    # filename containing the client private key\n    KAFKA_SSL_KEY = None\n\n    # filename of ca file to use in certificate verification\n    KAFKA_SSL_CABUNDLE = None\n\n    # password for decrypting the client private key\n    SSL_KEY_PASSWORD = None\n\n    SSL_CONTEXT = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, cafile=KAFKA_SSL_CABUNDLE)\n\n    SSL_CONTEXT.load_cert_chain(KAFKA_SSL_CERT, keyfile=KAFKA_SSL_KEY, password=SSL_KEY_PASSWORD)\n```\n\nThe settings also include a basic logging and store configuration:\n\n```python\nSTORE_URI = \"rocksdb://\" # If rocksdb is  disabled is \"memory://\"\n\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n    'formatters': {\n        'default': {\n            'format': '%(asctime)s %(levelname)s %(name)s %(message)s',\n        },\n    },\n    'handlers': {\n        'console': {\n            'level': '{{cookiecutter.log_level}}',\n            'class': 'logging.StreamHandler',\n            'formatter': 'default',\n        },\n    },\n    'loggers': {\n        'page_views': {\n            'handlers': ['console'],\n            'level': '{{cookiecutter.log_level}}',\n        },\n    },\n}\n```\n\n### Docker and Docker Compose\n\nThe `Dockerfile` is based on  `python:3.7-slim`. The example is [here](https://github.com/marcosschroh/cookiecutter-faust/blob/master/%7B%7Bcookiecutter.project_slug%7D%7D/Dockerfile)\n\nThe `docker-compose.yaml` includes `zookeeper`, `kafka` and `schema-registry` (if selected for inclusion) based on `confluent-inc`.\nFor more information you can go to [confluentinc](https://docs.confluent.io/current/installation/docker/docs/index.html) and see the docker compose example [here](https://github.com/confluentinc/examples/blob/5.3.1-post/cp-all-in-one/docker-compose.yml)\n\nUseful `ENVIRONMENT` variables that you may change:\n\n|Variable| description  | default |\n|--------|--------------|---------|\n| WORKER_PORT | Worker port | Autogenerated by `cookiecutter`. |\n| KAFKA_BOOTSTRAP_SERVER | Kafka servers. | `kafka://localhost:29092` |\n| KAFKA_BOOTSTRAP_SERVER_NAME | Kafka server name| `kafka` |\n| KAFKA_BOOTSTRAP_SERVER_PORT | Kafka server port | `29092` |\n| SCHEMA_REGISTRY_SERVER | Schema registry server name | `schema-registry` |\n| SCHEMA_REGISTRY_SERVER_PORT | Schema registry server port | `8081` |\n| SCHEMA_REGISTRY_URL | Schema Registry Server url | `http://schema-registry:8081` |\n\n### Development\n\nRun tests:\n\n```bash\n./scripts/test\n```\n\nRun code linting\n\n```bash\n./scripts/lint\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcosschroh%2Fcookiecutter-faust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcosschroh%2Fcookiecutter-faust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcosschroh%2Fcookiecutter-faust/lists"}