{"id":19643229,"url":"https://github.com/goethite/lucygw","last_synced_at":"2025-04-28T13:30:42.746Z","repository":{"id":50189659,"uuid":"189885641","full_name":"goethite/lucygw","owner":"goethite","description":"Experiments in Microservices Loose Coupling with Go, Kafka and Python","archived":true,"fork":false,"pushed_at":"2021-06-01T23:53:35.000Z","size":70,"stargazers_count":1,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-26T23:26:56.453Z","etag":null,"topics":["ansible","golang","kafka","kubeless","kubernetes","loose-coupling","microservices","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/goethite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-02T19:18:40.000Z","updated_at":"2024-10-02T14:11:21.000Z","dependencies_parsed_at":"2022-09-24T07:11:45.769Z","dependency_job_id":null,"html_url":"https://github.com/goethite/lucygw","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Flucygw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Flucygw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Flucygw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Flucygw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/goethite","download_url":"https://codeload.github.com/goethite/lucygw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251319747,"owners_count":21570450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","golang","kafka","kubeless","kubernetes","loose-coupling","microservices","python"],"created_at":"2024-11-11T14:19:42.732Z","updated_at":"2025-04-28T13:30:42.701Z","avatar_url":"https://github.com/goethite.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# lucygw - Experiments in Microservices Loose Coupling with Go, Kafka and Python\n\nProvide a generic ingress api routing to Kafka topic queues for backend apps\nto consume (e.g. using kubeless functions or dedicated services) as loosely\ncouple apis.\n\nThis project is a playground for loose-coupling microservice apis. Contributions\nare welcome - so feel free to raise Issues/Discussions/PRs.\n\n## Setup Environment\nMy current setup is a MicroK8s server running the\n[incubator/kafka](https://github.com/helm/charts/tree/master/incubator/kafka)\nhelm chart (with a few minor tweaks (for NodePort access to the brokers) -\nwhich I will post soon on github) and Vagrant running lucygw:\n\n```bash\nvagrant up\nvagrant ssh\ncd lucygw\ngo run main.go\n```\n\n## Configure\nSee [config.yaml](./config.yaml) for exposed api paths and related kafka queue topics.\n\n## API Discovery\n```bash\ncurl -sS http://127.0.0.1:3303 | jq\n{\n  \"services\": [\n    {\n      \"handler\": \"Kafka\",\n      \"name\": \"ping\",\n      \"path\": \"/ping\",\n      \"request_topic\": \"ping\",\n      \"response_topic\": \"ping\"\n    },\n    {\n      \"handler\": \"Kafka\",\n      \"name\": \"automation_v1\",\n      \"path\": \"/automation/v1\",\n      \"request_topic\": \"automation_v1_request\",\n      \"response_topic\": \"automation_v1_response\"\n    }\n  ],\n  \"site\": {\n    \"_links\": {\n      \"curies\": [\n        {\n          \"href\": \"https://github.example.com/mysite/docs.md\",\n          \"name\": \"Site API\"\n        }\n      ],\n      \"self\": {\n        \"href\": \"/\"\n      }\n    },\n    \"description\": \"lucygw - Loose Coupling API Proxy\",\n    \"title\": \"lucygw - Loose Coupling API Proxy\"\n  }\n}\n```\n\n## Feed requests to a Kafka Queue\n(e.g. to be picked up by Kubeless function)\n\n### POST a Request\n```bash\ncurl -sS http://127.0.0.1:3303/ping \\\n  -X POST \\\n  -H \"Content-type: application/json\" \\\n  --data '{\"foo\":\"bar\"}'\n```\n\n```bash\ncurl -sS http://127.0.0.1:3303/automation/v1 \\\n  -X POST \\\n  -H \"Content-type: application/json\" \\\n  --data '{\"foo\":\"bar\"}'\n```\n\nAnsible ping:\n```bash\ncurl -sS http://127.0.0.1:3303/automation/v1/ping \\\n  -X POST \\\n  -H \"Content-type: application/json\" \\\n  --data '{}' \\\n  | jq .data.log -r\n```\n\n```\n[WARNING]: Unable to parse /etc/ansible/hosts as an inventory source\n[WARNING]: No inventory was parsed, only implicit localhost is available\n[WARNING]: provided hosts list is empty, only localhost is available. Note\nthat the implicit localhost does not match 'all'\n127.0.0.1 | SUCCESS =\u003e {\n   \"changed\": false,\n   \"ping\": \"pong\"\n}\n```\n\nRun playbook\n```bash\ncurl -sS \\\n  'http://127.0.0.1:3303/automation/v1/play?group=goethite%2fgostint-ansible%3a2.7.5\u0026name=dump.yml' \\\n  -X POST \\\n  -H \"Content-type: application/json\" \\\n  --data '{}' | jq .data.log -r\n```\n\n```\nPLAY [all] *********************************************************************\n\nTASK [Gathering Facts] *********************************************************\nok: [127.0.0.1]\n\nTASK [debug] *******************************************************************\nok: [127.0.0.1] =\u003e {\n    \"ansible_password\": \"VARIABLE IS NOT DEFINED!\"\n}\n\nTASK [debug] *******************************************************************\nok: [127.0.0.1] =\u003e {\n    \"ansible_connection\": \"local\"\n}\n\nTASK [shell] *******************************************************************\nchanged: [127.0.0.1]\n\nPLAY RECAP *********************************************************************\n127.0.0.1                  : ok=4    changed=1    unreachable=0    failed=0\n```\n\n### POST a request with Asynchronous Callback\nNOTE: This PoC implementation is currently for demo purposes only.\n\nStart a test listener for the callback:\n```bash\n(ansible-k8s) \u003e FLASK_APP=test/callback-listener.py flask run -h 0.0.0.0\n```\n(defaults to port 5000)\n\nPost a request with the `X-Lucygw-Cb` header:\n```bash\n$ curl -sS \\\n  'http://127.0.0.1:3303/automation/v1/play?group=goethite\u0026name=dump.yml' \\\n  -X POST \\\n  -H \"Content-type: application/json\" \\\n  --data '{}' \\\n  -H 'X-Lucygw-Cb: http://172.17.0.1:5000/callback'\n```\n(the IP 172.17.0.1 is the default docker ip of the parent host in the lucygw\nservice container )\n\nreturns immediately with:\n```json\n{\"callback_url\":\"http://172.17.0.1:5000/callback\",\"event_uuid\":\"ee85d16d-96ae-11e9-ba39-0242ac110002\"}\n```\n\nand later `callback-listener.py` reports the completed job:\n```\nreq: {\n  \"data\": {\n    \"log\": \"\\nPLAY [all] *********************************************************************\\n\\nTASK [Gathering Facts] *********************************************************\\nok: [127.0.0.1]\\n\\nTASK [debug] *******************************************************************\\nok: [127.0.0.1] =\u003e {\\n    \\\"ansible_password\\\": \\\"VARIABLE IS NOT DEFINED!\\\"\\n}\\n\\nTASK [debug] *******************************************************************\\nok: [127.0.0.1] =\u003e {\\n    \\\"ansible_connection\\\": \\\"local\\\"\\n}\\n\\nTASK [shell] *******************************************************************\\nchanged: [127.0.0.1]\\n\\nPLAY RECAP *********************************************************************\\n127.0.0.1                  : ok=4    changed=1    unreachable=0    failed=0   \\n\\n\",\n    \"status\": {\n      \"active\": null,\n      \"failed\": null,\n      \"succeeded\": 1\n    }\n  },\n  \"event_uuid\": \"ee85d16d-96ae-11e9-ba39-0242ac110002\"\n}\n172.17.0.2 - - [24/Jun/2019 19:36:17] \"POST /callback HTTP/1.1\" 200 -\n```\n\n## Kubeless consumer examples\n\nSee:\n* [ansible-coupler.py](kubeless/ansible-coupler.py) run ansible in docker\n  containers as k8s batch jobs.\n* [gostint-coupler.py](kubeless/gostint-coupler.py) (in progress) PoC stub for\n  gostint api.\n\n## Consumer service examples\n\nSee:\n* [ansible-k8s](services/ansible-k8s) like ansible-coupler above, but as a\nkafka consumer service.\n* [callback-listener.py](services/ansible-k8s/test/callback-listener.py) - a\ndemo callback listener.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoethite%2Flucygw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoethite%2Flucygw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoethite%2Flucygw/lists"}