{"id":31801347,"url":"https://github.com/egasimov/news-system","last_synced_at":"2026-05-05T18:39:54.797Z","repository":{"id":79220412,"uuid":"565099460","full_name":"egasimov/news-system","owner":"egasimov","description":"System for retrieving news from 3rd party APIs and asynchronously serving them to clients over web socket","archived":false,"fork":false,"pushed_at":"2022-11-20T16:12:12.000Z","size":525,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-10T23:59:50.577Z","etag":null,"topics":["docker","docker-compose","golang","memphis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/egasimov.png","metadata":{"files":{"readme":"README.md","changelog":"news-collector/.env.local","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-11-12T10:42:09.000Z","updated_at":"2024-05-20T14:45:18.000Z","dependencies_parsed_at":"2023-07-14T03:30:26.927Z","dependency_job_id":null,"html_url":"https://github.com/egasimov/news-system","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/egasimov/news-system","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/egasimov%2Fnews-system","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/egasimov%2Fnews-system/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/egasimov%2Fnews-system/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/egasimov%2Fnews-system/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/egasimov","download_url":"https://codeload.github.com/egasimov/news-system/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/egasimov%2Fnews-system/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32663429,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","golang","memphis"],"created_at":"2025-10-10T23:59:07.595Z","updated_at":"2026-05-05T18:39:54.792Z","avatar_url":"https://github.com/egasimov.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# news-system\n\nA system is composed of three modules:\n\n1. [news-collector](./news-collector)\n* Collecting news from some predefined sources\n(e.g [NewsAPI](https://documenter.getpostman.com/view/3479169/Szf7zncp?version=latest#intro), [TheGuardianNews](https://content.guardianapis.com))\n* Publish the batch of news into [memphis broker](https://github.com/memphisdev/memphis-broker)\n\n2. [news-presenter](./news-presenter)\n* Consume the batch of news from [memphis broker](https://github.com/memphisdev/memphis-broker)\n* Expose that data over websocket with predefined port\n\n3. [memphis-broker](https://github.com/memphisdev/memphis-broker)\n*  Provides end-to-end support for in-app streaming use cases using Memphis distributed message broker.\n\n## _Customization of behaviour_\nTo customize \nbehaviour or change config kinda values(_scrape interval, switching source of news, external system credentials etc._)\n\nPlease refer to environment variables(e.g [news-collector environment variables](./news-collector/.env.local)) and configuration files(e.g [news-collector config file](./news-collector/config/config.json)).\n\n## _Testing on Docker_\n\n1. First clone the application into your machine\n\n``git clone https://github.com/egasimov/news-system.git``\n\n2. Make sure that docker already installed and running\n\n``docker info``\n\n3. Use docker-compose to run the application in your machine (_P:S it might take some time_)\n\n``docker-compose -f ./docker-compose.yml up -d``\n\n4. Verify containers are properly running on machine.\n\n``docker ps``\n\n![alt text](./doc/output-docker-ps.png)\n\n5. Open [simple file](./news-presenter/client-index.html) in the browser, and verify it is working\n\n``open ./news-presenter/client-index.html``\n\n\n## _Testing on K8s_\n1. First clone the application into your machine\n\n``git clone https://github.com/egasimov/news-system.git``\n\n2. Make sure that kubernetes already configured and running, kubectl properly configured with k8s\n\n``kubectl cluster-info``\n\n4. Make sure that memphis already configured and running properly\n\n``helm repo add memphis https://k8s.memphis.dev/charts/ --force-update \u0026\u0026 helm install memphis memphis/memphis --create-namespace --namespace memphis``\n\nMemphis Deployment Information\n### Secrets ###\nUI/CLI/SDK root username        - root\n\nUI/CLI root Password            - kubectl get secret memphis-creds -n memphis -o jsonpath=\"{.data.ROOT_PASSWORD}\" | base64 --decode\n\nSDK root connection token       - kubectl get secret memphis-creds -n memphis -o jsonpath=\"{.data.CONNECTION_TOKEN}\" | base64 --decode\n\nReplace MEMPHIS_TOKEN(in [k8s-deployment.yml](./k8s-deployment.yaml)) with value from SDK root connection token.\n\n5.Use and apply below deployment file \n\n``kubectl apply -f k8s-deployment.yaml``\n\n6.Verify pods are properly running on machine.\n\n``kubectl get pods --namespace news-system``\n\n![alt text](./doc/output-k8s-get-pods.png)\n\n7. Do port forwarding from k8s to your local machine to test it.\n\n``kubectl port-forward service/news-presenter-go-svc 8081:80 --namespace news-system``\n\n8. Open [simple file](./news-presenter/client-index.html) in the browser, and verify it is working\n\n``open ./news-presenter/client-index.html``\n\n## Result:\n![alt text](./doc/output_1.png)\n![alt text](./doc/output_2.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fegasimov%2Fnews-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fegasimov%2Fnews-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fegasimov%2Fnews-system/lists"}