{"id":18941435,"url":"https://github.com/mchmarny/dapr-pipeline","last_synced_at":"2025-04-15T20:32:03.363Z","repository":{"id":64307258,"uuid":"254732681","full_name":"mchmarny/dapr-pipeline","owner":"mchmarny","description":"Example of Twitter event processing pipeline using dapr framewrk","archived":false,"fork":false,"pushed_at":"2023-05-05T02:25:31.000Z","size":50574,"stargazers_count":20,"open_issues_count":7,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T15:16:12.677Z","etag":null,"topics":["dapr","eventing","framework","pipeline","processing","serverless","twitter"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mchmarny.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-10T20:44:51.000Z","updated_at":"2021-12-07T06:07:30.000Z","dependencies_parsed_at":"2024-06-20T13:15:16.440Z","dependency_job_id":null,"html_url":"https://github.com/mchmarny/dapr-pipeline","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fdapr-pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fdapr-pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fdapr-pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fdapr-pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mchmarny","download_url":"https://codeload.github.com/mchmarny/dapr-pipeline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249148018,"owners_count":21220463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dapr","eventing","framework","pipeline","processing","serverless","twitter"],"created_at":"2024-11-08T12:28:03.269Z","updated_at":"2025-04-15T20:31:58.348Z","avatar_url":"https://github.com/mchmarny.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dapr-pipeline\n\nExample of Twitter event processing pipeline using dapr framework.\n\n![alt text](resource/image/overview-local.png \"Local Pipeline Overview\")\n\n\u003e This readme covers local deployment. For Kubernetes deployment instructions [see here](./deployment/)\n\n\u003e To keep it DRY, this demo uses [godapr](https://github.com/mchmarny/godapr) HTTP client \n\n## Prerequisites\n\n### Dapr\n\nTo run this demo locally, you will have to install [Dapr](https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md).\n\n### Twitter\n\nTo use the Dapr twitter binding you will also need Twitter API consumer keys and secrets. You can get these by registering your Twitter application [here](https://developer.twitter.com/en/apps/create).\n\n### Cognitive Services\n\nTo analyze sentiment of each text, you will also need an API token for the Azure [Cognitive Services](https://azure.microsoft.com/en-us/services/cognitive-services/). You can learn more about the API and how to configure it [here](https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-sentiment-analysis?tabs=version-2#sentiment-analysis-versions-and-features).\n\n## Setup\n\nAssuming you have all the prerequisites mentioned above you can demo this dapr pipeline in following steps. First, start by cloning this repo:\n\n```shell\ngit clone https://github.com/mchmarny/dapr-pipeline.git\n```\n\nand then navigate into the `dapr-pipeline` directory:\n\n```shell\ncd dapr-pipeline\n```\n\nand build the executables for your OS\n\n```shell\nbin/build\n```\n\n\u003e Note, if you don't have `go` you can download the pre-built executables for Mac, Windows or Linux from the [latest release](https://github.com/mchmarny/dapr-pipeline/releases/latest).\n\n## Run\n\nThis pipeline consists of three microservices: Processor, Sentimenter, and Viewer. Still inside of the the `dapr-pipeline` you cloned above, follow these instructions to launching each one of these services:\n\n### Sentimenter\n\nTo starting `sentimenter`, first export your Azure Cognitive Services API key (see the [Prerequisites](#prerequisites) section for details).\n\n```shell\nexport CS_TOKEN=\"\u003cyour key here\u003e\"\n```\n\nAnd then launch it using Dapr:\n\n```shell\ndapr run dist/sentimenter --app-id sentimenter --app-port 8082 --protocol http\n```\n\nIf everything goes well when launch these services using Dapr you will see following message:\n\n```shell\nℹ️  Updating metadata for app command: bin/sentimenter\n✅  You're up and running! Both Dapr and your app logs will appear here.\n```\n\n### Viewer\n\nNext, start `viewer`. In yet another terminal window navigate to the `dapr-pipeline` directory and run:\n\n```shell\ndapr run dist/viewer --app-id viewer --app-port 8083 --protocol http\n```\n\nWhile there still won't be any data, at this point you should be able to navigate to the viewer UI\n\nhttp://localhost:8083/\n\n\n### Processor\n\nFinally, to start the `processor`, first edit the Dapr [Twitter input binding component](components/twitter.yaml) using the Twitter API secrets your created in [Prerequisites](#prerequisites)):\n\n```yaml\napiVersion: dapr.io/v1alpha1\nkind: Component\nmetadata:\n  name: tweets\nspec:\n  type: bindings.twitter\n  metadata:\n  - name: consumerKey\n    value: \"\"\n  - name: consumerSecret\n    value: \"\"\n  - name: accessToken\n    value: \"\"\n  - name: accessSecret\n    value: \"\"\n  - name: query\n    value: \"serverless\"  \n```\n\n\u003e For demo purposes use a search term that appears often on Twitter (e.g. serverless)\n\nOnce the Twitter API secrets are set, you are ready to run the `processor`:\n\n```shell\ndapr run dist/processor --app-id processor --app-port 8081 --protocol http --port 3500\n```\n\n### Dashboard\n\nOnce all three microservices are running, you can go back to the `viewer` (http://localhost:8083/). After a few moments you should see something like this:\n\n![](resource/image/ui.png)  \n\nThe icon left of the tweet author's username indicates the sentiment (positive \u003cimg src=\"resource/static/img/s1.svg\" width=\"25\" style=\"vertical-align:middle\"\u003e, negative \u003cimg src=\"resource/static/img/s0.svg\" width=\"25\" style=\"vertical-align:middle\"\u003e, or neutral \u003cimg src=\"resource/static/img/neutral.svg\" width=\"25\" style=\"vertical-align:middle\"\u003e). The Twitter logo right of the username is also linked to the original tweet on https://twitter.com.\n\n\u003e YOu can find how-to on deployment this demo on Kubernetes [here](./deployment/)\n\n## Disclaimer\n\nThis is my personal project and it does not represent my employer. I take no responsibility for issues caused by this code. I do my best to ensure that everything works, but if something goes wrong, my apologies is all you will get.\n\n## License\nThis software is released under the [Apache v2 License](./LICENSE)\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmchmarny%2Fdapr-pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmchmarny%2Fdapr-pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmchmarny%2Fdapr-pipeline/lists"}