{"id":18941461,"url":"https://github.com/mchmarny/stockercm","last_synced_at":"2025-10-23T07:35:01.256Z","repository":{"id":77051956,"uuid":"184933827","full_name":"mchmarny/stockercm","owner":"mchmarny","description":"Sentiment processor using Cloud Functions, Cloud PubSub, Natural Language Processing API, and Cloud Dataflow.","archived":false,"fork":false,"pushed_at":"2019-05-09T02:58:42.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-25T10:41:14.962Z","etag":null,"topics":["api","cloud","dataflow","function","nlp","pubsub"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mchmarny.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-04T19:01:09.000Z","updated_at":"2019-05-09T03:17:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"6856b95d-647b-446d-aa10-fb08f4449826","html_url":"https://github.com/mchmarny/stockercm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mchmarny/stockercm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fstockercm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fstockercm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fstockercm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fstockercm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mchmarny","download_url":"https://codeload.github.com/mchmarny/stockercm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mchmarny%2Fstockercm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266738345,"owners_count":23976417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","cloud","dataflow","function","nlp","pubsub"],"created_at":"2024-11-08T12:28:09.234Z","updated_at":"2025-10-23T07:34:56.208Z","avatar_url":"https://github.com/mchmarny.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# stockercm\n\nSentiment processor using `Cloud Functions`, `Cloud PubSub`, `Natural Language Processing API`, and `Cloud Dataflow`.\n\n## Setup\n\nCreate PubSub topics\n\n```shell\ngcloud pubsub topics create stocker-source\ngcloud pubsub topics create stocker-processed\n```\n\nCreating BigQuery content table\n\n```shell\nbq mk stocker\nbq query --use_legacy_sql=false \"\n  CREATE OR REPLACE TABLE stocker.content (\n    symbol STRING NOT NULL,\n    cid STRING NOT NULL,\n    created TIMESTAMP NOT NULL,\n    author STRING NOT NULL,\n    lang STRING NOT NULL,\n    source STRING NOT NULL,\n    content STRING NOT NULL,\n    magnitude FLOAT64 NOT NULL,\n    score FLOAT64 NOT NULL,\n    retweet BOOL NOT NULL\n)\"\n```\n\nCreate Cloud Dataflow job to drain processed topic to BigQuery\n\n```shell\ngcloud dataflow jobs run stocker-processed-to-bq-pump \\\n  --gcs-location gs://dataflow-templates/latest/PubSub_to_BigQuery \\\n  --parameters topic=projects/${GCP_PROJECT}/topics/stocker-processed,\\\n    table=${GCP_PROJECT}:stocker.content \\\n  --region us-central1\n```\n\n## Deploy\n\nDeploy a Cloud Function to process the events from `stocker-source` topic and push results to `stocker-processed`.\n\n```shell\ngcloud functions deploy stocker-process \\\n  --entry-point ProcessorSentiment \\\n  --set-env-vars \"PID=${GCP_PROJECT}\" \\\n  --memory 256MB \\\n  --region us-central1 \\\n  --runtime go112 \\\n  --trigger-topic stocker-source \\\n  --timeout=300s\n```\n\n## Cleanup\n\nDelete the Cloud Function\n\n```shell\ngcloud functions delete stocker-process --region us-central1\n```\n\nDelete the Cloud Dataflow job\n\n```shell\ngcloud dataflow jobs cancel stocker-processed-to-bq-pump --region us-central1\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmchmarny%2Fstockercm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmchmarny%2Fstockercm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmchmarny%2Fstockercm/lists"}