{"id":13743501,"url":"https://github.com/elithrar/centiment","last_synced_at":"2025-05-09T01:30:56.635Z","repository":{"id":51774690,"uuid":"115298529","full_name":"elithrar/centiment","owner":"elithrar","description":" 🤖 Crypto-currency sentiment analysis via Google Natural Language \u0026 Twitter.","archived":true,"fork":false,"pushed_at":"2021-05-10T03:32:41.000Z","size":287,"stargazers_count":26,"open_issues_count":6,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-06T03:57:16.076Z","etag":null,"topics":["appengine","bitcoin","cryptocurrency","ethereum","firestore","golang","google-cloud","sentiment-analysis"],"latest_commit_sha":null,"homepage":"https://github.com/elithrar/centiment","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elithrar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-25T02:10:53.000Z","updated_at":"2023-12-22T16:52:57.000Z","dependencies_parsed_at":"2022-08-03T03:00:22.067Z","dependency_job_id":null,"html_url":"https://github.com/elithrar/centiment","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fcentiment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fcentiment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fcentiment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fcentiment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elithrar","download_url":"https://codeload.github.com/elithrar/centiment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253174239,"owners_count":21865836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appengine","bitcoin","cryptocurrency","ethereum","firestore","golang","google-cloud","sentiment-analysis"],"created_at":"2024-08-03T05:00:48.842Z","updated_at":"2025-05-09T01:30:56.243Z","avatar_url":"https://github.com/elithrar.png","language":"Go","funding_links":[],"categories":["Golang"],"sub_categories":["Sentiment Intensity Analyzer"],"readme":"# 🤖 centiment\n\n[![GoDoc](https://godoc.org/github.com/elithrar/centiment?status.svg)](https://godoc.org/github.com/elithrar/centiment)\n[![CircleCI](https://circleci.com/gh/elithrar/centiment.svg?style=svg)](https://circleci.com/gh/elithrar/centiment)\n\nCentiment is a service that performs sentiment analysis of tweets using Google's [Natural Language APIs](https://cloud.google.com/natural-language/). It was designed with the goal of searching for cryptocurrency tweets, but can be used to analyze and aggregate sentiments for any search terms.\n\n* It will search Twitter for tweets matching the configured search terms, and store the aggregate \"sentiment\" (negative, neutral or positive) and magnitude each time it runs a search.\n* Search terms can be easily added without writing code via `cmd/centimentd/search.toml`\n* The aggregate results are made available via a REST API.\n\nThe goal is to see whether written sentiment about cryptocurrencies has correlation with prices - e.g. does a negative sentiment predict or otherwise reinforce a drop in price?\n\n## Usage\n\nCentiment relies on Google's [Natural Language APIs](https://cloud.google.com/natural-language/docs/analyzing-sentiment) and [Firestore](https://firebase.google.com/docs/firestore/), but otherwise can run anywhere provided it can reach these services.\n\nAt a minimum, you'll need to:\n\n* Install the [Google Cloud SDK](https://cloud.google.com/sdk/) \u0026 create a new project with billing enabled.\n* Create a new Firestore instance \u0026 enable the Natural Language API via the [Google Cloud API Dashboard](https://console.cloud.google.com/apis/dashboard).\n* Create a [new Twitter application](https://apps.twitter.com/) \u0026 retrieve your API credentials.\n* Install the Firebase SDK via `npm install -g firebase-tools`\n\n### Running Locally\n\nYou can run Centiment locally with a [properly configured Go toolchain](https://golang.org/doc/install) and [Service Account](https://console.cloud.google.com/apis/credentials) credentials saved locally.\n\n```sh\n# Fetch Centiment \u0026 its dependencies\ngo get github.com/elithrar/centiment/...\n\n# Initialize the Firebase SDK \u0026 create the required indexes\ncentiment/ $ firebase login\ncentiment/ $ firebase deploy --only firestore:indexes\n\n# Set the required configuration as env. variables, or pass via flags (see: `centiment --help`)\nexport TWITTER_CONSUMER_KEY=\"key\"; \\\n  export TWITTER_CONSUMER_SECRET=\"secret\"; \\\n  export TWITTER_ACCESS_TOKEN=\"at\"; \\\n  export TWITTER_ACCESS_KEY=\"ak\"; \\\n  export CENTIMENT_PROJECT_ID=\"your-gcp-project-id\"; \\\n  export GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/creds.json\";\n\n# Run centimentd (the server) in the foreground, provided its on your PATH:\n$ centimentd\n```\n\n### Deploy to App Engine Flexible\n\nApp Engine Flexible makes running Centiment fairly easy: no need to set up or secure an environment.\n\n* `git clone` or `go get` this repository: `git clone https://github.com/elithrar/centiment.git`\n* Copy `app.example.yaml` to `app.yaml` and add your Twitter API keys under `env_variables` - important: don't check these credentials into your source-code! The `.gitignore` file included in the repo should help to prevent that.\n\nThe service can then be deployed via:\n\n```\ncentiment $ cd cmd/centimentd\ncmd/centimentd $ gcloud app deploy\n```\n\n#### Cost\n\nSome notes on running this yourself:\n\n* The default `app.example.yaml` included alongside is designed to use the minimum set of resources on App Engine Flex. Centiment is extremely efficient (it's written in Go) and runs quickly on a single CPU core + 600MB RAM. At the time of writing (Jan 2018), running a 1CPU / 1GB RAM / 10GB disk App Engine Flex instance for a month is ~USD$44/month.\n* Cloud Function pricing is fairly cheap for our use-case: if you're running a search every 10 minutes, that's 6 times an hour \\* 730 hours per month = 4380 invocations per search term per month. That falls into the [free tier](https://cloud.google.com/functions/pricing) of Cloud Functions pricing.\n* The Natural Language API is where the majority of the costs will lie if you choose to run Centiment more aggressively (more tweets, more often). _Searching for up to 50 tweets (per search term) every 10 minutes is 219,000 [Sentiment Analysis records](https://cloud.google.com/natural-language/pricing) per month, and results in a total of USD$219 per search term per month (as of Jan 2018), excluding the small free tier (first 5k)_\n\n\u003e Note: Make sure to do the math before tweaking the `CENTIMENT_RUN_INTERVAL` or `CENTIMENT_MAX_TWEETS` environmental variables, or adding additional search terms to `cmd/centimentd/search.toml`.\n\n### Using BigQuery for Analysis\n\nIn order to make analysis easier, you can import data directly into BigQuery after each run via a [Cloud Function](https://firebase.google.com/docs/functions/firestore-events) that is triggered from every database write.\n\n#### Pre-requisites\n\nYou'll need to:\n\n* Create a [BigQuery dataset](https://cloud.google.com/bigquery/docs/datasets#create-dataset) called \"Centiment\" and [a table](https://cloud.google.com/bigquery/docs/tables) called \"sentiments\". You can opt to use different names, but you will need to make sure to use `config:set` within the Firebase SDK so that our function works.\n\n```sh\n# Create an empty table with our schema using the bq CLI tool (installed with the gcloud SDK)\ncentiment/ $ bq mk --schema bigquery.schema.json -t centiment.sentiments\n```\n\n* [Install the Firebase SDK](https://firebase.google.com/docs/functions/get-started) so that we can deploy the Cloud Function with the Firestore trigger.\n\n```sh\ncentiment $ cd _functions\n# Log into your Google Cloud Platform account\n_functions $ firebase login\n# Set the dataset and table names\n_functions $ firebase functions:config:set centiment.dataset=\"Centiment\" centiment.table=\"sentiments\"\n# Deploy this secific function.\n_functions $ firebase deploy --only functions:sentimentsToBQ\n# Done!\n```\n\n### Docker\n\nTODO(matt): Create a `Dockerfile` - for this `FROM alpine:latest`\n\n#### Running Elsewhere\n\nIf you're running Centiment elsewhere, you'll need to provide the [application with credentials](https://cloud.google.com/docs/authentication/production) to reach Firestore and the Natural Language APIs by setting the `GOOGLE_APPLICATION_CREDENTIALS` environmental variable to the location of your credentials file.\n\nFurther, the `Store` interface allows you to provide alternate backend datastores (e.g. PostgreSQL), if you want to run Centiment on alternative infrastructure.\n\n### REST API\n\nCentiment exposes its analysis as JSON via a REST API. Requests are not authenticated by default.\n\n```sh\n# Get the latest sentiments for the named currency (\"bitcoin\", in this case)\nGET /sentiments/bitcoin\n\n[\n  {\n    \"id\": \"lwnXwJmNbxRoE0mzXff0\",\n    \"topic\": \"bitcoin\",\n    \"slug\": \"bitcoin\",\n    \"query\": \"bitcoin OR BTC OR #bitcoin OR #BTC -filter:retweets\",\n    \"count\": 154,\n    \"score\": 0.11818181921715863,\n    \"stdDev\": 0.3425117817511681,\n    \"variance\": 0.11731432063835981,\n    \"fetchedAt\": \"2018-02-12T05:24:15.44671Z\"\n  }\n]\n```\n\n## Contributing\n\nPRs are welcome, but any non-trivial changes should be raised as an issue first to discuss the design and avoid having your hard work rejected!\n\nSuggestions for contributors:\n\n* Additional sentiment analysis adapters (e.g. Azure Cognitive Services, IBM Watson)\n* Alternative backend datastores\n\n## License\n\nBSD licensed. See the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felithrar%2Fcentiment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felithrar%2Fcentiment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felithrar%2Fcentiment/lists"}