{"id":28092432,"url":"https://github.com/mathrailsai/sentiment_insights","last_synced_at":"2026-02-28T07:04:47.874Z","repository":{"id":291332890,"uuid":"977307222","full_name":"mathrailsAI/sentiment_insights","owner":"mathrailsAI","description":"SentimentInsights is a Ruby gem for extracting actionable insights from qualitative survey responses. It provides sentiment analysis, key phrase extraction, and named entity recognition using multiple NLP providers including OpenAI and AWS Comprehend.","archived":false,"fork":false,"pushed_at":"2025-05-04T01:41:23.000Z","size":27,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-13T13:14:39.430Z","etag":null,"topics":["aws-comprehend","entity-recognition","key-phrases","nlp","openai","ruby","ruby-gem","sentiment-analysis","survey-insights","text-analysis"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mathrailsAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-03T22:27:36.000Z","updated_at":"2025-05-13T01:46:47.000Z","dependencies_parsed_at":"2025-05-03T23:36:20.613Z","dependency_job_id":null,"html_url":"https://github.com/mathrailsAI/sentiment_insights","commit_stats":null,"previous_names":["mathrailsai/sentiment_insights"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathrailsAI%2Fsentiment_insights","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathrailsAI%2Fsentiment_insights/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathrailsAI%2Fsentiment_insights/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathrailsAI%2Fsentiment_insights/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mathrailsAI","download_url":"https://codeload.github.com/mathrailsAI/sentiment_insights/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253948503,"owners_count":21988961,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-comprehend","entity-recognition","key-phrases","nlp","openai","ruby","ruby-gem","sentiment-analysis","survey-insights","text-analysis"],"created_at":"2025-05-13T13:14:43.787Z","updated_at":"2026-02-28T07:04:47.865Z","avatar_url":"https://github.com/mathrailsAI.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SentimentInsights\n\n**SentimentInsights** is a Ruby gem for extracting sentiment, key phrases, and named entities from survey responses or free-form textual data. It offers a plug-and-play interface to different NLP providers, including OpenAI, Claude AI, and AWS.\n\n---\n\n## Table of Contents\n\n- [Installation](#installation)\n- [Configuration](#configuration)\n- [Usage](#usage)\n    - [Sentiment Analysis](#sentiment-analysis)\n    - [Key Phrase Extraction](#key-phrase-extraction)\n    - [Entity Extraction](#entity-extraction)\n- [Export \u0026 Output Formats](#export--output-formats)\n- [Provider Options \u0026 Custom Prompts](#provider-options--custom-prompts)\n- [Full Example](#full-example)\n- [Contributing](#contributing)\n- [License](#license)\n\n---\n\n## Installation\n\nAdd to your Gemfile:\n\n```ruby\ngem 'sentiment_insights'\n```\n\nThen install:\n\n```bash\nbundle install\n```\n\nOr install it directly:\n\n```bash\ngem install sentiment_insights\n```\n\n---\n\n## Configuration\n\nConfigure the provider and (if using OpenAI, Claude AI, or AWS) your API key:\n\n```ruby\nrequire 'sentiment_insights'\n\n# For OpenAI\nSentimentInsights.configure do |config|\n  config.provider = :openai\n  config.openai_api_key = ENV[\"OPENAI_API_KEY\"]\nend\n\n# For Claude AI\nSentimentInsights.configure do |config|\n  config.provider = :claude\n  config.claude_api_key = ENV[\"CLAUDE_API_KEY\"]\nend\n\n# For AWS\nSentimentInsights.configure do |config|\n  config.provider = :aws\n  config.aws_region = 'us-east-1'\nend\n\n# For sentimental\nSentimentInsights.configure do |config|\n  config.provider = :sentimental\nend\n```\n\nSupported providers:\n- `:openai`\n- `:claude`\n- `:aws`\n- `:sentimental` (local fallback, limited feature set)\n\n---\n\n## Usage\n\nData entries should be hashes with at least an `:answer` key. Optionally include segmentation info under `:segment`.\n\n```ruby\nentries = [\n  { answer: \"Amazon Checkout was smooth!\", segment: { age_group: \"18-25\", gender: \"Female\" } },\n  { answer: \"Walmart Shipping was delayed.\", segment: { age_group: \"18-25\", gender: \"Female\" } },\n  { answer: \"Target Support was decent.\", segment: { age_group: \"26-35\", gender: \"Male\" } },\n  { answer: \"Loved the product!\", segment: { age_group: \"18-25\", gender: \"Male\" } }\n]\n```\n\n---\n\n### Sentiment Analysis\n\nQuickly classify and summarize user responses as positive, neutral, or negative — globally or by segment (e.g., age, region).\n\n#### 🔍 Example Call\n\n```ruby\ninsight = SentimentInsights::Insights::Sentiment.new\nresult = insight.analyze(entries)\n```\n\nWith options:\n\n```ruby\ncustom_prompt = \u003c\u003c~PROMPT\n  For each of the following customer responses, classify the sentiment as Positive, Neutral, or Negative, and assign a score between -1.0 (very negative) and 1.0 (very positive).\n\n            Reply with a numbered list like:\n            1. Positive (0.9)\n            2. Negative (-0.8)\n            3. Neutral (0.0)\nPROMPT\n\ninsight = SentimentInsights::Insights::Sentiment.new\nresult = insight.analyze(\n  entries,\n  question: \"How was your experience today?\",\n  prompt: custom_prompt,\n  batch_size: 10\n)\n```\n\n#### Available Options (`analyze`)\n| Option        | Type    | Description                                                            | Provider           |\n|---------------|---------|------------------------------------------------------------------------|--------------------|\n| `question`    | String  | Contextual question for the batch                                     | OpenAI, Claude only |\n| `prompt`      | String  | Custom prompt text for LLM                                            | OpenAI, Claude only |\n| `batch_size`  | Integer | Number of entries per completion call (default: 50)                  | OpenAI, Claude only |\n\n#### 📾 Sample Output\n\n```ruby\n{:global_summary=\u003e\n   {:total_count=\u003e5,\n    :positive_count=\u003e3,\n    :neutral_count=\u003e0,\n    :negative_count=\u003e2,\n    :positive_percentage=\u003e60.0,\n    :neutral_percentage=\u003e0.0,\n    :negative_percentage=\u003e40.0,\n    :net_sentiment_score=\u003e20.0},\n :segment_summary=\u003e\n   {:age=\u003e\n      {\"25-34\"=\u003e\n         {:total_count=\u003e3,\n          :positive_count=\u003e3,\n          :neutral_count=\u003e0,\n          :negative_count=\u003e0,\n          :positive_percentage=\u003e100.0,\n          :neutral_percentage=\u003e0.0,\n          :negative_percentage=\u003e0.0,\n          :net_sentiment_score=\u003e100.0}},\n    :top_positive_comments=\u003e\n      [{:answer=\u003e\n          \"I absolutely loved the experience shopping with Everlane. The website is clean,\\n\" +\n            \"product descriptions are spot-on, and my jeans arrived two days early with eco-friendly packaging.\",\n        :score=\u003e0.9}],\n    :top_negative_comments=\u003e\n      [{:answer=\u003e\n          \"The checkout flow on your site was a nightmare. The promo code from your Instagram campaign didn’t work,\\n\" +\n            \"and it kept redirecting me to the homepage. Shopify integration needs a serious fix.\",\n        :score=\u003e-0.7}],\n    :responses=\u003e\n      [{:answer=\u003e\n          \"I absolutely loved the experience shopping with Everlane. The website is clean,\\n\" +\n            \"product descriptions are spot-on, and my jeans arrived two days early with eco-friendly packaging.\",\n        :segment=\u003e{:age=\u003e\"25-34\", :region=\u003e\"West\"},\n        :sentiment_label=\u003e:positive,\n        :sentiment_score=\u003e0.9}]}}\n```\n\n---\n\n### Key Phrase Extraction\n\nExtract frequently mentioned phrases and identify their associated sentiment and segment spread.\n\n```ruby\ninsight = SentimentInsights::Insights::KeyPhrases.new\nresult = insight.extract(entries)\n```\n\nWith options:\n\n```ruby\nkey_phrase_prompt = \u003c\u003c~PROMPT.strip\n  Extract the most important key phrases that represent the main ideas or feedback in the sentence below.\n  Ignore stop words and return each key phrase in its natural form, comma-separated.\n\n  Question: %{question}\n\n  Text: %{text}\nPROMPT\n\nsentiment_prompt = \u003c\u003c~PROMPT\n  For each of the following customer responses, classify the sentiment as Positive, Neutral, or Negative, and assign a score between -1.0 (very negative) and 1.0 (very positive).\n\n            Reply with a numbered list like:\n            1. Positive (0.9)\n            2. Negative (-0.8)\n            3. Neutral (0.0)\nPROMPT\n\ninsight = SentimentInsights::Insights::KeyPhrases.new\nresult = insight.extract(\n  entries,\n  question: \"What are the recurring themes?\",\n  key_phrase_prompt: key_phrase_prompt,\n  sentiment_prompt: sentiment_prompt\n)\n```\n\n#### Available Options (`extract`)\n| Option             | Type    | Description                                                | Provider           |\n|--------------------|---------|------------------------------------------------------------|--------------------|\n| `question`         | String  | Context question to help guide phrase extraction           | OpenAI, Claude only |\n| `key_phrase_prompt`| String  | Custom prompt for extracting key phrases                   | OpenAI, Claude only |\n| `sentiment_prompt` | String  | Custom prompt for classifying tone of extracted phrases    | OpenAI, Claude only |\n\n#### 📾 Sample Output\n\n```ruby\n{:phrases=\u003e\n   [{:phrase=\u003e\"everlane\",\n     :mentions=\u003e[\"r_1\"],\n     :summary=\u003e\n       {:total_mentions=\u003e1,\n        :sentiment_distribution=\u003e{:positive=\u003e1, :negative=\u003e0, :neutral=\u003e0},\n        :segment_distribution=\u003e{:age=\u003e{\"25-34\"=\u003e1}, :region=\u003e{\"West\"=\u003e1}}}}],\n :responses=\u003e\n   [{:id=\u003e\"r_1\",\n     :sentence=\u003e\n       \"I absolutely loved the experience shopping with Everlane. The website is clean,\\n\" +\n         \"product descriptions are spot-on, and my jeans arrived two days early with eco-friendly packaging.\",\n     :sentiment=\u003e:positive,\n     :segment=\u003e{:age=\u003e\"25-34\", :region=\u003e\"West\"}}]}\n```\n\n---\n\n### Entity Extraction\n\n```ruby\ninsight = SentimentInsights::Insights::Entities.new\nresult = insight.extract(entries)\n```\n\nWith options:\n\n```ruby\nentity_prompt = \u003c\u003c~PROMPT.strip\n  Identify brand names, competitors, and product references in the sentence below.\n  Return each as a JSON object with \"text\" and \"type\" (e.g., BRAND, PRODUCT, COMPANY).\n\n  Question: %{question}\n\n  Sentence: \"%{text}\"\nPROMPT\n\ninsight = SentimentInsights::Insights::Entities.new\nresult = insight.extract(\n  entries,\n  question: \"Which products or brands are mentioned?\",\n  prompt: entity_prompt\n)\n\n```\n\n#### Available Options (`extract`)\n| Option      | Type    | Description                                       | Provider           |\n|-------------|---------|---------------------------------------------------|--------------------|\n| `question`  | String  | Context question to guide entity extraction       | OpenAI, Claude only |\n| `prompt`    | String  | Custom instructions for entity extraction         | OpenAI, Claude only |\n\n#### 📾 Sample Output\n\n```ruby\n{:entities=\u003e\n   [{:entity=\u003e\"everlane\",\n     :type=\u003e\"ORGANIZATION\",\n     :mentions=\u003e[\"r_1\"],\n     :summary=\u003e\n       {:total_mentions=\u003e1,\n        :segment_distribution=\u003e{:age=\u003e{\"25-34\"=\u003e1}, :region=\u003e{\"West\"=\u003e1}}}},\n    {:entity=\u003e\"jeans\",\n     :type=\u003e\"PRODUCT\",\n     :mentions=\u003e[\"r_1\"],\n     :summary=\u003e\n       {:total_mentions=\u003e1,\n        :segment_distribution=\u003e{:age=\u003e{\"25-34\"=\u003e1}, :region=\u003e{\"West\"=\u003e1}}}},\n    {:entity=\u003e\"24 hours\",\n     :type=\u003e\"TIME\",\n     :mentions=\u003e[\"r_4\"],\n     :summary=\u003e\n       {:total_mentions=\u003e1,\n        :segment_distribution=\u003e{:age=\u003e{\"45-54\"=\u003e1}, :region=\u003e{\"Midwest\"=\u003e1}}}}],\n :responses=\u003e\n   [{:id=\u003e\"r_1\",\n     :sentence=\u003e\n       \"I absolutely loved the experience shopping with Everlane. The website is clean,\\n\" +\n         \"product descriptions are spot-on, and my jeans arrived two days early with eco-friendly packaging.\",\n     :segment=\u003e{:age=\u003e\"25-34\", :region=\u003e\"West\"}},\n    {:id=\u003e\"r_4\",\n     :sentence=\u003e\n       \"I reached out to your Zendesk support team about a missing package, and while they responded within 24 hours,\\n\" +\n         \"the response was copy-paste and didn't address my issue directly.\",\n     :segment=\u003e{:age=\u003e\"45-54\", :region=\u003e\"Midwest\"}}]}\n```\n\n---\n\n## Export \u0026 Output Formats\n\nSentimentInsights provides flexible export options for all analysis results. Export to CSV, Excel, or JSON formats with advanced filtering and customization.\n\n### Quick Export Examples\n\n```ruby\nresult = SentimentInsights::Insights::Sentiment.new.analyze(entries)\n\n# Direct format methods\nresult.to_csv(\"analysis.csv\")           # CSV file\nresult.to_json(\"analysis.json\")         # JSON file  \nresult.to_excel(\"analysis.xlsx\")        # Excel file\n\n# Auto-detect format from filename\nresult.export(\"analysis.csv\")           # Creates CSV\nresult.export(\"analysis.json\")          # Creates JSON\nresult.export(\"analysis.xlsx\")          # Creates Excel\n\n# API-friendly formats (no files)\napi_data = result.to_hash               # Structured Hash\njson_string = result.to_json_string     # JSON string\n```\n\n### Advanced Export Options\n\n```ruby\n# Filtered exports\nresult.export_positive(:csv, \"positive_feedback.csv\")\nresult.export_negative(:excel, \"issues_to_address.xlsx\")\n\n# Fluent interface with filtering\nresult\n  .filter_sentiment(:positive)\n  .filter_segments(age_group: [\"18-25\", \"26-35\"])\n  .to_excel(\"young_positive_feedback.xlsx\")\n\n# Export all formats at once\nfiles = result.export_all(\"comprehensive_analysis\")\n# Returns: { csv: \"file.csv\", json: \"file.json\", excel: \"file.xlsx\" }\n```\n\n### CSV Export Structure\n\n```csv\nresponse_id,text,sentiment_label,sentiment_score,segment_age_group,segment_region,timestamp\nr_1,\"I love this product!\",positive,0.9,18-25,North,2024-06-28T10:30:00Z\n\nSUMMARY STATISTICS\nTotal Responses,150\nPositive Percentage,60.0%\nNet Sentiment Score,40.0\n\nSEGMENT ANALYSIS\nSegment Type,Segment Value,Total Count,Positive %,Net Score\nAge Group,18-25,75,65.3%,49.3\n```\n\n### JSON Export Structure\n\n```json\n{\n  \"metadata\": {\n    \"export_timestamp\": \"2024-06-28T10:30:00Z\",\n    \"analysis_type\": \"sentiment\",\n    \"total_responses\": 150,\n    \"provider_used\": \"claude\"\n  },\n  \"analysis\": {\n    \"responses\": [...],\n    \"global_summary\": {...},\n    \"segment_summary\": {...}\n  }\n}\n```\n\n**📋 For complete export documentation, see [EXPORT_USAGE.md](EXPORT_USAGE.md)**\n\n---\n\n## Provider Options \u0026 Custom Prompts\n\n\u003e ⚠️ All advanced options (`question`, `prompt`, `key_phrase_prompt`, `sentiment_prompt`, `batch_size`) apply only to the `:openai` and `:claude` providers.  \n\u003e They are safely ignored for `:aws` and `:sentimental`.\n\n---\n\n## 🔑 Environment Variables\n\n### OpenAI\n\n```bash\nOPENAI_API_KEY=your_openai_key_here\n```\n\n### Claude AI\n\n```bash\nCLAUDE_API_KEY=your_claude_key_here\n```\n\n### AWS Comprehend\n\n```bash\nAWS_ACCESS_KEY_ID=your_aws_key\nAWS_SECRET_ACCESS_KEY=your_aws_secret\nAWS_REGION=us-east-1\n```\n\n---\n\n## 💎 Ruby Compatibility\n\n- **Minimum Ruby version:** 2.7\n\n---\n\n## 🔮 Testing\n\n```bash\nbundle exec rspec\n```\n\n---\n\n## 📋 Roadmap\n\n- [x] Sentiment Analysis\n- [x] Key Phrase Extraction\n- [x] Entity Recognition\n- [x] Export Functionality (CSV, Excel, JSON)\n- [ ] Topic Modeling\n- [ ] Visual Dashboard Add-on\n\n---\n\n## 📄 License\n\nMIT License\n\n---\n\n## 🙌 Contributing\n\nPull requests welcome! Please open an issue to discuss major changes first.\n\n---\n\n## 💬 Acknowledgements\n\n- [OpenAI GPT](https://platform.openai.com/docs)\n- [Claude AI](https://docs.anthropic.com/claude/reference/getting-started-with-the-api)\n- [AWS Comprehend](https://docs.aws.amazon.com/comprehend/latest/dg/what-is.html)\n- [Sentimental Gem](https://github.com/7compass/sentimental)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathrailsai%2Fsentiment_insights","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathrailsai%2Fsentiment_insights","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathrailsai%2Fsentiment_insights/lists"}