{"id":22740469,"url":"https://github.com/azlinrusnan/optimizing_csat_through_sentiment-analysis_and_predictive-modeling","last_synced_at":"2025-06-25T23:35:41.540Z","repository":{"id":263706421,"uuid":"891238332","full_name":"AzlinRusnan/Optimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling","owner":"AzlinRusnan","description":"This study focuses on optimizing Customer Satisfaction (CSAT) analysis through Sentiment Analysis and Predictive Modeling. Using Natural Language Processing (NLP) and Machine Learning (ML), the project improves sentiment classification accuracy, aligns sentiment scores with CSAT ratings, and enhances predictive modeling performance.","archived":false,"fork":false,"pushed_at":"2025-05-19T07:02:57.000Z","size":2080,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-19T07:47:05.662Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AzlinRusnan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-20T01:16:19.000Z","updated_at":"2025-05-19T07:03:00.000Z","dependencies_parsed_at":"2024-11-20T02:21:58.159Z","dependency_job_id":"20e3d6ed-cf1a-4f32-9063-e3841436c79d","html_url":"https://github.com/AzlinRusnan/Optimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling","commit_stats":null,"previous_names":["azlinrusnan/optimizing-customer-satisfaction-csat-through-sentiment-analysis-and-predictive-ml-techniques","azlinrusnan/optimizing_csat_through_sentiment-analysis_and_predictive-modeling"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AzlinRusnan%2FOptimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AzlinRusnan%2FOptimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AzlinRusnan%2FOptimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AzlinRusnan%2FOptimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AzlinRusnan","download_url":"https://codeload.github.com/AzlinRusnan/Optimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AzlinRusnan%2FOptimizing_CSAT_Through_Sentiment-Analysis_and_Predictive-Modeling/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":258540127,"owners_count":22717334,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-10T23:08:56.968Z","updated_at":"2025-06-25T23:35:41.523Z","avatar_url":"https://github.com/AzlinRusnan.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cdiv align=\"center\"\u003e Optimizing CSAT Through Sentiment Analysis and Predictive Modeling Techniques\u003c/div\u003e\n\n## Summary\n\nCustomer Satisfaction (CSAT) is a key performance metric in any service-driven organisation. However, CSAT scores often fail to capture the full customer experience, especially when sentiment in textual feedback contradicts the given score.\n\nTo address this, we developed an AI-based analytics pipeline that combines **Natural Language Processing (NLP)** and **Machine Learning (ML)** to:\n\n- Automatically extract sentiment from customer comments.\n\n- Align this sentiment with CSAT scores to identify misclassifications or hidden dissatisfaction.\n\n- Predict satisfaction levels from structured support ticket.\n\nThe result is a smarter, scalable system for understanding customer satisfaction, enabling better resource planning and faster issue resolution.\n\n## Solution Overview\n\n### 🧠 Phase 1: Sentiment Detection from Customer Comments\n\n- Started with AFINN, a rule-based lexicon, which produced inaccurate classifications.\n\n- However, as shown below, it fails to capture the actual sentiment of customer feedback correctly.\n\n![1](images/lexiconresults.png)\n  \n- Due to AFINN’s limitations, we explored a more advanced BERT model, which significantly improved sentiment detection.\n  \n- Upgraded to a fine-tuned BERT model, significantly improving sentiment accuracy by understanding context, negation, and domain-specific phrasing.\n\n![2](images/BERTRES.png)\n\n**Result:** A multilingual BERT model fine-tuned on historical feedback data achieved high classification accuracy and contextual understanding.\n\n### 🔁 Phase 2: Aligning Sentiment with CSAT Scores\n\n![3](images/objtwo.png)\n\n- The model detected 5.75% of user-labeled Negative feedback as actually Positive, improving sentiment alignment.\n\n- It also detected 0.65% of user-labeled Positive feedback as actually Negative, uncovering hidden dissatisfaction.\n\n- Business Impact: This alignment improves the credibility of CSAT reporting and helps surface hidden service issues.\n\n### 🧩 Phase 3: Predicting CSAT with Machine Learning\n\nWe used structured ticket metadata (country, region, sentiment polarity, etc.) to train multiple classification models:\n\n- Logistic Regression\n\n- Random Forest\n\n- Support Vector Machine (SVM)\n\n- Gradient Boosting Machine (GBM)\n\nAll models achieved high accuracy, but **Logistic Regression** performed best on ROC-AUC, highlighting its strength in distinguishing satisfaction levels.\n\n- Evaluation Metrics:\n\n- Accuracy – Overall correctness of predictions.\n- ROC-AUC Score – Ability to distinguish sentiment polarity.\n- Precision, Recall, F1-score – Balance between false positives \u0026 false negatives.\n- Confusion Matrix – Insights into correct vs. misclassified instances.\n\n**Key Finding:**\n\n![3](images/mlresults.png)\n\n- Logistic Regression achieved the highest ROC-AUC (0.9512), demonstrating its superior ability to distinguish sentiment polarity, despite high accuracy across all models (97.62%).\n\n##  Summary of Outcomes\n\n✅ BERT Model outperforms traditional methods in interpreting customer sentiment.\n\n✅ Sentiment-CSAT misalignment highlights previously unseen issues.\n\n✅ Predictive ML models enable accurate classification of customer satisfaction.\n\n✅ Logistic Regression offers strong, interpretable performance for operational use.\n\n\n\n## Future Enhancements\n\n- Expand to Multilingual Feedback – Incorporate customer reviews in different languages to improve global applicability.\n\n- Feature Expansion – Add Ticket Priority, User Type, and additional metadata for better prediction accuracy.\n\n- Explore Advanced Transformers – Investigate more sophisticated NLP models for improved sentiment classification.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazlinrusnan%2Foptimizing_csat_through_sentiment-analysis_and_predictive-modeling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazlinrusnan%2Foptimizing_csat_through_sentiment-analysis_and_predictive-modeling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazlinrusnan%2Foptimizing_csat_through_sentiment-analysis_and_predictive-modeling/lists"}