{"id":25753573,"url":"https://github.com/gabya06/sentiment_analysis_glassdoor","last_synced_at":"2025-06-19T23:43:19.001Z","repository":{"id":277022984,"uuid":"665547031","full_name":"Gabya06/sentiment_analysis_glassdoor","owner":"Gabya06","description":"Sentiment Analysis on glassdoor.com data science jobs","archived":false,"fork":false,"pushed_at":"2025-02-11T17:32:47.000Z","size":951,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-11T18:34:33.106Z","etag":null,"topics":["nlp","nltk-python","sentiment-analysis","textblob-sentiment-analysis","vader-sentiment-analysis"],"latest_commit_sha":null,"homepage":"https://medium.com/@gabya06/python-sentiment-analysis-using-textblob-and-vader-for-glassdoor-reviews-cc9632babb73","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Gabya06.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-12T12:53:01.000Z","updated_at":"2025-02-11T17:32:51.000Z","dependencies_parsed_at":"2025-02-11T18:48:53.585Z","dependency_job_id":null,"html_url":"https://github.com/Gabya06/sentiment_analysis_glassdoor","commit_stats":null,"previous_names":["gabya06/sentiment_analysis_glassdoor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gabya06%2Fsentiment_analysis_glassdoor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gabya06%2Fsentiment_analysis_glassdoor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gabya06%2Fsentiment_analysis_glassdoor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gabya06%2Fsentiment_analysis_glassdoor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Gabya06","download_url":"https://codeload.github.com/Gabya06/sentiment_analysis_glassdoor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240876869,"owners_count":19871904,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nlp","nltk-python","sentiment-analysis","textblob-sentiment-analysis","vader-sentiment-analysis"],"created_at":"2025-02-26T15:18:46.755Z","updated_at":"2025-02-26T15:18:47.334Z","avatar_url":"https://github.com/Gabya06.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sentiment Analysis of Data Science Job Reviews on Glassdoor\n\n## Project Overview\nThis project involves **web scraping** Glassdoor reviews for data science job postings to **analyze sentiment** and explore how review ratings correlate with textual sentiment. The goal is to understand how employees describe their experiences and whether sentiment aligns with the number of stars given.\n\n📖 Read the full article on [medium.com](https://medium.com/@gabya06/python-sentiment-analysis-using-textblob-and-vader-for-glassdoor-reviews-cc9632babb73).\n\n\n## Table of Contents\n\n* [Web Scrapping](#webscrapping)\n* [Text Pre-processing](#textprocessing)\n* [Data Visualization](#viz)\n* [Sentiment Analysis](#sentiment) \n* [Conclusion](#conclusion)\n\n\n\u003ca id=\"webscrapping\"\u003e\u003c/a\u003e\n\n### Web Scrapping Glassdoor Data \nSince Glassdoor lacks a public API, I used `Selenium` and `BeautifulSoup` to extract job listings and reviews. However, scraping was limited due to a `403 Forbidden` error after a small number of requests.\n\nExample Scraping Function:\n\n\n```python\ndef get_page(url, headers):\n    \"\"\"Fetch webpage content using BeautifulSoup.\"\"\"\n    try:\n        req = Request(url, headers=headers)\n        page = urlopen(req)\n        soup = BeautifulSoup(page, \"html.parser\")\n        return soup\n    except HTTPError as e:\n        print(f\"Error opening page {e}\")\n```\n\u003ca id=\"textprocessing\"\u003e\u003c/a\u003e\n\n### Text Preprocessing\nIn order to proceed with sentiment analysis, I had to perform text preprocessing which involved the below. When working on NLP projects, I am usually a big fan of the NLTK library, but this time I wanted to try out TextBlob. \n\n* Removed punctuation, numbers, and stopwords\n* Tokenized text using **TextBlob** instead of **NLTK**\n\n\nExample of cleaning reviews:\n```python\n# assign new clean review column\ndf = df.assign(clean_review = df.reviews.map(lambda x: ' '.join(TextBlob(str(x)).words)))\n```\n\u003ca id=\"viz\"\u003e\u003c/a\u003e\n\n### Data Exploration \u0026 Visualization\nOnce the data is cleaned I can finally move on to the fun part, visualizations! I created a **word cloud** from the most frequent words in the reviews:\n\n\n![review_wordcloud](/img/review_wordcloud.png)\n\nTop words included *good, work, people, great, benefits, culture, balance, pay, management, life*, reflecting common themes in workplace discussions.\n\n\n\u003ca id=\"sentiment\"\u003e\u003c/a\u003e\n\n## Sentiment Analysis\n### TextBlob Sentiment Analysis\nTextBlob measures:\n\n* **Polarity** (-1 to 1): Negative to positive sentiment\n* **Subjectivity** (0 to 1): Objective to subjective content\n\nHere is an example:\n\n```python \nsample_review = df['clean_review'].iloc[0]\nTextBlob(sample_review).sentiment  \n```\n\n*Observations:*\n\n* Higher-rated companies didn’t always have higher polarity scores.\n* Some lower-rated reviews had neutral or even positive sentiment.\n\n\n### VADER Sentiment Analysis\nVADER provides:\n\n* **Positive, Neutral, Negative scores**\n* **Compound score** (-1 to 1), summarizing overall sentiment\n\nHere is an example:\n\n```python \nsid = SentimentIntensityAnalyzer()\nsid.polarity_scores(sample_review)\n```\n\n\n### Sentiment vs. Star Ratings\nWhile VADER's compound scores showed a clearer relationship with ratings, inconsistencies still emerged.\n\n### Polarity vs. Stars\n![textblob_sentiments](/img/textblob_sentiments.png)\n\n### VADER Compound Scores\n![vader_compoundscores](/img/vader_compoundscores.png)\n\n\n\u003ca id=\"conclusion\"\u003e\u003c/a\u003e\n\n## Conclusion\nWhile sentiment analysis provides valuable insights, review ratings don’t always align with textual sentiment. This raises questions about how employees rate companies and whether numerical ratings alone reflect job satisfaction.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgabya06%2Fsentiment_analysis_glassdoor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgabya06%2Fsentiment_analysis_glassdoor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgabya06%2Fsentiment_analysis_glassdoor/lists"}