{"id":26134364,"url":"https://github.com/lioccoumd/news-sentiment-analysis","last_synced_at":"2026-02-16T17:31:35.196Z","repository":{"id":279880727,"uuid":"940304201","full_name":"LIoccoUMD/news-sentiment-analysis","owner":"LIoccoUMD","description":"This scrapes a variety of news stations websites for article titles and evaluates them using distilBERT pretrained model for sentiment analysis.","archived":false,"fork":false,"pushed_at":"2025-03-09T20:19:52.000Z","size":31,"stargazers_count":1,"open_issues_count":5,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-09T21:23:01.862Z","etag":null,"topics":["beautiful","datasets","distilbert","news","nltk","numpy","python","pytorch","reques","scraping-websites","sentiment-analysis","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LIoccoUMD.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-28T00:19:24.000Z","updated_at":"2025-03-09T20:19:56.000Z","dependencies_parsed_at":"2025-02-28T09:01:21.870Z","dependency_job_id":"64a71b22-ad11-45ac-85cb-d4842b8c1068","html_url":"https://github.com/LIoccoUMD/news-sentiment-analysis","commit_stats":null,"previous_names":["lioccoumd/news-sentiment-analysis"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LIoccoUMD%2Fnews-sentiment-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LIoccoUMD%2Fnews-sentiment-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LIoccoUMD%2Fnews-sentiment-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LIoccoUMD%2Fnews-sentiment-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LIoccoUMD","download_url":"https://codeload.github.com/LIoccoUMD/news-sentiment-analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242945837,"owners_count":20210762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautiful","datasets","distilbert","news","nltk","numpy","python","pytorch","reques","scraping-websites","sentiment-analysis","transformers"],"created_at":"2025-03-11T00:00:14.856Z","updated_at":"2026-02-16T17:31:35.190Z","avatar_url":"https://github.com/LIoccoUMD.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# News Sentiment Analyzer\n\nThis Python project scrapes article titles from news sitemaps, performs sentiment analysis on the headlines, and provides a breakdown of positive and negative sentiments. Using web scraping and natural language processing (NLP) techniques, it offers insights into the emotional tone of recent news coverage. Analysis is performed on current events.\n\n## Features\n\n- Scrapes article titles from XML sitemaps:\n  - CNN: `https://www.cnn.com/sitemap/news.xml`\n  - Fox News: `https://www.foxnews.com/sitemap.xml?type=news`\n  - BBC: `https://www.bbc.com/sitemaps/https-sitemap-com-news-2.xml`\n  - Reuters: `https://www.reuters.com/arc/outboundfeeds/news-sitemap/?outputType=xml`\n  - etc.\n- Performs sentiment analysis using the `distilbert-base-uncased-finetuned-sst-2-english` model from Hugging Face's Transformers library.\n- Optimizes performance with batch processing and GPU support (CUDA) when available.\n- Outputs the count of positive and negative headlines for each news source.\n\n## Technologies Used\n\n- **Python Libraries**:\n  - `requests` for HTTP requests\n  - `BeautifulSoup` for XML parsing\n  - `transformers` for sentiment analysis\n  - `torch` for GPU acceleration\n  - `datasets` for efficient data handling\n  - `nltk` for tokenization\n  - `numpy` for numerical operations\n- **NLP**: Pre-trained DistilBERT model for sentiment classification\n- **Web Scraping**: XML parsing with BeautifulSoup and `lxml`\n\n## How It Works\n\n1. Fetches XML sitemap data using `requests`.\n2. Extracts article titles with `BeautifulSoup`.\n3. Converts titles into a `Dataset` object for batch processing.\n4. Runs sentiment analysis to classify titles as \"POSITIVE\" or \"NEGATIVE\" with confidence scores.\n5. Summarizes results by counting positive and negative sentiments for each news source.\n\n## Installation\n\n1. Clone the repository: ```git clone https://github.com/LIoccoUMD/news-sentiment-analyzer.git```\n2. Navigate to the directory: ``` cd news-sentiment-analyzer ```\n3. Install the required dependencies: ```pip install -r requirements.txt```\n\n## Example Output\n```\nCUDA availability: True\nDevice set to use cuda:0\nDevice set to use cuda:0\nDevice set to use cuda:0\nDevice set to use cuda:0\nCNN Count of POSITIVE scores: 91\nCNN Count of NEGATIVE scores: 185\nFOX Count of POSITIVE scores: 99\nFOX Count of NEGATIVE scores: 223\nBBC Count of POSITIVE scores: 389\nBBC Count of NEGATIVE scores: 608\nReuters Count of POSITIVE scores: 11\nReuters Count of NEGATIVE scores: 39\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flioccoumd%2Fnews-sentiment-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flioccoumd%2Fnews-sentiment-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flioccoumd%2Fnews-sentiment-analysis/lists"}