{"id":24431914,"url":"https://github.com/arrismo/news-dashboard","last_synced_at":"2026-04-25T15:35:31.176Z","repository":{"id":261862226,"uuid":"885562069","full_name":"arrismo/news-dashboard","owner":"arrismo","description":"Dashboard using News API, Snowflake, dbt, Textblob and Tableau","archived":false,"fork":false,"pushed_at":"2024-12-14T21:39:25.000Z","size":21671,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-31T01:59:16.119Z","etag":null,"topics":["dbt","snowflake","tableau","textblob"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arrismo.png","metadata":{"files":{"readme":"README.md","changelog":"news.png","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-08T20:44:23.000Z","updated_at":"2025-04-17T15:24:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"dc0f0efa-efc9-4941-a51d-90a4877ca461","html_url":"https://github.com/arrismo/news-dashboard","commit_stats":null,"previous_names":["arrismo/gas-prices","arrismo/news-dashboard"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arrismo/news-dashboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arrismo%2Fnews-dashboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arrismo%2Fnews-dashboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arrismo%2Fnews-dashboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arrismo%2Fnews-dashboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arrismo","download_url":"https://codeload.github.com/arrismo/news-dashboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arrismo%2Fnews-dashboard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32267710,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T09:15:33.318Z","status":"ssl_error","status_checked_at":"2026-04-25T09:15:31.997Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dbt","snowflake","tableau","textblob"],"created_at":"2025-01-20T15:27:01.737Z","updated_at":"2026-04-25T15:35:31.139Z","avatar_url":"https://github.com/arrismo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# News Article Pipeline and Analysis\n\n\n\nThis project leverages the News API to collect recent articles about artificial intelligence, utilizing TextBlob for natural language processing and sentiment analysis. By extracting and analyzing news content, the pipeline processes articles to determine sentiment trends in AI-related reporting. Snowflake serves as the data warehouse, providing robust storage and scalability for large volumes of news data, while dbt (data build tool) enables efficient data transformation and modeling. The workflow involves retrieving news articles, performing sentiment analysis to categorize them as positive, negative, or neutral, and creating structured datasets that can reveal insights into media perception and narrative trends surrounding artificial intelligence technologies.\n\n## Architecture\n![Screenshot](news.png)\n\n## Project Setup and Installation\n\n### Prerequisites\n- Python 3.8+\n- Snowflake account\n- [News API key](https://newsapi.org/)\n\n\n### Installation Steps\n1. Clone the repository:\ngit clone https://github.com/arrismo/news-dashboard.git\ncd news-dashboard\n\n### Create Virtual Environment \n`python -m venv venv`\n`source venv/bin/activate`  # On Windows, use `venv\\Scripts\\activate`\n\n### Install dependencies\n`pip install -r requirements.txt`\n\n### Set up environment variables:\n\n# Create a .env file in the project root\n`touch .env`\n\n# Add the following:\n`NEWS_API_KEY=your_news_api_key` \\\n`SNOWFLAKE_ACCOUNT=your_snowflake_account` \\\n`SNOWFLAKE_USER=your_username` \\\n`SNOWFLAKE_PASSWORD=your_password` \n\n### Running the pipeline\n\nExtract News Data \\\n`python tap_news.py`\n\nRun dbt transformations \\\n### Configurations\n\nUpdate `news_project/models/` for custom transformations\n\n`dbt run`\n\n\n\nGenerate Setiment analysis \\\n`python sentiment_analysis.py`\n\nLoad to Snowflake \\\n`python target-snowflake.py`\n\n# Tableau Dashboard\n![Screenshot](dashboard.png)\nKey Components\n1. Sentiment by News Source\n\n    Description: A stacked bar chart showing the sentiment (positive, neutral, negative) of articles for each news source.\n    Purpose: Understand which sources produce the most articles and their overall tone.\n    Insights:\n        \"Business Insider\" and \"Quartz India\" are leading contributors.\n        Sentiment varies widely across sources, with most articles showing positive or negative sentiment.\n\n2. Sentiment Over Time\n\n    Description: A line chart tracking the average sentiment score of articles over a series of weeks. A red reference line marks the overall average sentiment.\n    Purpose: Monitor how sentiment changes over time, detect trends, and highlight anomalies.\n    Insights:\n        A sharp drop in sentiment occurred around November 17, followed by gradual recovery.\n        Sentiment peaked in early November and rose again in December.\n\n3. Number of Times Google is Mentioned\n\n    Description: A treemap visualizing the frequency of Google mentions in article titles, grouped by sentiment and news source.\n    Purpose: Identify which sources highlight Google the most and assess the tone of coverage.\n    Insights:\n        \"Quartz India\" leads with the highest number of mentions, all positive.\n        Other sources, such as \"Business Insider\" and \"Android Central,\" also contribute positively.\n\n4. Author Analysis\n\n    Description: A horizontal bar chart displaying the top authors by article count.\n    Purpose: Highlight the most prolific authors contributing to AI-related coverage.\n    Insights:\n        \"Britney Nguyen\" published the most articles, far exceeding other authors.\n        Authors like \"Kurt Knutsson\" and \"Rocio Fabbro\" also contributed significantly.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farrismo%2Fnews-dashboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farrismo%2Fnews-dashboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farrismo%2Fnews-dashboard/lists"}