{"id":22432805,"url":"https://github.com/gappeah/xrp_sentiment_analysis","last_synced_at":"2025-03-27T07:43:03.249Z","repository":{"id":254195617,"uuid":"816380325","full_name":"gappeah/xrp_sentiment_analysis","owner":"gappeah","description":"This project aims to create a powerful tool for performing sentiment analysis on XRP-related news and social media data, followed by visualizing the results to gain insights into market trends. The analysis will involve extracting sentiment from various text sources.","archived":false,"fork":false,"pushed_at":"2025-02-02T01:36:07.000Z","size":561,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-02T02:24:41.862Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gappeah.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-17T16:17:29.000Z","updated_at":"2025-02-02T01:36:10.000Z","dependencies_parsed_at":"2024-08-22T01:04:59.877Z","dependency_job_id":"d5627c0f-efc1-4b47-8ff9-6e11d006f2b4","html_url":"https://github.com/gappeah/xrp_sentiment_analysis","commit_stats":null,"previous_names":["gappeah/oil-market-price-sentiment","gappeah/sentiment-analysis-and-visualisation-of-xrp","gappeah/xrp_sentiment_analysis"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gappeah%2Fxrp_sentiment_analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gappeah%2Fxrp_sentiment_analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gappeah%2Fxrp_sentiment_analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gappeah%2Fxrp_sentiment_analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gappeah","download_url":"https://codeload.github.com/gappeah/xrp_sentiment_analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245806081,"owners_count":20675293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-05T22:12:56.063Z","updated_at":"2025-03-27T07:43:03.236Z","avatar_url":"https://github.com/gappeah.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# XRP Sentiment Analysis Model\n\n## Introduction\n\nThis project is designed to perform sentiment analysis on news headlines related to XRP, a popular cryptocurrency. The goal is to analyze the sentiment of news articles from various sources to determine whether the overall sentiment is bullish, bearish, or neutral. This can be useful for traders, investors, and enthusiasts who want to gauge market sentiment and make informed decisions.\n\n## About XRP\n\nXRP is a digital asset and cryptocurrency that was created by Ripple Labs Inc. It is designed to facilitate fast, low-cost international payments and is often used by financial institutions for cross-border transactions. Unlike Bitcoin, which uses a proof-of-work consensus mechanism, XRP uses a unique consensus algorithm called the Ripple Protocol Consensus Algorithm (RPCA). This allows for faster transaction times and lower energy consumption compared to traditional blockchain systems.\n\n### Infrastructure and Governance\n\nXRP operates on a decentralized ledger known as the XRP Ledger (XRPL). The XRPL is maintained by a network of independent validators that confirm transactions and maintain the integrity of the ledger. Ripple Labs, the company behind XRP, plays a significant role in the development and promotion of the XRP ecosystem, but the ledger itself is decentralized and not controlled by any single entity.\n\n### How Transactions Work\n\nTransactions on the XRP Ledger are processed in a matter of seconds, with low transaction fees. When a user initiates a transaction, it is broadcast to the network of validators. These validators then work together to reach a consensus on the validity of the transaction. Once consensus is reached, the transaction is added to the ledger, and the balance of the involved accounts is updated.\n\n### Example of XRP Transaction (Pseudo Code)\n\n```python\n# Example code for an XRP transaction\nfrom xrpl.clients import JsonRpcClient\nfrom xrpl.wallet import Wallet\nfrom xrpl.models import Payment\nfrom xrpl.transaction import safe_sign_and_submit_transaction\nfrom xrpl.account import get_balance\nfrom xrpl.utils import xrp_to_drops\n\n# Connect to the XRPL testnet\nTESTNET_URL = \"https://s.altnet.rippletest.net:51234\"\nclient = JsonRpcClient(TESTNET_URL)\n\n# Generate sender and receiver wallets\nsender_wallet = Wallet.create()\nreceiver_wallet = Wallet.create()\n\nprint(\"Sender address:\", sender_wallet.classic_address)\nprint(\"Receiver address:\", receiver_wallet.classic_address)\n\n# Fund sender wallet using testnet faucet\nprint(\"Funding sender wallet...\")\nclient.request({\n    \"command\": \"faucet\",\n    \"address\": sender_wallet.classic_address\n})\n\n# Prepare payment transaction\npayment = Payment(\n    account=sender_wallet.classic_address,\n    amount=xrp_to_drops(22),  # Convert XRP to drops (1 XRP = 1,000,000 drops)\n    destination=receiver_wallet.classic_address,\n    fee=\"10\"  # Minimum transaction fee in drops\n)\n\n# Sign and submit the transaction\nprint(\"Submitting transaction...\")\nresponse = safe_sign_and_submit_transaction(payment, client, sender_wallet)\n\n# Check transaction results\nif response.is_successful():\n    result = response.result\n    print(\"Transaction successful!\")\n    print(f\"Transaction hash: {result['hash']}\")\n    print(f\"Sender balance: {get_balance(sender_wallet.classic_address, client)} XRP\")\n    print(f\"Receiver balance: {get_balance(receiver_wallet.classic_address, client)} XRP\")\nelse:\n    print(\"Transaction failed:\")\n    print(response.result)\n```\n\n## Project Description\n\nThis project scrapes news headlines related to XRP from multiple sources, including Crypto News, The Crypto Basic, and Yahoo Finance. It then performs sentiment analysis on these headlines using two popular sentiment analysis tools: TextBlob and VADER. The results are visualized using Matplotlib and Seaborn, and the data is saved to a CSV file for further analysis.\n\n### How to Launch the Project\n\n1. **Clone the Repository**: Start by cloning this repository to your local machine.\n   ```bash\n   git clone https://github.com/gappeah/xrp-sentiment-analysis.git\n   ```\n\n2. **Install Dependencies**: Ensure you have Python installed, then install the required dependencies using pip.\n   ```bash\n   pip install pandas numpy beautifulsoup4 requests textblob nltk vaderSentiment matplotlib seaborn\n   ```\n\n3. **Download NLTK Resources**: The project uses NLTK for text processing. You need to download the necessary NLTK resources.\n   ```python\n   import nltk\n   nltk.download('punkt')\n   nltk.download('stopwords')\n   nltk.download('wordnet')\n   ```\n\n4. **Run the Script**: Execute the script to scrape news headlines, perform sentiment analysis, and generate visualizations.\n   ```bash\n   python xrp_sentiment_analysis.py\n   ```\n\n5. **View Results**: The script will generate a CSV file (`crypto_headlines_sentiment_analysis.csv`) containing the sentiment analysis results. It will also display visualizations of the sentiment scores over time.\n\nHere is a detailed breakdown of the **code differences** between the **new version** and the **old version** of the XRP Sentiment Analysis Model. The differences are categorized by functionality and highlighted with explanations.\n\n---\n\n## 1. **Date Handling and Time-Based Analysis**\n\n### New Version\n```python\n# Function to scrape headlines and dates from Crypto News\ndef scrape_crypto_news():\n    headlines = []\n    dates = []\n    for page in range(1, 15):\n        url = f\"https://crypto.news/page/{page}/?s=xrp\"\n        response = requests.get(url)\n        soup = BeautifulSoup(response.text, 'html.parser')\n        for h3, date_div in zip(soup.find_all('h3'), soup.find_all('div', class_='search-result-loop__date')):\n            headline = h3.get_text()\n            raw_date = date_div.get_text(strip=True).split(\" at \")[0]  # Remove time\n            parsed_date = datetime.strptime(raw_date, \"%B %d, %Y\").strftime('%d/%m/%Y')\n            headlines.append(headline)\n            dates.append(parsed_date)\n    return headlines, dates\n```\n\n### Old Version\n```python\n# Function to scrape headlines from Crypto News\ndef scrape_crypto_news():\n    headlines = []\n    for page in range(1, 6):\n        url = f\"https://crypto.news/page/{page}/?s=xrp\"\n        response = requests.get(url)\n        soup = BeautifulSoup(response.text, 'html.parser')\n        for h3 in soup.find_all('h3'):\n            headlines.append(h3.get_text())\n    return headlines\n```\n\n### Key Differences:\n- **New Version**: Extracts **dates** alongside headlines and parses them into a consistent format (`DD/MM/YYYY`).\n- **Old Version**: Only extracts headlines, with no date handling.\n\n---\n\n## 2. **Yahoo Finance Date Parsing**\n\n### New Version\n```python\n# Function to scrape headlines and dates from Yahoo Finance\ndef scrape_yahoo_finance():\n    yahoo_headlines = []\n    dates = []\n    url = \"https://finance.yahoo.com/quote/XRP-USD/news/\"\n    response = requests.get(url)\n    soup = BeautifulSoup(response.text, 'html.parser')\n    articles = soup.find_all('h3', class_='Mb(5px)')\n    times = soup.find_all('div', class_='publishing yf-1weyqlp')\n\n    for article, time_div in zip(articles[:50], times[:50]):  # Limit to 20 articles\n        headline = article.get_text()\n        time_text = time_div.get_text(strip=True).split(\"•\")[-1].strip()\n        # Calculate date from \"XX hours/days ago\"\n        today = datetime.today()\n        if \"hour\" in time_text:\n            hours_ago = int(re.search(r\"\\d+\", time_text).group())\n            article_date = today - timedelta(hours=hours_ago)\n        elif \"day\" in time_text:\n            days_ago = int(re.search(r\"\\d+\", time_text).group())\n            article_date = today - timedelta(days=days_ago)\n        elif \"week\" in time_text:\n            weeks_ago = int(re.search(r\"\\d+\", time_text).group())\n            article_date = today - timedelta(weeks=weeks_ago)\n        elif \"month\" in time_text:\n            months_ago = int(re.search(r\"\\d+\", time_text).group())\n            article_date = today - timedelta(days=months_ago * 30)\n        else:\n            article_date = today\n\n        parsed_date = article_date.strftime('%d/%m/%Y')\n        yahoo_headlines.append(headline)\n        dates.append(parsed_date)\n\n    return yahoo_headlines, dates\n```\n\n### Old Version\n```python\n# Function to scrape headlines from Yahoo Finance\ndef scrape_yahoo_finance():\n    yahoo_headlines = []\n    url = \"https://finance.yahoo.com/quote/XRP-USD/news/\"\n    response = requests.get(url)\n    soup = BeautifulSoup(response.text, 'html.parser')\n    for h3 in soup.find_all('h3', class_='Mb(5px)'):\n        yahoo_headlines.append(h3.get_text())\n    return yahoo_headlines\n```\n\n### Key Differences:\n- **New Version**: Dynamically calculates the date of each article based on relative time descriptions (e.g., \"2 hours ago\").\n- **Old Version**: Does not extract or handle dates.\n\n---\n\n## 3. **Sentiment Categorization**\n\n### New Version\n```python\n# Sentiment categorization in the new version\nsentiment_category = (\n    \"Bullish\" if average_score \u003e= 0.5 else\n    \"Slightly Bullish\" if 0.2 \u003c average_score \u003c 0.4 else\n    \"Neutral\" if -0.2 \u003c= average_score \u003c= 0.2 else\n    \"Slightly Bearish\" if -0.5 \u003c= average_score \u003c -0.2 else\n    \"Bearish\"\n)\n```\n\n### Old Version\n```python\n# Sentiment categorization in the old version\nsentiment_category = (\n    \"Bullish\" if average_score \u003e 0.5 else\n    \"Slightly Bullish\" if 0.2 \u003c average_score \u003c= 0.5 else\n    \"Neutral\" if -0.2 \u003c= average_score \u003c= 0.2 else\n    \"Slightly Bearish\" if -0.5 \u003c= average_score \u003c -0.2 else\n    \"Bearish\"\n)\n```\n\n### Key Differences:\n- **New Version**: Uses a more granular categorization with clearer boundaries (e.g., `\u003e= 0.5` for \"Bullish\").\n- **Old Version**: Uses slightly different boundaries (e.g., `\u003e 0.5` for \"Bullish\").\n\n---\n\n## 4. **Visualization**\n\n### New Version\n```python\n# Visualization in the new version\nfig, ax = plt.subplots(figsize=(15, 8))\n\nsns.barplot(x=\"date\", y=\"vader_score\", data=df, color=\"blue\", label=\"Vader Score\", ax=ax)\nsns.barplot(x=\"date\", y=\"textblob_score\", data=df, color=\"orange\", label=\"TextBlob Score\", ax=ax)\n\nax.set_title(\"Sentiment Analysis of Cryptocurrency Headlines\")\nax.set_xlabel(\"Dates (DD/MM/YYYY)\")\nax.set_ylabel(\"Sentiment Score\")\nax.tick_params(axis='x', rotation=45)  # Rotate x-axis labels for clarity\nax.legend()\n\nplt.tight_layout()\nplt.show()\n```\n\n### Old Version\n```python\n# Visualization in the old version\nfig, ax = plt.subplots(figsize=(10, 6))\n\nsns.barplot(x=\"headline\", y=\"vader_score\", data=df, label=\"Vader Score\", ax=ax)\nsns.barplot(x=\"headline\", y=\"textblob_score\", data=df, label=\"TextBlob Score\", ax=ax)\n\nax.set_title(\"Sentiment Analysis of Cryptocurrency Headlines\")\nax.set_xlabel(\"Headlines\")\nax.set_ylabel(\"Sentiment Score\")\nax.legend()\n```\n\n### Key Differences:\n- **New Version**: Visualizes sentiment scores **over time** with dates on the x-axis.\n- **Old Version**: Visualizes sentiment scores per headline, which is less informative.\n\n---\n\n## 5. **Data Storage**\n\n### New Version\n```python\n# Data storage in the new version\ndata.append({\n    'headline': headline,\n    'date': date,\n    'textblob_score': textblob_score,\n    'vader_score': vader_score,\n    'average_score': average_score,\n    'sentiment_category': sentiment_category\n})\n```\n\n### Old Version\n```python\n# Data storage in the old version\ndata.append({\n    'headline': headline,\n    'cleaned_headline': cleaned_headline,\n    'textblob_score': textblob_score,\n    'vader_score': vader_score,\n    'average_score': average_score,\n    'sentiment_category': sentiment_category\n})\n```\n\n### Key Differences:\n- **New Version**: Includes **dates** in the stored data.\n- **Old Version**: Does not include dates.\n\n---\n\n## 6. **Descriptive Statistics**\n\n### New Version\n```python\n# Descriptive statistics in the new version\ndef descriptive_statistics():\n    print(\"Descriptive Statistics:\\n\")\n    print(df[['textblob_score', 'vader_score', 'average_score']].describe())\n```\n\n### Old Version\n```python\n# Descriptive statistics in the old version\ndef descriptive_statistics(df):\n    print(\"Descriptive Statistics of Sentiment Scores:\")\n    print(df[['textblob_score', 'vader_score', 'average_score']].describe())\n```\n\n### Key Differences:\n- **New Version**: Simplified function call (no `df` parameter needed).\n- **Old Version**: Requires `df` as a parameter.\n\n\n### Visualizations\nBetween the old version and new version of the codebase improvement in the visualisation results in better quality graphs. \nThe project includes several visualizations to help you understand the sentiment trends:\n\n1. **Sentiment Scores Over Time**: A bar plot showing the sentiment scores (TextBlob and VADER) over time.\n2. **TextBlob Sentiment Distribution**: A histogram showing the distribution of TextBlob sentiment scores.\n3. **VADER Sentiment Distribution**: A histogram showing the distribution of VADER sentiment scores.\n4. **TextBlob vs VADER Sentiment Scores**: A scatter plot comparing TextBlob and VADER sentiment scores.\n\n### Old Version\n![Figure_3](https://github.com/user-attachments/assets/4bd6b46b-2b89-43d6-9200-7077c3cf2147)\n![Figure_2](https://github.com/user-attachments/assets/4567eee1-3ebb-41c3-8800-6f1ec40be8a0)\n![Figure_1](https://github.com/user-attachments/assets/50645ed0-e4e4-4bcd-848e-9808f44a5002)\n![Figure_4](https://github.com/user-attachments/assets/91b25b52-6f09-46e3-a1d8-e6feade6a0e5)\n\n### New Version\n![Figure_3_Revised](https://github.com/user-attachments/assets/713ee340-e33a-4c6e-b7de-34dddfab540b)\n![Figure_2_Revised](https://github.com/user-attachments/assets/3204cab1-e0db-4efc-bd58-2706f2a6dd76)\n![Figure_1_Revised](https://github.com/user-attachments/assets/1b1efcba-c118-447e-8287-715eefc5a75a)\n![Figure_4_Revised](https://github.com/user-attachments/assets/bc81c202-face-4d93-8080-283dc7e15d65)\n\n---\n\n## Summary of Code Differences\n\n| Feature                     | New Version Code Changes                                                                 | Old Version Code Changes                                                                 |\n|-----------------------------|-----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|\n| **Date Handling**            | Extracts and parses dates from all sources.                                             | No date handling.                                                                       |\n| **Yahoo Finance Parsing**    | Dynamically calculates dates from relative time descriptions.                           | No date extraction.                                                                     |\n| **Sentiment Categorization** | More granular and clear boundaries for sentiment categories.                            | Simpler categorization with less granularity.                                           |\n| **Visualization**            | Time-based visualizations with dates on the x-axis.                                     | Headline-based visualizations without time context.                                     |\n| **Data Storage**             | Includes dates in the stored data.                                                      | Does not include dates.                                                                 |\n| **Descriptive Statistics**   | Simplified function call.                                                               | Requires `df` as a parameter.                                                           |\n\n---\n\n## Conclusion\n\nThis project provides a comprehensive analysis of sentiment trends in XRP-related news headlines. By combining web scraping, sentiment analysis, and data visualization, it offers valuable insights into market sentiment, which can be used to inform trading and investment decisions.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgappeah%2Fxrp_sentiment_analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgappeah%2Fxrp_sentiment_analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgappeah%2Fxrp_sentiment_analysis/lists"}