{"id":45201970,"url":"https://github.com/pmatorras/financial-sentiment-llm","last_synced_at":"2026-02-20T14:03:18.613Z","repository":{"id":329266225,"uuid":"1117477390","full_name":"pmatorras/financial-sentiment-llm","owner":"pmatorras","description":"Multi-task FinBERT achieving 85% accuracy on financial sentiment (News/Social/Forums). Explores LoRA efficiency and domain adaptation for trading signals.","archived":false,"fork":false,"pushed_at":"2026-01-22T13:15:26.000Z","size":102,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-23T05:38:22.188Z","etag":null,"topics":["bert","deep-learning","finance","financial-nlp","finbert","fine-tuning","llm","lora","machine-learning","natural-language-processing","pytorch","sentiment-analysis","transformers"],"latest_commit_sha":null,"homepage":"https://pablo.matorras.com/projects/finsentiment.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pmatorras.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-16T11:17:46.000Z","updated_at":"2026-01-22T13:25:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/pmatorras/financial-sentiment-llm","commit_stats":null,"previous_names":["pmatorras/financial-sentiment-llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pmatorras/financial-sentiment-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmatorras%2Ffinancial-sentiment-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmatorras%2Ffinancial-sentiment-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmatorras%2Ffinancial-sentiment-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmatorras%2Ffinancial-sentiment-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pmatorras","download_url":"https://codeload.github.com/pmatorras/financial-sentiment-llm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmatorras%2Ffinancial-sentiment-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29653477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T09:27:29.698Z","status":"ssl_error","status_checked_at":"2026-02-20T09:26:12.373Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","deep-learning","finance","financial-nlp","finbert","fine-tuning","llm","lora","machine-learning","natural-language-processing","pytorch","sentiment-analysis","transformers"],"created_at":"2026-02-20T14:03:15.671Z","updated_at":"2026-02-20T14:03:18.608Z","avatar_url":"https://github.com/pmatorras.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Financial Sentiment LLM\n[![Open in Spaces](https://huggingface.co/datasets/huggingface/badges/resolve/main/open-in-hf-spaces-md.svg)](https://huggingface.co/spaces/pmatorras/financial-sentiment-demo)\n\n![Python](https://img.shields.io/badge/python-3.10-blue.svg)\n![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-ee4c2c.svg)\n![Transformers](https://img.shields.io/badge/🤗%20Transformers-4.0+-yellow.svg)\n\nFine-tuning FinBERT for financial sentiment analysis to integrate sentiment features into equity selection pipelines ([financial-ML](https://github.com/pmatorras/financial-ML)).\n\n\n## Key Results (Phase 2)\n\n*Optimisation Complete - Jan 22, 2026*\n\n\n| Category | Model | Accuracy | Storage | Key Strengths |\n| :-- | :-- | :-- | :-- | :-- |\n| **Best Performance** | **FinBERT Multi-Task** | **85.4%** | ~420 MB | -  Best generalisation (+4.4%)\u003cbr\u003e-  Necessary for forum discussions (FiQA) |\n| **Best Efficiency** | **FinBERT LoRA (r16)** | 83.2% | **~5 MB** | -  **99% Storage Savings**\u003cbr\u003e-  Matches performance on News/Twitter |\n\n**🚀 [Try Live Demo](https://pablo.matorras.com/projects/finsentiment.html)** - Test the model on real financial text\n\n**Next Steps:** Phase 3 - Deployment \\\u0026 Integration into Financial-ML pipeline.\n\n## Pipeline Architecture\n\n```mermaid\nflowchart LR\n    subgraph Data[\"Training Data (40K+ samples)\"]\n        PB[Financial PhraseBank\u003cbr/\u003e\u003ci\u003eProfessional News\u003c/i\u003e]\n        TW[Twitter Financial\u003cbr/\u003e\u003ci\u003eSocial Media\u003c/i\u003e]\n        FQ[FiQA\u003cbr/\u003e\u003ci\u003eForum Discussions\u003c/i\u003e]\n    end\n    \n    subgraph Training[\"Offline Training (Local/GPU)\"]\n        PB --\u003e Train[FinBERT Multi-Task\u003cbr/\u003eClassification + Regression]\n        TW --\u003e Train\n        FQ --\u003e Train\n        Train --\u003e|Save| Model[Model Checkpoint\u003cbr/\u003e420MB .pt file]\n    end\n    \n    subgraph Serving[\"Production Serving (HF Spaces)\"]\n        Model --\u003e|Load| App[Gradio Interface]\n        App --\u003e|Inference| Logic[Sentiment Output\u003cbr/\u003e3-class + score]\n    end\n    \n    Browser[User] --\u003e|Input Text| App\n    App --\u003e|JSON Response| Browser\n\n```\n**Design Decisions:**\n- **Offline Training**: Computationally expensive fine-tuning runs locally on GPU (2 min/epoch), producing versioned model checkpoints\n- **Lightweight Serving**: Hugging Face Spaces hosts the inference endpoint with automatic scaling and global CDN\n- **Model Versioning**: Training outputs are saved as `.pt` files, enabling rollback and A/B testing\n- **Live Demo**: Try the deployed model at [pablo.matorras.com/projects/finsentiment.html](https://pablo.matorras.com/projects/finsentiment.html)\n## Model Architecture \u0026 Training\n\nTo handle the diverse nature of financial text, this project implements a **Multi-Task FinBERT** architecture. Unlike standard classifiers, this model shares a BERT backbone with two task-specific heads:\n\n1. **Classification Head:** Predicts Negative/Neutral/Positive (for news/tweets).\n2. **Regression Head:** Predicts continuous sentiment scores (for FiQA).\n\n**Training data**\n\n- [Financial PhraseBank](https://huggingface.co/datasets/takala/financial_phrasebank) (Professional news)\n- [Twitter Financial News](https://huggingface.co/datasets/zeroshot/twitter-financial-news-sentiment) (Social media)\n- [FiQA Sentiment](https://huggingface.co/datasets/TheFinAI/fiqa-sentiment-classification) (Forum discussions)\n\n\u003e ***Note**: Training uses 2:1 Twitter/PhraseBank ratio with decoupled sampling. See [EXPERIMENTS.md](EXPERIMENTS.md) for methodology.*\n\n**Single vs Multi-task**\n\nWe compared our Multi-Task architecture against a standard Single-Task FinBERT baseline. The Multi-Task approach yielded a **+4.4%** increase in overall accuracy (85.4% vs 81.0%). The improvement was consistent across all data sources but was especially relevant in Twitter/Social Media, where accuracy increased by **+6.1%** (from 77.2% to 83.3%)\n\n**Technical Strategy**\n- **Multi-Task Loss:** We combine Cross-Entropy (for classes) and MSE (for scores) to handle diverse data formats simultaneously.\n- **Robust Training:** Includes Early Stopping (patience=2) to prevent overfitting and fixed random seeds for Reproducibility.\n- **Loss weighting:** Multi-task loss uses Cross-Entropy (classification) + weighted MSE (regression); defaults are rescaled to `1/10` for implementation convenience while preserving the original 1:10 ratio.\n\nflowchart LR\n    subgraph Training[\"Offline Training (Local/GPU)\"]\n        Raw[Financial PhraseBank] --\u003e Train[FinBERT Fine-Tuning]\n        Train --\u003e|Export| Model[Model Artifact .bin]\n    end\n\n    subgraph Serving[\"Model Serving (Hugging Face Spaces)\"]\n        Model --\u003e|Load| App[FastAPI/Gradio App]\n        App --\u003e|Inference| Logic[Sentiment Logic]\n    end\n\n    subgraph Client[\"User Interface\"]\n        Browser[Web Browser] --\u003e|Input Text| App\n        App --\u003e|JSON/Prediction| Browser\n    end\n\n**Lightweight alternative (LoRA)**\n\nWe also tested **Low-Rank Adaptation (LoRA)** as a lightweight alternative to determine if we could maintain performance while drastically reducing model size.\n- **Concept:** Freezes the 110M FinBERT parameters and injects small trainable rank decomposition matrices.\n- **Result:** We achieved **83.2% accuracy** (vs 85.4% baseline) using only **5MB** of trainable weights.\n- **Trade-off:**  LoRA matches the full model on standard news text but consistently underperforms on complex forum discussions (**FiQA**), both in the single and multi-task architectures, suggesting the full parameter space is needed for this specific domain.\n\n## Validation Results\n*Phase 2 (Jan 2026)*\n\nWe benchmarked multiple architectures on a fixed test set (N=1,896) to balance accuracy, efficiency, and robustness. \n\n| Model | Architecture | Overall Acc | F1-Score | PhraseBank | Twitter | FiQA | Storage |\n| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |\n| **FinBERT (Full)** | **Multi-Task** | **85.4%** | **0.83** | 95.9% | **83.3%** | **81.5%** | ~420 MB | -\u003echecked\n| **FinBERT (LoRA)** | Multi-Task | 83.2% |0.80 | **97.1%** | 80.5% | 72.6% | **~5 MB** |\n| FinBERT (Full) | Single-Task | 81.0% | 0.77 | 95.3% | 77.2% | 80.6% | ~420 MB | \n| FinBERT (LoRA) | Single-Task | 79.0% | 0.79 | 95.8% | 75.4% | 73.4% | ~5 MB |\n| BERT-Base | Multi-Task | 83.0% | 0.81 | 92.7% | 81.9% | 75.0% | 110M | ~420 MB |\n| DistilBERT | Multi-Task | 82.0% | 0.79 | 90.3% | 80.7% | 75.0% | ~260 MB |\n\n**Key results**\n- **Best Overall Performance:** FinBERT Multi-Task achieves 85.4% accuracy, outperforming generic BERT (+2.4%) and DistilBERT (+3.4%), especially on professional news text.\n- **Production (Cloud):** Use **FinBERT (Full)** for maximum robustness.\n- **Production (Edge/Lightweight):** Use **FinBERT (LoRA)**. While total inference parameters are similar, the **trainable/storage footprint is 99% smaller (5MB)**.\n\n\n### Detailed Performance comparison\nThe Full Model maintains stronger precision across \"Negative\" and \"Neutral\" classes, while LoRA achieves near-parity on \"Positive\" sentiment detection (0.87 F1 vs 0.89). This performance gap reflects LoRA's constrained capacity on complex financial text.\n\n| Class | **FinBERT**  |  | | **LoRA** | | |\n|-------|-----------|--------|----|-----------|--------|----|\n|       | Precision | Recall | F1 | Precision | Recall | F1 |\n| Negative | 0.78 | 0.77 | 0.77 | 0.72 | 0.76 | 0.74 |\n| Neutral | 0.87 | 0.78 | 0.82 | 0.81 | 0.78 | 0.79 |\n| Positive | 0.87 | 0.91 | 0.89 | 0.86 | 0.87 | 0.87 |\n\n\n## Project Structure\n```bash\n├── src/\n│   └── finsentiment/\n│       ├── cli/                  # CLI entrypoints\n│       │   ├── train.py\n│       │   ├── evaluate.py\n│       │   └── parser.py\n│       ├── datasets/             # Data loading + splitting + dataset class\n│       │   ├── __init__.py\n│       │   ├── load.py           # Download/load HF datasets → pandas\n│       │   ├── preprocessing.py  # Split / balance / combine datasets\n│       │   ├── registry.py       # List of datasets used with weights\n│       │   ├── sentiment.py      # Dataset wrapper (task_type-aware)\n│       │   └── clean_data.py     # Optional cleaning utilities (default OFF)\n│       ├── modeling/             # Model definition(s)\n│       │   ├── __init__.py\n│       │   ├── bert.py           # FinancialSentimentModel (cls + reg heads)\n│       │   └── bert.py           # LoRAFinancialSentimentModel (cls + reg heads)\n│       ├── training/             # Training loop(s)\n│       │   ├── __init__.py\n│       │   └── trainer.py\n│       ├── evaluation/           # Metrics\n│       │   ├── __init__.py\n│       │   └── metrics.py\n│       ├── config.py             # Global configuration\n│       ├── main.py               # Application entry point\n│       └── __main__.py           # python -m finsentiment\n├── data/\n│ └── raw/ # Auto-downloaded datasets\n├── scripts/       # Stand alone scripts to run\n├── models/        # Saved checkpoints\n├── logs/          # Evaluation results  \n├── app.py         # HuggingFace Space demo app\n├── Dockerfile     # Demo containerisation\n├── .gitignore     # Git ignore rules\n├── EXPERIMENTS.md # Description of each experiment made\n├── PROJECT.md     # Detailed roadmap \u0026 progress\n└── README.md      # This file\n```\n\n## Installation\n\n### Install PyTorch\n\nChoose based on your hardware:\n\n**GPU (NVIDIA CUDA):**\n```bash\n# GPU version\npip install torch --index-url https://download.pytorch.org/whl/cu121\n\n# CPU version  \npip install torch --index-url https://download.pytorch.org/whl/cpu\n\n```\n### Install Project\n\n```bash\n# Clone and setup\ngit clone https://github.com/pmatorras/financial-sentiment-llm.git\ncd financial-sentiment-llm\npython -m venv .venv\nsource .venv/bin/activate # Windows: .venv\\Scripts\\activate\npip install -e . \npip install -e \".[dev]\" #with dev dependencies\n```\n\n### Verify Setup\n```bash\npython -c \"import torch; print('CUDA available:', torch.cuda.is_available())\"\n```\n\n## Usage\n\n### Training\n```bash\n# Train with FinBERT (default, recommended)\npython -m finsentiment train\n\n# Train with LoRA (Efficient - 5MB checkpoint)\npython -m finsentiment train --model finbert-lora-tuned\n\n# Experiment with other models\npython -m finsentiment train --model-name bert        # Generic BERT\npython -m finsentiment train --model-name distilbert  # Lightweight variant\n\n# All models support multi-task architecture (default) or single-task\npython -m finsentiment train --model-type single\n```\n### Run the Demo locally\n```bash\n# Install demo dependencies\npip install -e \".[demo]\"\nmv models/finbert_multi_task_model.pt models/multi_task_model_phase2.pt #Or any other chosen model\n# Launch the web interface\npython app.py\n# Opens at http://0.0.0.0:7860\n```\n\u003e The local demo runs the same interface as the [**live HuggingFace Space**](https://huggingface.co/spaces/pmatorras/financial-sentiment-demo)\n\n**Training Time:**\n- **GPU (RTX 4050):** ~2 minutes per epoch\n- **CPU (Intel Core Ultra 7):** ~20 minutes per epoch\n\u003e *Note: DistilBERT showed unexpectedly slower training in this configuration, likely due to dataloader bottleneck.*\n### Evaluation\n```bash\n#Evaluate trained model on test set\npython -m finsentiment evaluate # Defaults to Multi-Task architecture. Use --model-type multi for baseline.\n```\n\n\n\n## Resources\n\n- [PROJECT.md](PROJECT.md) - Detailed roadmap and progress tracking\n- [FinBERT Paper](https://arxiv.org/abs/1908.10063) - Financial domain BERT\n- [HuggingFace PEFT docs](https://huggingface.co/docs/peft) - For upcoming LoRA implementation\n\n---\n\n**Note**: This is a learning project to develop production-grade LLM and NLP skills. Documentation and results are continuously updated as experimentation progresses.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmatorras%2Ffinancial-sentiment-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpmatorras%2Ffinancial-sentiment-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmatorras%2Ffinancial-sentiment-llm/lists"}