{"id":30346433,"url":"https://github.com/connergroth/timbrality","last_synced_at":"2026-04-13T03:03:41.583Z","repository":{"id":273429778,"uuid":"919696878","full_name":"connergroth/Timbrality","owner":"connergroth","description":"Hybrid music recommender combining NMF collaborative filtering, two-tower content embeddings, audio feature synthesis, and meta-learning fusion for adaptive personalization.","archived":false,"fork":false,"pushed_at":"2025-10-08T03:01:07.000Z","size":68220,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-11T03:29:29.580Z","etag":null,"topics":["beautifulsoup","cloudscraper","docker","fastapi","lastfm-api","machine-learning","music","postgresql","python","pytorch","redis","scikit-learn","spotify-api","supabase"],"latest_commit_sha":null,"homepage":"https://timbrality.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/connergroth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-20T20:54:30.000Z","updated_at":"2025-08-21T07:27:06.000Z","dependencies_parsed_at":"2025-02-24T23:23:02.949Z","dependency_job_id":"8f91a1f2-0123-4d7b-b598-9deea211cd99","html_url":"https://github.com/connergroth/Timbrality","commit_stats":null,"previous_names":["connergroth/sonic-lens","connergroth/sonance","connergroth/tensoe","connergroth/timbre","connergroth/timbrality"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/connergroth/Timbrality","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/connergroth%2FTimbrality","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/connergroth%2FTimbrality/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/connergroth%2FTimbrality/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/connergroth%2FTimbrality/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/connergroth","download_url":"https://codeload.github.com/connergroth/Timbrality/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/connergroth%2FTimbrality/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31737850,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-12T22:19:12.206Z","status":"online","status_checked_at":"2026-04-13T02:00:06.623Z","response_time":93,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","cloudscraper","docker","fastapi","lastfm-api","machine-learning","music","postgresql","python","pytorch","redis","scikit-learn","spotify-api","supabase"],"created_at":"2025-08-18T15:38:39.830Z","updated_at":"2026-04-13T03:03:41.528Z","avatar_url":"https://github.com/connergroth.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/user-attachments/assets/5498a533-8497-46e6-a4f5-46d8cfacd6f3\" alt=\"Timbrality\" width=\"350px\" /\u003e\n  \u003ch1\u003eTimbrality - AI Powered Music Discovery\u003c/h1\u003e\n\u003c/div\u003e\n\n\u003e **Timbrality** — a machine learning-powered music recommendation engine that uses AI agents to create personalized music experiences.\n\nTimbrality is an intelligent music recommendation platform that combines data from **Spotify**, **Last.fm**, and **Album of the Year (AOTY)** to provide personalized music suggestions through conversational AI agents. The platform features a hybrid recommendation system powered by the **Timbral** ML engine and modern web interface built with React and Next.js.\n\n---\n\n## Features\n\n- **🤖 AI-Powered Music Agent**  \n  Conversational AI agent that understands music preferences and provides intelligent recommendations through natural language interactions.\n\n- **🎧 Personalized Recommendations**  \n  Hybrid recommendation system combining collaborative filtering and content-based approaches using listening behavior and audio features.\n\n- **🔗 Multi-Platform Integration**  \n  Seamlessly connects with **Spotify**, **Last.fm**, and **Album of the Year** to gather comprehensive music data and preferences.\n\n- **📱 Modern Web Interface**  \n  Clean, responsive UI built with React/Next.js featuring chat interface, playlist management, and real-time music discovery.\n\n- **🎵 Smart Playlist Creation**  \n  AI-generated playlists with Spotify integration for seamless music discovery and playlist management.\n\n- **⚡ High-Performance Backend**  \n  FastAPI-powered backend with multi-tier caching (Redis + in-memory), rate limiting, and async processing.\n\n- **📊 Rich Music Metadata**  \n  Enhanced with AOTY ratings, reviews, tags, and similar album data through sophisticated CloudScraper-based pipeline with rating count extraction.\n\n---\n\n## Architecture Overview\n\n### Backend (FastAPI)\n\n- **Multi-tier caching**: Redis primary + in-memory fallback\n- **Rate limiting**: 30 requests/minute via SlowAPI\n- **Database**: PostgreSQL with SQLAlchemy ORM and Alembic migrations\n- **Web scraping**: CloudScraper with async processing for comprehensive AOTY data extraction\n- **AI Agent**: NLP processor with tool registry for music recommendations\n\n### ML Service (Timbral Engine)\n\n- **Hybrid recommendation engine**: NMF collaborative + BERT content-based filtering\n- **Dedicated FastAPI service**: Port 8001 with ML-specific endpoints\n- **Model serving**: Redis-cached recommendations with explainability\n- **HTTP integration**: Proxied through main backend at `/timbral/*` routes\n\n### Frontend\n\n- **Main site**: Vite + React + shadcn/ui components\n- **Auth app**: Next.js application for OAuth flows\n- **State management**: React Context + Supabase auth\n\n### Key Components\n\n- `/backend/agent/`: AI agent core, tools, and NLP processing\n- `/backend/routes/`: API endpoints (agent, albums, playlists, users, timbral)\n- `/backend/services/`: Business logic (Spotify, Last.fm, ML, AOTY)\n- `/backend/ingestion/`: Data pipeline for music metadata\n- `/ml/timbral/`: Timbral ML engine (models, training, inference)\n- `/frontend/app/`: Next.js authentication and chat interface\n\n---\n\n## Tech Stack\n\n### 💻 Backend Technologies\n\n- **FastAPI** – Async Python web framework with automatic OpenAPI docs\n- **PostgreSQL + SQLAlchemy** – Relational database with async ORM\n- **Redis** – High-performance caching layer\n- **CloudScraper** – Advanced web scraping with anti-bot protection bypass for AOTY data\n- **Pydantic** – Data validation and serialization\n\n### 📊 Data Sources \u0026 APIs\n\n- **Spotify Web API** – User listening data, playlists, and audio features\n- **Last.fm API** – Scrobbling data and music discovery\n- **AOTY Custom Scraper** – Album ratings, reviews, rating counts, and comprehensive metadata\n- **Supabase** – Authentication and user management\n\n### 🤖 AI \u0026 Machine Learning\n\n- **AI Agent Architecture** – Tool-based agent for music recommendations\n- **NLP Processing** – Natural language understanding for music queries\n- **Timbral Engine** – Dedicated ML microservice with hybrid recommendation engine\n- **NMF Collaborative Filtering** – User-item matrix factorization for personalized suggestions\n- **BERT Content-Based Filtering** – Semantic understanding of music metadata and genres\n- **Model Explainability** – Built-in recommendation reasoning and explanations\n\n---\n\n## Model Design\n\n### 🔸 Collaborative Filtering (CF)\n\n- Built from play counts and listening behavior\n- Uses Non-negative Matrix Factorization (NMF)\n- Predicts latent user-track affinities\n\n### 🔹 Content-Based Filtering (CBF)\n\n- Embeds mood, genre, and tags using Sentence-BERT\n- Computes track similarity with cosine distance\n- Useful for cold-starts and fallback recs\n\n### 🔶 Hybrid Fusion\n\n- Weighted blending of CF + CBF scores\n- Tunable or learnable fusion logic\n- Produces rich, explainable recs per user or seed\n\n---\n\n## AI Agent\n\nTimbrality features an advanced AI agent with **dual-store memory architecture** that combines fast Redis working memory with durable PostgreSQL long-term storage for intelligent, context-aware music recommendations.\n\n### **Memory Architecture:**\n\n- **Redis Working Memory** – Sub-millisecond access to recent conversations (last 50-200 turns per chat)\n- **PostgreSQL + pgvector** – Semantic search and long-term memory with embeddings\n- **Context Assembly** – Intelligent retrieval combining recent turns, relevant memories, and user preferences\n- **Background Processing** – Async summarization, fact extraction, and topic analysis\n\n### **Agent Capabilities:**\n\n- **Conversational Memory** – Remembers user preferences, music tastes, and conversation context\n- **Semantic Understanding** – Uses Sentence-BERT embeddings for natural language processing\n- **Tool Integration** – Access to music databases, recommendation engines, and analysis tools\n- **Streaming Responses** – Real-time interaction with memory context updates\n- **Automatic Learning** – Extracts user facts, preferences, and music patterns over time\n\n### **Memory Features:**\n\n- **Working Memory** – Fast access to recent chat context with configurable TTL (24-72 hours)\n- **Long-term Memory** – Durable storage of important facts, preferences, and conversation summaries\n- **Semantic Search** – Vector similarity search for relevant context using pgvector\n- **Importance Scoring** – Memory prioritization based on user interaction patterns\n- **Topic Tracking** – Automatic extraction and trending of music-related topics\n\n### **API Endpoints:**\n\n```bash\nPOST /api/agent/chat          # Enhanced chat with memory integration\nPOST /api/agent/chat/stream   # Streaming responses with context\nGET  /api/agent/memory/stats  # User memory statistics\nPOST /api/agent/memory/process # Trigger background memory processing\n```\n\n---\n\n## AOTY Data Scraper\n\nTimbrality includes a sophisticated web scraper that extracts rich music metadata from **Album of the Year (AOTY)**, one of the most comprehensive music databases available. This custom scraper enhances the platform's recommendation capabilities with detailed album ratings, reviews, and metadata.\n\n### 🎯 What It Scrapes\n\n**Albums:**\n\n- User scores and rating counts (e.g., \"Based on 37,040 ratings\")\n- Critic reviews from major publications\n- Popular user reviews with like counts\n- Genre tags and metadata\n- Similar album recommendations\n- \"Must Hear\" designations\n\n**Artists:**\n\n- Overall user ratings and rating counts\n- Biography and formation details\n- Geographic location data\n- Complete discography listings\n- Genre classifications\n\n**Tracks:**\n\n- Individual track ratings and rating counts\n- Track-level metadata and features\n- Featured artist information\n- Track length and positioning data\n\n### 🛠 Technical Implementation\n\n**Web Scraping Engine:**\n\n- **CloudScraper** for bypassing anti-bot protection\n- **BeautifulSoup** for robust HTML parsing\n- **Async/await** processing for high performance\n- **Custom retry logic** with exponential backoff\n- **Rate limiting** to respect AOTY's servers\n\n**Data Models:**\n\n```python\nclass Album(BaseModel):\n    title: str\n    artist: str\n    user_score: Optional[float]\n    num_ratings: int\n    tracks: List[Track]\n    critic_reviews: List[CriticReview]\n    popular_reviews: List[AlbumUserReview]\n\nclass Track(BaseModel):\n    title: str\n    rating: Optional[int]\n    num_ratings: int\n    featured_artists: List[str]\n```\n\n**API Endpoints:**\n\n```bash\nGET /scraper/album?artist=Radiohead\u0026album=OK+Computer\nGET /scraper/similar?artist=Radiohead\u0026album=OK+Computer\nGET /scraper/artist?name=Radiohead\n```\n\n### 🔄 Data Pipeline Integration\n\n**Automated Population:**\n\n```bash\n# Add rating count columns to existing tables\npsql $DATABASE_URL -f backend/add_aoty_rating_counts.sql\n\n# Populate rating counts for all entities\npython backend/populate_aoty_rating_counts.py --type all --batch-size 10\n```\n\n**Database Enhancement:**\n\n- Adds `aoty_num_ratings` columns to albums, artists, and tracks tables\n- Batch processing with configurable limits\n- Resume capability for interrupted runs\n- Error handling and logging for production use\n\n**Caching Strategy:**\n\n- **Redis caching** for scraped data with configurable TTL\n- **In-memory fallback** when Redis is unavailable\n- **Smart cache keys** based on artist/album combinations\n- **Cache warming** for popular albums and artists\n\n### 🎵 Use Cases\n\n**Recommendation Enhancement:**\n\n- Weight recommendations by AOTY rating popularity\n- Surface critically acclaimed but undiscovered albums\n- Filter by minimum rating thresholds\n- Include review-based reasoning in AI responses\n\n**Music Discovery:**\n\n- \"Similar Albums\" recommendations from AOTY's algorithm\n- Genre-based exploration using AOTY's tagging system\n- Critical consensus analysis for new releases\n- User review sentiment for recommendation explanations\n\n**Data Quality:**\n\n- Cross-reference Spotify/Last.fm data with AOTY metadata\n- Resolve artist/album name discrepancies\n- Enrich sparse metadata with comprehensive AOTY details\n- Validate music catalog completeness\n\n---\n\n## Getting Started\n\n### Prerequisites\n\n- Python 3.8+\n- Node.js 18+\n- PostgreSQL\n- Redis (optional, falls back to in-memory cache)\n\n### Full Stack Setup (Docker)\n\n```bash\ndocker-compose up\n```\n\n### Manual Setup\n\n#### Backend\n\n```bash\ncd backend\npip install -r requirements.txt\nuvicorn main:app --reload  # Port 8000\n```\n\n#### ML Service\n\n```bash\ncd ml\npip install -r requirements.txt\npython main.py  # Port 8001\n```\n\n#### Frontend\n\n```bash\n# Main site\ncd frontend \u0026\u0026 npm install \u0026\u0026 npm run dev  # Port 3001\n\n# Auth app\ncd frontend/app \u0026\u0026 npm install \u0026\u0026 npm run dev  # Port 3000\n```\n\n### Environment Variables\n\nConfigure `.env` files in `backend/`, `ml/`, and `frontend/app/` directories with your API keys for Spotify, Last.fm, Supabase, and OpenAI.\n\n---\n\n## Current Status\n\n✅ **Completed:**\n\n- AI agent architecture with conversational interface\n- Multi-platform data integration (Spotify, Last.fm, AOTY)\n- Modern React/Next.js frontend with chat interface\n- FastAPI backend with caching and rate limiting\n\n🚧 **In Progress:**\n\n- Enhanced playlist management features\n- Performance optimizations and deployment preparation\n- Advanced ML model training and fine-tuning\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconnergroth%2Ftimbrality","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconnergroth%2Ftimbrality","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconnergroth%2Ftimbrality/lists"}