{"id":32037693,"url":"https://github.com/cterence/lastfm-scrobble-deduplicator","last_synced_at":"2026-04-17T13:32:46.333Z","repository":{"id":318963790,"uuid":"1036990679","full_name":"cterence/lastfm-scrobble-deduplicator","owner":"cterence","description":"Automatically detects and removes duplicate scrobbles from your Last.fm profile","archived":false,"fork":false,"pushed_at":"2026-04-15T12:13:44.000Z","size":91,"stargazers_count":1,"open_issues_count":9,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-15T14:20:10.249Z","etag":null,"topics":["browserless","docker","golang","lastfm","scrobble","scrobbles"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cterence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-12T22:34:19.000Z","updated_at":"2025-12-31T09:46:11.000Z","dependencies_parsed_at":"2026-03-08T11:10:37.082Z","dependency_job_id":null,"html_url":"https://github.com/cterence/lastfm-scrobble-deduplicator","commit_stats":null,"previous_names":["cterence/lastfm-scrobble-deduplicator"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/cterence/lastfm-scrobble-deduplicator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cterence%2Flastfm-scrobble-deduplicator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cterence%2Flastfm-scrobble-deduplicator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cterence%2Flastfm-scrobble-deduplicator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cterence%2Flastfm-scrobble-deduplicator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cterence","download_url":"https://codeload.github.com/cterence/lastfm-scrobble-deduplicator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cterence%2Flastfm-scrobble-deduplicator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31931408,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T12:37:54.787Z","status":"ssl_error","status_checked_at":"2026-04-17T12:37:25.095Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["browserless","docker","golang","lastfm","scrobble","scrobbles"],"created_at":"2025-10-17T09:51:18.143Z","updated_at":"2026-04-17T13:32:46.326Z","avatar_url":"https://github.com/cterence.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Last.fm Scrobble Deduplicator\n\nA Go-based tool that automatically detects and removes duplicate scrobbles from your Last.fm profile using intelligent duplicate detection algorithms and browser automation.\n\n## 🎯 Why This Tool Exists\n\nMany music scrobblers create duplicate entries due to:\n\n- Network interruptions during scrobbling\n- Multiple devices scrobbling the same track simultaneously\n- Scrobblers retrying failed requests\n- Manual corrections that create duplicates\n\nInstead of manually fixing these issues or dealing with problematic scrobblers, this tool periodically cleans up your profile by intelligently identifying and removing duplicate scrobbles.\n\n## ✨ Features\n\n- **Smart Duplicate Detection**: Based on track duration and timing thresholds\n- **Incomplete Scrobble Detection**: Finds scrobbles that were interrupted before completion\n- **Multiple Cache Backends**: Redis, file-based, and in-memory caching\n- **MusicBrainz Integration**: Automatic track duration fetching\n- **Browser Automation**: Chrome/Chromium interaction with Last.fm\n- **Telegram Notifications**: Optional completion notifications\n- **Docker Support**: Full containerization with Redis and Chromium\n- **Cross-platform**: Linux, macOS, and Windows binaries\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- **Binary releases**: Chrome or Chromium browser (required for chromedp)\n- **Source builds**: Go 1.24+, Chrome or Chromium browser\n- Docker setup (alternative to local browser)\n- Last.fm account credentials\n\n### Using Pre-built Binaries (Recommended)\n\n1. **Download from [GitHub Releases](https://github.com/cterence/scrobble-deduplicator/releases):**\n   - Linux: `scrobble-deduplicator-linux-amd64` or `scrobble-deduplicator-linux-arm64`\n   - macOS: `scrobble-deduplicator-darwin-amd64` or `scrobble-deduplicator-darwin-arm64`\n   - Windows: `scrobble-deduplicator-windows-amd64.exe`\n\n2. **Setup:**\n\n   ```bash\n   chmod +x scrobble-deduplicator-linux-amd64  # Linux/macOS\n   curl -O https://raw.githubusercontent.com/cterence/scrobble-deduplicator/main/config.example.yaml\n   mv config.example.yaml config.yaml\n   # Edit config.yaml with your credentials\n   ```\n\n3. **Run:**\n\n   ```bash\n   ./scrobble-deduplicator-linux-amd64 -c config.yaml\n   ```\n\n### Using Docker\n\n```bash\n# Download the production compose file\ncurl -O https://raw.githubusercontent.com/cterence/scrobble-deduplicator/main/compose.yaml\n\n# Download and edit configuration\ncurl -O https://raw.githubusercontent.com/cterence/scrobble-deduplicator/main/config.example.yaml\nmv config.example.yaml config.yaml\n# Edit config.yaml with your credentials\n\n# Run with Docker\ndocker compose up\n```\n\n### Building from Source\n\n```bash\nnix develop\ngo build -o scrobble-deduplicator .\n./scrobble-deduplicator -c config.yaml\n```\n\n## 📋 Configuration\n\nCreate `config.yaml`:\n\n```yaml\ncacheType: inmemory  # redis|file|inmemory\nlastfm:\n  username: your_username\n  password: your_password\nfrom: 01-01-2025  # Optional date range\nto: 01-03-2025\nbrowserHeadful: false # Set to true to open a browser window\ncanDelete: false  # Set to true to enable deletion\nduplicateThreshold: 90  # Percentage threshold\ncompleteThreshold: 50   # Completion threshold\ndataDir: ./data\ntelegramBotToken: \"\"  # Optional\ntelegramChatID: \"\"    # Optional\n```\n\n### Command Line Options\n\n```bash\n# Basic usage\n./scrobble-deduplicator -u username -p password\n\n# With date range\n./scrobble-deduplicator -u username -p password --from 01-01-2025 --to 01-03-2025\n\n# Enable deletion\n./scrobble-deduplicator -u username -p password --delete\n\n# Custom thresholds\n./scrobble-deduplicator -u username -p password --duplicate-threshold 85\n```\n\n## 🔧 How It Works\n\n### Duplicate Detection\n\n- Compares consecutive scrobbles of the same track\n- Calculates time difference between scrobbles\n- Determines if time difference is less than configurable percentage of track duration\n- Uses MusicBrainz API for accurate track durations\n\n**Example**: If a 4-minute track has two scrobbles 2 minutes apart, the second is considered a duplicate if threshold is 90% (since 2 minutes is only 50% of track duration).\n\n### Browser Automation\n\n- Navigates through Last.fm library pages\n- Extracts scrobble information from HTML\n- Automatically deletes identified duplicates\n- Handles cookie management and consent banners\n\n### Caching Strategy\n\n- **In-Memory**: Fastest, data lost on restart\n- **File**: Persistent, good for single instances\n- **Redis**: Distributed, ideal for production\n\n## 🐳 Docker Development\n\n```bash\n# Development (with hot reload)\ndocker compose -f compose.dev.yaml up --build\n\n# Production\ndocker compose up --build\n```\n\n## 🔍 Development\n\n```bash\nnix develop\ngo test ./...\n```\n\n### Code Quality\n\n- Pre-commit hooks for formatting, linting, and static analysis\n- Automated testing and CI/CD pipeline\n\n## 📊 Output and Reporting\n\n- **CSV Export**: Deleted scrobbles with timestamps\n- **Statistics**: Cache hits/misses, processing time, error counts\n- **Telegram Notifications**: Optional completion reports\n- **Logging**: Comprehensive audit trail\n\n## 🚨 Safety Features\n\n- **Dry-run by default**: Set `canDelete: true` to enable deletion\n- **Configurable thresholds**: Fine-tune detection sensitivity\n- **Date range limits**: Process only specific time periods\n- **Comprehensive logging**: Full audit trail\n- **Error handling**: Robust retry mechanisms\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run tests and quality checks\n5. Submit a pull request\n\n## 🆘 Troubleshooting\n\n### Common Issues\n\n- **Browser Connection**: Ensure Chrome/Chromium is accessible, use `browserHeadful: true` for debugging\n- **Authentication**: Verify Last.fm credentials\n- **Cache Issues**: Verify Redis connection, check file permissions\n- **Performance**: Use Redis cache for large libraries, adjust date ranges\n\n### Getting Help\n\n- Check logs for detailed error information\n- Review configuration options\n- Open an issue with detailed error logs\n\n---\n\n**Note**: This tool modifies your Last.fm profile data. Always test with `canDelete: false` first.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcterence%2Flastfm-scrobble-deduplicator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcterence%2Flastfm-scrobble-deduplicator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcterence%2Flastfm-scrobble-deduplicator/lists"}