{"id":46571657,"url":"https://github.com/chrisgleissner/sidflow","last_synced_at":"2026-04-01T21:39:58.422Z","repository":{"id":322152967,"uuid":"1088304206","full_name":"chrisgleissner/sidflow","owner":"chrisgleissner","description":"Stream C64 SID music that evolves with your feedback, creating custom mood-based stations from your ratings and song similarity.","archived":false,"fork":false,"pushed_at":"2026-03-24T00:10:59.000Z","size":38302,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-24T04:54:43.087Z","etag":null,"topics":["c64","classification","commodore","commodore-64","music","nextjs16","react","retrocomputing","sid","song","streaming","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrisgleissner.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-11-02T17:57:11.000Z","updated_at":"2026-03-24T00:11:03.000Z","dependencies_parsed_at":"2026-02-20T10:24:38.097Z","dependency_job_id":null,"html_url":"https://github.com/chrisgleissner/sidflow","commit_stats":null,"previous_names":["chrisgleissner/sidflow"],"tags_count":62,"template":false,"template_full_name":null,"purl":"pkg:github/chrisgleissner/sidflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisgleissner%2Fsidflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisgleissner%2Fsidflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisgleissner%2Fsidflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisgleissner%2Fsidflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrisgleissner","download_url":"https://codeload.github.com/chrisgleissner/sidflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisgleissner%2Fsidflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292458,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c64","classification","commodore","commodore-64","music","nextjs16","react","retrocomputing","sid","song","streaming","typescript"],"created_at":"2026-03-07T08:33:23.771Z","updated_at":"2026-04-01T21:39:58.410Z","avatar_url":"https://github.com/chrisgleissner.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- markdownlint-disable-next-line MD041 --\u003e\n![Logo](./doc/img/logo.png)\n\n# SIDFlow\n\nA seamless stream of similar Commodore 64 SID songs.\n\n[![CI](https://img.shields.io/github/actions/workflow/status/chrisgleissner/sidflow/ci.yaml?branch=main\u0026logo=github\u0026label=CI)](https://github.com/chrisgleissner/sidflow/actions/workflows/ci.yaml)\n[![codecov](https://codecov.io/github/chrisgleissner/sidflow/graph/badge.svg?token=ynAHHsMqMG)](https://codecov.io/github/chrisgleissner/sidflow)\n[![License: GPL v2](https://img.shields.io/badge/License-GPL%20v2-blue.svg)](https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html)\n[![Platform](https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20Windows-forestgreen)](doc/developer.md)\n\n\nSIDFlow analyses your Commodore 64 SID music collection. It extracts audio features, learns your taste, and generates a continuous stream of similar tracks.\n\n\u003e [!NOTE]\n\u003e This project is under active development. Some documented features may not yet be fully functional. \n\n## Quick Start\n\nThree commands to a running local player:\n\n**1. Install [Bun](https://bun.com/docs/installation)**\n\n```sh\n# macOS / Linux\ncurl -fsSL https://bun.com/install | bash\n\n# Windows\npowershell -c \"irm bun.sh/install.ps1|iex\"\n```\n\n**2. Clone and build**\n\n```bash\ngit clone https://github.com/chrisgleissner/sidflow.git\ncd sidflow\nbun run build\n```\n\n**3. Start the web player**\n\n```bash\ncd packages/sidflow-web\nbun run dev\n```\n\nOpen **\u003chttp://localhost:3000\u003e**.\n\nThe first-time setup wizard guides you through downloading HVSC and configuring your collection. Local dev mode defaults to `admin / password` for the admin console at **\u003chttp://localhost:3000/admin\u003e**.\n\n### System ROMs\n\nMany SID songs require the Commodore 64 system ROM files in **`workspace/roms/`** at the repository root:\n\n| File name (preferred) | ROM | Size |\n|-----------------------|-----|------|\n| `kernal.901227-03.bin` (or `kernal.bin`) | C64 Kernal | 8 KB |\n| `basic.901226-01.bin` (or `basic.bin`) | C64 BASIC | 8 KB |\n| `characters.901225-01.bin` (or `chargen.bin`) | Character generator | 4 KB |\n\nObtain these files from a physical Commodore 64 or a BIOS dump. The `workspace/roms/` directory is git-ignored so the ROM files are never committed.\n\nAlternative locations (checked in order):\n1. `$SIDFLOW_ROMS_DIR` or `$SIDFLOW_ROM_DIR` environment variable\n2. `$SIDFLOW_ROOT/workspace/roms`\n3. `workspace/roms/` ← **recommended default**\n4. `public/roms/`\n\n---\n\n## Web UI\n\nSIDFlow ships a **Next.js + React** interface with two access points:\n\n| URL | Purpose |\n|-----|---------|\n| `http://localhost:3000` | Public player |\n| `http://localhost:3000/admin` | Admin console |\n\n### Public Player\n\nPick a mood preset and hit play. The queue fills automatically with similar tracks.\n\n![play panel](./doc/web-screenshots/07-play.png)\n\n**Keyboard shortcuts (Play tab):** `Space` play/pause · `←/→` prev/next · `↑/↓` volume · `M` mute · `S` focus search · `?` help\n\n**Favorites** are stored server-side (`data/.sidflow-preferences.json`) and shared across all browsers pointing at the same server.  \n**Recently played** is stored per-browser in localStorage (up to 100 entries).\n\n### Admin Console\n\nThe admin console (`/admin`) controls the full pipeline. Authenticate with `SIDFLOW_ADMIN_USER` / `SIDFLOW_ADMIN_PASSWORD` (defaults to `admin/password` in local dev only).\n\n#### Wizard - first-time setup\n\nSelect your HVSC root and confirm cache locations.\n\n![wizard panel](./doc/web-screenshots/01-wizard.png)\n\n#### Preferences\n\nTweak themes, fonts, render engines, ROM paths, and collection settings.\n\n![preferences panel](./doc/web-screenshots/02-prefs.png)\n\n#### Fetch - download HVSC\n\nSync the High Voltage SID Collection from official mirrors.\n\n![fetch panel](./doc/web-screenshots/03-fetch.png)\n\n#### Rate - tag your collection\n\nManually rate songs on energy, complexity, mood, and preference. Ratings feed into the training pipeline.\n\n![rate panel](./doc/web-screenshots/04-rate-playback.png)\n\n#### Classify - audio feature extraction\n\nAutomatically analyse your entire collection. Progress is displayed in real time.\n\n![classify panel](./doc/web-screenshots/05-classify-progress.png)\n\nFor more details on routes and the REST API, see [packages/sidflow-web/README.md](packages/sidflow-web/README.md) and the [OpenAPI spec](packages/sidflow-web/openapi.yaml).\n\n---\n\n## How It Works\n\nSIDFlow is a CLI-first pipeline. Each stage reads and writes JSONL under `data/` and is configured via `.sidflow.json`:\n\n```\nsidflow-fetch → sidflow-classify → sidflow-train → sidflow-play\n     ↓                 ↓                 ↓               ↓\n  HVSC sync      audio features      ML model       playlists\n```\n\n1. **Fetch** - downloads and synchronises HVSC (or any local SID collection).\n2. **Classify** - renders each SID to WAV cache, extracts structural and audio features, and writes JSONL.\n3. **Train** - consumes classified JSONL plus manual feedback to produce LanceDB model artifacts.\n4. **Play** - uses similarity search against the model to generate context-aware queues.\n\nThe web UI, Docker image, and CLI tools are all thin wrappers over these same pipeline stages.\n\n---\n\n## CLI Tools\n\nAll pipeline stages are available as standalone CLIs for automation and scripting:\n\n| CLI | Description |\n|-----|-------------|\n| **[sidflow-fetch](packages/sidflow-fetch/README.md)** | Sync HVSC from official mirrors |\n| **[sidflow-classify](packages/sidflow-classify/README.md)** | Render WAV cache + extract features |\n| **[sidflow-train](packages/sidflow-train/README.md)** | Train / update model artifacts |\n| **[sidflow-rate](packages/sidflow-rate/README.md)** | Write manual rating/tag files |\n| **[sidflow-play](packages/sidflow-play/README.md)** | Generate playlists via similarity search |\n\nFull CLI reference: [Technical Reference](./doc/technical-reference.md).\n\n### SID Station - command line radio\n\nLaunch a self-contained radio station in a Bash terminal that selects and streams similar SID tracks:\n\n![SID Flow Station](./doc/cli-screenshots/sidflow-station.png)\n\n```bash\n./scripts/sid-station.sh\n```\n\nFor playback on a real Commodore 64 Ultimate over your LAN:\n\n```bash\n./scripts/sid-station.sh --c64u-host c64u\n```\n\nIf `workspace/hvsc` is missing or empty, the script bootstraps HVSC automatically before starting.\n\n---\n\n## Config\n\n`.sidflow.json` controls all runtime paths. The defaults work out of the box:\n\n```json\n{\n  \"sidPath\": \"./workspace/hvsc\",\n  \"audioCachePath\": \"./workspace/audio-cache\",\n  \"tagsPath\": \"./workspace/tags\",\n  \"threads\": 0,\n  \"classificationDepth\": 3\n}\n```\n\nPass `--config /path/to/custom.json` to any CLI or set `SIDFLOW_CONFIG` for the web server to override the config location.\n\n---\n\n## Deployment\n\n### Docker\n\nPre-built images: [`ghcr.io/chrisgleissner/sidflow:latest`](https://github.com/chrisgleissner/sidflow/pkgs/container/sidflow)\n\n```bash\ndocker run -p 3000:3000 \\\n  -e SIDFLOW_ADMIN_USER=admin \\\n  -e SIDFLOW_ADMIN_PASSWORD='your-password' \\\n  -e SIDFLOW_ADMIN_SECRET='replace-with-a-32-character-secret-minimum' \\\n  -e JWT_SECRET='replace-with-a-32-character-secret-minimum' \\\n  -v /path/to/hvsc:/sidflow/workspace/hvsc \\\n  -v /path/to/audio-cache:/sidflow/workspace/audio-cache \\\n  -v /path/to/tags:/sidflow/workspace/tags \\\n  -v /path/to/data:/sidflow/data \\\n  ghcr.io/chrisgleissner/sidflow:latest\n```\n\nWeb UI at **\u003chttp://localhost:3000\u003e**, admin at `/admin`.\n\nProduction startup rejects default credentials, derived secrets, or a missing `JWT_SECRET`. Full Docker instructions, health checks, and smoke-testing are in [doc/deployment.md](doc/deployment.md).\n\n### Fly.io\n\nFly deployment is supported as a single stateful machine:\n\n```bash\ncurl -L https://fly.io/install.sh | sh          # install flyctl\n./scripts/deploy/fly-deploy.sh -e stg            # staging\n./scripts/deploy/fly-deploy.sh -e prd -t \u003ctag\u003e   # production\n```\n\nSee [Deployment Guide](doc/deployment.md) for details.\n\n---\n\n## Portable Similarity Export\n\nProduces a self-contained SQLite bundle containing per-track ratings, feedback aggregates, and 24-dimensional perceptual vectors (WAV + SID-native hybrid) for offline and downstream consumers.\n\nPrerequisites: `bun` 1.3.1+, `ffmpeg`, `sidplayfp`, `curl`, `python3` (plus `gh` authenticated for step 3/publish). Many SID songs also rqeuire [C64 system ROMs](#system-roms) in `workspace/roms/`.\n\n**0. Download HVSC:**\n\nDownload the latest High Voltage SID Collection:\n\n```bash\n./scripts/sidflow-fetch\n```\n\nThis downloads the latest HVSC base archive plus any delta archives from `https://hvsc.brona.dk/HVSC/` and extracts them into `workspace/hvsc/` (i.e. the `sidPath` configured in `.sidflow.json`). The extracted SID files will be under `workspace/hvsc/C64Music/`. \n\nIf you already have a local HVSC copy elsewhere, point `sidPath` in `.sidflow.json` at that directory instead and skip this step.\n\n**1. Reclassify the entire HVSC collection and generate the export:**\n\nClassifying all 60,572 SID songs (as of HVSC version 84 in March 2026) takes about 3 hours on an Intel 14600K CPU using Kubuntu 24.04:\n\n```bash\nbash scripts/run-similarity-export.sh --mode local --full-rerun true\n```\n\nExpected logs:\n```\n✔ ~/dev/c64/sidflow [main|✔] \n08:33 $ bash scripts/run-similarity-export.sh --mode local --full-rerun true\n[sidcorr] Mode is full rerun: existing classified data and export artifacts will be ignored and replaced\n[sidcorr] Full rerun: removing prior classified JSONL artifacts from /home/chris/dev/c64/sidflow/data/classified\n[sidcorr] Mode: local\n[sidcorr] Installing dependencies for local mode\n[sidcorr] Starting local web server on port 3000\n[sidcorr] Triggering classification with payload {\"async\":false,\"skipAlreadyClassified\":false,\"deleteWavAfterClassification\":true,\"forceRebuild\":true}\n[sidcorr] Classification request started\n[sidcorr] Waiting for classification to finish\n[sidcorr] progress update: completed=2451 remaining=84623 total=87074 elapsed=55s eta=31m 46s rate=44.41 songs/s percent=2.8 phase=tagging phases[analyzing=done, metadata=done, building=done, tagging=now, completed=todo] stageCounts[rendered=2464, extracted=2451, tagged=2451]\n[sidcorr] progress update: completed=4398 remaining=82676 total=87074 elapsed=1m 25s eta=26m 42s rate=51.60 songs/s percent=5.1 phase=tagging phases[analyzing=done, metadata=done, building=done, tagging=now, completed=todo] stageCounts[rendered=4415, extracted=4398, tagged=4398]\n\n...snip...\n\n[sidcorr] progress update: completed=45898 remaining=41176 total=87074 elapsed=38m 2s eta=34m 7s rate=20.11 songs/s percent=52.7 phase=tagging phases[analyzing=done, metadata=done, building=done, tagging=now, completed=todo] stageCounts[rendered=45848, extracted=45845, tagged=45898]\n[sidcorr] progress update: completed=46778 remaining=40296 total=87074 elapsed=38m 32s eta=33m 12s rate=20.23 songs/s percent=53.7 phase=tagging phases[analyzing=done, metadata=done, building=done, tagging=now, completed=todo] stageCounts[rendered=46728, extracted=46725, tagged=46778]\n\n```\n\nOutput: `data/exports/sidcorr-hvsc-full-sidcorr-1.sqlite` and `sidcorr-hvsc-full-sidcorr-1.manifest.json`.\n\n**2. Regenerate the export from existing classified data (skip reclassification):**\n\n```bash\nbun run export:similarity -- --profile full\n```\n\n**3. Publish the export as a release to `chrisgleissner/sidflow-data`:**\n\nThe following command requires permissions to create new releases on `chrisgleissner/sidflow-data` and is only intended for the repo maintainers:\n\n```bash\nbash scripts/run-similarity-export.sh --workflow publish-only --mode local --publish-release true\n```\n\nFull schema and consumer workflow: [doc/similarity-export.md](doc/similarity-export.md).\n\n### Classification Vector Reference\n\nEach exported song also gets a 24-number similarity vector. It mixes what SIDFlow hears in the rendered WAV with what it reads from the SID chip write trace. The raw per-song feature dump is larger, but these 24 fields are the compact fingerprint used for similarity search and station building.\n\nSample record: [doc/examples/classification-vector-sample.json](doc/examples/classification-vector-sample.json)\n\n| Internal name | Source | Meaning |\n|---------------|--------|---------|\n| `tempoFused` | Hybrid | Overall speed feel |\n| `onsetDensityFused` | Hybrid | How often new notes or hits happen |\n| `rhythmicRegularityFused` | Hybrid | How steady the rhythm feels |\n| `syncopationSid` | SID | How much the beat pushes off the obvious pulse |\n| `arpeggioRateSid` | SID | How much fast chord-cycling the tune uses |\n| `waveTriangleRatio` | SID | Share of smooth triangle tone |\n| `waveSawRatio` | SID | Share of buzzy saw tone |\n| `wavePulseRatio` | SID | Share of hollow pulse tone |\n| `waveNoiseRatio` | SID | Share of noisy/percussion-like tone |\n| `pwmActivitySid` | SID | How much pulse-width modulation is moving |\n| `filterCutoffMeanSid` | SID | Typical brightness of the SID filter |\n| `filterMotionFused` | Hybrid | How much the tone color sweeps over time |\n| `samplePlaybackRate` | SID | How much digi-sample playback is present |\n| `melodicClarityFused` | Hybrid | How clearly a lead melody stands out |\n| `bassPresenceFused` | Hybrid | How bass-heavy the tune feels |\n| `accompanimentShareSid` | SID | How much of the arrangement acts as backing parts |\n| `voiceRoleEntropySid` | SID | How evenly the SID voices split their jobs |\n| `adsrPluckRatioSid` | SID | How often notes sound short and plucky |\n| `adsrPadRatioSid` | SID | How often notes sound long and sustained |\n| `loudnessFused` | Hybrid | Overall strength/loudness impression |\n| `dynamicRangeWav` | WAV | Difference between softer and louder moments |\n| `inharmonicityWav` | WAV | How rough or bell-like the spectrum is |\n| `mfccResidual1` | Hybrid | Timbre detail left after obvious SID waveform patterns are removed |\n| `mfccResidual2` | Hybrid | Another timbre detail channel for fine tonal differences |\n\n`Source` means:\n\n| Value | Meaning |\n|-------|---------|\n| `WAV` | Measured from the rendered audio |\n| `SID` | Derived from SID register-write traces |\n| `Hybrid` | SIDFlow combines WAV and SID evidence |\n\n---\n\n## Performance Tests\n\nJourney-driven performance suite (k6 + optional Playwright):\n\n```bash\n# Run against a local server\nbun run perf:run -- --env local --base-url http://localhost:3000 --results performance/results --tmp performance/tmp --execute\n```\n\n| Option | Notes |\n|--------|-------|\n| `--profile smoke\\|reduced\\|standard\\|scale` | Defaults: local→smoke, CI→reduced |\n| `--profile scale` | Hundreds-of-users load; remote-only guard |\n| `--env remote --enable-remote` | Fly.io / Raspberry Pi targets |\n\nJourneys live in `performance/journeys/`; outputs in `performance/results/\u003ctimestamp\u003e/`. CI uses `--profile reduced` with k6-only for stability.\n\n---\n\n## Developer Documentation\n\n- **[Technical Reference](doc/technical-reference.md)** - architecture, CLI tools, APIs\n- **[Developer Guide](doc/developer.md)** - setup, testing, contributions\n\n---\n\n## Acknowledgements\n\nSIDFlow is [GPLv2](LICENSE)-licensed and builds upon open-source software and datasets:\n\n| Component | License | Source | Credit |\n|-----------|---------|--------|--------|\n| **Bun** | MIT | [github.com/oven-sh/bun](https://github.com/oven-sh/bun) | JS runtime and tooling |\n| **libsidplayfp** | GPL v2+ | [github.com/libsidplayfp/libsidplayfp](https://github.com/libsidplayfp/libsidplayfp) | SID emulator compiled to WASM for browser playback |\n| **High Voltage SID Collection (HVSC)** | Free for personal use | [hvsc.c64.org](https://www.hvsc.c64.org/) | Largest SID collection |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisgleissner%2Fsidflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrisgleissner%2Fsidflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisgleissner%2Fsidflow/lists"}