{"id":48505263,"url":"https://github.com/izzoa/magsync","last_synced_at":"2026-04-07T16:02:31.539Z","repository":{"id":349582555,"uuid":"1202986008","full_name":"izzoa/magsync","owner":"izzoa","description":"A CLI/TUI tool for indexing and downloading PDF magazines from freemagazines.top, organized into a clean local library.","archived":false,"fork":false,"pushed_at":"2026-04-06T17:13:37.000Z","size":94,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-06T17:29:14.126Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/izzoa.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-06T15:54:22.000Z","updated_at":"2026-04-06T17:09:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/izzoa/magsync","commit_stats":null,"previous_names":["izzoa/magsync"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/izzoa/magsync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izzoa%2Fmagsync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izzoa%2Fmagsync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izzoa%2Fmagsync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izzoa%2Fmagsync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/izzoa","download_url":"https://codeload.github.com/izzoa/magsync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/izzoa%2Fmagsync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31518643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T03:10:19.677Z","status":"ssl_error","status_checked_at":"2026-04-07T03:10:13.982Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-07T16:02:30.603Z","updated_at":"2026-04-07T16:02:31.528Z","avatar_url":"https://github.com/izzoa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003emagsync\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\u003ci\u003eMagazine downloader and organizer for your local library\u003c/i\u003e\u003c/p\u003e\n\nA CLI/TUI tool for indexing and downloading PDF magazines from [freemagazines.top](https://freemagazines.top), organized into a clean local library.\n\n```\n~/Magazines/\n├── The New Yorker/\n│   ├── 2026/\n│   │   ├── 04/\n│   │   │   ├── The_New_Yorker_-_April_06,_2026.pdf\n│   │   │   └── The_New_Yorker_-_April_13,_2026.pdf\n│   │   ├── 03/\n│   │   └── 02/\n│   └── 2025/\n├── The Economist/\n└── Science News/\n```\n\n## Features\n\n- **Search** magazines by title with full pagination across freemagazines.top\n- **Download** PDFs automatically — handles LimeWire's E2E encryption entirely in Python (no browser needed)\n- **Organize** into `[Magazine Title]/[YYYY]/[MM]/` directory structure with smart date parsing\n- **Track** what you've downloaded with a local SQLite index — never re-download the same issue\n- **Update** your index on demand to discover new issues for tracked magazines\n- **TUI** for interactive browsing, or **CLI** for scripted/headless use\n- **Self-healing** — automatically refreshes encryption constants when LimeWire updates their JS bundles\n\n## Installation\n\nRequires Python 3.11+.\n\n```bash\n# From source\ngit clone https://github.com/yourusername/magsync.git\ncd magsync\npip install -e .\n\n# Or with pipx (recommended)\npipx install .\n```\n\n## Quick Start\n\n### TUI (Interactive)\n\n```bash\nmagsync\n```\n\nLaunches the terminal UI with three tabs:\n- **Search** — type a magazine name, browse results, select issues, download\n- **Downloads** — watch download progress\n- **Library** — browse your indexed magazines by title/year/month\n\nKeyboard shortcuts: `s` (search), `a` (select all), `d` (download selected), `q` (quit)\n\n### CLI\n\n```bash\n# Search for a magazine\nmagsync search \"The Economist\"\n\n# Download all issues from March 2026 onward\nmagsync fetch \"The New Yorker\" --since 2026-03\n\n# Preview what would be downloaded (no actual download)\nmagsync fetch \"The New Yorker\" --since 2026-03 --dry-run\n\n# Download to a custom directory\nmagsync fetch \"Science News\" --since 2025-01 --output ~/MyMags\n\n# Update index for all tracked magazines\nmagsync update\n\n# View/change configuration\nmagsync config\nmagsync config output_dir ~/MyMagazines\n```\n\n## Subscriptions\n\nManage magazine subscriptions for daemon mode:\n\n```bash\n# Add a subscription\nmagsync subscribe \"The New Yorker\" --since 2025-01\nmagsync subscribe \"The Economist\" --since 2024-06\n\n# List subscriptions\nmagsync subscribe\n\n# Remove a subscription\nmagsync unsubscribe \"The Economist\"\n```\n\nOr configure in `config.toml`:\n\n```toml\n[[subscriptions]]\nquery = \"The New Yorker\"\nsince = \"2025-01\"\n\n[[subscriptions]]\nquery = \"The Economist\"\nsince = \"2024-06\"\n```\n\n## Docker\n\nRun magsync as an unattended daemon in Docker. Automatically fetches new issues on a schedule.\n\n### Quick Start\n\n```bash\n# Pull the image\ndocker pull ghcr.io/izzoa/magsync:latest\n\n# Or build locally\ndocker build -t magsync .\n\n# Start the daemon\ndocker compose up -d\n```\n\n### docker-compose.yml\n\n```yaml\nservices:\n  magsync:\n    image: ghcr.io/izzoa/magsync:latest\n    container_name: magsync\n    volumes:\n      - ./config.toml:/config/config.toml\n      - magsync_data:/data\n      - /path/to/magazines:/magazines\n    environment:\n      - MAGSYNC_OUTPUT_DIR=/magazines\n      - MAGSYNC_INTERVAL=6h\n      - MAGSYNC_SUBSCRIPTIONS=The New Yorker:2025-01,The Economist:2024-06\n      # - MAGSYNC_APPRISE_URLS=gotify://server/token\n    restart: unless-stopped\n\nvolumes:\n  magsync_data:\n```\n\n### Multi-Architecture\n\nBuild for both amd64 and arm64 (Raspberry Pi, Synology NAS):\n\n```bash\ndocker buildx build --platform linux/amd64,linux/arm64 -t magsync:latest .\n```\n\n### Environment Variables\n\nAll config values can be overridden via environment variables:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `MAGSYNC_OUTPUT_DIR` | Magazine output directory | `~/Magazines` |\n| `MAGSYNC_INTERVAL` | Daemon cycle interval | `6h` |\n| `MAGSYNC_SUBSCRIPTIONS` | Comma-separated `query:since` pairs | (none) |\n| `MAGSYNC_APPRISE_URLS` | Comma-separated [Apprise](https://github.com/caronc/apprise/wiki) notification URLs | (none) |\n| `MAGSYNC_CONFIG_DIR` | Config directory path | `~/.magsync` |\n| `MAGSYNC_DB_PATH` | SQLite index path | `{config_dir}/index.db` |\n| `MAGSYNC_DOWNLOAD__MAX_CONCURRENT` | Max parallel downloads | `3` |\n| `MAGSYNC_DOWNLOAD__SCRAPE_DELAY` | Delay between scrape requests (seconds) | `1.0` |\n\n### NAS Deployment (Synology, QNAP)\n\n```yaml\nvolumes:\n  - /volume1/docker/magsync/config.toml:/config/config.toml:ro\n  - magsync_data:/data\n  - /volume1/magazines:/magazines\nenvironment:\n  - MAGSYNC_OUTPUT_DIR=/magazines\n# Match your NAS user ID:\nuser: \"1026:100\"\n```\n\n### Notifications\n\nmagsync sends notifications via [Apprise](https://github.com/caronc/apprise/wiki) when new issues are downloaded. Supports 90+ services including Gotify, Discord, Slack, ntfy, email, and more.\n\n```bash\n# Via environment variable\nMAGSYNC_APPRISE_URLS=gotify://myserver:8080/token,discord://webhook_id/webhook_token\n```\n\nOr in `config.toml`:\n\n```toml\n[notifications]\nenabled = true\napprise_urls = [\"gotify://myserver:8080/token\"]\n```\n\n## Configuration\n\nConfig lives at `~/.magsync/config.toml` (or `$MAGSYNC_CONFIG_DIR/config.toml`):\n\n```toml\n[general]\noutput_dir = \"~/Magazines\"\n\n[download]\nmax_concurrent = 3\nretry_attempts = 3\nscrape_delay = 1.0\n\n# [limewire] section is auto-populated on first download via self-healing.\n# You do not need to configure this manually.\n\n[notifications]\nenabled = false\napprise_urls = []\n\n[[subscriptions]]\nquery = \"The New Yorker\"\nsince = \"2025-01\"\n```\n\n## How It Works\n\n### Scraping\n\nmagsync searches freemagazines.top (a WordPress site) via its search endpoint (`/?s=query`), follows pagination, and scrapes individual magazine detail pages to extract metadata and download links.\n\n### Downloading\n\nDownloads go through [LimeWire](https://limewire.com), a file-sharing service that uses end-to-end encryption. magsync implements the full decryption pipeline natively in Python:\n\n1. Visit the LimeWire share page to get a session (JWT + CSRF token)\n2. Extract metadata from the server-rendered HTML (bucket ID, encryption keys, etc.)\n3. Derive the AES decryption key from the URL fragment:\n   - **Short links** (`/d/bjAa5#passphrase`): PBKDF2 → AES-KW unwrap → ECDH key agreement\n   - **UUID links** (`/d/{uuid}#base64key`): Direct ECDH key agreement\n4. Fetch the presigned S3 download URL via LimeWire's API\n5. Download the encrypted blob and decrypt with AES-256-CTR\n\nNo browser, Playwright, or Selenium required.\n\n### Self-Healing\n\nLimeWire's encryption constants (salt, IVs) are embedded in their JavaScript bundles and may change on deploys. If decryption produces an invalid PDF, magsync automatically:\n\n1. Fetches LimeWire's current JS bundles\n2. Extracts updated encryption constants\n3. Retries decryption with the new constants\n4. Persists working constants to your config file\n\nIf auto-extraction fails, see [UPDATE_KEYS.md](UPDATE_KEYS.md) for manual extraction instructions.\n\n### Organization\n\nMagazine titles are parsed to extract dates using regex patterns that handle:\n- `April 13, 2026` → `2026/04/`\n- `February 16-23, 2026` → `2026/02/`\n- `March-April 2026` → `2026/03/`\n- `Spring 2026` → `2026/03/`\n- `Vol 208 No 05, May 2026` → `2026/05/`\n- Undatable issues → `YYYY/00-Unknown/`\n\n## Dependencies\n\n| Package | Purpose |\n|---------|---------|\n| [httpx](https://www.python-httpx.org/) | Async HTTP client for scraping and API calls |\n| [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/) | HTML parsing |\n| [cryptography](https://cryptography.io/) | PBKDF2, AES-KW, ECDH P-256, AES-256-CTR |\n| [textual](https://textual.textualize.io/) | Terminal UI framework |\n| [typer](https://typer.tiangolo.com/) | CLI framework |\n| [rich](https://rich.readthedocs.io/) | Terminal formatting and progress bars |\n| [apprise](https://github.com/caronc/apprise) | Notifications (optional) |\n\n## Disclaimer\n\nThis software is provided for educational and personal use only. The authors and contributors of this project are not responsible for how it is used. Users are solely responsible for ensuring their use of this tool complies with all applicable laws and regulations in their jurisdiction, including but not limited to copyright law. Downloading copyrighted material without authorization may be illegal in your country. The authors do not endorse, encourage, or condone the use of this software for copyright infringement or any other unlawful activity.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fizzoa%2Fmagsync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fizzoa%2Fmagsync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fizzoa%2Fmagsync/lists"}