{"id":13993809,"url":"https://github.com/thisisparker/xword-dl","last_synced_at":"2025-10-04T00:32:28.525Z","repository":{"id":39915905,"uuid":"174920359","full_name":"thisisparker/xword-dl","owner":"thisisparker","description":"⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛","archived":false,"fork":false,"pushed_at":"2025-09-27T22:20:15.000Z","size":453,"stargazers_count":175,"open_issues_count":38,"forks_count":38,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-09-27T22:24:41.176Z","etag":null,"topics":["cli","crosswords","python","scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thisisparker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-03-11T03:37:16.000Z","updated_at":"2025-09-27T21:15:16.000Z","dependencies_parsed_at":"2023-10-04T11:02:54.290Z","dependency_job_id":"23697ff8-0fd7-4bb4-b4a3-449204b3adaf","html_url":"https://github.com/thisisparker/xword-dl","commit_stats":{"total_commits":241,"total_committers":10,"mean_commits":24.1,"dds":0.1203319502074689,"last_synced_commit":"f926a739b9d572341d3e94c3f170981ada804f87"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/thisisparker/xword-dl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thisisparker%2Fxword-dl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thisisparker%2Fxword-dl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thisisparker%2Fxword-dl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thisisparker%2Fxword-dl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thisisparker","download_url":"https://codeload.github.com/thisisparker/xword-dl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thisisparker%2Fxword-dl/sbom","scorecard":{"id":881681,"data":{"date":"2025-08-11","repo":{"name":"github.com/thisisparker/xword-dl","commit":"7b22ce5eea9f2fba2d317e9919edb0155108fc70"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.8,"checks":[{"name":"Maintained","score":10,"reason":"22 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 2/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/lint-format-typecheck.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/lint-format-typecheck.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/lint-format-typecheck.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/lint-format-typecheck.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/lint-format-typecheck.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/lint-format-typecheck.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish-release.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/publish-release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-release.yml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/publish-release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-release.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/publish-release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/status-check-outlets.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/thisisparker/xword-dl/status-check-outlets.yml/main?enable=pin","Info:   1 out of   4 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   4 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: jobLevel 'contents' permission set to 'write': .github/workflows/publish-release.yml:14","Warn: no topLevel permission defined: .github/workflows/lint-format-typecheck.yml:1","Warn: no topLevel permission defined: .github/workflows/publish-release.yml:1","Warn: no topLevel permission defined: .github/workflows/status-check-outlets.yml:1"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-24T08:27:23.950Z","repository_id":39915905,"created_at":"2025-08-24T08:27:23.950Z","updated_at":"2025-08-24T08:27:23.950Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278249448,"owners_count":25955834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","crosswords","python","scraping"],"created_at":"2024-08-09T14:02:34.045Z","updated_at":"2025-10-04T00:32:28.517Z","avatar_url":"https://github.com/thisisparker.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# xword-dl\n\n`xword-dl` is a command-line tool to download .puz files for online crossword puzzles from supported outlets or arbitrary URLs with embedded crossword solvers. For a supported outlet, you can easily download the latest puzzle, or specify one from the archives.\n\nSupported outlets:\n\n|Outlet|Keyword|Download latest|Search by date|Search by URL|\n|------|-------|:-------------:|:------------:|:-----------:|\n|*Atlantic*|`atl`|✔️|✔️||\n|*Crossword Club*|`club`|✔️|✔️|✔️|\n|*The Daily Beast*|`db`|✔️|||\n|*Daily Pop*|`pop`|✔️|✔️||\n|*Der Standard*|`std`|✔️||✔️|\n|*Guardian Cryptic*|`grdc`|✔️||✔️|\n|*Guardian Everyman*|`grde`|✔️||✔️|\n|*Guardian Prize*|`grdp`|✔️||✔️|\n|*Guardian Quick*|`grdq`|✔️||✔️|\n|*Guardian Quiptic*|`grdu`|✔️||✔️|\n|*Guardian Speedy*|`grds`|✔️||✔️|\n|*Guardian Weekend*|`grdw`|✔️||✔️|\n|*Los Angeles Times*|`lat`|✔️|✔️||\n|*Los Angeles Times Mini*|`latm`|✔️|✔️||\n|*New York Times*|`nyt`|✔️|✔️|✔️|\n|*New York Times Mini*|`nytm`|✔️|✔️|✔️|\n|*New York Times Variety*|`nytv`||✔️||\n|*The New Yorker*|`tny`|✔️|✔️|✔️|\n|*The New Yorker Mini*|`tnym`|✔️|✔️|✔️|\n|*Newsday*|`nd`|✔️|✔️||\n|*Observer Everyman*|`ever`|✔️||✔️|\n|*Observer Speedy*|`spdy`|✔️||✔️|\n|*Puzzmo*|`pzm`|✔️|✔️|✔️|\n|*Puzzmo Big*|`pzmb`|✔️|✔️|✔️|\n|*Simply Daily Puzzles*|`sdp`|✔️|✔️|✔️|\n|*Simply Daily Puzzles Cryptic*|`sdpc`|✔️|✔️|✔️|\n|*Simply Daily Puzzles Quick*|`sdpq`|✔️|✔️|✔️|\n|*Universal*|`uni`|✔️|✔️||\n|*USA Today*|`usa`|✔️|✔️||\n|*Vox*|`vox`|✔️|||\n|*The Walrus*|`wal`|✔️|||\n|*Washington Post*|`wp`|✔️|✔️||\n\nTo download a puzzle, install `xword-dl` and run it on the command line.\n\n## Installation\n\nThe easiest way to install `xword-dl` is with the [`uv` package manager](https://docs.astral.sh/uv/). After [installing `uv`](https://docs.astral.sh/uv/#installation), you can install the latest released version of `xword-dl` with:\n\n```\nuv tool install xword-dl\n```\n\nYou can also invoke it without installing, by running:\n\n```\nuvx xword-dl\n```\n\nRun an unreleased version or alternative branch, with or without installation.\n\n```\nuvx --from git+https://github.com/thisisparker/xword-dl.git\u003c@featurebranch\u003e xword-dl\n```\n\nIt is also available through a standard `pip install`, if you want to manage your own virtual environments.\n\n## Usage\n\nOnce installed, you can invoke `xword-dl`, providing the short code of the site from which to download. If you run `xword-dl` without providing a site keyword, it will print some usage instructions and then exit.\n\nFor example, to download the latest Newsday puzzle, you could run:\n\n```\nxword-dl nd --latest\n```\n\nor simply\n\n```\nxword-dl nd\n```\n\nYou can also download puzzles that are embedded in AmuseLabs solvers or on supported sites by providing a URL, such as:\n\n```\nxword-dl https://rosswordpuzzles.com/2021/01/03/cover-up/\n```\n\nIn either case, the resulting .puz file can be opened with [`cursewords`](https://github.com/thisisparker/cursewords) or any other puz file reader.\n\nDue to the constraints of the .puz format, the `xword-dl`'s conversion may be a bit lossy. For example, the most common form of .puz files only support [Latin-1 text encoding](https://en.wikipedia.org/wiki/ISO/IEC_8859-1), which means that some special characters (and even “curly quotes”) need to be converted before saving.\n\n`xword-dl` will also, by default, convert provided HTML to plaintext markdown. If you want to skip that step, you can provide the `--preserve-html` flag at runtime or set the `preserve-html` key to `True` in your config file. \n\n### Specifying puzzle date\n\nSome outlets allow specification of a puzzle to download by date using the `--date` or `-d` flag. For example, to download the Universal puzzle from September 22, 2021, you could run:\n\n```\nxword-dl uni --date 9/22/21\n```\n\nThe argument provided after the flag is parsed pretty liberally, and you can use relative descriptors such as \"yesterday\" or  \"monday\". Use quotes if your date contains spaces (such as \"June 16, 2022\").\n\n### Specifying filenames\n\nBy default, files will be given a descriptive name based on puzzle metadata. If you want to specify a name for a given download, you can do so with the `-o` or `--output` flag. The following tokens are available:\n\n|Token    |Value|\n|---------|-----|\n|`%outlet`|Outlet name|\n|`%prefix`|Hardcoded outlet prefix|\n|`%title` |Puzzle title|\n|`%author`|Puzzle author|\n|`%cmd`   |Puzzle outlet keyword|\n|`%netloc`|Network location (domain and subdomain)|\n|`date tokens`|[`strftime` tokens](https://strftime.org/)|\n\n### Configuration file\n\nWhen running `xword-dl`, a configuration file is created to store persistent settings. By default, this file is located at `~/.config/xword-dl/xword-dl.yaml`. You can manually edit this file to pass options to `xword-dl` at runtime.\n\nMost settings are specified by the command keyword. For example, if you want to save *USA Today* puzzles in this format:\n\n```\nUSA Today - By Brooke Husic  Ed. Erik Agard - Right Turns - 221115.puz\n```\n\nyou can specify that by editing your config file to include the following lines:\n\n```\nusa:\n  filename: '%prefix - %author - %title - %y%m%d'\n```\n\nIn addition to command keywords, you can also use the keys `general` (to apply to all puzzles), `url` (to apply to embedded puzzles selected by URL at runtime) or with a given `netloc` (to apply to embedded puzzles at a given domain or subdomain).\n\n### New York Times authentication\n\nNew York Times puzzles are only available to subscribers. Attempting to download with the `nyt` keyword without authentication will fail. To authenticate, run:\n\n```\nxword-dl nyt --authenticate\n```\n\nand you will be prompted for your New York Times username and password. (Those credentials can also be passed at runtime with the `--username` and `--password` flags.)\n\nIf authentication is successful, an authentication token will be stored in a config file. Once that token is stored, you can download puzzles with `xword-dl nyt`.\n\nIn some cases, the authentication may fail because of anti-automation efforts on New York Times servers. If the automatic authentication doesn't work for you, you can [manually find your NYT-S token](https://xwstats.com/link) and save it in your config file.\n\n## Contributing\n\n`xword-dl` is open-source and freely licensed, and I welcome contributions. It is usually helpful to start by opening a new issue for discussion. Generally, only downloaders that pull crossword data from a first-party source are included in official releases.\n\nAny merged code should pass `pyright` and `ruff` checks, which are run automatically on open pull requests. Both tools are included as dev dependencies and configured in `pyproject.toml`, if you'd like to run them locally:\n\n```\nuv sync --dev\nuv run pyright\nuv run ruff check\nuv run ruff format\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthisisparker%2Fxword-dl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthisisparker%2Fxword-dl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthisisparker%2Fxword-dl/lists"}