https://github.com/century-arcade/xd

a futureproof crossword corpus toolset
https://github.com/century-arcade/xd

Last synced: 4 months ago
JSON representation

a futureproof crossword corpus toolset

• Host: GitHub
• URL: https://github.com/century-arcade/xd
• Owner: century-arcade
• License: mit
• Created: 2016-01-18T04:43:33.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2024-01-03T04:37:30.000Z (6 months ago)
• Last Synced: 2024-01-15T06:09:56.423Z (5 months ago)
• Language: Python
• Size: 669 KB
• Stars: 209
• Watchers: 10
• Forks: 26
• Open Issues: 18
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Lists

• awesome-stars - xd - arcade | 187 | (Python)
• awesome-stars - xd - a futureproof crossword corpus toolset (Python)
• awesome-stars - century-arcade/xd - a futureproof crossword corpus toolset (Python)

README

        
# xd

The xd project includes a [text format for crossword puzzles](doc/xd-format.md) and a pipeline for downloading, parsing, analyzing puzzles, and producing the website and released data at [xd.saul.pw](https://xd.saul.pw).

## Requirements

- python 3.7+

- git

- markdown (to build website)

- sqlite (for grid comparison)

- gcc (to build sqlite plugin)

- aws-cli (to deploy)

# Running the pipeline

1. Checkout the gxd repo (private; join [#crosswords on the Discord](https://saul.pw/chat) to discuss getting access).

    make setup

2. Download new puzzles from known sources, convert to .xd, shelve, and commit to gxd repo.

    make import

Raw puz/etc files saved to .zip in /tmp, and .xd files saved to `gxd` directory.

3. Analyze puzzles

    make analyze

Output in `pub` directory.

4. Build website

    make website

Output in `wwwroot` directory.

5. Generate `gxd.sqlite` database (400MB)

    make gxd.sqlite

6. Find similar grids (takes ~12 hours)

    make gridmatches

Similarity scores saved to `gridmatches` table in gxd.sqlite.