{"id":13564151,"url":"https://github.com/eschluntz/compress","last_synced_at":"2025-05-16T05:04:11.969Z","repository":{"id":149713319,"uuid":"415462977","full_name":"eschluntz/compress","owner":"eschluntz","description":"Text compression for generating keyboard expansions","archived":false,"fork":false,"pushed_at":"2023-09-30T17:24:33.000Z","size":868,"stargazers_count":1415,"open_issues_count":3,"forks_count":29,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-05-16T05:03:34.931Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eschluntz.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-10-10T01:56:19.000Z","updated_at":"2025-04-30T19:31:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"fcdae315-26b5-476d-bac0-8a5a77ae7514","html_url":"https://github.com/eschluntz/compress","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eschluntz%2Fcompress","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eschluntz%2Fcompress/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eschluntz%2Fcompress/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eschluntz%2Fcompress/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eschluntz","download_url":"https://codeload.github.com/eschluntz/compress/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254471061,"owners_count":22076585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:27.242Z","updated_at":"2025-05-16T05:04:11.949Z","avatar_url":"https://github.com/eschluntz.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![codecov](https://codecov.io/gh/eschluntz/compress/branch/master/graph/badge.svg?token=CML5P28ELL)](https://codecov.io/gh/eschluntz/compress)\n\n[![Build Status](https://github.com/eschluntz/compress/actions/workflows/run_tests.yml/badge.svg)](https://github.com/eschluntz/compress/actions)\n\n# Compress\n![demo](img/expand-demo.gif)\n\nThis is a tool for automatically creating typing shortcuts from a corpus of your own writing! I use these shortcuts mainly for email and slack:\n\n![email](img/email-expand.gif)\n\n![slack](img/slack-expand.gif)\n\nThis repo parses a corpus of text and suggest what shortcuts you should use to save the most letters while typing. It then generates config files for [Autokey](https://github.com/autokey/autokey), a linux program that implements keyboard shortcuts!\n\nIt also contains a tool for optionally parsing a Slack Data Export of your messages to create a corpus.\n\n# What phrases should I abbreviate?\nThe code looks through the corpus to find common n-grams that can be replaced with much shorter phrases. The suggestions are ranked by `[characters saved] * [frequency of phrase]`. \n\nI was surprised that very short and frequent words topped this list, such as `the -\u003e t`, instead of longer phrases that I use a lot, such as `what do you think -\u003e wdytk`.\n\n![results](img/chars-saved.png)\n\nJust reading through the results was amusing to see how repetitive some of my writing is :)\n\n# How to pick abbreviations?\nThis is largely preferences and heuristics to try to generate memorable abbreviations for different phrases. Some of my design philosphies were:\n\n1. The abbrev cannot be a word that I want to type. Right now this is done with a blacklist, but I should change it to use my actual corpus.\n2. The goal is being memorable. 1st letter is top choice, and 1st letter + last letter is next choice.\n3. More common phrases get priority for more memorable abbrevs.\n\nThis is currently done as a manual post-process step, but I like to make \"families\"\nof abbrevs to make them more memorable. Some example heuristics for this are:\n\n1. Plurals should have the same abbrev as the singular, but with an \"s\". For example `robot -\u003e r` and `robots -\u003e rs`. \n2. If a word has an abbrev, a phrase that contains that word should contain the abbrev. For example:\n```\nthe       -\u003e t\nrobot     -\u003e r\nthe robot -\u003e tr\n```\n3. Think about how similar words' abbrevs can be similar as well. i.e. \n```\nsome      -\u003e s\nsomeone   -\u003e sn\nsomething -\u003e st\nsometime  -\u003e sti\n```\n\n# Instructions\n\n1. run `install.sh` to install dependencies. Currently tested on python 3.10.12\n1. Put any corpus of your text that you want to compress in `data/corpus/*.txt`\n2. If you want to use your slack history as a corpus:\n    1. export it to a folder called `data/slack_export`. Only slack workspace admins can do this (and it only exports public channels).\n    2. Change `USERNAME_TO_EXPORT` at the top of the file to your slack username.\n    3. Run `parse_slack.py`. This will generate a new corpus document in `data/corpus/`\n    4. DELETE YOUR SLACK EXPORT WITH `srm` \n3. Run `find_suggested_phrases.py`. This will generate a list of the top 200 suggested shortcuts to `output/suggested_shortcuts.yaml`\n4. Edit or add any shortcuts that you want, then copy the file to `shortcuts.yaml`. \n    - This is a manual step so you can customize it without it being blown out every time you run the script again. \n    - It's also saved in git even though it's an output so that I can keep it in sync across multiple of my computers :) \n    - If you're starting out, I suggest just going with 10-20 shortcuts to make it easier to remember them\n5. Run `generate_autokeys.py` to convert `shortcuts.yaml` into actual config files for `autokey`.\n6. Install [Autokey](https://github.com/autokey/autokey)\n    - Right now, Autokey is only supported on linux with X11, not Wayland\n7. Check that your autokey config is located at `~/.config/autokey/data/My Phrases/`. If it is somewhere else, change `reload.sh:8` to point to your config location\n8. From now on when you edit `shortcuts.yaml` you can re-generate and reload autokey with `reload.sh` \n\n# Notes\n[Autokey](https://github.com/autokey/autokey) Uses simulated keyboard input to replace phrases with your abbreviations. I tried several chrome extensions but this worked much more reliably without conflicting with sites' own javascript.\n\nThe config files I generate are set to only apply when Chrome is in focus because that's where I do most of my english typing. I found that keeping this active in terminal and vscode caused way more problems than it was solved because my abbreviations overlapped with common short linux commands and variable names i.e. `t`. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feschluntz%2Fcompress","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feschluntz%2Fcompress","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feschluntz%2Fcompress/lists"}