{"id":13446151,"url":"https://github.com/quackduck/aces","last_synced_at":"2025-04-23T01:50:56.163Z","repository":{"id":63738354,"uuid":"394143914","full_name":"quackduck/aces","owner":"quackduck","description":"Encode in a character set of your choice","archived":false,"fork":false,"pushed_at":"2024-05-03T22:09:08.000Z","size":54,"stargazers_count":68,"open_issues_count":3,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-17T18:23:18.423Z","etag":null,"topics":["decoder","encoding","hacktoberfest","library","variable-charset-encoding"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quackduck.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-09T04:06:37.000Z","updated_at":"2025-03-30T21:48:07.000Z","dependencies_parsed_at":"2024-01-13T17:14:17.872Z","dependency_job_id":"641bab6d-d301-4a75-b442-45767e227f30","html_url":"https://github.com/quackduck/aces","commit_stats":{"total_commits":57,"total_committers":2,"mean_commits":28.5,"dds":0.01754385964912286,"last_synced_commit":"a0ca5b97bac108283251597ee132da8bea2b0425"},"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackduck%2Faces","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackduck%2Faces/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackduck%2Faces/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackduck%2Faces/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quackduck","download_url":"https://codeload.github.com/quackduck/aces/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250354294,"owners_count":21416751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decoder","encoding","hacktoberfest","library","variable-charset-encoding"],"created_at":"2024-07-31T05:00:47.132Z","updated_at":"2025-04-23T01:50:56.135Z","avatar_url":"https://github.com/quackduck.png","language":"Go","funding_links":[],"categories":["CLI Tools"],"sub_categories":[],"readme":"# Aces\n\n[comment]: \u003c\u003e (**A**ny **C**haracter **E**ncoding **S**et)\nAny Character Encoding Set\n\nAces is a command line utility that lets you encode any data to a character set of your choice.\n\n\u003csup\u003e_Psst... it is also now a library that you can use for encoding and decoding and also writing and reading at a bit level! See documentation [here](https://pkg.go.dev/github.com/quackduck/aces)._\u003c/sup\u003e\n\nFor example, you could encode \"Foo Bar\" to a combination of these four characters: \"HhAa\", resulting in this ~~hilarious~~ sequence of laughs:\n```text\nhHhAhAaahAaaHAHHhHHAhAHhhaHA\n```\nWith Aces installed, you can actually do that with:\n```shell\n$ echo -n \"Foo Bar\" | aces HhAa\nhHhAhAaahAaaHAHHhHHAhAHhhaHA\n```\nThis was the original use of Aces (it was called `ha`, increased data size by 4X and had no decoder)\n\nIf you're on macOS, you can even convert that output to speech:\n```shell\necho -n \"Matthew Stanciu\" | aces HhAa | say\n```\n\nMake your own wacky encoding:\n```shell\n$ echo HELLO WORLD | aces \"DORK BUM\"\nRRD RBO RKD M  DRBU MBRRRKD RDOR\n```\n\nYou can also use emojis:\n```shell\n$ echo -n yay | aces 🥇🥈🥉\n🥇🥈🥉🥇🥉🥇🥉🥉🥇🥉🥉🥇🥈🥇🥉🥇🥉🥉🥇🥉🥇🥈🥇\n```\n\nWith Aces, you can see the actual 0s and 1s of files:\n```shell\naces 01 \u003c $(which echo)\n```\nYou can also write hex/octal/binary/your own format by hand:\n```shell\necho C2A70A   | aces -d 0123456789ABCDEF # try this!\necho .+=...++ | aces -d ./+=\n```\nConvert binary to hex:\n```shell\necho 01001010 | aces -d 01 | aces 0123456789ABCDEF\n```\n\n_Also check out the examples!_\n## Installing\n\n### macOS or Linux with linuxbrew\n```shell\nbrew install quackduck/tap/aces\n```\n\n### Other platforms\nHead over to [releases](https://github.com/quackduck/aces/releases) and download the latest binary!\n\n## Usage\n```yaml\nAces - Encode in any character set\n\nUsage:\n  aces \u003ccharset\u003e                  - encode data from STDIN into \u003ccharset\u003e\n  aces -d/--decode \u003ccharset\u003e      - decode data from STDIN from \u003ccharset\u003e\n  aces -v/--version | -h/--help   - print version or this help message\n\n  Aces reads from STDIN for your data and outputs the result to STDOUT. An optimized algorithm is used\n  for character sets with a power of 2 length. Newlines are ignored when decoding.\n\nExamples:\n  echo hello world | aces \"\u003c\u003e(){}[]\" | aces --decode \"\u003c\u003e(){}[]\"      # basic usage\n  echo matthew stanciu | aces HhAa | say                             # make funny sounds (macOS)\n  aces \" X\" \u003c /bin/echo                                              # see binaries visually\n  echo 0100100100100001 | aces -d 01 | aces 0123456789abcdef         # convert bases\n  echo Calculus | aces 01                                            # what's stuff in binary?\n  echo Aces™ | base64 | aces -d\n  ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/   # even decode base64\n  echo -n yay | aces 🥇🥈🥉                                          # emojis work too! \n  Set the encoding/decoding buffer size with --bufsize \u003csize\u003e (default 16KiB).\n\n  File issues, contribute or star at github.com/quackduck/aces\n```\n\n## How does it work?\nTo answer that, we need to know how encoding works in general. Let's take the example of Base64.\n\n### Base64\n```text\nABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/\n```\nThat is the Base64 character set. As you may expect, it's 64 characters long. \n\nLet's say we want to somehow represent these two bytes in those 64 characters:\n```text\n00001001 10010010    # 09 92 in hex\n```\nTo do that, Base64 does something very smart: it uses the bits, interpreted as a number, as indexes of the character set.\n\nTo explain what that means, let's consider what possible values 6 bits can represent: `000000` (decimal 0) to `111111` (decimal 63).\nSince 0 to 63 is the exact range of indices that can be used with the 64 element character set, we'll group our 8 bit chunks (bytes) of data in 6 bit chunks (to use as indices):\n```text\n000010 011001 0010\n```\n`000010` is 2 in decimal, so by using it as an index of the character set, Base64 adds `C` (index 2) to the result.\n\n`011001` is 16 + 8 + 1 = 25 in decimal, so Base64 appends `Z` (index 25) to the result.\n\nYou may have spotted a problem with the next chunk - it's only 4 bits long!\n\nTo get around this, Base64 pretends it's a 6 bit chunk and simply appends how many zeros are needed:\n```\n0010 + 00 =\u003e 001000\n```\n`001000` is 8 in decimal, so Base64 appends `I` to the result\n\nBut then, on the decoding side, how do you know where real data ends and where the pretend data starts?\n\nIt turns out that we don't need to do anything. On the decoding side, we know that the decoded data _has_ to be a multiple of 8 bits. So, the decoder ignores the bits which make the output _not_ a multiple of 8 bits, which will always be the extra bits we added.\n\nFinally, encoding `00001001 10010010` to Base64 should result in `CZI`\n\nTry this in your terminal with the real Base64!\n```shell\necho -n -e \\\\x09\\\\x92 | base64 # base64 also adds a \"=\" character called \"padding\" to fit to a standard input length to output length ratio\n```\n\n### Aces\n\nNow we generalize this to all character sets of any length.\n\nGeneralizing the characters is easy, we just switch out the characters of the array storing the character set.\n\nChanging the length of the character set is harder. For every character set length, we need to figure out how many bits the chunked data should have. \n\nIn the Base64 example, the chunk length (let's call it that) was 6. The character set length was 64.\n\n[comment]: \u003c\u003e (Let's do another example: in octal, the character set length is 8 and the chunk length will be 3 \u0026#40;`000` to `111` = 0 to 7\u0026#41;)\n\n[comment]: \u003c\u003e (For a character set length of 4, we'd need a chunk length of 2 \u0026#40;`00` to `11` is 0 to 3\u0026#41;)\n\n[comment]: \u003c\u003e (```text)\n\n[comment]: \u003c\u003e (set len =\u003e chunk len)\n\n[comment]: \u003c\u003e (     4  =\u003e 2)\n\n[comment]: \u003c\u003e (     8  =\u003e 3)\n\n[comment]: \u003c\u003e (     64 =\u003e 6)\n\n[comment]: \u003c\u003e (```)\nIt looks like `2^(chunk len) = set len`. We can prove this is true with this observation:\n\nEvery bit can either be 1 or 0, so the total possible values of a certain number of bits will just be `2^(number of bits)` (if you need further proof, observe that every bit we add doubles the total possibilities since there's an additional choice: the new bit being 0 or the new bit being 1)\n\nThe total possible values is the length of the character set (of course, since we need the indices to cover all the characters of the set)\n\nSo, to find the number of bits the chunked data should have, we just do `log2(character set length)`. Then, we divide the bytes into chunks of that many bits (which was pretty hard to implement: knowing when to read more bytes, crossing over into the next byte to fetch more bits, etc, etc.), use those bits as indices for the user-supplied character set, and print the result.\n\nUnfortunately, this algorithm only works for character sets with a length that is a power of 2. For character sets with a length that is not a power of 2, we need to do something else.\n\n\nSets that are not power of 2 in length use an algorithm that may not have the same output as other encoders with the\nsame character set. For example, using the base58 character set does not mean that the output will be the same as a base58-specific encoder.\nThis is because most encoders interpret data as a number and use a base conversion algorithm to convert it to the\ncharacter set. For non-power-of-2 charsets, this requires all data to be read before encoding, which is not possible\nwith streams. To enable stream encoding for non-power-of-2 charsets, Aces converts the base of a default of 8 bytes of data at a time, which is not the same as converting the base of the entire data.\n\nEasy! (Nope, this is the work of several showers and a lot of late night pondering :) \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackduck%2Faces","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquackduck%2Faces","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackduck%2Faces/lists"}