{"id":22555166,"url":"https://github.com/tdjsnelling/babel","last_synced_at":"2025-04-09T13:03:52.356Z","repository":{"id":76807726,"uuid":"463869271","full_name":"tdjsnelling/babel","owner":"tdjsnelling","description":"An open source Library of Babel implementation in TypeScript using the GNU Multiple Precision Arithmetic Library","archived":false,"fork":false,"pushed_at":"2025-01-07T15:01:00.000Z","size":19031,"stargazers_count":82,"open_issues_count":1,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-02T12:11:43.647Z","etag":null,"topics":["babel","generative-art","gmp","library","library-of-babel","nodejs","pug","typescript"],"latest_commit_sha":null,"homepage":"https://libraryofbabel.app","language":"Pug","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tdjsnelling.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-26T14:03:14.000Z","updated_at":"2025-03-23T02:20:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"9f89aa39-8266-4c27-ac2d-7f899231a159","html_url":"https://github.com/tdjsnelling/babel","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdjsnelling%2Fbabel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdjsnelling%2Fbabel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdjsnelling%2Fbabel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdjsnelling%2Fbabel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tdjsnelling","download_url":"https://codeload.github.com/tdjsnelling/babel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045230,"owners_count":21038553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["babel","generative-art","gmp","library","library-of-babel","nodejs","pug","typescript"],"created_at":"2024-12-07T19:06:50.014Z","updated_at":"2025-04-09T13:03:52.322Z","avatar_url":"https://github.com/tdjsnelling.png","language":"Pug","funding_links":[],"categories":[],"sub_categories":[],"readme":"# babel\n\nA functional, complete, true-to-scale re-creation of the [Library of Babel](https://en.wikipedia.org/wiki/The_Library_of_Babel) [[.pdf](https://libraryofbabel.app/pdf/Borges-The-Library-of-Babel.pdf)].\n\n### What is it?\n\nThe Library of Babel is a short story written by [Jorge Luis Borges](https://en.wikipedia.org/wiki/Jorge_Luis_Borges).\n\nIt describes a library made up of an “indefinite, perhaps infinite” number hexagonal of rooms, each lined on four sides by a bookshelf of five shelves, each self containing thirty-two books. Each book is four hundred and ten pages of forty lines, each line of eighty characters.\n\nIn the story, each book is made up of “twenty-five orthographic symbols” consisting of twenty-two lowercase letters, the comma, the full-stop, and the space. This version expands that to thirty-two symbols: the twenty-six lower-case letters of the English alphabet, the comma, full-stop, exclamation mark, question mark, hyphen, and space.\n\nIn its pages, the library contains every possible combination of these characters. No two books are the same — meaning that the library is “total — perfect, complete, and whole”. Everything that ever has been, or ever will be written using these thirty-two characters is contained somewhere within the library.\n\n\u003e All-the detailed history of the future, the autobiographies of the archangels, the faithful catalog of the Library, thousands and thousands of false catalogs, the proof of the falsity of those false catalogs, a proof of the falsity of the true catalog, the gnostic gospel of Basilides, the commentary upon that gospel, the commentary on the commentary on that gospel, the true story of your death, the translation of every book into every language, the interpolations of every book into all books, the treatise Bede could have written (but did not) on the mythology of the Saxon people, the lost books of Tacitus. \n\nThis is an explorable representation of the complete library, written in TypeScript.\n\n### Version 3\n\nThis is the third version of my implementation of a Library of Babel. \n\n#### v1\n\nThe first was written purely in JavaScript, and contained only all unique *pages*, rather than all unique *books*. \n\nThis meant that a page of prose followed by a page of random characters could not also exist in the library followed by a page of continuing prose. \n\nThe [bignumber.js](https://mikemcl.github.io/bignumber.js/) simply was not capable of working with numbers big enough to build a complete library – in the realm of 29^1312000 rather than 29^3200.\n\nThis initial, limited implementation is still available on the `v1` branch.\n\n#### v2\n\nIn the second version, the core logic was re-written in C to make use of the [GMP](https://gmplib.org/) library. This combination allowed for much enabled computation using the extremely large numbers required. \n\nI am not a C programmer, so the code was a mess and likely had some glaring issues that I was not aware of – but it worked. \n\nThe web front-end was still written in JS + Pug, and interacted with the C core via the [Node-API](https://nodejs.org/api/n-api.html).\n\nThis version was *slow*, prompting another iteration.\n\nThis implementation is still available on the `v2` branch.\n\n#### v3\n\nIn this third iteration, the C code is removed in favour of the [gmp-wasm](https://github.com/Daninet/gmp-wasm) GMP\u003c-\u003eJS binding.\n\nThis means I can work in JS rather than C—which I am much more comfortable with—while still getting the benefits that come with using GMP.\n\nThis also means I can do away with the awkward Node-API code and GYP build process. The JavaScript is replaced with TypeScript which helped me catch a few bugs.\n\nImportantly, the 29 character alphabet is expanded to contain 32 characters. This means that calculations are now done in base-32 rather than base-29, resulting much faster conversions in and out of this base-X representation — as this was the primary bottleneck in the slow v2.\n\n### How do I use it?\n\nYou can play with a live instance at [libraryofbabel.app](https://libraryofbabel.app).\n\nAlternatively, you can clone this repo and run it yourself.\n\nTo build and start on `http://localhost:3000`:\n\n```\n$ yarn install\n$ yarn build\n$ yarn start\n```\n\nYou can then look up a page at the `/ref/...` endpoint, e.g. `/ref/1.1.1.1.1`.\n\nYou can search for a page containing some content at `/search`, navigate to a specific page at `/browse`, and you can visit a random page at `/random`.\n\n### How does it work?\n\nThe contents of each book is intrinsically linked to it’s index in the library. Books are not generated and stored as you search for them, as the storage requirements would be beyond possibility. Instead, the book index is run through an algorithm that produces the contents of the book. This algorithm is reversible, so we can give it the contents of a book and determine the index of the book it appears in.\n\nEach book will contain the same contents forever. There is no trickery going on to simply show you a made-up book containing the text that you search for.\n\nYou can read more on the [home](https://libraryofbabel.app) and [about](https://libraryofbabel.app/about) pages.\n\n#### Detailed explanation\n\nFirstly, each individual page in the library is given an identifier. For readability, this is represented in the form `ROOM.WALL.SHELF.BOOK.PAGE`, where:\n\n* _ROOM_ is an alphanumeric (`[0-9a-v]`) string treated as a base-32 integer.\n* _WALL_ is an integer 1-4.\n* _SHELF_ is an integer 1-5.\n* _BOOK_ is an integer 1-32.\n* _PAGE_ is an integer 1-410.\n\nSo using this representation, the 200th page of the 16th book on the 3rd shelf on the 1st wall of the 90th room would be `2q.1.3.16.200`.\n\nOnce we have an identifier, from this we can determine a globally unique book index (pages are not considered here). For example:\n\n* The identifier `1.1.1.1.1` gives us book 1.\n* The identifier `1.1.1.2.1` gives us book 2.\n* The identifier `1.1.2.1.1` gives us book 33\n* The identifier `1.2.1.1.1` gives us book 161, and so on.\n\nNow that we have a unique book index, we can do some mathematical operations to generate the contents of the book using the book index as a sort of 'seed'.\n\nWe work in base-32, as that is the number of characters in our alphabet. We need to define some constants, which can be found in the `numbers` file.\n\nThis file contains `N`, `C` and `I` on 3 separate lines. They are: \n\n* `N` is largest possible base-32 number of 1,312,000 digits (1,312,000 is the number of characters in a book).\n* `C` is a large, randomly generated number that is coprime to `N`.\n* `I` is the [modular multiplicative inverse](https://en.wikipedia.org/wiki/Modular_multiplicative_inverse) to `C`.\n\nThe code to generate the `numbers` file can be found in `src/utils/gen-constants.ts`. Changing `C` and `I` will change the book index \u003c-\u003e book contents mapping.\n\nA better explanation of why we need these constants can be found in the blog post [A practical use of multiplicative inverses](https://ericlippert.com/2013/11/14/a-practical-use-of-multiplicative-inverses).\n\nThe operations we need to perform are:\n\n* `bookContentValue = bookIndex * C % N`, and\n* `bookIndex = bookContentValue * I % N`.\n\n`bookContentValue` is a 1,312,000 digit base-32 number. To determine the actual content of the book, we take each digit of this number one-by-one, convert it to base-10, and then select the character at that index of our [alphabet array](./src/constants.ts):\n* `0` becomes an `a`\n* `k` (or base-10 `20`) becomes a `t`\n* `u` (or base-10 `30`) becomes a `?`, and so on.\n\nTo turn a book’s text content back into a `bookContentValue`, we just do the opposite: take the index in our alphabet array of each character in the book, convert it to base-32, and append it to a value that becomes our 1,312,000 digit base-32 number `bookContentValue`. Then we can calculate the book index, and thus look it up in the library.\n\nWhen a specific page is accessed, the entire book is generated and split into 410 pages, and the relevant page is returned.\n\n### Why is a database required if books are not stored on disk?\n\nThe alphanumeric room identifiers get very long, up to around 1,000,000 characters. This makes them too long to use in URLs, so the database is responsible for holding 'bookmarks' to each room.\n\nWhen a room is visited for the first time, it’s SHA-256 hash is calculated and stored in a LevelDB database alongside the actual room identifier. This hash is used in the URL in place of the real room identifier.\n\n#### But are there not way more unique room identifiers than unique hashes?\n\nCorrect. In theory, when enough unique rooms are discovered hashes will start to collide, and old bookmarks will start to be overwritten as new rooms are discovered. In reality this very unlikely to happen, as this many rooms will not be visited. \n\n### Prior art\n\n[Jonathan Basile’s](https://jonathanbasile.info/) brilliant site [libraryofbabel.info](https://libraryofbabel.info/) contains another implementation of the Library of Babel, which contains every unique page, but not every unique book. Jonathan’s site also contains lots of great supplemental theory and ideas around the library. His book on the subject, [Tar for Mortar](https://punctumbooks.com/titles/tar-for-mortar/), is also worth the read.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftdjsnelling%2Fbabel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftdjsnelling%2Fbabel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftdjsnelling%2Fbabel/lists"}