{"id":20278846,"url":"https://github.com/madladsquad/chitankaextractor","last_synced_at":"2025-06-26T07:03:59.531Z","repository":{"id":198184034,"uuid":"699982173","full_name":"MadLadSquad/ChitankaExtractor","owner":"MadLadSquad","description":"Small scripts to extract text from an archive of the chitanka.info website","archived":false,"fork":false,"pushed_at":"2025-03-13T22:18:11.000Z","size":27,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-03T08:30:06.531Z","etag":null,"topics":["bulgarian","chitanka","shell"],"latest_commit_sha":null,"homepage":"https://madladsquad.com/untitled-desktop/subprojects/i18n","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MadLadSquad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yaml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"ko_fi":"madladsquad"}},"created_at":"2023-10-03T18:03:27.000Z","updated_at":"2025-03-13T22:18:14.000Z","dependencies_parsed_at":"2023-12-25T00:23:15.046Z","dependency_job_id":"a875b783-74eb-4524-bde2-88356aaa77cf","html_url":"https://github.com/MadLadSquad/ChitankaExtractor","commit_stats":null,"previous_names":["madladsquad/chitankaextractor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MadLadSquad/ChitankaExtractor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLadSquad%2FChitankaExtractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLadSquad%2FChitankaExtractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLadSquad%2FChitankaExtractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLadSquad%2FChitankaExtractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MadLadSquad","download_url":"https://codeload.github.com/MadLadSquad/ChitankaExtractor/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLadSquad%2FChitankaExtractor/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262018760,"owners_count":23245619,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bulgarian","chitanka","shell"],"created_at":"2024-11-14T13:26:45.493Z","updated_at":"2025-06-26T07:03:59.514Z","avatar_url":"https://github.com/MadLadSquad.png","language":"Shell","funding_links":["https://ko-fi.com/madladsquad"],"categories":[],"sub_categories":[],"readme":"# ChitankaExtractor\n[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://lbesson.mit-license.org/)\n[![trello](https://img.shields.io/badge/Trello-UDE-blue])](https://trello.com/b/HmfuRY2K/untitleddesktop)\n[![Discord](https://img.shields.io/discord/717037253292982315.svg?label=\u0026logo=discord\u0026logoColor=ffffff\u0026color=7389D8\u0026labelColor=6A7EC2)](https://discord.gg/4wgH8ZE)\n\nSmall scripts to extract text from an archive of the [chitanka.info](https://chitanka.info) website.\n\n## About this project\nThe chitanka website offers archives of their whole royalty-free bulgarian book library. This data can be useful for many applications, such as LLMs.\nWe needed this data in our specific format, so we made this project to automatically convert the books, which come in `.epub` format, to `.html` files.\n\nAdditinally, scripts for setting the name of the file to not be a number and further converting the `.html` files to plaintext files, where each paragraph is on\na new line.\n\n## Usage\n### Getting the data\nYou can download an archive of the chitanka website from [this link](https://chitanka.info/resources/archives), as a torrent. When downloading an archive, download the\n`epub` version with numbers. For those who don't speak bulgarian, click the link saying `EPUB с номера`.\n\nThese torrents have a good number of seeders, but are only located in Bulgaria, so it may take significantly longer for one to download the data from a place outside of \nEurope, North Africa and the Middle East.\n\n### Unzipping the archive\nClone this repository, enter it and unzip the archive in this folder, so that the folders containing the `epub` files are in the same directory as the scripts. To convert\nthe `epub` files to `html`, run `./to-html.sh` script. The script requires `pandoc` and `parallel` to be installed.\n\n### Renaming\nTo rename the files with the HTML metadata, run `./rename-with-data.sh`.\n\n### Converting to plain text\nTo convert to plain text format, run `./to-plain-text.sh`. This script, strips all tags, except for `\u003cp\u003e` and replaces headings with new lines.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadladsquad%2Fchitankaextractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmadladsquad%2Fchitankaextractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadladsquad%2Fchitankaextractor/lists"}