{"id":19410187,"url":"https://github.com/yurisizuku/toolkit-localization","last_synced_at":"2025-04-24T10:32:24.197Z","repository":{"id":213114482,"uuid":"733056058","full_name":"YuriSizuku/toolkit-Localization","owner":"YuriSizuku","description":"My toolkit for galgame localization","archived":false,"fork":false,"pushed_at":"2024-10-31T16:45:05.000Z","size":573,"stargazers_count":54,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T03:33:29.466Z","etag":null,"topics":["ftext","galgame","localization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YuriSizuku.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-18T13:16:11.000Z","updated_at":"2025-03-30T06:15:42.000Z","dependencies_parsed_at":"2023-12-18T15:12:47.958Z","dependency_job_id":"10e127e4-cf7f-4431-8f80-a7b2a5e4ab44","html_url":"https://github.com/YuriSizuku/toolkit-Localization","commit_stats":null,"previous_names":["yurisizuku/localizationtool","yurisizuku/toolkit-localization"],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuriSizuku%2Ftoolkit-Localization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuriSizuku%2Ftoolkit-Localization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuriSizuku%2Ftoolkit-Localization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuriSizuku%2Ftoolkit-Localization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YuriSizuku","download_url":"https://codeload.github.com/YuriSizuku/toolkit-Localization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250609133,"owners_count":21458434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ftext","galgame","localization"],"created_at":"2024-11-10T12:15:04.921Z","updated_at":"2025-04-24T10:32:23.335Z","avatar_url":"https://github.com/YuriSizuku.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LocalizationTool\n\n![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/YuriSizuku/LocalizationTool?label=LocalizationTool\u0026color=green) ![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/YuriSizuku/LocalizationTool/build_pyexe.yml?label=pyexe)  ![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/YuriSizuku/LocalizationTool/build_pysrc.yml?label=pysrc)  \n\n🍀 General localization tools for galgame, seperated from my [WinReverse](https://github.com/YuriSizuku/toolkit-WinReverse)  \nSee also, [GalgameReverse](https://github.com/YuriSizuku/GalgameReverse) for specific galgames.  \n\n## Components\n\n### cross platfrom libraries\n\n* `libutil.py`, basic serilization functions for ftext and tbl  \n* `libtext.py`, for text exporting and importing, checking\n* `libimage.py`, something about texture and picture convert  \n* `libfont.py`, for extracting, building tile font, or generating font picture.\n* `libword.py`, some matching and statistic method for text  \n* `ftextcvt.py`, convert the `ftext` format made by `libtext.py`  \n* `ftextpack.py`, method for packing `ftext` in a bin file with lower memory  \n\n### windows platform libraries\n\n* `winconsole.js`,  allocate a console for game  \n* `winfile.js` , view information for both `CreateFile`, `ReadFile`, `WriteFile`, `fopen`,`fread`, `fwrite`  \n* `winredirect.js`, redirect font, codepage, and paths in games  \n\n``` mermaid\n%%{init: {'theme':'forest'}}%%\ngraph LR;\nl1[libutil.py]\nl2[libtext.py];\nl3[libimage.py]\nl4[libfont.py]\nl5[libword.py]\nt1[ftextcvt.py]\nt2[ftextpack.py]\nw1[winconsole.js]\nw2[winfile.js]\nw3[windirect.js]\n\nl1 --\u003e l2 --\u003e t2\nl1 --\u003e l5  \nl1 --\u003e l4\nl2 --\u003e t1\nl1 --\u003e l3 --\u003e l4\nw1 --- w2 --- w3\n```\n\n## CLI Example\n\nInstall these libraries if you want to use python script.  \n\n```shell\npython -m pip install python-docx # ftextcvt\npython -m pip install numpy numba pillow # libfont, libimage\npython -m pip install scikit-learn # libimage, for kmeans method\n```\n\nUse these scripts to testing\n\n```shell\nsh project/pysrc_all/test_pyunit.sh\nsh -c \"source project/pysrc_all/test_pycli.sh \u0026\u0026 test_all\"\n```\n\n**We use \"\u003e\" to load or save files in zip, such as `path1/file1.zip\u003epath2/file2`.**\n**As for the `--batch` mode, replace the inpath and outpath files to the list (txtfile, or string like `dir1;path1;path2...`), usually the first path is for the base directory location**\n\nSee [test_pycli.sh](project/pysrc_all/test_pycli.sh) for details, binary build on winwdows are in [release](https://github.com/YuriSizuku/LocalizationTool/releases).  \n\n### libtext\n\n```shell\n# insert ftext (save direct or in gz file)\npython src/libtext.py insert test/sample/COM001 test/sample/COM001.txt --refer test/sample/COM001 -t test/sample/COM001.tbl -o project/pysrc_all/build/COM001_rebuild.bin --log_level info --bytes_padding \"2020\" --bytes_fallback \"815A\" --insert_shorter --insert_longer  --text_replace \"季\" \"季季季\" --text_replace \"煌びやかな光\" \"你你你你你\" \npython src/libtext.py insert test/sample/COM001 test/sample/COM001.txt --refer test/sample/COM001 -t test/sample/COM001.tbl -o project/pysrc_all/build/COM001_rebuild.bin.gz --log_level info\n\n# extract ftext from bin file (save direct or in zip file)\npython src/libtext.py extract project/pysrc_all/build/COM001_rebuild.bin -o \"project/pysrc_all/build/COM001.zip\u003eCOM001/COM001_rebuild.txt\" --log_level info -e sjis --has_cjk --min_len 4 --skip 0x16 --size 1024\n\n# check ftext (direct or in zip file)\npython src/libtext.py check \"project/pysrc_all/build/COM001.zip\u003eCOM001/COM001_rebuild.txt\" --refer project/pysrc_all/build/COM001_rebuild.bin -o \"project/pysrc_all/build/COM001.zip\u003eCOM001/COM001_rebuild_check.txt\" --log_level info -e sjis\n```\n\n### libfont\n\n```shell\n# font tbl operation\npython src/libfont.py tbl_make cp932 --tchar_replace \"亜\" \"亚\" -o \"project/pysrc_all/build/sjis.tbl\"\npython src/libfont.py tbl_make cp936 -o \"project/pysrc_all/build/gb2312.tbl\"\npython src/libfont.py tbl_align \"project/pysrc_all/build/sjis.tbl\" -o \"project/pysrc_all/build/sjis_align.tbl\" --gap_static --tbl_padding \"ff\" \"x\" --gap 0 2 --gap 2 -2\npython src/libfont.py tbl_merge --intersect \"project/pysrc_all/build/sjis.tbl\" \"project/pysrc_all/build/gb2312.tbl\" -o \"project/pysrc_all/build/sjis_gb2312_merge.tbl\" --range_reserve 0 70\n\n# make glphy operation\npython src/libfont.py font_make --format image \"C:\\Windows\\Fonts\\simhei.ttf\" --tbl \"test/sample/COM001.tbl\" -o \"project/pysrc_all/build/com001_font24.png\" --tilew 24 --tileh 24\npython src/libfont.py font_make --format tile \"C:\\Windows\\Fonts\\simhei.ttf\" --tbl \"test/sample/COM001.tbl\" -o \"project/pysrc_all/build/com001_font2418_8bpp.bin\" --tilew 24 --tileh 18 --tilebpp 8\npython src/libfont.py font_make --format tile \"C:\\Windows\\Fonts\\simhei.ttf\" --tbl \"test/sample/COM001.tbl\" -o \"project/pysrc_all/build/com001_font1614_2bpp.bin\" --tilew 16 --tileh 14 --tilebpp 2 --palette \"00 00 00 00 ff ff ff 60 ff ff ff a0 ff ff ff ff\"\n\n# extract glphy operation\nmkdir -p \"project/pysrc_all/build/com001_font24\" \nmkdir -p \"project/pysrc_all/build/it\"\npython src/libfont.py font_extract --format image \"project/pysrc_all/build/com001_font24.png\" -o \"project/pysrc_all/build/com001_font24\" --split_glphy --tilew 24 --tileh 24\npython src/libfont.py font_extract --format tile \"test/sample/it.bin\" -o \"project/pysrc_all/build/it\" --split_glphy --tilew 20 --tileh 18 --tilebpp 2 --tilesize 92 --palette \"ff ff ff 00 ff ff ff 3f ff ff ff 8f ff ff ff ff\"\npython src/libfont.py font_extract --format tile \"test/sample/it.bin\" -o \"project/pysrc_all/build/it.jpg\" --tilew 20 --tileh 18 --tilebpp 2 --tilesize 92 --palette \"ff ff ff 00 ff ff ff 3f ff ff ff 8f ff ff ff ff\"\n\n```\n\n### libimage\n\n```shell\n# decode tile to image\npython src/libimage.py decode --format tile \"test/sample/it.bin\" -o \"project/pysrc_all/build/it_decode.png\" --tilew 20 --tileh 18 --tilebpp 2 --tilesize 92 --palette \"ff ff ff 00 ff ff ff 3f ff ff ff 8f ff ff ff ff\" \n\n# encode image to 1 tile\npython src/libimage.py encode --format tile \"project/pysrc_all/build/it_decode.png\" -o \"project/pysrc_all/build/it_encode1.bin\" --tilebpp 2 --palette \"ff ff ff 00 ff ff ff 3f ff ff ff 8f ff ff ff ff\"\n```\n\n### libword\n\n```shell\n# match ftext_now\npython src/libword.py match --format ftext_now test/sample/COM001.txt test/sample/COM001.txt -o project/pysrc_all/build/COM001_match.csv\n\n# count chars in file\npython src/libword.py count --format ftext_org test/sample/COM001.txt -o project/pysrc_all/build/COM001_count.csv -n 100\n```\n\n### ftextpack\n\n```shell\n# pack compact mode in zip file, and batch example\npython src/ftextpack.py test/sample/COM001 test/sample/COM001.txt -o \"project/pysrc_all/build/COM001.zip\u003eCOM001/COM001.fp01\" -t test/sample/COM001.tbl --pack_compact\npython src/ftextpack.py --batch \"test/sample;COM001\" \"test/sample;COM001.txt\" -o \"project/pysrc_all/build;COM001.zip\u003eCOM001/COM001.fp02\" -t test/sample/COM001.tbl --pack_compact\n```\n\n### ftextcvt\n\n```shell\n# json convert\npython src/ftextcvt.py test/sample/COM001.txt -o project/pysrc_all/build/COM001.json\npython src/ftextcvt.py project/pysrc_all/build/COM001.json -o project/pysrc_all/build/COM001.json.txt\n\n# split merge ftext\npython src/ftextcvt.py test/sample/COM001.txt -o project/pysrc_all/build/COM001_split.txt --split 3\npython src/ftextcvt.py test/sample/COM001_split_*.txt -o project/pysrc_all/build/COM001_merge.txt --merge 0\n```\n\n## File Formats\n\n### ftext (translation format text)  \n\nThe ftext files are using `utf-8 unix lf` format to store. In the ftexts,  we use `●num|addr|size● org_text` for origin text reference and `○num|addr|size○ trans_text` for translation text edit.  Do not modify the index information within `●` or `○`, and must leave a space after `●` or `○`.  \n\nInside the ftext, `\\r` and `\\n` are replaced to `[\\r]` and `[\\n]`. We also use `{{}}` for input some custom formats or informations to process.  \n\n``` shell\n# ftext example  \n○00002|00018D|04C○ 湧き出る温泉と豊かな自然に包まれた風光明媚な地で、知る人ぞ知る観光地である。\n●00002|00018D|04C● 此地温泉涌流，自然繁茂，风光明媚。可谓是内行人都知晓的胜地。\n\n○00003|0001FD|00A○ 季節は夏。\n●00003|0001FD|00A● 时值夏日。{{b'\\xff'}}\n\n○00004|000253|068○ 残月島にある唯一の街\\n『@r紅霞市（こうかし）@0』では、ここ最近の不況が嘘のように盛り上がりを見せていた。\n●00004|000253|068● 在残月岛{{'唯一'.encode('sjis')}}的市区“@r红霞市@0”里，近来经济之萧条每况愈下，已是人心惶惶。\n\n○00005|000307|056○ 『@r花柳街（かりゅうがい）@0』の一郭に存在する置屋に、上流階級のお客様が現れたからだ。\n●00005|000307|056● 因为有上流社会的客人来到了@r花柳街@0某郭的置屋。\n\n○00006|000381|032○ ――今、煌びやかな光の中を一人の美しい女性が往く。\n●00006|000381|032● ――此刻，正有一位美丽的女性，身披华光，款款行来。\n```\n\n### fpack (translation format text pack, ftextpack)  \n\nPacking ftext files into a bin file, for optimizing the memory usage.  Usually use `ftextpack.py` to pack ftext files and `ftextpack.h` to search ftext in the game dynamic translation.  \n\n### tbl (translation word encoding table)  \n\nIn the format of `tcode=tchar`, usally used for custom codepage and glphy mapping.  \n\n```shell\n8140=　\n8141=、\n8142=。\n8143=，\n8144=．\n8145=・\n8146=：\n8147=；\n8148=？\n8149=！\n814A=゛\n814B=゜\n814C=´\n814D=｀\n212F=¨\n```\n\n## Roadmap\n\n* [x] seperate Localizetion Tool from ReverseTool Repo, [v0.3.5sep](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.3.5sep)\n* [x] make unit test and cli test script\n* [x] write documentation about the project, such as format and cli example  \n* [x] remake `libtext.py`, `libutil.py` to make more pythonic and easy to understand, [v0.4beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4beta)\n* [x] remake `ftextpack.py`, `ftextcvt.py` and use unified format  \n* [x] remake `libfont.py`, `libimage.py`, use numba to improve performance, [v0.4.2beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4.2beta)\n* [x] finish `libfont.py` cli , [v0.4.3beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4.3beta), [0.4.5beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4.5beta)\n* [x] finish `libimage.py` cli, [v0.4.4beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4.4beta)\n* [x] remake `libword.py`, [v0.4.6beta](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.4.6beta)\n* [x] add collated batch files input to improve io performance [v0.5](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.5)\n* [x] add split/merge ftexts in `ftextcvt.py` [v0.5.3](https://github.com/YuriSizuku/LocalizationTool/releases/tag/v0.5.3)\n\n## History\n\nSee [History](project/pysrc_all/History.md).  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyurisizuku%2Ftoolkit-localization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyurisizuku%2Ftoolkit-localization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyurisizuku%2Ftoolkit-localization/lists"}