{"id":16269556,"url":"https://github.com/hiyouga/information-theory-experiment","last_synced_at":"2025-04-08T15:18:13.817Z","repository":{"id":111025127,"uuid":"191120052","full_name":"hiyouga/information-theory-experiment","owner":"hiyouga","description":"BUAA CST Spring 2019 Information Theory Experiment","archived":false,"fork":false,"pushed_at":"2019-11-23T09:16:46.000Z","size":145,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-14T11:33:50.643Z","etag":null,"topics":["aep","compression-algorithm","information-theory"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hiyouga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-10T07:36:42.000Z","updated_at":"2021-06-10T01:41:25.000Z","dependencies_parsed_at":"2024-03-15T12:04:59.008Z","dependency_job_id":null,"html_url":"https://github.com/hiyouga/information-theory-experiment","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hiyouga%2Finformation-theory-experiment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hiyouga%2Finformation-theory-experiment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hiyouga%2Finformation-theory-experiment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hiyouga%2Finformation-theory-experiment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hiyouga","download_url":"https://codeload.github.com/hiyouga/information-theory-experiment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247867365,"owners_count":21009240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aep","compression-algorithm","information-theory"],"created_at":"2024-10-10T18:08:37.359Z","updated_at":"2025-04-08T15:18:13.798Z","avatar_url":"https://github.com/hiyouga.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 信息论课程实验\n\nBUAA CST Spring 2019 Information Theory Experiment\n\n懒得用英文了，反正就是三个很水的小程序，但是还蛮好玩的。\n\n## [aep.py](src/aep.py)\n\n### Requirement\n\n- Python 3\n- scipy\n- decimal\n\n### Introduction\n\n（赶着DDL写出来的，原理对了就行）\n\n验证了随机信源序列的**渐近等同分割性**(Asymptotic Equipartition Property, AEP)：\n\n\u003e 当随机变量的序列足够长时，其中一部分序列出现的概率近乎相等，每个信源符号的平均信息量接近于信源的熵H(U)，且这部分序列出现的概率和趋近于1，称为**典型序列集**，而其余的非典型序列集出现的概率和趋近于0。典型序列又称为渐近等概序列或AEP序列。实质上就是大数定律在信息论里的应用吧。\n\n实验中使用的是离散无记忆信源输出的消息序列，因为浮点数精度问题用了高精度模块计算，实验现象与理论吻合。\n\n### Usage\n\n```bash\npython aep.py\n```\n\n### Results\n\n参数：`epsilon=0.05, P(X=1)=0.6`\n\n| 序列长度 | 典型集概率和 | 典型序列数量 | 典型序列数量占比 |\n| -------- | ------------ | ------------ | ---------------- |\n| 25       | 0.69         | 1.6*10^7     | 0.48             |\n| 50       | 0.81         | 5.0*10^14    | 0.44             |\n| 100      | 0.92         | 4.8*10^29    | 0.38             |\n| 200      | 0.99         | 5.8*10^59    | 0.36             |\n| 500      | 0.99         | 8.2*10^149   | 0.25             |\n| 1000     | 0.99         | 1.9*10^300   | 0.18             |\n\n## [huffman.py](src/huffman.py)\n\n### Requirement\n\n- Python 3\n- queue\n- argparse\n- matplotlib\n\n### Introduction\n\n使用Huffman最佳不等长编码对文件进行编码（压缩）和译码（解压缩）。由于我们并不知道信源的真正分布，所以只能通过文件中的统计规律近似其分布。\n\n编码过程基本就是构造Huffman树，然后遍历树得到每个符号的不等长编码，对文件进行编码。同时还需要把码字对照表存到文件里用于译码，由于存储文件时需要字节对齐，所以要在后面对填充，为了避免填充的符号使译码时候中产生困惑，也同时记录了文件大小。最后模仿传统压缩文件，把文件名也记录了进去，最终压缩文件的存储结构是：\n\n`文件名|0x00|文件大小|0x00|对应码表|文件内容编码|00...`\n\n由于我们将每个字节视为一个信源符号，所以共有256个信源符号，每个符号都按序存在码字对照表中，按序可以省去我们存储信源符号的空间，存储结构是：\n\n`码字长度|对应码字|00...`\n\n译码过程首先恢复码字对照表，然后按照一般异字头编码的译码规则即可。\n\n最终也是由于时间原因，优化应该没有做到头，至少原理是没错的，Python的优化也是各种玄学。（还是因为自己对Python底层懂得太少）\n\n### Usage\n\n```bash\npython huffman.py -p file_path -j [encode|decode|eval] -b show_bar -i display_info\n```\n\n显示帮助：\n\n```bash\npython huffman.py -h\n```\n\n### Results\n\n为了写实验报告，我对许多文件做了测试，但是不在这里一一贴出来了，大致说一下程序性能：\n\n英文文本文件的压缩率：**~60%**\n\n位图文件的压缩率：**~65%**\n\n其余已经经过压缩的文件格式(pdf/docx/jpg/png/mp3/mp4)：**~100%**\n\n编码速度：**850KB/s±200KB/s**\n\n译码速度：**300KB/s±50KB/s**\n\n## [lz78.py](src/lz78.py)\n\n### Requirement\n\n- Python 3\n- argparse\n- matplotlib\n\n### Introduction\n\n使用LZ78算法对文件进行编码（压缩）和译码（解压缩），LZ系列算法并不需要预先知道信源的分布，利用的是字典技术，而且字典本身不需要传输，所以是一种很巧妙的方法，与Huffman编码算法一样可以逼近信息熵的极限。\n\n编码过程是首先将输入序列分段，分段规则是与之前分段均不相同的最短序列，然后从分段数量得到段号的码长，结合符号的编码，对每个分段进行编码。具体算法不再赘述，可以参考《信息论与编码 第二版》（王育民著）中的介绍。\n\n最终压缩文件的存储结构是：\n\n`文件名|0x00|文件大小|0x00|段号码长|0x00|文件内容编码|00...`\n\n译码时候一边译码一边建立字典，从而恢复出文件内容。\n\n（~~优化 Go Away~~）\n\n### Usage\n\n```bash\npython lz78.py -p file_path -j [encode|decode|eval] -b show_bar -i display_info\n```\n\n显示帮助：\n\n```bash\npython lz78.py -h\n```\n\n### Results\n\n英文文本文件的压缩率：**~50%**\n\n位图文件的压缩率：**~50%**\n\n其余已经经过压缩的文件格式(pdf/docx/jpg/png/mp3/mp4)：**~100%**\n\n编码速度：**450KB/s±200KB/s**\n\n译码速度：**550KB/s±50KB/s**\n\n（为什么编码速度浮动这么大？不等长编码下由于信源分布不同导致？）\n\n## Example\n\n放几个编码程序的运行效果图=w=\n\n### 编码\n\n![encode](assets/encode.gif)\n\n### 评估\n\n![eval](assets/eval.gif)\n\n## License\n\nThis project is licensed under the terms of the [MIT license](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiyouga%2Finformation-theory-experiment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhiyouga%2Finformation-theory-experiment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiyouga%2Finformation-theory-experiment/lists"}