{"id":18899930,"url":"https://github.com/hcengine/hcproto","last_synced_at":"2025-04-15T02:35:06.693Z","repository":{"id":259414794,"uuid":"875986681","full_name":"hcengine/hcproto","owner":"hcengine","description":"simple binary proto","archived":false,"fork":false,"pushed_at":"2025-01-03T08:24:23.000Z","size":180,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T07:06:56.332Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hcengine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-21T08:00:39.000Z","updated_at":"2025-02-15T09:45:06.000Z","dependencies_parsed_at":"2024-10-25T07:24:19.961Z","dependency_job_id":"2c201049-2699-4e30-b78c-f9ba49e5e90d","html_url":"https://github.com/hcengine/hcproto","commit_stats":null,"previous_names":["hcengine/hcproto"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hcengine%2Fhcproto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hcengine%2Fhcproto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hcengine%2Fhcproto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hcengine%2Fhcproto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hcengine","download_url":"https://codeload.github.com/hcengine/hcproto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248994862,"owners_count":21195462,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T08:48:00.329Z","updated_at":"2025-04-15T02:35:06.658Z","avatar_url":"https://github.com/hcengine.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hcproto\nsimple binary proto\n一种对标JSON的二进制数据协议, 支持JSON的所有类型的动态组合\n\n## 支持的数据类型\n基本支持的类型 \"u8\",   \"i8\",   \"u16\",   \"i16\",   \"u32\",   \"i32\", \"u64\",   \"i64\", \"varint\", \"f32\", \"f64\", \"string\",  \"raw\", \"array\",  \"map\"\n\n### 各种数值类型格式说明\n- u8/i8 用一个字节进行写入\n- u16/i16/u32/i32/u64/i64 分别对应大小的数据写入, 小端模式\n- f32 按IEEE754进行写入\n- f64 按IEEE754进行写入\n- varint 可变长的整型数据\n\u003e 如果是正数则*2, 如果是负数则-(x + 1) * 2, 相当于0-\u003e0, -1-\u003e1, 1-\u003e2,-2-\u003e3,2-\u003e4来做处理, 因为是小子节的数比较多, 每bit里的第一位则表示是否是最后一位, 如果10000001, 则表示还要继续往下读如果是00000001则表示这是最后一位\n- str 字符串类型, 则先用varint表示str的长度, 然后再写入str的值\n- str_idx 字符串索引值, 在str的arr表中的第几位, 重复的str则在同一个位置, 用varint表示\n- array 数组类型, 先用varint表示array的长度, 然后再写入各个value的数值\n- map map类型, 先用varint表示map的长度, 然后先写入key, 再写入value, 依次循环到结束\n\n## 与protobuf差异\n\u003e 相对protobuf, 无需预先定义任何的数据格式, 更好的适应多变的场景, 或者客户端不好更新的情况, 拥有更好的自适应性, 简单开封即用, 和JSON一样, 在可支持的数据类型里, 可以自由的进行转换\n## 与JSON的差异\n\u003e 可以把这个看做是二进制的JSON格式, 有更好的压缩率和更快的解析速度\n\n\n## 数据使用, 以Rust为例\n```rust\nuse algorithm::buf::Bt;\nuse hcproto::{Buffer, Value};\nuse std::time::SystemTime;\n\nmod test_data;\nuse std::collections::HashMap;\n\nfn test_level4_json() {\n    let mut now = SystemTime::now();\n    let parsed = test_data::get_json();\n    println!(\"解析JSON用时 = {:?}\", now.elapsed());\n    now = SystemTime::now();\n    // println!(\"ok!!! parsed= {:?}\", parsed);\n    let mut buffer = Buffer::new();\n    hcproto::encode_proto(\u0026mut buffer, \u0026\"cmd_level4_full\".to_string(), vec![parsed]).unwrap();\n    println!(\n        \"用tunm_proto压缩test_level4_json的长度 = {}k\",\n        buffer.remaining() / 1024\n    );\n\n    println!(\"压缩JSON耗时 = {:?}\", now.elapsed());\n    now = SystemTime::now();\n    let read = hcproto::decode_proto(\u0026mut buffer).unwrap();\n    println!(\"解析buffer耗时 = {:?}\", now.elapsed());\n    match read {\n        (name, _val) =\u003e {\n            assert_eq!(name, \"cmd_level4_full\".to_string());\n            // println!(\"value === {:?}\", val);\n        }\n    }\n}\n\n```\n\n### 格式说明\n数据协议分为三部分(协议名称, 字符串索引区, 数据区(默认为数组))\n如数据协议名为cmd_test_op, 数据为[\"tunm_proto\", {\"name\": \"tunm_proto\", \"tunm_proto\": 1}]\n1. 那么数据将先压缩协议名cmd_test_op, 将先写下可变长度(varint)值为11占用1字节, 然后再写入cmd_test_op的utf8的字节数\n2. 接下来准备写入字符串索引区, 索引数据用到的字符串为[\"tunm_proto\", \"name\"]两个字符串, 即将写入可变长度(varint)值为2占用一字节, 然后分别写入字符串tunm_proto和name两个字符串, 这样子字符串相接近有利于压缩, 且如果有相同的字符串可以更好的进行复用\n3. 接下来准备写入数据区, \n首先判断为一个数组, 写入类型u8(TYPE_ARR=16), 写入数组长度varint(2), 准备开始写第一个数据, 字符串tunm_proto, 已转成id, 则写入类型u8(TYPE_STR_IDX=14), 查索引号0, 则写入varint(0), 第一个字段写入完毕, 接下来第二个字段是一个map数据, 写入map长度varint(2), 然后进行遍历得到key值为name, 则写入写入类型u8(TYPE_STR_IDX=14),查索引号1, 则写入varint(1), 然后开始写name对应的值tunm_proto, 写入TYPE_STR_IDX类型的0值, 则这组key写入完毕, 依此类推写入第二组数据\n\n测试打印的结果\n用完整的level-full4.json\n\n```\n解析JSON用时 = Ok(3.104869s)\n用tunm_proto压缩test_level4_json的长度 = 370k\n压缩JSON耗时 = Ok(22.2712ms)\n解析buffer耗时 = Ok(16.9738ms)\n普通文本的长度 = 90\n```\n解析速度约为JSON的68倍, 符合预期, 大小为明文的0.16倍, 符合压缩比\n\n#### 相关连接\n[协议地址https://github.com/hcengine/hcproto](https://github.com/hcengine/hcproto)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhcengine%2Fhcproto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhcengine%2Fhcproto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhcengine%2Fhcproto/lists"}