{"id":13413801,"url":"https://github.com/pascaldekloe/colfer","last_synced_at":"2025-04-09T05:07:59.583Z","repository":{"id":37445186,"uuid":"41968300","full_name":"pascaldekloe/colfer","owner":"pascaldekloe","description":"binary serialization format","archived":false,"fork":false,"pushed_at":"2024-08-07T15:14:28.000Z","size":30228,"stargazers_count":748,"open_issues_count":10,"forks_count":54,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-04-02T03:17:22.672Z","etag":null,"topics":["android","binary-data","c","code-generation","code-generator","cross-platform","golang","high-performance","java","javascript","node-js","serializable-objects","serialization","serialization-format"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pascaldekloe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["pascaldekloe"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2015-09-05T16:42:41.000Z","updated_at":"2025-03-30T13:44:36.000Z","dependencies_parsed_at":"2023-12-18T22:26:57.170Z","dependency_job_id":"3f9ae8cf-bcea-4c18-98b8-4b3519c45108","html_url":"https://github.com/pascaldekloe/colfer","commit_stats":{"total_commits":363,"total_committers":13,"mean_commits":"27.923076923076923","dds":"0.12396694214876036","last_synced_commit":"af86cc29afae2723efccb1a0aee3c58e1a62700b"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pascaldekloe%2Fcolfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pascaldekloe%2Fcolfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pascaldekloe%2Fcolfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pascaldekloe%2Fcolfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pascaldekloe","download_url":"https://codeload.github.com/pascaldekloe/colfer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247980836,"owners_count":21027808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","binary-data","c","code-generation","code-generator","cross-platform","golang","high-performance","java","javascript","node-js","serializable-objects","serialization","serialization-format"],"created_at":"2024-07-30T20:01:49.681Z","updated_at":"2025-04-09T05:07:59.557Z","avatar_url":"https://github.com/pascaldekloe.png","language":"Java","funding_links":["https://github.com/sponsors/pascaldekloe"],"categories":["Relational Databases","Serialization","Java","序列化","Memory and concurrency","\u003cspan id=\"序列化-serialization\"\u003e序列化 Serialization\u003c/span\u003e","安全领域相关库"],"sub_categories":["HTTP Clients","Advanced Console UIs","HTTP客户端","高级控制台界面","查询语","交流","\u003cspan id=\"高级控制台用户界面-advanced-console-uis\"\u003e高级控制台用户界面 Advanced Console UIs\u003c/span\u003e","高級控制台界面"],"readme":"# Colfer\n\nColfer is a binary serialization [format](https://github.com/pascaldekloe/colfer/wiki/Spec)\noptimized for speed and size.\n\nThe project's compiler `colf(1)` generates source code from schema definitions\nto marshal and unmarshall data structures.\n\nThis is free and unencumbered software released into the\n[public domain](http://creativecommons.org/publicdomain/zero/1.0).\nThe format is inspired by Proto**col** Buf**fer**s.\n\n[![CI](https://github.com/pascaldekloe/colfer/actions/workflows/ci.yml/badge.svg)](https://github.com/pascaldekloe/colfer/actions/workflows/ci.yml)\n\n\n#### Language Support\n\n* **C**, ISO/IEC 9899:2011 compliant a.k.a. C11, C++ compatible\n* **Go**, a.k.a. golang\n* **Java**, Android compatible\n* **JavaScript**, a.k.a. ECMAScript, NodeJS compatible\n* 🚧 Gergely Bódi realised a functional **Dart** [port](https://github.com/vendelin8/colfer).\n* 🚧 Karthik Kumar Viswanathan has a **Python** [alternative](https://github.com/guilt/colfer-python) under construction.\n\n\n#### Features\n\n* Simple and straightforward in use\n* No dependencies other than the core library\n* Both faster and smaller than the competition\n* [Robust](#security) against malicious input\n* Maximum of 127 fields per data structure\n* No support for enumerations\n* Framed; suitable for concatenation/streaming\n\n#### TODO's\n\n* Rust and Python support\n* Protocol [revision](https://github.com/pascaldekloe/colfer/commits/v2)\n\n\n## Use\n\nDownload a [prebuilt compiler](https://github.com/pascaldekloe/colfer/releases)\nor run `go get -u github.com/pascaldekloe/colfer/cmd/colf` to make one yourself.\nHomebrew users can also `brew install colfer`.\n\nThe command prints its own manual when invoked without arguments.\n\n```\nNAME\n\tcolf — compile Colfer schemas\n\nSYNOPSIS\n\tcolf [-h]\n\tcolf [-vf] [-b directory] [-p package] \\\n\t\t[-s expression] [-l expression] C [file ...]\n\tcolf [-vf] [-b directory] [-p package] [-t files] \\\n\t\t[-s expression] [-l expression] Go [file ...]\n\tcolf [-vf] [-b directory] [-p package] [-t files] \\\n\t\t[-x class] [-i interfaces] [-c file] \\\n\t\t[-s expression] [-l expression] Java [file ...]\n\tcolf [-vf] [-b directory] [-p package] \\\n\t\t[-s expression] [-l expression] JavaScript [file ...]\n\nDESCRIPTION\n\tThe output is source code for either C, Go, Java or JavaScript.\n\n\tFor each operand that names a file of a type other than\n\tdirectory, colf reads the content as schema input. For each\n\tnamed directory, colf reads all files with a .colf extension\n\twithin that directory. If no operands are given, the contents of\n\tthe current directory are used.\n\n\tA package definition may be spread over several schema files.\n\tThe directory hierarchy of the input is not relevant to the\n\tgenerated code.\n\nOPTIONS\n  -b directory\n    \tUse a base directory for the generated code. (default \".\")\n  -c file\n    \tInsert a code snippet from a file.\n  -f\tNormalize the format of all schema input on the fly.\n  -h\tPrints the manual to standard error.\n  -i interfaces\n    \tMake all generated classes implement one or more interfaces.\n    \tUse commas as a list separator.\n  -l expression\n    \tSet the default upper limit for the number of elements in a\n    \tlist. The expression is applied to the target language under\n    \tthe name ColferListMax. (default \"64 * 1024\")\n  -p package\n    \tCompile to a package prefix.\n  -s expression\n    \tSet the default upper limit for serial byte sizes. The\n    \texpression is applied to the target language under the name\n    \tColferSizeMax. (default \"16 * 1024 * 1024\")\n  -t files\n    \tSupply custom tags with one or more files. Use commas as a list\n    \tseparator. See the TAGS section for details.\n  -v\tEnable verbose reporting to standard error.\n  -x class\n    \tMake all generated classes extend a super class.\n\nTAGS\n\tTags, a.k.a. annotations, are source code additions for structs\n\tand/or fields. Input for the compiler can be specified with the\n\t-t option. The data format is line-oriented.\n\n\t\t\u003cline\u003e :≡ \u003cqual\u003e \u003cspace\u003e \u003ccode\u003e ;\n\t\t\u003cqual\u003e :≡ \u003cpackage\u003e '.' \u003cdest\u003e ;\n\t\t\u003cdest\u003e :≡ \u003cstruct\u003e | \u003cstruct\u003e '.' \u003cfield\u003e ;\n\n\tLines starting with a '#' are ignored (as comments). Java output\n\tcan take multiple tag lines for the same struct or field. Each\n\tcode line is applied in order of appearance.\n\nEXIT STATUS\n\tThe command exits 0 on success, 1 on error and 2 when invoked\n\twithout arguments.\n\nEXAMPLES\n\tCompile ./io.colf with compact limits as C:\n\n\t\tcolf -b src -s 2048 -l 96 C io.colf\n\n\tCompile ./*.colf with a common parent as Java:\n\n\t\tcolf -p com.example.model -x com.example.io.IOBean Java\n\nBUGS\n\tReport bugs at \u003chttps://github.com/pascaldekloe/colfer/issues\u003e.\n\n\tText validation is not part of the marshalling and unmarshalling\n\tprocess. C and Go just pass any malformed UTF-8 characters. Java\n\tand JavaScript replace unmappable content with the '?' character\n\t(ASCII 63).\n\nSEE ALSO\n\tprotoc(1), flatc(1)\n```\n\nIt is recommended to commit the generated source code into the respective\nversion control to preserve build consistency and minimise the need for compiler\ninstallations. Alternatively, you may use the\n[Maven plugin](https://github.com/pascaldekloe/colfer/wiki/Java#maven).\n\n```xml\n\u003cplugin\u003e\n\t\u003cgroupId\u003enet.quies.colfer\u003c/groupId\u003e\n\t\u003cartifactId\u003ecolfer-maven-plugin\u003c/artifactId\u003e\n\t\u003cversion\u003e1.11.2\u003c/version\u003e\n\t\u003cconfiguration\u003e\n\t\t\u003cpackagePrefix\u003ecom/example\u003c/packagePrefix\u003e\n\t\u003c/configuration\u003e\n\u003c/plugin\u003e\n```\n\n\n\n## Schema\n\nData structures are defined in `.colf` files. The format is quite self-explanatory.\n\n```\n// Package demo offers a demonstration.\n// These comment lines will end up in the generated code.\npackage demo\n\n// Course is the grounds where the game of golf is played.\ntype course struct {\n\tID    uint64\n\tname  text\n\tholes []hole\n\timage binary\n\ttags  []text\n}\n\ntype hole struct {\n\t// Lat is the latitude of the cup.\n\tlat float64\n\t// Lon is the longitude of the cup.\n\tlon float64\n\t// Par is the difficulty index.\n\tpar uint8\n\t// Water marks the presence of water.\n\twater bool\n\t// Sand marks the presence of sand.\n\tsand bool\n}\n```\n\nSee what the generated code looks like in\n[C](https://gist.github.com/pascaldekloe/05e903f12a4f02a995f71d0c18872b65),\n[Go](https://gist.github.com/pascaldekloe/786fd46e6e4710c14fee7da1f480c2d4),\n[Java](https://gist.github.com/pascaldekloe/b54326e6b7c5e9f036911a8cbea6ccbf) or\n[JavaScript](https://gist.github.com/pascaldekloe/5653c8bb074ebd29ffcc0deece7495a4).\n\nThe following table shows how Colfer data types are applied per language.\n\n| Colfer\t| C\t\t\t| Go\t\t| Java\t\t| JavaScript\t|\n|:--------------|:----------------------|:--------------|:--------------|:--------------|\n| bool\t\t| char\t\t\t| bool\t\t| boolean\t| Boolean\t|\n| uint8\t\t| uint8_t\t\t| uint8\t\t| byte †\t| Number\t|\n| uint16\t| uint16_t\t\t| uint16\t| short †\t| Number\t|\n| uint32\t| uint32_t\t\t| uint32\t| int †\t\t| Number\t|\n| uint64\t| uint64_t\t\t| uint64\t| long †\t| Number ‡\t|\n| int32\t\t| int32_t\t\t| int32\t\t| int\t\t| Number\t|\n| int64\t\t| int64_t\t\t| int64\t\t| long\t\t| Number ‡\t|\n| float32\t| float\t\t\t| float32\t| float\t\t| Number\t|\n| float64\t| double\t\t| float64\t| double\t| Number\t|\n| timestamp\t| timespec\t\t| time.Time ††\t| time.Instant\t| Date + Number\t|\n| text\t\t| const char* + size_t\t| string\t| String\t| String\t|\n| binary\t| uint8_t* + size_t\t| []byte\t| byte[]\t| Uint8Array\t|\n| list\t\t| * + size_t\t\t| slice\t\t| array\t\t| Array\t\t|\n\n* † signed representation of unsigned data, i.e. may overflow to negative.\n* ‡ range limited to [1 - 2⁵³, 2⁵³ - 1]\n* †† timezone not preserved\n\nLists may contain floating points, text, binaries or data structures.\n\n\n\n## Security\n\nColfer is suited for untrusted data sources such as network I/O or bulk streams.\nMarshalling and unmarshalling comes with built-in size protection to ensure\npredictable memory consumption. The format prevents memory bombs by design.\n\nThe marshaller may not produce malformed output, regardless of the data input.\nIn no event may the unmarshaller read outside the boundaries of a serial. Fuzz\ntesting did not reveal any volnurabilities yet. Computing power is welcome.\n\n\n## Compatibility\n\nName changes do not affect the serialization format. Deprecated fields should be\nrenamed to clearly discourage their use. For backwards compatibility new fields\nmust be added to the end of colfer structs. Thus the number of fields can be\nseen as the schema version.\n\n\n\n## Performance\n\nColfer aims to be the fastest and the smallest format without compromising on\nreliability. See the\n[benchmark wiki](https://github.com/pascaldekloe/colfer/wiki/Benchmark) for a\ncomparison. Suboptimal performance is treated like a bug.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpascaldekloe%2Fcolfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpascaldekloe%2Fcolfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpascaldekloe%2Fcolfer/lists"}