{"id":13605881,"url":"https://github.com/adobe/bin2c","last_synced_at":"2025-04-12T05:35:07.311Z","repository":{"id":66034806,"uuid":"259588323","full_name":"adobe/bin2c","owner":"adobe","description":"Convert to/Embed binary files in C source files, quickly and efficiently.","archived":false,"fork":false,"pushed_at":"2024-05-01T17:21:59.000Z","size":53,"stargazers_count":49,"open_issues_count":1,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-06-12T09:38:52.928Z","etag":null,"topics":["c","cli","compiler"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adobe.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-04-28T09:15:46.000Z","updated_at":"2024-03-13T03:03:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"c778bae4-ba07-4eb3-9883-68bd8f35296e","html_url":"https://github.com/adobe/bin2c","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fbin2c","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fbin2c/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fbin2c/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fbin2c/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adobe","download_url":"https://codeload.github.com/adobe/bin2c/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213572306,"owners_count":15607068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","cli","compiler"],"created_at":"2024-08-01T19:01:03.875Z","updated_at":"2024-08-01T19:06:50.589Z","avatar_url":"https://github.com/adobe.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"# bin2c\n\nbin2c can be used to embed binary \u0026 text files inside C binaries.\n\nIt works by formatting the contents of the file in such a way that they\ncan be used inside a C89 compliant C string (albeit one that may be over the\nC90 maximumg string length of 509 bytes).\n\nThis method of embedding binary data is highly portable and very fast (although\nnot as fast as using the linker to generate an object file directly). It is fast enough\nto process even excessively large files (hundreds of MB) in two or three seconds.\n\n## Requirements\n\nFor the header only version:\n\n* A C11 compatible compiler\n\nTo compile the library:\n\n* A GCC, C11 compatible compiler (compatibility with MSVC would require a custom build config)\n* GNU Make\n\nTo run tests and benchmarks:\n\n* Bash 4 or newer (install with Brew under OSX)\n\n## Setup\n\nCompile and run:\n\n```bash\n$ make\n$ ./build/bin2c\n```\n\nCompile and install:\n\n```bash\n$ make install prefix=/usr/local\n```\n\nCompile and run tests:\n\n```bash\n$ make test\n```\n\nCompile and run benchmarks:\n\n```bash\n$ make test\n```\n\nThis will create a file with the benchmark results and print\naveraged benchmark results on exit. If you want to regenerate\nthese avrages from the raw data manually at a later time, you\ncan use:\n\n```bash\n$ bash ./benchmark.sh evaluate \u003c benchmark-2020-04-20-22:35:23.txt\n```\n\n## Examples\n\nCompile `myfile.txt`:\n\n```bash\n$ bin2c myfile \u003c myfile.txt \u003e myfile.c\n$ cc -c myfile.c -o myfile.o\n```\n\nManually producing the C armor:\n\n```bash\n$ echo '#include \u003cstdlib.h\u003e'\n$ echo -n 'const char myfile[] = \"'\n$ bin2c \u003c myfile.txt\n$ echo '\";'\n$ echo 'const size_t myfile_len = sizeof(myfile) - 1;';\n```\n\nWithout using a temporary file:\n\n```bash\n$ bin2c myfile \u003c myfile.txt | cc -x c -c - -o myfile.o\n```\n\nPrinting the contents of the temporary file to stdout if the file\ncontains text:\n\n```C\n#include \u003cstdio.h\u003e\n\nint main() {\n  extern const char myfile[];\n  printf(\"%s\", myfile);\n  return 0;\n}\n```\n\nIf the file contains binary data:\n\n```C\n#include \u003cstdlib.h\u003e\n#include \u003cstdio.h\u003e\n\nint main() {\n  extern const char myfile[];\n  extern const size_t myfile_len;\n  fwrite(myfile, 1, myfile_len, stdout);\n  return 0;\n}\n```\n\nUsage in C++17 with string_view:\n\n```C++\n#include \u003ccstdlib\u003e\n#include \u003cstring_view\u003e\n#include \u003ciostream\u003e\n\nextern \"C\" const char myfile[];\nextern \"C\" const size_t myfile_len;\n\nint main() {\n  std::string_view myfile_s{myfile, myfile_len};\n  std::cout \u003c\u003c myfile_s;\n  return 0;\n}\n```\n\nYou could also convert the file to a different text encoding before embedding it:\n\n```bash\n$ iconv -f utf-8 -t utf-32 \u003c myfile.txt \\\n  | bin2c myfile \\\n  | cc -x c -c - -o myfile.o\n```\n\nThis is a more elaborate example; it is a makefile rule for embedding shaders.\n\n```Makefile\nbuild/shader_%.o: shaders/%.glsl\n\t$(CPP) -P -I \"shaders/\" $\u003c \\\n\t  | bin2c shader_$* \\\n\t\t| $(CC) $(CPPFLAGS) $(CFLAGS) -x c -c - -o $@\n```\n\nThe first line here defines the build pattern; generate an object file\ncalled `shader_\u003cname\u003e` from a file called `\u003cname\u003e.glsl`.\nThen we use the C preprocessor to add support for `#include` directives\ninside our shader.\n\nThen we perform the actual conversion to compilable C; the name of our\nexported symbols is automatically calculated to be the same as our object\nfile.\n\nFinally we compile the produced C code.\n\n## Library usage\n\n```C\n// Header: bin2c.h, Lib: libbin2c.a\n\n/// If this is defined, the bin2c function will included as an inline\n/// function.\n#define BIN2C_INLINE 1\n\n/// If this is defined, bin2c can be used a header-only library.\n/// Note that this enables an alternate, slower algorithm not based\n/// on a lookup table.\n#define BIN2C_HEADER_ONLY 1\n\n/// Escape binary data\nvoid bin2c(\n  const uint8_t **in, const uint8_t *in_end,\n  char **out, const char *out_end);\n```\n\nThe bin2c function reads binary data from `[*in; in_end)`, escapes the\ndata and writes the resuling string to `[*out; out_end)`.\nThe function writes as many bytes as possible without overflowing either\nin or out buffers. If necessary, the output will be truncated.\n\n`in`/`out` are incremented to indicate the number of bytes red/written.\n\n### Example\n\n```C\n// cc -c example.c -o example.o -I\"${PWD}/src/\"\n// cc example.o build/libbin2c.a -o example\n#include \u003cstdlib.h\u003e\n#include \u003cstdint.h\u003e\n#include \u003cstring.h\u003e\n#include \u003cbin2c.h\u003e\n#include \u003cstdio.h\u003e\n\nint main(int argc, const char **argv) {\n  const char *in = argc \u003e 1 ? argv[1] : \"Hello World\";\n  char out[6];\n\n  const char *ip = in;\n  char *op = out;\n  bin2c(\n    (const uint8_t**)(\u0026ip), (const uint8_t*)(in + strlen(in)),\n    // Out buffer, leaving space for terminating zero byte\n    \u0026op, out+sizeof(out)-1);\n\n  *op = '\\0';\n\n  printf(\"Converted `%s` to `%s`. Red %i bytes and wrote %i chars.%s\\n\",\n    in, out, ip-in, op-out,\n    (ip-in \u003c strlen(in) ? \" String was truncated.\" : \"\"));\n\n  return 0;\n}\n```\n\n## Comparison to other tools\n\nThe most common way of including a file in C is using xxd:\n\n```\n$ echo \"const uint8_t myfile[] = {\"\n$ echo \"foo\" | xxd -i\n$ echo \"};\"\n```\n\nThis generates an array of hex characters:\n\n```\nconst uint8_t myfile[] = {\n  0x66, 0x6f, 0x6f, 0x0a\n};\n```\n\nThis method of embedding a file is both slower than using bin2c\nand results in bigger source file sizes. xxd produces an overhead\nof 6.2 bytes per byte while bin2c just needs 3.1 bytes for binary\nand 1.2 bytes for text.\n\nThe format produced by bin2c is more readable for text, xxd's format\nis more readable for binary. xxd also does not append a zero terminator.\n\nThe most efficient way of embedding binary data is just using ld to\ndirectly produce an object file:\n\n```bash\n$ ld -r -b binary myfile -o myfile.o -m elf_x86_64\n```\n\nThis method is not as portable as generating a source file, both because\nld needs to be available and you need to specify the architecture.\nUsing the produced symbols is also a bit more unwieldy:\n\n```C\nint main() {\n  extern const char _binary_myfile_start, _binary_myfile_end;\n  const char *myfile = \u0026_binary_myfile_start;\n  const size_t myfile_len = \u0026_binary_myfile_end - \u0026_binary_myfile_start;\n  fwrite(myfile, 1, myfile_len, stdout);\n  return 0;\n}\n```\n\nUse ld, when portability is not an issue.\n\nUse xxd, when\n\n1. You really do not wish to include the zero terminator.\n2. You are encoding a binary file and readability is important.\n\nUse bin2c, when\n\n1. You need a source file (just an object file will not do)\n2. You are encoding text and readability is an issue\n3. Encoding speed \u0026 efficiency are of importance to you.\n\n## Encoding\n\nAlphabetic letters, digits, SPACE, as well as the following symbols are\nexported as is:\n\n    !#%\u0026'()*+,-./:;\u003c=\u003e?[]^_`{|}~\n\nThe following simple escape sequences are used:\n\n    \\a\\b\\t\\v\\f\\r\\\"\\\\\n\nNewline is encoded as a newline escape code, followed by an actual, escaped\nnewline in the source. This improves readability of text. E.g.\n\n    This is the first line\\n\\\n    This is the second.\n\nAll other characters are encoded as a three letter octal escape code:\n\n    \\000\\001\\002\\003\n\nThis includes the zero terminator, `@`, `$` (which are not permitted\ncharacters in C89) and `?` to avoid confusion with trigraphs.\n\n### Text files\n\nText files stay mostly readable as long as only ascii characters are used.\n\n### Unicode Support\n\nNo special steps are taken to encode unicode. The embedded file will use whatever\nencoding the original file did use.\n\n## Performance\n\nThe fastest way to embed a binary or text file is using the linker\ndirectly (as shown above). Failing that, bin2c is quite quick.\n\nbin2c is both quicker than xxd at actually generating the source file (by a\nfactor of around 30x to 40x) and both clang and gcc are faster at embedding\nthe source generated by bin2c (20x-30x faster than xxd generated source).\n\n### Benchmarks\n\nBenchmarks Recorded on a i7-9700K CPU @ 3.60GHz.\nBigger numbers are better.\n\nComparing raw xxd and bin2c encoding performance.\n\n```\nxxd                    13.9045  Mb/s\nbin2c                  547.613  Mb/s\n```\n\nComparing compilation speeds using xxd, bin2c and bypassing the compiler\nalthogether:\n\n```\nxxd_gcc                  0.918  Mb/s\nxxd_clang                1.160  Mb/s\nbin2c_gcc               18.879  Mb/s\nbin2c_clang             32.761  Mb/s\ncompile_ld            1559.670  Mb/s\n```\n\nComparing bin2c_gcc (generating the source file on the fly) and bin2c_gcc_baseline\ncompiling a cached bin2c generated source file. (Indicates how much impact improving\nthe performance of bin2c would have on the total compile time).\n\n```\nbin2c_gcc               18.879  Mb/s\nbin2c_gcc_baseline      19.126  Mb/s\nbin2c_clang             32.761  Mb/s\nbin2c_clang_baseline    33.570  Mb/s\n```\n\n## LICENSE\n\nAuthor: Karolin Varner \u003ckaro@cupdev.net\u003e\n\nCopyright 2019 Adobe. All rights reserved.\nThis file is licensed to you under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License. You may obtain a copy\nof the License at http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under\nthe License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS\nOF ANY KIND, either express or implied. See the License for the specific language\ngoverning permissions and limitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fbin2c","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadobe%2Fbin2c","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fbin2c/lists"}