{"id":21404702,"url":"https://github.com/checkedc/checkedc-libjpeg-tutorial","last_synced_at":"2026-01-03T10:15:39.591Z","repository":{"id":139990799,"uuid":"407233892","full_name":"checkedc/checkedc-libjpeg-tutorial","owner":"checkedc","description":null,"archived":false,"fork":false,"pushed_at":"2021-10-04T18:26:20.000Z","size":73,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"converted","last_synced_at":"2025-01-23T03:41:23.278Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/checkedc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-16T16:17:47.000Z","updated_at":"2022-06-08T22:54:17.000Z","dependencies_parsed_at":"2023-07-22T15:15:10.193Z","dependency_job_id":null,"html_url":"https://github.com/checkedc/checkedc-libjpeg-tutorial","commit_stats":null,"previous_names":["checkedc/checkedc-libjpeg-tutorial"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/checkedc%2Fcheckedc-libjpeg-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/checkedc%2Fcheckedc-libjpeg-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/checkedc%2Fcheckedc-libjpeg-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/checkedc%2Fcheckedc-libjpeg-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/checkedc","download_url":"https://codeload.github.com/checkedc/checkedc-libjpeg-tutorial/tar.gz/refs/heads/converted","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243902291,"owners_count":20366259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T16:17:39.522Z","updated_at":"2026-01-03T10:15:39.531Z","avatar_url":"https://github.com/checkedc.png","language":"C","readme":"\u003c!--\n```{.bash file=tutorial.sh}\n#!/bin/bash -ex\ngit diff-index --quiet HEAD --\ngit branch -D converted\ngit checkout -b converted\nif [[ -z \"$LLVM_OBJ\" ]]; then\n  export LLVM_OBJ=~/checkedc-clang/build\nfi\nexport PATH=\"$LLVM_OBJ/bin/:$PATH\"\n```\n--\u003e\n\nThis tutorial will demonstrate the 3C conversion process by converting\na small C program into Checked C. The program uses the C library\n[libjpeg](http://www.ijg.org/) to read a JPEG image file provided as a command\nline argument, and write it to standard output in\n[PPM format](https://en.wikipedia.org/wiki/Netpbm#PPM_example). The program\nbeing converted is adapted from sample code provided as part of the libjpeg\ndistribution.\n\nThis tutorial is intended to specifically demonstrate how 3C can be used to\nport programs that interact with existing C libraries. For a first\ntutorial on 3C, see our port of\n[tiny-bignum-c](https://github.com/correctcomputation/checkedc-tiny-bignum-c).\n\n# Requirements\n\nFor conversion of C code into Checked C and compilation of Checked C to C, both\n`3c` and the Checked C version of `clang` must be built from the repository\n[correctcomputation/checkedc-clang](https://github.com/correctcomputation/checkedc-clang).\nThe Checked C standard library headers from\n[correctcomputation/checkedc](https://github.com/correctcomputation/checkedc/)\nare also required. These should already be available if 3C was built according\nto the\n[3C build instructions](https://github.com/correctcomputation/checkedc-clang/blob/main/clang/docs/checkedc/3C/INSTALL.md).\nThis tutorial uses these repositories at commits\n[2a0df40](https://github.com/correctcomputation/checkedc-clang/commit/2a0df403e4c803edecbe8ed6019320d7b170708a) and\n[7c6f388](https://github.com/correctcomputation/checkedc/commit/7c6f388a816bcca14cf2144f9b78b6e27957a09d)\nrespectively.\n\nThe premise of this tutorial is to convert C code that depends on libjpeg, so\nthe library should be installed. On Ubunutu, it can be installed with\n`sudo apt install libjpeg-dev`.\n\n\u003c!--\nNOTE: The tutorial currently doesn't work properly without libjpeg installed,\n      so I've commented this part out. It should be able to work in priniple.\n\nA copy of the relevant header file is included\nwith the tutorial, so the conversion steps can be followed regardless, but it\nmust be installed to compile and execute the converted program.\n--\u003e\n\nThe makefile and conversion script included with this tutorial assume that the\n`3c` Checked C version of `clang` are on your path. If you cloned 3c into your\nhome directory and built it according to the instructions linked above, then\nthe path can be configured with\n```bash\nexport PATH=\"~/checkedc-clang/build/bin/:$PATH\"\n```\n\n# Initial Conversion\n\nThe tutorial begins with unconverted code on\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/660412ea837da94b43133dd38cdffc960331ec00\" data-commit-start=\"true\"\u003ethis commit\u003c/a\u003e.\nTo convert the sample program, we will execute 3C through the provided\n`convert_all.sh` script. This script invokes 3C on the source file with\nsome options to control the specifics of the conversion process. (The\npurpose of the options is explained in the [tiny-bignum-c tutorial](https://github.com/correctcomputation/checkedc-tiny-bignum-c).\n\nHere we assume that you do not know beforehand that you will need to create a new\nchecked header file for libjpeg, and show how to determine that it is\nnecessary. If this is known ahead of time, the conversion script could be\nmodified to include the options required for converting with the libjpeg header\nfile from the beginning.\n\nThe command in `convert_all.sh` can be simple because only one source file is\nbeing converted, and there are no complicated compiler flags that need to be\nprovided.\n```bash\n#!/bin/bash\n3c -alltypes -warn-root-cause -output-dir=out ./to_ppm.c --\n```\n\nRunning the conversion script (`./convert_all.sh`)  will execute the initial\nconversion and place the converted files in `./out/` according to the specified\n`-output-dir`. The converted code is left in the output directory for the\nmoment because subsequent changes will be applied to the original, unconverted,\ncode. The converted code will be copied from the output directory when we are\nhappy with the progress of the conversion, and want to begin making final\nadjustments to the Checked C code.\n\n```{.bash file=tutorial.sh}\n./convert_all.sh\ngit add out\ngit commit -m \"[3c] Initial conversion\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/b418e4bcfedcc870e6002e65f0eb8d60b7cfe302\" data-commit-subject=\"Initial\"\u003e(view changes)\u003c/a\u003e.\n\n\n# Converting `jpeglib.h`\n\nRunning the conversion script also emitted *root cause diagnostics* as specified\nby the `-warn-root-cause` option. These diagnostics identify code patterns\nthat have specifically prevented a pointer from being made checked (it\nremains \"wild\"); this code is a root cause in the sense that it directly affects\nthe identified pointer (which in turn may affect other pointers with a dataflow\nrelationship). To decide how to proceed with the conversion,\nwe can examine the information in the root-cause diagnostics.\nBy far, the most common root cause of wildness is that there are\nfunction declarations in unwritable files. These functions are declared in the\nsystem version of the libjpeg header file (`jpeglib.h`) which is located\noutside of the current directory. By default, 3C considers files outside the\ncurrent directory \"unwritable\", and so will never make changes to such files.\nBefore we can have 3C infer Checked C declarations for these functions, we\nfirst need to create a local, writable copy of the header so that 3C can\nrewrite it.\n\nFor consistency, a copy of the header file is already included with this\ntutorial, but, in general, the header file should be a copy of the system\nversion of the header file. In this case, we would need to create a local\ninclude directory and copy the system header into it:\n```bash\nmkdir include\ncp /usr/include/jpeglib.h ./include\n```\nThe local include directory and header file already exist, so we only need to\nupdate the conversion script so that the local header is found before the\nsystem header. Edit `convert_all.sh` to append `-I ./include/` to the end of\nthe argument list. This must be placed after the dashes (`--`) which denote the\nend of arguments passed to 3C and the start of arguments passed to clang. For\nthis tutorial, we have included patch files for this and all other necessary\nmanual changes.\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/update_include.patch\ngit add convert_all.sh\ngit commit -m \"[Manual] Update include path\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/c8003d910d4693c97d4f7441c4abecd53a73f4b5\" data-commit-subject=\"Update include path\"\u003e(view changes)\u003c/a\u003e.\n\nRe-running `./convert_all.sh` will attempt conversion again, but this time with\nthe writable copy of `jpeglib.h`, so 3C is capable of making changes to it. This\nresolves the \"unwritable file\" root causes, but they are replaced by root\ncauses due to \"undefined functions\".\n\n```{.bash file=tutorial.sh}\n./convert_all.sh\ngit add out\ngit commit -m \"[3C] Convert with local header\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/c73e57afbcbb14b33cf0a0acd18ced43c7ccef04\" data-commit-subject=\"Convert with local header\"\u003e(view changes)\u003c/a\u003e.\n\nBy default, 3C will not rewrite the types of functions that are not defined. To\nmake it infer the types of such functions, 3C can be given the flag\n`-infer-types-for-undefs`. This disables constraints that are typically\ngenerated which force all undefined functions to remain unchecked, allowing\nChecked C types to be inferred and added to the header as so-called\n\"interop types\" (itypes).\n\nThere are some important limitations to this flag:\n\n- 3C will still only infer types for functions that are declared in files that\n  it is able to rewrite. In particular, this means that functions defined in\n  system library headers will not be automatically converted to use itypes.\n  This is why a local copy of `jpeglib.h` was created in the same directory\n  as the code being converted in the previous steps. This allows 3C to rewrite\n  the declaration, so checked types will be inferred.\n- 3C will still treat any undefined function as internally unsafe. This means\n  that an undefined function will only be converted to an `itype` type, and\n  never a fully checked type. If the body of the function is later made\n  available, 3C can be rerun on the code and will rewrite itypes into fully checked types.\n- 3c is not able to examine the bodies of the functions, so inferred pointer\n  types and array bounds are not guaranteed to be correct. The compiler will\n  check that the functions are called in accordance with the inferred types,\n  but doing so risks missing spatial safety violations when the types do not\n  accurately specify the behavior of the implementation. For this reason, the\n  inferred types must be validated against the implementation or the behavior\n  specified in available documentation.\n\nTo proceed with conversion of the libjpeg header file (`jpeglib.h`) we must\nnow update the options passed to 3C to enable conversion of undefined\nfunctions. Update `convert_all.sh` again to add `-infer-types-for-undefs` to\nthe list of options passed to 3C. When making this edit, be careful to add\nthe option before the two dashes `--` which mark the end of the options passed\nto 3C.\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/add_option.patch\ngit add convert_all.sh\ngit commit -m \"[Manual] Add new option\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/2fb01f452b04cb14f90965cbdee416476a8fe1d6\" data-commit-subject=\"Add new option\"\u003e(view changes)\u003c/a\u003e.\n\nRunning the 3C conversion for a third time and examining the root cause\ndiagnostics will show that the undefined functions are no longer reported as\ncauses of wildness.\n\n```{.bash file=tutorial.sh}\n./convert_all.sh\ngit add out\ngit commit -m \"[3C] Convert with new option\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/ec51716c9629e84d3a08b53f6c97a3a32e841b48\" data-commit-subject=\"Convert with new option\"\u003e(view changes)\u003c/a\u003e.\n\n# A Final Root Cause\n\nAfter this point, the conversion can proceed using the same process\ndescribed in the tiny-bignum-c tutorial. Any remaining root causes of unchecked\npointers will be resolved before committing to using the converted version of\nthe code, and finally placing the entire program in a checked region to finish\nconversion.\n\nOne of the root causes that remains indicates that a pointer is declared in a\nmacro. 3C cannot rewrite inside macros, so we must either fully expand the\nmacro, so that 3C can annotate each expansion individually, or manually add\nChecked C annotations inside the macros.\n\nIn this case, the macro `jpeg_common_fields` is used to declare the fields of\na structure. It is easy to add an itype to the field declarations so that each\nstructure defined using the macro is now defined with itypes on its fields.\nOnly two of the fields (`err` and `mem`) are actually relevant to the current\nconversion. Itypes should be added onto these fields so that they are now\ndeclared as `struct jpeg_error_mgr *err : itype(_Ptr\u003cstruct jpeg_error_mgr\u003e)`\nand `struct jpeg_memory_mgr *mem : itype(_Ptr\u003cstruct jpeg_memory_mgr\u003e)`\nrespectively. The conversion script should then be run again to propagate the\nchange.\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/insert_itypes.patch\ngit add ./include/jpeglib.h\ngit commit -m \"[Manual] Add itypes in macro\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/eb5a93bd826d626bee61ccce85a47c5bc53841f3\" data-commit-subject=\"Add itypes in macro\"\u003e(view changes)\u003c/a\u003e.\n\n```{.bash file=tutorial.sh}\n./convert_all.sh\ngit add out\ngit commit -m \"[3C] Convert with itypes in macro\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/c381097b1fe20bffb5ba9c87c0cad4215c25e5e1\" data-commit-subject=\"Convert with itypes in macro\"\u003e(view changes)\u003c/a\u003e.\n\nAt this point, the major issues preventing conversion of the example program\n(unwritable code, undefined functions, and `struct` fields defined in macros)\nhave been resolved. A few unchecked pointers remain, but these are either in\nthe header file only, or can be fixed individually in the partially converted\ncode. To do this, the converted versions of the source files need to be copied\ninto the working directory.\n\n```{.bash file=tutorial.sh}\ncp -r ./out/* .\ngit add include/jpeglib.h to_ppm.c\ngit commit -m \"[Manual] Copy converted files\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/c969724fe46a42db7bbe0efb9d48b9ca8cea7868\" data-commit-subject=\"Copy converted files\"\u003e(view changes)\u003c/a\u003e.\n\n# Compiling the Converted Code\n\nAttempting to compile the code by executing `make` in its current state will\nshow that a few manual fixes need to be made to the converted code.\n\u003c!--\n```{.bash file=tutorial.sh}\nmake || true\n```\n--\u003e\n\nA few of the errors are syntax errors introduced by 3C while rewriting the header\nfile. These are not interesting and can be fixed easily. The other, more\ninteresting, error is raised in `to_ppm.c`.\n\n3C infers that a pointer is an array, but is unable to infer any bounds.\nChecked C raises an error when the pointer is later accessed at constant index\n`0`. Due to the constant index, the length of the array must be at least `1`.\nIt is not used anywhere else, so the bounds of the array can be exactly\n`count(1)`. This pointer is actually a two dimensional array\n(`_Array_ptr\u003c_Array_ptr\u003cunsigned char\u003e\u003e`), but Checked C only supports bounds\non the outer array pointer. The lack of bounds on the nested pointers will need\nto be addressed before the program can be considered fully checked.\n\nWith bounds added onto the array Checked C now checks that the bounds are\npreserved on assignment to the array. There is an assignment to the array from\nthe return value of a function declared in `jpeglib.h`. 3C did not infer bounds\nfor this function, so they must now be added manually. The function is passed\na number of rows and the size of each row. As noted above, the array pointer is\na two-dimensional array, and we cannot annotate the inner array, so its length\nis equal to the number of rows (`count(numrows)`). This is consistent with the\nactual arguments at the function call: `numrows` is `1`, and the return value\nis expected to have bounds `count(1)`.\n\nThe final error complains about an unsafe cast from a `struct jpeg_error_mgr`\npointer to `struct my_error_mgr`. While the cast isn't safe, the size of\n`struct my_error_mgr` is at least as large as `struct jpeg_error_mgr`, so we\ncan instruct Checked C to accept the cast with explicit bounds checking by\nreplacing the cast `(my_error_ptr)` with\na `_Dynamic_bounds_cast\u003cmy_error_ptr\u003e`, so\n```c\nmy_error_ptr myerr = (my_error_ptr) cinfo-\u003eerr;\n```\nbecomes\n```c\nmy_error_ptr myerr = _Dynamic_bounds_cast\u003cmy_error_ptr\u003e(cinfo-\u003eerr);\n```\n\nThe exact fixes are captured in the patch file:\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/fix_errors.patch\ngit add include/jpeglib.h to_ppm.c\ngit commit -m  \"[Manual] Fix errors in converted code.\"\nmake\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/6c1fccf39ac166a08ba50faa9aa6192afb684f0c\" data-commit-subject=\"Fix errors in converted code\"\u003e(view changes)\u003c/a\u003e.\n\n# Finishing the Conversion\n\nWith the initial conversion working, the final goal is to \"fully convert\" the\nsource file, where full conversion means enabling the `CHECKED_SCOPE` pragma\nwhile having a minimal number of `_Unchecked` regions in the program.\n\nAdd `#pragma CHECKED_SCOPE on` at the top of `to_ppm.c`, before the function\ndefinitions but after all `#include`s. Attempting to compile again with `make`\nafter adding the pragma reveals new errors raised by the Checked C compiler.\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/add_pragma.patch\ngit add to_ppm.c\ngit commit -m \"[Manual] Add CHECKED_SCOPE pragma.\"\nmake || true\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/3841a63ad554109c3e1f3621cef7e8bb9902c302\" data-commit-subject=\"Add CHECKED_SCOPE pragma\"\u003e(view changes)\u003c/a\u003e.\n\nThe first error is a result of a bounds cast inserted by 3C during conversion.\nBefore turning the `CHECKED_SCOPE` pragma on, we fixed a different error by\nadding a dynamic cast so that Checked C would check a\nbound at run time. Here, an `_Assume_bounds_cast` was added by 3C to avoid checking\na bound that cannot be verified even at run time. This is clearly unsafe, and\nso it is not allowed in a checked region. Ideally, we would be able to\nrefactor the code to avoid needing the `_Assume_bounds_cast`, but that is not possible\nwithout a substantial change to the functions exported by libjpeg. libjpeg\nworks with two dimensional arrays, but Checked C currently does not have\na mechanism for annotating bounds on the inner array pointers. We need to place\nthe `_Assume_bounds_cast` in an unchecked region before the Checked C compiler will\naccept the program.\n\nThe simplest approach would be to place the whole statement with the cast in an\nunchecked block, but this would result in the function call and array access\nbeing unchecked, which should be avoided. Instead, the statement is split so\nthat only the unsafe cast is unchecked, minimizing the amount of code in\nunchecked blocks. The statement\n```c\nput_scanline_someplace(_Assume_bounds_cast\u003cJSAMPROW \u003e(buffer[0],  count(row_stride)), row_stride);\n```\nis split into four statements.\n```c\nJSAMPROW row_unkb : bounds(unknown) = buffer[0];\nJSAMPROW row : count(row_stride) = 0;\n_Unchecked {\n  row = _Assume_bounds_cast\u003cJSAMPROW\u003e(row_unkb,  count(row_stride));\n}\nput_scanline_someplace(row, row_stride);\n```\n\nAnother compiler error is raised because libjpeg using the standard library\nheader `setjmp.h` for error handling. This header file does not yet have\na checked version, so we need to surround calls to its functions in\n`_Unchecked` blocks.\n\nThe exact fixes required are again provided in a patch file.\n\n```{.bash file=tutorial.sh}\ngit apply \u003c patches/pragma_fixes.patch\ngit add -u\ngit commit -m  \"[Manual] Fix errors in checked scope.\"\n```\n\u003ca href=\"https://github.com/correctcomputation/libjpeg_tutorial/commit/931f8b3754c5ca43047fe1aacbbc390723f4608d\" data-commit-subject=\"Fix errors in checked scope\"\u003e(view changes)\u003c/a\u003e.\n\nWith these changes, the program compiles successfully with the `CHECKED_SCOPE`\npragma enabled. As a final step, we can check that the program can run without\nerror on sample input.\n\n```{.bash file=tutorial.sh}\nmake\n./to_ppm test.jpg | display\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheckedc%2Fcheckedc-libjpeg-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcheckedc%2Fcheckedc-libjpeg-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheckedc%2Fcheckedc-libjpeg-tutorial/lists"}