{"id":28476439,"url":"https://github.com/kyuupichan/kcpp","last_synced_at":"2026-02-25T22:04:25.662Z","repository":{"id":282776312,"uuid":"949354149","full_name":"kyuupichan/kcpp","owner":"kyuupichan","description":"A C++23 and C23 preprocessor implemented in Python","archived":false,"fork":false,"pushed_at":"2025-04-26T11:18:18.000Z","size":2781,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-27T23:59:18.539Z","etag":null,"topics":["c-preprocessor","c23","cpp-preprocessor","cpp23","preprocessor"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kyuupichan.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-16T09:08:19.000Z","updated_at":"2025-08-28T20:03:33.000Z","dependencies_parsed_at":"2025-04-14T14:26:49.913Z","dependency_job_id":"3b37278e-9dbf-46c8-9918-87adb8dd375b","html_url":"https://github.com/kyuupichan/kcpp","commit_stats":null,"previous_names":["kyuupichan/kcpp"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/kyuupichan/kcpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyuupichan%2Fkcpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyuupichan%2Fkcpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyuupichan%2Fkcpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyuupichan%2Fkcpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kyuupichan","download_url":"https://codeload.github.com/kyuupichan/kcpp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyuupichan%2Fkcpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29842886,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T21:18:31.832Z","status":"ssl_error","status_checked_at":"2026-02-25T21:18:29.265Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c-preprocessor","c23","cpp-preprocessor","cpp23","preprocessor"],"created_at":"2025-06-07T15:06:56.250Z","updated_at":"2026-02-25T22:04:25.656Z","avatar_url":"https://github.com/kyuupichan.png","language":"Python","readme":"====\nkcpp\n====\n\nA preprocessor for C23 and C++23 writen in Python, implemented as a library.\n\n  :Licence: MIT\n  :Language: Python (\u003e= 3.10)\n  :Author: Neil Booth\n\n\nGetting started\n===============\n\nPut this in ``/tmp/foo.cpp``::\n\n  #define div 1 / 0\n  #define g(x) 2 + x\n  #if g(div)\n  #endif\n\nThen::\n\n  $ pip install kcpp\n  $ kcpp /tmp/foo.cpp\n  #line 1 \"/tmp/foo.cpp\"\n  #line 1 \"\u003cpredefines\u003e\"\n  #line 1 \"/tmp/foo.cpp\"\n  \"/tmp/foo.cpp\", line 3: error: division by zero\n      3 | #if g(div)\n        |       ^~~\n      \"/tmp/foo.cpp\", line 1: note: in expansion of macro 'div'\n          1 | #define div 1 / 0\n            |               ^ ~\n      \"/tmp/foo.cpp\", line 2: note: in expansion of macro 'g'\n          2 | #define g(x) 2 + x\n            |                  ^\n  1 error generated compiling \"/tmp/foo.cpp\".\n\n\nGoals\n=====\n\nThis project is a standards-conforming and efficient (to the extent possible in Python)\npreprocessor that provides high quality diagnostics.  It is host and target independent\n(in the compiler sense).  The code is intended to be clean and easy to understand.\n\nPerhaps more importantly, it should be a reference implementation that can be easily\ntranscoded to an quivalent but more efficient C or C++ implementation by a decent\nprogrammer of those languages.  There is no reason such a re-implementation should not be\na par or even exceed Clang or GCC with respect to performance and quality, and at the same\ntime I would expect it to be significantly smaller and easier to understand and maintain.\n\nSome design choices (such as treating source files as binary rather than as Python Unicode\nstrings, and not using Python's built-in Unicode support) were made because those features\ndon't exist in C and C++.  I want it to be easy to translate this Python implementation to\na C or C++ equivalent.\n\nI intend to do such a transcoding to C++ once the Python code is mostly complete and\ncleaned up later in 2025 as part of my goal of learning C++ properly.\n\n\nWhy write a preprocessor in Python?\n===================================\n\nGood question.  Essentially because Python makes it very easy to refactor code to find the\ncleanest and most efficient implementation of an idea.  It is ideal for a reference\nimplementation that can be transcoded to C or C++.  I believe the result would be much\nbetter than could be achieved from scratch in a similar timeframe in those languages alone.\n\nI was a co-maintainer of GCC's preprocessor from 1999 to 2003.  During this time we\nconverted it from a standalone executable that would output to a pipe, to an integrated\n(kind-of) libary ``libcpp`` in the compiler proper.  Compilers are addictive, and between\n2005 and 2007 I wrote a C99 front-end in C (which is not public).  LLVM was lacking an\nimplementation of compile-time host- and target-independent IEEE-conforming floating point\narithmetic, so I contributed the one from my front-end (after translating it from C to\nC++).  Chris Lattner incorporated it into Clang/LLVM as APFloat.cpp in 2007.\n\nMy experience writing a front-end made clear the difficulty of refactoring and\nrestructuring C or C++ code to make improvements.  Another reason it is avoided is fear of\nbreaking things subtly owing to poor testsuite coverage, or having to update hundreds or\nthousands of tests to account for changes in output or diagnostics that a refactoring\ntends to cause.  Can compiler testing be improved?\n\nA glance at, e.g., the expression parsing and evalation code of GCC and Clang, or their\ndiagnostic subsystems, and trying to comprehend them reveals creeping complexity and loss\nof clarity.  I remember Clang's original preprocessor from 2007 as being quite clean and\nefficient; I'm not sure that could ever have been said of libcpp that I worked on.\n\nIn 2012 I learnt Python and have come to love its simplicity and elegance.  In 2016 with\nElectrumX I proved Python can efficiently process challenging workloads.  More recently I\nhave become interested in learning C++ properly - although able to write basic C++ from\naround the mid 1990s, I used to prefer the simplicity of C.\n\nRecently I noticed C++ was \"getting its act together\" and took a look at C++ standard\ndrafts.  I became curious and decided \"Hmm, let's try something a little insane and write\na C++23 preprocessor in Python.\"  So kcpp was born in mid-January 2025.\n\nCan a performant and standards-conforming preprocessor be written in Python?\n\n\nWhat about the other open source preprocessors?\n===============================================\n\nThere are several publicly available preprocessors typically written in C or C++.  Usually\nthey claim to be standards conforming but are usually far from it.  It is significant work\nto be 90% conforming, but the last 10% is really quite hard.  There are endless corner\ncases, and more recent preprocessor features like extended identifiers, raw strings,\nvariable-argument macros (particularly the addition of ``__VA_OPT__``), and handling UCNs\nare each significant work.  None of the preprocessors I'm aware of (other than those of\nthe much larger projects GCC and Clang) make an effort at high-quality diagnostics or\nserious standards compliance.\n\nTo my surprise two or three Python preprocessors exist as well, but have similar defects\nand/or have other goals such as visualization (``cpip`` is a cool example of this).  None\nappear to be actively maintained.\n\nIt is worthwhile comparing the code of other preprocessors with that of ``kcpp`` and\ntesting them with tricky preprocessing cases.\n\n\nFeatures\n========\n\nAs a **genuinely** conforming C++23 preprocessor ``kcpp`` is essentially complete.\nSpecifically, the following are implemented:\n\n- lexing (including UCNs, extended identifiers, and raw string literals)\n- macro expansion, including variable arguments, ``__VA_OPT__``, and whitespace-correctness\n- all standard directives\n- ``_Pragma`` operator\n- predefined and built-in macros, presently limited to those defined in the standard\n- interpretation of character, string and numeric literals\n- expression parsing with proper error recovery\n- expression evaluation\n- ``__has_include``, ``__has_cpp_attribute`` preprocessor conditional operators\n- conversion of Unicode character names (those in ``\\N{}`` escapes) to codepoints.  I\n  implemented it based on the ideas described by **cor3ntin** at\n  https://cor3ntin.github.io/posts/cp_to_name/.  I added some ideas and improvements of my\n  own to achieve another 20% compaction - see\n  https://github.com/kyuupichan/kcpp/blob/master/src/kcpp/unicode/cp_name_db.py.\n- module-related directives with import-keyword, export-keyword, module-keyword\n\nIn addition the following are complete:\n\n- preprocessed output\n- a full diagnostic framework.  This includes changing diagnostic severities from the\n  command line, Colourized output to a Unicode terminal, and translations (none\n  provided!).  The framework could be hooked up to an IDE.\n- diagnostics can display the macro expansion stack with precise caret locations and range\n  highlights, with proper handling of multibyte characters, tabstops and CJK terminal\n  character widths\n\nFor C23 preprocessing, C++-specific features are disabled (alternative operators, raw\nstring literals, ``\u003c=\u003e``, ``.*`` and ``-\u003e*`` tokens, ``__has_cpp_attribute`` and\nuser-defined suffixes).  The following C23 features are missing simply because I\nimplemented C++23 first.  They are all easy, apart from ``#embed`` which is quite a new\nconcept (and in C++26) which will need some thought on its implementation:\n\n- bit-precise integer suffixes\n- decimal floating-point suffixes\n- slightly different charcter literal semantics\n- different predefined macros\n- ``__has_c_attribute``\n- ``#embed``, ``__has_embed``, etc.\n\nFuture\n======\n\n- the multiple-include optimization is not yet implemented\n- some GCC and Clang extensions should be supported\n- features like ``Makefile`` output are worth considering going forwards.\n- pprecompiled headers are possibly an idea.  An implementation would probably share a lot\n  with modules.  Python is a good place to experiment before attempting an implementation\n  in C++, but there is little point doing this until a compiler frontend exists\n- add a C and C++ front-end in Python, perhaps as a single parser and codebase?\n\nIt should be easy to extend the code to provide hooks for analysis or other tools needing\na preprocessor to do grunt work.\n\n\nDocumentation\n=============\n\nI will write some at some point.  The code is well-commented and reasonably clean though -\nit shouldn't be hard to figure out.\n\n\nTests\n=====\n\nI have a testuite but it is mostly private.  Test case submissions for the public repo\n(using pytest) are welcome, as are bug reports.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyuupichan%2Fkcpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyuupichan%2Fkcpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyuupichan%2Fkcpp/lists"}