{"id":23642641,"url":"https://github.com/tleonhardt/python_interface_cpp","last_synced_at":"2025-08-31T19:30:46.943Z","repository":{"id":53977300,"uuid":"78489687","full_name":"tleonhardt/Python_Interface_Cpp","owner":"tleonhardt","description":"Example code for interfacing with C and C++ from Python using Cython, SWIG, CFFI, PyPy, and pybind11","archived":false,"fork":false,"pushed_at":"2019-11-09T18:12:51.000Z","size":1042,"stargazers_count":61,"open_issues_count":0,"forks_count":13,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-06T14:38:54.164Z","etag":null,"topics":["c","cffi","cpp","cython","pybind11","pypy","python","swig"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tleonhardt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-10T02:28:01.000Z","updated_at":"2022-06-20T19:56:45.000Z","dependencies_parsed_at":"2022-08-13T05:40:12.827Z","dependency_job_id":null,"html_url":"https://github.com/tleonhardt/Python_Interface_Cpp","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tleonhardt%2FPython_Interface_Cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tleonhardt%2FPython_Interface_Cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tleonhardt%2FPython_Interface_Cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tleonhardt%2FPython_Interface_Cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tleonhardt","download_url":"https://codeload.github.com/tleonhardt/Python_Interface_Cpp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231615472,"owners_count":18400983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","cffi","cpp","cython","pybind11","pypy","python","swig"],"created_at":"2024-12-28T10:51:53.370Z","updated_at":"2024-12-28T10:51:54.068Z","avatar_url":"https://github.com/tleonhardt.png","language":"Python","readme":"# Python_Interface_Cpp\nExample code for interfacing with C and C++ from Python using Cython, SWIG, PyPy, CFFI, and pybind11:\n\n* [Cython](http://cython.org)\n* [SWIG](http://www.swig.org)\n* [PyPy](https://pypy.org)\n* [CFFI](http://cffi.readthedocs.io)\n* [pybind11](https://github.com/pybind/pybind11)\n\n## Fibonacci Example\nThis repository has example code for interfacing with some C code for calculating the Nth\nfibonacci number from Python using the four different tools.\n\nThis is the one universal example present for all tools and is thus intended to provide\nan apples-to-apples comparison both in terms of API/usage and performance.\n\n### Fibonacci Performance Benchmarks\nOne metric for evaluating these tools is performance.  Here is a table which shows speedup\nfactor relative to the pure Python implementation of the Fibonacci code.  So a speedup of **2**\nmeans that code ran twice as fast as the pure Python version.\n\n| Tool               | Speedup |\n| ------------------ | -------:|\n| pypy + CFFI        | 40      |\n| Cython (optimized) | 27      |\n| Cython (wrapper)   | 25      |\n| SWIG               | 14      |\n| pybind11           | 10      |\n| pypy               |  8      |\n| CFFI               |  6      |\n| Python             |  1      |\n\nNOTE: These numbers were originally measured on a 2013 15\" Mac Book Pro using Python 3.6 via Anaconda distro\nwith the latest versions of all tools installed using the conda package manager.  I have more recently measured\nusing a Python 3.7 virtual environment via [Pipenv](https://github.com/pypa/pipenv) and confirmed that they\nstayed approximately the same and relative rankings in terms of performance are identical.\n\nThe Fibonacci example presented here is pretty trivial.  Your mileage may vary depending on your\napplication.  But overall these performance measurements are fairly representative of what you\nmay typically expect from each of these tools.\n\nIn general whenever you see nested loops in your Python code, that is where you should expect the\nmost speedup can be gained by optimizing with C/C++.  Speedups on the order of 1000 are not\nuncommon if you are dealing with triply-nested loops for things like image processing code.\n\n## Other Examples\nSome of the tools have other examples provided as well, typically for more advanced features.\nBut no attempt is made to provide an apples-to-apples cross-tool comparison for these other\nexamples.\n\n# Meet the tools\n\n## Cython\nCython is the C-Extensions for Python module and it is an optimising static transpiler which\nlets you use optional C static types embedded in your Python code and automatically translate\nthat to C or C++ code and then automatically compile it.\n\nBut all of that happens pretty much behind the scenes, and you are left with code that looks\nmostly like Python, but has the performance of C.\n\nIt has a bit of a learning curve, but it is amazing.  In particular it is awesome for\noptimizing existing Python code.  But it is also good for providing really high performance\nwrappers of existing C/C++ code with a clean Pythonic API.  It just requires you to manually\ndo the wrapping, so it involves significantly more work on your part to do so than using SWIG.\n\nThe [Cython documentation](http://docs.cython.org/en/latest/) is excellent and there is a great\n[book](https://www.amazon.com/Cython-Programmers-Kurt-W-Smith/dp/1491901551) from OReilly and also some great\nvideo presentations on [YouTube](https://www.youtube.com/watch?v=gMvkiQ-gOW8\u0026t=14s) and on [Safari](http://shop.oreilly.com/product/0636920046813.do).\n\n## SWIG\nSWIG is the Simplified Wrapper and Interface Generator.  It is capable of wrapping C and C++\ncode/libraries in about 20 different target languages including Python, Java, C#, etc.\n\nSWIG largely automates the process of wrapping existing C/C++ code so it is faster and easier\nto use for that purpose than Cython.\n\nThe [SWIG documentation](http://www.swig.org/Doc3.0/index.html) can sometimes leave something to be desired, but it is an extremely powerful\ntool which is an excellent choice in many situations.  A professor from the University of Oslo has\na decent [video tutorial](https://www.youtube.com/watch?v=J-iVTLp6M9I) with [example code](https://github.com/UiO-INF3331/code-snippets/tree/master/mixed/swig) on GitHub.\n\nSWIG is the granddaddy of all of the tools here.  It has been around for a long time.  Given how\nlong it has been around and the wealth of target languages, it is probably the most widely used\nof these tools.\n\n## PyPy\nPyPy is a fast, compliant alternative implementation of the Python language (2.7.13 and 3.5.3) in Python.\nThe normal implementation of Python is in C and is referred to as CPython.  PyPy uses a Just-in-Time (JIT)\ncompiler, so Python programs often run significantly faster on PyPy.  It also employs various memory\noptimizations compared to CPython.\n\nThe major advantage of PyPy is that it can deliver some quick and easy wins.  You don't need to change\nyour existing Python code in any way!  You just install PyPy and run your Python program using **pypy**\ninstead of **python**.\n\nThe major disadvantage of PyPy is that not all 3rd-party Python libraries work with PyPy.  Detailed\ninformation on which modules are compatible can be found [here](https://pypy.org/compat.html).\n\n## CFFI\nCFFI is the C Foreign Function Interface for Python.  It is basically an improved version of\nPython's built-in ctypes.\n\nIt can call functions in a C dynamic library just by writing code inline in Python.  There\nisn't any sort of need for an external wrapper.  Hence, it is the quickest and easiest\nof these libraries to use to interface with existing C dynamic libraries.  However, the\nperformance is decidedly worse than the other options presented here unless it is used in combination\nwith PyPy, in which case the performance is truly excellent.\n\nIts real strength lies in 100% compatibility with PyPy and lower overhead present when using CFFI\nwith PyPy's JIT.  So it is the go-to choice if you are\nusing the PyPy JIT.  That being said, I don't think I would recommend using it if you aren't using\nPyPy because either Cython or SWIG tend to be a better fit for most applications when used with\nthe normal CPython implementation of Python.\n\nCFFI has decent [documentation](https://cffi.readthedocs.io/en/latest/) and here is a [tutorial video](https://www.youtube.com/watch?v=ThDFmuXH15k).\n\n## pybind11\npybind11 is essentially what arose from the ashes of Boost.Python.  It is the newest of the tools\npresented here, but it is already better than\nBoost.Python ever was.\n\nIt only works with bleeding-edge C++11 compilers.  My experience is that I couldn't get it to work on Mac OS X at all and I tried with both Python 3.6 and Python 2.7, both from Anaconda distro and using default LLVM compiler from Xcode on Mac OS X\n10.12.4.  I also could not get it working on either Ubuntu 16.04 or 14.04 with either Python 2.7 or 3.6. I was able to get it working on Debian 9 with both Python 2.7 and 3.5 as installed from apt-get.  Given that experience I wouldn't even consider it remotely stable yet.\n\nThe performance is worse than either SWIG or Cython and the ease of use is not as easy to use as SWIG.\nSo does a niche exist for pybind11?  pybind11 does make it easy to embedded Python within a C++ project and it is well\nsupported on Windows.  Here is a [good video](https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Embedding-Python-in-a-C-Project) that demonstrates scripting a\nC++ application with Python using pybind11 and Visual Studio 2017.\n\n[pybind11 documentation](http://pybind11.readthedocs.io/en/stable/) is decent and here is a [conference video](https://www.youtube.com/watch?v=jQedHfF1Jfw).\n\npybind11 does appear to be under rapid development, so maybe it will get better in the future; but at this time I can\nonly recommend it for embedding Python within a C++ project (though it is probably worth noting that Cython is also\ncapable of doing that).\n\n# Conclusion / Recommendations\nOk, here are some of my thoughts.  They are my opinions and are hence subjective in nature, though\nthey are grounded in my having actually evaluated these tools at length.\n\nIf what you care about most is performance, then Cython is the clear winner by a wide margin.  But on\nthe flip side, this tool has the largest learning curve, at least if you are looking at wrapping\nexisting C++ code.\n\nIf what you care about most is productivity, then SWIG is the clear winner if you want to wrap existing\nC/C++ code and Cython is the clear winner if you want to optimize existing Python code.\n\nIf all of the 3rd-party Python libraries you are using are compatible with PyPy, then using PyPy can\nprovide a painless performance boost for free.  Given that there are no code changes required to use it,\nit is very much worth trying it and seeing what sort of speedup you get.\n\nCFFI is pretty lame unless you are using PyPy.  But it is super easy to use with near zero learning curve.\nSo if you just want to call a function or two from an existing C library, it may be your best bet.  On\nthe other hand, if all of the 3rd party libraries you are using work with PyPy and you only want to\ninterface with C code (not C++), then the combination of PyPy + CFFI can result in some truly impressive\nperformance improvements.\n\npybind11 struggles with easy cross-platform compatibility and its performance is worse than SWIG, but it is more of a\npain to use than SWIG.  So I'd recommend staying away from it for now unless you are looking to embed Python code within\na C++ project on Windows.\n\n## Updates for 2019\nRecent re-evaluation of all these options firmly confirms my original conclusions.  `Cython` and `SWIG` are both awesome \nwith distinct tradeoffs.  The combination of `PyPy` + `CFFI` is attractive and shows a lot of potential.  While the \ncross-platform compatibility issues previously seen with `pybind11` appear to have been resolved, I still can't see a\nuse case for which it would be superior to one of the other options; but it has a ton of stars on GitHub so perhaps I \nam missing something?\n\n# Running Example Code Yourself\nFor information on getting all of the necessary prerequisites installed, see\n[Prerequisites for Running Example Code](https://github.com/tleonhardt/Python_Interface_Cpp/blob/master/Prerequisites.md).\n\nFor info on how to build and run each particular example, see the **Readme.md** in the example directory.\n\n\n# Excercises\nThere are a few examples here which are intended as guided exercises to help build your knowledge of how\nto use Cython and SWIG.\n\nThe exercises exist on the **master** branch, while the solutions exist on the **solutions** branch.\n\nMost exercises have **TODO:** comments in the locations where they want you to write some code and have\ninstructions in the Readme.md for that exercise.\n\n## Cython Exercises\n\n### Intro to Cython Exercise\nThe [integrate](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/cython/integrate) Cython\nexample serves as a good basic introduction to using Cython to optimize existing Python code.  It\ndetails a relatively typical series of steps which are followed in using Cython for process of\nprogressively optimizing existing Python code.\n\n### Cython for Wrapping\nThe [wrap_arrays](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/cython/wrap_arrays)\nCython example serves as an introduction to using Cython to wrap existing C code.  It purposely uses\nan example where the C functions take pointers to arrays, so it can help you learn how to generate\nwrapper code for this common type of scenario.\n\nThere are better (more optimal) ways of doing this than presented in the solution.  The solution tries\nto keep it simple.\n\n## SWIG Exercises\n\n### Cross-language Polymorphism in SWIG\nThe [logger](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/swig/logger) SWIG example\nserves as an introduction to how to achieve true cross-language polymorphism in SWIG by using directors.\nThis allows you to have a base class defined in C++, inherit from this class in Python, and then\ninstantiate a C++ class which takes a pointer to the base class and you pass it a pointer to an instance\nof the derived class and your C++ class will end up calling virtual methods defined in Python.\n\nWhile this sounds complicated and abstract, it is actually pretty simple to make use of it and is of\ngreat practical utility.\n\nAs a side benefit, this example also covers how to wrap STL std::string strings and effectively auto-cast\nthem to Python str objects.\n\n### Using STL Containers in SWIG\nThe [fastlz](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/swig/fastlz) SWIG example\nserves as an introduction to how to use STL containers such as vectors in your SWIG Python wrappers.  Also\nserves as an example of how to get your SWIG wrappers to link to dynamic libraries.\n\n## Solutions\nSolutions to all exercises are in the same location as the exercise, but on the **solutions** branch.\n\n\n# Presentation\nThe repository also has an overview [presentation](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/Interfacing_C_C++_with_Python.pdf) covering the basics of Cython, SWIG, and CFFI as well\nas when you might want to use each one.\n\nTo accompany this presentation, there is a [lab guide](https://github.com/tleonhardt/Python_Interface_Cpp/tree/master/Interfacing_C_C++_with_Python-Exercises.pdf) \nwhich provides an overview of the Cython and SWIG hand-on exercises in this repository.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftleonhardt%2Fpython_interface_cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftleonhardt%2Fpython_interface_cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftleonhardt%2Fpython_interface_cpp/lists"}