Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tqchen/ffi-navigator
https://github.com/tqchen/ffi-navigator
ffi language-server tvm vscode-extension
Last synced: 6 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/tqchen/ffi-navigator
- Owner: tqchen
- License: apache-2.0
- Created: 2019-12-27T16:44:58.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2023-02-05T16:25:17.000Z (almost 2 years ago)
- Last Synced: 2025-01-19T14:12:19.894Z (13 days ago)
- Topics: ffi, language-server, tvm, vscode-extension
- Language: Python
- Size: 155 KB
- Stars: 231
- Watchers: 13
- Forks: 24
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# FFI Navigator
[](https://dev.azure.com/ffi-navigator/ffi-navigator/_build/latest?definitionId=1&branchName=master)
Most modern IDEs support find function definition within the same language(e.g. python or c++). However, it is very hard to do that for cross-language FFI calls. While solving this general problem can be very technically challenging, we can get around it by building a project-specific analyzer that matches the FFI registration code patterns and recovers the necessary information.
This project is an example of that. Currently, it supports the PackedFunc FFI in the Apache TVM project. It is implemented as a [language server](https://microsoft.github.io/language-server-protocol/) that provides getDefinition function for FFI calls and returns the location of the corresponding C++ API in the TVM project. It complements the IDE tools that support navigation within the same language. We also have preliminary support for MXNet, DGL, and PyTorch, so we can do goto-definition from Python to C++ in these projects too.
## Installation
Install python package
```bash
pip install --user ffi-navigator
```
### VSCode
See [vscode-extension](vscode-extension)
### Emacs
Install [lsp-mode](https://github.com/emacs-lsp/lsp-mode)
Add the following configuration
```el
(lsp-register-client
(make-lsp-client
:new-connection (lsp-stdio-connection '("python3" "-m" "ffi_navigator.langserver"))
:major-modes '(python-mode c++-mode)
:server-id 'ffi-navigator
:add-on? t))
```
- Use commands like `M-x` `lsp-find-definition` and `M-x` `lsp-find-references`
If you use eglot instead, check out [this PR](https://github.com/tqchen/ffi-navigator/pull/1).
eglot does not support multiple servers per language at the moment, the PR above contains a workaround.
### Other editors/IDEs
It should be straightforward to add support for other editors or IDEs that have a LSP client implementation.
Please refer to this [site](https://langserver.org/) for the availability of clients.
Since ffi-navigator is intended to be used with other general purpose servers such as [pyls](https://github.com/palantir/python-language-server), your LSP client needs to be able to talk to multiple servers per language. If your client does not have such feature, check out [this PR](https://github.com/tqchen/ffi-navigator/pull/1) and the discussion there for a workaround. This [branch](https://github.com/masahi/tvm-ffi-navigator/tree/pyls-fallback) has fallback to pyls and it is always kept up to date with the master branch.
## Features
### TVM FFI
- Find definition/references of FFI objects(e.g. PackedFunc in TVM) on the python and c++ side.
- Jump from a python PackedFunc into ```TVM_REGISTER_GLOBAL```, ```@register_func```
- Find definition/references of FFI objects on the python and c++ side.
- move cursor to the object class name on the python side.
- move cursor to the ```_type_key``` string on the c++ side.
### PyTorch
- Jump to C10 registered ops. In python they corresponds to functions under `torch.ops` namespace.
- Example: `torch.ops.quantized.conv2d (py)` -> `c10::RegisterOperators().op("quantized::conv2d", ...) (cpp)`
- Jump to cpp functions wrapped by pybind.
- Example: `torch._C._jit_script_class_compile (py)` -> `m.def( "_jit_script_class_compile", ...) (cpp)`
## Development
For developing the python package locally, we can just make sure ffi_navigator is in your python path in bashrc.
```bash
export PYTHONPATH=${PYTHONPATH}:/path/to/ffi-navigator/python
```
### Project Structure
- python/ffi_navigator The analysis code and language server
- python/ffi_navigator/dialect Per project dialects
- vscode-extension language server extension for vscode
### Adding Support for New FFI Patterns
Add your FFI convention to [dialect namespace](python/ffi_navigator/dialect).
## Demo
### VSCode
See [vscode-extension](vscode-extension)
### Emacs
#### Goto definition from Python to C++
#### Goto definition from C++ to Python
#### Find reference across Python and C++
#### Goto definition in PyTorch
