Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jorenham/unpy
Python stub backporter
https://github.com/jorenham/unpy
code-generation pep484 python python-typing
Last synced: 17 days ago
JSON representation
Python stub backporter
- Host: GitHub
- URL: https://github.com/jorenham/unpy
- Owner: jorenham
- License: bsd-3-clause
- Created: 2024-09-12T20:23:59.000Z (2 months ago)
- Default Branch: master
- Last Pushed: 2024-10-05T02:52:59.000Z (about 1 month ago)
- Last Synced: 2024-10-14T13:37:03.190Z (30 days ago)
- Topics: code-generation, pep484, python, python-typing
- Language: Python
- Homepage:
- Size: 438 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 29
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
unpy
Unified Python
Transpiles .pyi stubs from Python 3.13 to 3.10
---
> [!IMPORTANT]
> This project is in the alpha stage:
> You probably shouldn't use it in production.
## Installation
```console
$ pip install unpy
```
## Usage
```console
$ unpy --help
Usage: unpy [OPTIONS] SOURCE [OUTPUT]
Arguments:
SOURCE Path to the input .pyi file or '-' to read from stdin. [required]
[OUTPUT] Path to the output .pyi file. Defaults to stdout.
Options:
--version Show the version and exit
--diff Show the changes between the input and
output in unified diff format
--target [3.10|3.11|3.12|3.13] The minimum Python version that should be
supported. [default: 3.10]
--help Show this message and exit.
```
## Examples
Some simple examples of Python 3.13 stubs that are backported to Python 3.10.
### Imports
```console
$ unpy --target 3.10 --diff examples/imports.pyi
```
```diff
+++ -
@@ -1,6 +1,4 @@
- from types import CapsuleType
- from typing import override
- from warnings import deprecated
+ from typing_extensions import CapsuleType, deprecated, override
@deprecated("RTFM")
class Spam:
__pyx_capi__: dict[str, CapsuleType]
@override
def __hash__(self, /) -> int: ...
```
Note the alphabetical order of the generated imports.
### Type Aliases
```console
$ unpy --target 3.10 --diff examples/type_aliases.pyi
```
```diff
+++ -
@@ -1,7 +1,15 @@
from collections.abc import Callable
+ from typing import ParamSpec, TypeAlias, TypeVar
+ from typing_extensions import TypeAliasType, TypeVarTuple, Unpack
- type Binary = bytes | bytearray | memoryview
- type Vector[R: float] = tuple[R, ...]
- type tciD[V, K] = dict[K, V]
- type Things[*Ts] = tuple[*Ts]
- type Callback[**Tss] = Callable[Tss, None]
+ _R = TypeVar("_R", bound=float)
+ _V = TypeVar("_V")
+ _K = TypeVar("_K")
+ _Ts = TypeVarTuple("_Ts")
+ _Tss = ParamSpec("_Tss")
+
+ Binary: TypeAlias = bytes | bytearray | memoryview
+ Vector: TypeAlias = tuple[_R, ...]
+ tciD = TypeAliasType("tciD", dict[_K, _V], type_params=(_V, _K))
+ Things: TypeAlias = tuple[Unpack[_Ts]]
+ Callback: TypeAlias = Callable[_Tss, None]
```
Note that `TypeAlias` cannot be used with `tciD` because the definition order of the
type parameters (at the left-hand side) does not match the order in which they are
accessed (at the right-hand side), and the backported `TypeAliasType` must be used
instead.
### Functions
```console
$ unpy --target 3.10 --diff examples/functions.pyi
```
```diff
+++ -
@@ -1,6 +1,11 @@
+ _T = TypeVar("_T")
+ _S = TypeVar("_S", str, bytes)
+ _X = TypeVar("_X")
+ _Theta = ParamSpec("_Theta")
+ _Y = TypeVar("_Y")
from collections.abc import Callable as Def
- from typing import Concatenate as Concat
+ from typing import Concatenate as Concat, ParamSpec, TypeVar
- def noop[T](x: T, /) -> T: ...
- def concat[S: (str, bytes)](left: S, right: S) -> S: ...
- def curry[X, **Theta, Y](f: Def[Concat[X, Theta], Y], /) -> Def[[X], Def[Theta, Y]]: ...
+ def noop(x: _T, /) -> _T: ...
+ def concat(left: _S, right: _S) -> _S: ...
+ def curry(f: Def[Concat[_X, _Theta], _Y], /) -> Def[[_X], Def[_Theta, _Y]]: ...
```
### Generic classes and protocols
```console
$ unpy --target 3.10 --diff examples/generics.pyi
```
```diff
+++ -
@@ -1,17 +1,25 @@
- from typing import Protocol, overload
+ from typing import Generic, Protocol, overload
+ from typing_extensions import TypeVar
+
+ _T_contra = TypeVar("_T_contra", contravariant=True)
+ _T_co = TypeVar("_T_co", covariant=True)
+ _T = TypeVar("_T", infer_variance=True)
+ _D = TypeVar("_D")
+ _NameT = TypeVar("_NameT", infer_variance=True, bound=str)
+ _QualNameT = TypeVar("_QualNameT", infer_variance=True, bound=str, default=_NameT)
class Boring: ...
- class CanGetItem[T_contra, T_co](Protocol):
- def __getitem__(self, k: T_contra, /) -> T_co: ...
+ class CanGetItem(Protocol[_T_contra, _T_co]):
+ def __getitem__(self, k: _T_contra, /) -> _T_co: ...
- class Stack[T]:
- def push(self, value: T, /) -> None: ...
+ class Stack(Generic[_T, _D]):
+ def push(self, value: _T, /) -> None: ...
@overload
- def pop(self, /) -> T: ...
+ def pop(self, /) -> _T: ...
@overload
- def pop[D](self, default: D, /) -> T | D: ...
+ def pop(self, default: _D, /) -> _T | _D: ...
- class Named[NameT: str, QualNameT: str = NameT]:
- __name__: NameT
- __qualname__: QualNameT
+ class Named(Generic[_NameT, _QualNameT]):
+ __name__: _NameT
+ __qualname__: _QualNameT
```
Note how `TypeVar` is (only) imported from `typing_extensions` here, which wasn't the
case in the previous example. This is a consequence of the `infer_variance` parameter,
which has been added in Python 3.12.
## Project goals
Here's the alpha version of a prototype of a rough sketch of some initial ideas for the
potential goals of `unpy`:
1. Towards the past
- [x] Get frustrated while [stubbing scipy](https://github.com/jorenham/scipy-stubs)
- [x] Transpile Python 3.13 `.pyi` stubs to Python 3.10 stubs
- [ ] Package-level analysis and conversion
- [ ] Tooling for stub-only project integration
- [ ] Use this in [`scipy-stubs`](https://github.com/jorenham/scipy-stubs)
- [ ] Gradually introduce this into [`numpy`](https://github.com/numpy/numpy)
2. Towards the future
- [ ] Beyond Python: $\text{Unpy} \supset \text{Python}$
- [ ] Language support & tooling for *all* `.py` projects
3. Towards each other
- [ ] Unified typechecking: Fast, reasonable, and language-agnostic
## Features
### Tooling
- Target Python versions
- [x] `3.13`
- [x] `3.12`
- [x] `3.11`
- [x] `3.10`
- [ ] `3.9`
- Language support
- [x] `.pyi`
- [ ] `.py`
- Conversion
- [x] stdin => stdout
- [x] module => module
- [ ] package => package
- [ ] project => project (including the `pyproject.toml`)
- Configuration
- [x] `--diff`: Unified diffs
- [x] `--target`: Target Python version, defaults to `3.10`
- [ ] Project-based config in `pyproject.toml` under `[tools.unpy]`
- [ ] ...
- Integration
- [ ] File watcher
- [ ] Pre-commit
- [ ] LSP
- [ ] UV
- [ ] VSCode extension
- [ ] (based)mypy plugin
- [ ] Project build tools
- [ ] Configurable type-checker integration
- [ ] Configurable formatter integration, e.g. `ruff format`
- Performance
- [ ] Limit conversion to changed files
### Stub backporting
- Python 3.13 => 3.12
- [PEP 742][PEP742]
- `typing.TypeIs` => `typing_extensions.TypeIs`
- [PEP 705][PEP705]
- `typing.ReadOnly` => `typing_extensions.ReadOnly`
- [PEP 702][PEP702]
- `warnings.deprecated` => `typing_extensions.deprecated`
- [PEP 696][PEP696]
- Backport [PEP 695][PEP695] type signatures with a default
- `typing.NoDefault` => `typing_extensions.NoDefault`
- Exceptions
- `asyncio.QueueShutDown` => `builtins.Exception`
- `pathlib.UnsupportedOperation` => `builtins.NotImplementedError`
- `queue.ShutDown` => `builtins.Exception`
- `re.PatternError` => `re.error`
- Typing
- `types.CapsuleType` => `typing_extensions.CapsuleType`
- `typing.{ClassVar,Final}` => `typing_extensions.{ClassVar,Final}` when
nested
- Python 3.12 => 3.11
- [PEP 698][PEP698]
- `typing.override` => `typing_extensions.override`
- [PEP 695][PEP695]
- Backport `type _` aliases
- Backport generic functions
- Backport generic classes and protocols
- `typing.TypeAliasType` => `typing_extensions.TypeAliasType`
- [PEP 688][PEP688]
- `collections.abc.Buffer` => `typing_extensions.Buffer`
- `inspect.BufferFlags` => `int`
- Python 3.11 => 3.10
- [PEP 681][PEP681]
- `typing.dataclass_transform` => `typing_extensions.dataclass_transform`
- [PEP 675][PEP675]
- `typing.LiteralString` => `typing_extensions.LiteralString`
- [PEP 673][PEP673]
- `typing.Self` => `typing_extensions.Self`
- [PEP 655][PEP655]
- `typing.[Not]Required` => `typing_extensions.[Not]Required`
- [PEP 654][PEP654]
- ~`builtins.BaseExceptionGroup`~
- ~`builtins.ExceptionGroup`~
- [PEP 646][PEP646]
- `typing.TypeVarTuple` => `typing_extensions.TypeVarTuple`
- `typing.Unpack` => `typing_extensions.Unpack`
- `*Ts` => `typing_extensions.Unpack[Ts]` with `Ts: TypeVarTuple`
- `asyncio`
- ~`asyncio.TaskGroup`~
- `enum`
- `enum.ReprEnum` => `enum.Enum`
- `enum.StrEnum` => `str & enum.Enum`
- `typing`
- `typing.Any` => `typing_extensions.Any` if subclassed (not recommended)
- Generated `TypeVar`s
- [x] De-duplicate extracted typevar-likes with same name if equivalent
- [x] Prefix the names of extracted typevar-likes with `_`
- [ ] Rename incompatible typevar-likes with the same name (jorenham/unpy#86)
### Simplification and refactoring
- Generic type parameters
- [x] Convert `default=Any` with `bound=T` to `default=T`
- [x] Remove `bound=Any` and `bound=object`
- [ ] Infer variance of PEP 695 type parameters (jorenham/unpy#44)
- [ ] If never used, it's redundant (and bivariant) (jorenham/unpy#46)
- [x] If constraints are specified, it's `invariant`
- [x] If suffixed with `_co`/`_contra`, it's `covariant`/`contravariant`
- [ ] If used as public instance attribute, it's `invariant`
- [ ] If only used as return-type (excluding `__init__` and `__new__`), or for
read-only attributes, it's `covariant`
- [ ] If only used as parameter-type, it's `contravariant`
- [ ] Otherwise, assume it's `invariant`
- Methods
- [ ] Default return types for specific "special method" (jorenham/unpy#55)
- [ ] Transform `self` method parameters to be positional-only
- Typing operators
- [ ] `type[S] | type[T]` => `type[S | T]`
- [ ] Flatten & de-duplicate unions of literals
- [ ] Remove redundant union values, e.g. `bool | int` => `int`
### Beyond Python
- [ ] `@sealed` types (jorenham/unpy#42)
- [ ] Unified type-ignore comments (jorenham/unpy#68)
- [ ] Set-based `Literal` syntax (jorenham/unpy#76)
- [ ] Reusable method signature definitions (jorenham/unpy#97, jorenham/unpy#98)
- [ ] Type-mappings, a DRY alternative to `@overload`
- [ ] Intersection types (as implemented in [basedmypy][BMP-ISEC])
- [ ] Higher-kinded types (see python/typing#548)
- [ ] Inline callable types (inspired by [PEP 677][PEP677])
[PEP646]: https://peps.python.org/pep-0646/
[PEP654]: https://peps.python.org/pep-0654/
[PEP655]: https://peps.python.org/pep-0655/
[PEP673]: https://peps.python.org/pep-0673/
[PEP675]: https://peps.python.org/pep-0675/
[PEP677]: https://peps.python.org/pep-0677/
[PEP681]: https://peps.python.org/pep-0681/
[PEP688]: https://peps.python.org/pep-0688/
[PEP695]: https://peps.python.org/pep-0695/
[PEP696]: https://peps.python.org/pep-0696/
[PEP698]: https://peps.python.org/pep-0698/
[PEP702]: https://peps.python.org/pep-0702/
[PEP705]: https://peps.python.org/pep-0705/
[PEP742]: https://peps.python.org/pep-0705/
[BMP-ISEC]: https://github.com/KotlinIsland/basedmypy#intersection-types