{"id":24092732,"url":"https://github.com/liarprincess/violet","last_synced_at":"2025-04-04T17:04:17.056Z","repository":{"id":40421092,"uuid":"414187036","full_name":"LiarPrincess/Violet","owner":"LiarPrincess","description":"Python VM written in Swift","archived":false,"fork":false,"pushed_at":"2025-02-13T11:14:10.000Z","size":8293,"stargazers_count":189,"open_issues_count":7,"forks_count":7,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T16:04:01.679Z","etag":null,"topics":["compiler","language","python3","swift","vm"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LiarPrincess.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"LiarPrincess"}},"created_at":"2021-10-06T11:37:36.000Z","updated_at":"2025-02-13T11:14:15.000Z","dependencies_parsed_at":"2025-03-21T15:05:33.949Z","dependency_job_id":"e95a7089-f702-45cc-8173-e48c5ecb1461","html_url":"https://github.com/LiarPrincess/Violet","commit_stats":{"total_commits":2217,"total_committers":5,"mean_commits":443.4,"dds":0.003157419936851591,"last_synced_commit":"404d10f644a7c750bf95f219827c8a9334379228"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiarPrincess%2FViolet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiarPrincess%2FViolet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiarPrincess%2FViolet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiarPrincess%2FViolet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LiarPrincess","download_url":"https://codeload.github.com/LiarPrincess/Violet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217174,"owners_count":20903008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","language","python3","swift","vm"],"created_at":"2025-01-10T08:37:19.997Z","updated_at":"2025-04-04T17:04:17.030Z","avatar_url":"https://github.com/LiarPrincess.png","language":"Swift","funding_links":["https://github.com/sponsors/LiarPrincess"],"categories":[],"sub_categories":[],"readme":"# Violet\n\nViolet is one of those Swift \u003c-\u003e Python interop thingies, except that this time we implement the whole language from scratch. Name comes from [Violet Evergarden](https://www.netflix.com/pl-en/title/80182123).\n\n[Many](https://www.imdb.com/title/tt7923710) [unwatched](https://www.imdb.com/title/tt12451520) [k-drama](https://www.imdb.com/title/tt10220588) [hours](https://www.imdb.com/title/tt10850932) [were](https://www.imdb.com/title/tt8242904) [put](https://www.imdb.com/title/tt14169770) [into](https://www.imdb.com/title/tt13067118) [this](https://www.imdb.com/title/tt6263222), so any ⭐  would be appreciated.\n\nIf something is not working, you have an interesting idea or maybe just a question, then you can start an issue or discussion. You can also contact us on twitter [@itBrokeAgain](https://twitter.com/itBrokeAgain) (optimism, yay!).\n\n- [Violet](#violet)\n  - [Requirements](#requirements)\n  - [Features](#features)\n  - [Future plans](#future-plans)\n  - [Sources](#sources)\n  - [Tests](#tests)\n  - [Code style](#code-style)\n  - [License](#license)\n\n## Requirements\n\n- 64 bit - for `BigInt` and hash\n- Platforms\n    - macOS\n      - Intel\n        - 11.6.2 (Big Sur) + Xcode 12.4 (Swift 5.3.2)\n        - 11.6.2 (Big Sur) + Xcode 13.0 (Swift 5.5)\n      - Apple\n        - 12.3.1 (Monterey) + Xcode 13.3.1 (Swift 5.5.3)\n    - Ubuntu\n      - 21.04 + Swift 5.4.2 - use `make test` and `make pytest`\n    - Docker\n      - `swift:latest` (5.6.0) - use `make docker-test` and `make docker-pytest`\n      - `swift:5.3.2` - use `make docker-test-old` and `make docker-pytest-old`\n\nThe whole Violet was written on 2014 rMBP (lowest spec: 8GB of ram + 128 GB storage), so it is safe to say that there are no other requirements.\n\n## Features\n\nWe aim for compatibility with Python 3.7 feature set.\n\nWe are only interested in the language itself without additional modules. This means that importing anything except for most basic modules (`sys`, `builtins` and a few others) is not supported (although you can import other Python files).\n\nSee `Documentation` directory for a list of known unimplemented features. There is no list of unknown unimplemented features though…\n\n## Future plans\n\n- **Garbage collection** is a nifty feature. Currently we allocate objects, but the only way to deallocate them is to call `py.destroy()` which destroys the whole Python context (and all of the objects that it owns).\n\n    Btw. please remember to use [with statement](https://www.python.org/dev/peps/pep-0343/) to manage resources, do not rely on object lifetime (especially for the file descriptors).\n\n- **Tail allocated `tuples`**. Currently we store `tuple` elements inside Swift array (`elements: [PyObject]`). The better idea would be to allocate more space *after* the tuple and store elements there (this is called [flexible array  member](https://en.wikipedia.org/wiki/Flexible_array_member) in `C`). This saves a pointer indirection and is better for cache, since we can fit a few first elements in the same line as `type`, `__dict__` etc. We can also do this for other immutable container types:\n  - `str` - currently native Swift `String`. This would force us to implement our own `String` type - not hard, but takes a lot of time.\n  - `int` - currently our own `BigInt` implementation (which does store values in `Int32` range inside the pointer).\n\n## Sources\n\nYou can browse all of the module exports - `open`/`public` declarations - [here]( https://github.com/LiarPrincess/Violet/tree/main/Documentation/Module%20exports) (generated by [Ariel](https://github.com/LiarPrincess/Ariel)).\n\nCore modules\n- **VioletCore** — shared module imported by all of the other modules.\n    - Contains things like `NonEmptyArray`, `SourceLocation`, [SipHash](https://131002.net/siphash/), `trap` and `unreachable`.\n- **BigInt** — our implementation of unlimited integers\n    - While it implements all of the operations expected of `BigInt` type, in reality it mostly focuses on performance of small integers — Python has only one `int` type and small numbers are most common.\n    - Under the hood it is a union (via [tagged pointer](https://en.wikipedia.org/wiki/Tagged_pointer)) of `Int32` (called `Smi`, after [V8](https://github.com/v8/v8)) and a heap allocation (magnitude + sign representation) with ARC for garbage collection. \u003c\u003c That's mouthful 💤\n    - While the whole Violet tries to be as easy-to-read/accessible as possible, this does not apply to `BigInt` module. Numbers are hard, and for some reason humanity decided that “division” is a thing.\n- **FileSystem** — our version of `Foundation.FileManager`.\n    - Code quality varies. Most of the time it was “ehh… I need to implement another IO thing”. Then, later, all of those “ehs…” were put into a single module. In so-called meantime the wild [swift-system 🐯](https://github.com/apple/swift-system) appeared, so maybe it is time to use it?\n    - Main reason why we do not support other platforms (Windows etc.).\n- **UnicodeData** — apparently we also bundle our own Unicode database, because why not…\n  - This is [kind of important](https://hsivonen.fi/string-length/).\n\nViolet\n- **VioletLexer** — transforms Python source code into a stream of tokens.\n- **VioletParser** — transforms a stream of tokens (from `Lexer`) into an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) (`AST`).\n    - Yet Another [Recursive Descent Parser](https://en.wikipedia.org/wiki/Recursive_descent_parser) with minor hacks for ambiguous grammar.\n    - `AST` type definitions are generated by `Elsa` module from `Elsa definitions/ast.letitgo`.\n- **VioletBytecode** — instruction set of our VM.\n    - 2-bytes per `enum Instruction`. There are a few interesting cases, like `.formatValue(conversion: StringConversion, hasFormat: Bool)` (where `StringConversion` is an `enum` with 4 possible values), but the compiler is expected to deal with it.\n    - No relative jumps, only absolute (via additional `labels` array).\n    - Instruction set is generated by `Elsa` module from `Elsa definitions/opcodes.letitgo`.\n    - Use `CodeObjectBuilder` to create `CodeObjects`  (whoa… what a surprise!).\n    - Includes a tiny [peephole optimizer](https://en.wikipedia.org/wiki/Peephole_optimization), because sometimes the semantics depends on it (for example for [short-circuit evaluation](https://en.wikipedia.org/wiki/Short-circuit_evaluation)).\n- **VioletCompiler** — responsible for transforming `Parser.AST` into `Bytecode.CodeObject`.\n- **VioletObjects** — contains all of the Python objects and modules.\n    - `Py` represents a Python context. Common usage: `py.newInt(2)` or `py.add(lhs, rhs)`.\n    - Contains `int`, `str`, `list` and 100+ other Python types.\n    - Python object is represented as a Swift `struct` with a single `ptr: RawPtr` stored property. The `ptr` points to a heap allocated storage with custom layout. Layout is generated by [Sourcery](https://github.com/krzysztofzablocki/Sourcery) using `sourcery: storedProperty` annotations. Read the docs in the `Documentation` directory!\n\n        ```Swift\n        // sourcery: pytype = int\n        public struct PyInt: PyObjectMixin {\n          // sourcery: storedProperty\n          public var value: BigInt { self.valuePtr.pointee }\n\n          public let ptr: RawPtr\n        }\n        ```\n\n    - Contains modules required to bootstrap Python: `builtins`, `sys`, `_imp`, `_os` and `_warnings`.\n    - Does not contain `importlib` and `importlib_external` modules, because those are written in Python. They are a little bit different than CPython versions (we have 80% of the code, but only 20% of the functionality \u003cgreat-success-meme.gif\u003e).\n    - `PyResult\u003cWrapped\u003e = Wrapped | PyBaseException` is used for error handling.\n- **VioletVM** — manipulates Python objects according to the instructions from `Bytecode.CodeObject`, so that the output vaguely resembles what `CPython` does.\n    - Mainly a massive `switch` over each possible `Instruction`.\n- **Violet** — main executable (duh…).\n- **PyTests** — runs tests written in Python from the `PyTests` directory.\n\nTools/support\n- **Elsa** — tiny DSL for code generation.\n    - Uses `.letitgo` files from `Elsa definitions` directory.\n    - Used for `Parser.AST` and `Bytecode.Instruction` types.\n- **Rapunzel** — pretty printer based on “[A prettier printer](http://homepages.inf.ed.ac.uk/wadler/papers/prettier/prettier.pdf)” by Philip Wadler.\n    - Used to print `AST` in digestible manner.\n\n## Tests\n\nThere are 2 types of tests in Violet:\n- Swift tests — standard Swift unit tests stored inside the `./Tests` directory. You can run them by typing `make test` in repository root.\n\n    You may want to disable unit tests for `BigInt` and `UnicodeData` if you are not touching those modules:\n    - `BigInt` — we went with [property based testing](https://en.wikipedia.org/wiki/Property_testing) with means that we test millions of inputs to check if the general rule holds (for example: `a+b=c -\u003e c-a=b` etc.). This takes time, but pays for itself by finding weird overflows in bit operations (we store “sign + magnitude”, so bit operations are a bit difficult to implement).\n    - `UnicodeData`\n        - In one of our tests we go through all of the Unicode code points and try to access various properties (crash -\u003e fail). There are `0x11_0000` values to test, so… it is not fast.\n        - We also have a few thousands of tests generated by Python. Things like: “is the `COMBINING VERTICAL LINE ABOVE (U+030d)` alpha-numeric?” (Answer: no, it is not. But you have to watch out because `HANGUL CHOSEONG THIEUTH (U+1110)` is).\n\n- Python tests — tests written in Python stored inside the `./PyTests` directory.  You can run them by typing `make pytest` in repository root (there is also `make pytest-r` for release mode).\n    - Violet - tests written specially for “Violet”.\n    - RustPython - tests taken from [github.com/RustPython](https://github.com/RustPython/RustPython).\n\n    Those tests are executed when you run `PyTests` module.\n\n## Code style\n\n- 2-space indents and no tabs at all\n- 80 characters per line\n    - You will get a [SwiftLint](https://github.com/realm/SwiftLint) warning if you go over 100.\n    - Over 120 will result in a compilation error.\n    - If 80 doesn't give you enough room to code, your code is too complicated - consider using subroutines (advice from [PEP-7](https://www.python.org/dev/peps/pep-0007/)).\n- Required `self` in methods and computed properties\n    - All of the other method arguments are named, so we will require it for this one.\n    - `Self`/`type name` for static methods is recommended, but not required.\n    - I’m sure that they will depreciate the implicit `self` in the next major Swift version 🤞. All of that source breakage is completely justified.\n- No whitespace at the end of the line\n    - Some editors may remove it as a matter of routine and we don’t want weird git diffs.\n- (pet peeve) Try to introduce a named variable for every `if` condition.\n    - You can use a single logical operator - something like `if !isPrincess` or `if isDisnepCharacter \u0026\u0026 isPrincess` is allowed.\n    - Do not use `\u0026\u0026` and `||` in the same expression, create a variable for one of them.\n    - If you need parens then it is already too complicated.\n\nAnyway, just use [SwiftLint](https://github.com/realm/SwiftLint) and [SwiftFormat](https://github.com/nicklockwood/SwiftFormat) with provided presets (see [.swiftlint.yml](.swiftlint.yml) and [.swiftformat](.swiftformat) files).\n\n## License\n\n“Violet” is licensed under the MIT License.\nSee [LICENSE](LICENSE) file for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliarprincess%2Fviolet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fliarprincess%2Fviolet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliarprincess%2Fviolet/lists"}