{"id":21331556,"url":"https://github.com/lukechampine/ply","last_synced_at":"2025-07-12T10:30:50.501Z","repository":{"id":67869506,"uuid":"75534691","full_name":"lukechampine/ply","owner":"lukechampine","description":"Painless polymorphism","archived":false,"fork":false,"pushed_at":"2017-03-21T05:29:02.000Z","size":398,"stargazers_count":125,"open_issues_count":10,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-06-20T17:41:48.009Z","etag":null,"topics":["generics","transpiler"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lukechampine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-04T12:00:23.000Z","updated_at":"2024-05-01T11:28:39.000Z","dependencies_parsed_at":"2023-03-03T06:45:46.585Z","dependency_job_id":null,"html_url":"https://github.com/lukechampine/ply","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukechampine%2Fply","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukechampine%2Fply/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukechampine%2Fply/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukechampine%2Fply/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lukechampine","download_url":"https://codeload.github.com/lukechampine/ply/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225814913,"owners_count":17528295,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generics","transpiler"],"created_at":"2024-11-21T22:42:20.818Z","updated_at":"2024-11-21T22:42:21.391Z","avatar_url":"https://github.com/lukechampine.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"ply\n===\n\n`ply` is an experimental compile-to-Go language. Its syntax and semantics are\nbasically identical to Go's, but with more builtin functions for manipulating\ngeneric containers (slices, arrays, maps). This is accomplished by forking\nGo's type-checker, running it on the `.ply` file, and using the resolved types\nto generate specific versions of the generic function. For example, given the\nfollowing Ply code:\n\n```go\nm1 := map[int]int{1: 1}\nm2 := map[int]int{2: 2}\nm3 := merge(m1, m2)\n```\n\n`merge` is a generic function. After type-checking, the Ply compiler knows the\ntypes of `m1` and `m2`, so it can generate a specific function for these types:\n\n```go\nfunc mergeintint(m1, m2 map[int]int) map[int]int {\n\tm3 := make(map[int]int)\n\tfor k, v := range m1 {\n\t\tm3[k] = v\n\t}\n\tfor k, v := range m2 {\n\t\tm3[k] = v\n\t}\n\treturn m3\n}\n```\n\n`mergeintint` is then substituted for `merge` in the relevant expression, and\nthe modified source can then be passed to the Go compiler.\n\nA similar approach is used to implement generic methods:\n\n```go\nxs := []int{1, 2, 3, 4, 6, 20}\nb := xs.filter(func(x int) bool { return x \u003e 3 }).\n        morph(func(x int) bool { return x % 2 == 0 }).\n        fold(func(x, y bool) bool { return x \u0026\u0026 y })\n```\n\nIn the above, `b` is true because all the integers in `xs` greater than 3 are\neven. To compile this, `xs` is wrapped in a new type that has a `filter`\nmethod. Then, that call is wrapped in a new type that has a `morph` method,\nand so on.\n\nNote that in most cases, Ply can combine these method chains into a single\n\"pipeline\" that **does not allocate any intermediate slices**. Without\npipelining, `filter` would allocate a slice and pass it to `morph`, which\nwould allocate another slice and pass it to `fold`. But Ply is able to merge\nthese methods into a single transformation that does not require allocations,\nthe same way a (good) human programmer would write it.\n\nUsage\n-----\n\nFirst, install the Ply compiler:\n\n```\ngo get github.com/lukechampine/ply\n```\n\nThe `ply` command behaves similarly to the `go` command. In fact, you can run\nany `go` subcommand through `ply`, including `build`, `run`, `install`, and\neven `test`.\n\nWhen you run `ply run test.ply`, `ply` parses `test.ply` and generates a\n`ply-impls.go` file containing the specific implementations of any generics\nused in `test.ply`. It then rewrites `test.ply` as a standard Go file,\n`ply-test.go`, that calls those implementations. Finally, `go run` is invoked\non `ply-test.go` and `ply-impls.go`.\n\n\nSupported Functions and Methods\n-------------------------------\n\n**Builtins:** `enum`, `max`, `merge`, `min`, `not`, `zip`\n\n- Planned: `repeat`, `compose`\n\n**Methods:** `all`, `any`, `contains`, `drop`, `dropWhile`, `elems`, `filter`,\n`fold`, `foreach`, `keys`, `morph`, `reverse`, `sort`, `take`, `takeWhile`,\n`tee`, `toMap`, `toSet`, `uniq`\n\n- Planned: `join`, `replace`, `split`\n\nAll functions and methods are documented in the [`ply` pseudo-package](https://godoc.org/github.com/lukechampine/ply/doc).\n\n\nSupported Optimizations\n-----------------------\n\nIn many cases we can reduce allocations when using Ply functions and methods.\nThe Ply compiler will automatically apply these optimizations when it is safe\nto do so. However, all optimizations have trade-offs. If performance is\nimportant, you should always read the docstring of each method in order to\nunderstand what optimizations may be applied. Depending on your use case, it\nmay be necessary to write your own implementation to squeeze out maximum\nperformance.\n\n**Pipelining:**\n\nPipelining means chaining together multiple Ply functions and/or methods.\nCurrently only method chaining is supported. For example:\n\n```go\nxs := []int{1, 2, 3, 4, 6, 20}\nb := xs.filter(func(x int) bool { return x \u003e 3 }).\n        morph(func(x int) bool { return x % 2 == 0 }).\n        fold(func(acc, x bool) bool { return acc \u0026\u0026 x })\n```\n\nAs written, this chain requires allocating a new slice for the `filter` and a\nnew slice for the `morph`. But if we were writing this transformation by hand,\nwe could optimize it like so:\n\n```go\nb := true\nfor _, x := range xs {\n\tif x \u003e 3 {\n\t\tb = b \u0026\u0026 (x % 2 == 0)\n\t}\n}\n```\n\n(A good rule of thumb is that, for most chains, only the allocations in the\nfinal method are required. `fold` doesn't require any allocations, but if the\nchain stopped at `morph`, then of course we would still need to allocate\nmemory in order to return the morphed slice.)\n\nPly is able to perform the above optimization automatically. The bodies of\n`filter`, `morph`, and `fold` are combined into a single method, `pipe`, and\nthe callsite is rewritten to supply the arguments of each chained function:\n\n```go\nxs := []int{1, 2, 3, 4, 6, 20}\nb := filtermorphfold(xs).pipe(\n\t\tfunc(x int) bool { return x \u003e 3 },\n\t\tfunc(x int) bool { return x % 2 == 0 },\n\t\tfunc(x, y bool) bool { return x \u0026\u0026 y })\n```\n\nHowever, not all methods can be pipelined. `reverse` is a good example. If\n`reverse` is the first method in the chain, then we can eliminate an\nallocation by reversing the order in which we iterate through the slice. We\ncan also eliminate an allocation if `reverse` is the last method in the chain,\nsince we can reverse the result in-place. But what do we do if `reverse` is in\nthe middle? Consider this chain:\n\n```go\nxs.takeWhile(even).reverse().morph(square)\n```\n\nSince we don't know what `takeWhile` will return, there is no way to pass its\nreversed elements to `morph` without allocating an intermediate slice. So we\nresort to a less-efficient form, splitting the chain into `takeWhile(even)`\nand `reverse().morph(square)`, each of which will perform an allocation.\n\nFortunately, it is usually possible to reorder the chain such that `reverse`\nis the first or last method. In the above, we know that `morph` doesn't affect\nthe length or order of the slice, so we can move `reverse` to the end and the\nresult will be the same. Ply can't perform this reordering automatically\nthough: methods may have side effects that the programmer is relying upon.\n\nSide effects are also problematic because pipelining can change the number of\ntimes a function is called. For example, in this expression:\n\n```go\n[]int{1, 2, 3, 4, 6, 20}.morph(fn).take(3)\n```\n\nWithout pipelining, `fn` is called on every element of the slice. But with\npipelining, it is only called 3 times. So the best practice is to avoid side\neffects in functions passed to `morph`, `filter`, etc.\n\nLastly, it's worth pointing out that pipelining cannot eliminate any\nallocations performed inside function arguments. For example, in this chain:\n\n```go\nmyEnum := func(n int) []int {\n\tr := make([]int, n)\n\tfor i := range r {\n\t\tr[i] = i\n\t}\n\treturn r\n}\nconcat := func(x, y []int) []int { return append(x, y...) }\nlist := xs.morph(myEnum).fold(concat)\n```\n\nA handwritten version of this chain could eliminate the allocations performed\nby `myEnum`, but there is no way to do so programmatically.\n\n\n**Parallelization (planned):**\n\nFunctor operations like `morph` can be trivially parallelized, but this\noptimization should not be applied automatically. For small lists, the\noverhead is probably not worth it. More importantly, if the function has side\neffects, parallelizing may cause a race condition. So this optimization must\nbe specifically requested by the caller via separate identifiers, e.g.\n`pmorph`, `pfilter`, etc.\n\n**Reassignment (planned):**\n\nIt is a common pattern to reassign the result of a transformation to the\noriginal variable, for example when filtering or reversing a slice. In such\ncases, we would like to reuse the existing slice's memory instead of\nallocating a new one. At one time, Ply did this automatically (by detecting\nreassignment), but the feature was later removed because it is not provably\nsafe. If the underlying slice memory is referenced by a different variable,\nthen silently performing this optimization would affect that memory as well,\nwhich is surprising behavior.\n\nHowever, this optimization remains important. It is directly in line with\nPly's goal of generating code that is as good as the hand-written version. We\njust need a different approach; probably a more explicit one. This could take\nthe form of separate identifiers (e.g. `rfilter`), similar to parallelization.\nBut this leads to an unfortunate bifurcation: what if you want both\nreassignment and parallelization? So now we need four different forms:\nstandard, parallel, reassigned, and parallel reassigned, each with its own\nidentifier. More identifiers means more burden on the programmer, so I'm\nhesistant to implement this approach until I've given it more thought.\n\n**Compile-time evaluation:**\n\nA few functions (currently just `max` and `min`) can be evaluated at compile\ntime if their arguments are also known at compile time. This is similar to how\nthe builtin `len` and `cap` functions work:\n\n```go\nlen([3]int) // known at compile-time; compiles to 3\n\nmax(1, min(2, 3)) // known at compile time; compiles to 2\n```\n\nIn theory, it is also possible to perform compile-time evaluation on certain\nliterals. For example:\n\n```go\n[]int{1, 2, 3}.contains(3) // compile to true?\n```\n\nWe could even go further and support arbitrary compile-time execution. But\nthat seems a little dangerous. At best, it's useful for things like computing\na large table instead of including it in the source. But I don't think that\nsingle case warrants such a powerful feature.\n\n**Function hoisting (planned):**\n\n`not` currently returns a function that wraps its argument. Instead, `not`\ncould generate a new top-level function definition, and replace the callsite\nwholesale. For example, given these definitions:\n\n```go\neven := func(i int) bool { return i % 2 == 0 }\nodd := not(even)\n```\n\nThe compiled code currently looks like this:\n\n```go\nfunc not_int(fn func(int) bool) func(int) bool {\n\treturn func(i int) bool {\n\t\treturn !fn(i)\n\t}\n}\n\neven := func(i int) bool { return i % 2 == 0 }\nodd := not_int(even)\n```\n\nBut we could improve upon this by generating a top-level `not_even` function:\n\n```go\nfunc not_even(i int) bool {\n\treturn !even(i)\n}\n\neven := func(i int) bool { return i % 2 == 0 }\nodd := not_even\n```\n\nThis is non-trivial, though, because `even` is not in the top-level scope; we\nwould need to hoist its definition into the function body of `not_even`.\nAlternatively, we could simply not consider local functions for this\noptimization -- but we'd still need a way to distinguish global functions from\nlocal functions.\n\nThe motivation for this optimization is that the Go compiler is more likely to\ninline top-level functions (AFAIK). Eliminating the overhead of a function\ncall could be significant when, say, filtering a large slice. Benchmarks are\nneeded to confirm that this would actually result in a significant speedup.\n\n\nFAQ\n---\n\n**Why wouldn't you just use [existing generics solution]?**\n\nThere are basically two options: runtime generics (via reflection) and\ncompile-time generics (via codegen). They both suck for different reasons:\nreflection is slow, and codegen is cumbersome. Ply is an attempt at making\ncodegen suck a bit less. You don't need to grapple with magic annotations or\ncustom types; you can just start using `filter` and `fold` as though Go had\nalways supported them.\n\n**What are the downsides of this approach?**\n\nThe most obvious is that it's less flexible; you can only use the functions\nand methods that Ply provides. Another annoyance is that since they behave\nlike builtins, you can't pass them around as first-class values. Fortunately\nthis is a pretty rare thing to do, and it's possible to work around it in most\ncases. (For example, you can wrap the call in a `func`.)\n\nGenerating a specific implementation of every generic function call produces\nvery fast code, at the cost of slower compilation, larger binaries, and less\nhelpful error messages. Your build process will also be more complicated,\nthough hopefully not as complicated as writing template code and using `go\ngenerate`. The fact of the matter is that *there is no silver bullet*: every\nimplementation of generics has its downsides. Do your research before deciding\nwhether Ply is the right approach for your project.\n\n**What if I want to define my own generic functions?**\n\nSorry, that's not in the cards. The purpose of Ply is to make polymorphism as\npainless as possible. Supporting custom generics would mean defining some kind\nof template syntax, and that adds a lot of complexity to the language.\nRestricting the set of generic functions also allows the Ply compiler to apply\ndeep optimizations, such as pipelining.\n\nI understand that this is a controversial position, and Ply's set of functions\nmay not suit everyone's needs. My rationale is that by adding a small set of\nnew functions, Go can be made much more productive without becoming any harder\nto parse (by computer or by human). If you have suggestions for new functions,\n[open an issue](https://github.com/lukechampine/ply/issues) and I'll consider\nadding them.\n\n**What about generic data structures?**\n\nGo seems pretty productive without them. Slices and maps are sufficient for\nthe vast majority of programs. Adding new generic data structures would\ncomplicate Go's syntax (do we overload `make` for our new `RedBlackTree`\ntype?) and I really want to avoid that. Go's simplicity is one of its biggest\nstrengths.\n\n**How does Ply interact with the existing Go toolchain?**\n\nOne nice thing about Ply is that because it has the same syntax as Go, many\ntools built for Go will \"just work\" with Ply. For example, you can run `gofmt`\nand `golint` on `.ply` files. Other tools (like `go vet`) are pickier about\ntheir input filenames ending in `.go`, but will work if you rename your `.ply`\nfiles. Lastly, tools that require type information will fail, because Go's\ntype-checker does not understand Ply builtins.\n\nOne current deficiency is that Ply will not automatically compile imported\n`.ply` files. So you can't write pure-Ply packages (yet).\n\n**Will you add support for feature X?**\n\nOpen an issue and I will gladly consider it.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flukechampine%2Fply","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flukechampine%2Fply","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flukechampine%2Fply/lists"}