Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/awkay/porting-tool

A generic tool for automating arbitrary transforms of Clojure/Clojurescript/CLJC files
https://github.com/awkay/porting-tool

clj cljc cljs clojure clojurescript porting refactoring

Last synced: 3 months ago
JSON representation

A generic tool for automating arbitrary transforms of Clojure/Clojurescript/CLJC files

Awesome Lists containing this project

README 

        
== Porting Tool



WARNING: I've abandoned this effort. I ended up not wanting to 

spend the time refining it (I have enough jobs), and the

approach didn't work very well (the zipper approach was very slow at

runtime). Do not contact me about it, but feel free

to look around or play with it. Don't remember if it compiles on the

current commit...you might have to back up a few.



This is a small library that can parse clj/cljs/cljc files and apply transforms.

It is not a full interpreter for Clojure, so it has limited understanding of the source, but it does understand namespaces and aliasing so that you can do the most common operations necessary for custom source refactorings.



The following transforms are supplied, but it is also easy to write you own:



* Renaming artifact symbols that moved from one ns to another (including adding require with an alias, if necessary).

* Rename namespaces in require clauses.



The tool supports CLJ, CLJS, and CLJC files with conditional reads.



=== Intended Use



The original motivation for this library is porting Fulcro 2 to Fulcro 3.

The new version of that library created new namespaces for all APIs, changed some semantics, and a few API signatures.

The projects that use Fulcro use CLJ, CLJS, and CLJC files, and there were no tools I could find that would make it easy to do what I wanted: write generic transforms against forms where the handling of some of the grunt work was handled for me.



This library is intended to:



* Make it possible for you to work with clj/cljs/cljc in a seamless but separately configurable way.

* Comprehend the namespaces in a file and provide you with that information so you can more easily write transforms.

* Traverse the forms in a contextual way (automatically handling reader conditionals) so that you can concentrate on just the actual code transform.



Ultimately this tools is just meant to be a very advanced form of `sed`:

Run through a file a "fix up" the code.



If you're doing refactoring on your own source, and you're just working on CLJ and using emacs you should have a look at `clj-refactor`.

It does a number of refactorings that may be sufficient, but it doesn't support CLJS/CLJC (https://github.com/clojure-emacs/refactor-nrepl/issues/195).



IntelliJ has a few minor refactorings that are more powerful for the specific things that they do (e.g. renaming a namespaced keyword), since that tool does try to comprehend your source code in a more complete manner.



== Usage



Add the lastest git SHA to your `deps.edn` file.



```

com.fulcrologic/porting-tool {:git/url "https://github.com/fulcrologic/porting-tool.git"

                              :sha "1d58d2d89f1704a29828fedc0f0d2be2f1573f55"}}}

```



Then code a simple project file that can run the processing on files of your choice:



```

(ns my-porting-tool

  (:require

    [com.fulcrologic.porting.core :refer [process-file]]))



(defn config {...}) ; see below



(process-file "sample.cljc" config)

```



At the moment this will output to `*out*` (stdout by default) using `println`

and `pprint`.



=== Configuration



You must supply a configuration that describes what you want done to the file(s) that are processed.

You must understand that the tool needs to know configuration for each feature (language) that will be affected, so the configuration file has a section for each:



```

{:clj {:transforms       [...]

       :let-forms        #{encore/if-let}

       :defn-forms       #{'com.fulcrologic.fulcro.components/defsc 'ghostwheel.core/>defn}

       :namespace->alias {'com.fulcrologic.fulcro.components 'comp

                          'com.fulcrologic.fulcro.dom        'dom

                          'com.fulcrologic.fulcro.dom-server 'dom}}

```



The `:transforms` is a vector of tuples `[predicate transform]` that are to be performed.

The rest of the keys in the config are dictated by these transforms, but many of them can create new requires in the file.

The

`:namespace->alias` map indicates what alias is preferred for new namespaces.



The `:let-forms` and `defn-forms` tell the tool what macros you have that should be seen as let-like or defn-like things, so that potential shadowing and renaming problems can be properly detected.

The tool has a built-in list of these, so you only need to configure them if you have extra ones (and the defaults are always merged into whatever you configure).



==== CLJC



Working with CLJC files is a bit more work.

A CLJC file (for our purposes) really is three things in one: forms for *just* CLJ, forms for *just* CLJS, and forms that are "language agnostic".



When a transform is given a form it will also be told the "feature context".

The transform will use that to figure out which of the configurations to apply to the form it is working on.

The feature contexts are `:clj`, `cljs`, and `:agnostic`; therefore, a configuration for a CLJC transform will commonly look something like this:



```

(let [base {:fqname-old->new  {'fulcro.client.primitives/defsc        'com.fulcrologic.fulcro.components/defsc

                               'fulcro.client.primitives/get-computed 'com.fulcrologic.fulcro.components/get-computed}

            :transforms       [rename/rename-artifacts-transform

                               rename/rename-namespaces-transform

                               rename/add-missing-namespaces-transform]

            :namespace->alias {'com.fulcrologic.fulcro.components 'comp

                               'com.fulcrologic.fulcro.dom        'dom

                               'com.fulcrologic.fulcro.dom-server 'dom}}

      config {:agnostic base

              :cljs     (merge base {:namespace-old->new {'fulcro.client.dom 'com.fulcrologic.fulcro.dom}})

              :clj      (merge base {:namespace-old->new {'fulcro.client.dom-server 'com.fulcrologic.fulcro.dom-server}})}]

    ...)

```



NOTE: the `:transforms` themselves must be configured for each feature of a file, since some transforms may only make sense for a given aspect of the file.



== Transforms



This library's implementation provides the core tools: namespace comprehension and traversal of forms in a language feature-sensitive manner.

This makes is easy to write transforms.



It is important to understand that this tool is *not* a global refactoring tool that could actually move an artifact from one disk file to another.

It is local transforms on *one file at a time* where you can indicate changes you'd like to make to the *forms* within that file.



As such, when we speak of a "rename" we are almost always referring to the "global name" of some artifact (e.g. its fully-qualified name).



=== How the Processing Works



The processing of a file is done in two phases:



- Phase 1:

All forms are traversed.

All transforms are invoked.

A processing environment (`env`) is passed to transforms and will include a `:state-atom` holding a map that transforms can use to "save information" about the things they see.

This is useful for doing things like gathering up a list of namespaces that are used so that the namespace form can be fixed on the second pass.

- Phase 2:

Identical to 1 except the `env` now includes `:state`, which is cumulative result of what was in the `:state-atom` at then end of phase 1.



Each phase does the same steps (some of which have multiple passes):



* Analyzes the ns form for each feature (e.g. :clj, :cljs, etc) that is necessary for the file.

It records what namespaces are required in the file, and what symbols are referred (aliased to simple symbols).

The result of this step becomes the *parsing environment*.

* Forms are traversed recursively, but in a "context sensitive" manner (one pass for each feature of the file).

Transforms only see forms for the a single feature context at a time.

For example if the source had `#?(:clj a :cljs b)`

and you were in the `:clj` context, the transform function would only see `a`, and whatever it returned would only *affect* the CLJ side of the reader conditional.

The `:agnostic` feature pass *skips* reader conditionals altogether.

** `let`-like and `defn`-like forms are analyzed for possible naming confusion, and are used to modify the parsing environment and issue warnings.

Any local symbol bindings will remove conflicting namespace `refer`s, but since

code comprehension is not part of this library's purpose it will just issue warnings when that might result in

a problem with the output.

* Transforms are applied in order for each form.



NOTE: CLJC files require some care.

The :clj, :cljs, *and* :agnostic feature passes will see the same (non-conditional) form.

Ideally, only the agnostic transform would be configured to respond for that form (or all feature configs would be configured identically for it).

A transform *is allowed* to output a Reader Conditional (TODO: document how to do that), which means a transform could convert

something from language agnostic to conditional.



=== The Transform `env`



Your transform processing `env` will include a number of useful things:



`:parsing-envs`:: A map from feature key (e.g. :clj) to the `parsing-env` for the features of the current file.

`:zloc`:: A current rewrite-clj zipper set to the location of the form being processed.

`:config`:: The map from feature to config that you supplied on start.

`:feature-context`:: The current feature being processed.

`:current-ns`:: The name of ns of the file being processed.



Each `parsing-env` will include feature-specific details of the namespace:

`:nsalias->ns`:: A map from namespace aliases to the real namespace (from the `:as` clauses in the requires).

If there is no alias for a ns it will still be listed as itself.

`:ns->alias`:: A reverse of from ns to its alias.

All nses are included (e.g. no alias will have same k as v).

`:raw-sym->fqsym`:: A map from raw symbols to their fully-qualified name (from the `:refer` clauses in the requires)



=== Reporting Problems



Sometimes there is no transform possible and you just need to inform the user that there is a problem.

The

`com.fulcrologic.porting.parsing.util/report-warning!` and

`com.fulcrologic.porting.parsing.util/report-error!` functions should be used for this.

The latter throws an exception to halt processing.

They will include the file and line for you as a prefix to your message.



=== Writing Your Own Transform



See the source of the built-in transforms for some examples of how to write them.



=== Built-in Transforms



=== Function Rename



See the docstring of `com.fulcrologic.porting.transforms.rename/rename-artifacts-transform` for usage.



Say the function `some.lib/f` is moved and renamed to `other.thing/g`:



Your old file might be:



```

(ns my.thing

  (:require

    [some.lib :as lib :refer [f]]))



(lib/f)

(f)

```



and the desired new file would be:



```

(ns my.thing

  (:require

    [other.thing :as thing]))



(thing/g)

(thing/g)

```



=== Adding Missing Namespaces



This transform is a companion of the `rename-artifacts-transform` (which must appear *before* it).



See the docstring of `com.fulcrologic.porting.transforms.rename/add-missing-namespaces` for usage.



=== Renaming Namespaces



Sometimes the only real change is that of the namespace itself.

You could (tediously) list out every single function from the old to the new namespace in the artifact renaming, but in the case of a simple namespace rename this is overkill.



See the docstring of `com.fulcrologic.porting.transforms.rename/rename-namespaces-transform` for usage.



== Limitations



This library is *not* a full compiler, and as such it cannot possibly comprehend your code.

Clojure(script) macros can create bindings that *should* shadow namespace aliases, but this library has limited support for figuring out when shadowing is happening.



If you have a macro that behaves like `defn` or `let` you should configure it as described above.