Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/uber-go/gopatch

Refactoring and code transformation tool for Go.
https://github.com/uber-go/gopatch
go golang refactoring
Last synced: 12 days ago
JSON representation
Refactoring and code transformation tool for Go.
Host: GitHub
URL: https://github.com/uber-go/gopatch
Owner: uber-go
License: mit
Created: 2020-11-30T13:37:43.000Z (almost 4 years ago)
Default Branch: main
Last Pushed: 2024-07-23T17:36:07.000Z (4 months ago)
Last Synced: 2024-08-01T22:50:21.775Z (3 months ago)
Topics: go, golang, refactoring
Language: Go
Homepage:
Size: 677 KB
Stars: 860
Watchers: 13
Forks: 32
Open Issues: 23
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project

my-awesome - uber-go/gopatch - 10 star:0.9k fork:0.0k Refactoring and code transformation tool for Go. (Go)
README

        # gopatch [![Go](https://github.com/uber-go/gopatch/actions/workflows/go.yml/badge.svg)](https://github.com/uber-go/gopatch/actions/workflows/go.yml) [![codecov](https://codecov.io/gh/uber-go/gopatch/branch/main/graph/badge.svg?token=tFsx23GSTB)](https://codecov.io/gh/uber-go/gopatch)

gopatch is a tool to match and transform Go code. It is meant to aid in

refactoring and restyling.

# Table of contents

- [Introduction](#introduction)

- [Getting started](#getting-started)

  - [Installation](#installation)

  - [Your first patch](#your-first-patch)

  - [Apply the patch](#apply-the-patch)

  - [Next steps](#next-steps)

- [Usage](#usage)

  - [Options](#options)

- [Patches](#patches)

  - [Metavariables](#metavariables)

  - [Statements](#statements)

  - [Elision](#elision)

  - [Comments](#comments)

    - [Description comments](#description-comments)

    - [Usage in diff mode](#usage-during-diff-mode)

- [Examples](#examples)

- [Project status](#project-status)

  - [Goals](#goals)

  - [Known issues](#known-issues)

  - [Upcoming](#upcoming)

- [Similar Projects](#similar-projects)

- [Credits](#credits)

# Introduction

gopatch operates like the Unix `patch` tool: given a patch file and another

file as input, it applies the changes specified in the patch to the provided

file.

```

 .-------.                      .-------.

/_|      |.                    /_|      |.

|        ||.    +---------+    |        ||.

|   .go  |||>-->| gopatch |>-->|   .go  |||

|        |||    +---------+    |        |||

'--------'||      ^            '--------'||

 '--------'|      |             '--------'|

  '--------'      |              '--------'

     .-------.    |

    /_|      |    |

    |        +----'

    | .patch |

    |        |

    '--------'

```

What specifically differentiates it from `patch` is that unlike plain text

transformations, it can be smarter because it understands Go syntax.

# Getting started

## Installation

Download a **pre-built binary** of gopatch from the [Releases page] or by

running the following command in your terminal and place it on your `$PATH`.

  [Releases page]: https://github.com/uber-go/gopatch/releases

```bash

VERSION=0.4.0

URL="https://github.com/uber-go/gopatch/releases/download/v$VERSION/gopatch_${VERSION}_$(uname -s)_$(uname -m).tar.gz"

curl -L "$URL" | tar xzv gopatch

```

Alternatively, if you have Go installed, **build it from source** and install

it with the following command.

```bash

go install github.com/uber-go/gopatch@latest

```

Note: If you're using Go < 1.16, use `go get github.com/uber-go/gopatch@latest` instead.

## Your first patch

Write your first patch.

```shell

$ cat > ~/s1028.patch

# Replace redundant fmt.Sprintf with fmt.Errorf

@@

@@

-import "errors"

-errors.New(fmt.Sprintf(...))

+fmt.Errorf(...)

```

This patch is a fix for staticcheck [S1028]. It searches for uses of

[`fmt.Sprintf`] with [`errors.New`], and simplifies them by replacing them

with [`fmt.Errorf`].

  [S1028]: https://staticcheck.io/docs/checks#S1028

  [`fmt.Sprintf`]: https://golang.org/pkg/fmt/#Sprintf

  [`errors.New`]: https://golang.org/pkg/errors/#New

  [`fmt.Errorf`]: https://golang.org/pkg/fmt/#Errorf

For example,

```go

return errors.New(fmt.Sprintf("invalid port: %v", err))

// becomes

return fmt.Errorf("invalid port: %v", err)

```

## Apply the patch

-  `cd` to your Go project's directory.

  ```shell

  $ cd ~/go/src/example.com/myproject

  ```

  Run `gopatch` on the project, supplying the previously written patch with the

  `-p` flag.

  ```shell

  $ gopatch -p ~/s1028.patch ./...

  ```

  This will apply the patch on all Go code in your project.

  Check if there were any instances of this issue in your code by running

  `git diff`.

- Instead, `cd` to your Go project's directory.

  ```shell

  $ cd ~/go/src/example.com/myproject

  ```

  Run `gopatch` on the project, supplying the previously written patch with the

  `-p` flag along with '-d' flag.

  ```shell

  $ gopatch -d -p ~/s1028.patch ./...

  ```

  This will turn on diff mode and will write the diff to stdout instead of modifying all

  the Go code in your project. To provide more context on what the patch does, if

  there were description comments in the patch, they will also get displayed at 

  the top. To learn more about description comments jump to section [here](#description-comments)

  

  For example if we applied patch ~/s1028 to our testfile error.go

  ```shell

  $ gopatch -d -p ~/s1028.patch ./testdata/test_files/diff_example/

  ```

  Output would be : 

  ```

  gopatch/testdata/test_files/diff_example/error.go:Replace redundant fmt.Sprintf with fmt.Errorf

  --- gopatch/testdata/test_files/diff_example/error.go

  +++ gopatch/testdata/test_files/diff_example/error.go

  @@ -7,7 +7,7 @@

 

  func foo() error {

          err := errors.New("test")

  -       return errors.New(fmt.Sprintf("error: %v", err))

  +       return fmt.Errorf("error: %v", err)

  }

 

   func main() {

  ```

  Note: Only the description comments of patches that actually **apply** are displayed.

## Next steps

To learn how to write your own patches, move on to the [Patches] section. To

dive deeper into patches, check out [Patches in depth].

  [Patches in depth]: docs/PatchesInDepth.md

To experiment with other sample patches, check out the [Examples] section.

  [Patches]: #patches

  [Examples]: #examples

# Usage

To use the gopatch command line tool, provide the following arguments.

```

gopatch [options] pattern ...

```

Where pattern specifies one or more Go files, or directories containing Go

files. For directories, all Go code inside them and their descendants will be

considered by gopatch.

## Options

gopatch supports the following command line options.

- `-p file`, `--patch=file`

    Path to a patch file specifying a transformation. Read more about the

    patch file format in [Patches].

    Provide this flag multiple times to apply multiple patches in-order.

    ```shell

    $ gopatch -p foo.patch -p bar.patch path/to/my/project

    ```

    If this flag is omitted, a patch is expected on stdin.

    ```shell

    $ gopatch path/to/my/project << EOF

    @@

    @@

    -foo

    +bar

    EOF

    ```

- `-d`, `--diff`

    Flag to turn on diff mode. Provide this flag to write the diff to stdout instead

    of modifying the file and display applied patches' [description comments](#description-comments) if they exist. 

    Use in conjunction with -p to provide patch file.

    

    Only need to apply the flag once to turn on diff mode

    ```shell

    $ gopatch -d -p foo.patch -p bar.patch path/to/my/project

    ```

    If this flag is omitted, normal patching occurs which modifies the

    file instead.

- `--print-only`

  

  Flag to turn on print-only mode. Provide this flag to write the changed code to stdout instead of modifying the

  file and display applied patches' description comments to stderr if they exist.

  

    ```shell

    $ gopatch --print-only -p foo.patch -p bar.patch path/to/my/project

    ```  

- `--skip-import-processing`

  

  Flag to turn on skip-import-processing mode. Provide this flag to disable

  import formatting for imports that were not part of the patch changes.

    ```shell

    $ gopatch --skip-import-processing -p foo.patch -p bar.patch path/to/my/project

    ```

- `--skip-generated`

  

  Flag to turn on skip-generated code mode. Provide this flag to skip running the

  tool on generated code. A file is considered containing generated code if it 

  has `@generated` or `^// Code generated .* DO NOT EDIT\.$` in the comment

  header.

    ```shell

    $ gopatch --skip-generated -p foo.patch -p bar.patch path/to/my/project

    ```

# Patches

Patch files are the input to gopatch that specify how to transform code. Each

patch file contains one or more patches. This section provides an introduction

to writing patches; look at [Patches in depth] for a more detailed

explanation.

Each patch specifies a code transformation. These are formatted like unified

diffs: lines prefixed with `-` specify matching code should be deleted, and

lines prefixed with `+` specify that new code should be added.

Consider the following patch.

```diff

@@

@@

-foo

+bar

```

It specifies that we want to search for references to the identifier `foo` and

replace them with references to `bar`. (Ignore the lines with `@@` for now.

We will cover those below.)

A more selective version of this patch will search for uses of `foo` where it

is called as a function with specific arguments.

```diff

@@

@@

-foo(42)

+bar(42)

```

This will search for invocations of `foo` as a function with the specified

argument, and replace only those with `bar`.

gopatch understands Go syntax, so the above is equivalent to the following.

```diff

@@

@@

-foo(

+bar(

  42,

 )

```

## Metavariables

Searching for hard-coded exact parameters is limited. We should be able to

generalize our patches.

The previously ignored `@@` section of patches is referred to as the

**metavariable section**. That is where we specify **metavariables** for the

patch.

Metavariables will match any code, to be reproduced later. Think of them like

holes to be filled by the code we match. For example,

```diff

@@

var x expression

@@

# rest of the patch

```

This specifies that `x` should match any Go expression and record its match

for later reuse.

> **What is a Go expression?**

>

> Expressions usually refer to code that has value. You can pass these as

> arguments to functions. These include `x`, `foo()`, `user.Name`, etc.

>

> Check the [Identifiers vs expressions vs statements] section of the appendix

> for more.

  [Identifiers vs expressions vs statements]: docs/Appendix.md#identifiers-vs-expressions-vs-statements

So the following patch will search for invocations of `foo` with a single

argument---any argument---and replace them with invocations of `bar` with the

same argument.

```diff

@@

var x expression

@@

-foo(x)

+bar(x)

```

| Input              | Output             |

|--------------------|--------------------|

| `foo(42)`          | `bar(42)`          |

| `foo(answer)`      | `bar(answer)`      |

| `foo(getAnswer())` | `bar(getAnswer())` |

Metavariables hold the entire matched value, so we can add code around them

without risk of breaking anything.

```diff

@@

var x expression

@@

-foo(x)

+bar(x + 3, true)

```

| Input              | Output                       |

|--------------------|------------------------------|

| `foo(42)`          | `bar(42 + 3, true)`          |

| `foo(answer)`      | `bar(answer + 3, true)`      |

| `foo(getAnswer())` | `bar(getAnswer() + 3, true)` |

For more on metavariables see [Patches in depth/Metavariables].

  [Patches in depth/Metavariables]: docs/PatchesInDepth.md#metavariables

## Statements

gopatch patches are not limited to transforming basic expressions. You can

also transform statements.

> **What is a Go statements?**

>

> Statements are instructions to do things, and do not have value. They cannot

> be passed as parameters to other functions. These include assignments

> (`foo := bar()`), if statements (`if foo { bar() }`), variable declarations

> (`var foo Bar`), and so on.

>

> Check the [Identifiers vs expressions vs statements] section of the appendix

> for more.

For example, consider the following patch.

```diff

@@

var f expression

var err identifier

@@

-err = f

-if err != nil {

+if err := f; err != nil {

   return err

 }

```

The patch declares two metavariables:

- `f`: This represents an operation that possibly returns an `error`

- `err`: This represents the name of the `error` variable

The patch will search for code that assigns to an error variable immediately

before returning it, and inlines the assignment into the `if` statement. This

effectively [reduces the scope of the variable] to just the `if` statement.

  [reduces the scope of the variable]: https://github.com/uber-go/guide/blob/master/style.md#reduce-scope-of-variables

InputOutput

```go

err = foo(bar, baz)

if err != nil {

   return err

}

```

```go

if err := foo(bar, baz); err != nil {

   return err

}

```

```go

err = comment.Submit(ctx)

if err != nil {

  return err

}

```

```go

if err := comment.Submit(ctx); err != nil {

  return err

}

```

For more on transforming statements, see [Patches In Depth/Statements].

  [Patches In Depth/Statements]: docs/PatchesInDepth.md#statements

## Elision

Matching a single argument is still too selective and we may want to match a

wider criteria.

For this, gopatch supports **elision** of code by adding `...` in many places.

For example,

```diff

@@

@@

-foo(...)

+bar(...)

```

The patch above looks for all calls to the function `foo` and replaces them

with calls to the function `bar`, regardless of the number of arguments they

have.

| Input                      | Output                     |

|----------------------------|----------------------------|

| `foo(42)`                  | `bar(42)`                  |

| `foo(42, true, 1)`         | `bar(42, true, 1)`         |

| `foo(getAnswer(), x(y()))` | `bar(getAnswer(), x(y()))` |

Going back to the patch from [Statements], we can instead write the following

patch.

  [Statements]: #statements

```diff

@@

var f expression

var err identifier

@@

-err = f

-if err != nil {

+if err := f; err != nil {

   return ..., err

 }

```

This patch is almost exactly the same as before except the `return` statement

was changed to `return ..., err`. This will allow the patch to operate even on

functions that return multiple values.

InputOutput

```go

err = foo()

if err != nil {

   return false, err

}

```

```go

if err := foo(); err != nil {

   return false, err

}

```

For more on elision, see [Patches in depth/Elision].

  [Patches in depth/Elision]: docs/PatchesInDepth.md#elision

## Comments

Patches come with comments to give more context about what they do.

Comments are prefixed by '#'

For example:

```

# Replace time.Now().Sub(x) with time.Since(x)

@@

# var x is in the metavariable section 

var x identifier

@@

-time.Now().Sub(x)

+time.Since(x)

# We replace time.Now().Sub(x)

# with time.Since(x)

```

#### Description comments

Description comments are comments that appear directly above a patch's first

`@@` line.

gopatch will record these descriptions and display them to users with use of

the `--diff` or `--print-only` flags.

For example,

```

# Replace time.Now().Sub(x) with time.Since(x)

@@

# Not a description comment

var x identifier

@@

-time.Now().Sub(x)

+time.Since(x)

# Not a description comment

# Not a description comment

```

Patch files with multiple patches can have a separate description for each

patch.

```

# Replace redundant fmt.Sprintf with fmt.Errorf

@@

@@

-import "errors"

-errors.New(fmt.Sprintf(...))

+fmt.Errorf(...)

# Replace time.Now().Sub(x) with time.Since(x)

@@

var x identifier

@@

-time.Now().Sub(x)

+time.Since(x)

# Not a description comment

```

As these are messages that will be printed to users of the patch,

we recommend the following best practices for description comments.

- Keep them short and on a single-line

- Use imperative mood ("replace X with Y", not "replaces X with Y")

#### Usage with `--diff`

When diff mode is turned on by the `-d`/`--diff` flag, gopatch will print

description comments for patches that matched different files to stderr.

```shell

$ gopatch -d -p ~/s1028.patch testdata/test_files/diff_example/error.go

error.go:Replace redundant fmt.Sprintf with fmt.Errorf

--- error.go

+++ error.go

@@ -7,7 +7,7 @@

func foo() error {

        err := errors.New("test")

-       return errors.New(fmt.Sprintf("error: %v", err))

+       return fmt.Errorf("error: %v", err)

}

 func main() {

```

Note that gopatch will print only the description comments in diff mode.

Other comments will be ignored.

# Examples

This section lists various example patches you can try in your code.

Note that some of these patches are not perfect and may have false positives.

- [s1012.patch](examples/s1012.patch): Fix for staticcheck [S1012](https://staticcheck.io/docs/checks#S1012).

- [s1028.patch](examples/s1028.patch): Fix for staticcheck [S1028](https://staticcheck.io/docs/checks#S1028).

- [s1038.patch](examples/s1038.patch): Fix for staticcheck [S1038](https://staticcheck.io/docs/checks#S1038).

- [gomock-v1.5.0.patch](examples/gomock-v1.5.0.patch): Drops unnecessary call to `Finish` method for users of gomock.

- [destutter.patch](examples/destutter.patch): Demonstrates renaming a type and updating its consumers.

# Project status

The project is currently is in a beta state. It works but significant features

are planned that may result in breaking changes to the patch format.

## Goals

gopatch aims to be a generic power tool that you can use in lieu of simple

search-and-replace.

gopatch will attempt to do 80% of the work for you in a transformation, but it

cannot guarantee 100% correctness or completeness. Part of this is owing to

the decision that gopatch must be able to operate on code that doesn't yet

compile, which can often be the case in the middle of a refactor. We may add

features in the future that require compilable code, but we plan to always

support transformation of partially-valid Go code.

## Known issues

Beyond the known issues highlighted above, there are a handful of other issues

with using gopatch today.

- It's very quiet, so there's no indication of progress. [#7]

- Error messages for invalid patch files are hard to decipher. [#8]

- Matching elisions between the `-` and `+` sections does not always work in a

  desirable way. We may consider replacing anonymous `...` elision with a

  different named elision syntax to address this issue. [#9]

- When elision is used, gopatch stops replacing after the first instance in

  the given scope which is often not what you want. [#10]

- Formatting of output generated by gopatch isn't always perfect.

  [#7]: https://github.com/uber-go/gopatch/issues/7

  [#8]: https://github.com/uber-go/gopatch/issues/8

  [#9]: https://github.com/uber-go/gopatch/issues/9

  [#10]: https://github.com/uber-go/gopatch/issues/10

## Upcoming

Besides addressing the various limitations and issues we've already mentioned,

we have a number of features planned for gopatch.

- Contextual matching: match context (like a function declaration), and then

  run a transformation inside the function body repeatedly, at any depth. [#11]

- Collateral changes: Match and capture values in one patch, and use those in

  a following patch in the same file.

- Metavariable constraints: Specify constraints on metavariables, e.g.

  matching a string, or part of another metavariable.

- Condition elision: An elision should match only if a specified condition is

  also true.

  [#11]: https://github.com/uber-go/gopatch/issues/11

# Contributing

If you'd like to contribute to gopatch, you may find the following documents

useful:

- [HACKING](docs/HACKING.md) documents the architecture, code organization, and

  other information necessary to contribute to the project.

- [RELEASE](docs/RELEASE.md) documents the process for releasing a new version

  of gopatch.

# Similar Projects

- [rf] is a refactoring tool with a custom DSL

- [gofmt rewrite rules] support simple transformations on expressions

- [eg] supports basic example-based refactoring

- [Coccinelle] is a tool for C from which gopatch takes inspiration heavily

- [Semgrep] is a cross-language semantic search tool

- [Comby] is a language-agnostic search and transformation tool

  [gofmt rewrite rules]: https://golang.org/cmd/gofmt/

  [eg]: https://godoc.org/golang.org/x/tools/cmd/eg

  [Coccinelle]: https://coccinelle.gitlabpages.inria.fr/website/

  [Semgrep]: https://semgrep.dev/

  [Comby]: https://comby.dev/

  [rf]: https://github.com/rsc/rf

# Credits

gopatch is heavily inspired by [Coccinelle].