Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/kwk/buildbot-app

Your GitHub App to make Buildbot a part of your Pull Request workflow.
https://github.com/kwk/buildbot-app

buildbot ci git llvm

Last synced: 3 months ago
JSON representation

Your GitHub App to make Buildbot a part of your Pull Request workflow.

Awesome Lists containing this project

README 

        
// DO NOT MODIFY THIS DOCUMENT DIRECTLY!

// MODIFY docs/README.in.adoc INSTEAD!

// THEN RUN make docs!

= Your GitHub App to make Buildbot a part of your Pull Request workflow.

Konrad Kleine ;

:description: A study to showcase how GitHub pull request comments and Check Runs can be used in combination with aminimally modified buildbot.

:toc: left

:toclevels: 5

:showtitle:

:experimental:

:sectnums:

:stem:

:sectlinks:

:listing-caption: Listing

:sectanchors:

// :icons: font

:source-highlighter: pygments



// See https://gist.github.com/dcode/0cfbf2699a1fe9b46ff04c41721dda74#admonitions

ifdef::env-github[]

:tip-caption: :bulb:

:note-caption: :information_source:

:important-caption: :heavy_exclamation_mark:

:caution-caption: :fire:

:warning-caption: :warning:

endif::[]



// We always want our images to be displayed directly from github

// See https://gist.github.com/dcode/0cfbf2699a1fe9b46ff04c41721dda74#images

// ifdef::env-github[]

:imagesdir: https://raw.githubusercontent.com/kwk/buildbot-app/main/

// endif::[]



// toc::[]

[IMPORTANT]

====

THIS REPOSITORY IS NO LONGER ACTIVELY MAINTAINED.

====



Let’s build a GitHub App that lets you control your buildbot through

`/buildbot` comments on GitHub Pull Requests.



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/logo/logo-round-small.png[logo

round small]



image:https://codecov.io/gh/kwk/buildbot-app/branch/main/graph/badge.svg?token=ZQ83LL4XLY[CodeCov]



image:https://dl.circleci.com/status-badge/img/gh/kwk/buildbot-app/tree/main.svg?style=svg[CircleCI]



[[_scenarios]]

== Scenarios



This section lists the ideas and sometimes even fully implemented

scenarios we have for this GitHub app.



[[_comment_buildbot_on_pull_request]]

=== Comment `/buildbot` on Pull Request



In this scenario a user authors a Pull Request comment with the comment

body being `/buildbot`.



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/screenshots/author-buildbot-comment.png[author

buildbot comment]



The `buildbot-app` gets notified about a new comment and checks if it

matches a regular expression. This EBNF diagram shows the current

command syntax:



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/command-ebnf.svg[command

ebnf]



For the purpose of this demonstration, `/buildbot` is simply enough.



Internally a command comment will be converted into this structure:



*Command structure (cmd/buildbot-app/command/command.go)*



[source%linesnum,go]

----

// A Command represents all information about a /buildbot command

type Command struct {

    // When true, the command has to pass for the PR in order pass gating

    // (default: true).

    IsMandatory bool

    // Case-sensitive, sorted list of builders without duplicates to run build on.

    // TODO(kwk): Maybe we can default to something reasonable here?

    BuilderNames []string

    // The user's GitHub login that issued the /buildbot comment

    CommentAuthor string

    // When true, we'll try to run the build even if the PR has already been

    // tested at this stage (default: false).

    Force bool

}

----



There’s a regular expression that a string comment must match (see

`StringIsCommand()`) in order for it to be a valid command string:



*Regular Rexpression (cmd/buildbot-app/command/command.go)*



[source%linesnum,go]

----

const (

    // BuildbotCommand is the command that triggers the buildbot workflow in a

    // GitHub comment.

    BuildbotCommand = "/buildbot"



    // CommandOptionMandatory is the boolean option to make a check run

    // mandatory.

    CommandOptionMandatory = "mandatory"



    // CommandOptionBuilder is the option that can be used multiple times in a

    // command comment. The resulting builders will be a case-sensitive, sorted

    // list of builders with no duplicates.

    CommandOptionBuilder = "builder"



    // CommandOptionForce is the boolean option to enforce a new build even if

    // one is already present.

    CommandOptionForce = "force"

)



// StringIsCommand returns true if the given string is a valid /buildbot command.

func StringIsCommand(s string) bool {

    // force=yes|no : Can be used to allow for PRs to be build even when

    // they are closed or when a check run for the exact same SHA has been

    // run already.

    return regexp.MustCompile(buildRegexPattern()).MatchString(s)

}



// buildRegexPattern returns the regex pattern to match a string against a

// /buildbot command

func buildRegexPattern() string {

    tfOptions := `(yes|no|true|false|f|t|y|n|0|1)`

    mandatoryOption := fmt.Sprintf(`%s=%s`, CommandOptionMandatory, tfOptions)

    forceOption := fmt.Sprintf(`%s=%s`, CommandOptionForce, tfOptions)

    builderOption := fmt.Sprintf(`%s=(\w+)`, CommandOptionBuilder)

    return fmt.Sprintf(`^%s(\s+|%s|%s|%s)*$`, BuildbotCommand, mandatoryOption, forceOption, builderOption)

}

----



[[_build_log_comment]]

==== Build Log Comment



The `buildbot-app` then creates a *Thank-you*-comment that serves two

purposes:



[arabic]

. It shows the user that we understood the request and are thankful for

it and that we are working on it.

. It is a perfect placeholder to store short build state changes for

future lookups. That is why we call this comment the

*build-log-comment*.

+

image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/screenshots/build-log-comment.png[build

log comment]

+

Just imagine, your PR gets updated and you want to see the previous

build results. The _build-log-comment_ is there for you too look it up.



The code for creating the comment is straight-forward:



*Thank You! (cmd/buildbot-app/on_issue_comment_event.go)*



[source%linesnum,go]

----

     // This comment will be used all over the place

        thankYouComment := fmt.Sprintf(

            `Thank you @%s for using the %s command here! `,

            command.BuildbotCommand,

            *event.Comment.User.Login,

            *event.Comment.HTMLURL,

        )



        newComment, _, err := gh.Issues.CreateComment(context.Background(), repoOwner, repoName, prNumber, &github.IssueComment{

            Body: github.String(thankYouComment +

                `This very comment will be used to continously log build state changes for your request. We decided to do this in addition to using Github's Check Runs below so you can inspect previous check runs better.`,

            ),

        })

----



[[_check_run]]

==== Check run



Of course, we are also using GitHub’s check runs as you can see here:



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/screenshots/check-run-overview.png[check

run overview]



[NOTE]

====

I really like that we can dynamically create check runs on request and

give them good names.

====



When you click on *Details* next to a check run, you’re brought to this

page on GitHub:



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/screenshots/check-run-details.png[check

run details]



[[_video_walkthrough]]

==== Video walkthrough



We walk you through the creation of a Pull Request and authoring the

`/buildbot` comment in this in this short video:

https://www.youtube.com/watch?v=9NpbKEmkvt8



[[_uml_sequence_diagram]]

==== UML sequence diagram



The sequence diagram for this scenario is layed out here. It includes

some of the internals of the processing.



image:https://raw.githubusercontent.com/kwk/buildbot-app/main/docs/media/on-buildbot-comment.svg[on

buildbot comment]



[[_testing]]

=== Testing



[[_testing_github_interaction]]

==== Testing GitHub interaction



We’re using a fantastic library to run to simulate sequential GitHub

interaction: https://github.com/migueleliasweb/go-github-mock.



For example, when `/buildbot` comment is authored on a pull request we

don’t want a build to run if the pull request is not mergable. Therefore

we first have to take the event input and get the pull request from

GitHub before we check if is mergable:



*Get PR and check mergability

(cmd/buildbot-app/on_issue_comment_event.go)*



[source%linesnum,go]

----

     commentUser := *event.Comment.User.Login

        repoOwner := *event.Repo.Owner.Login

        repoName := *event.Repo.Name

        prNumber := *event.Issue.Number

        pr, _, err := gh.PullRequests.Get(context.Background(), repoOwner, repoName, prNumber)

        if !pr.GetMergeable() {

        }

----



In order to test that a PR is not mergable, we can simply create a valid

`github.PullRequest` object (see `prOK()`) and set the `Mergable` member

to `false`. The mock server will return it as the first request and

afterwards create a `POST` a comment about the pull request not being

mergable:



*Test: Get PR and check mergability

(cmd/buildbot-app/on_issue_comment_event_test.go)*



[source%linesnum,go]

----

func TestOnIssueCommentEventAny(t *testing.T) {

    t.Run("pr not mergable", func(t *testing.T) {

        t.Run("comment writable", func(t *testing.T) {

            prNotMergable := prOK()

            prNotMergable.Mergeable = github.Bool(false)

            srv := NewMockServer(

                // Get PR for comment event

                mock.WithRequestMatch(

                    mock.GetReposPullsByOwnerByRepoByPullNumber,

                    prNotMergable,

                ),

                // Create comment on about PR not being mergable

                mock.WithRequestMatch(

                    mock.PostReposIssuesCommentsByOwnerByRepoByIssueNumber,

                    github.IssueComment{

                        Body: github.String("blabla"),

                    },

                ),

            )

            fn := OnIssueCommentEventAny(srv)

            err := fn("1234", "created", issueCommentEventOK())

            require.ErrorContains(t, err, "pr is not mergable", "expected and error because pr is not mergable, yet")

        })

    })

}

----



For this trick to work we have to use dependency injection by passing a

Go interface (`Server`) instead of a real server object to functions in

various places:



*Server interface (cmd/buildbot-app/server.go)*



[source%linesnum,go]

----

// Server specifies the interface that we need to implement from the AppServer

// object in order to provide a decent mock in tests.

type Server interface {



    // NewGithubClient returns a new GitHub client object for the given

    // application ID.

    NewGithubClient(appInstallationID int64) (*github.Client, error)



    // RunTryBot runs a "buildbot try" command

    RunTryBot(responsibleGithubLogin string, githubRepoOwner string, githubRepoName string, properties ...string) (string, error)

}

----



[[_todos]]

==== TODOs



* ❏ Reset check run to neutral after Pull Request was updated.

* ❏ Deal with buttons shown at the top of check run details page.



[[_developer_setup]]

== Developer Setup



I’m using a Fedora Linux 37 on my local machine and for most of the

containers.



[source,console]

----

$ git clone https://github.com/kwk/buildbot-app.git && cd buildbot-app 

$ sudo dnf install -y direnv golang podman podman-compose buildbot pandoc asciidoctor 

$ gem install asciidoctor-lists pygments.rb 

$ go install github.com/cespare/reflex@latest 

$ cat <> ~/.bashrc 

export PATH=\${PATH}:~/go/bin

eval "\$(direnv hook bash)"

EOF

$ source ~/.bashrc 

$ direnv allow . 

$ make infra-start 

$ make app 

----



* Clone the repo.

* Install tools we need/use for development locally. If this was a

deployment site the only requirement is buildbot so that the github app

can make a call to `buildbot try`.

* Install extension to create list of figures etc. and install pygments

for source code highlighting.

* Install hot-reload tool.

* Make tools above available upon next source of `.bashrc`.

* Reload `.bashrc` to have `direnv` and `reflex` working in your current

shell.

* Navgigate out and back into the project directory to have `direnv`

kickin. If this doesn’t work, try `direnv allow .`.

* Bring up local containers for a buildbot setup with one master and

three workers.

* Run and hot reload the app code upon changes being made to any of your

`*.go` files or your `.envrc` file.



[[_useful_links]]

== Useful links



[[_llvm_links]]

== LLVM links



* Discussion on LLVM Discourse:

https://discourse.llvm.org/t/rfc-prototyping-pre-commit-testing-using-buildbot/69900?u=kwk



[[_github_app_documents]]

== Github App documents



* Github Webhook Events and Payloads:

https://docs.github.com/en/webhooks-and-events/webhooks/webhook-events-and-payloads

* Github Apps documentation: https://docs.github.com/en/apps



[[_interacting_with_github]]

== Interacting with Github



* Forwarding Github Webhooks to your local dev machine:

https://dashboard.ngrok.com/get-started/setup

* Github Emoji Cheat Sheet:

https://github.com/ikatyang/emoji-cheat-sheet/blob/master/README.md



[[_golang_libraries]]

== Golang libraries



* For using Github API v3 from Golang:

https://github.com/google/go-github

* GraphQL Go Library for Github API v4:

https://github.com/shurcooL/githubv4

* For mocking the above repo responses:

https://github.com/migueleliasweb/go-github-mock

* Go web framework: https://github.com/labstack/echo

* For handling github events: https://github.com/cbrgm/githubevents

* For authentication of Github App from private key file:

https://github.com/bradleyfalzon/ghinstallation



[[_buildbot_links]]

== Buildbot links



* System Architecture:

https://docs.buildbot.net/latest/manual/introduction.html#system-architecture

* Custom services (Might be worth looking into):

https://docs.buildbot.net/latest/manual/configuration/services/index.html



[[_misc_links]]

== Misc links



* Recording terminal sessions: https://github.com/faressoft/terminalizer

* For automatic reloading: https://github.com/cespare/reflex

* Per-Directory environment files: https://direnv.net/



[[_lists]]

== Lists



[[_list_of_figures]]

== List of figures



[[_list_of_tables]]

== List of tables



[[_list_of_code_snippets]]

== List of code snippets



link:#cd841b04-3fc5-4388-ba3c-1aca6d7f50c1[Listing 1.] Command structure

(cmd/buildbot-app/command/command.go) +

link:#39ad47a6-8c85-47c8-bf25-f65e425de282[Listing 2.] Regular

Rexpression (cmd/buildbot-app/command/command.go) +

link:#5ac04254-7db6-46f2-b371-cb91339ace8e[Listing 3.] Thank You!

(cmd/buildbot-app/on_issue_comment_event.go) +

link:#bf2bda60-e550-4332-b406-c32431dde101[Listing 4.] Get PR and check

mergability (cmd/buildbot-app/on_issue_comment_event.go) +

link:#fae37627-0f58-4a6a-adb0-d03494b5c305[Listing 5.] Test: Get PR and

check mergability (cmd/buildbot-app/on_issue_comment_event_test.go) +

link:#868c092a-3941-4971-b2f5-d570212c3c31[Listing 6.] Server interface

(cmd/buildbot-app/server.go) +



[[_todo]]

== TODO



* ❏ properly document developer setup with ngrok and how to setup the

`.envrc` file

* ❏ hook into buildbots event system and send feedback to buildbot app

from there?



[[_terminology]]

== Terminology



PR or Pull Request::

  "Pull requests let you tell others about changes you’ve pushed to a

  branch in a repository on GitHub. Once a pull request is opened, you

  can discuss and review the potential changes with collaborators and

  add follow-up commits before your changes are merged into the base

  branch."

   — (https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests[About

  pull requests])

Buildmaster or Buildbot Master::

  "Buildbot consists of a single buildmaster and one or more workers

  that connect to the master. The buildmaster makes all decisions about

  what, when, and how to

  build." — (https://docs.buildbot.net/latest/manual/introduction.html[Buildbot

  System Architecture])

Buildbot Worker::

  "The workers only connect to master and execute whatever commands they

  are instructed to

  execute." — (https://docs.buildbot.net/latest/manual/introduction.html[Buildbot

  System Architecture])

Builder::

  "A builder is a user-configurable description of how to perform a

  build. It defines what steps a new build will have, what workers it

  may run on and a couple of other properties. A builder takes a build

  request which specifies the intention to create a build for specific

  versions of code and produces a build which is a concrete description

  of a build including a list of steps to perform, the worker this needs

  to be performed on and so

  on." — (https://docs.buildbot.net/latest/manual/introduction.html[Buildbot

  System Architecture])

Scheduler::

  "A scheduler is a user-configurable component that decides when to

  start a build. The decision could be based on time, on new code being

  committed or on similar

  events." — (https://docs.buildbot.net/latest/manual/introduction.html[Buildbot

  System Architecture])

Reporters::

  Reporters are user-configurable components that send information about

  started or completed builds to external sources. Buildbot provides its

  own web application to observe this data, so reporters are optional.

  However they can be used to provide up to date build status on

  platforms such as GitHub or sending

  emails. — (https://docs.buildbot.net/latest/manual/introduction.html[Introduction])