{"id":17127417,"url":"https://github.com/tnir/gitlab-workhorse","last_synced_at":"2025-04-06T03:13:06.119Z","repository":{"id":40281340,"uuid":"296168244","full_name":"tnir/gitlab-workhorse","owner":"tnir","description":null,"archived":false,"fork":false,"pushed_at":"2023-02-25T00:49:54.000Z","size":24055,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-12T08:58:15.040Z","etag":null,"topics":["gitlab","gitlab-workhorse","hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tnir.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-16T23:20:27.000Z","updated_at":"2020-10-04T05:56:36.000Z","dependencies_parsed_at":"2024-06-21T08:34:52.064Z","dependency_job_id":"b9afb93d-e459-459e-a394-507517577ec9","html_url":"https://github.com/tnir/gitlab-workhorse","commit_stats":{"total_commits":1063,"total_committers":78,"mean_commits":"13.628205128205128","dds":0.702728127939793,"last_synced_commit":"c8575e0b2e193464c76993b1abb9fbc846b38759"},"previous_names":[],"tags_count":211,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tnir%2Fgitlab-workhorse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tnir%2Fgitlab-workhorse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tnir%2Fgitlab-workhorse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tnir%2Fgitlab-workhorse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tnir","download_url":"https://codeload.github.com/tnir/gitlab-workhorse/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247427012,"owners_count":20937214,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gitlab","gitlab-workhorse","hacktoberfest"],"created_at":"2024-10-14T19:04:37.579Z","updated_at":"2025-04-06T03:13:06.088Z","avatar_url":"https://github.com/tnir.png","language":"Go","readme":"# gitlab-workhorse\n\nGitlab-workhorse is a smart reverse proxy for GitLab. It handles\n\"large\" HTTP requests such as file downloads, file uploads, Git\npush/pull and Git archive downloads.\n\n## Quick facts (how does Workhorse work)\n\n-   Workhorse can handle some requests without involving Rails at all:\n    for example, JavaScript files and CSS files are served straight\n    from disk.\n-   Workhorse can modify responses sent by Rails: for example if you use\n    `send_file` in Rails then gitlab-workhorse will open the file on\n    disk and send its contents as the response body to the client.\n-   Workhorse can take over requests after asking permission from Rails.\n    Example: handling `git clone`.\n-   Workhorse can modify requests before passing them to Rails. Example:\n    when handling a Git LFS upload Workhorse first asks permission from\n    Rails, then it stores the request body in a tempfile, then it sends\n    a modified request containing the tempfile path to Rails.\n-   Workhorse can manage long-lived WebSocket connections for Rails.\n    Example: handling the terminal websocket for environments.\n-   Workhorse does not connect to Postgres, only to Rails and (optionally) Redis.\n-   We assume that all requests that reach Workhorse pass through an\n    upstream proxy such as NGINX or Apache first.\n-   Workhorse does not accept HTTPS connections.\n-   Workhorse does not clean up idle client connections.\n-   We assume that all requests to Rails pass through Workhorse.\n\nFor more information see ['A brief history of\ngitlab-workhorse'][brief-history-blog].\n\n## Usage\n\n```\n  gitlab-workhorse [OPTIONS]\n\nOptions:\n  -apiCiLongPollingDuration duration\n      Long polling duration for job requesting for runners (default 50s - enabled) (default 50ns)\n  -apiLimit uint\n      Number of API requests allowed at single time\n  -apiQueueDuration duration\n      Maximum queueing duration of requests (default 30s)\n  -apiQueueLimit uint\n      Number of API requests allowed to be queued\n  -authBackend string\n      Authentication/authorization backend (default \"http://localhost:8080\")\n  -authSocket string\n      Optional: Unix domain socket to dial authBackend at\n  -cableBackend string\n      Optional: ActionCable backend (default authBackend)\n  -cableSocket string\n      Optional: Unix domain socket to dial cableBackend at (default authSocket)\n  -config string\n      TOML file to load config from\n  -developmentMode\n      Allow the assets to be served from Rails app\n  -documentRoot string\n      Path to static files content (default \"public\")\n  -listenAddr string\n      Listen address for HTTP server (default \"localhost:8181\")\n  -listenNetwork string\n      Listen 'network' (tcp, tcp4, tcp6, unix) (default \"tcp\")\n  -listenUmask int\n      Umask for Unix socket\n  -logFile string\n      Log file location\n  -logFormat string\n      Log format to use defaults to text (text, json, structured, none) (default \"text\")\n  -pprofListenAddr string\n      pprof listening address, e.g. 'localhost:6060'\n  -prometheusListenAddr string\n      Prometheus listening address, e.g. 'localhost:9229'\n  -proxyHeadersTimeout duration\n      How long to wait for response headers when proxying the request (default 5m0s)\n  -secretPath string\n      File with secret key to authenticate with authBackend (default \"./.gitlab_workhorse_secret\")\n  -version\n      Print version and exit\n```\n\nThe 'auth backend' refers to the GitLab Rails application. The name is\na holdover from when gitlab-workhorse only handled Git push/pull over\nHTTP.\n\nGitlab-workhorse can listen on either a TCP or a Unix domain socket. It\ncan also open a second listening TCP listening socket with the Go\n[net/http/pprof profiler server](http://golang.org/pkg/net/http/pprof/).\n\nGitlab-workhorse can listen on redis events (currently only builds/register\nfor runners). This requires you to pass a valid TOML config file via\n`-config` flag.\nFor regular setups it only requires the following (replacing the string\nwith the actual socket)\n\n### Redis\n\nGitlab-workhorse integrates with Redis to do long polling for CI build\nrequests. This is configured via two things:\n\n-   Redis settings in the TOML config file\n-   The `-apiCiLongPollingDuration` command line flag to control polling\n    behavior for CI build requests\n\nIt is OK to enable Redis in the config file but to leave CI polling\ndisabled; this just results in an idle Redis pubsub connection. The\nopposite is not possible: CI long polling requires a correct Redis\nconfiguration.\n\nBelow we discuss the options for the `[redis]` section in the config\nfile.\n\n```\n[redis]\nURL = \"unix:///var/run/gitlab/redis.sock\"\nPassword = \"my_awesome_password\"\nSentinel = [ \"tcp://sentinel1:23456\", \"tcp://sentinel2:23456\" ]\nSentinelMaster = \"mymaster\"\n```\n\n- `URL` takes a string in the format `unix://path/to/redis.sock` or\n`tcp://host:port`.\n- `Password` is only required if your redis instance is password-protected\n- `Sentinel` is used if you are using Sentinel.\n  *NOTE* that if both `Sentinel` and `URL` are given, only `Sentinel` will be used\n\nOptional fields are as follows:\n```\n[redis]\nDB = 0\nReadTimeout = \"1s\"\nKeepAlivePeriod = \"5m\"\nMaxIdle = 1\nMaxActive = 1\n```\n\n- `DB` is the Database to connect to. Defaults to `0`\n- `ReadTimeout` is how long a redis read-command can take. Defaults to `1s`\n- `KeepAlivePeriod` is how long the redis connection is to be kept alive without anything flowing through it. Defaults to `5m`\n- `MaxIdle` is how many idle connections can be in the redis-pool at once. Defaults to 1\n- `MaxActive` is how many connections the pool can keep. Defaults to 1\n\n### Relative URL support\n\nIf you are mounting GitLab at a relative URL, e.g.\n`example.com/gitlab`, then you should also use this relative URL in\nthe `authBackend` setting:\n\n```\ngitlab-workhorse -authBackend http://localhost:8080/gitlab\n```\n\n### Interaction of authBackend and authSocket\n\nThe interaction between `authBackend` and `authSocket` can be a bit\nconfusing. It comes down to: if `authSocket` is set it overrides the\n_host_ part of `authBackend` but not the relative path.\n\nIn table form:\n\n|authBackend|authSocket|Workhorse connects to?|Rails relative URL|\n|---|---|---|---|\n|unset|unset|`localhost:8080`|`/`|\n|`http://localhost:3000`|unset|`localhost:3000`|`/`|\n|`http://localhost:3000/gitlab`|unset|`localhost:3000`|`/gitlab`|\n|unset|`/path/to/socket`|`/path/to/socket`|`/`|\n|`http://localhost:3000`|`/path/to/socket`|`/path/to/socket`|`/`|\n|`http://localhost:3000/gitlab`|`/path/to/socket`|`/path/to/socket`|`/gitlab`|\n\nThe same applies to `cableBackend` and `cableSocket`.\n\n## Installation\n\nTo install gitlab-workhorse you need [Go 1.8 or\nnewer](https://golang.org/dl) and [GNU\nMake](https://www.gnu.org/software/make/).\n\nTo install into `/usr/local/bin` run `make install`.\n\n```\nmake install\n```\n\nTo install into `/foo/bin` set the PREFIX variable.\n\n```\nmake install PREFIX=/foo\n```\n\nOn some operating systems, such as FreeBSD, you may have to use\n`gmake` instead of `make`.\n\n## Dependencies\n\n### Exiftool\n\nWorkhorse uses [exiftool](https://www.sno.phy.queensu.ca/~phil/exiftool/) for\nremoving EXIF data (which may contain sensitive information) from uploaded\nimages. If you installed GitLab:\n\n-   Using the Omnibus package, you're all set.\n    *NOTE* that if you are using CentOS Minimal, you may need to install `perl`\n    package: `yum install perl`\n-   From source, make sure `exiftool` is installed:\n\n    ```sh\n    # Debian/Ubuntu\n    sudo apt-get install libimage-exiftool-perl\n\n    # RHEL/CentOS\n    sudo yum install perl-Image-ExifTool\n    ```\n\n### GraphicsMagick (**experimental**)\n\nWorkhorse has an experimental feature that allows us to rescale images on-the-fly.\nIf you do not run Workhorse in a container where the `gm` tool is already installed,\nyou will have to install it on your host machine instead:\n\n#### macOS\n\n```sh\nbrew install graphicsmagick\n```\n\n#### Debian/Ubuntu\n\n```sh\nsudo apt-get install graphicsmagick\n```\n\nFor installation on other platforms, please consult http://www.graphicsmagick.org/README.html.\n\nNote that Omnibus containers already come with `gm` installed.\n\n## Error tracking\n\nGitLab-Workhorse supports remote error tracking with\n[Sentry](https://sentry.io). To enable this feature set the\n`GITLAB_WORKHORSE_SENTRY_DSN` environment variable.\nYou can also set the `GITLAB_WORKHORSE_SENTRY_ENVIRONMENT` environment variable to\nuse the Sentry environment functionality to separate staging, production and\ndevelopment.\n\nOmnibus (`/etc/gitlab/gitlab.rb`):\n\n```\ngitlab_workhorse['env'] = {\n    'GITLAB_WORKHORSE_SENTRY_DSN' =\u003e 'https://foobar'\n    'GITLAB_WORKHORSE_SENTRY_ENVIRONMENT' =\u003e 'production'\n}\n```\n\nSource installations (`/etc/default/gitlab`):\n\n```\nexport GITLAB_WORKHORSE_SENTRY_DSN='https://foobar'\nexport GITLAB_WORKHORSE_SENTRY_ENVIRONMENT='production'\n```\n\n## Tests\n\nRun the tests with:\n\n```\nmake clean test\n```\n\n### Coverage / what to test\n\nEach feature in gitlab-workhorse should have an integration test that\nverifies that the feature 'kicks in' on the right requests and leaves\nother requests unaffected. It is better to also have package-level tests\nfor specific behavior but the high-level integration tests should have\nthe first priority during development.\n\nIt is OK if a feature is only covered by integration tests.\n\n## Distributed Tracing\n\nWorkhorse supports distributed tracing through [LabKit][] using [OpenTracing APIs](https://opentracing.io).\n\nBy default, no tracing implementation is linked into the binary, but different OpenTracing providers can be linked in using [build tags][build-tags]/[build constraints][build-tags]. This can be done by setting the `BUILD_TAGS` make variable.\n\nFor more details of the supported providers, see LabKit, but as an example, for Jaeger tracing support, include the tags: `BUILD_TAGS=\"tracer_static tracer_static_jaeger\"`.\n\n```shell\nmake BUILD_TAGS=\"tracer_static tracer_static_jaeger\"\n```\n\nOnce Workhorse is compiled with an opentracing provider, the tracing configuration is configured via the `GITLAB_TRACING` environment variable.\n\nFor example:\n\n```shell\nGITLAB_TRACING=opentracing://jaeger ./gitlab-workhorse\n```\n\n## Continuous Profiling\n\nWorkhorse supports continuous profiling through [LabKit][] using [Stackdriver Profiler](https://cloud.google.com/profiler).\n\nBy default, the Stackdriver Profiler implementation is linked in the binary using [build tags][build-tags], though it's not\nrequired and can be skipped.\n\nFor example:\n\n```shell\nmake BUILD_TAGS=\"\"\n```\n\nOnce Workhorse is compiled with Continuous Profiling, the profiler configuration can be set via `GITLAB_CONTINUOUS_PROFILING`\nenvironment variable.\n\nFor example:\n\n```shell\nGITLAB_CONTINUOUS_PROFILING=\"stackdriver?service=workhorse\u0026service_version=1.0.1\u0026project_id=test-123 ./gitlab-workhorse\"\n```\n\nMore information about see the [LabKit monitoring docs](https://gitlab.com/gitlab-org/labkit/-/blob/master/monitoring/doc.go).\n\n## License\n\nThis code is distributed under the MIT license, see the [LICENSE](LICENSE) file.\n\n[brief-history-blog]: https://about.gitlab.com/2016/04/12/a-brief-history-of-gitlab-workhorse/\n[LabKit]: https://gitlab.com/gitlab-org/labkit/\n[build-tags]: https://golang.org/pkg/go/build/#hdr-Build_Constraints\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftnir%2Fgitlab-workhorse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftnir%2Fgitlab-workhorse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftnir%2Fgitlab-workhorse/lists"}