{"id":36688204,"url":"https://github.com/converged-computing/goshare","last_synced_at":"2026-01-12T11:16:46.948Z","repository":{"id":183618002,"uuid":"670447414","full_name":"converged-computing/goshare","owner":"converged-computing","description":"Testing a producer/consumer model that spans namespaces for flux + application pods!","archived":false,"fork":false,"pushed_at":"2024-01-18T15:53:46.000Z","size":81,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-10T14:50:01.802Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/converged-computing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-25T04:41:12.000Z","updated_at":"2023-07-25T05:00:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"8044a230-d272-459b-99ae-d02843e49dbc","html_url":"https://github.com/converged-computing/goshare","commit_stats":null,"previous_names":["converged-computing/goshare"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/converged-computing/goshare","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/converged-computing%2Fgoshare","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/converged-computing%2Fgoshare/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/converged-computing%2Fgoshare/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/converged-computing%2Fgoshare/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/converged-computing","download_url":"https://codeload.github.com/converged-computing/goshare/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/converged-computing%2Fgoshare/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28338970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T10:58:46.209Z","status":"ssl_error","status_checked_at":"2026-01-12T10:58:42.742Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-12T11:16:46.835Z","updated_at":"2026-01-12T11:16:46.935Z","avatar_url":"https://github.com/converged-computing.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# goshare\n\nProducer / consumer model to share commands between containers in Kubernetes. We do this using gRPC over unix domain sockets (UDS) via:\n\n- Running a process in the consumer container with a listener. This creates a PID that the producer can find in `/proc/\u003cpid\u003e`\n- Start the producer, pointing it to the PID of the consumer, and expecting it to write a socket to a known path in `/proc/\u003cpid\u003e/root`\n\nAt this point, we can run the producer as many times as needed, providing a command to give to the consumer to execute. The consumer will:\n\n - Receive the command\n - Execute it (or return an error back it's not found, etc.)\n - Provide the pid back to the producer\n - The producer then needs to somehow watch this PID for it to complete (likely with some API that uses ps, need to think about this more because we don't want to be polling)\n\nI am starting from [this example](https://github.com/devlights/go-grpc-uds-example/tree/master) with an MIT license, included in [.github](.github).\nI need creative terminology for producer and consumer, so I'm stil thinking about this. Right now, client and server is probably logical\nfor a listener and message sender! I'm first going to test this small app (to make sure it works) and then I'll work on customizing it\nfor submitting jobs. I am reading that we should set `GOMAXPROCS` to be the number of concurrent jobs we will allow.\n\n## Usage\n\n### Quick Start\n\nBuild\n\n```bash\ntask install\ntask protoc\ntask build\nbuild-arm\ntask run\n```\n\n### Server\n\nYou will generally want to start a server. You can either run it as a background\nprocess or use `ps aux` from another terminal (or container with shared process namespace)\nto see the PID.\n\n```bash\n./bin/server\n```\n\nTo specify a different unix socket to use:\n\n```bash\n./bin/server -s /dinosaur.sock\n```\n\nSee all options (there are few!)\n\n```bash\n./bin/server --help\n```\n\nIf you leave out the socket, it by default will be written to '/tmp/goshare.sock'.\nThe same is true for the client, discussed next.\n\n### Client\n\nThe client should connect to the same unix socket as the server for proper communication!\n\n```bash\n./bin/client -s /dinosaur.sock \u003ccommand\u003e\n```\n\nSince the jobs are run possibly on a different system, you can also define the working directory.\n\n```bash\n./bin/client -w /working/directory \u003ccommand\u003e\n```\n\nIf you leave out a command, a dummy \"echo hello world\" is used for a test. \n\n### Wait\n\nThe wait executable is intended to take a command or full path to an executable, and return the PID. It will wait (and loop) until it finds a match.\nAs an example, here is with full verbosity:\n\n```bash\n$ ./bin/wait -c \"/usr/libexec/gvfsd-recent --spawner :1.2 /org/gtk/gvfs/exec_spaw/5\" \n```\n```console\n🟧️  wait: 2023/07/27 23:44:30 wait.go:38: /usr/libexec/gvfsd-recent --spawner :1.2 /org/gtk/gvfs/exec_spaw/5\n🟧️  wait: 2023/07/27 23:44:30 wait.go:64: Found matched command /usr/libexec/gvfsd-recent --spawner :1.2 /org/gtk/gvfs/exec_spaw/5 with pid 4013899\n4013899\n```\n\nWe can run in quiet mode (only print out the final PID):\n\n```bash\n./bin/wait -c \"/usr/libexec/gvfsd-recent --spawner :1.2 /org/gtk/gvfs/exec_spaw/5\" -q\n4013899\n```\n\nOr change the frequency of polling (in seconds):\n\n```bash\n./bin/wait -c \"/usr/libexec/gvfsd-recent --spawner :1.2 /org/gtk/gvfs/exec_spaw/5\" -q -w 10\n4013899\n```\n\nNote that you can also provide the executable (full path) to look for if it is unique, and both should\nbe in quotes in case of flags, etc.\n\n### Wait-fs\n\n`wait-fs` is a derivative of wait, but will wait for a path on the filesystem to exist.\nE.g., try running it with a path that doesn't exist in one terminal:\n\n```bash\n./bin/wait-fs -p ./does-not-exist.txt\n```\n```console\n🟧️  wait-fs: 2023/09/06 13:50:04 wait-fs.go:53: Path ./does-not-exist.txt does not exist yet, sleeping 5\n🟧️  wait-fs: 2023/09/06 13:50:09 wait-fs.go:53: Path ./does-not-exist.txt does not exist yet, sleeping 5\n```\n\nAnd then try creating it in another:\n\n```bash\n$ touch does-not-exist.txt\n```\n\nYou'll see it discovered and the script exit.\n\n```console\n🟧️  wait-fs: 2023/09/06 13:50:04 wait-fs.go:40: ./does-not-exist.txt\n🟧️  wait-fs: 2023/09/06 13:50:04 wait-fs.go:53: Path ./does-not-exist.txt does not exist yet, sleeping 5\n🟧️  wait-fs: 2023/09/06 13:50:09 wait-fs.go:53: Path ./does-not-exist.txt does not exist yet, sleeping 5\n🟧️  wait-fs: 2023/09/06 13:50:14 wait-fs.go:49: Found existing path ./does-not-exist.txt\n```\n\nAnd that's it! We use this for an operator where we are waiting for a particular file to exist.\n\n## Setup\n\nWe are going to use [go-task](https://taskfile.dev/) over a Makefile. To install, [download a release](https://github.com/go-task/task/releases) and I installed with dpkg.\n\n```sh\n$ task --list\ntask: Available tasks for this project:\n* build:                      build\n* build-arm:       build-arm\n* install-requirements:       install requirements\n* protoc:                     gen protoc\n* run:                        run\n```\n\n### Install gRPC and Go libraries\n\n```sh\ntask install\n```\n\n### Run protoc\n\nThe way I understand this, this compiles the code from [proto](proto) (the echo.proto) into the [internal](internal) folder\nwhere it can be used by the Go libraries under [cmd](cmd) to define the structure of messages.\n\n```bash\ntask protoc\n```\n\n### Build Server and Client\n\n```bash\ntask build\n```\n```console\ntask: [build] GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/wait cmd/wait/wait.go\ntask: [build] GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/server cmd/server/server.go\ntask: [build] GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/client cmd/client/client.go\ntask: [build] GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/wait-fs cmd/wait-fs/wait-fs.go\n```\n\nor for arm:\n\n```\ntask build-arm\n```\n\n\nThese are generated in [bin](bin)\n\n### Run Server and Client\n\n```sh\ntask run\n```\n```console\ntask: [run] ./bin/server \u0026\ntask: [run] sleep 1\ntask: [run] ./bin/client\n🟪️  client: 2023/07/25 15:57:16 client.go:40: socket path: /tmp/echo.sock\n🟪️  client: 2023/07/25 15:57:16 client.go:41: requested command: echo hello world\n🟪️  client: 2023/07/25 15:57:16 client.go:82: sent command: echo hello world\n🟦️ service: 2023/07/25 15:57:16 command.go:26: start new stream request\n🟦️ service: 2023/07/25 15:57:16 command.go:54: Received command echo hello world\n🟦️ service: 2023/07/25 15:57:16 command.go:67: send new pid=461564\n🟦️ service: 2023/07/25 15:57:16 command.go:70: Process started with PID: 461564\n🟦️ service: 2023/07/25 15:57:16 command.go:75: send final output: hello world\n🟪️  client: 2023/07/25 15:57:16 client.go:103: pid 461564 is active\n🟪️  client: 2023/07/25 15:57:16 client.go:88: closing send\n🟪️  client: 2023/07/25 15:57:16 client.go:103: pid 461564 is active\n🟪️  client: 2023/07/25 15:57:16 client.go:107: new output received: hello world\n🟪️  client: 2023/07/25 15:57:16 client.go:108: process is done, closing\n🟪️  client: 2023/07/25 15:57:16 client.go:130: finished with client request\n```\n\n## TODO next\n\n- add subcommands to client to run / cancel?\n- ensure we check for executable first\n - should be table of values that indicate what happened\n- test run with a sleep command, then cancel\n- try making a release we can install to a dummy jobset with a flux container and go + application\n\n## References\n\n### Examples of both server and client side\n\n - https://github.com/devlights/go-grpc-uds-example\n - https://github.com/pahanini/go-grpc-bidirectional-streaming-example\n - https://zenn.dev/hsaki/books/golang-grpc-starting/viewer/client\n - http://yamahiro0518.hatenablog.com/entry/2016/02/01/215908\n - https://stackoverflow.com/a/46279623\n - https://stackoverflow.com/a/18479916\n\n### gRPC with Unix Domain Socket example (server side)\n\n - https://qiita.com/hnakamur/items/848097aad846d40ae84b\n\n### gRPC with Unix Domain Socket example (client side)\n\n - https://qiita.com/marnie_ms4/items/4582a1a0db363fe246f3\n\n## License\n\nHPCIC DevTools is distributed under the terms of the MIT license.\nAll new contributions must be made under this license.\n\nSee [LICENSE](https://github.com/converged-computing/cloud-select/blob/main/LICENSE),\n[COPYRIGHT](https://github.com/converged-computing/cloud-select/blob/main/COPYRIGHT), and\n[NOTICE](https://github.com/converged-computing/cloud-select/blob/main/NOTICE) for details.\n\nSPDX-License-Identifier: (MIT)\n\nLLNL-CODE- 842614\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconverged-computing%2Fgoshare","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconverged-computing%2Fgoshare","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconverged-computing%2Fgoshare/lists"}