{"id":13582365,"url":"https://github.com/rancher/remotedialer","last_synced_at":"2025-04-06T14:30:53.193Z","repository":{"id":37663742,"uuid":"178285814","full_name":"rancher/remotedialer","owner":"rancher","description":"HTTP in TCP in Websockets in HTTP in TCP, Tunnel all the things!","archived":false,"fork":false,"pushed_at":"2025-03-21T13:37:10.000Z","size":397,"stargazers_count":294,"open_issues_count":15,"forks_count":104,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-03-21T23:35:44.375Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rancher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-28T21:36:54.000Z","updated_at":"2025-03-04T21:52:42.000Z","dependencies_parsed_at":"2022-07-18T05:21:42.765Z","dependency_job_id":"d98cde7e-d5ef-47e7-948f-c730cfcbb46a","html_url":"https://github.com/rancher/remotedialer","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fremotedialer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fremotedialer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fremotedialer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fremotedialer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rancher","download_url":"https://codeload.github.com/rancher/remotedialer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247495722,"owners_count":20948101,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:02:38.797Z","updated_at":"2025-04-06T14:30:53.186Z","avatar_url":"https://github.com/rancher.png","language":"Go","funding_links":[],"categories":["Go","\u003ca id=\"01e6651181d405ecdcd92a452989e7e0\"\u003e\u003c/a\u003e工具"],"sub_categories":["\u003ca id=\"9d6789f22a280f5bb6491d1353b02384\"\u003e\u003c/a\u003e隧道\u0026\u0026穿透"],"readme":"Reverse Tunneling Dialer\n========================\n\nremotedialer creates a two-way connection between a server and a client, so\nthat a `net.Dial` can be performed on the server and actually connects to the\nclient running services accessible on localhost or behind a NAT or firewall.\n\nArchitecture\n------------\n\n### Abstractions\n\nremotedialer consists of structs that organize and abstract a TCP connection\nbetween a server and a client. Both client and server use ``Session``s, which\nprovides a means to make a network connection to a remote endpoint and keep\ntrack of incoming connections. A server uses the ``Server`` object which\ncontains a ``sessionManager`` which governs one or more ``Session``s, while a\nclient creates a ``Session`` directly. A ``connection`` implements the\n``io.Reader`` and ``io.WriteCloser`` interfaces, so it can be read from and\nwritten to directly. The ``connection``'s internal ``readBuffer`` monitors the\nsize of the data it is carrying and uses ``backPressure`` to pause incoming\ndata transfer until the amount of data is below a threshold.\n\n![](./docs/remotedialer.png)\n\n### Data flow\n\n![](./docs/remotedialer-flow.png)\n\nA client establishes a session with a server using the server's URL. The server\nupgrades the connection to a websocket connection which it uses to create a\n``Session``. The client then also creates a ``Session`` with the websocket\nconnection.\n\nThe client sits in front of some kind of HTTP server, most often a kubernetes\nAPI server, and acts as a reverse proxy for that HTTP resource. When a user\nrequests a resource from this remote resource, request first goes to the\nremotedialer server. The application containing the server is responsible for\nrouting the request to the right client connection.\n\nThe request is sent through the websocket connection to the client and read\ninto the client's connection buffer. A pipe is created between the client and\nthe HTTP service which continually copies data between each socket. The request\nis forwarded through the pipe to the remote service, draining the buffer. The\nservice's response is copied back to the client's buffer, and then forwarded\nback to the server and copied into the server connection's own buffer. As the\nuser reads the response, the server's buffer is drained.\n\nThe pause/resume mechanism checks the size of the buffer for both the client\nand server. If it is greater than the threshold, a ``PAUSE`` message is sent\nback to the remote connection, as a suggestion not to send any more data. As\nthe buffer is drained, either by the pipe to the remote HTTP service or the\nuser's connection, the size is checked again. When it is lower than the\nthreshold, a ``RESUME`` message is sent, and the data transfer may continue.\n\n### remotedialer in the Rancher ecosystem\n\nremotedialer is used to connect Rancher to the downstream clusters it manages,\nenabling a user agent to access the cluster through an endpoint on the Rancher\nserver. remotedialer is used in three main ways:\n\n#### Agent config and tunnel server\n\nWhen the agent starts, it initially makes a client connection to the endpoint\n`/v3/connect/register`, which runs an authorizer that sets some initial data\nabout the node. The agent continues to connect to the endpoint `/v3/connect` on\na loop. On each connection, it runs an OnConnect handler which pulls down node\nconfiguration data from `/v3/connect/config`.\n\n#### Steve Aggregation\n\nThe steve aggregation server on the agent establishes a remotedialer Session\nwith Rancher, making the steve API on the downstream cluster accessible from\nthe Rancher server and facilitating resource watches.\n\n#### Health Check\n\nThe clusterconnected controller in Rancher uses the established tunnel to check\nthat clusters are still responsive and sets alert conditions on the cluster\nobject if they are not.\n\n#### HA operation (peering)\n\nremotedialer supports a mode where multiple servers can be configured as peers.\nIn that mode all servers maintain a mapping of all remotedialer client connections\nto all other servers, and can route incoming requests appropriately.\n\nTherefore, http requests referring any of the remotedialer clients can be resolved\nby any of the peer servers. This is useful for high availability, and Rancher\nleverages that functionality to distribute downstream clusters (running agents\nacting as remotedialer clients) among replica pods (acting as remotedialer\nserver peers). In case one Rancher replica pod breaks down, Rancher will\nreassign its downstream clusters to others.\n\nPeers authenticate to one another via a shared token.\n\nRunning Locally\n---------------\n\nremotedialer provides an example client and server which can be run in\nstandalone mode FOR TESTING ONLY. These are found in the `server/` and\n`client/` directories.`\n\n### Compile\n\nCompile the server and client:\n\n```\nmake server\nmake client\n```\n\n### Run\n\nStart the server first.\n\n```\n./server/server\n```\n\nThe server has debug mode off by default. Enable it with `--debug`.\n\nThe client proxies requests from the remotedialer server to a web server, so it\nneeds to be run somewhere where it can access the web server you want to\ntarget. The remotedialer server also needs to be reachable by the client.\n\nFor testing purposes, a basic HTTP file server is provided. Build the server with:\n\n```\nmake dummy\n```\n\nCreate a directory with files to serve, then run the web server from that directory:\n\n```\nmkdir www\ncd www\necho 'hello' \u003e bigfile\n/path/to/dummy\n```\n\nRun the client with\n\n```\n./client/client\n```\n\nBoth server and client can be run with even more verbose logging:\n\n```\nCATTLE_TUNNEL_DATA_DEBUG=true ./server/server --debug\nCATTLE_TUNNEL_DATA_DEBUG=true ./client/client\n```\n\n### Usage\n\nIf the remotedialer server is running on 192.168.0.42, and the web service that\nthe client can access is running at address 127.0.0.1:8125, make proxied\nrequests like this:\n\n```\ncurl http://192.168.0.42:8123/client/foo/http/127.0.0.1:8125/bigfile\n```\n\nwhere `foo` is the hardcoded client ID for this test server.\n\nThis test server only supports GET requests.\n\n### HA Usage\n\nTo test remotedialer in HA mode, first start the dummy server from an appropriate directory, eg.:\n\n```shell\ncd /tmp\nmkdir www\ncd www\necho 'hello' \u003e bigfile\n/path/to/dummy -listen :8125\n```\n\nThen start two peer remotedialer servers with the `-peers id:token:url` flag:\n\n```shell\n./server/server -debug -id first -token aaa -listen :8123 -peers second:aaa:ws://localhost:8124/connect \u0026\n./server/server -debug -id second -token aaa -listen :8124 -peers first:aaa:ws://localhost:8123/connect\n```\n\nThen connect a client to the first server, eg:\n```shell\n./client/client -id foo -connect ws://localhost:8123/connect\n```\n\nFinally, use the second server to make a request to the client via the first server:\n```\ncurl http://localhost:8124/client/foo/http/127.0.0.1:8125/\n```\n\n# Versioning\n\nSee [VERSION.md](VERSION.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher%2Fremotedialer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Francher%2Fremotedialer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher%2Fremotedialer/lists"}