{"id":13629471,"url":"https://github.com/u-root/cpu","last_synced_at":"2025-05-16T06:07:09.015Z","repository":{"id":38330864,"uuid":"209644546","full_name":"u-root/cpu","owner":"u-root","description":"cpu command in Go, inspired by the Plan 9 cpu command","archived":false,"fork":false,"pushed_at":"2025-04-17T21:30:16.000Z","size":3763,"stargazers_count":253,"open_issues_count":23,"forks_count":31,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-05-10T00:47:32.673Z","etag":null,"topics":["cpu","golang","plan9"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/u-root.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-09-19T20:38:23.000Z","updated_at":"2025-04-17T21:30:18.000Z","dependencies_parsed_at":"2023-02-16T02:00:59.990Z","dependency_job_id":"bf8fba99-8596-4930-b9ee-af62546588d7","html_url":"https://github.com/u-root/cpu","commit_stats":{"total_commits":370,"total_committers":18,"mean_commits":"20.555555555555557","dds":0.4297297297297298,"last_synced_commit":"50279c508ed0c6e7a48549ea941a65f7c4eed392"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fcpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fcpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fcpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u-root%2Fcpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/u-root","download_url":"https://codeload.github.com/u-root/cpu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478190,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpu","golang","plan9"],"created_at":"2024-08-01T22:01:11.505Z","updated_at":"2025-05-16T06:07:04.004Z","avatar_url":"https://github.com/u-root.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# cpu\n\n[![CircleCI](https://circleci.com/gh/u-root/cpu.svg?style=svg)](https://circleci.com/gh/u-root/cpu)\n[![Go Report Card](https://goreportcard.com/badge/github.com/u-root/cpu)](https://goreportcard.com/report/github.com/u-root/cpu)\n[![CodeQL](https://github.com/u-root/cpu/workflows/CodeQL/badge.svg)](https://github.com/u-root/cpu/actions?query=workflow%3ACodeQL)\n[![GoDoc](https://godoc.org/github.com/u-root/cpu?status.svg)](https://godoc.org/github.com/u-root/cpu)\n[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://github.com/u-root/cpu/blob/master/LICENSE)\n\nThis repo is an implementation the Plan 9 cpu command, both client and server, for Linux.\nMore detail is available in the [CPU chapter of the LinuxBoot book](https://book.linuxboot.org/utilities/cpu.html).\nUnlike the Plan 9 command, this version uses the ssh protocol for the underlying transport. It includes\nfeatures familiar to ssh users, such as support for the ssh config file.\n\n## Overview\nThe cpu command\nlets you log in from a local system to a remote system and see some or all of the files (how much is\nup to you) from the local system.\n\nThis is wonderfully convenient for embedded systems programmers. Because some or all the files\ncan come from your local machine, including binaries, the only thing you need installed\non the remote machine is the cpu daemon itself.\n\n### Motivation\nConsider the case of running a\ncomplex Python program on an embedded system.\nWe will need to either do a full install of some distro on that system, meaning we\nneed USB ports and local storage; or we will need to run the program over the\nnetwork.\n\nInstalling distros can turn into a mess. Some programs only work under specific distros.\nIn some cases, when two programs are needed in a pipeline, it can happen that they only work\nunder different distros!\nUsers are left juggling USB sticks and NVME cards, and this fails the first time there are\ntwo programs which need two different distros.\n\nRunning over a network is usually done with ssh, but ssh can not supply the programs and files.\nWe would need to either set up a network file system, meaning\nfinding a sysadmin willing to set it up, and keep it working; or, trying to figure out which\nfiles the program needs, and using rsync or scp to get them there. In some cases,\nthe target system might not have enough memory to hold those files!\n\nCpu looks like ssh, but with an important difference: it also provides a file transport\nso that the files your program needs are available via a 9p mount. For example, if I have an\nembedded system named camera, and I need to read the flash with the flashrom command, I simply type:\n\n```\ncpu camera flashrom -r rom.img\n```\n\n\u003cimg alt=\"IP camera robots\" src=\"doc/img/ip-camera-robot.jpg\" height=\"240px\" /\u003e\n\nBreaking this down: cpu is the cpu command; camera is the host name; flashrom is the command\nto run; the options are to do a read (-r) into a file called rom.img.\n\nWhere does that file end up? In whatever of my home directories I ran the cpu command from. I need\nnot worry about scp'ing it back, or any such thing; it's just there.\n\n### Building your own docker container\n\nYou can easily build your own docker container to try things out.\n\n```\ndocker build -t \"${USER}/cpu:latest\" .\n```\n\nor if you have installed a version of docker buildx, you can build a multi-arch manifest container and push it to docker hub:\n```\n% docker login\n% docker buildx build --platform \"linux/amd64,linux/arm64,linux/arm/v7\" --progress plain --pull -t \"${USER}/cpu:latest\" .\n```\n\n### Pre-built Docker container for trying out cpu (on arm64 \u0026 amd64 for now)\n\nWe have created a docker container so you can try cpu client and server:\n```\nghcr.io/u-root/cpu:main\n```\n\nIt includes both the cpud (server) and cpu (client) commands. In the\ncontainer, you only have access to date and cat commands, but that is enough to get\nthe idea.\n\nYou will need keys. You can either use your own SSH keys that you use for\nother things, for example:\n```\nexport KEY=~/.ssh/id_rsa\nexport KEY=~/.ssh/a_special_key_for_this_docker\n```\n\nor generate one and use it.\n```\nssh-keygen -f key -t rsa  -P \"\"\nexport KEY=`pwd`/key\n```\nNOTE! The name KEY is not required. Instead of KEY, you\ncan use any name you want, as long as you use it in the docker\ncommand below.\n\nTo start the cpud, you need docker installed. Once that is done, you need to create\na docker network and start the daemon, with public and private keys.\nThe --mount option allows docker to provide the keys, using a bind mount\nfor both the private and public key.\nThat is how we avoid\nstoring keys in the container itself.\n```\ndocker network create cpud\n# If you ran docker before and it failed in some way, you may need to remove the\n# old identity (e.g. docker rm cpud_test)\ndocker run --rm -v $KEY.pub:/key.pub -v $KEY:/key -v /tmp --name cpud_test --privileged=true -t -i -p 17010:17010 ghcr.io/u-root/cpu:main\n```\n\nThen you can try running a command or two by using the embedded cpu client in the docker container.  \n_NOTE_: when you run cpu in this way, it does not immediately have access to your host's file system, \nwhich means you won't be able to really leverage the back-mount and you'll have to artificially set \nenvironment variables (like PWD) so that the remote task can execute.\n\n```\ndocker exec -it -e PWD=/root cpud_test  /bin/cpu -key /key localhost /bin/date\n```\n\nRemember, this cpu command is running in the container. You need to use the name /key in the\ncontainer, not $KEY and you'll only be able to run binaries that have been pre-loaded into the \ncontainer (which in the public container is /bin/cat and /bin/date)\n\nTo see the mounts:\n```\ndocker exec -it -e PWD=/root cpud_test  /bin/cpu -key /key localhost /bin/cat /proc/mounts\n```\n\nYou might want to just get a cpu command to let you talk to the docker cpud\ndirectly:\n```\ngo install github.com/u-root/cpu/cmds/cpu@latest\n```\n\nAnd now you can run\n```\ncpu -key $KEY localhost date\n```\n\n_NOTE_: if you are running on OSX, remember that your cpud docker is linux, so if you try the above\ncommand you'll see:\n```\n$ cpu -key $KEY localhost date\n2022/10/13 18:52:17 CPUD(as remote):fork/exec /bin/date: exec format error\n```\n\nTo deal with that we'll have to play games with the namespace, which we'll cover next.\n\n### cpu on heterogeneous systems.\n\nThe cpu command sets up the various 9p mounts with a default namespace. Users can override this\ndefault with the -namespace switch. The argument to the switch looks like\nPATH variables, with comma-separated values, but with one extra option: users can, optionally,\nspecify the local path and the remote path. This is useful when running ARM binaries\nhosted from an x86 system.\n\nIn the example below, we show starting up a bash on an ARM system (solidrun honeycomb) using\na cpu command running on an x86 system.\n\n```\ncpu -namespace /home:/bin=`pwd`/bin:/lib=`pwd`/lib:/usr=`pwd`/usr honeycomb /bin/bash\n```\n\nBreaking this down, we set up the namespace so that:\n* the remote /home is from our /home\n* the remote /bin is from `pwd`/bin -- which, in this case, was an unpacked arm64 file system image\n* the remote /lib is from `pwd`/lib\n* the remote /usr is from `pwd`/usr\n\nWe can use the path /bin/bash, because /bin/bash on the remote points to `pwd`/bin/bash on the local\nmachine.\n\nWe can use the same trick to cpu to Linux from OSX, but instead of having an Arm tree under `pwd`\nwe'll need a Linux binary tree to pick binaries from.\n\n### cpu over USB\n\nThere are many IoT like devices that do not have an ethernet port.\nFear not though: The Linux USB gadget drivers offer ethernet via USB!\n\nThere are [tutorials out\nthere](https://linuxlink.timesys.com/docs/wiki/engineering/HOWTO_Use_USB_Gadget_Ethernet), and here is the gist:\n\n- enable the Linux kernel options\n  * `CONFIG_USB_GADGET`\n  * `CONFIG_USB_ETH`\n  * `CONFIG_USB_ETH_RNDIS` (for Windows support)\n  * `CONFIG_INET`\n- add the MAC addresses for your gadget device and the machine you connect to in\n  the kernel `CMDLINE`, e.g., `g_ether.dev_addr=12:34:56:78:9a:bc g_ether.host_addr=12:34:56:78:9a:bd`\n\n## cpu will be familiar to ssh users\n\nAs mentioned, cpu looks and feels a lot like ssh, to the point of honoring ssh config files.\nFor the honeycomb, for example, the ssh config entry looks like this (we shorten the name to 'h'\nfor convenience):\n\n```\nHost h\n\tHostName honeycomb\n\tPort 17010\n\tUser root\n\tIdentityFile ~/.ssh/apu2_rsa\n```\n\nNote that the cpu command is itself a 9p server; i.e., your instance of cpu runs your server. The remote\ncpu server may run as root, but all file accesses happen locally as you. Hence,\nthe cpu command does not grant greater access to the local machine than you already possess.\nI.e., there is no privilege escalation.\n\n## cpu and Docker\n\nMaintaining file system images is inconvenient.\nWe can use Docker containers on remote hosts instead.\nWe can take a standard Docker container and, with suitable options, use docker\nto start the container with cpu as the first program it runs.\n\nThat means we can use any Docker image, on any architecture, at any time; and\nwe can even run more than one at a time, since the namespaces are private.\n\nIn this example, we are starting a standard Ubuntu image:\n```\ndocker run -v /home/rminnich:/home/rminnich -v /home/rminnich/.ssh:/root/.ssh -v /etc/hosts:/etc/hosts --entrypoint /home/rminnich/go/bin/cpu -it ubuntu@sha256:073e060cec31fed4a86fcd45ad6f80b1f135109ac2c0b57272f01909c9626486 h\nUnable to find image 'ubuntu@sha256:073e060cec31fed4a86fcd45ad6f80b1f135109ac2c0b57272f01909c9626486' locally\ndocker.io/library/ubuntu@sha256:073e060cec31fed4a86fcd45ad6f80b1f135109ac2c0b57272f01909c9626486: Pulling from library/ubuntu\na9ca93140713: Pull complete\nDigest: sha256:073e060cec31fed4a86fcd45ad6f80b1f135109ac2c0b57272f01909c9626486\nStatus: Downloaded newer image for ubuntu@sha256:073e060cec31fed4a86fcd45ad6f80b1f135109ac2c0b57272f01909c9626486\nWARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested\n1970/01/01 21:37:32 CPUD:Warning: mounting /tmp/cpu/lib64 on /lib64 failed: no such file or directory\n# ls\nbbin  buildbin\tenv  go    init     lib    proc  tcz  ubin  var\nbin   dev\tetc  home  key.pub  lib64  sys\t tmp  usr\n#\n```\n\nNote that the image was updated and then started. The /lib64 mount fails, because there is no /lib64 directory in the image, but\nthat is harmless.\n\nOn the local host, on which we ran docker, this image will show up in docker ps:\n```rminnich@a300:~$ docker ps\nCONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS         PORTS     NAMES\nb92a3576229b   ubuntu    \"/home/rminnich/go/b…\"   9 seconds ago   Up 9 seconds             inspiring_mcnulty\n````\n\nEven though the binaries themselves are running on the remote ARM system.\n\n## Testing with vsock\n\nVsock is a useful transport layer available in Linux, and support by at least QEMU.\n\nWe use the mdlayher/vsock package.\n\nIn the cpu and cpud, the switch\n```\n-net vsock\n```\nwill enable vsock.\n\nIn the host kernel, you need ```vhost_vsock``` module:\n```\nsudo modprobe vhost_vsock\n```\n.\n\nWhen starting qemu, add\n```\n-device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3\n```\nto the command line. The '3' is arbitrary; it just needs to be agreed upon on both sides.\n\nWhen running a cpu command, the host name is the vsock guest-cid you specified in qemu:\n```\ncpu -net vsock 3 date\n```\n\nIf you want a different port, you can use the same -sp switch you use for other network types.\n\n## De-centralized cpu with DNS-SD\n\nA variation of the cpu and cpud commands now exist which use dns-sd to autodiscover and register cpu resources.  DNS-SD is a multi-cast DNS protocol that will multi-cast out requests for resources and get responses from participating cpud nodes.  Meta-data provides information such as architecture, OS, number of cores, free memory, load average, and number of existing cpu clients.  When you start a decpud you can specify additional meta-data which might be harder to auto-disocver such as near=storage, near=gateway, near=gpu, or secure=true (if running in a confidential computing domain).\n\nIn order to use this functionality, you can use decpu and decpud just as you would cpu and cpud.  decpud will enable dns-sd registration by default, and multicast its information in response to requests.  In order to use this from decpu, you can just specify decpu without a hostname and, by default, it will find the lowest loaded decpud with the same architecture and OS as the host you run the decpu command from.\n\nYou can specify additional constraints by using a dns-sd URI formulation:\n```\n  decpu dnssd://?requirement=value\\\u0026otherrequirement=\\\u003cothervalue\\\u0026sort=tenet\n```\n\nEssentially you can provide a set of key/value requirements that must be met in order for a decpud node to be considered.  You can override the defaults (arch/os), or you can specify a minimum core count or minimum amount of free memory.  Numeric values can use comparison operators (\u003c or \u003e) Finally, you can specify any number of numeric keys to sort based on.  If you want to ignore one of the default requirements (e.g. arch, os) then you can set to a \\*:\n```\n  decpu dnssd://?arch=\\*\\\u0026os=\\*\n```\n This can be useful if what you are running is a script or an interpretive language like python.  _If you are running on the shell, make sure you escape any characters the shell may have an interest in (\u003c, \u003e, !, \u0026, *)_\n\n### Runnign the DNS-SD tools inside Docker\n\nIn order to use dns-sd you have to be able to send/receive multicast.  The easiest way to do this on Linux is to use host networking when you start your docker containers (--network host).  This unfortunately does not work on Mac OSX, so you will need to run a relay on your system that tunnels multicast to/from the docker network.  The relay client can be run in every container or you can start all dns-sd containers in the same docker network and start a relay container client on that network to communicate for the group.\n\nThere is a container in the decent-e fork of the dnssd package: https://github.com/decent-e/dnssd under cmd/relay.  Running relay will start the server on the host, and running relay mode=client will start the client inside the docker container.  You can also use a pre-build docker image ( docker pull ghcr.io/decent-e/dnssd:main ) to start the client in a docker.\n\nExample: (on mac)\n```\n# presumes you have already setup your $KEY appropriately\n% docker create network cpud\n% go install github.com/decent-e/dnssd/cmd/relay@latest\n% export PATH=$GOBIN:$PATH\n% relay \u0026\n% docker run -d --network cpud ghcr.io/decent-e/dnssd:main\n% docker run -d --network cpud -v $KEY:/key -v $KEY.pub:/key.pub -v /tmp:/tmp --privileged --rm --name decpud ghcr.io/decent-e/cpu:decent-e /bin/decpud\n% docker exec -i -t -e PWD=/ decpud /bindecpu -key /key . /bin/date\n# you should also be able to see the service from your mac\n% dns-sd -B _ncpu._tcp\nBrowsing for _ncpu._tcp\nDATE: ---Sun 09 Oct 2022---\n18:35:56.925  ...STARTING...\nTimestamp     A/R    Flags  if Domain               Service Type         Instance Name\n18:35:56.925  Add        3  17 local.               _ncpu._tcp.          d3c196958c24-cpud\n18:35:56.925  Add        2  14 local.               _ncpu._tcp.          d3c196958c24-cpud\n```\n\n## Summary\nThe cpu command makes using small embedded systems dramatically easier. There is no need to install\na distro, or juggle distros; there is no need to scp files back and forth; just run commands\nas needed.\n\n## Development\n\nFor debugging, `tcpdump` is very handy. Read [a short tutorial](\nhttps://danielmiessler.com/study/tcpdump/) to get familiar with it.\n\n## Further reading\n\n### Talks\n\n* Short Talk \"building small stateless network-controlled appliances with\n  coreboot/linuxboot and u-root’s cpu command\"\n  * at Open Source Firmware Conference 2019\n   [slides](https://docs.google.com/presentation/d/1ee8kxuLBJAyAi-xQqE75EMk-lNM5d8Z6CarWoYlO6Ws/edit?usp=sharing) /\n   [recording](https://www.youtube.com/watch?v=mxribsZFDQQ)\n  * at BARC2021\n   [slides (PDF)](https://bostonarch.github.io/2021/presentations/U-root%20CPU%20command.pdf)\n* [Network Managed Processors at IoT World 2021](https://docs.google.com/presentation/d/1jREHiHci1EAMWdj--6uX9o0aVpeCPy21UQpnl1oTSwE/edit?usp=sharing)\n* [\"Plan 9 CPU command, in Go, for Linux - the network is the computer -- for real this time\" at FOSDEM 2022\n  ](https://fosdem.org/2022/schedule/event/plan_9_cpu_cmd/)\n* [\"Drivers From Outer Space at CLT 2022 - Fast, Simple Driver Development\"\n  ](https://chemnitzer.linux-tage.de/2022/de/programm/beitrag/226)\n* Short demo of [Attaching CPUs via USB](https://media.ccc.de/v/all-systems-go-2023-246-attaching-cpus-via-usb)\n\n### History\n\nThe first version of cpu was developed for Plan 9, and is described [here](http://man.cat-v.org/plan_9/1/cpu).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu-root%2Fcpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fu-root%2Fcpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu-root%2Fcpu/lists"}