{"id":18743172,"url":"https://github.com/kbingham/zc","last_synced_at":"2025-10-20T11:21:04.943Z","repository":{"id":145221057,"uuid":"80710014","full_name":"kbingham/zc","owner":"kbingham","description":null,"archived":false,"fork":false,"pushed_at":"2017-02-02T10:05:12.000Z","size":20,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-12-28T19:42:46.606Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kbingham.png","metadata":{"files":{"readme":"README.zc","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-02T09:20:59.000Z","updated_at":"2019-03-06T15:22:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"78932301-9403-41db-a410-65e2a1ff8b6a","html_url":"https://github.com/kbingham/zc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kbingham%2Fzc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kbingham%2Fzc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kbingham%2Fzc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kbingham%2Fzc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kbingham","download_url":"https://codeload.github.com/kbingham/zc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239625042,"owners_count":19670617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T16:10:27.539Z","updated_at":"2025-10-20T11:20:59.888Z","avatar_url":"https://github.com/kbingham.png","language":"C","readme":"\nZC - a tool for measuring the efficiency of Linux TCP transmission\nAndrew Morton \u003candrewm@uow.edu.au\u003e                     27 Jan 2001\n==================================================================\n\n\nOverview\n========\n\nZC consists of a client (zcc) and a server (zcs) and a CPU load\nmeasuring tool (cyclesoak).  The client sends a bunch of files to the\nserver.  The server monitors the throughput (kbytes/sec).\n\nThe CPU load measurement tool `cyclesoak' is very accurate.\n\nThe client can also use read()/write() or read()/send() for\ntransmission rather than sendfile(), so these can be compared.\n\n\nResults\n=======\n\n\nThe kernels which were tested were 2.4.1-pre10 with and without the\nzerocopy patch.  We only look at client load (the TCP sender).\n\nIn all tests the link throughput was 11.5 mbytes/sec at all times\n(saturated 100baseT) unless otherwise noted.\n\nThe client (the thing which sends data) is a dual 500MHz PII with a\n3c905C.\n\nFor the write() and send() tests, the chunk size was 64 kbytes.\n\nThe workload was 63 files with an average length of 350 kbytes.\n\nCPU figures are also given for the same machine receiving the 11.5\nmbyte/sec stream.  It is doing 16 kbyte read()s.\n\n                                                    3c905C 3c905C  3c905C 3c905C 3c905C    eepro100  eepro100  eepro100\n                                                     CPU   affine   ints  rx pps tx pps      MMIO     I/O ops   ints\n       \n    2.4.1-pre10+zerocopy, sendfile():                9.6%           4395   4106   8146      15.3%\n    2.4.1-pre10+zerocopy, send():                   24.1%           4449   4163   8196      20.2%\n    2.4.1-pre10+zerocopy, receiving:                18.7%           12332  8156   4189      17.6%\n\n    2.4.1-pre10+zerocopy, sendfile(), no xsum/SG:   16.2%                                  (15.3%)\n    2.4.1-pre10+zerocopy, send(), no xsum/SG:       21.5%                                  (20.2%)\n\n\n\n    2.4.1-pre10-vanilla, using sendfile():          17.1%  17.9%    5729   5296   8214      16.1%     16.8%\n    2.4.1-pre10-vanilla, using send():              21.1%           4629   4152   8191      20.3%     20.6%     6310\n    2.4.1-pre10-vanilla, receiving:                 18.3%          12333   8156   4188      17.1%     18.2%    12335\n\n\nBearing in mind that a large amount of the load is in the device\ndriver, the zerocopy patch makes a large improvement in sendfile\nefficiency.  But read() and send() performance is decreased by 10% -\nmore than this if you factor out the constant device driver overhead.\n\n\nTCP_CORK makes no difference.  The files being sent are much larger\nthan a single frame.  If you force small writes with `zcc -S -b 64'\nthen TCP_CORK saves some CPU at the receiver, but total throughput\nfalls a little.  More work needed here.\n\n\nWhen using TCP_CORK on send(), packets-per-second remained unchanged. \nThis is due to the large size of the writes, and is due to the\nnetworking stack's ability to coalesce data from separate calls to\nsys_send() into single frames.\n\n\nWithout the zc patch, there is a significant increase in the number of\nRx packets (acks, persumably) when data is sent using sendfile() as\nopposed to when the same data is sent with send().\n\n  Workload: 62 files, average size 350k.\n            sendfile() tries to send the entire file in one hit\n            send() breaks it up into 64kbyte chunks.\n\nWhen the zerocopy patch is applied, the Rx packet rate during\nsendfile() is the same as the rate during send().\n\n\neepro100 generates more interrupts doing TCP Tx, but not TCP Rx.  The\ndriver doesn't do Tx interrupt mitigation.\n\nChanging eepro100 to use IO operations instead of MMIO slows down this\ndual 500MHz machine by less than one percent at 100 mbps.\n\n\nConclusions:\n\n  For a NIC which cannot do scatter/gather/checksums, the zerocopy\n  patch makes no change in throughput in all case.\n\n  For a NIC which can do scatter/gather/checksums, sendfile()\n  efficiency is improved by 40% and send() efficiency is decreased by\n  10%.  The increase and decrease caused by the zerocopy patch will in\n  fact be significantly larger than these two figures, because the\n  measurements here include a constant base load caused by the device\n  driver.\n\n\nsend() results, against transfer buffer size\n============================================\n\n                  2.4.1-pre10+zc    2.4.1-pre10+zc     2.4.1-pre10+zc\n                   SG enabled         SG enabled        SG disabled\n                    MAXPGS=8           MAXPGS=1\n\n256                  34.9%               35.3%            32.3%\n512                  28.7%               29.0%            26.6%\n1024                 25.5%               25.8%            24.1%\n2048                 21.3%               21.6%            20.2%\n4096                 20.0%               20.2%            19.0%\n8192                 19.1%               19.4%            18.0%\n16384                19.7%               20.0%            18.7%\n32768                21.0%               21.6%            20.0%\n65536                24.3%               24.8%            22.2%\n\n\nNFS/UDP client results\n======================\n\nReading a 100 meg file across 100baseT.  The file is fully cached on\nthe server.  The client is the above machine.  You need to unmount the\nserver between runs to avoid client-side caching.\n\nThe server is mounted with various rsize and wsize options.\n\n  Kernel           rsize wsize   mbyte/sec     CPU\n\n  2.4.1-pre10+zc   1024  1024     2.4         10.3%\n  2.4.1-pre10+zc   2048  2048     3.7         11.4%\n  2.4.1-pre10+zc   4096  4096     10.1        29.0%\n  2.4.1-pre10+zc   8199  8192     11.9        28.2%\n  2.4.1-pre10+zc  16384 16384     11.9        28.2%\n\n  2.4.1-pre10      1024  1024      2.4         9.7%\n  2.4.1-pre10      2048  2048      3.7        11.8%\n  2.4.1-pre10      4096  4096     10.7        33.6%\n  2.4.1-pre10      8199  8192     11.9        29.5%\n  2.4.1-pre10     16384 16384     11.9        29.2%\n\nSmall diff at 8192.\n\n\nNFS/UDP server results\n======================\n\nReading a 100 meg file across 100baseT.  The file is fully cached on\nthe server.  The server is the above machine.\n\n  Kernel           rsize wsize   mbyte/sec     CPU\n\n  2.4.1-pre10+zc   1024  1024      2.6        19.1%\n  2.4.1-pre10+zc   2048  2048      3.9        18.8%\n  2.4.1-pre10+zc   4096  4096     10.0        34.5%\n  2.4.1-pre10+zc   8199  8192     11.8        28.9%\n  2.4.1-pre10+zc  16384 16384     11.8        29.0%\n\n  2.4.1-pre10      1024  1024      2.6        18.5%\n  2.4.1-pre10      2048  2048      3.9        18.6%\n  2.4.1-pre10      4096  4096     10.9        33.8%\n  2.4.1-pre10      8199  8192     11.8        29.0%\n  2.4.1-pre10     16384 16384     11.8        29.0%\n\nNo diff.\n\nLocal networking\n================\n\n./zcs\n./zcc -s 0 /usr/local/bin/* -n100\nmount -t nfs localhost:/usr/src /mnm -o rsize=8192,wsize=8192\n\n                                sendfile()     send(8k)   NFS\n\nNo explicit bonding\n  2.4.1:                          66600        70000     25600\n  2.4.1-zc:                      208000        69000     25000\n\nsems:                             69000        70000     25200\n                                  71800\n\nBond client and server to separate CPUs\n  2.4.1:                          66700        68000     27800\n  2.4.1-zc:                      213047        66000     25700\n\nBond client and server to same CPU:\n  2.4.1:                          56000        57000     23300\n  2.4.1-zc:                      176000        55000     22100\n\nttcp-sf Results\n===============\n\nJamal Hadi Salim has taught ttcp to use sendfile.  See\nhttp://www.cyberus.ca/~hadi/ttcp-sf.tar.gz\n\nUsing the same machine as above, and the following commands:\n\nSender:    ./ttcp-sf -t -c -l 32768 -v receiver_host\nReceiver:  ./ttcp-sf -c -r -l 32768 -v sender_host\n\n                                                        CPU\n\n    2.4.1-pre10-zerocopy, sending with ttcp-sf:        10.5%\n    2.4.1-pre10-zerocopy, receiving with ttcp-sf:      16.1%\n\n    2.4.1-pre10-vanilla, sending with ttcp-sf:         18.5%\n    2.4.1-pre10-vanilla, receiving with ttcp-sf:       16.0%\n\n\ncyclesoak\n=========\n\n`cyclesoak' calculates CPU load by a subtractive method: a background\ncycle-soaking task is executed on all CPUs and `cyclesoak' measures how\nmuch the throughput of the background tasks is degraded by running the\ntraffic.\n\nThis means that ALL effects of networking (or other userspace + kernel\nactivity) are measured - interrupt load, softirq handling, memory\nbandwidth usage, etc.  This is much more accurate than using Linux\nprocess accounting.\n\nBefore `cyclesoak' can be used, it must be calibrated with the\n`cyclesoak -C -N nr_cpus' option.  You need to tell `cyclesoak' how\nmany CPUs you have each time it is used.  If you get this wrong during\na benchmark run, you may get wrong results.\n\n`cyclesoak' sets the nice level of its soaker as low as possible.  But\nit is still possible for `cyclesoak' to steal CPU from the thing which\nyou are trying to measure.  For this reason it may be necessary to run\n`zcc' or `zcs' with \"Round Robin\" scheduling policy.  This will ensure\nthat `zcc' or `zcs' get all the CPU they need, and ensures that CPU\nload measurements are exact.\n\nIn practice, SCHED_RR is not needed by `zcc' and `zcs' because they can\nget all the foreground CPU they need.\n\nBe careful using SCHED_RR processes! You can lock your system if you\nrun a SCHED_RR process which doesn't terminate.\n\nYou should always run these tools on otherwise-unloaded systems.\n\nUsage: run_rr\n=============\n\n`run_rr' runs another command with SCHED_RR policy.  It requires root\npermissions.  When run with no arguments, `run_rr' starts an\ninteractive shell which has SCHED_RR policy.  All the shell's children\nwill also have SCHED_RR policy.\n\nUsage: cyclesoak\n================\n\nThis is the CPU load monitor.\n\nUsage: cyclesoak [-Cdh] [-N nr_cpus] [-p period]\n\n  -C:      Calibrate CPU load\n\n           This option must be used (on an unloaded system)\n           before `cyclesoak' can be used.\n\n           It sets a baseline for the throughput of the\n           cycle-soaker tasks and writes it to \"./counts_per_sec\".\n\n  -d:      Debug (more d's, more fun)\n\n  -h:      This message\n\n  -N:      Tell cyclesoak how many CPUs you have\n\n           This is compulsory when running `cyclesoak' to\n           measure load or when using `-C'.\n\n  -p:      Set the load sampling period (seconds)\n\n  -C:      Calibrate CPU load\n\n\nUsage: zcs\n==========\n\nzcs is the server.  zcs listens on a port for connections from zcc.  It\nreceives the files which zcc sends.\n\nUsage: zcs [-dh] [-B cpu] [-p port] [-i input_bufsize] [-o output_bufsize]\n\n  -B:      Bond zcs to the indicated CPU.  Requires root permissions.\n\n  -d:      Debug (more d's, more fun)\n\n  -h:      This message\n\n  -p:      TCP port to listen on\n\n           Set the port on which `zcs' listens for\n           connections.  Default is 2222.\n\n  -i:      Set TCP receive buffer size (bytes)\n\n           This option allows you to alter the size of thr TCP receive\n           buffer.  I haven't played with this.  If you want to set it\n           really big or really small you'll need to alter\n           /proc/sys/net/rmem_max\n\n  -o:      Set TCP send buffer size (bytes)\n\n           This option allows you to alter the size of the\n           TCP transmit buffer.  I haven't played with this.  If you\n           want to set it really big or really small you'll need to\n           alter /proc/sys/net/wmem_max\n\nUsage: zcc\n==========\n\nzcc is the client.  It sends a bunch of files to `zcs'.  Of course, you\ndon't want to just measure the throughput of your disks, so you should\nsend the files multiple times using the `-n' option.  Make sure the\nentire working set fits into main memory, and ignore the initial\nthroughput and CPU load results.\n\n\n\nUsage: zcc [-cCdkhSw] [-b buffer_size] [-B cpu] [-i input_bufsize] [-o output_bufsize]\n           [-n n_loops] [-N nr_cpus] [-p port] -s server filename[s]\n\n  -b:      For read()/write() or read()/send() mode, set the\n           amount of memory which is used to buffer transfers, in\n           bytes.  `zcc' reads this many bytes from disk and then\n           writes them to the network and then does this again for each\n           file.\n\n  -B:      Bond zcc to the indicated CPU.  Requires root permissions.\n\n  -d:      Debug (more d's, more fun)\n\n  -h:      This message\n\n  -k:      Use the TCP_CORK option on the outgoing connection.\n\n  -i:      Set TCP receive buffer size (bytes)\n  -o:      Set TCP send buffer size (bytes)\n\n           Same as with zcs\n\n  -n:      Send the fileset this many times\n\n           Loop across the files this many times.\n\n  -p:      TCP port to connect to\n\n           Should match the option given to `zcs'.  Default is 2222.\n\n  -s:      Server to connect to\n\n           A hostname or an IP address.\n\n  -S:      Don't use sendfile: use read/send instead\n\n           Use read() and write() or read() and send() into\n           a userspace buffer, rather than sendfile().\n\n  -w:      When using `-S', use write() to write to the socket,\n           rather than send().\n\n  -N:      Tell zcc how many CPUs you have\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkbingham%2Fzc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkbingham%2Fzc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkbingham%2Fzc/lists"}