{"id":13524146,"url":"https://github.com/mtcp-stack/mtcp","last_synced_at":"2026-03-14T06:02:53.898Z","repository":{"id":15576617,"uuid":"18312122","full_name":"mtcp-stack/mtcp","owner":"mtcp-stack","description":"mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems","archived":false,"fork":false,"pushed_at":"2024-07-04T03:20:20.000Z","size":123163,"stargazers_count":2046,"open_issues_count":97,"forks_count":444,"subscribers_count":152,"default_branch":"master","last_synced_at":"2025-04-11T15:57:00.407Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"coreos/coreos-github","license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mtcp-stack.png","metadata":{"files":{"readme":"README","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-04-01T00:00:32.000Z","updated_at":"2025-04-11T03:16:53.000Z","dependencies_parsed_at":"2022-07-20T04:18:16.063Z","dependency_job_id":"bf09d549-3ef7-48e3-bd08-8247a958d90d","html_url":"https://github.com/mtcp-stack/mtcp","commit_stats":{"total_commits":331,"total_committers":24,"mean_commits":"13.791666666666666","dds":0.4712990936555891,"last_synced_commit":"0463aad5ecb6b5bca85903156ce1e314a58efc19"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtcp-stack%2Fmtcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtcp-stack%2Fmtcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtcp-stack%2Fmtcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtcp-stack%2Fmtcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mtcp-stack","download_url":"https://codeload.github.com/mtcp-stack/mtcp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270646,"owners_count":22042859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T06:01:07.479Z","updated_at":"2025-12-16T19:06:15.396Z","avatar_url":"https://github.com/mtcp-stack.png","language":"C","readme":"========================================================================\n README for mTCP\n========================================================================\n\nmTCP is a highly scalable user-level TCP stack for multicore systems. \nmTCP source code is distributed under the Modified BSD License. For \nmore detail, please refer to the LICENSE. The license term of io_engine \ndriver and ported applications may differ from the mTCP’s.\n\n========================================================================\n PREREQUISITE\n========================================================================\n\nWe require the following libraries to run mTCP.\n - libps (PacketShader I/O engine library) OR libdpdk (Intel's DPDK package*) OR netmap driver\n - libnuma\n - libpthread\n - librt\n - libgmp (for DPDK/ONVM driver)\n Compling PSIO/DPDK/NETMAP/ONVM driver requires kernel headers.\n  - For Debian/Ubuntu, try apt-get install linux-headers-$(uname -r)\n\n\n* We have modified the dpdk package to export net_device stat data \n(for Intel-based Ethernet adapters only) to the OS. To achieve this, we have\ncreated a new LKM dpdk-iface-kmod. We also\nmodified mk/rte.app.mk file to ease the compilation process of mTCP applications.\nWe recommend using our package for DPDK installation. \n\n=======================================================================\nCCP support\n=======================================================================\n\nUsing CCP(https://ccp-project.github.io/) for congestion control (disabled by\ndefault), requires building and running a CCP algorithm. If you would like to\nenable CCP (ie. use the internal implementation of Reno), simply run configure\nscript with --enable-ccp option.\n\n1. Install Rust. Any installation method should be fine. We recommend using\n   rustup:\n\n    curl https://sh.rustup.rs -sSf | sh -- -y -v --default-toolchain nightly\n\n2. Build a CCP algorithm. The generic-cong-avoid(https://github.com/ccp-project/generic-cong-avoid)\n   package implements standard TCP Reno and Cubic, so this is probably best to\n   start with. The same steps can be followed to build any of the other\n   algorithms hosted in the ccp-project(https://github.com/ccp-project) organization, such as\n   bbr(https://github.com/ccp-project/bbr).\n\n   git clone https://github.com/ccp-project/generic-cong-avoid.git\n   cd generic-cong-avoid\n   cargo +nightly build\n\n3. Later, after you've built mTCP and started an mTCP application (such as\n   epserver or perf), you must start the CCP binary you just built. If you\n   try to start the CCP process *before* running an mTCP application, it will\n   report a \"connection refused\" error.\n   \n\n    cd generic-cong-avoid\n    sudo ./target/debug/reno --ipc unix\n    \n========================================================================\n INCLUDED DIRECTORIES\n========================================================================\n\nmtcp - mtcp source code directory\nmtcp/src - source code\nmtcp/src/include - mTCP’s internal header files\nmtcp/lib - library file\nmtcp/include - header files that applications will use\n\nio_engine - event-driven packet I/O engine (io_engine)\nio_engine/driver - driver source code\nio_engine/lib - io_engine library\nio_engine/include - io_engine header files\nio_engine/samples - sample io_engine applications (not mTCP’s)\n\ndpdk - Intel's Data Plane Development Kit\ndpdk/...\n\napps - mTCP applications\napps/example - example applications (see README)\napps/lighttpd-1.4.32 - mTCP-ported lighttpd (see INSTALL)\napps/apache_benchmark - mTCP-ported apache benchmark (ab) (see README-mtcp)\n\nutil - useful source code for applications\n\nconfig - sample mTCP configuration files (may not be necessary)\n\n========================================================================\n INSTALL GUIDES\n========================================================================\nmTCP can be prepared in three ways.\n\n- PSIO VERSION -\n----------------\n1. make in io_engine/driver:\n   \t   # make\n  - check ps_ixgbe.ko\n  - please note that psio only runs on linux-2.6.x kernels\n    (linux-2.6.32 ~ linux-2.6.38)\n\n2. install the driver:\n   \t   # ./install.py \u003c# cores\u003e \u003c# cores\u003e\n  - refer to http://shader.kaist.edu/packetshader/io_engine/\n  - you may need to change the ip address in install.py:46\n\n3. Setup mtcp library:\n   \t   # ./configure --with-psio-lib=\u003c$path_to_ioengine\u003e\n\t     ## e.g. ./configure --with-psio-lib=`echo $PWD`/io_engine\n   \t   # make\n  - By default, mTCP assumes that there are 16 CPUs in your system.\n    You can set the CPU limit, e.g. on a 8-core system, by using the following command:\n    \t   # ./configure --with-psio-lib=`echo $PWD`/io_engine CFLAGS=\"-DMAX_CPUS=8\"\n    Please note that your NIC should support RSS queues equal to the MAX_CPUS value\n    (since mTCP expects a one-to-one RSS queue to CPU binding).\t   \n  - In case `./configure' script prints an error, run the\n    following command; and then re-do step-3 (configure again):\n    \t   # autoreconf -ivf\n  - check libmtcp.a in mtcp/lib\n  - check header files in mtcp/include\n  - check example binary files in apps/example\n\n4. Check the configurations in apps/example\n  - epserver.conf for server-side configuration\n  - epwget.conf for client-side configuration\n  - you may write your own configuration file for your application\n\n5. Run the applications!\n\n\n- DPDK VERSION -\n----------------\n1. Set up DPDK first.\n\n       # bash setup_mtcp_dpdk_env.sh [\u003cpath to $RTE_SDK]]\n\n   Press [15] to compile x86_64-native-linuxapp-gcc version\n   Press [18] to install igb_uio driver for Intel NICs\n   Press [22] to setup 2048 2MB hugepages\n   Press [24] to register the Ethernet ports\n   Press [35] to quit the tool\n\n  Only those devices will work with DPDK drivers that are listed\n  on this page: http://dpdk.org/doc/nics. Please make sure that your\n  NIC is compatible before moving on to the next step.\n\n  We use dpdk/ as our DPDK driver. FYI, you can pass a different\n  dpdk source directory as command line argument.\n   \n2. Bring the dpdk compatible interfaces up, and then set RTE_SDK\nand RTE_TARGET environment variables. If you are using Intel NICs,\nthe interfaces will have dpdk prefix.\n  \n       # sudo ifconfig dpdk0 x.x.x.x netmask 255.255.255.0 up\n       # export RTE_SDK=`echo $PWD`/dpdk\n       # export RTE_TARGET=x86_64-native-linuxapp-gcc\n\n3. Setup mtcp library:\n\t   # ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET\n   \t   # make\n\n- By default, mTCP assumes that there are 16 CPUs in your system.\n    You can set the CPU limit, e.g. on a 32-core system, by using the following command:\n    \t   # ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET CFLAGS=\"-DMAX_CPUS=32\"\n    Please note that your NIC should support RSS queues equal to the MAX_CPUS value\n    (since mTCP expects a one-to-one RSS queue to CPU binding).\n  - In case `./configure' script prints an error, run the\n    following command; and then re-do step-4 (configure again):\n    \t   # autoreconf -ivf\n  - checksum offloading in the NIC is now ENABLED (by default)!!!\n    \t     - this only works for dpdk at the moment\n\t     - use \n\t       ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET --disable-hwcsum\n\t       to disable checksum offloading.\n  - check libmtcp.a in mtcp/lib\n  - check header files in mtcp/include\n  - check example binary files in apps/example\n\n5. Check the configurations in apps/example\n  - epserver.conf for server-side configuration\n  - epwget.conf for client-side configuration\n  - you may write your own configuration file for your application\n\n6. Run the applications!\n\n7. You can revert back all your changes by running the following script.\n\n       # bash setup_linux_env.sh [\u003cpath to $RTE_SDK]]\n   \n   Press [29] to unbind the Ethernet ports\n   Press [30] to remove igb_uio.ko driver\n   Press [33] to remove hugepage mappings\n   Press [34] to quit the tool\n\n\n- ONVM VERSION -\n----------------\n\n~NEW~: Now you can run mTCP applications (server + client) locally.\nA local setup is useful when only 1 machine is available for the experiment. \nONVM configurations are placed as `.conf` files in apps/example directory.\nONVM basics are explained in https://github.com/sdnfv/openNetVM.\n\n**Before running the applications make sure that onvm_mgr is running.**\nAlso, no core overlap between applications and onvm_mgr is allowed.\n\n1. Install openNetVM using the following instructions \n           https://github.com/sdnfv/openNetVM/blob/master/docs/Install.md\n\n2. Set up the dpdk interfaces:\n\t   # bash setup_mtcp_onvm_env.sh\n\n3. Next bring the dpdk-registered interfaces up. This can be setup using:\n\t   # sudo ifconfig dpdk0 x.x.x.x netmask 255.255.255.0 up\n\n4. Setup mtcp library\n\t   # ./configure --with-dpdk-lib=$\u003cpath_to_dpdk\u003e --with-onvm-lib=$\u003cpath_to_onvm_lib\u003e\n\t   # e.g. ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET --with-onvm-lib=`echo $ONVM_HOME`/onvm\n\t   # make\n\n  - By default, mTCP assumes that there are 16 CPUs in your system.\n    You can set the CPU limit, e.g. on a 32-core system, by using the following command:\n      # ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET --with-onvm-lib=$\u003cpath_to_onvm_lib\u003e CFLAGS=\"-DMAX_CPUS=32\"\n    Please note that your NIC should support RSS queues equal to the MAX_CPUS value\n    (since mTCP expects a one-to-one RSS queue to CPU binding).\n    \n  - In case `./configure' script prints an error, run the\n    following command; and then re-do step-4 (configure again):\n         # autoreconf -ivf\n  - checksum offloading in the NIC is now ENABLED (by default)!!!\n           - this only works for dpdk at the moment\n       - use \n         ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET --disable-hwcsum\n         to disable checksum offloading.\n  - check libmtcp.a in mtcp/lib\n  - check header files in mtcp/include\n  - check example binary files in apps/example\n\n5. Check the configurations in apps/example\n  - epserver.conf for server-side configuration\n  - epwget.conf for client-side configuration\n  - you may write your own configuration file for your application\n\n6. Run the applications!\n\n7. You can revert back all your changes by running the following script.\n\n       # bash setup_linux_env.sh\n   \n   Press [29] to unbind the Ethernet ports\n   Press [30] to remove igb_uio.ko driver\n   Press [33] to remove hugepage mappings\n   Press [34] to quit the tool\n   \n - Notes -\nOnce you have started onvm_mgr, sometimes an mTCP application may fail to get launched due\nto a error(s) resembling the one mentioned below:\n\n  # EAL: FATAL: Cannot init memory \n    OR\n  # Cannot mmap memory for rte_config at [0x7ffff7fb6000], got [0x7ffff7e74000] - please use '--base-virtaddr' option\n    OR\n  # EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:06:00.0/resource3 to address: 0x7ffff7ff1000 \n\nTo prevent this, use the base virtual address parameter to run the ONVM manager(core list arg `0xf8` isn't actually used by mtcp NFs but is required), e.g.:\n\ncd openNetVM/onvm\n./go.sh 1,2,3 1 0xf8 -s stdout -a 0x7f000000000\n\n\n- NETMAP VERSION -\n------------------\nSee README.netmap for details.\n\n========================================================================\n TESTED ENVIRONMENTS\n========================================================================\n\nmTCP runs on Linux-based operating systems (2.6.x for PSIO) with generic \nx86_64 CPUs, but to help evaluation, we provide our tested environments \nas follows.\n\nIntel Xeon E5-2690 octacore CPU @ 2.90 GHz\n32 GB of RAM (4 memory channels)\n10 GbE NIC with Intel 82599 chipset (specifically Intel X520-DA2)\nDebian 6.0.7 (Linux 2.6.32-5-amd64)\n\nIntel Core i7-3770 quadcore CPU @ 3.40 GHz\n16 GB of RAM (2 memory channels)\n10 GbE NIC with Intel 82599 chipset (specifically Intel X520-DA2)\nUbuntu 10.04 (Linux 2.6.32-47)\n\nEvent-driven PacketShader I/O engine (extended io_engine-0.2)\n - PSIO is currently only compatible with Linux-2.6.\n\nWe tested the DPDK version (polling driver) with Linux-3.13.0 kernel.\n========================================================================\n NOTES\n========================================================================\n\n1. mTCP currently runs with fixed memory pools. That means, the size of\n   TCP receive and send buffers are fixed at the startup and does not \n   increase dynamically. This could be performance limit to the large \n   long-lived connections. Be sure to configure the buffer size \n   appropriately to your size of workload.\n\n2. The client side of mTCP supports mtcp_init_rss() to create an \n   address pool that can be used to fetch available address space in \n   O(1). To easily congest the server side, this function should be \n   called at the application startup.\n\n3. The supported socket options are limited for right now. Please refer \n   to the mtcp/src/api.c for more detail.\n\n4. The counterpart of mTCP should enable TCP timestamp.\n\n5. mTCP has been tested with the following Ethernet adapters:\n\n   1. Intel-82598\t\tixgbe\t\t(Max-queue-limit: 16)\n   2. Intel-82599\t\tixgbe\t\t(Max-queue-limit: 16)\n   3. Intel-I350\t\tigb\t\t(Max-queue-limit: 08)\n   4. Intel-X710\t\ti40e\t\t(Max-queue-limit: ~)\n   5. Intel-X722\t\ti40e\t\t(Max-queue-limit: ~)\n \n========================================================================\n FREQUENTLY ASKED QUESTIONS\n========================================================================\n\n1. How can I quit the application?\n  - Use ^C to gracefully shutdown the application. Two consecutive \n    ^C (separated by 1 sec) will force quit.\n\n2. My application doesn't use the address specified from ifconfig.\n  - For some Linux distros(e.g. Ubuntu), NetworkManager may re-assign\n    a different IP address, or delete the assigned IP address.\n\n    Disable NetworkManager temporarily if that's the case.\n    NetworkManager will be re-enabled upon reboot.\n\n        # sudo service network-manager stop\n\n3. Can I statically set the routing or arp table?\n  - Yes, mTCP allows static route and arp configuration. Go to the \n    config directory and see sample_route.conf or sample_arp.conf. \n    Copy and adapt it to your condition and link (ln -s) the config \n    directory to the application directory. mTCP will find \n    config/route.conf and config/arp.conf for static configuration.\n\n========================================================================\n CAUTION\n========================================================================\n\n1. Do not remove I/O driver (ps_ixgbe/igb_uio) while running mTCP \n   applications. The application will panic!\n\n2. Use the ps_ixgbe/dpdk driver contained in this package, not the one \n   from some other place (e.g., from io_engine github).\n\n========================================================================\n\n                   Contact: mtcp-user at list.ndsl.kaist.edu\n                             April 2, 2015. \n                 EunYoung Jeong \u003cnotav at ndsl.kaist.edu\u003e\n\t\t M. Asim Jamshed \u003cajamshed at ndsl.kaist.edu\u003e\n","funding_links":[],"categories":["Networking","C","others","内存分配"],"sub_categories":["网络"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtcp-stack%2Fmtcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmtcp-stack%2Fmtcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtcp-stack%2Fmtcp/lists"}