https://github.com/withlin/xdp-redirect-lab
xdp redirect veth pair conn lab
https://github.com/withlin/xdp-redirect-lab
Last synced: 4 months ago
JSON representation
xdp redirect veth pair conn lab
- Host: GitHub
- URL: https://github.com/withlin/xdp-redirect-lab
- Owner: withlin
- Created: 2021-12-02T12:33:44.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2022-04-06T11:02:59.000Z (about 4 years ago)
- Last Synced: 2024-12-28T18:32:40.441Z (over 1 year ago)
- Size: 3.91 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# xdp-redirect-lab
xdp redirect veth pair conn lab
### XDP REDIRECT
# iperf1
```
172.17.0.2
vethd32424b
02:42:ac:11:00:02
./xdp_loader --dev vethd32424b --force --progsec xdp_redirect_map
./xdp_prog_user -d vethd32424b -r veth70d1189 --src-mac 02:42:ac:11:00:02 --dest-mac 02:42:ac:11:00:03
```
# iperf3
```
172.17.0.3
veth70d1189
02:42:ac:11:00:03
./xdp_loader --dev veth70d1189 --force --progsec xdp_redirect_map
./xdp_prog_user -d veth70d1189 -r vethd32424b --src-mac 02:42:ac:11:00:03 --dest-mac 02:42:ac:11:00:02
```
### Lab
### 没使用xdp加速
```
/ $ iperf3 -c 172.17.0.3
Connecting to host 172.17.0.3, port 5201
[ 5] local 172.17.0.2 port 40596 connected to 172.17.0.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 4.05 GBytes 34.8 Gbits/sec 388 930 KBytes
[ 5] 1.00-2.00 sec 3.98 GBytes 34.2 Gbits/sec 449 940 KBytes
[ 5] 2.00-3.00 sec 4.05 GBytes 34.8 Gbits/sec 493 963 KBytes
[ 5] 3.00-4.00 sec 3.71 GBytes 31.8 Gbits/sec 3 994 KBytes
[ 5] 4.00-5.00 sec 4.02 GBytes 34.5 Gbits/sec 3 1010 KBytes
[ 5] 5.00-6.00 sec 4.08 GBytes 35.1 Gbits/sec 5 1020 KBytes
[ 5] 6.00-7.00 sec 3.95 GBytes 34.0 Gbits/sec 2 1.01 MBytes
[ 5] 7.00-8.00 sec 4.01 GBytes 34.5 Gbits/sec 2 1.01 MBytes
[ 5] 8.00-9.00 sec 3.67 GBytes 31.5 Gbits/sec 5 1.01 MBytes
[ 5] 9.00-10.00 sec 3.91 GBytes 33.5 Gbits/sec 0 1.03 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 39.4 GBytes 33.9 Gbits/sec 1350 sender
[ 5] 0.00-10.00 sec 39.4 GBytes 33.9 Gbits/sec receiver
```
#### 使用XDP加速后
```
iperf Done.
/ $ iperf3 -c 172.17.0.3
Connecting to host 172.17.0.3, port 5201
[ 5] local 172.17.0.2 port 40722 connected to 172.17.0.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 782 MBytes 6.56 Gbits/sec 0 269 KBytes
[ 5] 1.00-2.00 sec 772 MBytes 6.48 Gbits/sec 0 426 KBytes
[ 5] 2.00-3.00 sec 796 MBytes 6.68 Gbits/sec 0 426 KBytes
[ 5] 3.00-4.00 sec 784 MBytes 6.58 Gbits/sec 104 199 KBytes
[ 5] 4.00-5.00 sec 719 MBytes 6.02 Gbits/sec 6 197 KBytes
[ 5] 5.00-6.00 sec 775 MBytes 6.50 Gbits/sec 0 225 KBytes
[ 5] 6.00-7.00 sec 573 MBytes 4.81 Gbits/sec 53 269 KBytes
[ 5] 7.00-8.00 sec 821 MBytes 6.89 Gbits/sec 0 448 KBytes
[ 5] 8.00-9.00 sec 650 MBytes 5.44 Gbits/sec 376 218 KBytes
[ 5] 9.00-10.00 sec 815 MBytes 6.83 Gbits/sec 0 249 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 7.31 GBytes 6.28 Gbits/sec 539 sender
[ 5] 0.00-10.00 sec 7.31 GBytes 6.28 Gbits/sec receiver
```
#### XDP status
```
XDP-action
XDP_ABORTED 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000435
XDP_DROP 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000441
XDP_PASS 659425 pkts ( 0 pps) 43524 Kbytes ( 0 Mbits/s) period:2.000438
XDP_TX 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000434
XDP_REDIRECT 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000429
```
### 尝试关掉一些一些参数
```
root@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/ff0a5996e2d2 ethtool -K eth0 tso offroot@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/ff0a5996e2d2 ethtool -K eth0 lro offroot@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/1eebaa5a17b4 ethtool -K eth0 tso offroot@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/1eebaa5a17b4 ethtool -K eth0 lro off
root@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/1eebaa5a17b4 ethtool -K eth0 gso off
root@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/1eebaa5a17b4 ethtool -K eth0 gro off
root@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/ff0a5996e2d2 ethtool -K eth0 gso offroot@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting# nsenter --net=/var/run/docker/netns/ff0a5996e2d2 ethtool -K eth0 gro offroot@ubuntu-hirsute:~/withlin/xdp-tutorial/packet03-redirecting#
```
#### 总结
> 加速 加了个寂寞
xdp在 veth pair 上会导致TSO 停止工作，数据包被线性化，导致TCP包在两个 veth之前吞吐量降低。该问题还在内核修复阶段，XDP_REDIRECT 在veth设备之间只会变的更慢。。。