Original link: https://wusiyu.me/mikrotik-ccr2004-1g-2xs-pcie-%E7%AE%80%E5%8D%95%E5%B8%A6%E5%AE%BD%E6%B5% 8B%E8%AF%95/
CCR2004-1G-2XS-PCIe is an interesting device. To be precise, it is a “router in the form of a PCIe network card”. Unlike ordinary routers, it has two SFP28 25G network ports on the baffle and one RJ45 thousand The “physical network ports” of the mega management port also have 4 25G virtual network interfaces directly connected to the host through PCIe 3.0 x8. From the perspective of the host, this is equivalent to 4 “normal” network interfaces that are directly connected to the router. ; Unlike some Smart NICs, CCR2004-1G-2XS-PCIe does not provide any offload function for the host, and functions such as RDMA and sr-iov should also not be supported, nor does it have a built-in switch chip, all from the host to the host. The data of the physical network port needs to be forwarded by the CPU of CCR2004-1G-2XS-PCIe. In other words, if you just use it as a network card, it is not as good as a normal network card. Its reasonable use should be to form a server and router. All-in-one solution. However, whether it is used as an “integrated router” or a “common network card”, it is worth our practice test, which is the next part of this article.
Since there is no 25G environment, this article is tested under the 10G intranet. The CCR2004-1G-2XS-PCIe is connected to the CRS305 10G switch through SFP+ DAC, and the other machine on the switch uses the x540 dual 10G electrical port network card. CCR2004-1G-2XS-PCIe is installed on a PCIe 3.0 x16 slot (CPU direct connection channel) of a Core i9-10850k mini-host (hereinafter referred to as host), and the system is Fedora Workstation 36.
Some excerpts of information about this device on the host:
# lspci 01:00.0 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet 01:00.1 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet 01:00.2 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet 01:00.3 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet # dmesg [ 1.971577] atl1c 0000:01:00.0: enabling device (0000 -> 0002) [ 1.994014] atl1c 0000:01:00.1: enabling device (0000 -> 0002) [ 2.014003] atl1c 0000:01:00.2: enabling device (0000 -> 0002) [ 2.034983] atl1c 0000:01:00.3: enabling device (0000 -> 0002) [ 2.057479] atl1c 0000:01:00.1 enp1s0f1: renamed from eth1 [ 2.073979] atl1c 0000:01:00.0 enp1s0f0: renamed from eth0 [ 2.090921] atl1c 0000:01:00.2 enp1s0f2: renamed from eth2 [ 2.105991] atl1c 0000:01:00.3 enp1s0f3: renamed from eth3 [ 8.700933] atl1c 0000:01:00.0: atl1c: enp1s0f0 NIC Link is Up<25000 Mbps Full Duplex> [ 8.703098] atl1c 0000:01:00.1: atl1c: enp1s0f1 NIC Link is Up<25000 Mbps Full Duplex> [ 8.705370] atl1c 0000:01:00.2: atl1c: enp1s0f2 NIC Link is Up<25000 Mbps Full Duplex> [ 8.707589] atl1c 0000:01:00.3: atl1c: enp1s0f3 NIC Link is Up<25000 Mbps Full Duplex> # ethtool enp1s0f0 Settings for enp1s0f0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Speed: 25000Mb/s Duplex: Full Auto-negotiation: on Port: Twisted Pair PHYAD: 0 Transceiver: internal MDI-X: Unknown Supports Wake-on: pg Wake-on: d Current message level: 0x0000003f (63) drv probe link timer ifdown ifup Link detected: yes # ethtool -i enp1s0f0 driver: atl1c version: 5.17.9-300.fc36.x86_64 firmware-version: expansion-rom-version: bus-info: 0000:01:00.0 supports-statistics: no supports-test: no supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no
It can be seen that it uses the atl1c driver under Linux, and the connection speed and function are normal, but the model recognition under lspci is problematic.
Switching performance test (“soft switch” mode)
In this mode, all interfaces are in a bridge, only the first virtual network port is enabled on the host, and an SFP28 port is connected to the room switch. remote is the host on the room switch. The upper limit of the bandwidth of the test line is 10G. Here, iperf3 is used for the test, that is, the large packet test. The Fast Forward of RouterOS is enabled by default.
| test | describe | bandwidth | CPU usage |
| #1 | One-way host -> remote | 9.39 Gbits/sec | 15% |
| #2 | One-way remote -> host | 9.41 Gbits/sec | 30% |
The CPU usage in the table refers to CCR2004-1G-2XS-PCIe. The default RouterOS is to enable Fast Forward of the bridge, and the result does not change significantly after manually closing it.
Routing + NAT performance test (“integrated router” mode)
In this mode, the common home router environment is simulated, but the WAN side uses DHCP to connect to the room switch through an SFP28 interface, obtain the address from the upper router, and the remote host is also on the room switch. The other interfaces are in a bridge, which is the LAN side. Like ordinary home routers, the LAN side has its own address segment, turn on the DHCP server, and turn on the NAT masquerade on the WAN.
The first is the performance when Fast Path is turned on:
| test | describe | bandwidth | CPU usage |
| #1 | One-way host -> remote | 9.35 Gbits/sec | twenty one% |
| #2 | One-way remote -> host | 9.37 Gbits/sec | 31% |
Then there is the performance when Fast Path is turned off:
| test | describe | bandwidth | CPU usage |
| #1 | One-way host -> remote | 9.37 Gbits/sec | 28% |
| #2 | One-way remote -> host | 2.80 Gbits/sec | 29% |
It can be seen that after closing Fast Path, remote -> host, that is, the performance in the WAN to LAN direction is not good. Although the CPU occupancy rate has not reached 100% at this time, only one of the 4 cores has a utilization rate close to 100%. It is estimated that there is a single-core bottleneck. The CPU occupancy at this time is as follows:
The post Mikrotik CCR2004-1G-2XS-PCIe Simple Bandwidth Test appeared first on WuSiYu Blog .
This article is reproduced from: https://wusiyu.me/mikrotik-ccr2004-1g-2xs-pcie-%E7%AE%80%E5%8D%95%E5%B8%A6%E5%AE%BD%E6%B5% 8B%E8%AF%95/
This site is for inclusion only, and the copyright belongs to the original author.