Original link: https://colobu.com/2023/04/22/batch-read-and-write-udp-packets-in-Go/
Although the network protocol stack provides a wealth of functions, allowing us to easily realize network data exchange, sometimes we are not so satisfied with the performance of the protocol stack. In the previous articles, I also introduced efficient processing through XDP and other technologies. The way of network data, but after all, XDP is not so widely used, and it is not so simple to use. If we use the standard library provided by the programming language to realize the reading and writing of data, how can we improve the performance? Today I will introduce a way to read and write data packets in batches.
Intuitively, we can also understand that sending and receiving network packets in batches is more efficient than sending and receiving network packets individually, because in the logic of ordinary single sending and receiving packets, at least one system call (Send/SendTo, Read/ RecvFrom), and in the batch processing mode, one system call can process multiple network packets, so from the theoretical analysis point of view, the batch processing mode is more effective. Not only network processing, but many message queues and data storage will also achieve better performance through batch processing.
In this article, I did not test the performance of batch processing network packets and ordinary processing of a single network packet (maybe I can make up for the 5.1 holiday), someone did a simple test , and found no batch processing belt Of course, I guess his test may be too single or simple. Cloudflare has also done millions of pps tests , and the performance of batch processing is still very good. I think you should also do a performance test based on your scenario when evaluating this technology, so as to ensure whether this technology is suitable for adoption.
The technology of sending and receiving packets in batches I mentioned is realized through system calls sendmmsg and recvmmsg , which currently only support Linux systems. As described in the man manual, they are system calls to send and receive multiple packets on the socket:
- sendmmsg – send multiple messages on a socket:
int sendmmsg(int sockfd, struct mmsghdr *msgvec, unsigned int vlen,int flags); - recvmmsg – receive multiple messages on a socket
int recvmmsg(int sockfd, struct mmsghdr *msgvec, unsigned int vlen,int flags, struct timespec *timeout);
The version of these two system calls started to be added to Linux is 3.0, and glibc has been added since version 2.14. OpenBSD 7.2 also added this syscall.
Note that recvmmsg is a blocking system call and will not return until it receives vlen messages or times out.
These two system calls are extensions to sendmsg and recvmsg . If you have studied before, you may know that there are multiple system calls send , sendto , sendmsg , sendmmsg and read , recv , ,
Man send and man recv have made a detailed introduction to them respectively.
- send: The send function is used to send a data packet, which can be a TCP connection or a UDP datagram. Similar to
write, except thatwritehas no flag setting. - sendto: The sendto function is similar to the send function, but it can specify the address of the receiver when sending the data packet. If it is a connection-oriented protocol such as TCP, dest_addr and addrlen can be ignored, otherwise such as UDP, these two parameters need to be specified.
- sendmsg: The sendmsg function can send data in multiple buffers. At the same time, it can also specify one or more additional data. Need to specify
- sendmmsg: The sendmmsg function can send multiple messages in one call, and each message can have one or more buffers. This can reduce the number of system calls and thus improve efficiency.
Similarly, the system calls for receiving mainly include the following:
- recv: recv is the most basic receive function, which receives data from the socket and returns the number of bytes received. It receives data without additional information (such as destination address, etc.). Similar to
read, except thatreadhas no flag setting. - recvfrom: recvfrom also receives data from the socket, but it also returns the address information of the sender, which is suitable for protocols with address information such as UDP.
- recvmsg: recvmsg can receive other related data information (such as whether the received data is truncated, the IP address of the sender, etc.) while receiving data. It supports the reception of multiple data buffers and also the reception of control messages (cmsg).
- recvmmsg: recvmmsg is a multi-message version of recvmsg, which can receive multiple messages at the same time, and is suitable for high-concurrency and high-throughput scenarios.
Corresponding to the conn method of the standard library, take UDPConn as an example:
- conn.Write(_) : use
writesystem call - conn.WriteTo(_, _) : use
sendtosystem call - conn.WriteToUDP(_, _) : use
sendtosystem call - conn.WriteMsgUDP(_, _, _) : use
sendmsgsystem call - conn.WriteMsgUDPAddrPort(_, _, _) : use
sendmsgsystem call - conn.Read(nil): use
readsystem call - conn.ReadFrom(nil): Use
recvfromsystem call - conn.ReadMsgUDP(nil, nil): use
recvmsgsystem call - conn.ReadMsgUDPAddrPort(nil, nil): use
recvmsgsystem call - conn.ReadFromUDP(nil): Use
recvmsgsystem call - conn.ReadFromUDPAddrPort(nil): Use
recvmsgsystem call
Unfortunately, the Go standard library does not provide wrappers for the system calls sendmmsg and recvmmsg . This system call is not even defined in syscall , so there is no corresponding batch processing method like net.UDPConn . In 2021, in go issue#45886 , bradfitz proposed to add a batch of read and write messages to *UDPConn . This proposal was approved by Russ Cox as an accepted status on November 10, 2022, but no one has yet accepted this proposal. accomplish.
However, go’s extension library golang.org/x/net provides the methods of ReadBatch and WriteBatch , which can provide methods for reading and writing messages in batches. It is actually an encapsulation of the system calls readmmsg and sendmmsg .
Of course, you can also implement the encapsulation of system calls in a similar way:
|
1
2
3
4
5
6
7
8
9
|
func recvmmsg(s uintptr , hs []mmsghdr, flags int ) ( int , error) {
n, _, errno := syscall.Syscall6(sysRECVMMSG, s, uintptr (unsafe.Pointer(&hs [0 ])), uintptr ( len (hs)), uintptr (flags), 0 , 0 )
return int (n), errnoErr(errno)
}
func sendmmsg(s uintptr , hs []mmsghdr, flags int ) ( int , error) {
n, _, errno := syscall. Syscall6(sysSENDMMSG, s, uintptr (unsafe. Pointer(&hs [0 ])), uintptr ( len (hs)), uintptr (flags), 0 , 0 )
return int (n), errnoErr(errno)
}
|
But currently they all have a problem, similar to ReadBatch is blocking, if not enough messages are received, the current thread will be blocked. There is no problem when the number of threads is small, but if the number of threads is large, there will be insufficient resources and performance problems. In the proposal, bradfitz hopes to integrate with the net poller of the standard library, so as to prevent threads from being blocked, so you can expect this function to be realized as soon as possible.
So, let’s take a look at the best way to read and write UDP messages in batches at present, that is, the way to use ipv4 packets.
Use ipv4.PacketConn
We can use the way of ipv4.PacketConn , which provides ReadBatch and WriteBatch methods:
- func (c *PacketConn) ReadBatch(ms []Message, flags int) (int, error): read a batch of messages, it returns the number of messages read, the maximum is len(ms)
- func (c *PacketConn) WriteBatch(ms []Message, flags int) (int, error): write a batch of messages, it returns the number of successfully written messages
Next, we use an example of a UDP client and server to demonstrate the ability to read and write in batches.
Below is the code for the client. It first creates an instance of *net.UDPConn using the standard library, and converts it to *ipv4.PacketConn ipv4.NewPacketConn using ipv4.NewPacketConn.
Next, 10 messages are prepared, which need to be prepared as ipv4.Message type. If you use it in a product, it is best to use Pool to pool these objects. This example is relatively simple, without considering performance issues, and it is worth demonstrating the function of batch reading and writing.
After the data is ready, call WriteBatch to write in batches. Here, we also consider how to deal with each write if some are successful, assuming that the write is successful.
Next, read the return packet, assuming that the server will return every message, so if the batch reading is not finished here, it will continue to read until all the messages are read back.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
twenty one
twenty two
twenty three
twenty four
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
|
package main
import (
“fmt”
“net”
“golang.org/x/net/ipv4”
)
func main() {
remote, err := net.ResolveUDPAddr( “udp” , “localhost:9999” )
if err != nil {
panic (err)
}
conn, err := net.Dial( “udp” , “localhost:9999” )
if err != nil {
panic (err)
}
defer conn. Close()
pconn := ipv4.NewPacketConn(conn.(*net.UDPConn))
// write with a batch of 10 messages
batch := 10
msgs := make ([]ipv4.Message, batch)
for i := 0 ; i < batch; i++ {
msgs[i] = ipv4.Message{
Buffers: [][] byte {[] byte (fmt. Sprintf( “hello batch %d” , i))},
Addr: remote,
}
}
n, err := pconn.WriteBatch(msgs, 0 )
if err != nil {
panic (err)
}
fmt.Printf( “sent %d messages\n” , n)
// read 10 messages with batch
count := 0
for count < batch {
n, err := pconn. ReadBatch(msgs, 0 )
if err != nil {
panic (err)
}
count += n
for i := 0 ; i < n; i++ {
fmt.Println( string (msgs[i].Buffers [0 ]))
}
}
}
|
The code on the server side is similar, first receive messages in batches, and then write the messages back in batches as they are:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
twenty one
twenty two
twenty three
twenty four
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
|
package main
import (
“fmt”
“net”
“golang.org/x/net/ipv4”
)
func main() {
addr, err := net.ResolveUDPAddr( “udp” , “:9999” )
if err != nil {
panic (err)
}
conn, err := net.ListenUDP( “udp” , addr)
if err != nil {
panic (err)
}
defer conn. Close()
pconn := ipv4.NewPacketConn(conn)
fmt.Println( “server listening on” , addr)
batch := 10
msgs := make ([]ipv4.Message, batch)
for i := 0 ; i < batch; i++ {
msgs[i] = ipv4.Message{
Buffers: [][] byte { make ([] byte , 1024 )},
}
}
for {
n, err := pconn. ReadBatch(msgs, 0 )
if err != nil {
panic (err)
}
tn := 0
for tn < n {
nn, err := pconn. WriteBatch(msgs[tn:n], 0 )
if err != nil {
panic (err)
}
tn += nn
}
}
}
|
Run the server and client, and you can see that all 10 messages have been received on the client:
|
1
2
3
4
5
6
7
8
9
10
11
12
|
ubuntu@lab:~/network-programming/ch04/mmsg/sendmmsg/client$ ./client
sent 10 messages
hello batch 0
hello batch 1
hello batch 2
hello batch 3
hello batch 4
hello batch 5
hello batch 6
hello batch 7
hello batch 8
hello batch 9
|
Use ipv4.Conn
In fact, using ipv4.Conn can also achieve batch reading and writing, and the bottom layer is the same. The bottom layer of this type is golang.org/x/net/internal/socket.Conn , which contains the following methods for batch reading and writing messages:
- SendMsgs(ms []Message, flags int) (int, error): Write messages in batches, wrap the system call
sendmmsg, and return the number of successfully sent messages - RecvMsgs(ms []Message, flags int) (int, error): Read messages in batches, wrap the system call
recvmmsg, and return the number of successfully received messages
The usage is almost the same as above, but the method name is different. Here is the client code:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
twenty one
twenty two
twenty three
twenty four
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
package main
import (
“fmt”
“net”
“golang.org/x/net/ipv4”
)
func main() {
remote, err := net.ResolveUDPAddr( “udp” , “localhost:9999” )
if err != nil {
panic (err)
}
conn, err := net.Dial( “udp” , “localhost:9999” )
if err != nil {
panic (err)
}
defer conn. Close()
pconn := ipv4.NewConn(conn)
// write with a batch of 10 messages
batch := 10
msgs := make ([]ipv4.Message, batch)
for i := 0 ; i < batch; i++ {
msgs[i] = ipv4.Message{
Buffers: [][] byte {[] byte (fmt. Sprintf( “hello batch %d” , i))},
Addr: remote,
}
}
n, err := pconn. SendMsgs(msgs, 0 )
if err != nil {
panic (err)
}
fmt.Printf( “sent %d messages\n” , n)
// read 10 messages with batch
count := 0
for count < batch {
n, err := pconn.RecvMsgs(msgs, 0 )
if err != nil {
panic (err)
}
count += n
for i := 0 ; i < n; i++ {
fmt.Println( string (msgs[i].Buffers [0 ]))
}
}
}
|
Here is the server code:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
twenty one
twenty two
twenty three
twenty four
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
|
package main
import (
“fmt”
“net”
“golang.org/x/net/ipv4”
)
func main() {
addr, err := net.ResolveUDPAddr( “udp” , “:9999” )
if err != nil {
panic (err)
}
conn, err := net.ListenUDP( “udp” , addr)
if err != nil {
panic (err)
}
defer conn. Close()
pconn := ipv4.NewConn(conn)
fmt.Println( “server listening on” , addr)
batch := 10
msgs := make ([]ipv4.Message, batch)
for i := 0 ; i < batch; i++ {
msgs[i] = ipv4.Message{
Buffers: [][] byte { make ([] byte , 1024 )},
}
}
for {
n, err := pconn.RecvMsgs(msgs, 0 )
if err != nil {
panic (err)
}
tn := 0
for tn < n {
nn, err := pconn. SendMsgs(msgs[tn:n], 0 )
if err != nil {
panic (err)
}
tn += nn
}
}
}
|
This article is transferred from: https://colobu.com/2023/04/22/batch-read-and-write-udp-packets-in-Go/
This site is only for collection, and the copyright belongs to the original author.