Common patterns of Go memory leaks

Original link: https://sund.site/posts/2023/goroutine-leak/

Recently, when I was investigating memory leaks in Go language at work, I found this blog written by Uber , which shared several common goroutine memory leak patterns, so I sorted out goroutine-related issues, hoping that more people will find this post Articles to help you quickly locate memory leaks.

Causes of Goroutine memory leaks

Memory leaks in Go are often caused by incorrect use of goroutines and channels. For example, the following situations:

1. Open a connection (such as gRPC) in a goroutine but forget to close it
2. The global variable object in the goroutine is not released
3. The channel is read in the goroutine, but it is blocked without writing to the end
4. Writing to an unbuffered channel in a goroutine, but blocked because the channel’s read end was closed by another goroutine
5. Writing to a buffered channel in a goroutine, but the channel buffer is full

These kinds of situations are usually mixed in the logic of complex code, and it is difficult to debug and find problems. Therefore, the following patterns that are most prone to problems in daily work are derived.

Common Goroutine memory leak patterns

Premature Function Return / function returns prematurely

A goroutine is about to write to a channel, but the other end exits unexpectedly causing the code that the channel reads to not execute.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

 func Example () { a := 1 c := make ( chan error ) go func () { c <- err return }() // do something if a > 0 { return } // do something err := <- c }

In the code, the main process returns at if a > 0 , which causes the channel to be blocked because it cannot be written.

One way to solve this problem is to convert the unbuffered channel into a channel with a buffer size of 1.

 1

 c := make ( chan error , 1 )

A buffered channel will not block even if there is no read operation.

The Timeout Leak / Timeout Leak

This is a problem encountered in our work, and it is often used when an asynchronous operation that may timeout needs to be performed.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

 func Example () { timeoutOption := SomeTimeoutOption () done := make ( chan any ) go func () { done <- result }() select { case <- done : return case <- timeoutOption . Timeout (): return } }

In this code, once the timeoutOption operation times out, select will be notified, and then the program exits, so the operation of goroutine writing done is blocked and cannot exit.

The solution is the same as the previous mode, using a buffered channel instead of an unbuffered channel.

The NCast Leak / Multi-terminal read and write leak

This can happen if the channel has only one read end, but multiple write ends.

 1 2 3 4 5 6 7 8 9 10 11

 func Example () { c := make ( chan any ) for _ , i := range items { go func ( c chan any ) { c <- result }( c ) } data := <- c return }

This situation also applies to the situation of “multiple write ends and one read end”. The solution is to set the channel to the same number of buffers as the number of writes or reads.

 1

 c := make ( chan any , len ( items ))

Channel Iteration Misuse / Channel Iteration Misuse

Go supports a feature “Range over channels” , which can be used to read the content of the channel in a loop.

But once the content cannot be read, the range will wait for the channel to be written, and if the range happens to be inside the goroutine, the goroutine will be blocked.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

 func Example () { wg := & sync . WaitGroup {} c := make ( chan any , 1 ) for _ , i := items { wg . Add ( 1 ) go func () { c <- data }() } go func () { for data := range c { wg . Done () } }() wg . Wait () }

The way to solve this problem is to manually define the closed channel.

 1 2 3 4

 wg := & sync . WaitGroup {} c := make ( chan any , 1 ) defer close ( c ) //...

In this way, after the WaitGroup is all over, the main program will close the channel, so that the range inside the asynchronous goroutine exits the loop waiting.

summary

Goroutine memory leaks are the most prone to memory leaks in the Go language, and they are usually accompanied by incorrect use of goroutines and channels. The special usage of channel, such as select and range, makes channel blocking more hidden and difficult to find, thus increasing the difficulty of troubleshooting memory leaks.

When writing goroutines and debugging memory leaks, focus on channel-related operations, especially the four types of patterns listed in the article: premature return of functions, timeout leaks, multi-terminal read and write leaks, and channel iteration misuse.

This article is transferred from: https://sund.site/posts/2023/goroutine-leak/
This site is only for collection, and the copyright belongs to the original author.