Title: “ServiceMesh Based on eBPF”
Category: networking
Link: https://isovalent.com/blog/post/2021-12-08-ebpf-servicemesh
This article is an article published by the isovalent company team behind Cilium at the end of 2021. It mainly discusses the possible benefits of a new Service Mesh architecture. The whole article is based on the background of Cillium + eBPF. I think if there is no eBPF In the case of a comprehensive understanding, in fact, you can only read the results that this article wants to bring, and there is no way to understand the overall implementation and operation principles. At the same time, because the uses of eBPF itself are becoming more and more important in addition to the network (Cilium). More and more low-level debugging tools are implemented through the concept of eBPF. Therefore, learning the concept of eBPF actually brings many benefits. If you have time, I recommend everyone to take the time to learn.
This article is mainly divided into several parts
- What is Service Mesh and current mainstream practices
- Talk about the historical development of Linux network transmission
- Service Mesh Architecture Based on eBPF
- Differences under different architectures and possible hidden costs
With the rise of decentralized application architecture, how to provide information about network connections for these scattered applications has always been a maintenance and operation problem. In the past, the easiest way was to import relevant frameworks for various development environments. Every application needs to be modified to integrate these frameworks, but with the development of the entire architecture, there are more and more requirements. For example, the development environment has different programming languages, and there are even unmodifiable third-party applications. To import authentication, authorization, load balancing and other functions, each application developer needs to refer to these frameworks. There is no way to meet all needs beautifully. Therefore, a transparent framework architecture that can ignore the application ontology has become a sought after and desired solution. .
Most of today’s Service Meshes adopt this transparent architecture, and use additional Proxy to intercept application packets for subsequent management and monitoring, so that
- Application developers focus on their own business logic development
- Third-party non-modifiable applications can also import these advanced network features
In the case of kubernetes, the current mainstream is through the concept of sidecar, so that each application is placed next to a Proxy application, and at the same time, Containers in the Pod can use localhost to communicate with each other to process connections.
The application itself calls the Proxy through localhost, and all external connections are handled by the Proxy, so all advanced functions are implemented on the Proxy.
Isovalent believes that this method is functionally feasible, but believes that there are many hidden costs if importing Sidecar
- According to the test, no matter which Service Mesh/Proxy solution will increase the latency of the real connection by 3~4 times, this is mainly due to the architecture of the Linux Kernel, all network packets must be detoured back and forth in the Linux Kernel Network Stack Many times, the back and forth of the package itself will involve various costs such as Context Switch, Memory Copy, etc., so the improvement of the overall Latency is inevitable.
- For the additional resource requirements of the system, each Pod needs an additional Proxy to handle. For a 500 node, and each node has 30 Pods, an additional 15,000 Proxy Containers need to be deployed in the entire environment. Each Container consumes 50MB requires at least an additional 750G of memory,
At the same time, it should be noted that as the number of Pods/Nodes increases, each Proxy may require more memory to maintain the information between these Meshes, so the amount of Memory used will only increase.
So Cillium/Isovalent wants to introduce an eBPF-based architecture to create a Service Mesh with a different architecture. Through the architecture of eBPF, the entire Service Mesh occurs at the Kernel stage, rather than an independent Uses Proxy.
The changes here are
- Based on the characteristics of eBPF, it has its own way to execute specific functions for all sockets on the system, so Cillium can secretly modify the network traffic of the application, whether it is modifying the packet content, debugging and monitoring, etc.
- There is no need to deploy a separate application for each Pod as before. Instead, a common eBPF program is written to provide various functions
- Since everything happens in the Kernel, it can even achieve packet processing based on Socket-level, so the packets do not need to go around, and the entire processing path is very short, so the resulting Latency is very small.
Those who are very interested in this series of wars will take some time to complete the concept of eBPF. The next series of wars and discussions will be able to have more background to understand
personal information
I currently have Kubernetes-related courses on the Hiskio platform. Interested people are welcome to refer to and share, which contains my various ideas about Kubernetes from the bottom to the actual combat.
For details, please refer to the online course details: https://course.hwchiu.com/
In addition, please click like to join my personal fan page, which will regularly share various articles, some are translated articles, and some are original articles, mainly focusing on the CNCF field
https://www.facebook.com/technologynoteniu
If you use Telegram, you can also subscribe to the following channels, where I will regularly push notifications of various articles
https://t.me/technologynote
Your donation will give me the motivation to grow my article
This article is reprinted from: https://www.hwchiu.com/read-notes-50.html
This site is for inclusion only, and the copyright belongs to the original author.