Dynamic k8s cluster implementation scheme

Original link: https://blog.besscroft.com/articles/2022/easy-k8s/

foreword

What this article is going to discuss is a very low-cost, but high-availability K8S cluster solution, which provides a variety of implementation ideas and some achievable solutions. I have stepped on some pits. Under the existing resources, there are actually many ways to play, but if a small partner asks: Why don’t you use the xx service of xxx’s family? Why not use the xxx method?

At the end of the day, if you’re thinking of “production grade”, that’s fine, reliability is the most important thing. But for me, the purpose of implementation is “white prostitution”, which is more suitable for tossing yourself as a toy, open source project demonstration station, etc. For example, the production level will configure LoadBalancer outside the cluster, but I will not consider it because it costs money. And I have to consider that if one of the node clusters hangs, or the number is gone, I can switch to a normal cluster in a very short period of time.

Program

ideas

Let me explain the meaning of this picture first. I use the Pisces-Cloud project as an example. Because it is a microservice with separated front and back ends, the front end is directly placed in vercel. Because I don’t think the front-end resources need to be placed in the cluster, but should be placed on the cdn, so that users can access static resources as quickly as possible, while reducing the pressure on the back-end (after all, the server only needs to process API requests.

Then the domain name is placed in Cloudflare for analysis. Currently, it is directly resolved to the Nginx entrance of the cluster, and then enters the cluster. Through the Virtual IP generated by the Service , a NodePort is provided to access, and Nginx performs reverse proxying on this NodePort.

Note that the Virtual IP in this step needs to create a stateless load to achieve, because the default should be Headless, only for internal use in the cluster.

In k8s, there are many ways to expose to the outside for access, such as hostNetWork, hostPort, NodePort, LoadBalancer, Ingress, etc. I used the Ingress domain name routing method before to forward L7, but there is a small problem with this method. The configured domain name needs to be accessed with a port number, which is very awkward. Later, I simply changed the port of the Ingress Controller directly to 80 and 443, so the port number was removed.

After the migration in the past few days, it was placed on the Nginx outside the cluster as shown in the picture, and the certificate was also configured on it. As for the Workers step, let’s talk about it later. I haven’t done it yet. I only provide ideas. I will finish it when I have time to complete it.

Then there is the entire back-end service, where various supporting monitoring and middleware are configured within the cluster. Since Helm can be used directly, when configuring one master and three slaves, it is much more convenient than running with Docker. After all, just change the yaml file (?

my implementation

The most important part of this set of gameplay is Cloudflare to the cluster. In the first picture, Host-0 and Host-1 are different clusters. Some friends may be curious, why do you want to create multiple separate clusters instead of one master corresponding to multiple nodes? I played this way before, but my machine is a 4C24G ARM machine from Oracle Cloud. I can only open one per account, so I can’t use a set of intranets, and the cross-regional delay is also high, and scheduling fails from time to time. . . Although several machines can be formed into an internal network through the public network, the delay is a big problem, and packets are lost from time to time.

Aside from being torturous, the usability aspect is okay. For example, if Host-0 is down, I directly resolve the domain name to Host-1, and the service can be used normally. Data synchronization. Alibaba Cloud used to sell RDS for 20 years. It was good for this, but it has expired now. Because the requirements are not high, I manually use Navicat to synchronize and backup regularly. This is barely a kind of remote disaster recovery of Low after all. . . It didn’t take long for the downtime from receiving the alert email to logging in to Cloudflare to switch DNS resolution.

You have to ask me what is my biggest weakness? I think it should be limited to the ARM architecture, which makes it difficult to install a Jenkins-based DevOps system (not impossible, compatibility issues need to be resolved.

Dynamic Load Balancing Based on Cloudflare Workers

As for this idea, I plan to do it based on the Reflare project. It should not be a big problem to use Cloudflare Workers itself as a reverse proxy. I have used it for a long time, and this thing is also supported in addition to reverse generation, building websites, and engaging in load balancing itself.

Reflare has an experimental feature called “Dynamic Route”, which is dynamic routing. In fact, routing is based on Worker KV access. In this case, is it possible to monitor the cluster, and then trigger the rules (for example, once every 30 seconds and fail more than 5 times), and then modify the route through the Cloudflare API? (The dashboard of this project is under development) It is also possible to directly replace Worker KV with Redis.

at last

The article may be a bit watery? But here is mainly to provide an idea. If you read this article, don’t play like this in the production environment… If you want to see the specific operation, in the documentation of this project, I will follow I will take the time to finish this part. However, the friends who are more familiar with k8s, in fact, have a general understanding after reading it.

Although I am a Java developer, what if the next company uses k8s? (In fact, this one is in use now) It’s not that we have to learn how to develop k8s, but at least we have to understand the general process of deploying the programs we developed on k8s. Although production is done by operation and maintenance, if we are familiar with it, we will be more relaxed when looking for problems ?

References:

This article is reprinted from: https://blog.besscroft.com/articles/2022/easy-k8s/
This site is for inclusion only, and the copyright belongs to the original author.