Side Project cost minimization operation and maintenance

Original link: https://www.kawabangga.com/posts/4718

Sometimes people forget that today’s computer resources are so powerful, how much a $5/month machine can do.

Someone was discussing the official website of redis.io before. The number of visits must be huge, and Redis commands can be run in real time. It must have used a lot of machines. It is a distributed system. But it’s actually running on a $5/month VPS single machine.

In my spare time, I will do some Hobby Projects out of interest. These projects can be divided into two types:

1. Tools and libraries, etc., can be used as long as they are downloaded, and no matter how large the number of users is, it will not increase my cost;
2. The type of website that provides services requires money to purchase domain names and servers, and maintenance costs;

For the second type, the project is only likely to last if costs are reduced:

1. At the beginning of each project, there may not be many people. If the cost is not saved, it is very likely that there will be few users at the beginning, and the high operating cost will make it impossible to continue.
2. Low cost can accumulate users for a long time

This blog writes about some ways on how to reduce running costs.

Operating platform

If it is a static website, there are many choices, cloudflare , Vercel , netlify , all are fine. Just upload the front-end files.

If it is a dynamic website, there are many good SaaS platforms to choose from in recent years: Sass deployment can choose fly.io , heroku , serverless can choose cloudflare, aws and so on. The free quota of these platforms can basically cover many scenarios.

But the little things I do basically need to run user-submitted code, so I usually use DigitalOcean’s virtual machine. The advantage is that it is cheap, you can completely control the VM yourself, and the bill is transparent. There will be no vendor lock, and you can use another VM at any time.

deploy

If you choose SaaS, you need to write a deployment description file according to the SaaS platform used.

If using a virtual machine to deploy, I generally use ansible to manage the deployment:

• Put the code in a repository, which can be public or private
• Use a private repository to store ansible configuration files, including some secret files, where host inventory is placed
• Every time a version is released, submit a tag in the code repository, build binary
• Then modify the deployment tag in the ansible repository and submit a deployment (this step can actually be integrated into CI, and run Ansible automatically for each change)
• Keep a principle: It is to give you a new set of IP, and you can build an identical cluster within a few minutes (this way you don’t have to worry about the original cluster being broken, and there is no operation and maintenance burden)

Logs and Metrics

Logs also has some Saas platforms that can be used, such as Datadog and sematext , but I have not used them. One is that it will increase management costs and operating costs. Even if Saas is free, you still need traffic to send logs, and I I am more accustomed to using command-line tools to read logs.

Regarding Metrics, you don’t need to care about it. The method introduced by jvns is enough for small services at the beginning: Monitoring tiny web services , that is, using black-box monitoring and detection, as well as the way of timed curl website, to ensure that the website is running normally from the hour level. This can already detect enough things, such as SSL certificate expired, database hang and so on.

If you want more accurate monitoring, a better way is to deploy Prometheus + Grafana, the two components are one for collecting metrics and the other for displaying metrics, both can be started with docker, and then use Ansible to manage configuration files for monitoring Coded and can be migrated at any time.

If you want to run user code…

A project I did before is called clock.sh, which is a service that executes user code regularly, which can be understood as serverless crontab. A project that is still running recently is called xbin.io , which provides some command-line tools that can be run without installation. Both projects require running user code.

Jvns has written a lot of tool analysis on her blog to solve similar needs, which is basically the same as what I want. I highly recommend reading it:

• Firecracker: start a VM in less than a second
• Notes on running containers with bubblewrap
• Running containers without Docker

The solution I use now is: Docker + runsc, using the latest version of docker to avoid 0day attacks. Then only a limited number of syscalls can be allowed with runsc, which can meet most of the security needs (there has been no problem for the time being).

the last point

For amateur projects, you should focus your energy on more important (happy) things. The less you have to maintain yourself, the better.

For example, to implement a function, ready-made SaaS > based on library implementation > own software implementation > using new open source project additional deployment system to implement. Try to solve the problem from the software itself. For example, to clean up non-running containers every hour, one way is to write a script to make a crontab run, but a better way is to run a background thread in the software itself and check it every hour. This way, there is no need to maintain crontab (which is actually run by crond).

The post Side Project Cost Minimizing Operations first appeared on Kawabanga! .

This article is reprinted from: https://www.kawabangga.com/posts/4718
This site is for inclusion only, and the copyright belongs to the original author.