Original link: https://blog.besscroft.com/articles/2022/distributed-storage-oss/
foreword
When it comes to object storage, many people should have used it. Let’s not talk about the services provided by foreign cloud vendors, let’s take domestic ones as an example, everyone should have used Alibaba Cloud OSS. Many small partners choose to use it as a map bed, blog, or even as a cold backup (after all, it is more reliable than Baidu network disk), but the expensive storage costs, traffic costs, and request times of object storage are all It is quite a headache for many individual users. This blog is to provide a new idea, using CloudFlare CDN for data transmission of OSS resources, in order to achieve the purpose of free flow.
How to achieve
introduce
To start with, the Bandwidth Alliance is a service launched by Cloudflare, a group of forward-thinking cloud services and networking companies dedicated to reducing or eliminating data transfer (bandwidth) charges for mutual customers. Why can it be very low or even cost-free? The direct data transmission between cloud vendors and Cloudflare is through private network interface (PNI) or dedicated interconnection, without any network provider (such as telecommunications) in the middle. Since it is a direct connection, there is no incremental cost of PNI. , of course it’s cheaper. Although Alibaba Cloud was not the first to join the Bandwidth Alliance, it has joined for more than a year now (I have been prostitution for more than a year?), and launched the Cloudflare+Alibaba Cloud OSS solution. This is still very conscientious. Although Tencent Cloud also joined later, I felt that it was not “clear and transparent” enough, so I never tried it.
Although sometimes Cloudflare is “decelerating CDN”, but since we have chosen to prostitute for nothing, we must follow through to the end?, for small partners with overseas business, this can save a lot of expenses. Unlike some vendors where bandwidth costs are extremely low, they charge exorbitant fees (that’s you, AWS).
There is some controversy on the Internet, saying that only Alibaba Cloud International can do this, not in countries. . . Paste the country description document here. Moreover, after my own test, it has been used for more than a year in the country, and it did not cost any money.
cost issue
The picture above shows the usage data of accounts in my country. Let me explain the cost-related issues first, and solve the doubts you may have, so as to avoid misuse and cost money!
- storage fee
According to the storage fee document , we can learn that in overseas (some) regions, the standard storage (local redundancy) capacity supports the use of a free quota of 5 GB/month (that is, when the monthly standard storage (local redundancy) capacity is less than or equal to 5 GB) , there is no standard storage (locally redundant) capacity charge).
Although it does not support the free 5GB storage in mainland China, it does not matter. If it is supported, I do not recommend you to use it. You will understand it by looking down.
- traffic cost
According to the traffic fee document , we can learn that in overseas (some) regions, the external network outbound traffic (oss_flow_out) supports the use of a free quota of 5 GB/month (that is, when the monthly external network outflow traffic is less than or equal to 5 GB, the external network will not be charged. outgoing traffic charges).
Some friends may be curious, since traffic can be avoided, what is the use of the 5GB external network outflow traffic? Looking at the picture above, you will find that traffic from the external network cannot be completely eliminated. I have a few KB of fish that slip through the net every month, and this is still under the circumstance that I use less, and the more I use, the more I may leak. The reason is that you can’t achieve 100% cache hit rate. Maybe my setting method is not optimal, but it is enough to be close to 100%. But we also need to prevent the traffic directly hitting the origin site. This 5GB is equivalent to a buffer. How to prevent it from hitting the origin site, I will also talk about it below.
- request fee
According to the request fee document , there is no need to read it. The request fee needs to be charged, but fortunately, it is cheap and almost negligible.
Presumably experienced friends have already noticed that this plan can prevent D but not C, but the plan is equal, but each has its own advantages and disadvantages. But for users who have this need themselves, it can indeed save money.
- Finally, it’s the cost on Cloudflare’s side
For Alibaba Cloud, there are zero export fees for transferring to Cloudflare.
One special note: Export transfer fee concessions or waivers may require registration with a hosting provider and do not apply to data transfers originating in mainland China. In other words, the OSS origin site must be an overseas node, so the mainland node does not have free 5GB storage, in fact, it doesn’t matter!
So how can I avoid the flow?
Don’t worry, let’s take it step by step! First of all, prepare an Alibaba Cloud account, whether it is a China account or an Alibaba Cloud international account is supported here! For the convenience of everyone, I will use my country account to demonstrate, but the country account is rarely used, and I mainly use Alibaba Cloud International to operate.
Create a Bucket, pay attention to the Bucket name must be selected, and then select the overseas region, for example, I chose the Hong Kong region of China here. Select standard storage for storage type, select public read for read and write permissions, and the default for others. Then we click OK to proceed to the next step.
At this time, we come to the bucket console, and we can see the domain name accessed from the external network. We copy this domain name, and then go to Cloudflare to configure the resolution.
For the resolution type here, select CNAME resolution, then fill in the domain name we just copied, and start the proxy!
Then we go back to the OSS console, find the transfer management – domain name management, bind the domain name, and add the domain name we just configured and resolved.
Then we looked at the domain name binding configuration and found that it was already bound. Then we go to upload a photo.
Choose your own domain name to resolve the domain name configured by yourself, we can see that the URL has become our custom!
Batch management is actually a custom domain name.
Then visit this address and find that it can be accessed normally!
some necessary configuration
The picture can be accessed, indicating that it can be used normally. However, there are a few more configurations that I recommend you add.
For anti-leech and cross-domain settings, you can do it according to your own needs. This is not mandatory.
Regarding the authorization policy, this must be added. Visit the Cloudflare IP Ranges page and add all IPs to ensure access security to the greatest extent.
The static page here, I suggest that you must configure it, otherwise, when requesting an empty resource, the requester can see your source site information, and then can bypass the CDN to directly request the source site through technical means, which is also this scheme. greatest risk. So why did I say above that I like to play this with my Alibaba Cloud International account, because it is my trumpet account, and I don’t worry about the “astronomical bill”.
Then we go to Cloudflare to add page rules to improve the cache hit rate.
at last
There are risks in adopting the business solution of Alibaba Cloud OSS. But for the right user, it can save money. Moreover, after my own test, I have opened a lot of storage buckets, one bucket has 5GB space, then I can manage these storage buckets, and I can play with distributed storage at the beginning (of course, it is not really distributed storage, the difference is It’s big ?). However, for general risk reduction, we all use Backblaze . This store still has 10GB of free storage, which is more than enough to be a blog image bed.
This article is reprinted from: https://blog.besscroft.com/articles/2022/distributed-storage-oss/
This site is for inclusion only, and the copyright belongs to the original author.