Rust for Linux is coming, and what it means for us

Author | Tina

As we all know, Linux is the spokesman of the C language. However, times have changed, Rust is emerging and gaining more and more support, and it is starting to take on the role of the Linux system language.

At this year’s Linux Foundation Open Source Summit, Linus Torvalds mentioned that he would like to see Rust incorporated in Linux Kernel 5.20. Kernel release cycles are typically 9 to 10 weeks, which means we might see 5.19 in early August. Then, if all goes well, we’ll see Rust in 5.20 in late October or early November 2022. And only last year, Linus Torvalds just commented on Rust, saying that he will never promote the Rust movement in Linux, “There is definitely complexity behind the advantages of Rust, so I will take a wait-and-see attitude to see if these advantages are real. works.”

So why did Rust get into the Linux kernel so quickly? What does this mean for the community, for companies using Linux and Rust? To answer these questions, we interviewed Shi Jicheng, co-founder of DatenLord, a cross-cloud storage company.

InfoQ: A few years ago, there were voices saying “it’s time to rewrite the OS in Rust”. From your observations, is this possible? Why?

Shi Jicheng: If we define “rewriting the operating system in Rust” as building a Rust language-based operating system from scratch and hoping it can be commercialized, it is unrealistic. The reason is that commercial operating systems require application scenarios. One or several operating systems will be popular in a certain application scenario for a long time, and it is not easy to change, such as Windows and MacOS in desktop operating systems, and mobile phone operation. Android and iOS in the system, as well as server operating systems Linux, BSD and Windows Server, etc. We can find that these mainstream operating systems have grown up with the development of the field, often with a history of decades, and once they form a dominant position, it is not easy to shake.

The main reason why users are reluctant to change operating systems is “ecological dependence” – many applications and development frameworks developed on these operating systems. If it is possible to write an operating system from scratch by a commercial company, then the complete migration of the upper-layer ecosystem of the operating system is a vast project that requires the participation of all human beings. So I don’t think it’s realistic to write a completely new operating system based on existing mature scenarios in Rust language.

Then Rust for Linux (hereinafter referred to as R4L) is another feasible solution. The main reason why this method is feasible is that it does not change the interface of the Linux operating system and is fully compatible with all existing applications on Linux. R4L just replaces some kernel modules inside Linux. These kernel modules completed in Rust can be perfectly combined with other kernel components. By slowly “cannibalizing”, more and more components in Linux are implemented in Rust language. Ultimately achieve the purpose of improving Linux security. Of course, R4L is still in its infancy at this stage, and there are still many uncertainties in the future, which need to be improved by the community and everyone.

InfoQ: It’s possible to write safe C code, so why do so many people think it’s important to add Rust to the Linux kernel? Also, what does adding Rust to the kernel mean for the community and for your company?

Shi Jicheng: “It is possible to write safe C code” is not wrong in itself. It is possible to write safe code in any language, but why should programmers choose a safe language? Because programmers are human beings, they make mistakes. The mistakes that programmers make in their code are known as bugs. Good programmers use various methods to avoid or reduce their own mistakes, and choosing a safe language, such as Rust, is one of these efforts. The Rust language provides features that keep programmers from making memory errors and thread concurrent access errors that tend to dominate kernel development. Taking the Windows operating system as an example, it was mentioned in a speech by Microsoft in 2019 that 70% of the security vulnerabilities in Windows are memory security issues. I believe the phenomenon in Linux should be similar. Based on this, adding Rust to the Linux kernel is critical to improving its security.

The R4L project is a major boon for the Rust language community, because any mainstream language needs one or several application scenarios to grow and develop. Rust has been proven to be used to build browsers – Firefox, and the R4L project has proved that Rust can be used. is used to build the operating system. With the continuous expansion of these mainstream application scenarios, Rust will be used by more and more people, and the ecology of the corresponding Rust language will get better and better, which will eventually reduce the difficulty of using the Rust language and attract more people to use Rust. is a forward cycle.

For our company (Beijing Datan Technology), the R4L project also helps us build a secure storage system more easily. Since our company mainly focuses on the storage system that combines software and hardware, it is inevitable to deal with the Linux kernel, and it is also necessary to write system kernel modules to complete the hardware adaptation work. R4L makes our security-related work easier and improves. System-level security. Since R4L is still in its infancy, we will also submit Patches to the Linux community during use to help the R4L system continue to improve.

InfoQ: What functions has Rust for Linux achieved so far, and what functions still need to be improved?

Shi Jicheng : The Rust for Linux project is mainly divided into two parts. The first part is to package the original C language interface in the Kernel with unsafe Rust code. We commonly call it binding. Most of this part of the work is automatically generated by the code. easier. The other part is how to encapsulate the unsafe Rust code into safe Rust code, that is, if the safety of the methods in Linux is guaranteed by the features of the Rust language, this part of the work is less complete.

Since the kernel modules are too complicated, only some of them can be listed here to illustrate the situation. Modules that have partially completed their functions include memory allocation management modules and basic data structure modules such as linked lists, while unfinished modules include kernel thread abstraction and asynchronous tasks. actuator etc.

InfoQ: What are the tradeoffs in this development history?

Shi Jicheng : As far as I know, the main trade-off point in the R4L project is still security. Because the security of Linux C language modules is mostly achieved by good programming standards and calling paradigms, many security guarantees are implicit in the code logic; for example, the correct release of memory requires the memory caller to guarantee, The paradigm that people often use is to add memory release code to all possible exits, or to control exit exits in one place. Many security points of the kernel are much more complicated than the above examples. How to encapsulate these implicit logic in Rust code is a big challenge. Providing the same performance as C language while being safe poses a greater challenge to function implementers. challenge. However, I firmly believe that with time and more contributors, these problems will eventually be solved perfectly.

InfoQ: Linus has made some changes in attitude towards Rust, such as from “waiting and seeing” at the very beginning, to acknowledging that Rust will be in Linux. The main reason, what do you think is it?

Shi Jicheng: I personally think there are several reasons:

1. Linus had a bad experience trying to write kernel code in C++ in his early years. He only tried it for two weeks, and finally gave up using C++ to write the kernel. This experience will make Linus more cautious about embracing the new language.

2. At the beginning, R4L’s attempt made everyone feel that the risk was very high, and there were many voices of doubt, but with the continuous efforts of the community in recent years, everyone saw more hope. Confidence is also more abundant.

InfoQ: What challenges do you think will be present in Rust when it comes to 5.20?

Shi Jicheng: When R4L entered the main branch, the real story just started. As we introduced before, the R4L project is more about scaffolding for kernel development, and the real kernel modules need to be replaced and rewritten one by one, such as file system and network communication modules. Some of these modules have a long history, and the complexity of the code structure means that switching any language level is not an easy task. How to smoothly migrate these large modules will be a huge challenge for the entire community.

InfoQ: Rust is still quite difficult for developers, and there are not many people familiar with Rust. If Rust appears in the next version, will it pose some problems for maintainers?

Shi Jicheng: This question can be answered from two aspects. First of all, the number of Rust language users has grown from 600,000 in early 2020 to 2.2 million in early 2022. The increase in language users means that more people can be invested in kernel development and maintenance, which can provide a lot of maintenance work. s help. On the other hand, Linus also discussed Rust from the perspective of language difficulty. He believes that the complexity of the system kernel is greater than that of the Rust language, so the Rust language should not be the most difficult problem in kernel development. At the same time, Linus also took Perl language as an example, thinking that this is a very difficult language to read, but this does not prevent the use of Perl language in Linux projects.

InfoQ: Speaking of Rust’s present and future, where do you think the language is now?

Shi Jicheng: The Rust language has been the most popular language on the StackOverflow website (the world’s largest programming question and answer website) for 7 consecutive years. The popularity of the Rust language is self-evident. Now that Rust will soon become the official second language of Linux (Linus’s original words are: “It’s getting to the point where real soon now”), once again verifying the practical value of the Rust language, it will definitely attract more programmers Come learn and try Rust. We have seen a very good growth momentum in Rust, and believe that with the continuous development of the Rust community, it will become one of the most mainstream programming languages, just like C and C++ are now.

Interview guests:

Shi Jicheng, co-founder and CTO of DatenLord, has worked for internationally renowned technology companies such as Google and Alibaba. He is good at operating system kernel development, distributed systems, embedded systems, and has in-depth research on distributed data consistency. Published several papers related to operating system kernel, accumulatively hundreds of citations.

The text and pictures in this article are from InfoQ