After 14 years in the industry, I still find programming difficult: writing code for big projects is boring and dangerous

Author | Zhu Lei (@piglei)

In 2020, I made a sharing in the company group. The PPT title at that time was “Ten Feelings After Ten Years of Programming”. After sharing the information on the intranet, a colleague saw it and commented that just watching the PPT is not enough. I hope I can expand it into an article, and I replied that there is no problem. Now 3 years later, I finally fulfilled my promise. On the last page of the PPT at that time, I used a pure white background to give a line of big bold characters: “Ten years are very short, programming is difficult”. Fast forward to the middle of that second decade now, and the second half of that statement still seems to hold true for me.

Many years ago, when I was a student, I would occasionally click on some “senior engineer” job postings. In these posts, dazzling technical terms were written, but what impressed me most was that they often appeared in Requirements for the post on the first line: “Requirements for this position: more than 5 years of work experience”. As a rookie who has never worked a day, these age requirements are exaggerated in my eyes. However, while looking forward to the ocean and sighing, I sometimes secretly look forward to it in my heart: “How amazing is a programmer with five years of work experience? Is writing code as easy as eating for them?”

Time flies, and more than ten years have passed in a flash. Looking back now, I have become an honorable worker with 14 years of work experience. After working in the software development industry for many years, I found many things that are very different from what I imagined in my senior year, such as:

• With the growth of experience, programming will not become much easier, “as simple as eating” only appears in dreams

• Writing code for many “big projects” is not only boring, but also dangerous, far less interesting than doing an algorithm problem on LeetCode

• If you only think about problems from a technical point of view, you will not be a good programmer. Some things are far more important than technology

Thinking about it carefully, there are many more feelings about programming. I sorted out 8 of them and wrote this article. I would be very happy if some of these points resonate with you.

1 Writing code is easy, but writing good code is hard

Programming used to be a professional skill with a high barrier to entry. In the past, when an ordinary person wanted to learn programming, the most common way was to learn through textbooks and books. However, most programming professional books are very difficult and obscure, and are very unfriendly to beginners. Therefore, many people give up halfway before they taste the fun of programming.

But nowadays, learning to program is getting easier and easier. Learning is no longer like before, you can only gnaw hard at books, but there are many new ways. Watch instructional videos, take interactive courses on Codecademy, or even learn to code by playing games directly on CodeCombat, there’s something for everyone.

“Mom, I’m not playing games, I’m learning programming! Look at the right side of the screen!”

In addition, programming languages ​​are becoming easier to use. The classic C and Java are no longer the first choice of most beginners. Many simpler and easier-to-use dynamically typed languages ​​are now very popular, and related tools such as IDEs are becoming more and more perfect. These factors further lower the learning threshold for programming.

All in all, programming has long since lost its mystery, from a mysterious skill that only a few people can master, to a common craft that everyone can learn.

But a lower learning threshold and a more friendly programming language do not mean that everyone can write good code. If you have already worked and participated in some projects, then I would like to ask you a question: “How is the code quality of these projects you come into contact with every day? Is there more good code or more bad code?”

I don’t know how you will answer, so let me give my answer first.

Good code is still rare

In 2010, I jumped to a large Internet company headquartered in Wudaokou, Beijing.

Before joining this company, I only worked in a small company with ten employees. Therefore, I have high expectations for the new company in all aspects, especially in terms of software quality. At that time, what I thought was probably like this: “This is a ‘big’ project that supports a product with tens of millions of users. Compared with the previous ones, the code quality must have a qualitative leap!”

It wasn’t until a week into my new company that I realized how wrong I was. The code quality of so-called “big” projects is far from what I expected. Open the IDE, hundreds of lines of functions and mysterious numeric literals abound, and it is extremely difficult to develop any small requirement.

Later, after staying in more companies and getting in touch with more software projects, I concluded a truth: no matter how big the company is or how good the projects are, it is still a small probability event to encounter good code in actual work.

What are the elements of good code?

Having said that, what kind of code is good code? In this regard, Martin Fowler has a saying that is often quoted by everyone:

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”

“Any fool can write code that a computer can understand. A good programmer writes code that a human can understand.”

I think it can be used as the origin of evaluating good code: good code must be readable, easy to read, and easy to understand. The first rule of writing good code is to put the human reader first.

In addition to readability, there are many other dimensions to evaluating code good or bad:

Compatible with the programming language: Is the recommended writing method of the current programming language used? Are language features and syntactic sugar used to the right degree?

• Compatible with the programming language : Is the recommended writing method of the current programming language used? Are language features and syntactic sugar used to the right degree?

• Ease of modification : Does the code design take into account future requirements changes, and when changes occur, is the code easy to modify?

• Well-designed API : Is the API well-designed and easy to use? A good API is easy to use in simple scenarios, and can be expanded with demand in advanced scenarios.

• Usable performance : Does the code performance meet the current business needs, while leaving room for improvement in the future?

• Avoid over-design : Is there any problem with over-design and premature optimization of the code?

• …

All in all, good code is not something that is readily available to programmers of any level. To write good code, you need to weigh repeatedly in many dimensions, carefully design, and finally continue to polish.

That being the case, if you want to master the skill of writing code as soon as possible, is there a shortcut?

Shortcuts to writing good code

On many levels, I think programming and writing are very similar. Both use text and symbols to express ideas, but in slightly different ways.

Speaking of writing, I want to ask a question about writers: “Have you ever heard of a writer who doesn’t read? Have you ever heard a writer say that he never reads anyone else’s work but only his own?” “. I guess the answer should be no.

If you look up relevant information, you will find that the daily life of many professional writers is a constant cycle of reading and writing. They spend a lot of time every day reading all kinds of texts and then writing.

Also “text workers”, programmers rarely pay attention to reading. But if you want to quickly improve your programming skills, reading is an indispensable part. In addition to the projects we come into contact with in our daily work, we should read more about those classic software projects to learn the skills of API design, module architecture and code writing.

Not only codes and technical documents, it is best to read some professional books on computer regularly to maintain the habit of reading books. In this regard, I think Jeff Atwood’s article “Programmers Don’t Read Books — But You Should” written 15 years ago is still relevant today.

The shortcut to improve programming ability is hidden in the endless cycle of “reading <-> programming”.

“What should a good programmer do?”

2 The essence of programming is “creation”

In the daily work of a programmer, there are many things that will make people feel a sense of accomplishment, and even can’t help but sigh “programming is so beautiful”. For example, fixing a bug that was extremely difficult to locate, doubling the performance of the code with a new algorithm, and so on. But of all such things, there is nothing like ” creating something with your own hands.”

When you’re programming, opportunities to create new things are literally everywhere. Because not only the release of a new software can be called “creation”. Writing a reusable tool function and designing a clear data model can all be classified as “creation”.

As a programmer, it is very important to maintain the enthusiasm for “creation”. Because it can help us:

• Learn More Efficiently : The most efficient way to learn a new technology is to use it to develop a real project. Learning in the process of creation has the best effect.

• Chance to stumble upon something amazing : Much of the world-changing open source software was originally created by authors out of pure interest, such as Linus Torvalds and Linux, Guido van Rossum and Python.

During the Christmas holidays in 1989, the Dutchman Guido van Rossum knocked down the first few lines of code for the Python language, which was originally only expected to be the successor of the ABC language, but later “swallowed” the world

Although “creation” has many benefits, and programmers have a lot of opportunities to do it, many people often lack the awareness of being a “creator”. As the popular little story goes: A philosopher asks workers who are laying bricks. Some people know that they are building a cathedral, while others think they are just laying bricks. Many programmers are exactly “seeing bricks, not churches”.

Once you position yourself as a creator, the way you see things changes drastically. For example, by adding error message text to the API, creators can jump out of the thinking trap of “just complete the requirements quickly”, take a step forward, and ask themselves some more important questions: “What kind of problems do I want to create for users?” product experience? What kind of error message can help me achieve this goal?”

Like any useful programming pattern, a “creator mindset” can be a huge boost to your career. So try asking yourself the question right now – “What will my next creation be?”

3 It is very important to create an efficient trial and error environment

I once participated in the development of an Internet product, which was beautifully designed, rich in functions, and used by a large number of users every day.

But it is such a product that is quite successful from the market point of view, but the engineering quality is very poor. If you open its back-end project and turn all the directories upside down, you can’t find any line of unit test code, and other automated testing processes are out of the question. But the business logic is very complicated. In the end, there are so many unexpected couplings between project codes that it is easy to mess up old functions when developing a new feature.

“What are you up to?” “Trying to fix a problem I made when I was fixing a problem that I made when I was fixing another problem, and that problem was me…”

Therefore, every time the project is released, all the development and product students have to be on the alert, and the atmosphere is very tense. The whole release process is also very exciting, and emergency rollbacks happen from time to time. Working alone in such an environment, regardless of technical growth, the psychological quality will definitely be greatly exercised.

Programming is originally a job full of fun, but programming for such a project is not fun at all. What exactly takes away the fun of programming?

Ideal programming experience ≈ “study questions”

LeetCode is a well-known programming learning website, which provides many programming questions covering various levels of difficulty, most of which are related to algorithms. Users can choose topics they are interested in, write codes (supporting more than a dozen programming languages) and execute them directly on the browser. A solution is considered successful if all test cases pass.

Do questions on LeetCode

Brushing questions on LeetCode is like playing a game, challenging and fun at the same time. The whole process of doing the questions actually perfectly shows an idealized programming experience:

• Separation of concerns : each topic is an independent individual, and developers can fully immerse themselves in one topic at the same time;

• Quickly obtain accurate feedback : developers can quickly obtain result feedback through automated testing after each adjustment of the code;

• Zero-cost trial and error : The written code has grammatical errors and logic problems, without any adverse consequences, and the psychological burden is small.

However, you in front of the screen will probably think that I am talking nonsense.

“Otherwise? Solving arithmetic problems and writing small scripts, isn’t that the experience? Is there anything special worth mentioning?” You will probably continue to add, “Do you know how complicated our company’s projects are? The scale is huge, There are so many modules, do you understand what I mean? Serving ××× million people every day, there are only a few sets of databases, and there are three types of message queues. Of course, it will be a little more troublesome to develop!”

Indeed, the software in the world is very different, and it is impossible to develop it as easily and happily as brushing questions on LeetCode. But that doesn’t mean we shouldn’t try to improve our programming environment, even a little bit.

To enhance the programming experience by improving the environment, available ideas and tools include:

• Modular thinking : Properly design each module in the project to reduce coupling and improve orthogonality

• Design principles : At the micro level, apply those classic design principles and patterns, such as the “SOLID” principle

• Automated testing : write standardized unit tests, use Mock technology when necessary, and cover business critical paths with automated tests

• Shorten the feedback loop : Switch to tools with faster compilation speeds, optimize single-test performance, and do everything possible to shorten the waiting time from “code modification” to “getting feedback”

• Microservice Architecture : When necessary, split the large monomer into multiple microservices with different responsibilities to disperse the complexity

• …

Pay attention to the programming environment, deliberately create a “code paradise” that allows efficient trial and error, and make work as easy and enjoyable as brushing questions. It’s one of the best things an experienced programmer can do for his or her team.

4 Avoid the code perfectionism trap

It is good to strive for excellence in code quality, but be careful not to fall into the trap of perfectionism. Because programming is not artistic creation, people are not encouraged to pursue the ultimate without limit. Writers can spend years polishing a handed down work, but programmers are very problematic when they dig into the corners of the code.

There is no such thing as perfect code. Most of the time, your code just needs to meet current needs and leave some room for future expansion. There have been a few times I’ve seen candidates label themselves “code obsessive” on resumes. Through the screen, although I can feel TA’s emphasis on code quality, in my heart, I actually hope that TA has already left the perfectionism trap far behind.

5 Technology is important, but “people” may be more important

In the field of software development, the “Single Responsibility Principle” (single responsibility principle, hereinafter referred to as SRP) is a very famous design principle. Its definition is very simple and can be summed up in one sentence: “Each software module should have only one reason to be modified”.

Single Responsibility Principle: Just because you can do it doesn’t mean you should

To master the SRP principle, the key lies in figuring out what the “reason to be modified” is. Obviously, a program is lifeless, and it cannot and does not need to actively change itself. Any reason to modify a program comes from the people involved with it, and people are the “culprits” that cause the modification.

Take a simple example. Take a look at the following two classes, which one violates the SRP principle?

1. A dictionary data class that supports two types of operations: storing data and fetching data;

2. An employee profile class that supports two types of operations: updating personal information and rendering a user profile card image.

In most people’s eyes, the first example is fine, but the second example is a clear violation of the SRP principle. To reach this conclusion, it seems that there is no need for any rigorous analysis and proof, just a little intuition. But if you do some serious analysis, the suspiciousness of the second example is that you can easily find two different reasons for modification:

1. The administrator believes that the “Personal Phone” field in the data cannot have illegal numbers, and needs to add simple verification logic;

2. An employee thinks that the “name” part on the profile card picture is too small and wants to increase the font size.

“It is people who request changes. And you don’t want to confuse those people, or yourself, by mixing together the code that many different people care about for different reasons.” —— “The Single Responsibility Principle”

“People are asking for software changes. You never want to mix code that different people care about for different reasons, only to confuse them with yourself.” – “Single Responsibility Principle”

The key to understanding the principles of SRP is to understand people and the role they play in software development.

Give another example. Microservice architecture is a hot technical topic in recent years. But when many people discuss it, they often only focus on the technology itself, but ignore the relationship between the microservice architecture and people.

The key to distinguish the microservice architectural style from other things is that after splitting the monolith into independent microservices, the boundaries between different modules can become clearer. Compared with a team of hundreds of people maintaining a large monolith, many small organizations each maintain independent microservices, which obviously has higher operational efficiency.

If there is a lack of a specific organizational scale (that is, “people”) as a premise, it is purely putting the cart before the horse to talk about the various technical advantages and tricks of microservices.

Technology is of course important. As technicians, the magnificent architecture diagrams and ingenious code details naturally attract our attention. However, please don’t turn a blind eye to another important factor in software development, “people”. When necessary, switch perspectives (from “technical” to “human”) to your benefit.

6 It is a good thing to be thirsty for knowledge, but you must also pay attention to the method

Everyone is talking about “lifelong learning” nowadays, and programmers are a profession that especially requires lifelong learning. Because the iterative update of computer technology is very fast, a framework or programming language that was popular three years ago is likely to be outdated a month ago.

What will happen in one minute? 70,000 more hours watched on Netflix; 3 million videos watched on Snapchat; 2.4 million new searches on Google; a new JS framework invented (this is not true )

To perform well at work, programmers need to learn a lot, covering all levels. Taking the back-end field that I am more familiar with as an example, a qualified back-end engineer needs to master at least the following:

One or more back-end programming languages/relational databases such as MySQL/common storage components such as Redis/design patterns/user experience/software engineering/compilation principles/operating systems/network foundations/distributed systems/…

Although there is a lot to learn, according to my observation, most programmers actually love learning (at least not repelling it), so mentality is not a problem. But sometimes, it is not enough to just have the mentality of “thirsty for knowledge”. When learning, we need to pay special attention to “cost-effectiveness”.

Focus on cost-effective learning

The graph below shows the relationship between learning outcomes and engagement.

Learning effect and investment relationship diagram, the horizontal axis is learning input, and the vertical axis is learning effect

It can be seen from the figure that in the initial stage of learning, when the investment is small, the results obtained increase rapidly. But when the effect exceeds a certain threshold, if you want to continue to improve, the required learning investment will increase exponentially.

Because of this, I suggest that when you learn any new thing, you first think about a question in your mind: “Where should I stop in the above picture?” instead of learning it.

The ocean of knowledge is boundless, and some things require us to learn and improve continuously over the years. There are also some things, and it is more than enough to learn a little superficially. Accurately judging and allocating your limited learning energy is even more important than studying hard itself.

Choose the right learning materials

With learning objectives, the next step is to find the right learning materials. In this regard, I would like to share a failure experience of my own.

For a while, I suddenly had a strong interest in product interaction design, thinking that I should improve in this area. So, I carefully selected a very classic professional book in the field: “About Face 4: The Essence of Interaction Design”, bought it home, and confidently believed that my interaction design ability could be improved quickly.

But things backfired. When I held that classic book in my hand, I found that I couldn’t even finish the first chapter smoothly-the old saying is true: “Every line is like a mountain”.

From this failure, I summed up a little experience. That is, when learning something new, we’d better choose learning materials that are easier to read and more suitable for “layman”. Don’t have “big eyes and small mouth”, just know to go for the most classic and authoritative materials.

Looking back on my previous experience, I think the following books are very suitable for laymen to learn and use, and they are very cost-effective:

• “A Design Book for Everyone”: Design Related

• “Tianshichengjin”: Web user experience related

• “Bird Brother’s Linux Private Kitchen”: Linux system related

Maybe everyone wants to be a learned person in his heart, who knows everything and knows everything. But the time and energy available is always limited, and we cannot and do not need to be experts in all fields.

7. The sooner you start writing unit tests, the better

I like unit testing very much. I think writing unit testing has had a great impact on my programming career. It is an exaggeration to say that if “start writing unit tests” is used as the dividing line to divide my career into two periods, the latter period is far more exciting than the former period.

There are many benefits of writing unit test, such as unit test can drive you to improve the design of the code, it can be used as a kind of documentation of the code, and so on. In addition, sound unit testing is also the key to building the ” efficient environment for making mistakes ” mentioned above.

I have written several articles about unit testing, such as “5 Tips for Unit Testing”, “What the Game “Azure” Taught Me About Programming”. So here, I’m not going to repeat it. Just one word: If you have never tried to write unit tests so far, or never paid attention to testing, I suggest you start writing them tomorrow .

Normally I don’t test my code, but if I test it, I test it in the production environment

8 What is the biggest enemy of programmers?

In most programmer jokes, the product manager often appears as the villain. The project requirements in their mouths are always changing, and a new idea emerges every day, which makes programmers miserable.

Customers are constantly modifying requirements every day, so we decided to “freeze” these requirements before the next release

Under the background of these jokes, the product manager who constantly revises the requirements seems to have really become the biggest enemy of programmers. It seems that as long as the product does not randomly change the demand, everyone’s working environment will immediately become a utopia.

Although it is interesting to spit out a few words about product managers occasionally, I still want to say something seriously: product managers are not enemies.

Because from a certain point of view, software is born to be modified (otherwise, guess why software is called “soft” software?). In this way, developing software is completely different from building a house. Because no one builds a building and says, “Let’s tear it down and rebuild! Same building, but with 30% less steel and cement!”

Therefore, product managers and unstable requirements are not the enemy of programmers. Moreover, the ability to write code that is easy to modify and adapt to changes is one of the important criteria for distinguishing ordinary programmers from excellent programmers.

So, what is the worst enemy of programmers?

Complexity is the worst enemy

As said in ” Code Encyclopedia 2 “: The core problem of software development is to manage complexity. Runaway complexity is a programmer’s worst enemy.

Let’s take a look at the factors that contribute to the growing complexity of the project:

• Constantly adding new features : more features equals more code, and more code usually means more complexity

• Demand for high availability : In order to achieve high availability, additional technical components and codes such as message queues are introduced

• Demand for high performance : In order to improve performance, cache and related module codes are introduced, and some modules are split and rewritten in high-performance languages

• Repeatedly postponed refactoring : Due to the tight project schedule, the imminent refactoring has been repeatedly postponed, and the technical debt is piling up

• Neglecting automated testing : no one writes unit tests, and no one cares about them

• …

One day, when the complexity of the project grows to a certain level, there will be a bang in the air. “Boom!”, a “big pit” that everyone is unwilling to change and dare not change appears in everyone’s IDE out of thin air.

Guess who dug the hole?

The process of slowing down the complexity growth

While it is always inevitable that complexity will continue to grow, there are many practices that can slow down the process. If everyone can do the following things, it is possible to keep the complexity within a reasonable range in the long run:

• Proficient in current programming languages ​​and tools, write clean code

• Use appropriate design patterns and programming patterns

• Zero tolerance for code duplication, abstract libraries and frameworks

• Appropriate use of clean architecture and domain-driven design ideas

• Write thorough documentation and comments Write well-specified and effective unit tests

• Separate those that change from those that don’t

• …

The requirements seem to be many, but in summary, the core is actually one sentence: write better code .

About the Author

Zhu Lei (@piglei) , a programmer, likes programming, reading, and video games. He is the author of the book “Python Craftsman: Cases, Techniques, and Engineering Practices”.

The text and pictures in this article are from InfoQ