Does it make sense to count the bug rate of thousands of lines of code?

Original link: https://www.kingname.info/2022/07/13/bug-rate/

My conclusion: Statistical bug rates make sense. But it is meaningless to count the bug rate of thousands of lines of code.

Why is the 1000 lines of code bug rate meaningless?

A company recently came up with a scheme to quantify the job performance of programmers. It is called千行代码Bug率. In a statistical cycle, the number of lines of code added or modified by the programmer and the number of bugs found by QA are calculated according to the following rules:

• 1000 lines of code, 1 bug, then the bug rate is 100%;
• 2000 lines of code, 4 bugs, then the bug rate is 200%;
• 5000 lines of code, 3 bugs, then the bug rate is 60%
• n lines of code, m bugs, then the bug rate is m / n * 1000

Let’s not consider whether the rule itself is problematic. In my opinion, all performance statistics linked to the number of lines of code are meaningless. Because the number of lines of code can be brushed. If a performance requires as few lines of code as possible, you can use the writing method with fewer lines; a performance requires as many lines of code as possible, then you can use the writing method with more lines.

For example, for a string assignment: a = '今天天气竟然有40度，我要被烤化了。' , which can be expanded as:

 1
 2
 3
 4
 5

 a = ( 'Today's weather'
 'There are 40'
 'Degree, I want'
 'Toasted. '
 )

Going even further, expand it to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11

 a = 'Today's weather'
 b = 'there are 40'
 c = 'degrees, i want'
 d = 'toasted. '

 e = (a
 + b
 + c
 + d
 )
 a = e

The three spellings have exactly the same effect.

There are also some functions, which can be done with one line of native code. But in order to increase the number of lines, a third-party library is used intentionally. In this way, the number of lines of code of the third-party library is also counted. The increase in the total number of lines of code is equivalent to an increase in the denominator, and the bug rate of thousands of lines of code is reduced.

It is also easy to abbreviate. In Python, if you use lambda expressions, you can shorten the normal code of 40 lines into 1 line through a very dazzling and anti-human writing method. But such a line of code is simply unmaintainable.

Why is the bug rate meaningful?

For a practically useful project code, the number of bugs is a systematic error that can only be reduced, but there is no way to become 0.

Also implement a function, good programmers can predict in advance how others will use it, deal with illegal logic and unreasonable data flow in advance, thereby reducing the number of bugs. And poor programmers write code that will cause problems when others use it. Therefore, I think it is reasonable to use the bug rate to judge the level of programmers. But from the number of bugs to the bug rate, this calculation method should be carefully designed.

development stage

If the company has QA, in the software development stage, the general product manager will first put forward the requirements, and then pull the development and QA to evaluate the requirements together. QA will design test cases after the requirements review meeting. These test cases are public and will be seen by every developer.

These public test cases, I think, can be used as a denominator. Programmers write code, but fail some of the test cases. That is the level of programmers is not good.失败的测试案例数/所有公开的测试案例数. It can be used as one of the reference indicators to measure the level of programmers. Good programmers should try to keep this ratio at 0.

But sometimes, in the process of testing, QA may temporarily add test cases that the programmers don’t know about in advance. Then if these cases fail the test, they can also be used as an indicator to judge whether the programmer has the ability to prevent in advance. But to be fair, you can multiply it by a factor小于1 to reduce its weight:

 1

 Development stage bug rate = (number of test cases that have been disclosed + coefficient × number of test cases that have been temporarily added) / total number of test cases

As a digression, today we do not consider issues such as the number of unit tests and unit test coverage. Because as far as I know, there are too few programmers who will actively write unit tests in domestic Internet companies. Sometimes, an excellent programmer who originally wanted to write unit tests, after entering some big factories, gradually gave up due to the pressure of business and construction schedule. So we only consider test cases for QA today.

online stage

If you only look at QA’s test cases, there may be problems with QA-oriented programming. Because people are very smart, there are policies above and countermeasures below. A case of QA testing the API interface, input 5, output 10. The programmer directly judges in the code, if the input is 5, directly returns 10, skipping all the logic in the middle. This will pass 100% of all test cases for QA. But doing so is of no value to the product itself.

The market is an important criterion for checking the quality of the code. Whether the quality of the program is good or not, after it goes online, let users evaluate it.

You never know how stupid your users are, and you never know how they will use your product.

Bugs reported by users can also be used to evaluate the quality of the code, which in turn reflects the programmer’s ability. But the following two situations need to be considered:

The same function, two programmers implement:

• As soon as the function written by programmer A goes online, users report bugs as soon as they use it
• The function written by programmer B has been online for a long time. Hundreds of thousands of users are using it normally, but a sand sculpture user is operating randomly, and a bug is accidentally exposed.

Everyone knows by subjective judgment that programmer B should be better than programmer A.

Let’s consider the second case, where programmer A implements the X function, and programmer B implements the Y function:

• The X function is used millions of times a day, and more than 20 bugs are found in a week
• The Y function is used a total of 3 times a month. No bugs found

In this case, we have no way to judge which of the two programmers AB is better based on the number of bugs. Maybe B programmers write X functions, and hundreds of bugs may be found in one day.

Therefore, based on these two situations, I patted my head and summed up an empirical formula:

 1

 Online bug rate of a function = number of bugs / (log(number of functions used + 1) + 1)

where log is the base 10 logarithm. Because a function can easily be used hundreds of thousands of times, and the number of bugs is generally single or double digits. Therefore, take the logarithm of the number of uses to avoid the bug rate being too small. +1 twice in the formula. Once because the logarithm of 0 is not possible, and the other time because the denominator cannot be 0.

For the bug rate statistics of multiple online functions developed by programmers, we can calculate as follows:

 1

 Programmer's online bug rate = A function online bug rate * function importance coefficient + B function online bug rate * function importance coefficient +  …

Among them, the functions of the same importance, their function importance coefficients should be the same. Functions of different importance, the more important the function, the larger the coefficient.

Here, we can discuss whether this coefficient should be a功能重要性系数or a功能复杂性系数. Personally, I think it is better to use the importance. On the one hand, the complexity of the code is not easy to quantify. The second is because programmers’ code quality and business cannot be viewed separately. For important functions, it should be done first and should be more attentive. In the case of more attentiveness, there are still so many bugs, doesn’t it mean that the ability is poor. For unimportant functions, do it last, it may be too late, and there are some bugs in rushing to complete the work. However, because this function is not used by many people, it has little impact on the business, and there are some bugs that are fine.

The comprehensive formula of slap the head

From the comprehensive development stage and the online stage, we can draw a comprehensive formula. Since in general, the value range of a certain rate should be 0-100%, after these two formulas are combined, the result is likely to be greater than 1. Therefore, we changed the name and called it the程序员Bug指数:

 1

 Programmer bug index = development stage bug rate * development stage coefficient + programmer online bug rate * online stage coefficient

The higher the index, the poorer the programmer’s ability.

Finally, I would like to emphasize that the above formula is what I came up with, and it is only for reference. But I think it should be worth much more than the thousand lines of code bug rate.

This article is reprinted from: https://www.kingname.info/2022/07/13/bug-rate/
This site is for inclusion only, and the copyright belongs to the original author.