March 24 21:45~23:14
▲Figure 1: Example of elevator specifications
foreword
The basic functions related to ezSpec and Gherkin have been introduced, and today I will introduce a function that Gherkin does not have: “Specifications for describing parallel behaviors”.
Gherkin’s Given, When, Then, And, But these Steps are executed sequentially according to the order in which they appear in the Scenario. For some systems that are inherently capable of parallel processing, for example, in IoT systems, multiple sensors or devices interact with each other are executed independently and in parallel. In this case, the behavior of these parallel executions cannot be expressed in Gherkin.
In early March, a group of IoT students in the laboratory introduced to me a tool they developed in python— concurrentSpec , which can extend the Gherkin semantics to allow developers to write and execute synchronous behavior specifications. Originally Teddy developed ezSpec and did not plan to support the description of parallel processing behavior. After listening to the students’ introduction, it took some time to change ezSpec to a function similar to concurrentSpec, which can help write specifications for synchronous behavior.
***
Examples of Parallel Operations
Please refer to Figure 1. This example is excerpted from the paper <Specifying Internet of Things Behaviors in Behavior-Driven Development: Concurrency Enhancement and Tool Support>. The author of the paper is Mr. Zheng, a member of the laboratory IoT team and Teddy’s supervisor. The paper mentions an example of testing an elevator. This example refers to two books written by Jackson: “Software Requirements & Specifications: A Lexicon of Practice, Principles and Prejudices” and “Problem Frames: Analyzing and Structuring Software Development Problems”, Chinese The description is to ask ChatGPT to help translate.
If this example is executed with standard Gherkin, assuming that the execution of line 29 fails (the emergency indicator cannot be turned on), the test case will stop at 29, and the subsequent assertion will not be verified. In the elevator example, this behavior is clearly incorrect. Because even if both lines 29 and 30 fail, as long as line 28 succeeds (the elevator has stopped at the nearest floor), line 39 should be checked (the elevator doors must open within five seconds).
Some test frameworks allow users to set: “Even if a certain assertion fails, the test case will continue to execute.” But even if the example in Figure 1 is implemented in this way, the final result may still be wrong. For example, line 28 fails but line 31 succeeds, which means: “The elevator failed to stop at the nearest floor, but the elevator doors eventually opened.” This is obviously not the behavior the user expects.
How to describe synchronization behavior with Gherkin? concurrentSpec proposes a very simple expansion method: “Under the premise of not adding Gherkin keywords, use Given, When, Then as the beginning of the synchronous execution group , and And and But after them will be executed synchronously with them. The entire synchronization After the execution of the group is completed, the execution of the next synchronization group will start.” In other words, Given, When, Then are executed sequentially with each other, but if they are followed by And or But, these And/But will be parallel to them implement.
In addition, you can also specify whether to continue executing each Step if the execution fails.
***
The program that uses ezSpec to describe the behavior of Figure 1 is shown in Figure 2. Teddy intentionally made errors in “turn on the emergency light” and “cancel the request for the car” (lines 839 and 842), but if errors occur in these two steps, The next Given/When/Thne will still continue to execute (because the ContinuousAfterFailure parameter is specified). Please note that the execution order of the Steps in Figure 2 is as follows. Given, When, Then, Then are executed sequentially, and the first Then and the latter two Ands are executed in parallel.
- Given
- when
- Then, And, And (lines 834, 838, 841): three Steps are executed synchronously
- then
There is a small detail in Figure 2 to pay attention to, that is, to make the Scenario execute in parallel, you must call ExecuteConcurrently() .
▲Figure 2: Specifications of the example of elevators using ezSpec seedlings
Figure 3 shows the execution results of Figure 2. It can be seen that the failure of two And executions does not affect the subsequent execution of Then.
▲Figure 3: Figure 2 execution result report
***
the elevator door won’t open
Teddy modifies the scenario of the elevator, deliberately making the elevator not stop at the nearest floor, please refer to Figure 4.
▲Figure 4: The simulated elevator does not stop at the nearest floor.
The modified Scenario execution result is shown in Figure 5. It can be seen that because the ContinuousAfterFailure parameter is not added to Then, the execution of the next Step will not continue after the entire synchronous execution group organization is completed.
▲Figure 5: Figure 4 execution result report
***
Seeing this, the villagers may think: “Regardless of whether the elevator stops at the nearest floor or not, I just want to perform the last step of the inspection!” You can also modify Then and add ContinuousAfterFailure to it, please refer to Figure 6.
▲Figure 6: Continue to perform subsequent verification steps even if the elevator is not stopped
The modified Scenario execution result is shown in Figure 7. If the system behavior is really like this, congratulations, you have found a bug. what bug? The elevator did not stop at the nearest floor, but the elevator door was open.
▲Figure 7: Figure 6 execution result report
***
in conclusion
If the folks’ system is similar to the IoT system, with obvious parallel behaviors, then the standard Gherkin law cannot describe these behaviors. Borrowing the extended Gherkin semantics from concurrentSpec, you can use ezSpec to describe the behavior of parallel systems without adding Gherkin keywords.
The function of ezSpec specifying behavior has almost been introduced, and the next episode will introduce the function of ezSpec to generate reports.
***
Yuzo’s inner monologue: We are almost at the top of the mountain.
This article is transferred from https://teddy-chen-tw.blogspot.com/2023/03/ezspec8.html
This site is only for collection, and the copyright belongs to the original author.