July 9 14:20~16:20
▲NotifyBoard implements Idempotent architecture diagram
foreword
Teddy introduced why Event Handler needs to have Idempotent in <Event Sourcing (16): Event Semantics and Idempotent in Distributed Systems>. This episode takes the Event Handler—NotifyBoard that generates GetBoardContent in the ezKanban system as an example (please refer to <Event Sourcing (10) ): Implement Projector ＞), and introduce how to implement Idempotent.
***
remember what you did
Implementing Idempotent can start in two directions:
- The operation itself is the Idempotent : If the operation of a system itself is the Idempotent, then the Event Handler does not need to specially handle the events it has seen, as long as the order of the events is determined to be correct, and the event can be executed once with closed eyes after receiving the event. For example, the delete operation itself is Idempotent, and it can be directly applied once after receiving the CardDeleted event (the card is deleted). Even if the same CardDeleted is executed repeatedly, it will not cause a system state error.
-
Memorize the events that have been processed : In general general systems, it is not easy to design all system operations to be Idempotent, so the Event Handler needs to record the event codes it has processed, and then go to it every time it receives an event. Check to see if the event has already been handled. If yes, discard the event; if not, process it and then record the event code after processing. There are two points to pay attention to here. First, the event code needs to be unique and not repeatable before it can be judged whether it has been processed. Secondly, the system state change caused by the processing event and the storage event must be completed in the same transaction , otherwise the system state may change but the processed event is not recorded, so that the next time When the same event is received, it will be re-executed once, and the Idempotent will not be reached.
In this article Teddy will use the second way to implement Idempotent.
***
Look at the test results first
Do the townspeople feel a sense of deja vu when they read the phrase “storing system state changes and event codes in the same transaction”? Yes, this is similar to the method that Teddy introduced in < Event Sourcing (4): Storing Aggregate to Outbox Store >.
Figure 1 is the test case of NotifyBoardContent, which generates a Board, a Workflow, and three Stages, and then adds a new card.
▲Figure 1: NotifyBoardContent test case
The BoardContentViewModel projected by the test case in Figure 1 is shown in Figure 2.
▲ Figure 2: BoardContentViewModel (JSON file)
In addition to projecting the BoardContentViewModel in the database, because NotifyBoardContent supports Idempotent, the Idempotent table of the database also records the seven events that NotifyBoardContent has processed and generated by the test case. The seven events are: BoardCreated, BoardMemberAdded, WorkflowCreated, StageCreated, StageCreated, StageCreated, CardCreated.
▲Figure 3: The database Idempotent table records which events the Event Handler has read
***
implement
The project method of NotifyBoardContent is shown in Figure 4. The 57-line indented switch description is the program logic responsible for projection. Here we should pay attention to:
- Lines 53~54: Call the isEventHandled method of the boardContentStateRepository to determine whether the domain event has been processed, and return directly if it has been processed.
- Line 241: Update the IdempotentData data structure on boardContentState, which is used to record which domain event the Event Handler is processing, as shown in Figure 5.
- Line 242: Store the boardContentState in the database. The boardContentStateRepository.save method will store the boardContentState in the board_content table (the JSON file in Figure 2) and the IdempotentData in the idempotent table in the same transaction, as shown in Figure 6.
▲Figure 4: The project method of NotifyBoardContent
▲Figure 5: IdempotentData category
▲Figure 6: Storing boardContentViewData and IdempotentData in the same transaction
***
Doesn’t seem to be difficult?
After reading the above implementation method, it seems that it is not difficult to make Event Handler reach Idempotent. However, there are still some practical details to consider. In ezKanban, both boardContentViewData and IdempotentData are stored in the PostgreSQL database, so the transaction of the relational database can be used to ensure the state consistency of these two operations. However, if the villagers store the read model in a NoSQL database, such as a document-based NoSQL database, this database may not provide the “cross-document” transaction control function. At this time, it is necessary to store the processed event numbers. On the document that represents the state of the read model.
***
end of first season
This series is almost written here. The key points and practical details of Event Sourcing and CQRS have been explained. If there is anything else in mind, I will add more explanations.
***
Youzo’s inner monologue: Thank you for watching.
This article is reprinted from https://teddy-chen-tw.blogspot.com/2022/07/18idempotent.html
This site is for inclusion only, and the copyright belongs to the original author.