Original link: https://hsiaofongw.notion.site/a6fd2feb9a7445b2a1657a8c90ad9258
This paper introduces the implementation method of multi-core parallel computing, and compares the effects of pipeline parallelism and data parallelism. The key to realizing pipeline parallelism is to use barrier synchronization mechanism to realize synchronization between workers and pipeline conveyor belt. In a more complex system, it may be necessary to explicitly move (copy) the data itself. Through testing, we can see that both pipeline parallelism and data parallelism are faster than serial calculations, but if the calculation steps are divided into small enough, the acceleration effect of pipeline parallelism will be better.
This article is transferred from: https://hsiaofongw.notion.site/a6fd2feb9a7445b2a1657a8c90ad9258
This site is only for collection, and the copyright belongs to the original author.