Document Type
Technical Report
Publication Date
6-1-2007
Technical Report Number
TR2007-596
Abstract
Pipelines that operate on buffers often work well to mitigate the high latency inherent in interprocessor communication and in accessing data on disk. Running a single pipeline on each node works well when each pipeline stage consumes and produces data at the same rate. If a stage might consume data faster or slower than it produces data, a single pipeline becomes unwieldy. We describe how we have extended the FG programming environment to support multiple pipelines in two forms. When a node might send and receive data at different rates during interprocessor communication, we use disjoint pipelines that send and receive on each node. When a node consumes and produces data from different streams on the node, we use multiple pipelines that intersect at a particular stage. Experimental results for two out-of-core sorting algorithms---one based on columnsort and the other a distribution-based sort---demonstrate the value of multiple pipelines.
Dartmouth Digital Commons Citation
Cormen, Thomas H.; Natarajan, Priya; and Davidson, Elena Riccio, "When One Pipeline Is Not Enough" (2007). Computer Science Technical Report TR2007-596. https://digitalcommons.dartmouth.edu/cs_tr/299