You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
<li><ahref="#IncludeTheScalablePipelineHeader">Include the Header</a></li>
<li><ahref="#CreateAScalablePipelineModuleTask">Create a Scalable Pipeline Module Task</a></li>
<li><ahref="#ResetAPlaceholderScalablePipeline">Reset a Placeholder Scalable Pipeline</a></li>
<li><ahref="#ScalablePipelineUseOtherIteratorTypes">Use Other Iterator Types</a></li>
<li><ahref="#ParallelScalablePipelineLearnMore">Learn More about Taskflow Pipeline</a></li>
</ul>
</nav>
<p>Unlike <ahref="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr/>Pipeline</a> (see <ahref="TaskParallelPipeline.html" class="m-doc">Task-parallel Pipeline</a>) that instantiates all pipes at the construction time, Taskflow provides a scalable alternative called <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline</a> to allow variable assignments of pipes using range iterators. A scalable pipeline is thus more flexible for applications to create a pipeline scheduling framework whose pipeline structure depends on runtime variables.</p><sectionid="IncludeTheScalablePipelineHeader"><h2><ahref="#IncludeTheScalablePipelineHeader">Include the Header</a></h2><p>You need to include the header file, <code>taskflow/algorithm/pipeline.hpp</code>, for creating a scalable pipeline scheduling framework.</p><preclass="m-code"><spanclass="cp">#include</span><spanclass="w"></span><spanclass="cpf"><taskflow/algorithm/pipeline.hpp></span></pre></section><sectionid="CreateAScalablePipelineModuleTask"><h2><ahref="#CreateAScalablePipelineModuleTask">Create a Scalable Pipeline Module Task</a></h2><p>Similar to <ahref="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr/>Pipeline</a>, <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline</a> is a composable graph object to implement a <em>pipeline scheduling framework</em> in a taskflow. The key difference between <ahref="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr/>Pipeline</a> and <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline</a> is that a scalable pipeline can accept <em>variable</em> assignments of pipes rather than instantiating all pipes at construction or programming time. Users define a linear range of pipes, each of the same callable type, and apply that range to construct a scalable pipeline. Between successive runs, users can reset the pipeline to a different range of pipes. The following code creates a scalable pipeline that uses four parallel lines to schedule tokens through three serial pipes in the given vector, then resetting that pipeline to a new range of five serial pipes:</p><preclass="m-code"><spanclass="n">tf</span><spanclass="o">::</span><spanclass="n">Taskflow</span><spanclass="w"></span><spanclass="n">taskflow</span><spanclass="p">(</span><spanclass="s">"pipeline"</span><spanclass="p">);</span>
<spanclass="n">executor</span><spanclass="p">.</span><spanclass="n">run</span><spanclass="p">(</span><spanclass="n">taskflow</span><spanclass="p">).</span><spanclass="n">wait</span><spanclass="p">();</span></pre><p>The program defines a uniform pipe type of <code>tf::Pipe<std::function<void(tf::Pipeflow&)>></code> and keep all pipes in a vector that is amenable to change. Then, it constructs a scalable pipeline using two range iterators, <code>[first, last)</code>, that point to the beginning and the end of the pipe vector, resulting in a pipeline of three serial stages:</p><divclass="m-graph"><svgstyle="width: 23.800rem; height: 29.200rem;" viewBox="0.00 0.00 238.00 292.00">
</div><p>Then, the program appends another two pipes into the vector and resets the pipeline to the new range of two additional pipes, resulting in a pipeline of five serial stages:</p><divclass="m-graph"><svgstyle="width: 38.200rem; height: 29.200rem;" viewBox="0.00 0.00 382.00 292.00">
</div><p>When resetting a scalable pipeline to a new range, it will start from the initial state as if it has just been constructed, i.e., the token number counts from zero.</p><asideclass="m-note m-warning"><h4>Attention</h4><p>Unlike <ahref="classtf_1_1Pipeline.html" class="m-doc">tf::<wbr/>Pipeline</a> that keeps the given pipes in a <ahref="http://en.cppreference.com/w/cpp/utility/tuple.html" class="m-doc-external">std::<wbr/>tuple</a> object, <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline</a> does not own the given pipe but maintains a vector of iterators to each pipe in the given range. It is your responsibility to keep those pipe objects alive during the execution of the pipeline task.</p></aside></section><sectionid="ResetAPlaceholderScalablePipeline"><h2><ahref="#ResetAPlaceholderScalablePipeline">Reset a Placeholder Scalable Pipeline</a></h2><p>It is possible to create a scalable pipeline as a placeholder using the constructor <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline(size_t num_lines)</a> and reset it to another range later in the application. The following code creates a task to emplace a range of pipes and reset the pipeline to that range, before running the pipeline task:</p><preclass="m-code"><spanclass="n">tf</span><spanclass="o">::</span><spanclass="n">Executor</span><spanclass="w"></span><spanclass="n">executor</span><spanclass="p">;</span>
<spanclass="n">executor</span><spanclass="p">.</span><spanclass="n">run</span><spanclass="p">(</span><spanclass="n">taskflow</span><spanclass="p">).</span><spanclass="n">wait</span><spanclass="p">();</span><spanclass="w"></span></pre><p>The task graph of this program is shown below:</p><divclass="m-graph"><svgstyle="width: 83.800rem; height: 17.900rem;" viewBox="0.00 0.00 838.00 178.50">
</div><asideclass="m-note m-warning"><h4>Attention</h4><p>It is your responsibility to ensure a scalable pipeline has a valid structure before running it. A valid pipeline must have at least one parallel line and one pipe, where the first pipe is a serial type.</p></aside><p>Similarly, you can create an empty scalable pipeline using the default constructor <ahref="classtf_1_1ScalablePipeline.html" class="m-doc">tf::<wbr/>ScalablePipeline()</a> and reset it later in your program.</p><preclass="m-code"><spanclass="n">std</span><spanclass="o">::</span><spanclass="n">vector</span><spanclass="o"><</span><spanclass="n">tf</span><spanclass="o">::</span><spanclass="n">Pipe</span><spanclass="o"><</span><spanclass="n">std</span><spanclass="o">::</span><spanclass="n">function</span><spanclass="o"><</span><spanclass="kt">void</span><spanclass="p">(</span><spanclass="n">tf</span><spanclass="o">::</span><spanclass="n">Pipeflow</span><spanclass="o">&</span><spanclass="p">)</span><spanclass="o">>>></span><spanclass="w"></span><spanclass="n">pipes</span><spanclass="p">;</span>
<spanclass="n">spl</span><spanclass="p">.</span><spanclass="n">reset</span><spanclass="p">(</span><spanclass="n">num_lines</span><spanclass="p">,</span><spanclass="w"></span><spanclass="n">pipes</span><spanclass="p">.</span><spanclass="n">begin</span><spanclass="p">(),</span><spanclass="w"></span><spanclass="n">pipes</span><spanclass="p">.</span><spanclass="n">end</span><spanclass="p">());</span></pre></section><sectionid="ScalablePipelineUseOtherIteratorTypes"><h2><ahref="#ScalablePipelineUseOtherIteratorTypes">Use Other Iterator Types</a></h2><p>When assigning a range to a scalable pipeline, the pipeline fetches all pipe iterators in that range to an internal vector. This organization allows invoking a pipe callable to be a random accessible operation, regardless of the pipe container type. Taskflow does not have much restriction on the iterator type, as long as these pipes can be iterated in a sequential order using the postfix increment operator, <code>++</code>.</p><preclass="m-code"><spanclass="c1">// use vector to store pipes</span>
<spanclass="n">tf</span><spanclass="o">::</span><spanclass="n">ScalablePipeline</span><spanclass="w"></span><spanclass="nf">spl2</span><spanclass="p">(</span><spanclass="n">num_lines</span><spanclass="p">,</span><spanclass="w"></span><spanclass="n">list</span><spanclass="p">.</span><spanclass="n">begin</span><spanclass="p">(),</span><spanclass="w"></span><spanclass="n">list</span><spanclass="p">.</span><spanclass="n">end</span><spanclass="p">());</span></pre></section><sectionid="ParallelScalablePipelineLearnMore"><h2><ahref="#ParallelScalablePipelineLearnMore">Learn More about Taskflow Pipeline</a></h2><p>Visit the following pages to learn more about pipeline:</p><ul><li><ahref="TaskParallelPipeline.html" class="m-doc">Task-parallel Pipeline</a></li><li><ahref="DataParallelPipeline.html" class="m-doc">Data-parallel Pipeline</a></li><li><ahref="TextProcessingPipeline.html" class="m-doc">Text Processing Pipeline</a></li><li><ahref="GraphProcessingPipeline.html" class="m-doc">Graph Processing Pipeline</a></li><li><ahref="TaskflowProcessingPipeline.html" class="m-doc">Taskflow Processing Pipeline</a></li></ul></section>
</div>
</div>
</div>
</article></main>
<divclass="m-doc-search" id="search">
<ahref="#!" onclick="return hideSearch()"></a>
<divclass="m-container">
<divclass="m-row">
<divclass="m-col-m-8 m-push-m-2">
<divclass="m-doc-search-header m-text m-small">
<div><spanclass="m-label m-default">Tab</span> / <spanclass="m-label m-default">T</span> to search, <spanclass="m-label m-default">Esc</span> to close</div>