<divclass="textblock"><p>We study a taskflow processing pipeline where each stage runs an entire task graph rather than a single function. This example demonstrates how Taskflow combines task graph parallelism <em>inside</em> each stage with pipeline parallelism <em>across</em> stages, achieving two levels of parallelism simultaneously.</p>
<p>Many real-world pipelines require each stage to run a parallel algorithm, not a single function. We model this as a sequence of tokens flowing through three serial pipes, where each pipe embeds a full taskflow graph:</p>
<divclass="dotgraph">
<iframescrolling="no" frameborder="0" src="dot_taskflow_processing_pipeline_2.svg" width="583" height="251"><p><b>This browser is not able to show SVG: try Firefox, Chrome, Safari, or Opera instead.</b></p></iframe></div>
<divclass="dotgraph">
<iframescrolling="no" frameborder="0" src="dot_taskflow_processing_pipeline_1.svg" width="622" height="423"><p><b>This browser is not able to show SVG: try Firefox, Chrome, Safari, or Opera instead.</b></p></iframe></div>
<p>Within each pipe, tasks inside the embedded taskflow run in parallel. Across pipes, the pipeline scheduler overlaps execution of different tokens in different stages. This produces two-level parallelism: <em>intra-stage</em> (task graph) and <em>inter-stage</em> (pipeline).</p>
<p>We define three taskflows — one per stage — each with a different internal structure to illustrate the flexibility of this model. We then create a pipeline of three serial pipes, each running its corresponding taskflow via <aclass="el" href="classtf_1_1Executor.html#a8fcd9e0557922bb8194999f0cd433ea8" title="runs a target graph and waits until it completes using an internal worker of this executor">tf::Executor::corun</a>:</p>
<divclass="ttc" id="aclasstf_1_1Executor_html_a519777f5783981d534e9e53b99712069"><divclass="ttname"><ahref="classtf_1_1Executor.html#a519777f5783981d534e9e53b99712069">tf::Executor::run</a></div><divclass="ttdeci">tf::Future< void > run(Taskflow &taskflow)</div><divclass="ttdoc">runs a taskflow once</div></div>
<divclass="ttc" id="aclasstf_1_1Executor_html_a8fcd9e0557922bb8194999f0cd433ea8"><divclass="ttname"><ahref="classtf_1_1Executor.html#a8fcd9e0557922bb8194999f0cd433ea8">tf::Executor::corun</a></div><divclass="ttdeci">void corun(T &target)</div><divclass="ttdoc">runs a target graph and waits until it completes using an internal worker of this executor</div></div>
<divclass="ttc" id="aclasstf_1_1FlowBuilder_html_a4d52a7fe2814b264846a2085e931652c"><divclass="ttname"><ahref="classtf_1_1FlowBuilder.html#a4d52a7fe2814b264846a2085e931652c">tf::FlowBuilder::emplace</a></div><divclass="ttdeci">Task emplace(C &&callable)</div><divclass="ttdoc">creates a static task</div><divclass="ttdef"><b>Definition</b> flow_builder.hpp:1781</div></div>
<divclass="ttc" id="aclasstf_1_1FlowBuilder_html_a90f3d9b9d6fcf4df8e7d7878dfdd130d"><divclass="ttname"><ahref="classtf_1_1FlowBuilder.html#a90f3d9b9d6fcf4df8e7d7878dfdd130d">tf::FlowBuilder::linearize</a></div><divclass="ttdeci">void linearize(std::vector< Task > &tasks)</div><divclass="ttdoc">adds adjacent dependency links to a linear list of tasks</div><divclass="ttdef"><b>Definition</b> flow_builder.hpp:1909</div></div>
<divclass="ttc" id="aclasstf_1_1Task_html_a8c78c453295a553c1c016e4062da8588"><divclass="ttname"><ahref="classtf_1_1Task.html#a8c78c453295a553c1c016e4062da8588">tf::Task::precede</a></div><divclass="ttdeci">Task & precede(Ts &&... tasks)</div><divclass="ttdoc">adds precedence links from this to other tasks</div><divclass="ttdef"><b>Definition</b> task.hpp:1305</div></div>
<divclass="ttc" id="aclasstf_1_1Taskflow_html"><divclass="ttname"><ahref="classtf_1_1Taskflow.html">tf::Taskflow</a></div><divclass="ttdoc">class to create a taskflow object</div><divclass="ttdef"><b>Definition</b> taskflow.hpp:64</div></div>
</div><!-- fragment --><h2><aclass="anchor" id="TaskflowPipelineCorun"></a>
Why corun Instead of run</h2>
<p>Each pipe callable is itself executed by a worker thread. If we called <code>executor.run(taskflows[...])</code>.wait() inside the pipe, that worker would block waiting for the inner taskflow — preventing it from helping with other available tasks and potentially causing deadlock if all workers are blocked.</p>
<p><aclass="el" href="classtf_1_1Executor.html#a8fcd9e0557922bb8194999f0cd433ea8" title="runs a target graph and waits until it completes using an internal worker of this executor">tf::Executor::corun</a> avoids this: the calling worker stays active in the work-stealing loop while the inner taskflow executes, ensuring forward progress and preventing deadlock:</p>
<divclass="fragment"><divclass="line"><spanclass="comment">// correct: calling worker participates in executing the inner taskflow</span></div>
</div><!-- fragment --><h2><aclass="anchor" id="TaskflowPipelineStorage"></a>
Taskflow Storage</h2>
<p>Since all three pipes are serial, at most one token occupies each stage at any time. A one-dimensional array of taskflows — one per stage — is therefore sufficient:</p>
</div><!-- fragment --><p>If any pipe were declared parallel, multiple tokens could be at the same stage simultaneously across different lines, requiring a two-dimensional storage of size <code></code>(num_lines × num_pipes).</p>
<p>The outer task graph, including pipeline composition, is shown below:</p>
<divclass="dotgraph">
<iframescrolling="no" frameborder="0" src="dot_taskflow_processing_pipeline_3.svg" width="468" height="350"><p><b>This browser is not able to show SVG: try Firefox, Chrome, Safari, or Opera instead.</b></p></iframe></div>
</div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
<!-- HTML footer for doxygen 1.13.1-->
<!-- start footer part -->
<divid="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<liclass="navelem"><aclass="el" href="Examples.html">Learning from Examples</a></li>
<liclass="footer">
Maintained by <ahref="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a>
—
Generated by <ahref="https://www.doxygen.org/index.html"><imgclass="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.13.1