<!-- iframe showing the search results (closed by default) -->
<divid="MSearchResultsWindow">
<iframesrc="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<divclass="header">
<divclass="headertitle">
<divclass="title">C6: Manage Threads and Executor </div></div>
</div><!--header-->
<divclass="contents">
<divclass="textblock"><p>We discuss in this chapter the thread management and task execution schemes in Cpp-Taskflow. We will go through the concept of <em>thread</em>, <em>ownership</em>, and <em>executor</em> in Cpp-Taskflow.</p>
<p>Cpp-Taskflow defines a strict relationship between the master and workers. Master is the thread that creates the taskflow object and workers are threads that invoke the callable target of a task. Each taskflow object owns an executor instance that implements the execution of a task, for example, by a thread in a shared pool. By default, Cpp-Taskflow uses <aclass="elRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a> to decide the number of worker threads and <aclass="elRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/get_id.html">std::thread::get_id</a> to identify the ownership between the master and workers.</p>
<divclass="fragment"><divclass="line"><aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/io/basic_ostream.html">std::cout</a> << <aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a>() << <aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/io/manip/endl.html">std::endl</a>; <spanclass="comment">// 8, for example</span></div><divclass="line"><aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/io/basic_ostream.html">std::cout</a> << <aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/get_id.html">std::thread::get_id</a>() << <aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/io/manip/endl.html">std::endl</a>; <spanclass="comment">// master thread id</span></div><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> tf1; <spanclass="comment">// create a taskflow object with the default number of workers</span></div><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> tf2{4}; <spanclass="comment">// create a taskflow object with four workers</span></div></div><!-- fragment --><p>In the above example, the master thread owns both taskflow objects. The first taskflow object <code>tf1</code> creates eight (default by <aclass="elRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency.html">std::thread::hardware_concurrency</a>) worker threads and the second taskflow object <code>tf2</code> creates four worker threads. Including the master thread, there will be a total of 1 + 8 + 4 = 13 threads running in this program. If you create a taskflow with zero workers, the master will carry out all the tasks by itself. That is, using one worker and zero worker are conceptually equivalent to each other since they both end up using one thread to run all tasks (see the snippet below).</p>
<divclass="fragment"><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> tf1(0); <spanclass="comment">// one master, zero worker (master to run tasks)</span></div><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> tf2(1); <spanclass="comment">// one master, one worker (one thread to run tasks)</span></div></div><!-- fragment --><p>In general, the master thread is exposed to users at programming time (main thread), while the worker threads are transparently maintained by the taskflow object. Each taskflow object owns an executor managed by <aclass="elRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/memory/shared_ptr.html">std::shared_ptr</a>. The default executor implements a work stealing scheduler to efficiently carry out tasks. The Taskflow class defines a member type <aclass="el" href="classtf_1_1BasicTaskflow.html#afa5ea834928f68f950f59889c626c2ff" title="alias of executor type ">tf::Taskflow::Executor</a> as an alias of the associated executor type. Users can acquire an ownership of the executor from a taskflow object through the method <aclass="el" href="classtf_1_1BasicTaskflow.html#abe76e5288016861aaf1dafc0218d3084" title="shares ownership of the executor associated with this taskflow object ">tf::Taskflow::share_executor</a>.</p>
<divclass="fragment"><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> taskflow;</div><divclass="line"><aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/memory/shared_ptr.html">std::shared_ptr<tf::Taskflow::Executor></a> ptr = taskflow.<aclass="code" href="classtf_1_1BasicTaskflow.html#abe76e5288016861aaf1dafc0218d3084">share_executor</a>(); <spanclass="comment">// share the executor</span></div><divclass="line">assert(ptr.use_count() == 2); <spanclass="comment">// the executor is owned by ptr and tf</span></div></div><!-- fragment --><p>The <em>shared</em> property allows users to create their own resource manager and construct a taskflow object on top. The executor has only one constructor that takes an unsigned integer indicating the number of worker threads to spawn.</p>
<divclass="fragment"><divclass="line"><spanclass="keyword">auto</span> ptr = std::make_shared<tf::Taskflow::Executor>(4); <spanclass="comment">// create an executor of 4 workers</span></div><divclass="line"><aclass="code" href="classtf_1_1BasicTaskflow.html">tf::Taskflow</a> taskflow(ptr); <spanclass="comment">// create a taskflow object on top of the executor</span></div><divclass="line">assert(ptr.use_count() == 2); <spanclass="comment">// the executor is owned by ptr and tf</span></div></div><!-- fragment --><h1><aclass="anchor" id="C6_ShareAnExecutorAmongTaskflowObjects"></a>
Share an Executor among Taskflow Objects</h1>
<p>It is sometime useful to share one executor among multiple taskflow objects in order to avoid the thread <em>over-subscription</em> problem. In the case of over-subscription, the number of threads running in a program exceeds the number of available logical cores, resulting in additional and unnecessary context switches. Context switch has nonzero cost and is especially costly when it crosses cores. The following example mimics the over-subscription problem through a creation of 100 taskflow objects each with its own executor of four threads, assuming only four logical cores present in the machine.</p>
<divclass="fragment"><divclass="line"><spanclass="comment">// create 100 taskflow objects on top of the same executor</span></div><divclass="line"><aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/list.html">std::list<tf::Taskflow></a> tfs;</div><divclass="line"><spanclass="keyword">auto</span> executor = std::make_shared<tf::Taskflow::Executor>(4);</div><divclass="line"><spanclass="keywordflow">for</span>(<spanclass="keywordtype">size_t</span> i=0; i<100; ++i) {</div><divclass="line"> assert(executor.use_count() == i + 1); <spanclass="comment">// by the executor and each taskflow</span></div><divclass="line"> tfs.emplace_back(executor); <spanclass="comment">// create a taskflow object from the executor</span></div><divclass="line">}</div><divclass="line"><spanclass="comment">// a total of 1 + 4 = 5 threads running in this program</span></div></div><!-- fragment --><h1><aclass="anchor" id="C6CustomizeYourExecutorInterface"></a>
Customize Your Executor Interface</h1>
<p>Cpp-Taskflow permits users to define their own executor interface and integrate it into the taskflow object being built. In most cases, the executor is implemented as a thread pool to run given tasks. Your executor class must obey the following concepts in order to work with Cpp-Taskflow:</p>
<divclass="fragment"><divclass="line"><spanclass="keyword">template</span> <<spanclass="keyword">typename</span> C></div><divclass="line"><spanclass="keyword">class </span>MyExecutor { <spanclass="comment">// closure type C, callable on operator ()</span></div><divclass="line"></div><divclass="line"><spanclass="keyword">public</span>:</div><divclass="line"></div><divclass="line"> MyExecutor(<spanclass="keywordtype">unsigned</span>); <spanclass="comment">// constructor on a number of worker threads (might be zero)</span></div><divclass="line"><spanclass="keywordtype">size_t</span> num_workers() <spanclass="keyword">const</span>; <spanclass="comment">// return the number of worker threads (might be zero)</span></div><divclass="line"><spanclass="keyword">template</span> <<spanclass="keyword">typename</span>... ArgsT></div><divclass="line"><spanclass="keywordtype">void</span> emplace(ArgsT&&...); <spanclass="comment">// arguments to construct the closure C</span></div><divclass="line"><spanclass="keyword">template</span> <<spanclass="keyword">typename</span> C></div><divclass="line"><spanclass="keywordtype">void</span> batch(<aclass="codeRef" doxygen="/Users/twhuang/PhD/Code/cpp-taskflow/docs/cppreference-doxygen-web.tag.xml:http://en.cppreference.com/w/" href="http://en.cppreference.com/w/cpp/container/vector.html">std::vector<C></a>&); <spanclass="comment">// a vector of closures for batch insertions</span></div><divclass="line">};</div><divclass="line"><spanclass="keyword">using</span> MyTaskflow = <aclass="code" href="classtf_1_1BasicTaskflow.html">tf::BasicTaskflow<MyExecutor></a>;</div></div><!-- fragment --><p>The executor class template with one parameter on the task type. The task type can be a generic polymorphic function wrapper, for instance, <code>std::function<void()></code>, or a callable class with fixed memory layout. It is completely up to users to define how to invoke the task. Your executor class must meet the following concepts:</p>
<ul>
<li>a constructor on a given number of worker threads </li>
<li>a constant method num_workers to return the number of worker threads </li>
<li>a method emplace to dispatch a task to thread with arguments to forward to the constructor of the task </li>
<li>a method batch to dispatch multiple tasks in a vector to thread</li>
</ul>
<p>Cpp-Taskflow requires little requirement on the executor class. Each taskflow object has its own internal data structure to keep track of the lifetime and execution status of a task. The executor only needs to guarantee a thread to run the task given by the methods emplace and batch. We recommend users to read our built-in executor implementation <aclass="el" href="classtf_1_1WorkStealingThreadpool.html" title="Executor that implements an efficient work stealing algorithm. ">tf::WorkStealingThreadpool</a> for more details.</p>
<h1><aclass="anchor" id="C6ThreadSafety"></a>
Thread Safety</h1>
<p>The Taskflow object is <em>NOT</em> thread-safe. Touching a taskflow object from multiple threads can result in <em>undefined behavior</em>. Thread safety has nothing to do with the master nor the workers. It is completely safe to access the taskflow object as long as only one thread presents at a time. However, we strongly recommend users to acknowledge the definition of the master and the workers, and separate the program control flow accordingly. Having a clear thread ownership can greatly reduce the chance of buggy implementations and undefined behaviors.</p>
<h1><aclass="anchor" id="C6Example1"></a>
Example 1: Impact of Over-subscription</h1>
<p>The example below demonstrates the impact of thread over-subscription. The workload is a task dependency graph of four tasks doing compute-intensive matrix multiplication. We benchmarked the performance between the two implementations with and without sharing an executor.</p>
<li>Line 1-14 creates a task dependency graph composed of compute-intensive matrix operations </li>
<li>Line 16-40 creates multiple independent taskflow objects without sharing the running threads </li>
<li>Line 42-68 creates multiple taskflow objects from the same executor to run on the same set of threads</li>
</ul>
<p>Running the program on different number of taskflow objects gives the following runtime values:</p>
<divclass="fragment"><divclass="line"># taskflows shared (ms) unique (ms)</div><divclass="line"> 1 120 114</div><divclass="line"> 2 225 229</div><divclass="line"> 4 451 452</div><divclass="line"> 8 908 904</div><divclass="line"> 16 1791 1837</div><divclass="line"> 32 3581 3782</div><divclass="line"> 64 7183 7636</div><divclass="line"> 128 14341 15482</div></div><!-- fragment --><p>As we increase the number of taskflow objects, the implementation without sharing the executor encounters more context switches among threads. This overhead reflected on the slower runtime (15482 vs 14341 on 128 taskflow objects). </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<divid="nav-path" class="navpath"><!-- id is needed for treeview function! -->