<p>Taskflow comes with a built-in profiler, <em>TFProf</em>, for you to profile and visualize taskflow programs.</p><imgclass="m-image" src="tfprof.png" alt="Image" style="width: 100%;" /><sectionid="ProfilerEnableTFProf"><h2><ahref="#ProfilerEnableTFProf">Enable Taskflow Profiler</a></h2><p>All taskflow programs come with a lightweight profiling module to observer worker activities in every executor. To enable the profiler, set the environment variable <code>TF_ENABLE_PROFILER</code> to a file name in which the profiling result will be stored.</p><preclass="m-console"><spanclass="go">~$ TF_ENABLE_PROFILER=result.json ./my_taskflow</span>
<spanclass="go">]</span></pre><p>When the program finishes, it generates and saves the profiling data to <code>result.json</code> in JavaScript Object Notation (JSON) format. You can then paste the JSON data to our web-based interface, <ahref="https://taskflow.github.io/tfprof/">Taskflow Profiler</a>, to visualize the execution timelines of tasks and workers. The web interface supports the following features:</p><ul><li>zoom into a selected window</li><li>double click to zoom back to the previously selected window</li><li>filter workers</li><li>mouse over to show the tooltip of the task</li><li>rank tasks in decreasing order of criticality (i.e., execution time)</li></ul><p>TFProf implements a clustering-based algorithm to efficiently visualize tasks and their execution timelines in a browser. Without losing much visual accuracy, each <em>clustered</em> task indicates a group of adjacent tasks clustered by the algorithm, and you can zoom in to see these tasks.</p></section><sectionid="ProfilerEnableTFProfServer"><h2><ahref="#ProfilerEnableTFProfServer">Enable Taskflow Profiler on a HTTP Server</a></h2><p>When profiling large taskflow programs, the method in the previous section may not work because of the limitation of processing large JSON files. For example, a taskflow program of a million tasks can produce several GBs of profiling data, and the profile may respond to your requests very slowly. To solve this problem, we have implemented a C++-based http server optimized for our profiling data. To compile the server, enable the cmake option <code>TF_BUILD_PROFILER</code>. You may visit <ahref="install.html" class="m-doc">Building and Installing</a> to understand Taskflow's build environment.</p><preclass="m-console"><spanclass="gp"># </span>under<spanclass="w"></span>the<spanclass="w"></span>build<spanclass="w"></span>directory
<spanclass="go">~$ make</span></pre><p>After successfully compiling the server, you can find the executable at <code>tfprof/server/tfprof</code>. Now, generate profiling data from running a taskflow program but specify the output file with extension .tfp.</p><preclass="m-console"><spanclass="go">~$ TF_ENABLE_PROFILER=my_taskflow.tfp ./my_taskflow</span>
<spanclass="go">~$ ls</span>
<spanclass="go">my_taskflow.tfp # my_taskflow.tfp is of binary format</span></pre><p>Launch the server program <code>tfprof/server/tfprof</code> and pass (1) the directory of <code>index.html</code> (default at <code>tfprof/</code>) via the option <code>--mount</code> and (2) the <code>my_taskflow.tfp</code> via the option <code>--input</code>.</p><preclass="m-console"><spanclass="gp"># </span>under<spanclass="w"></span>the<spanclass="w"></span>build/<spanclass="w"></span>directory
<spanclass="go">~$ ./tfprof/server/tfprof --mount ../tfprof/ --input my_taskflow.tfp</span></pre><p>Now, open your favorite browser at <code>localhost:8080</code> to visualize and profile your <code>my_taskflow</code> program.</p><imgclass="m-image" src="tfprof-local.png" alt="Image" /><p>The compiled profiler is a more powerful version than the pure JavaScript-based interface and it is able to more efficiently handle large profiling data under different queries. We currently support the following two view types:</p><ul><li>Cluster: visualize the profiling data using a clustering algorithm with a limit</li><li>Criticality: visualize the top-limit tasks in decreasing order of their execution times</li></ul></section><sectionid="ProfilerDisplayProfileSummary"><h2><ahref="#ProfilerDisplayProfileSummary">Display Profile Summary</a></h2><p>You can display a profile summary by specifying only the environment variable <code>TF_ENABLE_PROFILER</code> without any value. The Taskflow will generate a separate summary report of tasks and workers for each executor created by the program.</p><preclass="m-console"><spanclass="gp"># </span><spanclass="nb">enable</span><spanclass="w"></span>the<spanclass="w"></span>environment<spanclass="w"></span>variable<spanclass="w"></span>without<spanclass="w"></span>any<spanclass="w"></span>value
<spanclass="go"> 18 5 0.277778 0 4</span></pre><p>The report consists of two sections, task summary and worker summary. In the first section, the summary reports for each task type the number of executions (<code>Count</code>), the total execution time (<code>Time</code>), average execution time per task (<code>Avg</code>), and the minimum (<code>Min</code>) and the maximum (<code>Max</code>) execution time among all tasks. Similarly in the second section, the summary reports for each worker the task execution statistics.</p></section>
</div>
</div>
</div>
</article></main>
<divclass="m-doc-search" id="search">
<ahref="#!" onclick="return hideSearch()"></a>
<divclass="m-container">
<divclass="m-row">
<divclass="m-col-m-8 m-push-m-2">
<divclass="m-doc-search-header m-text m-small">
<div><spanclass="m-label m-default">Tab</span> / <spanclass="m-label m-default">T</span> to search, <spanclass="m-label m-default">Esc</span> to close</div>