CUDASTDScan
    Parallel Scan


Taskflow provides standard template methods for scanning a range of items on a CUDA GPU.
Scan a Range of Items
tf::cuda_inclusive_scan computes an inclusive prefix sum operation using the given binary operator over a range of elements specified by [first, last). The term "inclusive" means that the i-th input element is included in the i-th sum. The following code computes the inclusive prefix sum over an input array and stores the result in an output array.constsize_tN=1000000;
int*input=tf::cuda_malloc_shared<int>(N);//inputvector
int*output=tf::cuda_malloc_shared<int>(N);//outputvector

//initializesthedata
for(size_ti=0;i<N;i++)
input[i]=rand();
}
//computesinclusivescanoverinputandstorestheresultinoutput
tf::cuda_inclusive_scan(tf::cudaDefaultExecutionPolicy{},
input,input+N,output,[]__device__(inta,intb){returna+b;}
);
//verifiestheresult
for(size_ti=1;i<N;i++){
assert(output[i]==output[i-1]+input[i]);
}
On the other hand, tf::cuda_exclusive_scan computes an exclusive prefix sum operation. The term "exclusive" means that the i-th input element is NOT included in the i-th sum.//computesexclusivescanoverinputandstorestheresultinoutput
tf::cuda_exclusive_scan(tf::cudaDefaultExecutionPolicy{},
input,input+N,output,[]__device__(inta,intb){returna+b;}
);
//verifiestheresult
for(size_ti=1;i<N;i++){
assert(output[i]==output[i-1]+input[i-1]);
}


Scan a Range of Transformed Items
tf::cuda_transform_inclusive_scan transforms each item in the range [first, last) and computes an inclusive prefix sum over these transformed items. The following code multiplies each item by 10 and then compute the inclusive prefix sum over 1000000 transformed items.constsize_tN=1000000;
int*input=tf::cuda_malloc_shared<int>(N);//inputvector
int*output=tf::cuda_malloc_shared<int>(N);//outputvector

//initializesthedata
for(size_ti=0;i<N;i++)
input[i]=rand();
}
//computesinclusivescanovertransformedinputandstorestheresultinoutput
tf::cuda_transform_inclusive_scan(tf::cudaDefaultExecutionPolicy{},
input,input+N,output,
[]__device__(inta,intb){returna+b;},//binaryscanoperator
[]__device__(inta){returna*10;}//unarytransformoperator
);
//verifiestheresult
for(size_ti=1;i<N;i++){
assert(output[i]==output[i-1]+input[i]*10);
}
Similarly, tf::cuda_transform_exclusive_scan performs an exclusive prefix sum over a range of transformed items. The following code computes the exclusive prefix sum over 1000000 transformed items each multipled by 10.constsize_tN=1000000;
int*input=tf::cuda_malloc_shared<int>(N);//inputvector
int*output=tf::cuda_malloc_shared<int>(N);//outputvector

//initializesthedata
for(size_ti=0;i<N;i++)
input[i]=rand();
}

//computesexclusivescanovertransformedinputandstorestheresultinoutput
tf::cuda_transform_exclusive_scan(tf::cudaDefaultExecutionPolicy{},
input,input+N,output,
[]__device__(inta,intb){returna+b;},//binaryscanoperator
[]__device__(inta){returna*10;}//unarytransformoperator
);

//verifiestheresult
for(size_ti=1;i<N;i++){
assert(output[i]==output[i-1]+input[i-1]*10);
}


Invoke Scan Kernels Asynchronously
By default, scan functions block until all kernels finish. You can invoke these kernels asynchronously and explicitly synchronize them at another place of your program. Since our scan kernels rely on additional device memory, you need to provide a buffer of size in bytes equal to (or larger than) the value returned by tf::cuda_scan_buffer_size.constsize_tN=1000000;
int*input=tf::cuda_malloc_shared<int>(N);//inputvector
int*output=tf::cuda_malloc_shared<int>(N);//outputvector

//initializesthedata
for(size_ti=0;i<N;i++)
input[i]=rand();
}

//queriestherequiredbuffersizetoscanNelementsusingthegivenpolicy
autobytes=tf::cuda_scan_buffer_size<tf::cudaDefaultExecutionPolicy,int>(N);
void*buffer=tf::cuda_malloc_device<std::byte>(bytes);

//invoketheinclusivescankernelsasynchronously(sameforotherscanvariants)
tf::cuda_inclusive_scan_async(tf::cudaDefaultExecutionPolicy{my_stream},
input,input+N,output,[]__device__(inta,intb){returna+b;},buffer
);
//here,outputmaynotbereadyyet,askernelsarestillrunning...

//synchronizethestreamtomakeoutputready
cudaStreamSynchronize(my_stream);
The allocated buffer must remain alive until the asynchronous call completes.The stream given to an asynchronous scan function can be enabled to the capture mode to capture internal kernels into a CUDA graph.