Transcription of CUDA C/C++ Streams and Concurrency
{{id}} {{{paragraph}}}
Example: bachelor of science
CUDA C/C++ Streams and Concurrency
CUDA C/C++ Streams and ConcurrencySteve RennichNVIDIAC oncurrencyThe ability to perform multiple CUDA operations simultaneously(beyond multi-threaded parallelism)CUDA Kernel <<<>>>cudaMemcpyAsync (HostToDevice)cudaMemcpyAsync (DeviceToHost)Operations on the CPUF ermi architecture can simultaneously support(compute capability +)Up to 16 CUDA kernels on GPU2 cudaMemcpyAsyncs (must be in different directions)Computation on the CPUS treamsStreamA sequence of operations that execute in issue-order on the GPUP rogramming model used to effect concurrencyCUDA operations in different Streams may run concurrentlyCUDA operations from different Streams may be interleavedK1K2K3K4 Concurrency ExampleSerialConcurrent overlap kernel and D2H copycudaMemcpyAsync(H2D)cudaMemcpyAsync( D2H)Kernel<<<>>>timecudaMemcpyAsync(H2D) performance improvementstreamsHD2K2HD3DH2 Amount of ConcurrencySerial (1x)2-way Concurrency (up to 2x)3-way Concurrency (up to 3)
t i m e execution HD2 HD3 K1 K2 K3 DH1 DH2 DH3 HD1HD2 HD3 K2 K3 DH3 HD1 K1 DH1 DH2 HD2 HD3 K2 K3 DH3 Kernels no longer issued sequentially runtime = 5. Previous Architectures Compute Capability 1.0+ Support for GPU / CPU concurrency Compute Capability 1.1+ ( i.e. C1060 ) Adds support for asynchronous memcopies (single engine )
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: