CUDA Pro Tip: Write Flexible Kernels with Grid-Stride Loops
https://devblogs.nvidia.com/cuda-pro-tip-write-flexible-kernels-grid-stride-loops/
One of the most common tasks in CUDA programming is to parallelize a loop using a kernel. As an example, let’s use our old friend SAXPY. Here’s the basic sequential implementation, which uses a for loop. To efficiently parallelize this, we need to launch enough threads to fully utilize the GPU.
CUDA编程最常见的任务之一就是用一个kernel来并行化一个循环。比如,对于我们老朋友SAXPY,下面是一个基础的使用循环的实现。为了效率地并行化它,我们需要运行大量的线程来充分利用GPU。
void saxpy(int n, float a, float *x, float *y)
{
for (int i = ; i < n; ++i)
y[i] = a * x[i] + y[i];
}
Common CUDA guidance is to launch one thread per data element, which means to parallelize the above SAXPY loop we write a kernel that assumes we have enough threads to more than cover the array size.
通常CUDA指引会为每一个数据元素运行一个线程,意味着要并行化上述的SAXPY循环,我们需要假设我们写的kernel要有足够的线程以满足数组的大小。
__global__
void saxpy(int n, float a, float *x, float *y)
{
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n)
y[i] = a * x[i] + y[i];
}
I’ll refer to this style of kernel as a monolithic kernel, because it assumes a single large grid of threads to process the entire array in one pass. You might use the following code to launch the saxpy kernel to process one million elements.
我称这类kernel为monolithic kernel,因为它假设存在单个大的线程网格在一次同时处理,运行整个数组运算。你需要用下面的代码来运行一个具有百万元素的saxpy kernel
// Perform SAXPY on 1M elements
saxpy<<<,>>>(<<, 2.0, x, y);
Instead of completely eliminating the loop when parallelizing the computation, I recommend to use a grid-stride loop, as in the following kernel.
相比在并行化计算时完全消去循环,我更推荐使用一种grid-stride loop,如下
__global__
void saxpy(int n, float a, float *x, float *y)
{
for (int i = blockIdx.x * blockDim.x + threadIdx.x;
i < n;
i += blockDim.x * gridDim.x)
{
y[i] = a * x[i] + y[i];
}
}
Rather than assume that the thread grid is large enough to cover the entire data array, this kernel loops over the data array one grid-size at a time.
比起假设线程网格足够大得覆盖整个数组,这个kernel运行一次,就对数组进行一个grid-size的循环。
Notice that the stride of the loop is blockDim.x * gridDim.x
which is the total number of threads in the grid. So if there are 1280 threads in the grid, thread 0 will compute elements 0, 1280, 2560, etc. This is why I call this a grid-stride loop. By using a loop with stride equal to the grid size, we ensure that all addressing within warps is unit-stride, so we get maximum memory coalescing, just as in the monolithic version.
注意到这个循环的跨度是 blockDim.x * gridDim.x,它是一个线程网格中所有线程的数量。如果该线程网格中有1280个线程,那么编号为0的线程将执行元素0,1280,2560……这就是为什么我称之为“grid-stride loop”。使用一个跨度等于网格大小的循环,我们可以保证了所有地址都是unit-stride的,于是我们比起monolithic的版本减少了最大的内存消耗。
When launched with a grid large enough to cover all iterations of the loop, the grid-stride loop should have essentially the same instruction cost as the if
statement in the monolithic kernel, because the loop increment will only be evaluated when the loop condition evaluates to true.
grid-stride循环比起monolithic kernel,也会需要相同的计算消耗在if语句上,因为循环的条件为真时循环才会继续进行(在这里隐式地产生了if的消耗)。
There are several benefits to using a grid-stride loop.
1.Scalability and thread reuse. By using a loop, you can support any problem size even if it exceeds the largest grid size your CUDA device supports. Moreover, you can limit the number of blocks you use to tune performance. For example, it’s often useful to launch a number of blocks that is a multiple of the number of multiprocessors on the device, to balance utilization. As an example, we might launch the loop version of the kernel like this.
1.稳定性及线程复用。当使用一个循环,你可以支持任何显存大小的运算甚至包括它超出了CUDA设备(一次性)支持的最大值。除此之外,你可以限制线程块数量来调整运行效率。比如,为平衡资源使用,载入一定数量的具有不同multiprocessors的线程块,是非常有用的。
int numSMs;
cudaDeviceGetAttribute(&numSMs, cudaDevAttrMultiProcessorCount, devId);
// Perform SAXPY on 1M elements
saxpy<<<*numSMs, >>>( << , 2.0, x, y);
CUDA Pro Tip: Write Flexible Kernels with Grid-Stride Loops的更多相关文章
- CUDA Pro Tip: Optimized Filtering with Warp-Aggregated Atomics
In this post, I’ll introduce warp-aggregated atomics, a useful technique to improve performance when ...
- CUDA Pro:通过向量化内存访问提高性能
CUDA Pro:通过向量化内存访问提高性能 许多CUDA内核受带宽限制,而新硬件中触发器与带宽的比率不断提高,导致带宽受限制的内核更多.这使得采取措施减轻代码中的带宽瓶颈非常重要.本文将展示如何在C ...
- cuda编程-卷积优化
CUDA Convolution https://www.evl.uic.edu/sjames/cs525/final.html Improve Image Processing Using GPU ...
- CUDA 8混合精度编程
CUDA 8混合精度编程 Mixed-Precision Programming with CUDA 8 论文地址:https://devblogs.nvidia.com/mixed-precisio ...
- Ext.js中的tip事件实际使用
Ext.onReady(function () { // Init the singleton. Any tag-based quick tips will start working. Ext.ti ...
- CUDA学习笔记(二)——CUDA线程模型
转自:http://blog.sina.com.cn/s/blog_48b9e1f90100fm5b.html 一个grid中的所有线程执行相同的内核函数,通过坐标进行区分.这些线程有两级的坐标,bl ...
- ExtJs 4: How To Add Grid Cell Tooltip
最近忙一个项目的时候需要实现鼠标移到grid的某一行上提示消息.花了半天时间才解决.在网上找很久终于有找到一个有用的.我的版本是extjs4. 效果如图 Ext.onReady(function () ...
- CUDA性能优化----warp深度解析
本文转自:http://blog.163.com/wujiaxing009@126/blog/static/71988399201701224540201/ 1.引言 CUDA性能优化----sp, ...
- Linux下Qt+CUDA调试并运行
Qt与CUDA相结合具体的操作主要修改qt项目中的配置文件pro.下面以测试的项目为例. 因为这是一个测试案例,代码很简单,下面将这几个文件的代码贴出来,方面后面对应pro文件和Makefile文件中 ...
随机推荐
- Java IO: FileInputStream
原文链接 作者: Jakob Jenkov 译者: 李璟(jlee381344197@gmail.com) FileInputStream可以以字节流的形式读取文件内容.FileInputStream ...
- Windows下使用swoole的环境搭建
Cygwin 官方地址:http://www.cygwin.com/ swoole 官方下载地址:https://github.com/swoole/swoole-src/releases 方法/步骤 ...
- codeforces #332 div 2 D. Spongebob and Squares
http://codeforces.com/contest/599/problem/D 题意:给出总的方格数x,问有多少种不同尺寸的矩形满足题意,输出方案数和长宽(3,5和5,3算两种) 思路:比赛的 ...
- 【转载】python3安装scrapy之windows32位爬坑
python3安装scrapy之windows32位爬坑 原创 2016年11月06日 01:38:08 标签: scrapy / windows / python / 开源框架 / 网络爬虫 早 ...
- 4-CSS规范
4.1 命名规范:4.1.1 css文件命名 reset.css 重置样式,重置元素默认样式,使得页面在所有浏览器中有统一的外观 global.css 全局样式,全站公用,定义页面基础样式(常见的公共 ...
- sms短信服务
短信服务是app,电商类应用的基础功能.典型场景有: 用户注册,发送验证码 用户找回验证,发送验证码 用户账户异常,发送提示 用户账户变化,通知用户 短信服务开发有几个注意点: 供应商选型 短信模板 ...
- 【C++基础】008常量和变量
简介:常量和变量. 常量和变量 1. 常量 具体把数据写出来 2,3,4: 1.2,1.3: "Hello World!","C++": cout <&l ...
- 初识SpringAOP
概述 AOP(Aspect Oriented Programming),即面向切面编程 通过预编译方式和运行期动态代理实现程序功能的统一维护的一种技术.AOP是OOP的延伸,是软件系统开发中的一个 ...
- .NET Core 获取程序运行环境信息与反射的应用
目录 RuntimeInformation.Environment 获取信息 反射获取信息 获取某个属性的值以及别名 反射获取信息 使用 总结 笔者的九篇反射系统文章已经完结,但是笔者会持续更新反射在 ...
- 6487. 【GDOI2020模拟02.29】列强争霸war
题目描述 区间绝对众数 即出现次数>len/2下取整的数 对于区间[L,R]扫一遍,维护一个数x和出现次数s 当前数=x则s+1,否则s-1,若s已为0则把x设为当前数 若区间内存在绝对众数,那 ...