RealtimeRendering III

【RealtimeRendering III】

1、砖块渲染实例。

　　1）brick & mortar diffuse texture.

　　2）brick & mortar gloss texture.

　　3）applying bump mapping, the surface normals of the bricks may be varied so that when they are rendered, they do not appear to be perfectly smooth.

　　4）Parallx or Relief Mapping.

2、The pixels in the image texture are often called texels. Texture Pipeline：

　　　　

　　Parameter Space Coordinates 就是UV。

　　　　

3、Projector functions commonly used in modeling programs include spherical, cylindrical, and planar projections.  Other inputs can be used to a projector function. For example, the surface normal can be used to choose which of six planar projection directions is used for the surface.

4、Corresponder functions convert parameter-space coordinates to texturespace locations.

　　What happens outside of this range? Corresponder functions determine the behavior. In OpenGL, this type of corresponder function is called the “wrapping mode”; in DirectX, it is called the “texture addressing mode.” Common corresponder functions of this type are:

　　

　　　　• wrap (DirectX), repeat (OpenGL), or tile

　　　　• mirror

　　　　• clamp

　　　　• border

　　These corresponder functions can be assigned differently for each texture axis, e.g., the texture could repeat along the u-axis and be clamped on the v-axis.

5、 Texture Values

　　In the case of procedural texturing, the process of obtaining a texture value from a texture-space location does not involve a memory lookup, but rather the computation of a function.

6、The texture image size used in GPUs is usually 2^m × 2^n texels, where m and n are nonnegative integers.  Graphics accelerators have different upper limits on texture size. A typical DirectX 9 laptop allows a maximum of 2048 × 2048, a desktop 40962, and a DirectX 10 machine 81922 texels.

　　　　

　　　　1）nearest neighbor

　　　　　　individual texels may become apparent. This effect is called pixelation and occurs because the method takes the value of the nearest texel to each pixel center when magnifying, resulting in a blocky appearance.

　　　　2）bilinear interpolation

　　　　　　this kind of filtering finds the four neighboring texels and linearly interpolates in two dimensions to find a blended value for the pixel.  The result is blurrier, and much of the jaggedness from using the nearest neighbor method has disappeared.

　　　　3）cubic convolution（icubic interpolation）

　　　　　　the remaining blockiness is removed. It should be noted that bicubic filters are more expensive than bilinear filters.

7、bilinear interpolation。

　　

　　

　　A common solution to the blurriness that accompanies magnification is to use detail textures.

　　解决 magnification 的一种常用做法是 detail textures.

　　These are textures that represent fine surface details, from scratches on a cellphone to bushes on terrain. Such detail is overlaid onto the magnified texture as a separate texture, at a different scale.

　　The high-frequency repetitive pattern of the detail texture, combined with the low-frequency magnified texture, has a visual effect similar to the use of a single very high resolution texture.

8、Magnification的问题。　　

　　Using bilinear interpolation gives varying grayscale samples across the texture. By remapping so that, say, all grays lower than 0.4 are black, all grays higher than 0.6 are white, and those in between are stretched to fill the gap, the texture looks more like a checkerboard again.

　　在放大图片时，双线性插值有天生的问题，比如下图左1，将原清晰的小图放大并且双纯属插值后，得到一张非常模糊的图。我们实际需要的是一张清晰的大图。

　　　　

9、Minification.

　　最接近点采样，会产生很多锯齿。

　　　　

　　 Bilinear interpolation, again working exactly as in the magnification filter. This filter is only slightly better than the nearest neighbor approach for minification. It blends four texels instead of using just one, but when a pixel is influenced by more than four texels, the filter soon fails and produces aliasing.

　　Due to the Nyquist limit，we need to make sure that the texture's signal frequency is no greater than half the sample frequency.

       For example. Say an image is composed of alternating black and white lines, a texel apart. The wavelength is then two texels wide (from black line to black line), so the frequency is 1/2 .

　　To properly display this texture on a screen, the frequency must then be at least 2 × 1/2, i.e., at least one pixel per texel. So, for textures in general, there should be at most one texel per pixel to avoid aliasing.

10、Mipmapping

　　

　　Use of mipmapping on the GPU is why square textures with powers-of-two resolutions are the norm for use on models.

　　

　　For textures encoded in a nonlinear space (such as most color textures), ignoring gamma correction when filtering will modify the perceived brightness of the mipmap levels [121]. As you get farther away from the object and the uncorrected mipmaps get used, the object can look darker overall, and contrast and details can also be affected. For this reason, it is important to convert such textures into linear space, perform all mipmap filtering in that space, and convert the final results back into nonlinear space for storage.

　　为使用了 sRGB 的贴图生成 mipmap时，需要将期转到到 linear空间中计算，保存时再加上 sRGB。

　　The (u, v, d) triplet is used to access the mipmap. The value d is analogous to a texture level, but instead of an integer value, d has the fractional value of the distance between levels. The texture level above and the level below the d location is sampled. The (u, v) location is used to retrieve a bilinearly interpolated sample from each of these two texture levels. The resulting sample is then linearly interpolated, depending on the distance from each texture level to d. This entire process is called trilinear interpolation and is performed per pixel.

　　三线性过滤。

　　　　

　　One user control on the d coordinate is the level of detail bias (LOD bias)。

　　However, mipmapping has a number of flaws.  A major one is overblurring. When a viewer looks along a textured surface nearly edge-on.  In fact, it is possible to need minification along one axis of the texture and magnification along the other.

　　mipmapping 也会有瑕疵，如常见的 overblurring。原因如下：

　　The effect of accessing the mipmap is that square areas on the texture are retrieved; retrieving rectangular areas is not possible. To avoid aliasing, we choose the largest measure of the approximate coverage of the pixel cell on the texture. This results in the retrieved sample often being relatively blurry.

　　

11、ripmap.

　　One extension to mipmapping is the ripmap.  The idea is to extend the mipmap to include downsampled rectangular areas as subtextures that can be accessed [518]. The mipmap is stored 1 × 1, 2 × 2, 4 × 4, etc., but all possible rectangles are also stored 1 × 2, 2 × 4, 2 × 1, 4 × 1, 4 × 2, etc.

　　The u and v coverage determines the rectangle’s dimensions. While this technique produces better visuals than mipmapping, it comes at a high cost and so normally is avoided in favor of other methods. Instead of adding just one-third storage space, as is needed for mipmapping, the cost is three times additional space beyond the original image.

　　mipmap会增加 1/3的存储空间，而ripmap会增加3倍的存储空间。

12、Summed-Area Table

　　To use this method, one first creates an array that is the size of the texture but contains more bits of precision for the color stored (e.g., 16 bits or more for each of red, green, and blue). At each location in this array, one must compute and store the sum of all the corresponding texture’s texels in the rectangle formed by this location and texel (0, 0) (the origin).

　　　　　　

　　　　　　

　　　　　　

　　　　　　The lines going to the horizon are sharper near the right edge, but the diagonally crossing lines in the middle are still overblurred.

　　　　　　Ripmaps and summed-area tables are examples of what are called anisotropic filtering algorithms. Both schemes are memory intensive.  Such algorithms are schemes that can retrieve texel values over areas that are not square.

　　　　　　Summed-area tables take at least two times as much memory.

13、Unconstrained Anisotropic Filtering

　　　Instead of using a single mipmap sample to approximate this quad’s coverage, the algorithm uses a number of squares to cover the quad.  The shorter side of the quad can be used to determine d (unlike in mipmapping, where the longer side is often used); this makes the averaged area smaller (and so less blurred) for each mipmap sample. longer side is used to create a line of anisotropy parallel to the longer side and through the middle of the quad. When the amount of anisotropy is between 1:1 and 2:1, two samples are taken along this line (see Figure 6.17). At higher ratios of anisotropy, more samples are taken along the axis.

　　　　

　　This scheme allows the line of anisotropy to run in any direction, and so does not have the limitations that ripmaps and summed-area tables had.  It also requires no more texture memory than mipmaps do, since it uses the mipmap algorithm to do its sampling.

　　　　

14、Cubemap

　　Cube maps support bilinear filtering as well as mipmapping, but problems can occur along seams of the map, where the square faces join. all graphics hardware can not
sample across these boundaries when performing bilinear interpolation

　　Cubemap在bilinear filtering下会产生方块感。因为GPU不能跨纹理采样。

　　The angular size of a texel varies over a cube face. That is, a texel at the center of a cube face represents a greater visual solid angle than a texel at the corner.

　　边缘区域texel被采样时，视角必然更大，而中间区域采样时，视角更加垂直。所以边缘区域色彩通常会更低。

　　ATI CubeMapGen, take these factors into account when filtering. Neighboring samples from other faces are used to create the mipmap chain, and the angular extent of each texel is taken into account.

　　　　

15、Texture Caching

　　Clipmap.

　　The idea is that the entire dataset is treated as a mipmap, but only a small part of the lower levels of the mipmap is required for any particular view.

　　针对Mipmap的一些缺点，提出了Clipmap方法。Clipmap方法首先对原始大纹理创建Mipmap，然后选择一个分割尺寸，再把每一个Mipmap层分割成矩形网格状的片，避免了传统方法先把大纹理分割成小纹理再进行Mipmap过滤，而必须处理纹理边界的做法。这种分割是在大纹理被指定在单一的坐标系统中，对ＭMipmap每一层进行的，与几何无关。

16、Texture Compression

　　DXTC/BC， Encoding is done on 4x4 texel blocks, also called tiles. For each encoded quantity, two reference values are stored, and then for each of the 16 texels in the block, an interpolation factor is given to select a value along the line between the two reference values.

　　S3TC纹理压缩方法基于索引的思想。把纹理按4x4个单元（像素点）大小划分为块。每个块对应一张四色查找表，表中存有两个标准RGB565格式表示的16位颜色，另外使用标准插入算法在插入两个新的颜色值，由此构成四色查找表。4x4大小的纹理块中每个单元（像素点）用两个bit（00 01 10 11）表示，每一个都代表四色查找表中的一种颜色。可以看出，实质上是利用每个单元（像素点）中的两个bit来索引四色查找表中的颜色值。

　　对于不透明纹理贴图，每个单元（像素点）用2个bit表示，共32bit，加上每个块需要表示两个16位的颜色，总共，每个块被编码为64个bit。简单透明纹理（单色透明）保留四色查找表中的第四个颜色来表示该单元透明，第三个颜色是给出的两个颜色的平均值。区分不透明纹理和简单透明纹理的方法是看给出两个颜色的存放次序。

　　1） DXT1：上述标准算法。Taking up 8 bytes for the block, or 4 bits per texel. This represents a 6:1 texture compression ratio, compared to an uncompressed 24-bit RGB texture.

　　2） DXT3：In addition, each texel has a 4-bit alpha value stored separately.  takes up 16 bytes, or 8 bits per texel. This represents a 4:1 texture compression ratio, compared to an uncompressed 32-bit RGBA texture.

　　3）DXT5：In addition, alpha data is encoded using two 8-bit reference values.  Per texel
3-bit interpolation factor. 

17、法线压缩的 BC5/3Dc算法。

　　法线只需存储两个component，因为第三个component可以通过以下方程算出来。

　　

　　This compression technique reduces the memory required from 32 bytes (16 colors × 2 components/color × 1 byte/component) to 16 bytes.

　　The algorithm works on 4×4 blocks of texels. Instead of storing 16 colors for both components, the algorithm stores 2 reference colors for each component (red_0, red_1, green_0, and green_1) and 16 3-bit color indices for each component (red a through red p and green a through green p), as shown in the following diagram.

　　

　　The algorithm uses the 3-bit indices to look up colors from a color table that contains 8 colors. The first two colors—red_0 and red_1 (or green_0 and green_1)—are the minimum and maximum colors. The algorithm calculates the remaining colors using linear interpolation.

　　

参考：https://msdn.microsoft.com/zh-cn/library/windows/desktop/bb694531(v=vs.85).aspx#BC5

18、Texture Animatoin，即UV动画，可以用于实现瀑布。

19、Alpha Mapping.

　　decaling(贴花)。

　　cross tree，两个一样的模型，其中一个旋转90度，形成3D模型。

　　

　　　cross tree的问题是，从高处看时，会穿帮。

　　　

20、Alpha to Coverage　　

　　This idea is similar to screen-door transparency.

　　详细过程：对于每个fragment，计算每个sample的coverage。运行pixel shader得到alpha。以8x samples的framebuffer为例，如果一个fragment的 alpha是0.25，那么那么就以某种pattern丢掉6个sample, 只向framebuffer的两个sample上写入shading结果。最后利用multi-sample framebuffer 的resample过程来获得半透明的近似。

参考：https://www.zhihu.com/question/25822656/answer/31540272

21、BumpMapping，是一系列表面渲染技术的总称。包括以下。

　　1）offset map

　　stores in a texture two signed values, bu and bv, at each point. These two values correspond to the amount to vary the normal along the u and v image axes.   are used to scale two vectors that are perpendicular to the normal.

　　　　　　

　　2）heightfield

　　　　Each monochrome texture value represents a height, so in the texture, white is a high area and black a low one.

　　　　　　

　　3）normal-map

　　The normal map encodes (x, y, z) mapped to [−1, 1], e.g., for an 8-bit texture the x-axis value 0 represents −1.0 and 255 represents 1.0.  The color [128, 128, 255], a light blue,  would represent a flat surface for the color mapping shown, i.e., a normal

of [0, 0, 1].

　　

22、Parallax Mapping

　　　　原始点P，偏移点P^adj，P点高度h，视向量高度V^z，视向量水平宽度V^xy。

　　　　

　　　　this shifting works fairly well in practice if the bump heights change relatively slowly. 

　　　　When the view vector is near the surface's horizon, a small height change results in a large texture co ordinate shift. The approximation fails. 

　　　　一种改进方法是，限制偏移距离不大于h。如下：

　　　　

　　　　

　　　　One problem with relief mapping methods is that the illusion breaks down along the silhouette edges of objects, which will show the original surface’s smooth outlines.

　　　　

23、Height Field Texturing

　　

24、

参考：