磁盘异步I / O在Windows上显示为同步

概要

---

Microsoft Windows上的文件I / O可以是同步或异步的。I / O的默认行为是同步的，其中调用I / O函数并在I / O完成时返回。异步I / O允许I / O函数立即将执行返回给调用者，但是在未来的某个时间之前不会假定I / O完成。I / O完成后，操作系统会通知调用方。或者，调用者可以通过使用操作系统的服务来确定未完成的I / O操作的状态。 
异步I / O的优点是调用程序有时间完成其他工作或在I / O操作完成时发出更多请求。术语重叠I / O经常用于同步I / O的异步I / O和非重叠I / O. 本文使用术语异步和同步进行I / O操作。本文假定读者熟悉FileF / File函数，如CreateFile，ReadFile，WriteFile。 
通常，异步I / O操作的行为与同步I / O一样。本文在后面的部分中讨论的某些条件使I / O操作同步完成。调用者没有时间进行后台工作，因为I / O函数在I / O完成之前不会返回。
有几个函数与同步和异步I / O有关。本文使用ReadFile和WriteFile作为示例; 好的替代品是ReadFileEx和WriteFileEx。虽然本文仅讨论了磁盘I / O，但许多原则可以应用于其他类型的I / O，例如串行I / O或网络I / O.

更多信息

设置异步I / O.

打开文件时，必须在CreateFile中指定FILE_FLAG_OVERLAPPED标志。此标志允许异步执行文件上的I / O操作。这是一个例子：

   HANDLE hFile;
   hFile = CreateFile(szFileName,
                      GENERIC_READ,
                      0,
                      NULL,
                      OPEN_EXISTING,
                      FILE_FLAG_NORMAL | FILE_FLAG_OVERLAPPED,
                      NULL);

   if (hFile == INVALID_HANDLE_VALUE)
      ErrorOpeningFile();

在编写异步I / O时要小心，因为系统保留在需要时使操作同步的权利。因此，最好是编写程序以正确处理可以同步或异步完成的I / O操作。示例代码演示了这一考虑因素。
程序在等待异步操作完成时可以执行许多操作，例如排队其他操作或执行后台工作。例如，以下代码正确处理读取操作的重叠和非重叠完成。它只是等待未完成的I / O完成：

   if (!ReadFile(hFile,
                 pDataBuf,
                 dwSizeOfBuffer,
                 &NumberOfBytesRead,
                 &osReadOperation )
   {
      if (GetLastError() != ERROR_IO_PENDING)
      {
         // Some other error occurred while reading the file.
         ErrorReadingFile();
         ExitProcess(0);
      }
      else
         // Operation has been queued and
         // will complete in the future.
         fOverlapped = TRUE;
   }
   else
      // Operation has completed immediately.
      fOverlapped = FALSE;

   if (fOverlapped)
   {
      // Wait for the operation to complete before continuing.
      // You could do some background work if you wanted to.
      if (GetOverlappedResult( hFile,
                               &osReadOperation,
                               &NumberOfBytesTransferred,
                               TRUE))
         ReadHasCompleted(NumberOfBytesTransferred);
      else
         // Operation has completed, but it failed.
         ErrorReadingFile();
   }
   else
      ReadHasCompleted(NumberOfBytesRead);

请注意，传入ReadFile的＆NumberOfBytesRead与传递给GetOverlappedResult的＆NumberOfBytesTransferred不同。如果操作已异步，则使用GetOverlappedResult确定操作完成后在操作中传输的实际字节数。传递给ReadFile的＆NumberOfBytesRead没有意义。
另一方面，如果操作立即完成，则传入ReadFile的NumberOfBytesRead对读取的字节数有效。在这种情况下，忽略传递给ReadFile的OVERLAPPED结构; 不要将它与GetOverlappedResult或WaitForSingleObject一起使用。
异步操作的另一个警告是，在其挂起操作完成之前，不得使用OVERLAPPED结构。换句话说，如果您有三个未完成的I / O操作，则必须使用三个OVERLAPPED结构。如果重用OVERLAPPED结构，您将在I / O操作中收到不可预测的结果，并且可能会遇到数据损坏。此外，在您第一次使用OVERLAPPED结构之前，或者在先前操作完成之后重新使用OVERLAPPED结构之前，必须正确初始化它，以便没有遗留数据影响新操作。
相同类型的限制适用于操作中使用的数据缓冲区。在相应的I / O操作完成之前，不得读取或写入数据缓冲区; 读取或写入缓冲区可能会导致错误和数据损坏。

异步I / O仍然似乎是同步的

但是，如果按照本文前面的说明进行操作，则所有I / O操作通常仍然按照发出的顺序同步完成，并且没有任何ReadFile操作返回FALSE，GetLastError（）返回ERROR_IO_PENDING，这意味着您没有时间进行任何操作背景工作。为什么会这样？
即使您已编码异步操作，I / O操作同步完成的原因有很多：

压缩

异步操作的一个障碍是NTFS压缩。文件系统驱动程序不会异步访问压缩文件; 相反，所有操作都是同步的。这不适用于使用类似于COMPRESS或PKZIP的实用程序压缩的文件。

NTFS加密

与压缩类似，文件加密会导致系统驱动程序将异步I / O转换为同步。如果文件被解密，则I / O请求将是异步的。

扩展文件

I / O操作同步完成的另一个原因是操作本身。在Windows NT上，对扩展其长度的文件的任何写入操作都将是同步的。
注意：应用程序可以通过使用SetFileValidData函数更改文件的有效数据长度，然后发出WriteFile，使前面提到的写操作异步。
使用SetFileValidData（可在Windows XP及更高版本中使用），应用程序可以有效地扩展文件，而不会因为零填充而导致性能下降。
由于NTFS文件系统不会将数据填充到由SetFileValidData定义的有效数据长度（VDL），因此该函数具有安全隐患，其中可以为文件分配先前由其他文件占用的集群。因此，SetFileValidData要求调用者启用新的SeManageVolumePrivilege（默认情况下，此操作仅分配给管理员）。Microsoft建议ISV仔细考虑使用此功能的含义。

高速缓存

大多数I / O驱动程序（磁盘，通信和其他）都有特殊的大小写代码，如果可以“立即”完成I / O请求，则操作将完成，ReadFile或WriteFile函数将返回TRUE。在所有方面，这些类型的操作似乎是同步的。对于磁盘设备，通常，当数据缓存在内存中时，可以“立即”完成I / O请求。

数据不在缓存中

但是，如果数据不在缓存中，缓存方案可能会对您不利。Windows NT缓存使用文件映射在内部实现。Windows NT中的内存管理器不提供异步页面错误机制来管理缓存管理器使用的文件映射。但是，缓存管理器可以验证所请求的页面是否在内存中，因此如果您发出异步缓存读取，并且页面不在内存中，则文件系统驱动程序会假定您不希望您的线程被阻止并且请求将由有限的工作线程池处理。在ReadFile调用之后，控制将返回到您的程序，并且读取仍处于待定状态。
这适用于少量请求，但由于工作线程池有限（目前在16MB系统上有三个），因此在特定时间仍然只有少数请求排队到磁盘驱动器。如果对不在缓存中的数据发出大量I / O操作，则缓存管理器和内存管理器将变为饱和状态，并使您的请求同步。
还可以根据您是按顺序还是随机访问文件来影响缓存管理器的行为。在按顺序访问文件时，最常见的是缓存的好处。CreateFile调用中的FILE_FLAG_SEQUENTIAL_SCAN标志将优化此类访问的缓存。但是，如果以随机方式访问文件，请使用CreateFile中的FILE_FLAG_RANDOM_ACCESS标志指示缓存管理器优化其随机访问的行为。

不要使用缓存

FILE_FLAG_NO_BUFFERING标志对异步操作的文件系统行为影响最大。这是保证I / O请求实际上是异步的最佳方法。它指示文件系统根本不使用任何缓存机制。
警告：使用此标志有一些限制，这与数据缓冲区对齐和设备的扇区大小有关。有关正确使用此标志的详细信息，请参阅CreateFile函数文档中的函数参考。

真实世界的测试结果

以下是示例代码的一些测试结果。数字的大小在这里并不重要，并且因计算机而异，但数字之间的相互关系说明了标志对性能的一般影响。
您可以期望看到与以下内容类似的结果：

• 测试1

  异步，无缓冲的I / O：asynchio /f*.dat / n 
     操作已按请求顺序完​​成。
     500个请求在0.224264秒内排队。
     500个请求在4.982481秒内完成。

  此测试表明前面提到的程序很快发出了500个I / O请求，并且有很多时间做其他工作或发出更多请求。

• 测试2

  同步，无缓冲I / O：asynchio /f*.dat / s / n 
     按发出的顺序完成操作。
     500个请求在4.495806秒内排队并完成。

  此测试表明该程序花费4.495880秒调用ReadFile来完成其操作，而测试1仅花费0.224264秒来发出相同的请求。在测试2中，程序没有“额外”时间来完成任何后台工作。

• 测试3

  异步缓冲I / O：asynchio /f*.dat 
     按发出的顺序完成操作。
     在0.251670秒内发出并完成了500个请求。

  此测试演示了缓存的同步特性。所有读数均在0.251670秒内发布并完成。换句话说，异步请求是同步完成的。此测试还演示了数据在缓存中时缓存管理器的高性能。

• 测试4

  同步缓冲I / O：asynchio /f*.dat / s 
     按发出的顺序完成操作。
     500个请求，在0.217011秒内完成。

  此测试演示与测试3中相同的结果。请注意，从缓存执行的同步读取比从缓存中进行的异步读取要快一些。此测试还演示了数据在缓存中时缓存管理器的高性能。

结论

您可以决定哪种方法最好，因为它取决于程序执行的操作的类型，大小和数量。
没有为CreateFile指定任何特殊标志的默认文件访问是同步和缓存操作。
注意：在此模式下，您确实会获得一些自动异步行为，因为文件系统驱动程序会对已修改数据执行预测性异步预读和异步延迟写入。虽然这不会使应用程序的I / O异步，但它是绝大多数简单应用程序的理想情况。 
另一方面，如果您的应用程序不简单，您可能必须进行一些性能分析和性能监视以确定最佳方法，类似于本文前面所示的测试。分析在ReadFile或WriteFile函数中花费的时间，然后将此时间与实际I / O操作完成所需的时间进行比较非常有用。如果大部分时间花在实际发出I / O上，那么您的I / O将同步完成。但是，如果发出I / O请求所花费的时间与I / O操作完成所花费的时间相比相对较小，那么您的操作将被异步处理。本文前面提到的示例代码使用QueryPerformanceCounter函数进行自己的内部分析。
性能监视可以帮助确定程序使用磁盘和缓存的效率。跟踪Cache对象的任何性能计数器将指示缓存管理器的性能。跟踪物理磁盘或逻辑磁盘对象的性能计数器将指示磁盘系统的性能。
有几个实用程序有助于性能监视; PerfMon和DiskPerf特别有用。要使系统收集有关磁盘系统性能的数据，必须先发出diskperf -y命令。发出命令后，必须重新启动系统才能启动数据收集。

Summary

---

File I/O on Microsoft Windows can be synchronous or asynchronous. The default behavior for I/O is synchronous, where an I/O function is called and returns when the I/O is complete. Asynchronous I/O allows an I/O function to return execution back to the caller immediately, but the I/O is not assumed to be complete until some future time. The operating system notifies the caller when the I/O is complete. Alternatively, the caller can determine the status of the outstanding I/O operation by using services of the operating system.

The advantage of asynchronous I/O is that the caller has time to do other work or issue more requests while the I/O operation is being completed. The term Overlapped I/O is frequently used for Asynchronous I/O and Non-overlapped I/O for Synchronous I/O. This article uses the terms Asynchronous and Synchronous for I/O operations. This article assumes the reader has familiarity with the File I/O functions such as CreateFile, ReadFile, WriteFile.

Frequently, asynchronous I/O operations behave just as synchronous I/O. Certain conditions that this article discusses in the later sections make the I/O operations complete synchronously. The caller has no time for background work because the I/O functions do not return until the I/O is complete.

Several functions are related to synchronous and asynchronous I/O. This article uses ReadFile and WriteFile as examples; good alternatives would be ReadFileEx and WriteFileEx. Although this article discusses only disk I/O specifically, many of the principles can be applied to other types of I/O, such as serial I/O or network I/O.

More Information

---

Set Up Asynchronous I/O

The FILE_FLAG_OVERLAPPED flag must be specified in CreateFile when the file is opened. This flag allows I/O operations on the file to be performed asynchronously. Here is an example:

   HANDLE hFile;

   hFile = CreateFile(szFileName,
                      GENERIC_READ,
                      0,
                      NULL,
                      OPEN_EXISTING,
                      FILE_FLAG_NORMAL | FILE_FLAG_OVERLAPPED,
                      NULL);

   if (hFile == INVALID_HANDLE_VALUE)
      ErrorOpeningFile();

Be careful when coding for asynchronous I/O because the system reserves the right to make an operation synchronous if it needs to. Therefore, it is best if you write the program to correctly handle an I/O operation that may be completed either synchronously or asynchronously. The sample code demonstrates this consideration.

There are many things a program can do while waiting for asynchronous operations to complete, such as queuing additional operations, or doing background work. For example, the following code properly handles overlapped and non-overlapped completion of a read operation. It does nothing more than wait for the outstanding I/O to complete:

   if (!ReadFile(hFile,
                 pDataBuf,
                 dwSizeOfBuffer,
                 &NumberOfBytesRead,
                 &osReadOperation )
   {
      if (GetLastError() != ERROR_IO_PENDING)
      {
         // Some other error occurred while reading the file.
         ErrorReadingFile();
         ExitProcess(0);
      }
      else
         // Operation has been queued and
         // will complete in the future.
         fOverlapped = TRUE;
   }
   else
      // Operation has completed immediately.
      fOverlapped = FALSE;

   if (fOverlapped)
   {
      // Wait for the operation to complete before continuing.
      // You could do some background work if you wanted to.
      if (GetOverlappedResult( hFile,
                               &osReadOperation,
                               &NumberOfBytesTransferred,
                               TRUE))
         ReadHasCompleted(NumberOfBytesTransferred);
      else
         // Operation has completed, but it failed.
         ErrorReadingFile();
   }
   else
      ReadHasCompleted(NumberOfBytesRead);

Note that &NumberOfBytesRead passed into ReadFile is different from &NumberOfBytesTransferred passed into GetOverlappedResult. If an operation has been made asynchronous, then GetOverlappedResult is used to determine the actual number of bytes transferred in the operation after it has completed. The &NumberOfBytesRead passed into ReadFile is meaningless.

If, on the other hand, an operation is completed immediately, then &NumberOfBytesRead passed into ReadFile is valid for the number of bytes read. In this case, ignore the OVERLAPPED structure passed into ReadFile; do not use it with GetOverlappedResult or WaitForSingleObject.

Another caveat with asynchronous operation is that you must not use an OVERLAPPED structure until its pending operation has completed. In other words, if you have three outstanding I/O operations, you must use three OVERLAPPED structures. If you reuse an OVERLAPPED structure, you will receive unpredictable results in the I/O operations and you may experience data corruption. Additionally, before you can use an OVERLAPPED structure for the first time, or before you reuse it after an earlier operation has completed, you must correctly initialize it so no left-over data affects the new operation.

The same type of restriction applies to the data buffer used in an operation. A data buffer must not be read or written until its corresponding I/O operation has completed; reading or writing the buffer may cause errors and corrupted data.

Asynchronous I/O Still Appears to be Synchronous

If you followed the instructions earlier in this article, however, all your I/O operations still typically complete synchronously in the order issued, and none of the ReadFile operations returns FALSE with GetLastError() returning ERROR_IO_PENDING, this means you have no time for any background work. Why does this occur?

There are a number of reasons why I/O operations complete synchronously even if you have coded for asynchronous operation:

Compression

One obstruction to asynchronous operation is NTFS compression. The file system driver will not access compressed files asynchronously; instead all operations are just made synchronous. This does not apply to files that are compressed with utilities similar to COMPRESS or PKZIP.

NTFS Encryption

Similar to Compression, file encryption causes the system driver to convert asynchronous I/O to synchronous. If the files are decrypted, the I/O requests will be asynchronous.

Extending a File

Another reason that I/O operations are completed synchronously is the operations themselves. On Windows NT, any write operation to a file that extends its length will be synchronous.

NOTE: Applications can make the previously mentioned write operation asynchronous by changing the Valid Data Length of the file by using the SetFileValidData function, and then issuing a WriteFile.

Using SetFileValidData (which is available on Windows XP and later versions), applications can efficiently extend files without incurring a performance penalty for zero-filling them.

Because the NTFS file system does not zero-fill the data up to the valid data length (VDL) that is defined by SetFileValidData, this function has security implications where the file may be assigned clusters that were previously occupied by other files. Therefore, SetFileValidData requires that the caller have the new SeManageVolumePrivilege enabled (by default, this is assigned only to administrators). Microsoft recommends that ISVs carefully consider the implications of using this function.

Cache

Most I/O drivers (disk, communications, and others) have special case code where, if an I/O request can be completed "immediately," the operation will be completed and the ReadFile or WriteFile function will return TRUE. In all ways, these types of operations appear to be synchronous. For a disk device, typically, an I/O request can be completed "immediately" when the data is cached in memory.

Data Is not in Cache

The cache scheme can work against you, however, if the data is not in the cache. The Windows NT cache is implemented internally using file mappings. The memory manager in Windows NT does not provide an asynchronous page fault mechanism to manage the file mappings used by the cache manager. The cache manager can, however, verify whether the requested page is in memory, so if you issue an asynchronous cached read, and the pages are not in memory, the file system driver assumes that you do not want your thread blocked and the request will be handled by a limited pool of worker threads. Control is returned to your program after your ReadFile call with the read still pending.

This works fine for a small number of requests, but because the pool of worker threads is limited (currently three on a 16MB system), there will still be only a few requests queued to the disk driver at a particular time. If you issue a lot of I/O operations for data that is not in the cache, the cache manager and memory manager become saturated and your requests are made synchronous.

The behavior of the cache manager can also be influenced based on whether you access a file sequentially or randomly. Benefits of the cache are seen most when accessing files sequentially. The FILE_FLAG_SEQUENTIAL_SCAN flag in the CreateFile call will optimize the cache for this type of access. However, if you access files in a random fashion, use the FILE_FLAG_RANDOM_ACCESS flag in CreateFile to instruct the cache manager to optimize its behavior for random access.

Do not Use the Cache

The FILE_FLAG_NO_BUFFERING flag has the most effect on the behavior of the file system for asynchronous operation. This is the best way to guarantee that I/O requests are actually asynchronous. It instructs the file system to not use any cache mechanism at all.

WARNING: There are some restrictions to using this flag that have to do with the data buffer alignment and the device's sector size. See the function reference in the documentation for the CreateFile function for more information about using this flag properly.

Real World Test Results

Following are some test results from the sample code. The magnitude of the numbers is not important here and varies from computer to computer, but the relationship of the numbers compared to each other illuminates the general affect of the flags on performance.

You can expect to see results similar to the following:

• Test 1

  Asynchronous, unbuffered I/O:  asynchio /f*.dat /n
  
     Operations completed out of the order in which they were requested.
     500 requests queued in 0.224264 seconds.
     500 requests completed in 4.982481 seconds. 

  This test demonstrates that the previously mentioned program issued 500 I/O requests quickly and had a lot of time to do other work or issue more requests.

• Test 2

  Synchronous, unbuffered I/O: asynchio /f*.dat /s /n
  
     Operations completed in the order issued.
     500 requests queued and completed in 4.495806 seconds. 

  This test demonstrates that this program spent 4.495880 seconds calling ReadFile to complete its operations, whereas the test 1 spent only 0.224264 seconds to issue the same requests. In test 2, there was no "extra" time for the program to do any background work.

• Test 3

  Asynchronous, buffered I/O: asynchio /f*.dat
  
     Operations completed in the order issued.
     500 requests issued and completed in 0.251670 seconds. 

  This test demonstrates the synchronous nature of the cache. All reads were issued and completed in 0.251670 seconds. In other words, asynchronous requests were completed synchronously. This test also demonstrates the high performance of the cache manager when data is in the cache.

• Test 4

  Synchronous, buffered I/O: asynchio /f*.dat /s
  
     Operations completed in the order issued.
     500 requests and completed in 0.217011 seconds. 

  This test demonstrates the same results as in test 3. Note that synchronous reads from the cache complete a little faster than asynchronous reads from the cache. This test also demonstrates the high performance of the cache manager when data is in the cache.

CONCLUSION

You can decide which method is best because it all depends on the type, size, and number of operations that your program performs.

The default file access without specifying any special flags to CreateFile is a synchronous and cached operation.

NOTE: You do get some automatic asynchronous behavior in this mode because the file system driver does predictive asynchronous read-ahead and asynchronous lazy writing of modified data. Although this does not make the application's I/O asynchronous, it is the ideal case for the vast majority of simple applications.

If, on the other hand, your application is not simple, you may have to do some profiling and performance monitoring to determine the best method, similar to the tests illustrated earlier in this article. Profiling the time spent in the ReadFile or WriteFile function and then comparing this time to how long it takes for actual I/O operations to complete is extremely useful. If the majority of the time is spent in actually issuing the I/O, then your I/O is being completed synchronously. However, if the time spent issuing I/O requests is relatively small compared to the time it takes for the I/O operations to complete, then your operations are being treated asynchronously. The sample code mentioned earlier in this article uses the QueryPerformanceCounter function to do its own internal profiling.

Performance monitoring can help determine how efficiently your program is using the disk and the cache. Tracking any of the performance counters for the Cache object will indicate the performance of the cache manager. Tracking the performance counters for the Physical Disk or Logical Disk objects will indicate the performance of the disk systems.

There are several utilities that are helpful in performance monitoring; PerfMon and DiskPerf are especially useful. For the system to collect data on the performance of the disk systems, you must first issue the diskperf -y command. After you issue the command, you must restart the system to start the data collection.