There are use cases where data need to be read from source to a sink without modification. In code this might look quite simple: for example in Java, you may read data from one InputStream chunk by chunk into a small buffer (typically 8KB), and feed them into the OutputStream, or even better, you could create a PipedInputStream, which is basically just a util that maintains that buffer for you. However, if low latency is crucial to your software, this might be quite expensive from the OS perspective and I shall explain.

What happens under the hood

Well, here’s what happens when the above code is used:

  1. JVM sends read() syscall.
  2. OS context switches to kernel mode and reads data into the input socket buffer.
  3. OS kernel then copies data into user buffer, and context switches back to user mode. read() returns.
  4. JVM processes code logic and sends write() syscall.
  5. OS context switches to kernel mode and copies data from user buffer to output socket buffer.
  6. OS returns to user mode and logic in JVM continues.

This would be fine if latency and throughput aren’t your service’s concern or bottleneck, but it would be annoying if you do care, say for a static asset server. There are 4 context switches and 2 unnecessary copies for the above example.

OS-level zero copy for the rescue

Clearly in this use case, the copy from/to user space memory is totally unnecessary because we didn’t do anything other than dumping data to a different socket. Zero copy can thus be used here to save the 2 extra copies. The actual implementation doesn’t really have a standard and is up to the OS how to achieve that. Typically *nix systems will offer sendfile(). Its man page can be found here. Some say some operating systems have broken versions of that with one of them being OSX link. Honestly with such low-level feature, I wouldn’t trust Apple’s BSD-like system so never tested there.

With that, the diagram would be like this:

You may say OS still has to make a copy of the data in kernel memory space. Yes but from OS’s perspective this is already zero-copy because there’s no data copied from kernel space to user space. The reason why kernel needs to make a copy is because general hardware DMA access expects consecutive memory space (and hence the buffer). However this is avoidable if the hardware supports scatter-n-gather:

A lot of web servers do support zero-copy such as Tomcat and Apache. For example apache’s related doc can be found here but by default it’s off.

Note: Java’s NIO offers this through transferTo (doc).

mmap

The problem with the above zero-copy approach is that because there’s no user mode actually involved, code cannot do anything other than piping the stream. However, there’s a more expensive yet more useful approach - mmap, short for memory-map.

Mmap allows code to map file to kernel memory and access that directly as if it were in the application user space, thus avoiding the unnecessary copy. As a tradeoff, that will still involve 4 context switches. But since OS maps certain chunk of file into memory, you get all benefits from OS virtual memory management - hot content can be intelligently cached efficiently, and all data are page-aligned thus no buffer copying is needed to write stuff back.

However, nothing comes for free - while mmap does avoid that extra copy, it doesn’t guarantee the code will always be faster - depending on the OS implementation, there may be quite a bit of setup and teardown overhead (since it needs to find the space and maintain it in the TLB and make sure to flush it after unmapping) and page fault gets much more expensive since kernel now needs to read from hardware (like disk) to update the memory space and TLB. Hence, if performance is this critical, benchmark is always needed as abusing mmap() may yield worse performance than simply doing the copy.

The corresponding class in Java is MappedByteBuffer from NIO package. It’s actually a variation of DirectByteBuffer though there’s no direct relationship between classes. The actual usage is out of scope of this post.

NIO DirectByteBuffer

Java NIO introduces ByteBuffer which represents the buffer area used for channels. There are 3 main implementations of ByteBuffer:

  1. HeapByteBuffer

    This is used when ByteBuffer.allocate() is called. It’s called heap because it’s maintained in JVM’s heap space and hence you get all benefits like GC support and caching optimization. However, it’s not page aligned, which means if you need to talk to native code through JNI, JVM would have to make a copy to the aligned buffer space.

  2. DirectByteBuffer

    Used when ByteBuffer.allocateDirect() is called. JVM will allocate memory space outside the heap space using malloc(). Because it’s not managed by JVM, your memory space is page-aligned and not subject to GC, which makes it perfect candidate for working with native code (e.g. when writing OpenGL stuff). However, you are then “deteriorated” to C programmer as you’ll have to allocate and deallocate memory yourself to prevent memory leak.

  3. MappedByteBuffer

    Used when FileChannel.map() is called. Similar to DirectByteBuffer this is also outside of JVM heap. It essentially functions as a wrapper around OS mmap() system call in order for code to directly manipulate mapped physical memory data.

Conclusion

sendfile() and mmap() offer efficient, low-latency low-level solutions to data manipulation across sockets. Again, no code should assume these are silver bullets as real world scenarios may be complex and it might not be worth the effort to switch code to them if this is not the true bottleneck. For software engineering to get the most ROI, in most cases, it’s better to “make it right” and then “make it fast”. Without the guardrails offered by JVM, it’s easy to make software much more vulnerable to crashing (I literally mean crashing, not exceptions) when it comes to complicated logic.

https://xunnanxu.github.io/2016/09/10/It-s-all-about-buffers-zero-copy-mmap-and-Java-NIO/

IO的详细解释:It's all about buffers: zero-copy, mmap and Java NIO的更多相关文章

  1. Java NIO和IO的区别(转)

    原文链接:Java NIO和IO的区别 下表总结了Java NIO和IO之间的主要差别,我会更详细地描述表中每部分的差异. 复制代码代码如下: IO                NIO面向流     ...

  2. .htaccess语法之RewriteCond与RewriteRule指令格式详细解释

    htaccess语法之RewriteCond与RewriteRule指令格式详细解释 (2012-11-09 18:09:08) 转载▼ 标签:  htaccess it 分类: 网络 上文htacc ...

  3. cookie的详细解释

    突然看到网页上中英文切换的效果,不明白怎么弄得查了查 查到了cookie 并且附有详细解释 就copy留作 以后温习 http://blog.csdn.net/xidor/article/detail ...

  4. tar命令的详细解释

    tar命令的详细解释 标签: linuxfileoutputbashinputshell 2010-05-04 12:11 235881人阅读 评论(12) 收藏 举报  分类: linux/unix ...

  5. Linux学习笔记15——GDB 命令详细解释【转】

    GDB 命令详细解释 Linux中包含有一个很有用的调试工具--gdb(GNU Debuger),它可以用来调试C和C++程序,功能不亚于Windows下的许多图形界面的调试工具. 和所有常用的调试工 ...

  6. C语言 - 结构体(struct)比特字段(:) 详细解释

    结构体(struct)比特字段(:) 详细解释 本文地址: http://blog.csdn.net/caroline_wendy/article/details/26722511 结构体(struc ...

  7. 姿势体系结构的详细解释 -- C

    我基本上总结出以下4部分: 1.问题的足迹大小. 2.字节对齐问题. 3.特别保留位0. 4.这种结构被存储在存储器中的位置. #include <stdio.h> #include &l ...

  8. Java - 面向对象(object oriented)计划 详细解释

    面向对象(object oriented)计划 详细解释 本文地址: http://blog.csdn.net/caroline_wendy/article/details/24058107 程序包括 ...

  9. 设计模式 - 迭代模式(iterator pattern) Java 迭代器(Iterator) 详细解释

    迭代模式(iterator pattern) Java 迭代器(Iterator) 详细解释 本文地址: http://blog.csdn.net/caroline_wendy 參考迭代器模式(ite ...

随机推荐

  1. Java_Properties

    Java_Properties类 Hashtable与HashMap区别 主要:Hashtable线程安全,同步,效率相对低下 HashMap线程不安全,异步,效率高 父类:Hashtable父类是D ...

  2. 【Java每日一题】20170323

    20170322问题解析请点击今日问题下方的“[Java每日一题]20170323”查看(问题解析在公众号首发,公众号ID:weknow619) package Mar2017; public cla ...

  3. JS基础(一)dom小实例

    DOM的新增示例 <script language="JavaScript"> window.onload = function(){ //createDocument ...

  4. ASPxGridView 添加勾选列--全选 和 后端获取勾的行ID

    一.HTML 代码 <table style="width: 100%;"> <tr> <td> <asp:Button ID=" ...

  5. CSS制作镂空字体

    1.效果图 2.html内容: <!doctype html><html lang="en"><head> <meta charset=& ...

  6. 2 >&1 的准确含义

    1. 2代表标准错误,2 > 表示重定向,就是把标准错误重定向到 1中,这个1如果想表示标准输出的话,就必须在前面加 & 2. 正常情况下,下面这个会有很多错误信息,但是加上2>& ...

  7. ACL技术总结

    1.ACL的全称是访问控制列表,本质上是定义一组策略,以便指导报文在交换机内部的转发行为. 2.要配置策略,首先要明确ACL应用的对象,可以是针对端口,也可以是针对特殊的一条流. 针对端口,就是指端口 ...

  8. Python 标准类库-日期类型之datetime模块

    标准类库-日期类型之datetime模块    by:授客 QQ:1033553122 可用类型 3 实践出真知 4 timedelta对象 4 class datetime.timedelta(da ...

  9. Flutter 布局(三)- FittedBox、AspectRatio、ConstrainedBox详解

    本文主要介绍Flutter布局中的FittedBox.AspectRatio.ConstrainedBox,详细介绍了其布局行为以及使用场景,并对源码进行了分析. 1. FittedBox Scale ...

  10. 接口的绑定方案和动态SQL

    1. 接口绑定方案 MyBatis中, 提供了一套接口绑定方案. 程序员可以提供一个接口, 然后提供对应接口的一个mapper.xml文件. MyBatis会自动将接口和xml文件进行绑定. 实际上就 ...