I was recently reading a series on “Write Sequential Non-Blocking IO Code With Fibers in NodeJS” by Venkatesh.

Venki was essentially trying to emphasize that writing non-blocking code in NodeJS (either via callbacks, or using promises) can get hairy really fast. For example, this code demonstrates that aptly:

var express = require('express');
var app = express(); app.get('/users/:fbId', function(req, res) {
var id = req.params.id;
var key = 'user:' + id;
client.get(key, function(err, reply) {
if (err !== null) {
res.send(500);
return;
} if (reply === null) {
res.send(404);
return;
} res.send(200, {id: id, name: reply});
});
});

The exact code is available on GitHub (so is the promises driven version, but I won’t bother inlining it.)

What we actually wanted to write (if it were possible, was):

var express = require('express');
var app = express(); app.get('/users/:fbId', function(req, res) {
var id = req.params.id;
var key = 'user:' + id; try {
var reply = client.get(key);
if (reply === null) {
res.send(404);
return;
} res.send(200, {id: id, name: reply});
}
catch(err) {
res.send(500);
}
});

The magic would happen in line number 9 (above.) Instead of having to provide a cascade of callbacks (what if we wanted to do another lookup after we got the value back from the first), we could just write them serially, one after the other.

Well. Apparently we can!

Fibers

A fiber is a particularly lightweight thread of execution. Like threads, fibers share address space. However, fibers use co-operative multitasking while threads use pre-emptive multitasking. Threads often depend on the kernel’s thread scheduler to preempt a busy thread and resume another thread; fibers yield themselves to run another fiber while executing.

Fibers allow exactly this kind of black magic in NodeJS. It is still callbacks internally, but we are exposed to none of it in our application code. Sure you will end up writing a bunch of wrappers (or have some tool generate them for us), but we would have the sweet sweet pleasure of writing async IO code without having to jump through all the hoops. This is how the wrapper code for redis client looks like:

var Fiber = require('fibers');
var client = require('./redis-client'); exports.get = function(key) {
var err, reply;
var fiber = Fiber.current; client.get(key, function(_err, _reply) {
err = _err;
reply = _reply;
fiber.run();
}); Fiber.yield(); if (err != null) {
throw err;
} return reply;
};

(the real code is here in case you are curious)

I liked how the code looked. Having survided a ‘promising’ node.js project, I was definitely curious about this new style. Maybe this can be the saving grace (before generators and yield take over the JS world) for real world server side JavaScript.

Fibers you say

But the code (and the underlying technique which makes it tick) sounded very familiar, and reminded me of a similar technique which is used in Go to allow writing beautiful async IO code. For example, the same function from above in Go:

m.Get("/users/:id", func(db *DB, params martini.Params) (int, []byte) {
str := params["id"]
id, err := strconv.Atoi(str)
if err != nil {
return http.StatusBadRequest, []byte{}
} u, err := db.LoadUser(id)
if err != nil {
return http.StatusNotFound, []byte{}
}
return http.StatusOK, encoder.Must(enc.Encode(u))
})

Sure, there is a little more happening in here (Go is statically typed), buts its the exact same thing as the fibers example, without all the manual wrapping. Any call which does IO (like line 8) blocks the currently executing goroutine (just like a fiber, a lightweight thread.) The natural question to ask is, if the goroutine gets blocked, how do other requests get processed? Its quite simple actually. The Go runtime automatically schedules any other goroutine which is ready to run (their IO call is done) on the thread on which the current goroutine was running.

Since goroutines are light weight (stack size is just 4 KB in Go 1.3beta1 compared to the much larger ~2 MB thread stacks), it is not unusual to have hundreds of thousands of goroutines actively running in a single process, all humming along together. The best part, since the threads have to do less context switching (the same physical thread can continue running on the processor core, just the instruction pointer keeps changing as the goroutines shuffle in and out, just as in method calls), we are able to extract a lot more efficiency from the same unit of hardware than otherwise. Otherwise IO calls, which would otherwise cause the thread to block and wait, could cripple the system and bring it down to its knees. Read this article for more context on this.

Performance

A fellow ThoughtWorker asked me, “Does performance matter when choosing a framework?”

I know where he was coming from, and how we shouldn’t make decisions purely based on performance (we would all be doing assembly if that was the case.) While it is true that as a startup (or even in the case of a well established player), building the MVP and getting it to the users is paramount, you really dont want to face the situation where you suddenly have a huge influx of users (say it goes viral) and you are caught between a ROCK (scale horizontally by throwing compute units at the problem) and a HARD PLACE (have to rewrite the solution in a technology more amenable to scaling.) Both of these options are expensive, and can potentially be a deal breaker.

Therefore, provided everything else is more or less equal, choosing the more performant one is never a bad thing.

With this context, I decided to compare the two solutions for their performance, given that they more or less looked the same. I decided to allow the system under test to use as many cores as they wanted, and then hit them with 100 concurrent users, each of which is going full tilk for around 20 seconds (used the awesome wrktool for benchmarking.)

The results:

Golang  
Stdlib 134566 (3.81ms)
Gorilla 125092 (4.28ms)
Martini 51330 (9.51ms)
node.js  
Stdlib 54510 (7.78ms)
Callbacks* 36107 (10.84ms)
Fibers* 27372 (18.76ms)
Promises* 22665 (17.15ms)

* The Callbacks, Fibers and Promises versions are created using Express. The Stdlib versions use the http support in the corresponding standard libraries.

All the numbers are in req/s as given by wrk (higher is better.) The latency details are in brackets (lower is better.) Clicking the numbers will take you to the corresponding code in the GitHub repo (the README has the detailed numbers.)

The tests were done on an updated Ubuntu 14.04 box with a Intel i7 4770 processor, 16 GB of RAM and a SSD.

As you can see, the fibers method of doing async IO in node.js comes with a perceivable loss in throughput compared to the pure callbacks based approach, but looks relatively better than the promises version for this micro-benchmark.

At the same time, the default way of doing IO in Golang does very well for itself. More than 134,000 req/s with a 3.81 ms 99th percentile latency. All this without having to go through crazy callbacks/promises hoops. How cool is that?

How the tests were run?

Software versions

  • Go 1.3beta1
  • node.js 0.10.28
  • wrk 3.1.0

Command used to run

A more detailed description is available in the README but I will explain a simple version here:

  • Start the program (by say running ./start_martini.sh)
  • Run the benchmark (by running ./bench.sh)
  • Record the result
  • Rince and repeat 3 times and take the best run

Notes

  • All cores on the Intel i7 4770 were set to the performance governor
  • Redis was not tweaked
  • ulimit was not raised

Summary

This is part 1 in a multipart series looking at how async IO (and programming in general) is done in various languages/platforms. We will be going indepth into one language/platform with the every new article in the series. Future parts will look at Scala, Clojure, Java, C#, Python and Ruby based frameworks and try and present a holistic view of the async world.

But one thing is very clear, async IO is here to stay. Not embrassing it would be foolhardy given the need to stay lean. Hope these articles help you understand gravity of the decision.

While some might argue that what we did in Golang was not really async, as the call was blocking in nature. But the net result achieved, and the reason why Go is still able to provide an awesome throughput despite blocking IO calls, is because the Go runtime essentially does the heavy lifting for you. When one goroutine is busy waiting for the results of a IO call to come back, other goroutines can take their place and not waste CPU cycles. The fact that this mechanism allows us to get away with fewer threads that would be required otherwise, is the icing on top.

Async IO的更多相关文章

  1. 一款DMA性能优化记录:异步传输和指定实时信号做async IO

    关键词:DMA.sync.async.SIGIO.F_SETSIG. DMA本身用于减轻CPU负担,进行CPU off-load搬运工作. 在DMA驱动内部实现有同步和异步模式,异步模式使用dma_a ...

  2. [Functional Programming] Async IO Functor

    We will see a peculiar example of a pure function. This function contained a side-effect, but we dub ...

  3. 关于Blocking IO,non-Blokcing IO,async IO的区别和理解

    来源:http://shmilyaw-hotmail-com.iteye.com/blog/1896683 概括来说,一个IO操作可以分为两个部分:发出请求.结果完成.如果从发出请求到结果返回,一直B ...

  4. ORACLE数据库异步IO介绍

    异步IO概念 Linux 异步 I/O (AIO)是 Linux 内核中提供的一个增强的功能.它是Linux 2.6 版本内核的一个标准特性,当然我们在2.4 版本内核的补丁中也可以找到它.AIO 背 ...

  5. 为什么我们要使用Async、Await关键字

    前不久,在工作中由于默认(xihuan)使用Async.Await关键字受到了很多质问,所以由此引发这篇博文“为什么我们要用Async/Await关键字”,请听下面分解: Async/Await关键字 ...

  6. [翻译]各个类型的IO - 阻塞, 非阻塞,多路复用和异步

    同事推荐,感觉写的不错就试着翻译了下. 原文链接: https://www.rubberducking.com/2018/05/the-various-kinds-of-io-blocking-non ...

  7. 《Linux/UNIX系统编程手册》第63章 IO多路复用、信号驱动IO以及epoll

    关键词:fasync_helper.kill_async.sigsuspend.sigaction.fcntl.F_SETOWN_EX.F_SETSIG.select().poll().poll_wa ...

  8. linux io的cfq代码理解

    内核版本: 3.10内核. CFQ,即Completely Fair Queueing绝对公平调度器,原理是基于时间片的角度去保证公平,其实如果一台设备既有单队列,又有多队列,既有快速的NVME,又有 ...

  9. Haskell语言学习笔记(85)Async

    安装 async $ cabal install async async-2.2.1 installed async / wait / concurrently async :: IO a -> ...

随机推荐

  1. zsh 命令提示符 PROMPT

    使用上zsh后,发现命令提示符显示不了当前的路径,和一般的Linux系统默认提示不一致.配置自己的提示符: 更改配置文件.zshrc,添加配置PROMPT='%m:%. $',重新打开一个窗口生效. ...

  2. user agent stylesheet (浏览器默认样式)!

    "-webkit-margin-before". "-webkit-margin-after". "-webkit-margin-start" ...

  3. log4net配置

    1.configuration配置 <configSections> <section name="log4net" type="log4net.Con ...

  4. Blender 脚本之 Operator 初探

    addon(插件)用来扩展 Blender 的功能,跟其他软件里的 plugin(插件)一样,去掉不会影响软件的运行.插件可以加到 Blender 的用户偏好设置目录里,或者就在你所编辑的.blend ...

  5. Python 学习笔记(6)--常用模块(2)

    一.下载安装 下载安装有两种方式: yum\pip\apt-get 或者源码 下载源码 解压源码 进入目录 编译源码 python setup.py build 安装源码 python setup.p ...

  6. SQL Server客户端请求

    SQL Server是客户端 - 服务器平台.通过发送包含对数据库请求的命令是与后端数据库进行交互的唯一方法.你的应用程序和数据库之间通信的协议被称为TDS(表格数据流协议). 应用程序可以使用该协议 ...

  7. 公众号第三方平台开发 获取 component_verify_ticket 2015-07-05 10:16 59人阅读 评论(0) 收藏

    8.推送component_verify_ticket协议 在公众号第三方平台创建审核通过后,微信服务器会向其"授权事件接收URL"每隔10分钟定时推送component_veri ...

  8. 本人为巨杉数据库(开源NoSQL)写的C#驱动,支持Linq,全部开源,已提交github

    一.关于NoSQL的项目需求 这些年在做AgileEAS.NET SOA 中间件平台的推广.技术咨询服务过程之中,特别是针对我们最熟悉的医疗行业应用之中,针对大数据分析,大并发性能的需求,我们也在慢慢 ...

  9. 【leetcode】Valid Sudoku

    题目简述: Determine if a Sudoku is valid, according to: Sudoku Puzzles - The Rules. The Sudoku board cou ...

  10. WCF 学习篇

    写在前面 自从运用了.NET Remoting 之后,就想系统的学习下WCF,因为WCF是对现有分布式通信技术的整合.主要以 <WCF全面解析> 这本书为主,园子的资料和网上资料为辅,来学 ...