Architecture of a Highly Scalable NIO-Based Server
一。 thread-per-connection
The thread-per-connection approach uses an exclusive worker thread for each connection. Within the handling loop, a worker thread
waits for new incoming data, processes the request, returns the response data, and calls the blocking socket's read method
public class Server {
private ExecutorService executors = Executors.newFixedThreadPool(10);
private boolean isRunning = true; public static void main(String... args) throws ... {
new Server().launch(Integer.parseInt(args[0]));
} public void launch(int port) throws ... {
ServerSocket sso = new ServerSocket(port);
while (isRunning) {
Socket s = sso.accept();
executors.execute(new Worker(s));
}
} private class Worker implements Runnable {
private LineNumberReader in = null; Worker(Socket s) throws ... {
in = new LineNumberReader(new InputStreamReader(...));
out = ...
} public void run() {
while (isRunning) {
try {
// blocking read of a request (line)
String request = in.readLine(); // processing the request
String response = ... // return the response
out.write(resonse);
out.flush();
} catch (Exception e ) {
...
}
}
in.close();
...
}
}
}
Because each connection has an associated thread waiting on the server side, very good response times can be achieved. However,
higher loads require a higher number of running, concurrent threads, which limits scalability. In particular, long-living connections
like persistent HTTP connections lead to a lot of concurrent worker threads, which tend to waste their time waiting concurrent
threads can waste a great deal of stack space. Note, for example, that the default Java thread stack size for Solaris is 512 KB.
二。thread-on-event
If a readiness event occurs, an event handler will be notified to perform the appropriate processing within dedicated worker threads.
To participate in the event architecture, the connection's Channel
has to be registered on a Selector
. This will be done by calling
the register
method. Although this method is part of the SocketChannel
, the channel will be registered on the Selector
, not the
other way around.
SocketChannel channel = serverChannel.accept();
channel.configureBlocking(false); // register the connection
SelectionKey sk = channel.register(selector, SelectionKey.OP_READ);
To detect new events, the Selector
provides the capability to ask the registered channels for their readiness events. By calling the select
method, the Selector
collects the readiness events of the registered channels. This method call blocks until at least one event has been
occurred. In this case, the method returns the number of connections that have become ready for I/O operations since the last select
call.
The selected connections can be retrieved by calling the Selector's selectedKey
method. This method returns a set of SelectionKey
objects,
which holds the IO event status and the reference of the connection's Channel
.
A Selector
is held by the Dispatcher
. This is a single-threaded active class that surrounds the Selector
. The Dispatcher
is responsible to
retrieve the events and to dispatch the handling of the consumed events to the EventHandler
.
Within the dispatch loop, the Dispatcher
calls the Selector
's select
method to wait for new events. If at least one event has been occurred,
the method call returns and the associated channel for each event can be acquired by calling the selectedKeys
method.
while (isRunning) {
// blocking call, to wait for new readiness events
int eventCount = selector.select(); // get the events
Iterator it = selector.selectedKeys().iterator();
while (it.hasNext()) {
SelectionKey key = it.next();
it.remove(); // readable event?
if (key.isValid() && key.isReadable()) {
eventHandler.onReadableEvent(key.channel());
} // writable event?
if (key.isValid() && key.isWritable()) {
key.interestOps(SelectionKey.OP_READ); // reset to read only
eventHandler.onWriteableEvent(key.channel());
}
...
}
...
}
Because worker threads are not forced to waste time by waiting for new requests to open a connection, the scalability and
throughput of this approach is conceptually only limited by system resources like CPU or memory. That said, the response
times wouldn't be as good as for the thread-per-connection approach, because of the required thread switches and
synchronization. The challenge of the event-driven approach is therefore to minimize synchronizations and optimize thread
management, so that this overhead will be negligible.
三。构成
1.Acceptor
The Acceptor
is a single threaded active class. Because it is only responsible for handling the very short-running
client connection request, it is often sufficient to implement the Acceptor
using the blocking I/O model.
class Acceptor implements Runnable {
...
void init() {
ServerSocketChannel serverChannel = ServerSocketChannel.open();
serverChannel.configureBlocking(true);
serverChannel.socket().bind(new InetSocketAddress(serverPort));
} public void run() {
while (isRunning) {
try {
SocketChannel channel = serverChannel.accept(); Connection con = new Connection(channel, appHandler);
dispatcherPool.nextDispatcher().register(con);
} catch (...) {
...
}
}
}
}
2.Dispatcher
Because the scalability of a single Dispatcher
is limited, often a small pool of Dispatcher
s will be used. One reason for this limitation
is the operating-system-specific implementation of the Selector
.
Most popular operating systems map a SocketChannel
to a file handle in a one-to-one relationship. Depending on the concrete system,
the maximum number of file handles per Selector
is limited in adifferent way.
The Selector
manages the registered channels internally by using key sets. This means that by registering a channel, an associated
SelectionKey
will be created and be added to the Selector's registered key set. At the same time, the concurrent dispatcher thread
could call the Selector
's select
method, which also accesses the key set.
Because the key sets are not thread-safe, an unsynchronized registration in the context of the Acceptor
thread can lead to deadlocks
and race conditions. This can be solved by implementing the selector guard object idiom, which allows suspending the dispatcher
thread temporarily.
class Dispatcher implements Runnable {
private Object guard = new Object();
… void register(Connection con) {
// retrieve the guard lock and wake up the dispatcher thread
// to register the connection's channel
synchronized (guard) {
selector.wakeup();
con.getChannel().register(selector, SelectionKey.OP_READ, con);
} // notify the application EventHandler about the new connection
} void announceWriteNeed(Connection con) {
SelectionKey key = con.getChannel().keyFor(selector);
synchronized (guard) {
selector.wakeup();
key.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
}
} public void run() {
while (isRunning) {
synchronized (guard) {
// suspend the dispatcher thead if guard is locked
}
int eventCount = selector.select(); Iterator it = selector.selectedKeys().iterator();
while (it.hasNext()) {
SelectionKey key = it.next();
it.remove(); // read event?
if (key.isValid() && key.isReadable()) {
Connection con = (Connection) key.attachment();
disptacherEventHandler.onReadableEvent(con);
} // write event?
}
}
}
}
4.Dispatcher-Level EventHandler
5.Application-Level EventHandler
Architecture of a Highly Scalable NIO-Based Server的更多相关文章
- Java NIO: Non-blocking Server 非阻塞网络服务器
本文翻译自 Jakob Jenkov 的 Java NIO: Non-blocking Server ,原文地址:http://tutorials.jenkov.com/java-nio/non-bl ...
- Java NIO: Non-blocking Server
Even if you understand how the Java NIO non-blocking features work (Selector, Channel, Buffer etc.), ...
- NIO的一些相关链接
Architecture of a Highly Scalable NIO-Based Server Scalable IO in Java Tricks and Tips with NIO part ...
- 高吞吐高并发Java NIO服务的架构(NIO架构及应用之一)
高吞吐高并发Java NIO服务的架构(NIO架构及应用之一) http://maoyidao.iteye.com/blog/1149015 Java NIO成功的应用在了各种分布式.即时通信和中 ...
- 如何设计scalable 的系统 (转载)
Design a Scalable System Design a system that scales to millions of users (AWS based) Step 1: Outlin ...
- Cross-Domain Security For Data Vault
Cross-domain security for data vault is described. At least one database is accessible from a plural ...
- 可扩展的Web系统和分布式系统(Scalable Web Architecture and Distributed Systems)
Open source software has become a fundamental building block for some of the biggest websites. And a ...
- Scalable Web Architecture and Distributed Systems
转自:http://aosabook.org/en/distsys.html Scalable Web Architecture and Distributed Systems Kate Matsud ...
- 一段关于java NIO server端接受客户端socket连接;演示了关于channel,selector等组件的整合使用
public class ReactorDemo { public static void main(String[] args) throws IOException { ServerSocketC ...
随机推荐
- 系统启动时,spring配置文件解析失败,报”cvc-elt.1: 找不到元素 'beans' 的声明“异常
现象:spring加载配置文件applicationContext.xml出错,抛出nested exception is og.xml.sax.SAXParseException; lineNumb ...
- Unity3D脚本18:可视化辅助设置类 Gizmos
Gizmos 类 Gizmos用于场景中给出一个可视化的调试或辅助设置. 所有的Gizmos绘制都必须在脚本的OnDrawGizmos或OnDrawGizmosSelected函数中完成. OnD ...
- HDU 2255 & KM模板
题意: 一张完备二分图求最优完备匹配. SOL: 这题就不讲什么sol了...毕竟是裸的KM,不会的话可以看老人家的大白鼠,一些问题看代码注释.讲讲经历(悲惨的经历) 刚打完,自信地交上去发现MLE. ...
- 【BZOJ】1834: [ZJOI2010]network 网络扩容(最大流+费用流)
http://www.lydsy.com/JudgeOnline/problem.php?id=1834 我又思考人生了T_T,nd的数组开小了,一直wa,调了一个小时才发现啊!!!!!我一直以为我的 ...
- 【BZOJ】1934: [Shoi2007]Vote 善意的投票(网络流/-二分图匹配)
http://www.lydsy.com/JudgeOnline/problem.php?id=1934 一开始我想到了这是求最小割,但是我认为这题二分图可做,将1的放在左边,0的放在右边,然后朋友连 ...
- Uiautomator打包使用第三方库,报错的解决方案
问题引源: 在做自动化过程中,想在用例执行完毕后,自动生成该用例测试报告: 报告定义为Excel格式文件,且在用例执行过程中生成. 所以我在Uiautomator工程中引用了jxl.jar,用以处理E ...
- 二分查找算法java实现
今天看了一下JDK里面的二分法是实现,觉得有点小问题.二分法的实现有多种今天就给大家分享两种.一种是递归方式的,一种是非递归方式的.先来看看一些基础的东西. 1.算法概念. 二分查找算法也称为折半搜索 ...
- 提升 web 应用程序的性能(二)
最佳实践 本章将略述能帮助您提升 web 应用程序性能的最佳实践. 减少 HTTP 请求数 每个 HTTP 请求都有开销,包括查找 DNS.创建连接及等待响应,因此削减不必要的请求数可减少不必要的开销 ...
- Toll-Free Bridge
引 在深入了解桥接机制的时候看到一篇好文,虽然已经很久远,但是忍不住看了好几遍,心中诸多不解一扫而光.在此放上链接: 原文:http://ridiculousfish.com/blog/posts/b ...
- HDU 2594 Simpsons’ Hidden Talents(KMP的Next数组应用)
Simpsons’ Hidden Talents Time Limit: 2000/1000 MS (Java/Others) Memory Limit: 32768/32768 K (Java ...