https://groups.google.com/forum/#!topic/golang-nuts/I7a_3B8_9Gw

https://groups.google.com/forum/#!msg/golang-nuts/coc6bAl2kPM/ypNLG3I4mk0J

ask:   -----------------------

Hello,

I'm curious as to what the proper way of listening multiple simultaneous sockets is?
Please consider the two implementations I currently have: http://play.golang.org/p/LOd7q3aawd
 
Receive1 is a rather simple implementation, spawning a goroutine per connection, and simply blocking on each one via the ReadFromUDP method of the connection.
Receive2 is a more C-style approach, using the real select to block until either one of the sockets actually receives something, and then actually trying to read.
 
Now, Receive2 is _supposedly_ better, since we rely on the kernel to notify the select if one of the file descriptors is ready for reading. That would, under ideal conditions (and maybe less goroutines), allow the process itself to fall asleep until data starts pouring into the socket.
Receive1 is relying on the I/O read blocking until something comes along, and is doing so for each connection. Or at least that's what my understanding is, I'm not sure whether internally, the connection is still using select. Though even if it was, I don't thing the go scheduler is smart enough to put the whole process to sleep if two separate goroutines are waiting for I/O. That being said, Receive1 looks so much better than its counterpart.
 
If there is also a better way than either of these, please share it with me.
 
 
answer1:   ------------------------------------------
Recieve1 is better. Go will use asynchronous I/O (equivalent to select) under the covers for you. The go scheduler is smart enough to "put the whole process to sleep if [all] goroutines are waiting for I/O". Don't worry about it. Just use idiomatic Go.
 
answer2:   ------------------------------------------
Receive1 is certainly the Go way. I wonder however why you need to read from two UDP ports. UDP is connectionless, so you can support multiple clients with one open UDP port.

 

That being said you should know that any goroutine blocking in a system call consumes one kernel thread. This will not be a problem until you need to support thousands of connections. But at this scale the file descriptor bitmaps used by select become a performance bottleneck as well. In this situation you might want to look at epoll on Linux. On other systems poll might be an alternative. If you are in this territory I strongly recommend to have a look into Michael Kerrisk's excellent reference "The LINUX Programming Interface".

 
answer3:   ------------------------------------------
It's true that a goroutine blocking in a syscall consumes a kernel thread. However, Receive1 will *not* use any kernel threads while waiting in conn.ReadFromUDP, because under the covers, the Go runtime uses nonblocking I/O for all network activity. It's much better just to rely on the runtime implementation of network I/O rather than trying to roll your own. If you don't believe me, try doing syscall traces or profiling to prove it out.
 
answer4:   ------------------------------------------
receive2 approach is not portable (due to syscall), and is more complex. also, unless profiling can prove it, efficiency of the approach is a speculation.
 
answer5:   ------------------------------------------
Matt, thank you I learned something. During network access the Goroutine is not blocked in a syscall and Go is already using epoll internally. So unless you know what you are doing the Goroutine approach will work best.
 
 
这个问答中的 example code:
package main

import (
"fmt"
"net"
"os"
"syscall"
) func Receive1(conn1, conn2 *net.UDPConn, done chan struct{}) <-chan string {
res := make(chan string)
tokenChan := make(chan []string) for _, conn := range []*net.UDPConn{conn1, conn2} {
go func(conn *net.UDPConn) {
buf := make([]byte, )
for {
select {
case <-done:
return
default:
if n, _, err := conn.ReadFromUDP(buf); err == nil {
fmt.Println(string(buf[:n]))
res <- string(buf[:n])
}
}
}
}(conn)
} return res
} func Receive2(conn1, conn2 *net.UDPConn, done chan struct{}) <-chan string {
res := make(chan string)
fds := &syscall.FdSet{}
filemap := map[int]*os.File{}
var maxfd =
for _, conn := range []*net.UDPConn{conn1, conn2} {
if file, err := conn.File(); err == nil {
fd := int(file.Fd())
FD_SET(fds, fd)
filemap[fd] = file
if fd > maxfd {
maxfd = fd
}
}
} go func() {
for {
select {
case <-done:
return
default:
fdsetCopy := *fds
tv := syscall.Timeval{, }
if _, err := syscall.Select(maxfd+, &fdsetCopy, nil, nil, &tv); err == nil {
for fd, file := range filemap {
if !FD_ISSET(&fdsetCopy, fd) {
continue
} buf := make([]byte, )
if n, err := file.Read(buf); err == nil {
fmt.Println(string(buf[:n]))
res <- string(buf[:n])
}
}
}
}
}
}() return res
} func FD_SET(p *syscall.FdSet, i int) {
p.Bits[i/] |= << (uint(i) % )
} func FD_ISSET(p *syscall.FdSet, i int) bool {
return (p.Bits[i/] & ( << (uint(i) % ))) !=
} func main() {
fmt.Println("Hello, playground")
}
 
 
ask:  -------------------------------
Hello,
It is said that event-driven nonblocking model is not the preferred programming model in Go, so I use "one goroutine for one client" model, but is it OK to handle millions of concurrent goroutines in a server process?
 
And, how can I "select" millions of channel to see which goroutine has data received? The "select" statement can only select on predictable number of channels, not on a lot of unpredictable channels. And how can I "select" a TCP Connection (which is not a channel) to see if there is any data arrived? Is there any "design patterns" on concurrent programming in Go?
Thanks in advance.
 
answer: ----------------------------------

Hello,
 
It
is said that event-driven nonblocking model is not the preferred
programming model in Go, so I use "one goroutine for one client" model,
but is it OK to handle millions of concurrent goroutines in a server
process?
 

A goroutine itself is
4kb. So 1e6 goroutines would require 4gb of base memory. And then
whatever your server needs per goroutine that you add.

 
Any machine that might be handling 1e6 concurrent connections should have well over 4gb of memory. 
 
And, how can I "select" millions of channel to see which goroutine has data received?
 
That's
not how it works. You just try to .Read() in each goroutine, and the
select is done under the hood. The select{} statement is for channel
communication, specifically.
 
All IO in go is
event driven under the hood, but as far as the code you write, looks
linear. The Go runtime maintains a single thread that runs epoll or
kqueue or whatever under the hood, and wakes up a goroutine when new
data has arrived for that goroutine. 
 
The
"select" statement can only select on predictable number of channels,
not on a lot of unpredictable channels. And how can I "select" a TCP
Connection (which is not a channel) to see if there is any data arrived?
Is there any "design patterns" on concurrent programming in Go?
 
These problems you anticipate simply do not exist with go. Give it a shot!
 
 
 

golang 中处理大规模tcp socket网络连接的方法,相当于c语言的 poll 或 epoll的更多相关文章

  1. inux中,关于多路复用的使用,有三种不同的API,select、poll和epoll

    inux中,关于多路复用的使用,有三种不同的API,select.poll和epoll https://www.cnblogs.com/yearsj/p/9647135.html 在上一篇博文中提到了 ...

  2. 在c#中利用keep-alive处理socket网络异常断开的方法

    本文摘自 http://www.z6688.com/info/57987-1.htm 最近我负责一个IM项目的开发,服务端和客户端采用TCP协议连接.服务端采用C#开发,客户端采用Delphi开发.在 ...

  3. CentOS下netstat + awk 查看tcp的网络连接状态

    执行以下命令: #netstat -n | awk ‘/^tcp/ {++state[$NF]} END {for(key in state) print key."\t".sta ...

  4. 【虚拟机】在VMware中安装Server2008之后配置网络连接的几种方式

    VMware虚拟机的网络连接方式分为三种:桥接模式.NAT模式.仅主机(Host Only) (1)桥接模式 桥接模式即在虚拟机中虚拟一块网卡,这样主机和虚拟机在一个网段中就被看作是两个独立的IP地址 ...

  5. socket 网络连接基础

    socket 客户端 import socket 1.client = socket.socket()  # socket.TCP/IP 选择连接的类型,默认为本地连接 2.client.connec ...

  6. VC:检测网络连接的方法

    方法一: #include "stdafx.h" #include "windows.h" #include <Sensapi.h> #includ ...

  7. golang中浮点型底层存储原理和decimal使用方法

    var price float32 = 39.29 float64和float32类似,只是用于表示各部分的位数不同而已,其中:sign=1位,exponent=11位,fraction=52位,也就 ...

  8. 服务器中判断客户端socket断开连接的方法

    1, 如果服务端的Socket比客户端的Socket先关闭,会导致客户端出现TIME_WAIT状态,占用系统资源. 所以,必须等客户端先关闭Socket后,服务器端再关闭Socket才能避免TIME_ ...

  9. 服务器中判断客户端socket断开连接的方法【转】

    本文转载自:http://www.cnblogs.com/jacklikedogs/p/3976208.html 1, 如果服务端的Socket比客户端的Socket先关闭,会导致客户端出现TIME_ ...

随机推荐

  1. iOS 判断两个日期之间的间隔

    本文转载至 http://www.cnblogs.com/Ewenblog/p/3891791.html   两个时间段,判断之间的相差,做一些时间范围限制使用 NSDateFormatter * d ...

  2. 安卓手机优化 ROOT自启动管理 + 电量管理

    一.KingRoot + 净化大师 KingRoot 主要完成ROOT 和 自启动软件的管理 这里禁止的自启动管理 对于有些软件是不太管用 比如美团 手机百度等 净化大师 主要完成 高电量的软件优化 ...

  3. G - 生日蛋糕

    7月17日是Mr.W的生日,ACM-THU为此要制作一个体积为Nπ的M层生日蛋糕,每层都是一个圆柱体. 设从下往上数第i(1 <= i <= M)层蛋糕是半径为Ri, 高度为Hi的圆柱.当 ...

  4. php中memcached的使用

    文档所在:http://php.net/manual/zh/book.memcached.php 1.在php使用memcached操作之前要实例化memcached: [系统类]  2.添加服务器: ...

  5. Jenkins Docker安装及Docker build step插件部署配置

    生产部署环境:A:192.168.1.2 B:192.168.1.3  两台服务器系统均是Centos 7.3 , Docker版本都1.12.6 Jenkins安装操作步骤: 1.在A服务器上使用命 ...

  6. .NET Core开发日志——依赖注入

    依赖注入(DI)不是一个新的话题,它的出现是伴随着系统解耦的需要而几乎必然产生的. 在SOLID设计原则中,DIP(Dependency inversion principle)--依赖倒置,规定了& ...

  7. imu_tk标定算法原理

    imu_tk代码地址 https://bitbucket.org/alberto_pretto/imu_tk II. S ENSOR E RROR M ODEL 对于理想的IMU,加速度计三元组的3个 ...

  8. find实现特殊功能示例

    find列出目录下所有文件: # find /shell-script/ # find /shell-script/ -print find列出文件夹中所有开头为text的文件,参数-iname意思忽 ...

  9. 查询执行成本高(查询访问表数据行数多)而导致实例 CPU 使用率高是 MySQL 非常常见的问题

    MySQL CPU 使用率高的原因和解决方法_产品性能_常见问题_云数据库 RDS 版-阿里云 https://help.aliyun.com/knowledge_detail/51587.html ...

  10. A pointer is a variable whose value is the address of another variable 指针 null pointer 空指针 内存地址0 空指针检验

    小结: 1.指针的实际值为代表内存地址的16进制数: 2.不同指针的区别是他们指向的变量.常量的类型: https://www.tutorialspoint.com/cprogramming/c_po ...