golang语言中的context详解,Go Concurrency Patterns: Context
https://blog.golang.org/context
Introduction
In Go servers, each incoming request is handled in its own goroutine. Request handlers often start additional goroutines to access backends such as databases and RPC services. The set of goroutines working on a request typically needs access to request-specific values such as the identity of the end user, authorization tokens, and the request's deadline. When a request is canceled or times out, all the goroutines working on that request should exit quickly so the system can reclaim any resources they are using.
At Google, we developed a context
package that makes it easy to pass request-scoped values, cancelation signals, and deadlines across API boundaries to all the goroutines involved in handling a request. The package is publicly available as context. This article describes how to use the package and provides a complete working example.
Context
The core of the context
package is the Context
type:
// A Context carries a deadline, cancelation signal, and request-scoped values
// across API boundaries. Its methods are safe for simultaneous use by multiple
// goroutines.
type Context interface {
// Done returns a channel that is closed when this Context is canceled
// or times out.
Done() <-chan struct{} // Err indicates why this context was canceled, after the Done channel
// is closed.
Err() error // Deadline returns the time when this Context will be canceled, if any.
Deadline() (deadline time.Time, ok bool) // Value returns the value associated with key or nil if none.
Value(key interface{}) interface{}
}
(This description is condensed; the godoc is authoritative.)
The Done
method returns a channel that acts as a cancelation signal to functions running on behalf of the Context
: when the channel is closed, the functions should abandon their work and return. The Err
method returns an error indicating why the Context
was canceled. The Pipelines and Cancelation article discusses the Done
channel idiom in more detail.
A Context
does not have a Cancel
method for the same reason the Done
channel is receive-only: the function receiving a cancelation signal is usually not the one that sends the signal. In particular, when a parent operation starts goroutines for sub-operations, those sub-operations should not be able to cancel the parent. Instead, the WithCancel
function (described below) provides a way to cancel a new Context
value.
A Context
is safe for simultaneous use by multiple goroutines. Code can pass a single Context
to any number of goroutines and cancel that Context
to signal all of them.
The Deadline
method allows functions to determine whether they should start work at all; if too little time is left, it may not be worthwhile. Code may also use a deadline to set timeouts for I/O operations.
Value
allows a Context
to carry request-scoped data. That data must be safe for simultaneous use by multiple goroutines.
Derived contexts
The context
package provides functions to derive new Context
values from existing ones. These values form a tree: when a Context
is canceled, all Contexts
derived from it are also canceled.
Background
is the root of any Context
tree; it is never canceled:
// Background returns an empty Context. It is never canceled, has no deadline,
// and has no values. Background is typically used in main, init, and tests,
// and as the top-level Context for incoming requests.
func Background() Context
WithCancel
and WithTimeout
return derived Context
values that can be canceled sooner than the parent Context
. The Context
associated with an incoming request is typically canceled when the request handler returns. WithCancel
is also useful for canceling redundant requests when using multiple replicas. WithTimeout
is useful for setting a deadline on requests to backend servers:
// WithCancel returns a copy of parent whose Done channel is closed as soon as
// parent.Done is closed or cancel is called.
func WithCancel(parent Context) (ctx Context, cancel CancelFunc) // A CancelFunc cancels a Context.
type CancelFunc func() // WithTimeout returns a copy of parent whose Done channel is closed as soon as
// parent.Done is closed, cancel is called, or timeout elapses. The new
// Context's Deadline is the sooner of now+timeout and the parent's deadline, if
// any. If the timer is still running, the cancel function releases its
// resources.
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc)
WithValue
provides a way to associate request-scoped values with a Context
:
// WithValue returns a copy of parent whose Value method returns val for key.
func WithValue(parent Context, key interface{}, val interface{}) Context
The best way to see how to use the context
package is through a worked example.
Example: Google Web Search
Our example is an HTTP server that handles URLs like /search?q=golang&timeout=1s
by forwarding the query "golang" to the Google Web Search API and rendering the results. The timeout
parameter tells the server to cancel the request after that duration elapses.
The code is split across three packages:
- server provides the
main
function and the handler for/search
. - userip provides functions for extracting a user IP address from a request and associating it with a
Context
. - google provides the
Search
function for sending a query to Google.
The server program
The server program handles requests like /search?q=golang
by serving the first few Google search results for golang
. It registers handleSearch
to handle the /search
endpoint. The handler creates an initial Context
called ctx
and arranges for it to be canceled when the handler returns. If the request includes the timeout
URL parameter, the Context
is canceled automatically when the timeout elapses:
func handleSearch(w http.ResponseWriter, req *http.Request) {
// ctx is the Context for this handler. Calling cancel closes the
// ctx.Done channel, which is the cancellation signal for requests
// started by this handler.
var (
ctx context.Context
cancel context.CancelFunc
)
timeout, err := time.ParseDuration(req.FormValue("timeout"))
if err == nil {
// The request has a timeout, so create a context that is
// canceled automatically when the timeout expires.
ctx, cancel = context.WithTimeout(context.Background(), timeout)
} else {
ctx, cancel = context.WithCancel(context.Background())
}
defer cancel() // Cancel ctx as soon as handleSearch returns.
}
The handler extracts the query from the request and extracts the client's IP address by calling on the userip
package. The client's IP address is needed for backend requests, so handleSearch
attaches it to ctx
:
// Check the search query.
query := req.FormValue("q")
if query == "" {
http.Error(w, "no query", http.StatusBadRequest)
return
} // Store the user IP in ctx for use by code in other packages.
userIP, err := userip.FromRequest(req)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
ctx = userip.NewContext(ctx, userIP)
The handler calls google.Search
with ctx
and the query
:
// Run the Google search and print the results.
start := time.Now()
results, err := google.Search(ctx, query)
elapsed := time.Since(start)
If the search succeeds, the handler renders the results:
if err := resultsTemplate.Execute(w, struct {
Results google.Results
Timeout, Elapsed time.Duration
}{
Results: results,
Timeout: timeout,
Elapsed: elapsed,
}); err != nil {
log.Print(err)
return
}
Package userip
The userip package provides functions for extracting a user IP address from a request and associating it with a Context
. A Context
provides a key-value mapping, where the keys and values are both of type interface{}
. Key types must support equality, and values must be safe for simultaneous use by multiple goroutines. Packages like userip
hide the details of this mapping and provide strongly-typed access to a specific Context
value.
To avoid key collisions, userip
defines an unexported type key
and uses a value of this type as the context key:
// The key type is unexported to prevent collisions with context keys defined in
// other packages.
type key int // userIPkey is the context key for the user IP address. Its value of zero is
// arbitrary. If this package defined other context keys, they would have
// different integer values.
const userIPKey key =
FromRequest
extracts a userIP
value from an http.Request
:
func FromRequest(req *http.Request) (net.IP, error) {
ip, _, err := net.SplitHostPort(req.RemoteAddr)
if err != nil {
return nil, fmt.Errorf("userip: %q is not IP:port", req.RemoteAddr)
}
}
NewContext
returns a new Context
that carries a provided userIP
value:
func NewContext(ctx context.Context, userIP net.IP) context.Context {
return context.WithValue(ctx, userIPKey, userIP)
}
FromContext
extracts a userIP
from a Context
:
func FromContext(ctx context.Context) (net.IP, bool) {
// ctx.Value returns nil if ctx has no value for the key;
// the net.IP type assertion returns ok=false for nil.
userIP, ok := ctx.Value(userIPKey).(net.IP)
return userIP, ok
}
Package google
The google.Search function makes an HTTP request to the Google Web Search API and parses the JSON-encoded result. It accepts a Context
parameter ctx
and returns immediately if ctx.Done
is closed while the request is in flight.
The Google Web Search API request includes the search query and the user IP as query parameters:
func Search(ctx context.Context, query string) (Results, error) {
// Prepare the Google Search API request.
req, err := http.NewRequest("GET", "https://ajax.googleapis.com/ajax/services/search/web?v=1.0", nil)
if err != nil {
return nil, err
}
q := req.URL.Query()
q.Set("q", query) // If ctx is carrying the user IP address, forward it to the server.
// Google APIs use the user IP to distinguish server-initiated requests
// from end-user requests.
if userIP, ok := userip.FromContext(ctx); ok {
q.Set("userip", userIP.String())
}
req.URL.RawQuery = q.Encode()
}
Search
uses a helper function, httpDo
, to issue the HTTP request and cancel it if ctx.Done
is closed while the request or response is being processed. Search
passes a closure to httpDo
handle the HTTP response:
var results Results
err = httpDo(ctx, req, func(resp *http.Response, err error) error {
if err != nil {
return err
}
defer resp.Body.Close() // Parse the JSON search result.
// https://developers.google.com/web-search/docs/#fonje
var data struct {
ResponseData struct {
Results []struct {
TitleNoFormatting string
URL string
}
}
}
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return err
}
for _, res := range data.ResponseData.Results {
results = append(results, Result{Title: res.TitleNoFormatting, URL: res.URL})
}
return nil
})
// httpDo waits for the closure we provided to return, so it's safe to
// read results here.
return results, err
The httpDo
function runs the HTTP request and processes its response in a new goroutine. It cancels the request if ctx.Done
is closed before the goroutine exits:
func httpDo(ctx context.Context, req *http.Request, f func(*http.Response, error) error) error {
// Run the HTTP request in a goroutine and pass the response to f.
tr := &http.Transport{}
client := &http.Client{Transport: tr}
c := make(chan error, )
go func() { c <- f(client.Do(req)) }()
select {
case <-ctx.Done():
tr.CancelRequest(req)
<-c // Wait for f to return.
return ctx.Err()
case err := <-c:
return err
}
}
Adapting code for Contexts
Many server frameworks provide packages and types for carrying request-scoped values. We can define new implementations of the Context
interface to bridge between code using existing frameworks and code that expects a Context
parameter.
For example, Gorilla's github.com/gorilla/context package allows handlers to associate data with incoming requests by providing a mapping from HTTP requests to key-value pairs. In gorilla.go, we provide a Context
implementation whose Value
method returns the values associated with a specific HTTP request in the Gorilla package.
Other packages have provided cancelation support similar to Context
. For example, Tomb provides a Kill
method that signals cancelation by closing a Dying
channel. Tomb
also provides methods to wait for those goroutines to exit, similar to sync.WaitGroup
. In tomb.go, we provide a Context
implementation that is canceled when either its parent Context
is canceled or a provided Tomb
is killed.
Conclusion
At Google, we require that Go programmers pass a Context
parameter as the first argument to every function on the call path between incoming and outgoing requests. This allows Go code developed by many different teams to interoperate well. It provides simple control over timeouts and cancelation and ensures that critical values like security credentials transit Go programs properly.
Server frameworks that want to build on Context
should provide implementations of Context
to bridge between their packages and those that expect a Context
parameter. Their client libraries would then accept a Context
from the calling code. By establishing a common interface for request-scoped data and cancelation, Context
makes it easier for package developers to share code for creating scalable services.
By Sameer Ajmani
Using contexts to avoid leaking goroutines
https://rakyll.org/leakingctx/
The context package makes it possible to manage a chain of calls within the same call path by signaling context’s Done channel.
In this article, we will examine how to use the context package to avoid leaking goroutines.
Assume, you have a function that starts a goroutine internally. Once this function is called, the caller may not be able to terminate the goroutine started by the function.
// gen is a broken generator that will leak a goroutine.
func gen() <-chan int {
ch := make(chan int)
go func() {
var n int
for {
ch <- n
n++
}
}()
return ch
}
The generator above starts a goroutine with an infinite loop, but the caller consumes the values until n is equal to 5.
// The call site of gen doesn't have a
for n := range gen() {
fmt.Println(n)
if n == {
break
}
}
Once the caller is done with the generator (when it breaks the loop), the goroutine will run forever executing the infinite loop. Our code will leak a goroutine.
We can avoid the problem by signaling the internal goroutine with a stop channel but there is a better solution: cancellable contexts. The generator can select on a context’s Done channel and once the context is done, the internal goroutine can be cancelled.
// gen is a generator that can be cancellable by cancelling the ctx.
func gen(ctx context.Context) <-chan int {
ch := make(chan int)
go func() {
var n int
for {
select {
case <-ctx.Done():
return // avoid leaking of this goroutine when ctx is done.
case ch <- n:
n++
}
}
}()
return ch
}
Now, the caller can signal the generator when it is done consuming. Once cancel function is called, the internal goroutine will be returned.
ctx, cancel := context.WithCancel(context.Background())
defer cancel() // make sure all paths cancel the context to avoid context leak for n := range gen(ctx) {
fmt.Println(n)
if n == {
cancel()
break
}
} // ...
The full program is available as a gist.
Context and Cancellation of goroutines
http://dahernan.github.io/2015/02/04/context-and-cancellation-of-goroutines/
Yesterday I went to the event London Go Gathering, where all the talks had a great level, but particulary Peter Bourgon gave me idea to write about the excelent package context.
Context is used to pass request scoped variables, but in this case I’m only going to focus in cancelation signals.
Lets say that I have a program that execute a long running function, in this case work
and we run it in a separate go routine.
package main import (
"fmt"
"sync"
"time" ) var (
wg sync.WaitGroup
) func work() error {
defer wg.Done() for i := ; i < ; i++ {
select {
case <-time.After( * time.Second):
fmt.Println("Doing some work ", i)
}
}
return nil
} func main() {
fmt.Println("Hey, I'm going to do some work") wg.Add()
go work()
wg.Wait() fmt.Println("Finished. I'm going home")
}
$ go run work.go
Hey, I'm going to do some work
Doing some work
Doing some work
Doing some work
Doing some work
...
Doing some work
Finished. I'm going home
Now imagine that we have to call that work
function from a user interaction or a http request, we probably don’t want to wait forever for that goroutine to finish, so a common pattern is to set a timeout, using a buffered channel, like this:
package main import (
"fmt"
"log"
"time"
) func work() error {
for i := ; i < ; i++ {
select {
case <-time.After( * time.Second):
fmt.Println("Doing some work ", i)
}
}
return nil
} func main() {
fmt.Println("Hey, I'm going to do some work") ch := make(chan error, )
go func() {
ch <- work()
}() select {
case err := <-ch:
if err != nil {
log.Fatal("Something went wrong :(", err)
}
case <-time.After( * time.Second):
fmt.Println("Life is to short to wait that long")
} fmt.Println("Finished. I'm going home")
}
$ go run work.go
Hey, I'm going to do some work
Doing some work
Doing some work
Life is to short to wait that long
Finished. I'm going home
Now, is a little bit better because, the main execution doesn’t have to wait for work
if it’s timing out.
But it has a problem, if my program is still running like for example a web server, even if I don’t wait for the function work
to finish, the goroutine it would be running and consuming resources. So I need a way to cancel that goroutine.
For cancelation of the goroutine we can use the context package. We have to change the function to accept an argument of type context.Context
, by convention it’s usuallly the first argument.
package main import (
"fmt"
"sync"
"time" "golang.org/x/net/context"
) var (
wg sync.WaitGroup
) func work(ctx context.Context) error {
defer wg.Done() for i := ; i < ; i++ {
select {
case <-time.After( * time.Second):
fmt.Println("Doing some work ", i) // we received the signal of cancelation in this channel
case <-ctx.Done():
fmt.Println("Cancel the context ", i)
return ctx.Err()
}
}
return nil
} func main() {
ctx, cancel := context.WithTimeout(context.Background(), *time.Second)
defer cancel() fmt.Println("Hey, I'm going to do some work") wg.Add()
go work(ctx)
wg.Wait() fmt.Println("Finished. I'm going home")
}
$ go run work.go
Hey, I'm going to do some work
Doing some work
Cancel the context
Finished. I'm going home
This is pretty good!, apart that the code looks more simple to manage the timeout, now we are making sure that the function work
doesn’t waste any resource.
These examples are good to learn the basics, but let’s try to make it more real. Now the work
function is going to do an http request to a server and the server is going to be this other program:
package main // Lazy and Very Random Server
import (
"fmt"
"math/rand"
"net/http"
"time"
) func main() {
http.HandleFunc("/", LazyServer)
http.ListenAndServe(":1111", nil)
} // sometimes really fast server, sometimes really slow server
func LazyServer(w http.ResponseWriter, req *http.Request) {
headOrTails := rand.Intn() if headOrTails == {
time.Sleep( * time.Second)
fmt.Fprintf(w, "Go! slow %v", headOrTails)
fmt.Printf("Go! slow %v", headOrTails)
return
} fmt.Fprintf(w, "Go! quick %v", headOrTails)
fmt.Printf("Go! quick %v", headOrTails)
return
}
Randomly is going to be very quick or very slow, we can check that with curl
$ curl http://localhost:1111/
Go! quick
$ curl http://localhost:1111/
Go! quick
$ curl http://localhost:1111/
*some seconds later*
Go! slow
So we are going to make an http request to this server, in a goroutine, but if the server is slow we are going to Cancel the request and return quickly, so we can manage the cancellation and free the connection.
package main import (
"fmt"
"io/ioutil"
"net/http"
"sync"
"time" "golang.org/x/net/context"
) var (
wg sync.WaitGroup
) // main is not changed
func main() {
ctx, cancel := context.WithTimeout(context.Background(), *time.Second)
defer cancel() fmt.Println("Hey, I'm going to do some work") wg.Add()
go work(ctx)
wg.Wait() fmt.Println("Finished. I'm going home") } func work(ctx context.Context) error {
defer wg.Done() tr := &http.Transport{}
client := &http.Client{Transport: tr} // anonymous struct to pack and unpack data in the channel
c := make(chan struct {
r *http.Response
err error
}, ) req, _ := http.NewRequest("GET", "http://localhost:1111", nil)
go func() {
resp, err := client.Do(req)
fmt.Println("Doing http request is a hard job")
pack := struct {
r *http.Response
err error
}{resp, err}
c <- pack
}() select {
case <-ctx.Done():
tr.CancelRequest(req)
<-c // Wait for client.Do
fmt.Println("Cancel the context")
return ctx.Err()
case ok := <-c:
err := ok.err
resp := ok.r
if err != nil {
fmt.Println("Error ", err)
return err
} defer resp.Body.Close()
out, _ := ioutil.ReadAll(resp.Body)
fmt.Printf("Server Response: %s\n", out) }
return nil
}
$ go run work.go
Hey, I'm going to do some work
Doing http request is a hard job
Server Response: Go! quick
Finished. I'm going home $ go run work.go
Hey, I'm going to do some work
Doing http request is a hard job
Cancel the context
Finished. I'm going home
As you can see in the output, we avoid the slow responses from the server.
In the client the tcp connection is canceled so is not going to be busy waiting for a slow response, so we don’t waste resources.
Happy coding gophers!.
golang语言中的context详解,Go Concurrency Patterns: Context的更多相关文章
- Android中Context详解 ---- 你所不知道的Context
转自:http://blog.csdn.net/qinjuning/article/details/7310620Android中Context详解 ---- 你所不知道的Context 大家好, ...
- Android中Context详解 ---- 你所不知道的Context(转)
Android中Context详解 ---- 你所不知道的Context(转) 本文出处 :http://b ...
- 转:Android中Context详解 ---- 你所不知道的Context
转:http://blog.csdn.net/qinjuning/article/details/7310620 转:http://blog.csdn.net/lmj623565791/article ...
- C语言中static变量详解
Static翻译出来是“静态”“静止”的意思,在C语言中的意思其实和它的本意差不多,表示“静态”或者“全局”的意思,用来修饰变量和函数.经static修饰过后的变量或者函数的作用域或者存储域会发生变化 ...
- php 中函数获取可变参数的方法, 这个语法有点像 golang 语言中的
原文呢:http://php.net/manual/en/functions.arguments.php#functions.arguments.type-declaration.strict Onl ...
- Golang并发模型之Context详解
对于 Golang 开发者来说context(上下文)包一定不会陌生.但很多时候,我们懒惰的只是见过它,或能起到什么作用,并不会去深究它. 应用场景:在 Go http 包的 Server 中,每一个 ...
- tomcat 配置文件server.xml 详解 Connector Engine Host Context
目录 一 server.xml 1.1 server 配置 1.2 service 配置 1.3 Executor 1.4 Connector 配置 1.5 Engine 其他tocmat 文章 一 ...
- Golang GC 垃圾回收机制详解
摘要 在实际使用 go 语言的过程中,碰到了一些看似奇怪的内存占用现象,于是决定对go语言的垃圾回收模型进行一些研究.本文对研究的结果进行一下总结. 什么是垃圾回收? 曾几何时,内存管理是程序员开发应 ...
- Android面试收集录18 Android Context详解
Activity mActivity =new Activity() 作为Android开发者,不知道你有没有思考过这个问题,Activity可以new吗?Android的应用程序开发采用JAVA语言 ...
随机推荐
- listView优化方案
1.如果自定义适配器,那么在getView方法中要考虑方法传进来的参数contentView是否为null,如果为null就创建contentView并返回,如果不为null则直接使用.在这个方法中尽 ...
- struts.xml文件中配置tiles.xml
Apache Tiles是一个JavaEE应用的页面布局框架.Tiles框架提供了一种模板机制,可以为某一类页面定义一个通用的模板,该模板定义了页面的整体布局.布局由可以复用的多个块组成,每个页面可以 ...
- 字符串处理总结之一(C#String类)
C#(静态String类) C#中提供了比较全面的字符串处理方法,很多函数都进行了封装为我们的编程工作提供了很大的便利.System.String是最常用的字符串操作类,可以帮助开发者完成绝大部分的字 ...
- SharpGL学习笔记(一) 平台构建与Opengl的hello World
(一)平台构建与Opengl的hello World OpenGL就是3d绘图的API,微软针和它竞争推出D3D,也就是玩游戏时最常见的DirectorX组件中的3d功能. 所以不要指望windows ...
- Hive学习之数据去重
insert overwrite table store select t.p_key,t.sort_word from ( select p_key, sort_word , row_number( ...
- HTTP协议详解(真的很经典)(转载)
HTTP协议详解(真的很经典):http://www.cnblogs.com/li0803/archive/2008/11/03/1324746.html 引言 HTTP是一个属于应用层的面向对象的协 ...
- Doxygen生成C++中文文档配置注意事项
打开对应的Doxyfile,修改如下: 1.OUTPUT_LANGUAGE = Chinese. 2.INPUT_ENCODING = GB2312.
- 【转】C内存管理
在任何程序设计环境及语言中,内存管理都十分重要.在目前的计算机系统或嵌入式系统中,内存资源仍然是有限的.因此在程序设计中,有效地管理内存资源是程序员首先考虑的问题. 第1节主要介绍内存管理基本概念,重 ...
- Memcached概念、作用、运行原理、特性、不足简单梳理(1)
大家可能对memcached这种产品早有了解,或者已经应用在自己的网站中了,但是也有一些朋友从来都没有听说过或者使用过.这都没什么关系,本文旨在从各个角度综合的介绍这种产品,尽量深入浅出,如果能对您现 ...
- iOS - Block的简单使用
Block 的使用有两种: .独立Block .内联Block <一>独立Block 使用方式 一.定义一个Block Object,并调用. 1.定义 // 定义一个Bl ...