Data Race Detector 数据种类探测器:数据种类探测器手册

Introduction

Data races are among the most common and hardest to debug types of bugs in concurrent systems. A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write. See the The Go Memory Model for details.

Here is an example of a data race that can lead to crashes and memory corruption:

func main() {
c := make(chan bool)
m := make(map[string]string)
go func() {
m["1"] = "a" // First conflicting access.
c <- true
}()
m["2"] = "b" // Second conflicting access.
<-c
for k, v := range m {
fmt.Println(k, v)
}
}

Usage

To help diagnose such bugs, Go includes a built-in data race detector. To use it, add the -race flag to the go command:

$ go test -race mypkg    // to test the package
$ go run -race mysrc.go // to run the source file
$ go build -race mycmd // to build the command
$ go install -race mypkg // to install the package

Report Format

When the race detector finds a data race in the program, it prints a report. The report contains stack traces for conflicting accesses, as well as stacks where the involved goroutines were created. Here is an example:

WARNING: DATA RACE
Read by goroutine 185:
net.(*pollServer).AddFD()
src/net/fd_unix.go:89 +0x398
net.(*pollServer).WaitWrite()
src/net/fd_unix.go:247 +0x45
net.(*netFD).Write()
src/net/fd_unix.go:540 +0x4d4
net.(*conn).Write()
src/net/net.go:129 +0x101
net.func·060()
src/net/timeout_test.go:603 +0xaf Previous write by goroutine 184:
net.setWriteDeadline()
src/net/sockopt_posix.go:135 +0xdf
net.setDeadline()
src/net/sockopt_posix.go:144 +0x9c
net.(*conn).SetDeadline()
src/net/net.go:161 +0xe3
net.func·061()
src/net/timeout_test.go:616 +0x3ed Goroutine 185 (running) created at:
net.func·061()
src/net/timeout_test.go:609 +0x288 Goroutine 184 (running) created at:
net.TestProlongTimeout()
src/net/timeout_test.go:618 +0x298
testing.tRunner()
src/testing/testing.go:301 +0xe8

Options

The GORACE environment variable sets race detector options. The format is:

GORACE="option1=val1 option2=val2"

The options are:

  • log_path (default stderr): The race detector writes its report to a file named log_path.pid. The special names stdout and stderr cause reports to be written to standard output and standard error, respectively.
  • exitcode (default 66): The exit status to use when exiting after a detected race.
  • strip_path_prefix (default ""): Strip this prefix from all reported file paths, to make reports more concise.
  • history_size (default 1): The per-goroutine memory access history is 32K * 2**history_size elements. Increasing this value can avoid a "failed to restore the stack" error in reports, at the cost of increased memory usage.
  • halt_on_error (default 0): Controls whether the program exits after reporting first data race.

Example:

$ GORACE="log_path=/tmp/race/report strip_path_prefix=/my/go/sources/" go test -race

Excluding Tests

When you build with -race flag, the go command defines additional build tag race. You can use the tag to exclude some code and tests when running the race detector. Some examples:

// +build !race

package foo

// The test contains a data race. See issue 123.
func TestFoo(t *testing.T) {
// ...
} // The test fails under the race detector due to timeouts.
func TestBar(t *testing.T) {
// ...
} // The test takes too long under the race detector.
func TestBaz(t *testing.T) {
// ...
}

How To Use

To start, run your tests using the race detector (go test -race). The race detector only finds races that happen at runtime, so it can't find races in code paths that are not executed. If your tests have incomplete coverage, you may find more races by running a binary built with -race under a realistic workload.

Typical Data Races

Here are some typical data races. All of them can be detected with the race detector.

Race on loop counter

func main() {
var wg sync.WaitGroup
wg.Add(5)
for i := 0; i < 5; i++ {
go func() {
fmt.Println(i) // Not the 'i' you are looking for.
wg.Done()
}()
}
wg.Wait()
}

The variable i in the function literal is the same variable used by the loop, so the read in the goroutine races with the loop increment. (This program typically prints 55555, not 01234.) The program can be fixed by making a copy of the variable:

func main() {
var wg sync.WaitGroup
wg.Add(5)
for i := 0; i < 5; i++ {
go func(j int) {
fmt.Println(j) // Good. Read local copy of the loop counter.
wg.Done()
}(i)
}
wg.Wait()
}

Accidentally shared variable

// ParallelWrite writes data to file1 and file2, returns the errors.
func ParallelWrite(data []byte) chan error {
res := make(chan error, 2)
f1, err := os.Create("file1")
if err != nil {
res <- err
} else {
go func() {
// This err is shared with the main goroutine,
// so the write races with the write below.
_, err = f1.Write(data)
res <- err
f1.Close()
}()
}
f2, err := os.Create("file2") // The second conflicting write to err.
if err != nil {
res <- err
} else {
go func() {
_, err = f2.Write(data)
res <- err
f2.Close()
}()
}
return res
}

The fix is to introduce new variables in the goroutines (note the use of :=):

			...
_, err := f1.Write(data)
...
_, err := f2.Write(data)
...

Unprotected global variable

If the following code is called from several goroutines, it leads to races on the service map. Concurrent reads and writes of the same map are not safe:

var service map[string]net.Addr

func RegisterService(name string, addr net.Addr) {
service[name] = addr
} func LookupService(name string) net.Addr {
return service[name]
}

To make the code safe, protect the accesses with a mutex:

var (
service map[string]net.Addr
serviceMu sync.Mutex
) func RegisterService(name string, addr net.Addr) {
serviceMu.Lock()
defer serviceMu.Unlock()
service[name] = addr
} func LookupService(name string) net.Addr {
serviceMu.Lock()
defer serviceMu.Unlock()
return service[name]
}

Primitive unprotected variable

Data races can happen on variables of primitive types as well (boolintint64, etc.), as in this example:

type Watchdog struct{ last int64 }

func (w *Watchdog) KeepAlive() {
w.last = time.Now().UnixNano() // First conflicting access.
} func (w *Watchdog) Start() {
go func() {
for {
time.Sleep(time.Second)
// Second conflicting access.
if w.last < time.Now().Add(-10*time.Second).UnixNano() {
fmt.Println("No keepalives for 10 seconds. Dying.")
os.Exit(1)
}
}
}()
}

Even such "innocent" data races can lead to hard-to-debug problems caused by non-atomicity of the memory accesses, interference with compiler optimizations, or reordering issues accessing processor memory .

A typical fix for this race is to use a channel or a mutex. To preserve the lock-free behavior, one can also use thesync/atomic package.

type Watchdog struct{ last int64 }

func (w *Watchdog) KeepAlive() {
atomic.StoreInt64(&w.last, time.Now().UnixNano())
} func (w *Watchdog) Start() {
go func() {
for {
time.Sleep(time.Second)
if atomic.LoadInt64(&w.last) < time.Now().Add(-10*time.Second).UnixNano() {
fmt.Println("No keepalives for 10 seconds. Dying.")
os.Exit(1)
}
}
}()
}

Supported Systems

The race detector runs on darwin/amd64freebsd/amd64linux/amd64, and windows/amd64.

Runtime Overhead

The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.

28 Data Race Detector 数据种类探测器:数据种类探测器手册的更多相关文章

  1. 使用Spring Data ElasticSearch+Jsoup操作集群数据存储

    使用Spring Data ElasticSearch+Jsoup操作集群数据存储 1.使用Jsoup爬取京东商城的商品数据 1)获取商品名称.价格以及商品地址,并封装为一个Product对象,代码截 ...

  2. Spring Data:企业级Java的现代数据访问技术(影印版)

    <Spring Data:企业级Java的现代数据访问技术(影印版)>基本信息原书名:Spring Data:Modern Data Access for Enterprise Java作 ...

  3. 转:代码的坏味道之二十 :Data Class(纯稚的数据类)或POJO

    所谓Data Class是指:它们拥有一些值域(fields),以及用于访问(读写]这些值域的函数,除此之外一无长物.这样的classes只是一种「不会说话的数据容器」,它们几乎一定被其他classe ...

  4. 17.1.1.8?Setting Up Replication with Existing Data设置复制使用存在的数据

    17.1.1.8?Setting Up Replication with Existing Data设置复制使用存在的数据 当设置复制使用存在的数据,你需要确定如何最好的从master 得到数据到sl ...

  5. 【转】Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍

    Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍 要求: 今天要测试上千条数据,且每条数据要求执行多次,(模拟多用户多次抽奖) 1.用户id有175个,且没有任何排序规 ...

  6. Jmeter===Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍(转)

    Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍 要求: 今天要测试上千条数据,且每条数据要求执行多次,(模拟多用户多次抽奖) 1.用户id有175个,且没有任何排序规 ...

  7. elasticsearch负载均衡节点——客户端节点 node.master: false node.data: false 其他配置和master 数据节点一样

    elasticSearch的配置文件中有2个参数:node.master和node.data.这两个参 数搭配使用时,能够帮助提供服务器性能. 数据节点node.master: false node. ...

  8. iOS教程:如何使用Core Data – 预加载和引入数据

    这是接着上一次<iOS教程:Core Data数据持久性存储基础教程>的后续教程,程序也会使用上一次制作完成的. 再上一个教程中,我们只做了一个数据模型,之后我们使用这个数据模型中的数据创 ...

  9. Azure Data Factory(二)复制数据

    一,引言 上一篇主要只讲了Azure Data Factory的一些主要概念,今天开始新的内容,我们开始通过Azure DevOps 或者 git 管理 Azure Data Factory 中的源代 ...

随机推荐

  1. [HDU4787]GRE Words Revenge 解题报告

    这是我之前博客里提到的一道AC自动机的练手题,但是要完成这道题,我之前博客里提到的东西还不够,这里总结一下这道题. 这道题不是一般的裸的AC自动机,它的询问和插入是交叉出现的所以用我之前写的板子不大合 ...

  2. 《Linux内核设计与实现》学习总结 Chap18

    一.准备开始 1.一个确定的bug,但大部分bug通常都不是行为可靠且定义明确的. 2.一个藏匿bug的内核版本. 3.相关内核代码的知识和运气. 二.内核中的bug 1.bug的表象: 明白无误的错 ...

  3. MyBatis.2剖析

    上次给大家介绍了一下properties 和 environments 的配置, 接下来就正式开始看源码了: 上次例子中,我们以 SqlSessionFactoryBuilder 去创建 SqlSes ...

  4. spark 性能调优(一) 性能调优的本质、spark资源使用原理、调优要点分析

    转载:http://www.cnblogs.com/jcchoiling/p/6440709.html 一.大数据性能调优的本质 编程的时候发现一个惊人的规律,软件是不存在的!所有编程高手级别的人无论 ...

  5. 关于JavaScript诞生之初的趣事

    我在读很多优秀的JavaScript源码时候常常被它诡异的语法搞的精疲力尽,所以时不时的加固JavaScript基础知识是十分有必要的,这些知识每次温故或者你换个角度去思考都能收获颇多,那么如此深不可 ...

  6. golang 安装一个项目下的所有依赖

    go get -v ./...

  7. web项目中日志管理工具的使用

    在web项目中,很多时候会用到日志管理工具,常见的日志管理用具有:JDK logging(配置文件:logging.properties) 和log4j(配置文件:log4j.properties) ...

  8. iterm2切换显示屏vim乱行解决

    http://note.youdao.com/noteshare?id=5aec9d82cc3a95b6909e9966b4aa3227

  9. HOJ 13102 Super Shuttle (圆的反演变换)

    HOJ 13102 Super Shuttle 链接:http://49.123.82.55/online/?action=problem&type=show&id=13102 题意: ...

  10. 二叉树(前序,中序,后序,层序)遍历递归与循环的python实现

    二叉树的遍历是在面试使比较常见的项目了.对于二叉树的前中后层序遍历,每种遍历都可以递归和循环两种实现方法,且每种遍历的递归实现都比循环实现要简洁.下面做一个小结. 一.中序遍历 前中后序三种遍历方法对 ...