BTrace: DTrace for Java… ish

DTrace first peered into Java in early 2005 thanks to an early prototype by Jarod Jenson that led eventually to the inclusion of USDT probes in the HotSpot JVM. If you want to see where, say, the java.net.SocketOutputStream.write() method is called, you can simply run this DTrace script:

hotspot$target:::method-entry
/copyinstr(arg1, arg2) == "java/net/SocketOutputStream" &&
copyinstr(arg3, arg4) == "write"/
{
jstack(50, 8000);
}

And that will work as long as you rememember to start your JVM with the -XX:+ExtendedDTraceProbes option or you use the jinfo utility to enable it after the fact. And as long as you don’t mind a crippling performance penalty (hint: you probably do).

Inspired by dtrace.conf a few weeks ago, I wanted to sketch out what the real Java provider would look like:

java$target:java.net.SocketOutputStream:write:entry
{
        jstack(50,8000);
}

And check it out:

# jdtrace.pl -p $(pgrep java) -n 'java$target:java.net.SocketOutputStream::entry{ jstack(50,8000); }'
dtrace: script '/tmp/jdtrace.19092/jdtrace.d' matched 0 probes
CPU     ID                    FUNCTION:NAME
0  64991 Java_com_sun_btrace_BTraceRuntime_dtraceProbe0:event
libbtrace.so`Java_com_sun_btrace_BTraceRuntime_dtraceProbe0+0xbb
com/sun/btrace/BTraceRuntime.dtraceProbe0(Ljava/lang/String;Ljava/lang/String;II)I
com/sun/btrace/BTraceRuntime.dtraceProbe(Ljava/lang/String;Ljava/lang/String;II)I
com/sun/btrace/BTraceUtils$D.probe(Ljava/lang/String;Ljava/lang/String;II)I
com/sun/btrace/BTraceUtils$D.probe(Ljava/lang/String;Ljava/lang/String;)I
java/net/SocketOutputStream.$btrace$jdtrace$probe1(Ljava/lang/String;Ljava/lang/String;)V
java/net/SocketOutputStream.write([BII)V
sun/nio/cs/StreamEncoder.writeBytes()V
sun/nio/cs/StreamEncoder.implFlushBuffer()V
sun/nio/cs/StreamEncoder.implFlush()V
sun/nio/cs/StreamEncoder.flush()V
java/io/OutputStreamWriter.flush()V
java/io/BufferedWriter.flush()V
java/io/PrintWriter.newLine()V
java/io/PrintWriter.println()V
java/io/PrintWriter.println(Ljava/lang/String;)V
com/delphix/appliance/server/ham/impl/HAMonitorServerThread.run()V
java/util/concurrent/ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V
java/util/concurrent/ThreadPoolExecutor$Worker.run()V
java/lang/Thread.run()V
StubRoutines (1)
libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x21d
libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27
libjvm.so`__1cJJavaCallsMcall_virtual6FpnJJavaValue_nGHandle_nLKlassHandle_nMsymbolHandle_5pnGThread__v_+0x149
libjvm.so`__1cMthread_entry6FpnKJavaThread_pnGThread__v_+0x113
libjvm.so`__1cKJavaThreadDrun6M_v_+0x2c6
libjvm.so`java_start+0x1f2
libc.so.1`_thrp_setup+0x9b
libc.so.1`_lwp_start

Obviously there's something fishy going on. First, we're using perl -- the shibboleth of fake-o-ware -- and there's this BTrace stuff in the output.

Faking it with BTrace

BTrace is a dynamic instrumentation tool for Java; it is both inspired by DTrace and contains some DTrace integration. The perl script above takes the DTrace syntax and generates a DTrace script and a BTrace-enabled Java source file.

Like DTrace, BTrace lets you specify the points of instrumentation in your Java program as well as the actions to take. Here's what our generated source file looks like.

import com.sun.btrace.annotations.*;
import static com.sun.btrace.BTraceUtils.*;
@BTrace
public class jdtrace {
@OnMethod(clazz="java.net.SocketOutputStream", method="write", location=@Location(Kind.ENTRY))
public static void probe1(@ProbeClassName String c,
@ProbeMethodName String m) {
String name = "entry";
String p = Strings.strcat(c, Strings.strcat(":",
Strings.strcat(m, Strings.strcat(":", name))));
D.probe(p, "");
}
}

Note that we specify where to trace (this can be a regular expression), and then take the action of joining the class, method, and "entry" string into a single string that we pass to the D.probe() method that causes a BTrace USDT probe to fire.

Here's what the D script looks like:

btrace$target:::event
{
this->__jd_arg = copyinstr(arg0);
this->__jd_mod = strtok(this->__jd_arg, ":");
this->__jd_func = strtok(NULL, ":");
this->__jd_name = strtok(NULL, ":");
} btrace$target:::event
/((this->__jd_mod == "java.net.SocketOutputStream" &&
this->__jd_func == "write" &&
this->__jd_name == "entry"))/
{
jstack(50,8000);
}

It's pretty simple. We parse the string that was passed to D.probe(), and disassemble it into the DTrace notion of module, function, and name. We then use that information so that the specified actions are executed as appropriate (we could have specified different Java methods to probe, and different actions to take for each). Here's the code if you're interested.

This isn't the real Java provider, but is it close enough? Unfortunately not. The most glaring problem is that BTrace sometimes renders my Java process unresponsive. Other times it leaves instrumentation behind with no way of extracting it. The word "safe" appears as the third word on the BTrace website ("BTrace is safe"), but apparently there's still some way to go to achieve the requisite level of safety.

A Better BTrace

BTrace is an interesting tool for examining Java programs, but one obvious obstacle is that the programs are pretty cumbersome to write. With BTrace, we should be able to write a simple one-liner to see where we are when the java.net.SocketOutputStream.write() method is called, but instead we have to write a fairly cumbersome program:

import com.sun.btrace.annotations.*;
import static com.sun.btrace.BTraceUtils.*;
@BTrace
public class TraceWrite {
@OnMethod(clazz="java.net.SocketOutputStream", method="write", location=@Location(Kind.ENTRY))
public static void onWrite() {
jstack();
}
}

DTrace-inspired syntax would let users iterate much more quickly:

$ dbtrace -p $(pgrep -n java) -n 'java.net.SocketOutputStream:write:entry{ jstack(); }'
java.net.SocketOutputStream.write(SocketOutputStream.java)
sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
java.io.BufferedWriter.flush(BufferedWriter.java:236)
java.io.PrintWriter.newLine(PrintWriter.java:438)
java.io.PrintWriter.println(PrintWriter.java:585)
java.io.PrintWriter.println(PrintWriter.java:696)
com.delphix.appliance.server.ham.impl.HAMonitorServerThread.run(HAMonitorServerThread.java:56)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
java.lang.Thread.run(Thread.java:662)

With BTrace, you can trace nearly arbitrary information about a program's state, but instead of doing something like this:

dbtrace -p $(pgrep -n java) -n 'java.net.SocketOutputStream:write:entry{ printFields(this.impl); }'

You have to do this:

import com.sun.btrace.annotations.*;
import com.sun.btrace.AnyType;
import static com.sun.btrace.BTraceUtils.Reflective.*;
@BTrace
public class TraceWrite {
@OnMethod(clazz="java.net.SocketOutputStream", method="write", location=@Location(Kind.ENTRY))
public static void onWrite(@Self Object self) {
Object impl = get(field(classOf(self), "impl"), self);
printFields(impl);
}
}
$ ./bin/btrace $(pgrep -n java) TraceWrite.java
{server=null, port=1080, external_address=null, useV4=false, cmdsock=null, cmdIn=null, cmdOut=null, applicationSetProxy=false, timeout=0, trafficClass=0, shut_rd=false, shut_wr=false, socketInputStream=java.net.SocketInputStream@9993a1, fdUseCount=0, fdLock=java.lang.Object@ab5443, closePending=false, CONNECTION_NOT_RESET=0, CONNECTION_RESET_PENDING=1, CONNECTION_RESET=2, resetState=0, resetLock=java.lang.Object@292936, fd1=null, anyLocalBoundAddr=null, lastfd=-1, stream=false, socket=Socket[addr=/127.0.0.1,port=38832,localport=8765], serverSocket=null, fd=java.io.FileDescriptor@50abcc, address=/127.0.0.1, port=38832, localport=8765, }

BTrace needs a language that enables rapid iteration — piggybacking on Java is holding it back — and it needs some hard safety guarantees. With those, many developers and support engineers would use BTrace as part of their daily work — we certainly would here at Delphix.

Back to DTrace. Even with a useable solution for Java only, the ability to have lightweight and focused tracing for Java (and other dynamic languages) could be highly valuable. We’ll see how far BTrace can take us.

  
Posted on April 24, 2012 at 7:29 am by ahl · Permalink 
In: DTrace · Tagged with: BTraceDTraceJava

BTrace: DTrace for Java的更多相关文章

  1. BTrace: DTrace for Java2

    BTrace: DTrace for Java… ish 时间 2012-04-24 16:17:55  dtrace.org 原文  http://dtrace.org/blogs/ahl/2012 ...

  2. 基于BTrace监控调试Java代码

    BTrace是Java的一个动态代码追踪工具,通过编写btrace脚本,它可以动态的向目标应用程序的字节码注入追踪代码,通过修改字节码的方式,达到监控调试和定位问题的目的,是解决线上问题的利器. BT ...

  3. btrace拓展工具-java应用性能诊断优化利器

    Btrace是一个实时监控工具,可以无需修改应用代码(事实上它修改了字节码),来达到不可告人的秘密!这是性能调优和诊断的利器! 它可以获取应用程序代码的执行时间,他可以让你无需修改代码,帮你做时间的打 ...

  4. End-to-End Tracing of Ajax/Java Applications Using DTrace

    End-to-End Tracing of Ajax/Java Applications Using DTrace         By Amit Hurvitz, July 2007     Aja ...

  5. BTrace : Java 线上问题排查神器

    BTrace 是什么 BTrace 是检查和解决线上的问题的杀器,BTrace 可以通过编写脚本的方式,获取程序执行过程中的一切信息,并且,注意了,不用重启服务,是的,不用重启服务.写好脚本,直接用命 ...

  6. Java软件生产监控工具Btrace的使用

    Btrace BTrace是sun公司推出的一款Java 动态.安全追踪(监控)工具,可以在不用重启的情况下监控系统运行情况,方便的获取程序运行时的数据信息,如方法参数.返回值.全局变量和堆栈信息等, ...

  7. 认识一下java神器Btrace

    转载: http://calvin1978.blogcn.com/articles/btrace1.html BTrace是神器,每一个需要每天解决线上问题,但完全不用BTrace的Java工程师,都 ...

  8. BTrace学习总结

    一.简介: 在生产环境中经常遇到格式各样的问题,如OOM或者莫名其妙的进程死掉.一般情况下是通过修改程序,添加打印日志:然后重新发布程序来完成.然而,这不仅麻烦,而且带来很多不可控的因素.有没有一种方 ...

  9. btrace使用

    btrace使用 目录btracee是btrace的解压目录 btrace/btrace是btrace的项目工程 root@ubuntu:/usr/local/bogon/btrace# tree b ...

随机推荐

  1. Python的数据类型总结

    原地可变类型和不可变类型 原地不可变类型又叫可哈希(hashable)类型,原地可变类型又叫不可哈希类型. 原地不可变类型: 数字类型:int, float, decimal.Decimal, fra ...

  2. 守望先锋overwatch美服外服设置方法

    打开:C:\Users\你的用户名\AppData\Roaming\Battle.net\Battle.net.config 替换为下方内容: { "Client": { &quo ...

  3. python中的__init__ 、__new__、__call__等内置函数的剖析

    1.__new__(cls, *args, **kwargs)   创建对象时调用,返回当前对象的一个实例;注意:这里的第一个参数是cls即class本身2.__init__(self, *args, ...

  4. IAR中 C语言位定义

     __IO_REG8_BIT( SYS,             0xFFFFF802, __READ_WRITE ) #define __IO_REG8_BIT(NAME, ADDRESS, A ...

  5. NAT(NAPT)地址转换过程

    整理自NAT地址转换过程 注:本文实质讲的是NAPT(Network Address Port Translation),即网络端口地址转换.NAPT与动态地址NAT不同,它将内部连接映射到外部网络中 ...

  6. 【BZOJ1901】 Zju2112 Dynamic Rankings(树套树)

    [题意] 给定一个含有n个数的序列a[1],a[2],a[3]--a[n], 程序必须回答这样的询问:对于给定的i,j,k,在a[i],a[i+1],a[i+2]--a[j]中第k小的数是多少(1≤k ...

  7. [置顶] Objective-C,/,ios,/iphone开发基础:分类(category,又称类别)

    在c++中我们可以多继承来实现代码复用和封装使程序更加简练.在objective-c中只能单继承,不能多继承,那么除了协议protocol之外,我们可以实现类似多继承的一个方法就是,分类(catego ...

  8. bzoj1444

    显然自动机上高斯消元根据AC自动机上的转移可以列出一系列方程但这个样的方程解出来全0是一组解说明有线性组合的情况考虑除非没人能赢,否则每个人赢的概率和为1那么我们只要把原来的第一个方程换成这个即可 . ...

  9. UIImagePickerController 如何显示中文界面

    1: 添加中文的 Localizations Project --> Info --> Localizations  添加 "Chinese(Simplified)" ...

  10. Jquery UI dialog 详解 (中文)

    转载▼ 1 属性 1.11 autoOpen ,这个属性为true的时候dialog被调用的时候自动打开dialog窗口.当属性为false的时候,一开始隐藏窗口,知道.dialog("op ...