转自:http://blog.csdn.net/ruingman/article/details/53118202

 

定义

主线程在特定的时间内没有做完特定的事情

常见的场景

  1. A.input事件超过5S没有处理完成
  2. B.service executing 超时(bindcreatestartunbind等等),前台20s,后台200s
  3. C.广播处理超时,前台10S,后台60s
  4. D.ContentProvider执行超时,20s

常见的原因

  1. A.耗时操作,如复杂的layout,庞大的for循环,IO等。
  2. B.被Binder 对端block
  3. C.被子线程同步锁block
  4. D.Binder被占满导致主线程无法和SystemServer通信
  5. E.得不到系统资源(CPU/RAM/IO
  6. 其中ABCD比较好分析,而E比较困难。

如何分析?

指导思想:通过各种线索,寻求主线程在过去一段时间内发生block的可能原因。

  1. 线索包括:
  2. A.traces.txt/dropbox
  3. AMSANR发生的时候,dump相关进程(ANR的进程、systemservermediaserversurfaceFinger等)的当前栈到traces.txt
  4. traces.txt存在的几种形式:
  5. 1. adb pull /data/anr/
  6. 2. slog/2012xxxxx/misc/anr/snapshot_xxxx.log
  7. 3. slog/dropbox/data_app_anr_xxxxx.tgz
  8. 4. slog/dropbox/system_app_anr_xxxx.tgz
  9. 需要注意的是,traces.txt是抓取的是超时后(如input超时就是5s后)的snapshot,并不一定能够真实的反应出block的点。
  10. 也存在抓到主线程没有block,在idle的情况。
  11. B.Eventlog中的dvm_lock_sample.
  12. 在同步锁发生content的时候,虚拟机会将这个竞争记录在eventlog中:
  13. dvm_lock_sample: [system_server,1,Binder_7, 22, ActivityManagerService.java,15921,-,9628,4]
  14. 进程名 block线程 block时间(ms block的行号 持有者行号
  15. 实现可以参考art/runtime/monitor_android.cc LogContentionEvent函数
  16. 如果主线程是被binder对端、被同步锁block,那么eventlog中就很有可能会有dvm_lock_sample的打印。
  17. 华为项目上实现了更加强大的功能BlockMonitor,在主线程有耗时操作(如handlemsgBinder调用耗时)的时候会打印出栈。
  18. C.搜索主线程在发生ANR前后的main,systemlog,结合代码查看可能block在哪里。
  19. 现在发生ANR的时候,sprdruntimeinfomainlog中会Dump出比较多的信息,其中比较重要的是bindercpu
  20. D.ANR发生的时候,打印出的CPU的占用。
  21. 注意,并不能因为有进程cpu占用高就果断的去怀疑他。
  22. 因为最主要的线索traces.txt的有效性并不是非常高,所以ANR问题分析是存在一定的局限性的。

案例分析

1. Settings 被对端block

  1. 首先需要去看的就是traces.txt ANR进程的主线程的栈。
  2. settting的这次ANR为例:
  3. "main" prio=5 tid=1 Native
  4. | group="main" sCount=1 dsCount=0 obj=0x73ee6470 self=0xb4d76500
  5. | sysTid=22831 nice=0 cgrp=default sched=0/0 handle=0xb6f4bc00
  6. | state=S schedstat=( 0 0 0 ) utm=22 stm=22 core=0 HZ=100
  7. | stack=0xbe283000-0xbe285000 stackSize=8MB
  8. | held mutexes=
  9. native: #00 pc 000410ac /system/lib/libc.so (__ioctl+8)
  10. native: #01 pc 000477e5 /system/lib/libc.so (ioctl+14)
  11. native: #02 pc 0001e7c5 /system/lib/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+132)
  12. native: #03 pc 0001ee17 /system/lib/libbinder.so (android::IPCThreadState::waitForResponse(android::Parcel*, int*)+38)
  13. native: #04 pc 0001efcd /system/lib/libbinder.so (android::IPCThreadState::transact(int, unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+124)
  14. native: #05 pc 00019fb7 /system/lib/libbinder.so (android::BpBinder::transact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+30)
  15. native: #06 pc 00086de9 /system/lib/libandroid_runtime.so (???)
  16. native: #07 pc 00d94629 /data/dalvik-cache/arm/system@framework@boot.oat (Java_android_os_BinderProxy_transactNative__ILandroid_os_Parcel_2Landroid_os_Parcel_2I+140)
  17. at android.os.BinderProxy.transactNative(Native method)
  18. at android.os.BinderProxy.transact(Binder.java:503)
  19. at android.net.INetworkPolicyManager$Stub$Proxy.getNetworkPolicies(INetworkPolicyManager.java:410)
  20. at android.net.NetworkPolicyManager.getNetworkPolicies(NetworkPolicyManager.java:174)
  21. at com.android.settings.net.NetworkPolicyEditor.read(NetworkPolicyEditor.java:57)
  22. at com.android.settings.DataUsageSummary.onCreate(DataUsageSummary.java:361)
  23. at android.app.Fragment.performCreate(Fragment.java:2202)
  24. at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:942)
  25. at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1148)
  26. at android.app.BackStackRecord.run(BackStackRecord.java:793)
  27. at android.app.FragmentManagerImpl.execPendingActions(FragmentManager.java:1535)
  28. at android.app.FragmentManagerImpl.executePendingTransactions(FragmentManager.java:562)
  29. at com.android.settings.SettingsActivity.switchToFragment(SettingsActivity.java:1084)
  30. at com.android.settings.SettingsActivity.onCreate(SettingsActivity.java:657)
  31. at android.app.Activity.performCreate(Activity.java:6251)
  32. at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1110)
  33. at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2370)
  34. at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2477)
  35. at android.app.ActivityThread.-wrap11(ActivityThread.java:-1)
  36. at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1345)
  37. at android.os.Handler.dispatchMessage(Handler.java:102)
  38. at android.os.Looper.loop(Looper.java:148)
  39. at android.app.ActivityThread.main(ActivityThread.java:5438)
  40. at java.lang.reflect.Method.invoke!(Native method)
  41. at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:762)
  42. at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:652)
  43. 主线程被对端block,而对端是在systemserver中的NetworkPolicyManager
  44. 接下来继续去查NetworkPolicyManager为何会被block
  45. 通常情况下,可以搜索getNetworkPolicies,一般的对端的函数并不会修改函数名:
  46. 看到我们的对端:
  47. "Binder_4" prio=5 tid=56 Blocked
  48. | group="main" sCount=1 dsCount=0 obj=0x1321a0a0 self=0xad31e200
  49. | sysTid=2491 nice=0 cgrp=default sched=0/0 handle=0x9fd00930
  50. | state=S schedstat=( 0 0 0 ) utm=46441 stm=46582 core=1 HZ=100
  51. | stack=0x9fc04000-0x9fc06000 stackSize=1014KB
  52. | held mutexes=
  53. at com.android.server.net.NetworkPolicyManagerService.getNetworkPolicies(NetworkPolicyManagerService.java:1696)
  54. - waiting to lock <0x07439315> (a java.lang.Object) held by thread 35
  55. at android.net.INetworkPolicyManager$Stub.onTransact(INetworkPolicyManager.java:145)
  56. at android.os.Binder.execTransact(Binder.java:453
  57. Tid=35的人拿住了一把锁(0x07439315),那么继续看tid=35是谁,有两种方法:
  58. 1. 搜索tid=35
  59. 2. 搜索0x07439315,找到 - locked <0x07439315> (a java.lang.Object)
  60. "NetworkPolicy" prio=5 tid=35 TimedWaiting
  61. | group="main" sCount=1 dsCount=0 obj=0x12d98940 self=0x9f91f700
  62. | sysTid=2415 nice=0 cgrp=default sched=0/0 handle=0xa0f33930
  63. | state=S schedstat=( 0 0 0 ) utm=7681 stm=7783 core=0 HZ=100
  64. | stack=0xa0e31000-0xa0e33000 stackSize=1038KB
  65. | held mutexes=
  66. at java.lang.Object.wait!(Native method)
  67. - waiting on <0x02580c1b> (a java.lang.Object)
  68. at java.lang.Thread.parkFor$(Thread.java:1220)
  69. - locked <0x02580c1b> (a java.lang.Object)
  70. at sun.misc.Unsafe.park(Unsafe.java:299)
  71. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  72. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2053)
  73. at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:372)
  74. at com.android.server.NativeDaemonConnector$ResponseQueue.remove(NativeDaemonConnector.java:634)
  75. at com.android.server.NativeDaemonConnector.executeForList(NativeDaemonConnector.java:426)
  76. at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:345)
  77. at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:340)
  78. at com.android.server.NetworkManagementService.setInterfaceQuota(NetworkManagementService.java:1712)
  79. - locked <0x0b0f91b8> (a java.lang.Object)
  80. at com.android.server.net.NetworkPolicyManagerService.setInterfaceQuota(NetworkPolicyManagerService.java:2421)
  81. at com.android.server.net.NetworkPolicyManagerService.updateNetworkRulesLocked(NetworkPolicyManagerService.java:1232)
  82. at com.android.server.net.NetworkPolicyManagerService$14.onReceive(NetworkPolicyManagerService.java:1060)
  83. - locked <0x07439315> (a java.lang.Object)
  84. at android.app.LoadedApk$ReceiverDispatcher$Args.run(LoadedApk.java:881)
  85. at android.os.Handler.handleCallback(Handler.java:739)
  86. at android.os.Handler.dispatchMessage(Handler.java:95)
  87. at android.os.Looper.loop(Looper.java:148)
  88. at android.os.HandlerThread.run(HandlerThread.java:61)
  89. 可以看到,NetworkPolicy在通过NativeDaemonConnectornetd通信(setInterfaceQuota
  90. 我们结合log来看下是否有有用信息,按照之前的经验,netd在执行完cmd的时候,会打印出slow operation
  91. 在发生ANR的前后,查找netd相关的打印:
  92. 06-19 15:29:00.997 1235 1270 I am_anr : [0,22831,com.android.settings,818429509,Input dispatching timed out (Waiting because no window has focus but there is a focused application that may eventually add a window when it finishes starting up.)]
  93. 06-19 15:29:05.683 1235 2415 E NetdConnector: NDC Command {55445 bandwidth setiquota seth_w0 9223372036854775807} took too long (4755ms
  94. 06-19 15:29:05.723 1235 2491 I dvm_lock_sample: [system_server,1,Binder_4,4919,NetworkPolicyManagerService.java,1696,-,1056,100]
  95. eventslog中还可以发现,setting出现了两次连续的ANR,而上下文中都有类似上面的dvm_lock_sample NetworkPolicyManagerService相关的告警。
  96. 因此虽然15:28的这次ANR并没有打出有用的栈,但是我们还是可以猜测出这两次ANR的原因都是netdcmd耗时太久导致的(在主线程的调用路径上存在必然的block
  97. 那么,在netd可能无法修改的情况下,我们应该如何去resolve这个问题呢。
  98. 将可能存在block的操作放到非UI线程中去做。

2.com.huawei.hwvplayer.youku对端block:

  1. 打开dropbox中对应的system_app_anrxxxx:
  2. 查看主线程的栈:
  3. "main" prio=5 tid=1 Native
  4. | group="main" sCount=1 dsCount=0 obj=0x752b0000 self=0xb4276500
  5. | sysTid=25390 nice=-1 cgrp=default sched=3/0 handle=0xb6f18b34
  6. | state=S schedstat=( 0 0 0 ) utm=81 stm=12 core=2 HZ=100
  7. | stack=0xbe78b000-0xbe78d000 stackSize=8MB
  8. | held mutexes=
  9. kernel: (couldn't read /proc/self/task/25390/stack)
  10. native: #00 pc 000422d0 /system/lib/libc.so (__ioctl+8)
  11. native: #01 pc 00047825 /system/lib/libc.so (ioctl+14)
  12. native: #02 pc 0001e835 /system/lib/libbinder.so (_ZN7android14IPCThreadState14talkWithDriverEb+132)
  13. native: #03 pc 0001ee93 /system/lib/libbinder.so (_ZN7android14IPCThreadState15waitForResponseEPNS_6ParcelEPi+38)
  14. native: #04 pc 0001f049 /system/lib/libbinder.so (_ZN7android14IPCThreadState8transactEijRKNS_6ParcelEPS1_j+124)
  15. native: #05 pc 00019fe3 /system/lib/libbinder.so (_ZN7android8BpBinder8transactEjRKNS_6ParcelEPS1_j+30)
  16. native: #06 pc 0008a035 /system/lib/libandroid_runtime.so (???)
  17. native: #07 pc 00d78869 /data/dalvik-cache/arm/system@framework@boot.oat (Java_android_os_BinderProxy_transactNative__ILandroid_os_Parcel_2Landroid_os_Parcel_2I+140)
  18. at android.os.BinderProxy.transactNative(Native method)
  19. at android.os.BinderProxy.transact(Binder.java:510)
  20. at android.os.storage.IMountService$Stub$Proxy.getVolumeList(IMountService.java:771)
  21. at android.os.storage.StorageManager.getVolumeList(StorageManager.java:883)
  22. at android.os.Environment$UserEnvironment.getExternalDirs(Environment.java:95)
  23. at android.os.Environment.getExternalStorageDirectory(Environment.java:354)
  24. at com.huawei.common.utils.PathUtils.<clinit>(PathUtils.java:51)
  25. at com.huawei.common.utils.PathUtils.getWorkspacePath(PathUtils.java:80)
  26. at com.huawei.common.components.log.Logger.<clinit>(Logger.java:37)
  27. at com.huawei.common.components.log.Logger.i(Logger.java:162)
  28. at com.huawei.hwvplayer.data.db.DbProvider.attachInfo(DbProvider.java:89)
  29. at android.app.ActivityThread.installProvider(ActivityThread.java:5279)
  30. at android.app.ActivityThread.installContentProviders(ActivityThread.java:4868)
  31. at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4799)
  32. at android.app.ActivityThread.access$1600(ActivityThread.java:165)
  33. at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1436)
  34. at android.os.Handler.dispatchMessage(Handler.java:102)
  35. at android.os.Looper.loop(Looper.java:188)
  36. at android.app.ActivityThread.main(ActivityThread.java:5578)
  37. at java.lang.reflect.Method.invoke!(Native method)
  38. at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:794)
  39. at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:684)
  40. 主线程尝试去调用MountService的 getVolumeList接口,可能没有返回。
  41. 查看system_server中相关的,搜索getVolumeList
  42. 发现systemserver中有三个Binder线程和主线程被block,我们的对端是哪个Binder线程暂时无法确认,但这并不影响我们继续分析,因为他们被blockd的路径是一致的:
  43. "main" prio=5 tid=1 Blocked
  44. | group="main" sCount=1 dsCount=0 obj=0x752b0000 self=0xb4276500
  45. | sysTid=22735 nice=-2 cgrp=default sched=0/0 handle=0xb6f18b34
  46. | state=S schedstat=( 0 0 0 ) utm=432 stm=85 core=1 HZ=100
  47. | stack=0xbe78b000-0xbe78d000 stackSize=8MB
  48. | held mutexes=
  49. at com.android.server.MountService.getVolumeList(MountService.java:2759)
  50. - waiting to lock <0x0eeb54f1> (a java.lang.Object) held by thread 40
  51. at android.os.storage.StorageManager.getVolumeList(StorageManager.java:883)
  52. at android.os.storage.StorageManager.getVolumeList(StorageManager.java:858)
  53. at android.os.storage.StorageManager.getPrimaryVolume(StorageManager.java:906)
  54. at com.android.server.usb.UsbDeviceManager.systemReady(UsbDeviceManager.java:327)
  55. at com.android.server.usb.UsbService.systemReady(UsbService.java:181)
  56. at com.android.server.usb.UsbService$Lifecycle.onBootPhase(UsbService.java:78)
  57. at com.android.server.SystemServiceManager.startBootPhase(SystemServiceManager.java:135)
  58. at com.android.server.SystemServer$3.run(SystemServer.java:1489)
  59. at com.android.server.am.ActivityManagerService.systemReady(ActivityManagerService.java:12417)
  60. at com.android.server.am.HwActivityManagerService.systemReady(HwActivityManagerService.java:960)
  61. at com.android.server.SystemServer.startOtherServices(SystemServer.java:1485)
  62. at com.android.server.SystemServer.run(SystemServer.java:381)
  63. at com.android.server.SystemServer.main(SystemServer.java:272)
  64. at java.lang.reflect.Method.invoke!(Native method)
  65. at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:794)
  66. at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:684)
  67. "Binder_8" prio=5 tid=76 Blocked
  68. | group="main" sCount=1 dsCount=0 obj=0x13bd60a0 self=0x9c1abe00
  69. | sysTid=25191 nice=-1 cgrp=default sched=0/0 handle=0x97158930
  70. | state=S schedstat=( 0 0 0 ) utm=9 stm=7 core=2 HZ=100
  71. | stack=0x9705c000-0x9705e000 stackSize=1014KB
  72. | held mutexes=
  73. at com.android.server.MountService.getVolumeList(MountService.java:2759)
  74. - waiting to lock <0x0eeb54f1> (a java.lang.Object) held by thread 40
  75. at android.os.storage.IMountService$Stub.onTransact(IMountService.java:1634)
  76. at android.os.Binder.execTransact(Binder.java:453)
  77. "Binder_2" prio=5 tid=8 Blocked
  78. | group="main" sCount=1 dsCount=0 obj=0x12cac0a0 self=0xaebf0300
  79. | sysTid=22761 nice=-1 cgrp=default sched=0/0 handle=0xaef7d930
  80. | state=S schedstat=( 0 0 0 ) utm=41 stm=25 core=0 HZ=100
  81. | stack=0xaee81000-0xaee83000 stackSize=1014KB
  82. | held mutexes=
  83. at com.android.server.MountService.getVolumeList(MountService.java:2759)
  84. - waiting to lock <0x0eeb54f1> (a java.lang.Object) held by thread 40
  85. at android.os.storage.IMountService$Stub.onTransact(IMountService.java:1634)
  86. at android.os.Binder.execTransact(Binder.java:453)
  87. 他们均是被tid=40的人block,按照上面的方法搜索tid=40或者 0x0eeb54f1得到block的人:
  88. "MountService" prio=5 tid=40 TimedWaiting
  89. | group="main" sCount=1 dsCount=0 obj=0x132c1160 self=0x9ce57400
  90. | sysTid=23512 nice=0 cgrp=default sched=0/0 handle=0x9a239930
  91. | state=S schedstat=( 0 0 0 ) utm=29 stm=2 core=2 HZ=100
  92. | stack=0x9a137000-0x9a139000 stackSize=1038KB
  93. | held mutexes=
  94. at java.lang.Object.wait!(Native method)
  95. - waiting on <0x00fea1f3> (a java.lang.Object)
  96. at java.lang.Thread.parkFor$(Thread.java:1235)
  97. - locked <0x00fea1f3> (a java.lang.Object)
  98. at sun.misc.Unsafe.park(Unsafe.java:299)
  99. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  100. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2053)
  101. at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:372)
  102. at com.android.server.NativeDaemonConnector$ResponseQueue.remove(NativeDaemonConnector.java:777)
  103. at com.android.server.NativeDaemonConnector.executeForList(NativeDaemonConnector.java:489)
  104. at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:386)
  105. at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:381)
  106. at com.android.server.MountService.resetIfReadyAndConnectedLocked(MountService.java:827)
  107. at com.android.server.MountService.handleSystemReady(MountService.java:776)
  108. - locked <0x0eeb54f1> (a java.lang.Object)
  109. at com.android.server.MountService.access$500(MountService.java:152)
  110. at com.android.server.MountService$MountServiceHandler.handleMessage(MountService.java:596)
  111. at android.os.Handler.dispatchMessage(Handler.java:102)
  112. at android.os.Looper.loop(Looper.java:150)
  113. at android.os.HandlerThread.run(HandlerThread.java:61)
  114. 和上面netd类似的,mountservice也是通过ndc和vold通信,这里我们需要继续查看是否vold存在异常。
  115. 之前提到过,类似这种同步锁block的,dvm_lock_sample一定会有打印,于是先去找eventslog,不过这个是华为的log,是没有eventlog的。
  116. 而华为实现了一个blockMonitor的功能,和dvm_lock_sample类似,当某个操作特别耗时的时候,会将其打印出来:
  117. 在ANR的附近找到如下:
  118. 07-19 10:17:50.739 25271 25271 W BlockMonitor: The binder calling took 55209ms.
  119. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.BlockMonitor.checkBinderTime(BlockMonitor.java:141)
  120. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.BinderProxy.transact(Binder.java:511)
  121. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.storage.IMountService$Stub$Proxy.getVolumeList(IMountService.java:771)
  122. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.storage.StorageManager.getVolumeList(StorageManager.java:883)
  123. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.Environment$UserEnvironment.getExternalDirs(Environment.java:95)
  124. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.Environment.getExternalStorageDirectory(Environment.java:354)
  125. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.support.v4.content.FileProvider.parsePathStrategy(FileProvider.java:583)
  126. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.support.v4.content.FileProvider.getPathStrategy(FileProvider.java:534)
  127. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.support.v4.content.FileProvider.attachInfo(FileProvider.java:352)
  128. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread.installProvider(ActivityThread.java:5279)
  129. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread.installContentProviders(ActivityThread.java:4868)
  130. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread.handleBindApplication(ActivityThread.java:4799)
  131. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread.access$1600(ActivityThread.java:165)
  132. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread$H.handleMessage(ActivityThread.java:1436)
  133. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.Handler.dispatchMessage(Handler.java:102)
  134. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.os.Looper.loop(Looper.java:188)
  135. 07-19 10:17:50.739 25271 25271 W BlockMonitor: android.app.ActivityThread.main(ActivityThread.java:5578)
  136. 07-19 10:17:50.739 25271 25271 W BlockMonitor: java.lang.reflect.Method.invoke(Native Method)
  137. 07-19 10:17:50.739 25271 25271 W BlockMonitor: com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:794)
  138. 07-19 10:17:50.739 25271 25271 W BlockMonitor: com.android.internal.os.ZygoteInit.main(ZygoteInit.java:684)
  139. 再加上之前的:
  140. 07-19 10:17:50.729 22735 23512 E NativeDaemonConnector.ResponseQueue: Timeout waiting for response
  141. 07-19 10:17:50.729 22735 23512 E VoldConnector: timed-out waiting for response to 4 volume reset
  142. 07-19 10:17:50.729 22735 23512 E VoldConnector: timed-out waiting for response mOutputStream = android.net.LocalSocketImpl$SocketOutputStream@922644, mSocket = vold
  143. 07-19 10:17:50.731 22735 23512 W MountService: Failed to reset vold
  144. 07-19 10:17:50.731 22735 23512 W MountService: com.android.server.NativeDaemonTimeoutException: command '4 volume reset' failed with 'null'
  145. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.NativeDaemonConnector.executeForList(NativeDaemonConnector.java:501)
  146. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:386)
  147. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.NativeDaemonConnector.execute(NativeDaemonConnector.java:381)
  148. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.MountService.resetIfReadyAndConnectedLocked(MountService.java:827)
  149. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.MountService.handleSystemReady(MountService.java:776)
  150. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.MountService.access$500(MountService.java:152)
  151. 07-19 10:17:50.731 22735 23512 W MountService: at com.android.server.MountService$MountServiceHandler.handleMessage(MountService.java:596)
  152. 07-19 10:17:50.731 22735 23512 W MountService: at android.os.Handler.dispatchMessage(Handler.java:102)
  153. 07-19 10:17:50.731 22735 23512 W MountService: at android.os.Looper.loop(Looper.java:150)
  154. 07-19 10:17:50.731 22735 23512 W MountService: at android.os.HandlerThread.run(HandlerThread.java:61)
  155. 我们有理由去推断vold的状态是不对的,但是又没有vold的栈。
  156. 那么全局grep vold试试能不能找到线索,在kernellog中发现了vold的异常:
  157. 07-19 14:22:24.669 <3>[10772.492156] c0 Freezing of tasks failed after 20.008 seconds (1 tasks refusing to freeze, wq_busy=0):
  158. 07-19 14:22:24.669 <6>[10772.492217] c0 vold R running 0 224 1 0x00000001
  159. 07-19 14:22:24.669 <4>[10772.492278] c0 [<c05ebecc>] (__schedule+0x38c/0x5bc) from [<c05ea478>] (schedule_timeout+0x18/0x1e8)
  160. 07-19 14:22:24.669 <4>[10772.492309] c0 [<c05ea478>] (schedule_timeout+0x18/0x1e8) from [<c05eb90c>] (wait_for_common+0x11c/0x164)
  161. 07-19 14:22:24.669 <4>[10772.492309] c0 [<c05eb90c>] (wait_for_common+0x11c/0x164) from [<c03cd8c8>] (mmc_wait_for_req+0xb4/0xe4)
  162. 07-19 14:22:24.669 <4>[10772.492339] c0 [<c03cd8c8>] (mmc_wait_for_req+0xb4/0xe4) from [<c03cd95c>] (mmc_wait_for_cmd+0x64/0x74)
  163. 07-19 14:22:24.669 <4>[10772.492370] c0 [<c03cd95c>] (mmc_wait_for_cmd+0x64/0x74) from [<c03d41f0>] (mmc_send_status+0x6c/0x8c)
  164. 07-19 14:22:24.670 <4>[10772.492400] c0 [<c03d41f0>] (mmc_send_status+0x6c/0x8c) from [<c03d4504>] (sd_send_status+0x14/0x44)
  165. 07-19 14:22:24.670 <4>[10772.492431] c0 [<c03d4504>] (sd_send_status+0x14/0x44) from [<c03d491c>] (mmc_lock_unlock_by_buf+0xac/0x168)
  166. 07-19 14:22:24.670 <4>[10772.492431] c0 [<c03d491c>] (mmc_lock_unlock_by_buf+0xac/0x168) from [<c03dabd8>] (mmc_lockable_store+0x594/0x75c)
  167. 07-19 14:22:24.670 <4>[10772.492461] c0 [<c03dabd8>] (mmc_lockable_store+0x594/0x75c) from [<c029d560>] (dev_attr_store+0x18/0x24)
  168. 07-19 14:22:32.070 <4>[10772.492492] c0 [<c029d560>] (dev_attr_store+0x18/0x24) from [<c013b370>] (sysfs_write_file+0x104/0x148)
  169. 07-19 14:22:32.070 <4>[10772.492522] c0 [<c013b370>] (sysfs_write_file+0x104/0x148) from [<c00eabb4>] (vfs_write+0xd0/0x180)
  170. 07-19 14:22:32.070 <4>[10772.492553] c0 [<c00eabb4>] (vfs_write+0xd0/0x180) from [<c00eb070>] (SyS_write+0x38/0x68)
  171. 07-19 14:22:32.071 <4>[10772.492583] c0 [<c00eb070>] (SyS_write+0x38/0x68) from [<c000e840>] (ret_fast_syscall+0x0/0x30)
  172. vold一直在这个操作中没有退出来,所以不能响应客户端的请求,从而导致了ANR。
  173. 这个问题需要mmc的同事进一步去分析,目前怀疑是SD卡发生了错误。

3. setting 主线程耗时操作

  1. "main" prio=5 tid=1 Native
  2. | group="main" sCount=1 dsCount=0 obj=0x7425caf8 self=0xb4827800
  3. | sysTid=671 nice=0 cgrp=default sched=0/0 handle=0xb6febbec
  4. | state=S schedstat=( 0 0 0 ) utm=1853 stm=50 core=2 HZ=100
  5. | stack=0xbe7bd000-0xbe7bf000 stackSize=8MB
  6. | held mutexes=
  7. kernel: (couldn't read /proc/self/task/671/stack)
  8. native: #00 pc 000133cc /system/lib/libc.so (syscall+28)
  9. native: #01 pc 000a9a83 /system/lib/libart.so (art::ConditionVariable::Wait(art::Thread*)+82)
  10. native: #02 pc 001b16f1 /system/lib/libart.so (art::JNI::NewString(_JNIEnv*, unsigned short const*, int)+640)
  11. native: #03 pc 00075887 /system/lib/libandroid_runtime.so (???)
  12. native: #04 pc 008570ab /data/dalvik-cache/arm/system@framework@boot.oat (Java_android_database_CursorWindow_nativeGetString__JII+110)
  13. at android.database.CursorWindow.nativeGetString(Native method)
  14. at android.database.CursorWindow.getString(CursorWindow.java:438)
  15. at android.database.AbstractWindowedCursor.getString(AbstractWindowedCursor.java:51)
  16. at android.database.CursorWrapper.getString(CursorWrapper.java:114)
  17. at com.android.settings.ApnSettings.fillList(ApnSettings.java:259)
  18. at com.android.settings.ApnSettings.onResume(ApnSettings.java:208)
  19. at android.app.Fragment.performResume(Fragment.java:2096)
  20. at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:928)
  21. at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1067)
  22. at android.app.FragmentManagerImpl.moveToState(FragmentManager.java:1049)
  23. at android.app.FragmentManagerImpl.dispatchResume(FragmentManager.java:1879)
  24. at android.app.Activity.performResume(Activity.java:6113)
  25. at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3015)
  26. at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3061)
  27. at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2415)
  28. at android.app.ActivityThread.access$800(ActivityThread.java:151)
  29. at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313)
  30. at android.os.Handler.dispatchMessage(Handler.java:102)
  31. at android.os.Looper.loop(Looper.java:135)
  32. at android.app.ActivityThread.main(ActivityThread.java:5345)
  33. at java.lang.reflect.Method.invoke!(Native method)
  34. at java.lang.reflect.Method.invoke(Method.java:372)
  35. at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:947)
  36. at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:742)
  37. 从ApnSettings.java中可以看到fillList这个可能的耗时操作主线程处理的地方有OnResume和OnReceive。
  38. 且代码中存在使用HandlerThread异步处理这个操作的地方,可以借鉴下将这两处都放到HandlerThread中去做。

4. 系统资源紧张造成的ANR

  1. 通常情况下,一般这种问题的表现的可能形式是
  2. 1. traces.txt中的主线程的栈在一个非常common的操作中,如new一个变量,读取某个文件等
  3. 2. ANRtraceIOW相当高(IOW+CPU sys+usr=100
  4. 3. Kswapd非常活跃
  5. 4. Low Ram 配置
  6. 5. kernel log D状态的用户进程比较多,且都block在内存相关,且内存比较紧张.
  7. 6. 连续出现ANRslogams一直在dumptrace,造成比较大的IO压力
  8. 7. 前台正在玩一个很大的游戏
  9. 8. 某个应用正在安装做dex2oat.
  10. 出现这种情况下,我们需要做的是:
  11. 1. 针对性的调整lmk参数,减少后台的数量
  12. 2. 检查是否有不应该常驻的进程
  13. 3. 是否需要修改伪前台的adj,减少常驻内存
  14. 4. 是否需要关闭dex2oat
  15. 此类问题比较难解决。

5. Binder被占满导致ANR

  1. 1. traces.txt中主线程处于IDLE状态,要么是没有抓到现场,要么是出现了其他问题。
  2. 2. 分析log,找到ANR的时间点如下:后台服务执行超时,这个timeout200s
  3. 07-28 03:37:33.681 1719 1732 I am_anr : [0,15625,com.android.exchange,948452933,executing service com.android.exchange/.service.EasService]
  4. ----- pid 15625 at 2016-07-28 03:37:33 -----,dumptrace的时间点。
  5. 3. 查看是否有主线程的消息:
  6. 07-28 03:37:39.373 15625 15625 W BlockMonitor: Package name: com.android.exchange
  7. 07-28 03:37:39.373 15625 15625 W BlockMonitor: The Message{ what=115 obj=ServiceArgsData{token=android.os.BinderProxy@49814e2
  8. startId=6 args=Intent { act=com.android.email.EXCHANGE_INTENT flg=0x4 cmp=com.android.exchange/.service.EasService (has extras) }}
  9. target=android.app.ActivityThread$H } took 5174ms.
  10. 之前有说过,华为项目是没有dvmlocksample但是有blockMonitor的,如果有耗时的binder调用以及mesg处理,那么就会被他抓过来。
  11. 从代码中可以看到,what=115serviceArg消息,是startservice的一部分,而这个消息应该是34秒的时候才接受到并处理的。
  12. 那么,为什么33s的时候dumptrace并没有抓到主线程有卡顿?且后续并没有blockmonitor的打印,也就是说200s内主线程和systemserver并没有出现block
  13. 那么还有可能是什么原因导致了前200s并没有收到这个消息?
  14. 我们来看下这部分的代码:
  1. private final void sendServiceArgsLocked(ServiceRecord r, boolean execInFg,
  2. boolean oomAdjusted) throws TransactionTooLargeException {
  3. final int N = r.pendingStarts.size();
  4. if (N == 0) {
  5. return;
  6. }
  7. while (r.pendingStarts.size() > 0) {
  8. Exception caughtException = null;
  9. ServiceRecord.StartItem si;
  10. try {
  11. si = r.pendingStarts.remove(0);
  12. if (DEBUG_SERVICE) Slog.v(TAG_SERVICE, "Sending arguments to: "
  13. + r + " " + r.intent + " args=" + si.intent);
  14. if (si.intent == null && N > 1) {
  15. // If somehow we got a dummy null intent in the middle,
  16. // then skip it. DO NOT skip a null intent when it is
  17. // the only one in the list -- this is to support the
  18. // onStartCommand(null) case.
  19. continue;
  20. }
  21. si.deliveredTime = SystemClock.uptimeMillis();
  22. r.deliveredStarts.add(si);
  23. si.deliveryCount++;
  24. if (si.neededGrants != null) {
  25. mAm.grantUriPermissionUncheckedFromIntentLocked(si.neededGrants,
  26. si.getUriPermissionsLocked());
  27. }
  28. bumpServiceExecutingLocked(r, execInFg, "start"); //超时timer开始计算
  29. if (!oomAdjusted) {
  30. oomAdjusted = true;
  31. mAm.updateOomAdjLocked(r.app);
  32. }
  33. int flags = 0;
  34. if (si.deliveryCount > 1) {
  35. flags |= Service.START_FLAG_RETRY;
  36. }
  37. if (si.doneExecutingCount > 0) {
  38. flags |= Service.START_FLAG_REDELIVERY;
  39. }
  40. r.app.thread.scheduleServiceArgs(r, si.taskRemoved, si.id, flags, si.intent);//通过binder给App发送一个消息,ActivityThread会收到并处理。
  41. } catch (TransactionTooLargeException e) {
  42. if (DEBUG_SERVICE) Slog.v(TAG_SERVICE, "Transaction too large: intent="
  43. + si.intent);
  44. caughtException = e;
  45. } catch (RemoteException e) {
  46. // Remote process gone... we'll let the normal cleanup take care of this.
  47. if (DEBUG_SERVICE) Slog.v(TAG_SERVICE, "Crashed while sending args: " + r);
  48. caughtException = e;
  49. } catch (Exception e) {
  50. Slog.w(TAG, "Unexpected exception", e);
  51. caughtException = e;
  52. }
  53. if (caughtException != null) {
  54. // Keep nesting count correct
  55. final boolean inDestroying = mDestroyingServices.contains(r);
  56. serviceDoneExecutingLocked(r, inDestroying, inDestroying);
  57. if (caughtException instanceof TransactionTooLargeException) {
  58. throw (TransactionTooLargeException)caughtException;
  59. }
  60. break;
  61. }
  62. }
  63. }
  1. 从上面的代码可以看出,在启动计时器和给app发送消息之间并不存在耗时操作,且这个是在同步锁之内的,如果慢了一定会被blockmonitor抓出来。
  2. 那么问题的原因只有一种可能情形:
  3. 200s之前给app发送的消息,APP并没有及时的处理。
  4. 从经验可以猜测,一般这种情况都是appBinder被占满导致没有空闲的Binder线程能够处理这个消息导致。
  5. 而我们的traces正好dump到了这一幕:
  1. app所有的Binder线程都全部在做一件事情,一个依赖于网络的操作,是被另外一个人通过contentprovider.query调过来的,有点不太正常。
  2. "Binder_10" prio=5 tid=30 Native
  3. | group="main" sCount=1 dsCount=0 obj=0x22dba160 self=0xb2f26700
  4. | sysTid=4613 nice=10 cgrp=bg_non_interactive sched=0/0 handle=0x9eb80930
  5. | state=S schedstat=( 0 0 0 ) utm=0 stm=0 core=3 HZ=100
  6. | stack=0x9ea84000-0x9ea86000 stackSize=1014KB
  7. | held mutexes=
  8. kernel: (couldn't read /proc/self/task/4613/stack)
  9. native: #00 pc 000423e8 /system/lib/libc.so (__pselect6+20)
  10. native: #01 pc 0001c115 /system/lib/libc.so (select+60)
  11. native: #02 pc 0000a279 /system/lib/libjavacrypto.so (???)
  12. native: #03 pc 0000b29f /system/lib/libjavacrypto.so (???)
  13. native: #04 pc 004bc3fd /data/dalvik-cache/arm/system@framework@boot.oat (Java_com_android_org_conscrypt_NativeCrypto_SSL_1read__JLjava_io_FileDescriptor_2Lcom_android_org_conscrypt_NativeCrypto_00024SSLHandshakeCallbacks_2_3BIII+192)
  14. at com.android.org.conscrypt.NativeCrypto.SSL_read(Native method)
  15. at com.android.org.conscrypt.OpenSSLSocketImpl$SSLInputStream.read(OpenSSLSocketImpl.java:705)
  16. - locked <0x0508e00e> (a java.lang.Object)
  17. at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:108)
  18. at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:196)
  19. at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:88)
  20. at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:179)
  21. at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:185)
  22. at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:240)
  23. at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:264)
  24. at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:284)
  25. at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
  26. at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)
  27. at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:596)
  28. at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:517)
  29. at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:495)
  30. at com.android.exchange.EasResponse.fromHttpRequest(EasResponse.java:91)
  31. at com.android.exchange.service.EasServerConnection.executeHttpUriRequest(EasServerConnection.java:384)
  32. at com.android.exchange.eas.EasOperation.performOperation(EasOperation.java:382)
  33. at com.android.exchange.service.EasService.searchGal(EasService.java:706)
  34. at com.android.exchange.provider.ExchangeDirectoryProvider.query(ExchangeDirectoryProvider.java:350)
  35. at android.content.ContentProvider.query(ContentProvider.java:1017)
  36. at android.content.ContentProvider$Transport.query(ContentProvider.java:238)
  37. at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:112)
  38. at android.os.Binder.execTransact(Binder.java:453)
  1. sprdruntimeANR的时候会dump BinderState,搜索当前进程的pid,查看是否能够找到线索:
  2. 07-28 03:37:33.677 1719 1732 W SprdRuntimeInfo: proc 15625
  3. 07-28 03:37:33.677 1719 1732 W SprdRuntimeInfo: thread 4537: l 01
  4. 07-28 03:37:33.677 1719 1732 W SprdRuntimeInfo: incoming transaction 625339: ed9d1ec0 from 30228:4535 to 15625:4537 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb600d84
  5. 07-28 03:37:33.677 1719 1732 W SprdRuntimeInfo: thread 4538: l 01
  6. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 625344: c2c47080 from 30228:4536 to 15625:4538 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb601090
  7. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4563: l 01
  8. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 625759: e887e6c0 from 30228:4558 to 15625:4563 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb6013f4
  9. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4564: l 01
  10. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 625805: e6223a40 from 30228:4561 to 15625:4564 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601704
  11. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4578: l 01
  12. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626144: e96d0cc0 from 30228:4577 to 15625:4578 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601a70
  13. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4580: l 01
  14. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626162: d0a97440 from 30228:4579 to 15625:4580 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601db0
  15. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4591: l 01
  16. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626562: d34d60c0 from 30228:4588 to 15625:4591 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb6020ec
  17. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4592: l 01
  18. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626575: ecb23800 from 30228:4590 to 15625:4592 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602430
  19. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4602: l 01
  20. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626867: d3c64580 from 30228:4600 to 15625:4602 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602770
  21. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4603: l 01
  22. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 626883: e8b8bfc0 from 30228:4601 to 15625:4603 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602ab4
  23. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4612: l 01
  24. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 627170: e8b8bdc0 from 30228:4610 to 15625:4612 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602df4
  25. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 4613: l 01
  26. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 627176: e8ded5c0 from 30228:4611 to 15625:4613 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb603138
  27. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 15635: l 02
  28. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 623807: edbbfa40 from 30228:1612 to 15625:15635 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb600404
  29. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 15636: l 01
  30. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 624820: edc2bd80 from 30228:30240 to 15625:15636 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb600714
  31. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 15658: l 01
  32. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 624824: c2c47400 from 30228:30238 to 15625:15658 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb600a20
  33. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: thread 18343: l 01
  34. 07-28 03:37:33.678 1719 1732 W SprdRuntimeInfo: incoming transaction 623794: d5132380 from 30228:30241 to 15625:18343 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb6000f4
  35. 可以看到我们binder线程被30228给占满了,而同时我们也发现30228也有点不太对:
  36. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: proc 30228
  37. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 1612: l 11
  38. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 623807: edbbfa40 from 30228:1612 to 15625:15635 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb600404
  39. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 623738: e994e880 from 2332:4503 to 30228:1612 code 1 flags 10 pri 10 r1 node 538358 size 776:4 data f880009c
  40. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4535: l 11
  41. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 625339: ed9d1ec0 from 30228:4535 to 15625:4537 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb600d84
  42. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 625317: e82f4000 from 2332:4556 to 30228:4535 code 1 flags 10 pri 10 r1 node 538358 size 776:4 data f8800d6c
  43. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4536: l 11
  44. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 625344: c2c47080 from 30228:4536 to 15625:4538 code 1 flags 10 pri 10 r1 node 222269 size 736:4 data fb601090
  45. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 625326: ed9d1c40 from 2332:4557 to 30228:4536 code 1 flags 10 pri 10 r1 node 538358 size 776:4 data f88010a0
  46. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4558: l 11
  47. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 625759: e887e6c0 from 30228:4558 to 15625:4563 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb6013f4
  48. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 625733: e887e5c0 from 2332:4575 to 30228:4558 code 1 flags 10 pri 10 r1 node 538358 size 780:4 data f88013d4
  49. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4561: l 11
  50. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 625805: e6223a40 from 30228:4561 to 15625:4564 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601704
  51. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 625756: e8719880 from 2332:4576 to 30228:4561 code 1 flags 10 pri 10 r1 node 538358 size 780:4 data f880170c
  52. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4577: l 11
  53. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 626144: e96d0cc0 from 30228:4577 to 15625:4578 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601a70
  54. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 626125: d34d6ec0 from 2332:4585 to 30228:4577 code 1 flags 10 pri 10 r1 node 538358 size 780:4 data f8801a44
  55. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4579: l 11
  56. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 626162: d0a97440 from 30228:4579 to 15625:4580 code 1 flags 10 pri 10 r1 node 222269 size 740:4 data fb601db0
  57. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 626131: c3f98200 from 2332:4587 to 30228:4579 code 1 flags 10 pri 10 r1 node 538358 size 780:4 data f8801d7c
  58. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4588: l 11
  59. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 626562: d34d60c0 from 30228:4588 to 15625:4591 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb6020ec
  60. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: incoming transaction 626541: ede63540 from 2332:4598 to 30228:4588 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f88020b4
  61. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: thread 4590: l 11
  62. 07-28 03:37:33.562 1719 1732 W SprdRuntimeInfo: outgoing transaction 626575: ecb23800 from 30228:4590 to 15625:4592 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602430
  63. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: incoming transaction 626546: e9a32b80 from 2332:4599 to 30228:4590 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f88023f0
  64. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: thread 4600: l 11
  65. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: outgoing transaction 626867: d3c64580 from 30228:4600 to 15625:4602 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602770
  66. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: incoming transaction 626849: ecb23e00 from 2332:4608 to 30228:4600 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f880272c
  67. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: thread 4601: l 11
  68. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: outgoing transaction 626883: e8b8bfc0 from 30228:4601 to 15625:4603 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602ab4
  69. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: incoming transaction 626856: e8389f00 from 2332:4609 to 30228:4601 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f8802a68
  70. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: thread 4610: l 11
  71. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: outgoing transaction 627170: e8b8bdc0 from 30228:4610 to 15625:4612 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb602df4
  72. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: incoming transaction 627144: c27891c0 from 2332:4618 to 30228:4610 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f8802da4
  73. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: thread 4611: l 11
  74. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: outgoing transaction 627176: e8ded5c0 from 30228:4611 to 15625:4613 code 1 flags 10 pri 10 r1 node 222269 size 744:4 data fb603138
  75. 07-28 03:37:33.563 1719 1732 W SprdRuntimeInfo: incoming transaction 627151: e2b511c0 from 2332:4619 to 30228:4611 code 1 flags 10 pri 10 r1 node 538358 size 784:4 data f88030e0
  76. 也就是说实际上是2332query3022830228才会去query16625导致anr的。
  77. 那么2332是谁?
  78. eventslog中可以看到2332email30228android.process.acore
  79. 搜索traces.txt,看到我们在anr的时候有dump2332的栈:
  80. 发现了异常,有很多个线程(也就是上面dump到的2332的线程)在查询,
  81. "Filter" prio=5 tid=42 Native
  82. | group="main" sCount=1 dsCount=0 obj=0x130cc8e0 self=0x98637500
  83. | sysTid=4502 nice=10 cgrp=bg_non_interactive sched=0/0 handle=0x9877e930
  84. | state=S schedstat=( 0 0 0 ) utm=1 stm=0 core=0 HZ=100
  85. | stack=0x9867c000-0x9867e000 stackSize=1038KB
  86. | held mutexes=
  87. kernel: (couldn't read /proc/self/task/4502/stack)
  88. native: #00 pc 00017618 /system/lib/libc.so (syscall+28)
  89. native: #01 pc 000f5cbd /system/lib/libart.so (_ZN3art17ConditionVariable4WaitEPNS_6ThreadE+80)
  90. native: #02 pc 00275df1 /system/lib/libart.so (_ZN3art3JNI18CallBooleanMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+592)
  91. native: #03 pc 0008617b /system/lib/libandroid_runtime.so (???)
  92. native: #04 pc 00089c2b /system/lib/libandroid_runtime.so (???)
  93. native: #05 pc 000198dd /system/lib/libbinder.so (_ZN7android7BBinder8transactEjRKNS_6ParcelEPS1_j+60)
  94. native: #06 pc 0001ec25 /system/lib/libbinder.so (_ZN7android14IPCThreadState14executeCommandEi+584)
  95. native: #07 pc 0001ef73 /system/lib/libbinder.so (_ZN7android14IPCThreadState15waitForResponseEPNS_6ParcelEPi+262)
  96. native: #08 pc 0001f049 /system/lib/libbinder.so (_ZN7android14IPCThreadState8transactEijRKNS_6ParcelEPS1_j+124)
  97. native: #09 pc 00019fe3 /system/lib/libbinder.so (_ZN7android8BpBinder8transactEjRKNS_6ParcelEPS1_j+30)
  98. native: #10 pc 0008a035 /system/lib/libandroid_runtime.so (???)
  99. native: #11 pc 00d78919 /data/dalvik-cache/arm/system@framework@boot.oat (Java_android_os_BinderProxy_transactNative__ILandroid_os_Parcel_2Landroid_os_Parcel_2I+140)
  100. at android.os.BinderProxy.transactNative(Native method)
  101. at android.os.BinderProxy.transact(Binder.java:505)
  102. at android.content.ContentProviderProxy.query(ContentProviderNative.java:419)
  103. at android.content.ContentResolver.query(ContentResolver.java:502)
  104. at android.content.ContentResolver.query(ContentResolver.java:438)
  105. at com.huawei.mail.chips.BaseRecipientAdapter.doQuery(BaseRecipientAdapter.java:931)
  106. at com.huawei.mail.chips.BaseRecipientAdapter$DirectoryFilter.performFiltering(BaseRecipientAdapter.java:400)
  107. at android.widget.Filter$RequestHandler.handleMessage(Filter.java:234)
  108. at android.os.Handler.dispatchMessage(Handler.java:102)
  109. at android.os.Looper.loop(Looper.java:150)
  110. at android.os.HandlerThread.run(HandlerThread.java:61)
  111. 那么接下来就需要结合代码查看这么多线程创建的原因。
  112. 解决的思路也比较简单,是否需要采取线程池或者限制下线程的个数,防止由于此操作比较耗时(依赖于网络)导致对端ANR。
  1. Bug 582473 single touch point test,操作15次以上必现工厂测试anr
  1. dropbox中找到Validationtools对应的traces,查看主线程的栈,发现主线程在IDLE状态。
  2. "main" prio=5 tid=1 NATIVE
  3. | group="main" sCount=1 dsCount=0 obj=0x4155acc0 self=0x414943f0
  4. | sysTid=3287 nice=-1 sched=0/0 cgrp=apps handle=1074065748
  5. | state=S schedstat=( 0 0 0 ) utm=1241 stm=340 core=0
  6. #00 pc 00021764 /system/lib/libc.so (epoll_wait+12)
  7. #01 pc 000105e3 /system/lib/libutils.so (android::Looper::pollInner(int)+94)
  8. #02 pc 00010811 /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+92)
  9. #03 pc 0006b345 /system/lib/libandroid_runtime.so (android::NativeMessageQueue::pollOnce(_JNIEnv*, int)+22)
  10. #04 pc 0001e70c /system/lib/libdvm.so (dvmPlatformInvoke+112)
  11. #05 pc 0004dc43 /system/lib/libdvm.so (dvmCallJNIMethod(unsigned int const*, JValue*, Method const*, Thread*)+398)
  12. #06 pc 00027778 /system/lib/libdvm.so
  13. #07 pc 0002e410 /system/lib/libdvm.so (dvmMterpStd(Thread*)+76)
  14. #08 pc 0002bac8 /system/lib/libdvm.so (dvmInterpret(Thread*, Method const*, JValue*)+156)
  15. #09 pc 00060259 /system/lib/libdvm.so (dvmInvokeMethod(Object*, Method const*, ArrayObject*, ArrayObject*, ClassObject*, bool)+392)
  16. #10 pc 000683eb /system/lib/libdvm.so
  17. #11 pc 00027778 /system/lib/libdvm.so
  18. #12 pc 0002e410 /system/lib/libdvm.so (dvmMterpStd(Thread*)+76)
  19. #13 pc 0002bac8 /system/lib/libdvm.so (dvmInterpret(Thread*, Method const*, JValue*)+156)
  20. #14 pc 0005ff77 /system/lib/libdvm.so (dvmCallMethodV(Thread*, Method const*, Object*, bool, JValue*, std::__va_list)+338)
  21. #15 pc 0004982b /system/lib/libdvm.so
  22. #16 pc 0004de13 /system/lib/libandroid_runtime.so
  23. #17 pc 0004ecdd /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const*, char const*)+532)
  24. #18 pc 00001423 /system/bin/app_process
  25. #19 pc 0000e403 /system/lib/libc.so (__libc_init+50)
  26. #20 pc 00000f34 /system/bin/app_process
  27. at android.os.MessageQueue.nativePollOnce(Native Method)
  28. at android.os.MessageQueue.next(MessageQueue.java:138)
  29. at android.os.Looper.loop(Looper.java:123)
  30. at android.app.ActivityThread.main(ActivityThread.java:5294)
  31. at java.lang.reflect.Method.invokeNative(Native Method)
  32. at java.lang.reflect.Method.invoke(Method.java:515)
  33. at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:932)
  34. at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:748)
  35. at dalvik.system.NativeStart.main(Native Method)
  36. 然后发现第二个线程是Binder10 且在block状态,通常情况下,一个不是非常繁忙的需要对外提供服务的app是不许要16binder线程的。
  37. 于是接着查看是否其他binder也存在异常,结果发现所有的binder线程均处于同样的调用栈。
  38. "Binder_D" prio=5 tid=21 WAIT
  39. | group="main" sCount=1 dsCount=0 obj=0x41d6f468 self=0x555f4898
  40. | sysTid=3344 nice=0 sched=0/0 cgrp=apps handle=1442108096
  41. | state=S schedstat=( 0 0 0 ) utm=0 stm=0 core=3
  42. at java.lang.Object.wait(Native Method)
  43. - waiting on <0x41d6f4c0> (a java.lang.VMThread) held by tid=21 (Binder_D)
  44. at java.lang.Thread.parkFor(Thread.java:1205)
  45. at sun.misc.Unsafe.park(Unsafe.java:325)
  46. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:157)
  47. at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:813)
  48. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:846)
  49. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1175)
  50. at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:180)
  51. at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:256)
  52. at android.view.SurfaceView$MyWindow.resized(SurfaceView.java:655)
  53. at android.view.IWindow$Stub.onTransact(IWindow.java:108)
  54. at android.os.Binder.execTransact(Binder.java:427)
  55. at dalvik.system.NativeStart.run(Native Method)
  56. "Binder_F" prio=5 tid=23 WAIT
  57. | group="main" sCount=1 dsCount=0 obj=0x41d4e600 self=0x55cb63f8
  58. | sysTid=3351 nice=0 sched=0/0 cgrp=apps handle=1435714912
  59. | state=S schedstat=( 0 0 0 ) utm=1 stm=1 core=0
  60. at java.lang.Object.wait(Native Method)
  61. - waiting on <0x41d4e658> (a java.lang.VMThread) held by tid=23 (Binder_F)
  62. at java.lang.Thread.parkFor(Thread.java:1205)
  63. at sun.misc.Unsafe.park(Unsafe.java:325)
  64. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:157)
  65. at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:813)
  66. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:846)
  67. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1175)
  68. at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:180)
  69. at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:256)
  70. at android.view.SurfaceView$MyWindow.resized(SurfaceView.java:655)
  71. at android.view.IWindow$Stub.onTransact(IWindow.java:108)
  72. at android.os.Binder.execTransact(Binder.java:427)
  73. at dalvik.system.NativeStart.run(Native Method)
  74. 如上面一个例子,在这种Binder满的情况下,systemserverapp之间的通信是被block的,虽然这个是4.4并没有sprdruntimeInfo给我们dumpbinder的状态,我们还是可以推断出
  75. ANR的直接原因就是此处不合常理的Binder被占满。
  76. 首先通过代码可以看到,surfaceView中这行对应的代码为如下:
  77. @Override
  78. public void resized(Rect frame, Rect overscanInsets, Rect contentInsets,
  79. Rect visibleInsets, boolean reportDraw, Configuration newConfig) {
  80. SurfaceView surfaceView = mSurfaceView.get();
  81. if (surfaceView != null) {
  82. if (DEBUG) Log.v(
  83. "SurfaceView", surfaceView + " got resized: w=" + frame.width()
  84. + " h=" + frame.height() + ", cur w=" + mCurWidth + " h=" + mCurHeight);
  85. surfaceView.mSurfaceLock.lock();
  86. try {
  87. if (reportDraw) {
  88. surfaceView.mUpdateWindowNeeded = true;
  89. surfaceView.mReportDrawNeeded = true;
  90. surfaceView.mHandler.sendEmptyMessage(UPDATE_WINDOW_MSG);
  91. } else if (surfaceView.mWinFrame.width() != frame.width()
  92. || surfaceView.mWinFrame.height() != frame.height()) {
  93. surfaceView.mUpdateWindowNeeded = true;
  94. surfaceView.mHandler.sendEmptyMessage(UPDATE_WINDOW_MSG);
  95. }
  96. } finally {
  97. surfaceView.mSurfaceLock.unlock();
  98. }
  99. }
  100. }
  101. mSurfaceLocklock的路径还有如下:
  102. private void updateWindow(boolean force, boolean redrawNeeded)
  103. private final Canvas internalLockCanvas(Rect dirty)
  104. 由于这个是个必现问题,那么问题就简单了,我们可以在所有会lockunlock的地方都加上log,再复现一次来得到真相。
  105. 于是乎发现是在SingleTouchPointTest pass的时候忘记unlockCanvs导致。

套路总结

  1. 1. 查看traces.txt,如果有block栈则通过栈来找到block原因。
  2. 2. 应用自身耗时操作尝试修改异步,被Binder block则进一步确认下对端的情况,如果是依赖于外部状态的可能block操作,应用修改为异步,如果是common接口,转FW
  3. 3. 如果不存在栈,那么尽人事听天命,查看ANR点附近是否有主线程block的可能信息,如果有则顺腾摸瓜,没有就Needinfo

如何避免ANR

  1. A.减少复杂的layout
  2. B.主线程尽量不要做和显示无关的事情。
  3. C.如果存在可能会block、耗时的操作,不要放到主线程中,可以使用异步等方式来放到另外一个handlerthread或者asynctask中去做。
  4. 需要注意的是,使用handler.post或者sendmessage的时候需要确认清楚handlerlooper,如果是主线程,依然可能会ANR
  5. HandlerThread中如果需要处理和显示相关的,还需要到主线程中处理(非UI线程不能操作UI,在ViewRoot中有检查)

Android 开发

Android ANR分析(2)的更多相关文章

  1. Android ANR 分析解决方法

    一:什么是ANR ANR:Application Not Responding,即应用无响应 二:ANR的类型 ANR一般有三种类型: 1. KeyDispatchTimeout(5 seconds) ...

  2. Android ANR分析(1)

    转自:http://blog.csdn.net/itachi85/article/details/6918761 一:什么是ANR ANR:Application Not Responding,即应用 ...

  3. android anr分析方法

    目录(?)[+] 案例1关键词ContentResolver in AsyncTask onPostExecute high iowait 案例2关键词在UI线程进行网络数据的读写   一:什么是AN ...

  4. Android ANR分析及解决方案

    一:什么是ANR ANR:Application Not Responding,即应用无响应. ANR定义:在Android上,如果你的应用程序有一段时间响应不够灵敏,系统会向用户显示一个对话框,这个 ...

  5. [转]Android ANR 分析解决方法

    一:什么是ANR ANR:Application Not Responding,即应用无响应 二:ANR的类型 ANR一般有三种类型: 1. KeyDispatchTimeout(5 seconds) ...

  6. android ANR 分析定位问题

    ANR ? android 规定,Activity如果5秒钟之内无法响应屏幕触摸事件或者键盘输入事件,BroadcastReceiver 如果10s中之内还未执行完操作就会出现ANR 定位ANR问题 ...

  7. Android ANR分析(三)

    http://www.jianshu.com/p/8964812972be http://stackoverflow.com/questions/704311/android-how-do-i-inv ...

  8. Android ANR 分析

    首先贴一下trace 文件 Process: com.oppo.reader PID: 20358 Time: 2933175644_1545041895232 Flags: 0x38d83e44 P ...

  9. Android ANR分析

    1.发生anr时手机会生产traces文件 拉取trace文件:adb pull data/anr/traces.txt ./mytraces.txt 保存路径 参考

随机推荐

  1. Exploiting the Circulant Structure of Tracking-by-Detection with Kernels(二)

    之前给导师汇报时,主要是论文涉及公式的一些推导

  2. 使用PopupWindow

    PopupWindow可以用来实现弹出任意位置的菜单,比Context Menu和Option Menu灵活性更高.Android中弹出一个PopupWindow基本有两个方法: 1 2 //Disp ...

  3. neutron用linux_bridge部署provider网络

    网卡配置: # The loopback network interface auto lo iface lo inet loopback # The primary network interfac ...

  4. 【Networking】k8s容器网络 && golang相关

    Bookmarks flannel/proxy.c at master · coreos/flannel kubernetes/kubernetes: Production-Grade Contain ...

  5. 利用闪回查看Oracle表历史时刻数据

    利用闪回查看Oracle表历史时刻数据 1.查看表历史时刻数据 select * from tab_test AS OF TIMESTAMP to_timestamp('20140917 10:00: ...

  6. Python QRCODE

  7. Linux下安装Nginx服务器

    安装Nginx之前,首先要安装好编译环境gcc和g++,然后以CentOS为例安装Nginx,安装Nginx需要PRCE库.zlib库和ssl的支持,除了ssl外其他的我们都是去官网下载: Nginx ...

  8. [转]AndroidStudio导出jar包

    原文链接:http://blog.csdn.net/hjq842382134/article/details/38538097# 1. 不像在Eclipse,可以直接导出jar包.AndroidStu ...

  9. Xcode无法启动ios模拟器的问题

    一.问题描述 开发过程需要来回切换ios模拟器调试程序,开始在iPhone 4s下调试,然后切换到iPhone 6s Plus,再切换回iPhone 4s,遇到无法启动ios模拟器.错误提示如下: 二 ...

  10. codeforces 493B.Vasya and Wrestling 解题报告

    题目链接:http://codeforces.com/problemset/problem/493/B 题目意思:给出 n 个 techniques,每个 technique 的值为 ai. ai & ...