spark yarn任务的executor 无故 timeout之原因分析

问题：

用  spark-submit --master yarn --deploy-mode cluster --driver-memory 2G --num-executors 6 --executor-memory 2G ~~~

提交任务时，最后一个executor 执行时间 超过了 160s 导致 timeout而退出，造成任务重新执行造成用时过长。具体请看下面介绍：

// :: WARN spark.HeartbeatReceiver: Removing executor  with no recent heartbeats:  ms exceeds timeout  ms
// :: ERROR cluster.YarnClusterScheduler: Lost executor  on slave10: Executor heartbeat timed out after  ms
// :: WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 (TID , slave10): ExecutorLostFailure (executor  exited caused by one of the running tasks) Reason: Executor heartbeat timed out after  ms
// :: INFO scheduler.DAGScheduler: Executor lost:  (epoch )
// :: INFO cluster.YarnClusterSchedulerBackend: Requesting to kill executor(s)
// :: INFO scheduler.TaskSetManager: Starting task 0.1 in stage 0.0 (TID , slave06, partition ,RACK_LOCAL,  bytes)
// :: INFO storage.BlockManagerMasterEndpoint: Trying to remove executor  from BlockManagerMaster.
// :: INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(, slave10, )
// :: INFO storage.BlockManagerMaster: Removed  successfully in removeExecutor
// :: INFO scheduler.DAGScheduler: Host added was in lost list earlier: slave10
// :: INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) .
// :: INFO scheduler.TaskSetManager: Finished task 0.1 in stage 0.0 (TID ) in  ms on slave06 (/)
// :: INFO scheduler.DAGScheduler: ResultStage  (saveAsNewAPIHadoopFile at DataFrameFunctions.scala:) finished in 162.495 s

初步估计是  因为最后一步用到的计算多，但是 spark的堆外内存配置低 如下所示

spark.yarn.executor.memoryOverhead

executorMemory * 0.10, with minimum of 384

故加大配置，如下：
spark-submit --master yarn --deploy-mode cluster --driver-memory 2G --num-executors 6 --executor-memory 2G --conf spark.yarn.executor.memoryOverhead=512 --conf spark.yarn.driver.memoryOverhead=512

经测试上述问题不复存在！