Spark：java.net.BindException: Address already in use: Service 'SparkUI' failed after 16 retries!

Spark多任务提交运行时候报错。

java.net.BindException: Address already in use: Service 'SparkUI' failed after  retries!
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:)
at sun.nio.ch.Net.bind(Net.java:)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:)
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:)
at org.eclipse.jetty.server.Server.doStart(Server.java:)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:)
at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$(JettyUtils.scala:)
at org.apache.spark.ui.JettyUtils$$anonfun$.apply(JettyUtils.scala:)
at org.apache.spark.ui.JettyUtils$$anonfun$.apply(JettyUtils.scala:)
at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$.apply$mcVI$sp(Utils.scala:)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:)
at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:)
at org.apache.spark.SparkContext$$anonfun$.apply(SparkContext.scala:)
at org.apache.spark.SparkContext$$anonfun$.apply(SparkContext.scala:)
at scala.Option.foreach(Option.scala:)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:)
at com.Spark.FileInput.main(FileInput.java:)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:)
at java.lang.reflect.Method.invoke(Method.java:)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

错误原因：

每一个Spark任务都会占用一个SparkUI端口，默认为4040，如果被占用则依次递增端口重试。但是有个默认重试次数，为16次。16次重试都失败后，会放弃该任务的运行。

解决方法

初始化SparkConf时，添加conf.set(“spark.port.maxRetries”,“”)语句

使用spark-submit提交任务时，在命令行中添加-Dspark.port.maxRetries=

在spark-defaults.conf中添加spark.port.maxRetries