kUI' failed after 16 retries (starting from 4040)! Consider explicitly setting the appropriate port for the service 'SparkUI' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.

       at org.sparkproject.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)

       at org.sparkproject.jetty.server.ServerConnector.open(ServerConnector.java:310)

       at org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)

       at org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)

       at org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)

       at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:302)

       at org.apache.spark.ui.JettyUtils$.httpConnect$1(JettyUtils.scala:333)

       at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$5(JettyUtils.scala:336)

       at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$5$adapted(JettyUtils.scala:336)

       at org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2331)

       at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)

       at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2323)

       at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:337)

       at org.apache.spark.ui.WebUI.bind(WebUI.scala:146)

       at org.apache.spark.SparkContext.$anonfun$new$11(SparkContext.scala:486)

       at org.apache.spark.SparkContext.$anonfun$new$11$adapted(SparkContext.scala:486)

       at scala.Option.foreach(Option.scala:407)

       at org.apache.spark.SparkContext.<init>(SparkContext.scala:486)

       at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2672)

       at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:945)

       at scala.Option.getOrElse(Option.scala:189)

       at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:939)

       at com.mm.rdd.WordCount$.main(WordCount.scala:6)

       at com.mm.rdd.WordCount.main(WordCount.scala)

23/05/08 11:46:11 INFO DiskBlockManager: Shutdown hook called

23/05/08 11:46:11 INFO ShutdownHookManager: Shutdown hook called

解决办法:

最终解决方案:

我是在IntellijIdea下运行的,试着安装了windows版本下的hadoop3.2.4核spark3.2.4,都不行;最后是创建统一入口SparkSession,指令后加了一条.config("spark.port.maxRetries",10000):

val spark = SparkSession.builder().master("local[*]").appName("word count").config("spark.port.maxRetries",10000).getOrCreate();成功了。

 

Logo

腾讯云面向开发者汇聚海量精品云计算使用和开发经验,营造开放的云计算技术生态圈。

更多推荐