Exception in thread “main“ java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Lj
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class;
at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52)
at org.apache.spark.streaming.scheduler.JobGenerator.<init>(JobGenerator.scala:51)
at org.apache.spark.streaming.scheduler.JobScheduler.<init>(JobScheduler.scala:55)
at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:184)
at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:85)
at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20)
at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class;
at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52)
at org.apache.spark.streaming.scheduler.JobGenerator.<init>(JobGenerator.scala:51)
at org.apache.spark.streaming.scheduler.JobScheduler.<init>(JobScheduler.scala:55)
at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:184)
at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:85)
at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20)
at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)
原因 maven 导入的依赖版本问题
1.因为IDEA-maven的依赖包是直接在pom文件中添加代码,在初次使用时,很容易出现版本不一致的情况;
2.我的程序是spark-scala程序,还需要spark依赖的版本与scala匹配,比如scala2.12只适用于spark2.4以上;
我的scala SDK 是2.12


<!--2.4.8版本不可以-->
<!-- <dependency>-->
<!-- <groupId>org.apache.spark</groupId>-->
<!-- <artifactId>spark-streaming_2.12</artifactId>-->
<!-- <version>2.4.8</version>-->
<!--<!– <scope>provided</scope>–>-->
<!-- </dependency>-->
<!--这个版本3.1.0可以-->
<!-- <dependency>-->
<!-- <groupId>org.apache.spark</groupId>-->
<!-- <artifactId>spark-streaming_2.12</artifactId>-->
<!-- <version>3.1.0</version>-->
<!-- <!– <scope>provided</scope>–>-->
<!-- </dependency>-->
<!-- 这个版本3.0.0可以-->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.12</artifactId>
<version>3.0.0</version>
<!-- <scope>provided</scope>-->
</dependency>
更多推荐
所有评论(0)