问题

在spark集群运行scala程序报错

Error:scalac: error while loading package, class file 'C:\Program Files (x86)\scala\lib\scala-library.jar(scala/collection/immutable/package.class)' is broken
(class java.lang.RuntimeException/error reading Scala signature of package.class: Scala signature package has wrong version
 **expected: 5.0**
 found: 5.2 in package.class)

程序内容

object ActionDemo {

  def main(args: Array[String]): Unit = {
    val conf = new SparkConf().setAppName("ActionDemo").setMaster("spark://192.168.137.15:7077")
    val context = new SparkContext(conf)
    val rdd1:RDD[Int] = context.parallelize(List(1,2,3,4,5,6,4))
    println(rdd1.collect().toList)
    println(rdd1.count())
    val rdd2:Array[Int]=rdd1.top(3)
    println(rdd2.toBuffer)
    val rdd3:Array[Int]=rdd1.take(3)
    println(rdd3.toBuffer)
  }
}

原因 spark-core版本需要和scala版本一致

因为scala版本是这样的
<scala.version>2.12.15</scala.version>
所以需要修改spark-core依赖，其他的依赖也需要修改为_2.12版本

<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-core_2.12</artifactId>
			<version>3.2.1</version>
		</dependency>

完整maven

	<!-- 声明公有的属性 -->
	<properties>
		<maven.compiler.source>1.8</maven.compiler.source>
		<maven.compiler.target>1.8</maven.compiler.target>
		<encoding>UTF-8</encoding>
		<scala.version>2.12.15</scala.version>
		<spark.version>3.2.1</spark.version>
		<hadoop.version>2.7.1</hadoop.version>
		<mysql.version>8.0.13</mysql.version>
	</properties>
	<!-- 声明并引入公有的依赖 -->
	<dependencies>
		<dependency>
			<groupId>org.scala-lang</groupId>
			<artifactId>scala-library</artifactId>
			<version>${scala.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-core_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.hadoop</groupId>
			<artifactId>hadoop-client</artifactId>
			<version>${hadoop.version}</version>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<version>${mysql.version}</version>
		</dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka-0-10_2.12</artifactId>
            <version>${spark.version}</version>
        </dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-streaming_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-hive_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-mllib_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-hive-thriftserver_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-sql_2.12</artifactId>
			<version>${spark.version}</version>
		</dependency>
	</dependencies>