错误场景描述

1. spark_submit.sh 如下：

   spark-submit --master yarn --deploy-mode cluster --driver-memory 2g --num-executors 3 --executor-cores 1 --executor-memory 4g --queue spark --class com.aisino.controller.Test11 com_aisino_warehouse-1.0-SNAPSHOT-jar-with-dependencies.jar 
   
   

2. jar包在client模式下正常运行，但在cluster模式下运行错误。一直Accepted

   21/02/07 15:32:12 INFO Client: Application report for application_1612666934596_0042 (state: ACCEPTED)
   21/02/07 15:32:13 INFO Client: Application report for application_1612666934596_0042 (state: ACCEPTED)
   21/02/07 15:32:14 INFO Client: Application report for application_1612666934596_0042 (state: ACCEPTED)
   21/02/07 15:32:15 INFO Client: Application report for application_1612666934596_0042 (state: ACCEPTED)
   21/02/07 15:32:16 INFO Client: Application report for application_1612666934596_0042 (state: ACCEPTED)
   

3. 去yarn查看日志
   

4. 找到对应失败spark任务点进去
   

5. 错误日志如下：

   21/02/07 15:10:29 ERROR ApplicationMaster: Uncaught exception:
   java.lang.IllegalStateException: User did not initialize spark context!
   at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:510)
   at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$$runImpl(ApplicationMaster.scala:345)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$2.apply$mcV$sp(ApplicationMaster.scala:260)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$2.apply(ApplicationMaster.scala:260)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$2.apply(ApplicationMaster.scala:260)
   at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$5.run(ApplicationMaster.scala:815)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:422)
   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
   at org.apache.spark.deploy.yarn.ApplicationMaster.doAsUser(ApplicationMaster.scala:814)
   at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:259)
   at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:839)
   at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
   21/02/07 15:10:29 INFO SparkContext: Invoking stop() from shutdown hook
   

解决方法：

1. 好家伙，偷偷查看main函数，默默删掉setMaster(“local[*]”)
   

2. 重新打包提交，短暂accepted后开始running状态，最后finished