Flume整合SparkStreaming使用Poll方式拉取数据出现
UnsupportedClassVersionError: org/apache/spark/streaming/flume/sink/SparkFlumeProtocol : Unsupported major.minor version 52.0
问题：
INFO node.Application: Starting Source source1
18/06/12 03:34:54 ERROR lifecycle.LifecycleSupervisor: Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@14ed6a7 counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.UnsupportedClassVersionError: org/apache/spark/streaming/flume/sink/SparkFlumeProtocol : Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access100(URLClassLoader.java:71)atjava.net.URLClassLoader100(URLClassLoader.java:71)atjava.net.URLClassLoader<script type="math/tex" id="MathJax-Element-386">100(URLClassLoader.java:71) at java.net.URLClassLoader</script>1.run(URLClassLoader.java:361)
at java.net.URLClassLoader1.run(URLClassLoader.java:355)atjava.security.AccessController.doPrivileged(NativeMethod)atjava.net.URLClassLoader.findClass(URLClassLoader.java:354)atjava.lang.ClassLoader.loadClass(ClassLoader.java:425)atsun.misc.Launcher1.run(URLClassLoader.java:355)atjava.security.AccessController.doPrivileged(NativeMethod)atjava.net.URLClassLoader.findClass(URLClassLoader.java:354)atjava.lang.ClassLoader.loadClass(ClassLoader.java:425)atsun.misc.Launcher<script type="math/tex" id="MathJax-Element-387">1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher</script>AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access100(URLClassLoader.java:71)atjava.net.URLClassLoader100(URLClassLoader.java:71)atjava.net.URLClassLoader<script type="math/tex" id="MathJax-Element-388">100(URLClassLoader.java:71) at java.net.URLClassLoader</script>1.run(URLClassLoader.java:361)
at java.net.URLClassLoader1.run(URLClassLoader.java:355)atjava.security.AccessController.doPrivileged(NativeMethod)atjava.net.URLClassLoader.findClass(URLClassLoader.java:354)atjava.lang.ClassLoader.loadClass(ClassLoader.java:425)atsun.misc.Launcher1.run(URLClassLoader.java:355)atjava.security.AccessController.doPrivileged(NativeMethod)atjava.net.URLClassLoader.findClass(URLClassLoader.java:354)atjava.lang.ClassLoader.loadClass(ClassLoader.java:425)atsun.misc.Launcher<script type="math/tex" id="MathJax-Element-389">1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher</script>AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.spark.streaming.flume.sink.SparkSink.start(SparkSink.scala:84)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisorMonitorRunnable.run(LifecycleSupervisor.java:251)atjava.util.concurrent.ExecutorsMonitorRunnable.run(LifecycleSupervisor.java:251)atjava.util.concurrent.Executors<script type="math/tex" id="MathJax-Element-390">MonitorRunnable.run(LifecycleSupervisor.java:251) at java.util.concurrent.Executors</script>RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutorScheduledFutureTask.accessScheduledFutureTask.access<script type="math/tex" id="MathJax-Element-391">ScheduledFutureTask.access</script>301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutorScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)atjava.util.concurrent.ThreadPoolExecutorScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)atjava.util.concurrent.ThreadPoolExecutor<script type="math/tex" id="MathJax-Element-392">ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor</script>Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
18/06/12 03:34:54 INFO source.SpoolDirectorySource: SpoolDirectorySource source starting with directory: /usr/local/flume_logs
18/06/12 03:34:54 INFO sink.SparkSink: Stopping Spark Sink: sink1
解决方法:
云因是导入到flume lib中的spark-streaming-flume-sink_2.10-1.5.0.jar版本不对，开始使用的是2.10.1 出现version不匹配的问题
直接下载正确的jar导入到flume的lib中，
问题解决！