CDH flink命令行启动yarn-session报错java.lang.NoClassDefFoundError: Could not initialize class org.apache.ha

系统环境错误排查./bin/yarn-session.sh2020-07-09 11:22:01,187 INFOorg.apache.flink.configuration.GlobalConfiguration- Loading configuration property: gateway-port, 02020-07-09 11:22:01,327 ERROR org.apache.fli

bertramlau

4831人浏览 · 2020-07-09 13:50:59

bertramlau · 2020-07-09 13:50:59 发布

系统环境

CentOS 7.7.1908
CDH 6.3.1

错误排查

./bin/yarn-session.sh

2020-07-09 11:22:01,187 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: gateway-port, 0
2020-07-09 11:22:01,327 ERROR org.apache.flink.runtime.security.modules.HadoopModuleFactory  - Cannot create Hadoop Security Module.
java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
	at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:121)
	at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:237)
	at org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:242)
	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:210)
	at org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:202)
	at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:832)
	at org.apache.flink.runtime.util.HadoopUtils.getHadoopConfiguration(HadoopUtils.java:94)
	at org.apache.flink.runtime.security.modules.HadoopModuleFactory.createModule(HadoopModuleFactory.java:50)
	at org.apache.flink.runtime.security.SecurityUtils.install(SecurityUtils.java:64)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:857)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 12 more
2020-07-09 11:22:01,360 ERROR org.apache.flink.runtime.security.SecurityUtils               - Cannot install HadoopSecurityContext.
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
	at org.apache.flink.runtime.security.SecurityUtils.install(SecurityUtils.java:89)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:857)
2020-07-09 11:22:01,611 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Error while running the Flink Yarn session.
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
	at org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:136)
	at org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal(ConfiguredRMFailoverProxyProvider.java:77)
	at org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxy(ConfiguredRMFailoverProxyProvider.java:90)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:73)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:64)
	at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:59)
	at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:95)
	at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.getClusterDescriptor(FlinkYarnSessionCli.java:1008)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createDescriptor(FlinkYarnSessionCli.java:276)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createClusterDescriptor(FlinkYarnSessionCli.java:475)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:612)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:859)
	at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:859)

------------------------------------------------------------
 The program finished with the following exception:

java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
	at org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:136)
	at org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal(ConfiguredRMFailoverProxyProvider.java:77)
	at org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxy(ConfiguredRMFailoverProxyProvider.java:90)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:73)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:64)
	at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:59)
	at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:95)
	at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.getClusterDescriptor(FlinkYarnSessionCli.java:1008)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createDescriptor(FlinkYarnSessionCli.java:276)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createClusterDescriptor(FlinkYarnSessionCli.java:475)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:612)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:859)
	at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:859)

经查资料发现，是缺少commons-configuration-1.9.jar和hadoop-auth-3.0.0-cdh6.3.1.jar，遂拷贝相关包至flink lib目录下，继续执行：

[root@master01 flink]# ./bin/yarn-session.sh
Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR was set.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2020-07-09 11:24:57,112 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: tables, []
2020-07-09 11:24:57,114 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: functions, []
2020-07-09 11:24:57,114 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: catalogs, []
2020-07-09 11:24:57,114 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:88: "execution:"
2020-07-09 11:24:57,114 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: planner, old
2020-07-09 11:24:57,114 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: type, streaming
2020-07-09 11:24:57,115 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: time-characteristic, event-time
2020-07-09 11:24:57,115 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: periodic-watermarks-interval, 200
2020-07-09 11:24:57,115 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: result-mode, table
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-table-result-rows, 1000000
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: parallelism, 1
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-parallelism, 128
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: min-idle-state-retention, 0
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-idle-state-retention, 0
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: current-catalog, default_catalog
2020-07-09 11:24:57,118 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: current-database, default_database
2020-07-09 11:24:57,119 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:115: "  restart-strategy:"
2020-07-09 11:24:57,119 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: type, fallback
2020-07-09 11:24:57,119 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:141: "deployment:"
2020-07-09 11:24:57,119 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: response-timeout, 5000
2020-07-09 11:24:57,119 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: gateway-address, ""
2020-07-09 11:24:57,119 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: gateway-port, 0
2020-07-09 11:24:57,570 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to root (auth:SIMPLE)
2020-07-09 11:24:57,808 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Error while running the Flink Yarn session.
java.lang.NoClassDefFoundError: org/apache/htrace/Trace
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:213)
	at com.sun.proxy.$Proxy7.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:266)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy8.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:504)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:236)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:463)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:378)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:631)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:859)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:859)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 21 more

------------------------------------------------------------
 The program finished with the following exception:

java.lang.NoClassDefFoundError: org/apache/htrace/Trace
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:213)
	at com.sun.proxy.$Proxy7.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:266)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy8.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:504)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:236)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:463)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:378)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:631)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:859)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:859)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 21 more

缺少htrace相关jar包，继续拷贝

cp /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/spark/jars/htrace-core4-4.1.0-incubating.jar ./lib/
cp /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/hadoop/client/htrace-core4.jar ./lib/

被没有解决问题：

The program finished with the following exception:

java.lang.NoClassDefFoundError: org/apache/htrace/Trace
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:213)
	at com.sun.proxy.$Proxy7.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:266)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy8.getClusterNodes(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:504)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:236)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:463)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:378)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:631)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:859)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:859)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 21 more

为啥还是少class呢，老老实实grep jar吧：

for i in lib/*.jar; do jar -tvf "$i" | grep -Hsi org.apache.htrace.Trace && echo "$i"; done

什么输出也没有，证明htrace-core4根本没有org.apache.htrace.Trace这个类，Google一番，发现是htrace-core3.x才有这个类，见下图：
在这里插入图片描述
开始怀疑引入了错误的Hadoop包，因为htrace-core3.x对应的是CDH5的，接着去看lib目录：

居然有个hadoop2的包，遂mv之，以观后效：

[root@master01 lib]# mv flink-shaded-hadoop2-2.7.5-1.8.0.jar flink-shaded-hadoop2-2.7.5-1.8.0.jar.bak
[root@master01 lib]# cd ..
[root@master01 flink]# ./bin/yarn-session.sh
Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR was set.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2020-07-09 13:30:32,552 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: tables, []
2020-07-09 13:30:32,553 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: functions, []
2020-07-09 13:30:32,553 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: catalogs, []
2020-07-09 13:30:32,554 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:88: "execution:"
2020-07-09 13:30:32,554 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: planner, old
2020-07-09 13:30:32,554 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: type, streaming
2020-07-09 13:30:32,554 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: time-characteristic, event-time
2020-07-09 13:30:32,554 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: periodic-watermarks-interval, 200
2020-07-09 13:30:32,554 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: result-mode, table
2020-07-09 13:30:32,557 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-table-result-rows, 1000000
2020-07-09 13:30:32,557 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: parallelism, 1
2020-07-09 13:30:32,557 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-parallelism, 128
2020-07-09 13:30:32,558 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: min-idle-state-retention, 0
2020-07-09 13:30:32,558 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: max-idle-state-retention, 0
2020-07-09 13:30:32,558 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: current-catalog, default_catalog
2020-07-09 13:30:32,558 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: current-database, default_database
2020-07-09 13:30:32,558 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:115: "  restart-strategy:"
2020-07-09 13:30:32,558 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: type, fallback
2020-07-09 13:30:32,558 WARN  org.apache.flink.configuration.GlobalConfiguration            - Error while trying to split key and value in configuration file /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/conf/flink-conf.yaml:141: "deployment:"
2020-07-09 13:30:32,559 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: response-timeout, 5000
2020-07-09 13:30:32,559 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: gateway-address, ""
2020-07-09 13:30:32,559 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: gateway-port, 0
2020-07-09 13:30:32,939 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to root (auth:SIMPLE)
2020-07-09 13:30:33,301 INFO  org.apache.hadoop.conf.Configuration                          - resource-types.xml not found
2020-07-09 13:30:33,302 INFO  org.apache.hadoop.yarn.util.resource.ResourceUtils            - Unable to find 'resource-types.xml'.
2020-07-09 13:30:33,339 WARN  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - The JobManager or TaskManager memory is below the smallest possible YARN Container size. The value of 'yarn.scheduler.minimum-allocation-mb' is '5120'. Please increase the memory size.YARN will allocate the smaller containers but the scheduler will account for the minimum-allocation-mb, maybe not all instances you requested will start.
2020-07-09 13:30:33,339 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Cluster specification: ClusterSpecification{masterMemoryMB=5120, taskManagerMemoryMB=5120, numberTaskManagers=1, slotsPerTaskManager=1}
2020-07-09 13:30:33,828 WARN  org.apache.hadoop.util.NativeCodeLoader                       - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-07-09 13:30:34,677 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Submitting application master application_1593589036710_0049
2020-07-09 13:30:34,708 INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Submitted application application_1593589036710_0049
2020-07-09 13:30:34,708 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Waiting for the cluster to be allocated
2020-07-09 13:30:34,709 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deploying cluster, current state ACCEPTED
2020-07-09 13:30:37,723 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - YARN application has been deployed successfully.
2020-07-09 13:30:38,108 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest client endpoint started.
Flink JobManager is now running on slave03.cdh:40845 with leader id 00000000-0000-0000-0000-000000000000.
JobManager Web Interface: http://slave03.cdh:40845
^C2020-07-09 13:30:49,390 INFO  org.apache.flink.runtime.rest.RestClient                      - Shutting down rest endpoint.
2020-07-09 13:30:49,392 INFO  org.apache.flink.runtime.rest.RestClient                      - Rest endpoint shutdown complete.
2020-07-09 13:30:49,395 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Deleted Yarn properties file at /tmp/.yarn-properties-root

至此一切正常

总结

  引入正确的jar包很重要，一旦引入错误会带来一系列麻烦，出现问题不要慌，见招拆招吧。

参考资料

腾讯云开发者社区

腾讯云面向开发者汇聚海量精品云计算使用和开发经验，营造开放的云计算技术生态圈。

更多推荐

VSOMEIP — SOME/IP 开源实现

腾讯云开发者社区

DeepSeek 大爆发，带来了哪些新希望？

腾讯云开发者社区

李建忠：智能体正将互联网从信息网络重构为行动网络

腾讯云开发者社区

所有评论(0)

查看更多评论

bertramlau

@lhmood

已为社区贡献1条内容