[Presto] Kerberos trouble shooting
When I configured presto cluster to connect hive by kerberos, I met some problems which cost me too much time to solve them, so I summarized the problems, hope could help others.
1.Append -Djava.security.krb5.conf=”krb5.conf location” to etc/jvm.properties
8) Error in custom provider, java.lang.NoClassDefFoundError: Could not initialize class com.facebook.presto.hive.authentication.KerberosHadoopAuthentication
at com.facebook.presto.hive.authentication.AuthenticationModules$1.createHadoopAuthentication(AuthenticationModules.java:59) (via modules: com.facebook.presto.hive.authentication.HiveAuthenticationModule -> io.airlift.configuration.ConditionalModule -> com.facebook.presto.hive.authentication.AuthenticationModules$1)
while locating com.facebook.presto.hive.authentication.HadoopAuthentication annotated with @com.facebook.presto.hive.ForHiveMetastore()
for the 2nd parameter of com.facebook.presto.hive.authentication.KerberosHiveMetastoreAuthentication.<init>(KerberosHiveMetastoreAuthentication.java:44)
...
...
2.Specify hdfs-site.xml/core-site.xml in hive.properties like hive.config.resources=xxx/core-site.xml,xxx/hdfs-site.xml
Query 20180504_150148_00018_v6ndf failed: java.net.UnknownHostException: xxx
3.Download hadoop-lzo jar into plugin/hive-hadoop2
Query 20180504_150959_00002_3f2qe failed: Unable to create input format org.apache.hadoop.mapred.TextInputFormat
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 19 more
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 21 more
4.Export KRB5_CONFIG & get kerberos tgt, use kinit command
Query 20180504_153940_00000_nrsgy failed: Failed to list directory: hdfs://xxx/user/hive/warehouse/xxx.db/xxx
5.More than one coordinator in the cluster
2018-05-04T18:10:56.410Z WARN http-worker-4560 com.facebook.presto.execution.SqlTaskManager Switching coordinator affinity from hhbts to qhnep
2018-05-04T18:10:56.500Z WARN http-worker-4560 com.facebook.presto.execution.SqlTaskManager Switching coordinator affinity from qhnep to c83wr
2018-05-04T18:10:56.578Z WARN http-worker-4395 com.facebook.presto.execution.SqlTaskManager Switching coordinator affinity from c83wr to ujj9n
2018-05-04T18:10:56.749Z WARN http-worker-4432 com.facebook.presto.execution.SqlTaskManager Switching coordinator affinity from ujj9n to wdsxf
2018-05-04T18:10:57.009Z WARN http-worker-4584 com.facebook.presto.execution.SqlTaskManager Switching coordinator affinity from wdsxf to hhbts
Reply