高可用性-在此之前,我一直在获取java.net.UnnknownHostException:nameservice1



我使用presto 0.136和10个节点的设置,但尽管将core-site.xml和hdfs-site.xml放在适当的hive.properties文件中,我还是收到了下面的错误。如果有人能帮忙,请表示感谢。这不是一个重复的问题,因为所有类似的问题都没有此堆栈跟踪。。

2016-08-05T09:17:29.893Z    WARN    http-client-shared-scheduler    com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.195.0.44:8090/v1/memory: java.util.concurrent.TimeoutException: Idle timeout 2000ms
2016-08-05T09:17:39.900Z    WARN    http-client-shared-scheduler    com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.195.0.63:8090/v1/memory: java.util.concurrent.TimeoutException: Idle timeout 2000ms
2016-08-05T09:18:13.924Z    WARN    http-client-shared-scheduler    com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.195.0.62:8090/v1/memory: java.util.concurrent.TimeoutException: Idle timeout 2000ms
2016-08-05T09:18:51.534Z    ERROR   remote-task-callback-0  com.facebook.presto.execution.StageStateMachine Stage 20160805_091848_00000_z7g3h.1 failed
java.lang.IllegalArgumentException: java.net.UnknownHostException: nameservice1
    at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
    at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
    at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
    at org.apache.hadoop.fs.PrestoFileSystemCache.createFileSystem(PrestoFileSystemCache.java:74)
    at org.apache.hadoop.fs.PrestoFileSystemCache.getInternal(PrestoFileSystemCache.java:61)
    at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:43)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
    at parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:384)
    at com.facebook.presto.hive.parquet.ParquetHiveRecordCursor.createParquetRecordReader(ParquetHiveRecordCursor.java:364)
    at com.facebook.presto.hive.parquet.ParquetHiveRecordCursor.<init>(ParquetHiveRecordCursor.java:203)
    at com.facebook.presto.hive.parquet.ParquetRecordCursorProvider.createHiveRecordCursor(ParquetRecordCursorProvider.java:102)
    at com.facebook.presto.hive.HivePageSourceProvider.getHiveRecordCursor(HivePageSourceProvider.java:128)
    at com.facebook.presto.hive.HivePageSourceProvider.createPageSource(HivePageSourceProvider.java:106)
    at com.facebook.presto.spi.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:43)
    at com.facebook.presto.transaction.LegacyConnectorPageSourceProvider.createPageSource(LegacyConnectorPageSourceProvider.java:40)
    at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:48)
    at com.facebook.presto.operator.TableScanOperator.createSourceIfNecessary(TableScanOperator.java:268)
    at com.facebook.presto.operator.TableScanOperator.isFinished(TableScanOperator.java:210)
    at com.facebook.presto.operator.Driver.processInternal(Driver.java:377)
    at com.facebook.presto.operator.Driver.processFor(Driver.java:303)
    at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:580)
    at com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:505)
    at com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:640)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: nameservice1
    ... 30 more

2016-08-05T09:18:51.535Z    ERROR   query-execution-2   com.facebook.presto.execution.QueryStateMachine Query 20160805_091848_00000_z7g3h failed
java.lang.IllegalArgumentException: java.net.UnknownHostException: nameservice1
    at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
    at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
    at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
    at org.apache.hadoop.fs.PrestoFileSystemCache.createFileSystem(PrestoFileSystemCache.java:74)
    at org.apache.hadoop.fs.PrestoFileSystemCache.getInternal(PrestoFileSystemCache.java:61)
    at org.apache.hadoop.fs.PrestoFileSystemCache.get(PrestoFileSystemCache.java:43)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
    at parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:384)
    at com.facebook.presto.hive.parquet.ParquetHiveRecordCursor.createParquetRecordReader(ParquetHiveRecordCursor.java:364)
    at com.facebook.presto.hive.parquet.ParquetHiveRecordCursor.<init>(ParquetHiveRecordCursor.java:203)
    at com.facebook.presto.hive.parquet.ParquetRecordCursorProvider.createHiveRecordCursor(ParquetRecordCursorProvider.java:102)
    at com.facebook.presto.hive.HivePageSourceProvider.getHiveRecordCursor(HivePageSourceProvider.java:128)
    at com.facebook.presto.hive.HivePageSourceProvider.createPageSource(HivePageSourceProvider.java:106)
    at com.facebook.presto.spi.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:43)
    at com.facebook.presto.transaction.LegacyConnectorPageSourceProvider.createPageSource(LegacyConnectorPageSourceProvider.java:40)
    at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:48)
    at com.facebook.presto.operator.TableScanOperator.createSourceIfNecessary(TableScanOperator.java:268)
    at com.facebook.presto.operator.TableScanOperator.isFinished(TableScanOperator.java:210)
    at com.facebook.presto.operator.Driver.processInternal(Driver.java:377)
    at com.facebook.presto.operator.Driver.processFor(Driver.java:303)
    at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:580)
    at com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:505)
    at com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:640)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: nameservice1
    ... 30 more

请将核心站点和hdfs站点配置添加到/etc/catalog/hive.properties中,如下所示-

hive.config.resources=/haop/etc/haop/core-site.xml,/"/haop//etc/hadoop/hdfs-site.xml

其中"是hadoop二进制文件的位置。

如果您正在使用HA,请尝试禁用它。这应该有效。注意:当我在过去使用presto:0.123时尝试过这种方法时,HA支持不可用。我不知道现在的情况如何。

相关内容

  • 没有找到相关文章

最新更新