在 EMR 中为 Presto 设置配置单元配置属性

Question

我正在尝试更新/etc/presto/conf/catalog/hive.properties内部的 hive 连接器属性。我添加了以下内容：

hive.s3.max-client-retries=50
hive.s3.max-error-retries=50
hive.s3.max-connections=500
hive.s3.connect-timeout=5m
hive.s3.socket-timeout=5m

但出于某种原因，这些是不被接受的。presto 服务器重新启动失败，并显示以下错误：

2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   UNUSED PROPERTIES
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.connect-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-client-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-connections
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-error-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.socket-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   
2018-04-16T19:34:26.587Z    ERROR   main    com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:
1) Configuration property 'hive.s3.connect-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
2) Configuration property 'hive.s3.max-client-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
3) Configuration property 'hive.s3.max-connections' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
4) Configuration property 'hive.s3.max-error-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
5) Configuration property 'hive.s3.socket-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
5 errors

Answer 1

当hive.s3-file-system-type设置为 PRESTO 时，这些属性适用。当它是例如。 EMRFS ，这些配置属性不适用，设置它们会导致您观察到的错误。

Answer 2

正如 Piotr 所提到的，EMR 将从 emr-5.12.0 开始的默认文件系统实现从 PrestoS3FileSystem 更改为 EmrFS。PrestoS3文件系统属性将不可用，但您可以像配置任何其他应用程序一样配置 EmrFS。有关详细信息，请参阅 https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-considerations.html 上的"EMRFS 和 PrestoS3文件系统配置"。