Docker & Hive - Windows 上的端口 50070 端口权限被拒绝

Question

我想设置一个本地的hive服务器，并找到了这个repo:

https://github.com/big-data-europe/docker-hive

这是我使用的yaml文件。

version: "3"
services:
namenode:
image: bde2020/hadoop-namenode:2.0.0-hadoop2.7.4-java8
volumes:
- namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop-hive.env
ports:
- "50070:50070"
datanode:
image: bde2020/hadoop-datanode:2.0.0-hadoop2.7.4-java8
volumes:
- datanode:/hadoop/dfs/data
env_file:
- ./hadoop-hive.env
environment:
SERVICE_PRECONDITION: "namenode:50070"
ports:
- "50075:50075"
hive-server:
image: bde2020/hive:2.3.2-postgresql-metastore
env_file:
- ./hadoop-hive.env
environment:
HIVE_CORE_CONF_javax_jdo_option_ConnectionURL: "jdbc:postgresql://hive-metastore/metastore"
SERVICE_PRECONDITION: "hive-metastore:9083"
ports:
- "10000:10000"
hive-metastore:
image: bde2020/hive:2.3.2-postgresql-metastore
env_file:
- ./hadoop-hive.env
command: /opt/hive/bin/hive --service metastore
environment:
SERVICE_PRECONDITION: "namenode:50070 datanode:50075 hive-metastore-postgresql:5432"
ports:
- "9083:9083"
hive-metastore-postgresql:
image: bde2020/hive-metastore-postgresql:2.3.0
presto-coordinator:
image: shawnzhu/prestodb:0.181
ports:
- "8080:8080"
volumes:
namenode:
datanode:

错误:

Error starting userland proxy: Bind for 0.0.0.0:50075: unexpected error Permission denied

端口>50000在windows上被阻塞了，我在公司的电脑上没有管理员权限，所以我试着像这样映射端口:

ports:
- "40070:50070"
environment:
SERVICE_PRECONDITION: "namenode:40070 datanode:40075 hive-metastore-postgresql:5432"

这将让我启动容器，但容器似乎无法通信。

hive-metastore_1             | [1/100] check for namenode:40070...
hive-metastore_1             | [1/100] namenode:40070 is not available yet
hive-metastore_1             | [1/100] try in 5s once again ...
956a5237dbe2_docker-hive_datanode_1 | [4/100] check for namenode:40070...
956a5237dbe2_docker-hive_datanode_1 | [4/100] namenode:40070 is not available yet

我试图改变两个端口:

ports:
- "40070:40070"

这将不起作用，因为一些ip似乎是硬编码的:

ded7410db1b9_docker-hive_namenode_1 | 21/10/08 12:39:05 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:50070
ded7410db1b9_docker-hive_namenode_1 | 21/10/08 12:39:05 INFO http.HttpServer2: Jetty bound to port 50070

有人知道怎么让它运行吗?

Answer 1

与以下内容:

ports:
- "40070:50070"

你所做的只是将流量从主机端口40070引导到集装箱端口50070。

访问"namenode"例如:

localhost:40070

访问"namenode"在撰写网络内部:

namenode:50070

使用BDE的服务前提条件在建立自己的服务之前反复检查容器和端口，以查看服务是否正在运行，以确保一切就绪。您没有更改容器上运行的端口，因此您的容器应该仍然通过端口50070进行通信。

您错误地将前提条件更改为扫描主机端口40070，而它应该查找内部网络容器端口50070，而不管主机端口是什么。

改成如下:

ports:
- "40070:50070"

environment:
SERVICE_PRECONDITION: "namenode:50070 datanode:50075 hive-metastore-postgresql:5432"

您可以使用提供的环境变量文件更改Hive等的操作端口，但您不应该需要这样做。将主机端口40070暴露给集装箱端口50070，对docker业务无影响。