我正在尝试从python应用程序向docker-compose桥接网络中的gcs模拟器发出请求。当我尝试时，我发现不知何故，gcs客户端库试图使用IPv6向gcs模拟器发出请求并失败，因为mac上的docker不支持IPv6。
我已经实现了以下答案来正确IPv4，但它似乎仍然试图通过IPv6发出请求。
如何在docker-compose网络中从python成功请求gcs模拟器?
我已经确认从本地Python脚本到gcs模拟器的请求没有docker-compose是成功的。
docker-for-mac问题:https://github.com/docker/for-mac/issues/1432
引用的答案:强制请求使用IPv4/IPv6
gcs模拟器:https://github.com/fsouza/fake-gcs-server
样本docker-compose.yaml
version: '3'
services:
run:
build: .
container_name: run
ports:
- 9090:8080
env_file:
- ./.env
environment:
- PORT=8080
gcs:
image: fsouza/fake-gcs-server:latest
container_name: fake-gcs-server
ports:
- 4443:4443
env_file:
- ./.env
样本实现:
from google.cloud import storage
from google.api_core.client_options import ClientOptions
from google.auth.credentials import AnonymousCredentials
from unittest.mock import patch
from multijob_sample import variables as vs
import requests
import urllib3
import urllib3.util.connection
import traceback
import socket
orig_getaddrinfo = socket.getaddrinfo
def getaddrinfoIPv4(host, port, family=0, type=0, proto=0, flags=0):
print(f'running patched getaddrinfo')
return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET, type=type, proto=proto, flags=flags)
patcher = patch('socket.getaddrinfo', side_effect=getaddrinfoIPv4)
patcher.start()
# for fake-gcs-emulator
http_ssl_disabled = requests.Session()
http_ssl_disabled.verify = False
urllib3.disable_warnings(
urllib3.exceptions.InsecureRequestWarning
) # disable https warnings for https insecure certs
client = storage.Client(
credentials=AnonymousCredentials(),
project=vs.project_id,
client_options=ClientOptions(api_endpoint='https://gcs:4443'),
_http=http_ssl_disabled,
)
def put_file(bucket_id: str, file, blobname: str):
file.seek(0)
try:
client.get_bucket(bucket_id).blob(blob_name=blobname).upload_from_file(file)
print(f'file {blobname} uploaded')
except Exception as e:
print(f'failed to put file: {blobname}')
print(f'error: {e}')
print(f'trace: {traceback.format_exc()}')
put_file("bucketid", file, "blobname") # do put_file
错误信息:
run | running patched getaddrinfo
run | failed to put file: test.csv
run | error: HTTPSConnectionPool(host='::', port=4443): Max retries exceeded with url: /upload/resumable/efbbcde9c49cda2ff78e8da24371ea03 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8fb0765be0>: Failed to establish a new connection: [Errno -9] Address family for hostname not supported'))
run | trace: Traceback (most recent call last):
run | File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
run | conn = connection.create_connection(
run | File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
run | for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
run | File "/usr/local/lib/python3.9/unittest/mock.py", line 1093, in __call__
run | return self._mock_call(*args, **kwargs)
run | File "/usr/local/lib/python3.9/unittest/mock.py", line 1097, in _mock_call
run | return self._execute_mock_call(*args, **kwargs)
run | File "/usr/local/lib/python3.9/unittest/mock.py", line 1158, in _execute_mock_call
run | result = effect(*args, **kwargs)
run | File "/app/multijob_sample/storage.py", line 26, in getaddrinfoIPv4
run | return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET, type=type, proto=proto, flags=flags)
run | File "/usr/local/lib/python3.9/socket.py", line 954, in getaddrinfo
run | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
run | socket.gaierror: [Errno -9] Address family for hostname not supported
这是我在一段时间内不得不解决的最恼人的问题之一。解决方案是使用-external-url http://<your docker compose service name>:<port>选项运行仿真器。
这个问题只发生在文件上传，因为它只发生在可恢复的上传。对于可恢复上传，GCS客户端首先"启动"。在服务器上恢复上传，并且在响应中，服务器包含一个URL，供将来的请求访问(不确定为什么，但这似乎是一个复杂API的合理部分)。问题是模拟器不知道它自己的url!实际上，如果您查看模拟器的日志，您将看到它打印出server started at http://[::]:4443之类的东西。::和你在错误中看到的::是一样的。因此，模拟器响应它的::URL，然后过了一会儿，客户端崩溃试图解析该URL。
我仍然不确定为什么在docker-compose外运行，我猜在"","localhost"或"::" '周围有一些特殊的大小写。