我正在尝试使用moto来实现集成测试，并尝试测试一些要与cloudformation实现的自定义资源。我正在使用github页面中提供的示例。我稍微修改了示例以适应我的开发环境，并接受yaml文件而不是json文件，这也适合我的部署策略。我已经浏览了示例中的文档和代码，并启动了cf.create_stack命令所需的服务器。该示例调用
def wait_for_log_msg(expected_msg, log_group):
...
如果expected_msg在接收到的消息中，该方法返回True和set，否则返回False和set(代码见下文)。
问题:
根据示例，发送的消息是Status code: 200，该消息未收到，当我打印消息时，我得到failed executing http.request。我假设这是由于我使用motoserver的一些错误方法。
问题:
我怎么能得到motoserver，特别是logstream输出所需的Status code: 200消息?
尝试:
我尝试使用docker启动motoserver，但我得到以下错误:
{"error running docker: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))"}
然后切换到直接运行motoserver，在终端中使用
moto_server -p5000
，它产生一个失败，当我打印日志消息时，我得到(格式化后，使其更容易阅读):
{
'x1b[32mEND RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9x1b[0m',
"send(..) failed executing http.request(..): HTTPConnectionPool(host='host.docker.internal', port=5000): Max retries exceeded with url: /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eabf40>: Failed to establish a new connection: [Errno 111] Connection refused'))",
'x1b[32mSTART RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9 Version: $LATESTx1b[0m',
'',
"[WARNING]t2022-05-01T05:07:18.969Zt50fe51a1-46e2-16d2-c274-8a50604d2be9tRetrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab520>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
"[WARNING]t2022-05-01T05:07:18.968Zt50fe51a1-46e2-16d2-c274-8a50604d2be9tRetrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab1f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
'x1b[32mREPORT RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9tInit Duration: 183.50 mstDuration: 8.25 mstBilled Duration: 9 mstMemory Size: 128 MBtMax Memory Used: 27 MBtx1b[0m',
"{
'RequestType': 'Create',
'ServiceToken': 'arn:aws:lambda:us-east-1:123456789012:function:stack10d4d2-InfoFunction-M4ZP2MX8XYLL',
'ResponseURL': 'http://host.docker.internal:5000/cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'StackId': 'arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'RequestId': '046dc61f-d786-4672-b1a9-48eb286593e2',
'LogicalResourceId': 'CustomInfo',
'ResourceType': 'Custom::Info',
'ResourceProperties': {
'ServiceToken': 'arn:aws:lambda:us-east-1:123456789012:function:stack10d4d2-InfoFunction-M4ZP2MX8XYLL',
'Region': 'us-east-1',
'MyProperty': 'stuff'
}
}",
'http://host.docker.internal:5000/cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'{
"Status": "SUCCESS",
"Reason": "See the details in CloudWatch Log Stream: 2022/05/01/[$LATEST]b39c266d1989dc1bc8ec6dfecf2154d7",
"PhysicalResourceId": "2022/05/01/[$LATEST]b39c266d1989dc1bc8ec6dfecf2154d7",
"StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
"RequestId": "046dc61f-d786-4672-b1a9-48eb286593e2",
"LogicalResourceId": "CustomInfo",
"NoEcho": false,
"Data": {
"info_value": "special value"
}
}',
'Response body:',
'null',
"[WARNING]t2022-05-01T05:07:18.970Zt50fe51a1-46e2-16d2-c274-8a50604d2be9tRetrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab730>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776"
}
我认为最有趣的部分是:
"send(..) failed executing http.request(..): HTTPConnectionPool(host='host.docker.internal', port=5000): Max retries exceeded with url: /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eabf40>: Failed to establish a new connection: [Errno 111] Connection refused'))",
代码:
测试代码# Libraries
# Standard Libraries
import inspect
import json
import os
import sys
import time
from uuid import uuid4
import yaml
# Paths
PACKAGE_PARENT = ".."
SCRIPT_DIR = os.path.dirname(
os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__)))
)
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
# Third Party Librairies
import boto3
from moto import mock_cloudformation, mock_lambda, mock_logs, mock_s3, settings
import pytest
from pytest import assume
# User Defined Libraries
from .fixtures.custom_lambda import get_template
import src.app.App as lambda_function
from .test_awslambda.utilities import wait_for_log_msg
def get_log_group_name(cf, stack_name):
...
def get_outputs(cf, stack_name):
...
@pytest.fixture(scope="function")
def aws_credentials():
"""
Mocked AWS Credentials for moto.
"""
os.environ["AWS_ACCESS_KEY_ID"] = "testing"
os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
os.environ["AWS_SECURITY_TOKEN"] = "testing"
os.environ["AWS_SESSION_TOKEN"] = "testing"
os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
@mock_cloudformation
@mock_lambda
@mock_logs
@mock_s3
def test_create_custom_lambda_resource():
#########
# Integration test using a Custom Resource
# Create a Lambda
# CF will call the Lambda
# The Lambda should call CF, to indicate success (using the cfnresponse-module)
# This HTTP request will include any outputs that are now stored against the stack
# TEST: verify that this output is persisted
##########
if not settings.TEST_SERVER_MODE:
raise pytest.skip(
"Needs a standalone MotoServer, as cfnresponse needs to connect to something"
)
# Create cloudformation stack
stack_name = f"stack{str(uuid4())[0:6]}"
code = inspect.getsource(lambda_function)
template_body = get_template(code)
cf = boto3.client(
"cloudformation",
region_name="us-east-1",
endpoint_url="http://localhost:5000",
)
cf.create_stack(
StackName=stack_name,
TemplateBody=template_body,
# TemplateBody=json.dumps(template_body),
Capabilities=["CAPABILITY_IAM"],
)
# Verify CloudWatch contains the correct logs
log_group_name = get_log_group_name(cf, stack_name)
success, logs = wait_for_log_msg(
expected_msg="Status code: 200", log_group=log_group_name
)
print(f"Logs should indicate success: n{logs}")
# Verify the correct Output was returned
outputs = get_outputs(cf, stack_name)
with assume:
assert success is True
with assume:
assert len(outputs) == 1
所有这些几乎都是github页面上可用的复制粘贴，我更改代码以使用inspect从另一个文件导入函数，将其提供给cf模板的yaml版本，并传递它试图模拟自定义资源的创建
完整的wait_for_log_msg:
def wait_for_log_msg(expected_msg, log_group):
logs_conn = boto3.client("logs", region_name="us-east-1")
received_messages = []
start = time.time()
while (time.time() - start) < 30:
try:
result = logs_conn.describe_log_streams(logGroupName=log_group)
log_streams = result.get("logStreams")
except ClientError:
log_streams = None # LogGroupName does not yet exist
if not log_streams:
time.sleep(1)
continue
for log_stream in log_streams:
result = logs_conn.get_log_events(
logGroupName=log_group,
logStreamName=log_stream["logStreamName"],
)
received_messages.extend(
[event["message"] for event in result.get("events")]
)
for line in received_messages:
if expected_msg in line:
return True, set(received_messages)
time.sleep(1)
return False, set(received_messages)
是对github页面内容的复制粘贴。
默认情况下，外部连接无法访问MotoServer。用于创建CustomResource的lambdfunction是在Docker-container内部执行的，它被视为一个"外部"连接——这就是为什么你会看到NewConnectionError。
试着像这样启动MotoServer，这将打开服务器，让外部连接:
moto_server -h 0.0.0.0
(注意，我没有指定端口，因为5000是默认的-根据需要调整。)
Docker-exception与Moto无关，据我所知-查看这个问题的可能解决方案:docker.errors.DockerException:获取服务器API版本时出错