从云函数向谷歌云日志写入结构化日志时发生类型错误



我正试图将一个云函数中的日志写入云日志,但我在执行此操作时遇到了问题。我的代码如下,其中的消息是这样的日志:;20210319-11:39:58-CET-INFO-这是一条信息消息";而attributions只是字符串。

import base64
import json
import os
import pytz
from datetime import datetime
from google.cloud import logging
from google.cloud.logging_v2.resource import Resource
RESOURCE_TYPE = "cloudiot_devices"
PROJECT = os.environ.get("GCP_PROJECT")
DEVICE_REGISTRY = REGISTRY_ID
LOCATION = LOCATION
DATEFORMAT = "%Y%m%d-%H:%M:%S"
def iot_logger(event, context):
"""Triggered from a message on a Cloud Pub/Sub topic.
Args:
event (dict): Event payload.
context (google.cloud.functions.Context): Metadata for the event.
"""
log = base64.b64decode(event['data']).decode('utf-8')
attributes = event['attributes']
print(log)
print(json.dumps(attributes))
print("preparing timestamp")
# Extract elements from log
date_str,severity,message = log.split(" - ")
# This is the main content for the log
log_content = {"message":message}
# turn log timestamp str to an aware datetime object for our timestamp
tzname = date_str.split("-")[-1]
# remove time zone to avoid tripping up the datetime module
date_str = date_str[:-4]
date_unaware = datetime.strptime(date_str,DATEFORMAT)
date_aware = date_unaware.astimezone(pytz.timezone(tzname))
print(f"done preparing timestamp: {date_aware}")
# extract attributes
logger_name = attributes["logger_name"]
device_num_id = attributes["device_num_id"]
print("creating resource")
# create resource 
resource_labels = {
'project_id':PROJECT,
'device_num_id':device_num_id,
'device_registry_id':DEVICE_REGISTRY,
'location':LOCATION
}
log_resource = Resource(type=RESOURCE_TYPE,labels=resource_labels)
print("initiating client")
# initiate logging client
log_client = logging.Client()
logger = log_client.logger(logger_name)
print("creating structured log")
logger.log_struct(log_content,severity=severity,resource=log_resource,timestamp=date_aware)

然而,我一直面临log_struct命令的问题,我一直无法弄清楚问题出在哪里。最初我向时间戳传递一个字符串,但它不起作用,所以现在我传递一个日期对象。但现在的错误不同了:编辑:不止一个:

Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 589, in _ConvertFieldValuePair
self.ConvertMessage(value, sub_message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 485, in ConvertMessage
self._ConvertFieldValuePair(value, message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 603, in _ConvertFieldValuePair
raise ParseError('Failed to parse {0} field: {1}.'.format(name, e))
google.protobuf.json_format.ParseError: Failed to parse labels field: expected string or bytes-like object. 
Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 589, in _ConvertFieldValuePair
self.ConvertMessage(value, sub_message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 485, in ConvertMessage
self._ConvertFieldValuePair(value, message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 603, in _ConvertFieldValuePair
raise ParseError('Failed to parse {0} field: {1}.'.format(name, e))
google.protobuf.json_format.ParseError: Failed to parse labels field: expected string or bytes-like object. 
"Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/functions_framework/__init__.py", line 149, in view_func
function(data, context)
File "/workspace/main.py", line 64, in iot_logger
logger.log_struct(log_content,severity=severity,resource=log_resource,timestamp=date_aware)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/logging_v2/logger.py", line 180, in log_struct
self._do_log(client, StructEntry, info, **kw)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/logging_v2/logger.py", line 133, in _do_log
client.logging_api.write_entries([api_repr])
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/logging_v2/_gapic.py", line 140, in write_entries
log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/logging_v2/_gapic.py", line 140, in <listcomp>
log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/logging_v2/_gapic.py", line 507, in _log_entry_mapping_to_pb
ParseDict(mapping, entry_pb)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 454, in ParseDict
parser.ConvertMessage(js_dict, message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 485, in ConvertMessage
self._ConvertFieldValuePair(value, message)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/protobuf/json_format.py", line 597, in _ConvertFieldValuePair
raise ParseError('Failed to parse {0} field: {1}.'.format(name, e))
google.protobuf.json_format.ParseError: Failed to parse resource field: Failed to parse labels field: expected string or bytes-like object.."

它说它需要一个字符串或类似字节的对象,但我不确定在哪里。我想这与日期无关,因为当它是字符串时,它会给我一个错误。还有我认为应该有效的资源,我正在看一个关于这个stackoverflow问题的非常相似的例子,但它对我不起作用:

谷歌云功能Python日志问题

我做错了什么?

我在@jabbson的帮助下解决了这个问题。我遇到了一些问题,但主要是我在没有意识到的情况下给了它一些错误的东西

我使用的环境变量如下所示:https://cloud.google.com/functions/docs/env-var#nodejs_8_python_37_and_go_111但由于某种原因,我的项目id为null。此外,我的resource_type中也有一个拼写错误。

最后,我不得不摆弄时间戳,直到它真正像我希望的那样工作。LogEntry文档说它是一个字符串,但在这种情况下,我必须给它传递一个日期时间对象,并将本地时间转换为UTC,这样我才能在云日志中找到它。

相关内容

最新更新