在本地Azure表存储中处理批处理时遇到困难



我有这些包含超过100个实体的批处理,我试图在我的本地Azure存储帐户(Azurite)中拆分发送:

def save_results_in_database(self, operations, table_name):
table = self.table_service_client.get_table_client(table_name)
if table_name is None:
raise Exception(f"Table {table_name} does not exist !")
try:
table.submit_transaction(operations)
except Exception as e:
logging.exception(
f"Error while saving results in database for table {table_name}"
)
if 0 < len(results) < 100:
helpers.save_results_in_database(results, action)
else:
# if the number of results is greater than 100, we need to split them in batches of 50
# and save them in batches
batch_size = 50
for i in range(0, len(results), batch_size):
batch = results[i : i + batch_size]
helpers.save_results_in_database(batch, action)
sleep(10)

由于某些原因,有些批次通过了,有些批次没有通过。例如,如果我有107个实体要推送,前50个通过,后50个不通过,剩下的7个通过。

我试过睡觉,但我不确定它是否真的有帮助。下面是错误信息:

Error while saving results in database for table <table name>
File "/usr/local/lib/python3.10/site-packages/azure/data/tables/_table_client.py", line 734, in submit_transaction
return self._batch_send(self.table_name, *batched_requests.requests, **kwargs)  # type: ignore
File "/usr/local/lib/python3.10/site-packages/azure/data/tables/_base_client.py", line 334, in _batch_send
raise decoded
azure.data.tables._error.TableTransactionError: 2:An error occurred while processing this request.
ErrorCode:InvalidInput
Content: {"odata.error":{"code":"InvalidInput","message":{"lang":"en-US","value":"2:An error occurred while processing this request.nRequestId:994b1eca-1109-4ff9-aa56-0dc73ae97a18nTime:2023-04-06T13:08:29.278Z"}}}

我错过了什么吗?因为我相当确定输入在模式方面是有效的。TIA

事实证明我错了,其中一个rowkey有"/"在其中,破坏了操作。一定要检查你的数据!

相关内容

最新更新