Python with Azure blob service



我正在尝试使用 Azure blob 服务将视频文件上传到云。

我试图弄清楚如果我的互联网在传输过程中突然熄灭会发生什么。

当互联网出去时,似乎没有抛出的例外。

from azure.common import AzureException
from azure.storage.blob import AppendBlobService, BlockBlobService, ContentSettings
try:
self.append_blob_service.append_blob_from_path(self.container_name, blob_name, upload_queue.get(timeout=3))
except AzureException as ae:
print("hey i caught something") <-- this line never seem to run

如果我把互联网放回 blob 上,似乎会在大约 30 分钟后自行上传。我在文档中找不到有关此的任何信息。append_blob_from_path功能会持续尝试多长时间?

有 LinearRetry、ExponentialRetry、NoRetry 和自定义重试策略。

默认值为线性,最多 5 次尝试间隔 5 秒。因此,如果您的网络连接中断了 <25 秒,您的上传将继续。

我不确定您的互联网连接是否中断了 30 分钟。在这种情况下,它应该抛出并异常。

PS:您可以查找相应的 C# 文档以获取重试策略。

Python SDK for Azure Storage 是 OpenSource : https://github.com/Azure/azure-storage-python

如果我们查看来自append_blob_from_path()的电话,我们可以看到以下内容:

  1. 有一个默认的套接字超时:
# Socket timeout in seconds
DEFAULT_SOCKET_TIMEOUT = 20
  1. 最后它使用StorageClient(AppendBlobService(BaseBlobService) -> BaseBlobService(StorageClient)(中的函数,StorageClient使用:
self.retry = ExponentialRetry().retry
  1. >ExponentialRetry具有以下构造函数:
def __init__(self, initial_backoff=15, increment_base=3, max_attempts=3,
retry_to_secondary=False, random_jitter_range=3):
'''
Constructs an Exponential retry object. The initial_backoff is used for 
the first retry. Subsequent retries are retried after initial_backoff + 
increment_power^retry_count seconds. For example, by default the first retry 
occurs after 15 seconds, the second after (15+3^1) = 18 seconds, and the 
third after (15+3^2) = 24 seconds.
:param int initial_backoff: 
The initial backoff interval, in seconds, for the first retry.
:param int increment_base:
The base, in seconds, to increment the initial_backoff by after the 
first retry.
:param int max_attempts: 
The maximum number of retry attempts.
:param bool retry_to_secondary:
Whether the request should be retried to secondary, if able. This should 
only be enabled of RA-GRS accounts are used and potentially stale data 
can be handled.
:param int random_jitter_range:
A number in seconds which indicates a range to jitter/randomize for the back-off interval.
For example, a random_jitter_range of 3 results in the back-off interval x to vary between x+3 and x-3.
'''
  1. 还有一个RetryContext,这个 _retry(( 函数使用它来确定是否需要重试

如果在代码中启用 INFO 级别日志记录,您将看到所有重试:

# Basic configuration: configure the root logger, including 'azure.storage'
logging.basicConfig(format='%(asctime)s %(name)-20s %(levelname)-5s %(message)s', level=logging.INFO)

要恢复:

您有(20 秒的套接字超时 + 动态间隔从 15 秒开始,每次尝试随机递增(,并且您有 3 次尝试。您可以看到启用 INFO 级别日志记录时究竟发生了什么。

最新更新