提高 Blob 上传性能



我正在努力将c ++应用程序与Azure blob存储集成。为了实现这一点,我围绕wastorage 4.0 API实现了一个包装类。

#include "stdafx.h"
#include "AzureStorage.h"
// Microsoft Azure Library Header Includes.
#include "wasstorage_account.h"
#include "wasblob.h"
struct RadAzureData{
    azure::storage::cloud_storage_account storage_account;
    azure::storage::cloud_blob_client blob_client;
    azure::storage::cloud_blob_container container;
};
   RadAzureStorage::RadAzureStorage():
   RadCloudStorageInterface(RAD_STORAGE_TYPE::AZURE_CLOUD)
   {
   }
   RadAzureStorage::RadAzureStorage(std::string accountName1, std::string     accountKey1, std::string containerName1) :        RadCloudStorageInterface(RAD_STORAGE_TYPE::AZURE_CLOUD)
   {
       std::wstring accountNameWS(accountName1.begin(), accountName1.end());
       std::wstring accountKeyWS(accountKey1.begin(), accountKey1.end());
       std::wstring containerNameWS(containerName1.begin(), containerName1.end());
       d = new RadAzureData();
       accountName = accountNameWS;
       accountKey = accountKeyWS;
       containerName = containerNameWS;
       std::wstring connStr1 = L"AccountName=" + accountName + L";AccountKey=" + accountKey + L";DefaultEndpointsProtocol=https";
       d->storage_account =        azure::storage::cloud_storage_account::parse(connStr1.c_str());
       // Create a blob container
       d->blob_client = d->storage_account.create_cloud_blob_client();
       d->container = d->blob_client.get_container_reference(containerName.c_str());
       CreateContainer();
   }

   bool RadAzureStorage::CreateContainer()
   {
       try
       {
           d->container.create_if_not_exists();
       }
       catch (const azure::storage::storage_exception& e)
       {
          cout<<"Exception in container creation: " << e.what()<<endl;
          cout <<"The request that started at:" << e.result().start_time().to_string().c_str() << " and ended at " << e.result().end_time().to_string().c_str() << " resulted in HTTP status code " << e.result().http_status_code() << " and the request ID reported by the server was " << e.result().service_request_id().c_str()<<endl;
          return false;
       }
    return true;
   }
   bool RadAzureStorage::UploadFile(std::string blockBlobName, std::string dicomFileLocation)
   {
       std::wstring blockBlobNameWS(blockBlobName.begin(), blockBlobName.end());
       std::wstring dicomFileLocationWS(dicomFileLocation.begin(), dicomFileLocation.end());
       // Create a Block Blob Object.
       azure::storage::cloud_block_blob block_blob = d->container.get_block_blob_reference(blockBlobNameWS.c_str());
       // Upload Block Blob to container.
       try
       {
           block_blob.upload_from_file(dicomFileLocationWS.c_str());
       }
       catch (const azure::storage::storage_exception& e)
       {
           cout<< "Exception in file upload: " << e.what() << endl;
           cout<< "The request that started at:" << e.result().start_time().to_string().c_str() << " and ended at " << e.result().end_time().to_string().c_str() << " resulted in HTTP status code " << e.result().http_status_code() << " and the request ID reported by the server was " << e.result().service_request_id().c_str() << endl;
           return false;
       }
       return true;
   }
   #undef __FILENAME__

从应用程序中,它实例化RadAzureStorage类并调用UploadFile API。

   RadAzureStorage* clsi = new RadAzureStorage(accountname, acesskey,        containername);
   <<timer.start>>
   clsi->UploadFile(blockBlobName, file);
   <<timer.end>>
   cout << timer.ellapsedMilliSeconds<< "ms"<< endl;

上传文件 API 需要 14-16 毫秒,文件大小范围在 190-250 KB 之间。

是否有一些关于 wastorage 初始化的参数可以修改以在 10 毫秒内实现此目的。

当前测试环境托管在 Azure 印度南部:1. 虚拟机:视窗服务器 2016,4v 核心,16 GB 内存。2. 热层访问、存储帐户。

请注意:在 C# 上实现了类似的逻辑,对于同一数据集,每个文件的上传时间不到 10 毫秒。

此延迟也可能由使用的存储帐户类型引起。使用高级存储帐户时,与 GPV1 相比,可以获得更高的性能。

还有一个新产品仍在 Azure 高级 blob 中。高级存储帐户存储在 SSD 上,而不是 HDD 上,因此在使用 GPV2 高级版时也能获得更好的性能。

最新更新