Azure函数运行到System.Net.Sockets.SocketException中

Question

我有一个AzureFunctions应用程序，它有两个HTTP触发函数。两者都来自同一个类，但使用不同的url来获取数据。Azure数据工厂管道每天触发第一个HTTP函数另一个管道在1分钟内调用第二个函数

每个函数向第三方网站发出大约1300个HTTP请求，并将每个响应作为单独的json文件存储在Blob存储中。

问题几乎是每次(但并非总是)第二个函数抛出System.Net.Sockets.SocketException时，因为很少有出站请求会运行到公共的21秒TCP超时。我注意到了一件奇怪的事情——Azure可能出于某种原因抑制了我的出站请求：第一批需要300毫秒，下一个序列需要4.3秒，然后9.5秒，下一批达到21秒，除外

以下是出站请求定时增加的图像

异常堆栈跟踪：

System.Net.Http.HttpRequestException：连接尝试失败因为关联方在时间，或建立的连接失败，因为连接的主机未能响应--->System.Net.Sockets.SocketException:A由于连接方不正确，连接尝试失败一段时间后响应，或建立的连接失败因为连接的主机无法在响应System.Net.Http.ConnectHelper.ConnectAsync(字符串主机，Int32端口，CancellationToken CancellationToken)---内部异常结束堆栈跟踪--在System.Net.Http.ConnectHelper.ConnectAsync(字符串主机，Int32端口，CancellationToken取消令牌)位于的System.Threading.Tasks.ValueTask1.get_Result() at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Threading.Tasks.ValueTask1.get_Result()System.Net.Http.HttpConnectionPool.WWaitForCreatedConnectionAsync(ValueTask1 creationTask) at System.Threading.Tasks.ValueTask1.get_Result()
位于System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessagerequest，Boolean doRequestAuth，CancellationToken cancellionToken)
(HttpRequestMessage请求，CancellationToken取消令牌)System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage请求，CancellationToken取消令牌)System.Net.Http.HttpClient.FinishSendAsyncBuffered(任务`1 sendTask，HttpRequestMessage请求，CancellationTokenSource cts，布尔值disposeCts)FunctionApp.BaseFunc.<>c_DisplayClass7_2。<b_0>d.移动下一个()在里面E： \vsts-agent-win-1_work\339\s\Services\Host\Controllers\BaseFunc.cs:line102---在FunctionApp.BaseFunc.ProcessRun(ILoggerlog，字符串runId)E： \vsts-agent-win-1_work\339\s\Services\Host\Controllers\BaseFunc.cs:line122.

FunctionApp托管在AppService计划S1上，因此出站连接不限制为600(我相信是这样)

异常期间TCP连接的度量(最大值为498)：AzureFunction应用的度量

AzureFunction应用程序"解决问题"助手的TCP连接所有状态下的最大TCP连接数为502

异常期间应用程序服务计划的CPU和内存：应用程序服务计划指标

应用程序是.Net Core 2.2

我没能在我的本地电脑上复制这一点。但在Azure上，它几乎每天都会在每个环境(开发、测试、生产)中发生。失败后，Azure数据工厂在5分钟内重试，每次都是成功的。

以下是两个函数都使用的基类代码：

public abstract class BaseFunc
{
protected abstract string BlobFolderName { get; }
protected TelemetryClient telemetryClient;
private static HttpClient _httpClient;
static BaseFunc()
{
HttpClientHandler handler = new HttpClientHandler();
handler.MaxConnectionsPerServer = 300;
_httpClient = new HttpClient(handler);
}
protected async Task ProcessRun(ILogger log, string runId)
{
int processedItems = 0;
try
{
Stopwatch sw = Stopwatch.StartNew();
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
await Authentication("url", log, runId); //sets default Authorization header
string getIdeaResult = await _httpClient.GetStringAsync("url");
JObject jsonObject = JObject.Parse(getIdeaResult);
int ideaCount = (int)jsonObject.SelectToken("total_count");
List<Task> tasks = new List<Task>();
string DataPulledDate = DateTime.Now.ToString("dd-MMM-yyyy");
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("connection string");
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("container");
string getIdsUri = "url" + $"&limit={batchSize}&offset=";
int iterations = (int)Math.Ceiling((decimal)ideaCount/batchSize);
for (int i = 0; i < iterations; i++)
{
string result = await _httpClient.GetStringAsync("url" + i * 50);
JObject jsonIdsObject = JObject.Parse(result);
int[] ideaIds = jsonIdsObject["content"].Children().Values<int>("id").ToArray();
foreach (int id in ideaIds)
{
tasks.Add(Task.Run(async () =>
{
string content = null;
using (var response = await _httpClient.SendAsync(new HttpRequestMessage(HttpMethod.Get, "url"+ id))) //Exception is thrown on this line
{
content = await response.Content.ReadAsStringAsync();
response.EnsureSuccessStatusCode();
}
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference($"{DataPulledDate}/{BlobFolderName}/ideaId-{id}.json");
await cloudBlobContainer.CreateIfNotExistsAsync();
await cloudBlockBlob.UploadTextAsync(content);
Interlocked.Increment(ref processedItems);
}));
}
}
await Task.WhenAll(tasks);
sw.Stop();
}
catch (Exception ex)
{
log.LogError(ex, "{RunId}: Run failed. {Items} items processed successfully, Exception: {Exception}.", runId, processedItems, ex.ToString());
throw;
}
finally
{
if (telemetryClient != null)
{
telemetryClient.Flush();
Thread.Sleep(3000);
}
}
}
}

功能代码本身：

namespace FunctionApp
{
public class GetIdeas : BaseFunc
{
public GetIdeas(TelemetryClient telemetryClient)
{
this.telemetryClient = telemetryClient;
}
protected override string BlobFolderName { get => "folder"; }
protected override string GetItemUrl { get => "url"; }
[FunctionName("GetIdeasFn")]
public async Task Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req, ILogger log)
{
await ProcessRun(log, $"GetIdeasFn - {DateTime.UtcNow.Ticks}");
}
}
}

感谢您的帮助。

Answer 1

我也遇到了同样的问题，但在我的案例中，有一个代码补丁具有长循环机制，它向Microsoft GraphApi创建了数百个请求，而没有响应一个请求，它却创建了另一个请求。已更正&问题已解决！