Azure 函数 HttpClient 不给出任何响应，并不断重试

Question

我有一个在Azure中运行的功能应用程序。它是一个HttpTrigger，它是由流分析触发的，重试策略设置为Drop。它是用c#编写的，并通过CI/CD从DevOps部署。以下是一些配置。

FUNCTIONS_EXTENSION_VERSION: ~4
FUNCTIONS_WORKER_RUNTIME: dotnet-isolated
WEBSITE_RUN_FROM_PACKAGE: 1

该函数的目的是获取一些数据，并使用HttpClient(通过HttpClientFactory创建，超时时间为15秒)将其转发到不同服务器上的各种其他端点。到目前为止，一切都很顺利，直到最近在新服务器上添加了一个新端点。这个问题在某种程度上与这个新服务器有关，但我很难从我的端得到任何有用的日志，而且整个函数在发送数据时表现得很奇怪。下面是代码的结尾:

log.LogInformation($"Preparing to send...");
var content = new StringContent(jsonContent, Encoding.UTF8, "application/json");
var httpRequestMessage = new HttpRequestMessage
{
Method = HttpMethod.Post,
RequestUri = new Uri(uri),
Headers = {
{ "Authorization", $"Bearer {token}" },
{ "Accept", "*/*" },
},
Content = content
};
var response = await _httpClient.SendAsync(httpRequestMessage);
var responseString = await response.Content.ReadAsStringAsync();
log.LogInformation($"Returned content: {responseString}.");

我使用Application Insights进行监控。uri参数根据发送数据的位置不同而不同，它适用于除1个端点以外的任何地方。当我试图发送到这个"错误"时;端点，这是我在应用程序洞察中看到的:

01:27:37 PM Trace: Preparing to send...
01:27:37 PM Trace: Start processing HTTP request POST https://... (URI parameter)
01:27:37 PM Trace: Sending HTTP request POST https://... (URI Parameter)
01:27:37 PM Trace: End processing HTTP request after 188.05ms - OK
01:27:37 PM Trace: Received HTTP response after 187.92ms - OK
01:29:17 PM Exception: Exception while executing function: Functions.MyFunction
01:29:17 PM Trace:Executed 'Functions.MyFunction' (Failed, Id=3d9afe5b-c58a-470d-8f54-0d98ca60deb7, Duration=100005ms
01:29:17 PM Exception: Exception while executing function: Functions.MyFunction

当我检查异常时，它说:

Exception while executing function: Functions.MyFunction Result: Failure
Exception: System.AggregateException: One or more errors occurred. (A task was canceled.)
---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Threading.Tasks.Task.GetExceptions(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:a1ssrcDotNetWorker.CoreInvocationDefaultFunctionInvoker.cs:line 32
at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
at System.Threading.Tasks.Task.ExecuteFromThreadPool(Thread threadPoolThread)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()
--- End of stack trace from previous location ---
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:a1ssrcDotNetWorker.CoreInvocationDefaultFunctionInvoker.cs:line 32
at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:a1ssrcDotNetWorker.CoreInvocationDefaultFunctionExecutor.cs:line 45
at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:a1ssrcDotNetWorker.CoreOutputBindingsOutputBindingsMiddleware.cs:line 16
at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:a1ssrcDotNetWorker.GrpcGrpcWorker.cs:line 167
Stack:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionInvoker`2.<>c.<InvokeAsync>b__6_0(Task`1 t) in D:a1ssrcDotNetWorker.CoreInvocationDefaultFunctionInvoker.cs:line 32
at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:a1ssrcDotNetWorker.CoreInvocationDefaultFunctionExecutor.cs:line 45
at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:a1ssrcDotNetWorker.CoreOutputBindingsOutputBindingsMiddleware.cs:line 16
at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:a1ssrcDotNetWorker.GrpcGrpcWorker.cs:line 167 

然后似乎函数重试Http请求，我不明白为什么。"准备发送"日志不存在，但"开始处理"&;日志每2分钟出现一次，失败一次。由于我的自定义日志消息不在这里，我认为它是HttpClient正在以某种方式重试，而不是整个函数。

有人知道发生了什么事吗?为什么我的代码在读取response.Content时会卡住?我如何进一步调查出了什么问题，为什么HttpClient像这样重试自己?如果HttpClient失败，我希望函数崩溃/停止。

编辑:更正日志上的时间戳以匹配持续时间。有多个日志同时出现，我选错了。

Answer 1

很难帮忙，因为我们无法访问您的环境，但我想试试:

在我看来，日志的问题是因为它是一个异步操作，这个日志不代表内部/真正的异常。我建议您切换到同步操作并发送到此故障端点。一旦你得到真正的异常/适当的修复，你切换回异步操作。

Answer 2

从Thiago的提示更改异步，我收到了一个适当的异常，这帮助我把碎片放在一起。

新的错误:

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

我使用2个不同的httpclient，一个用于我的错误，另一个用于定时令牌更新程序。似乎我的第一个问题是我将15秒超时添加到错误的客户端(doh)。有了这个新信息，100005ms的持续时间就更有意义了。

接下来，HttpClient的超时异常没有被捕获，导致整个函数崩溃，产生"A task被取消";错误。

最后，它一直重启的原因似乎是由Azure流分析引起的。仔细看一下错误策略文档，似乎尽管设置为"Drop"，但在某些情况下，它仍然会根据错误重试。

我仍然有一个问题，为什么我的请求超时，但我原来的问题是，到底发生了什么是回答:)