nginx proxy_pass无法从输出流中获取完整的响应数据



后端端点使用k8s-client使用以下代码来监视POD日志。

@RequestMapping(value = "/{podName}/log", method = RequestMethod.GET)
public void queryLog(HttpServletResponse response, @PathVariable(name = "podName", required = true) String podName,
@RequestParam(name = "container", required = true) String container,
@RequestParam(name = "tailLines", required = false) Integer tailLines,
@RequestParam(name = "timestamps", required = false) boolean timestamps,
@RequestParam(name = "follow", required = false) boolean follow) {
try {
PodLogs logs = new PodLogs();
InputStream is = logs.streamNamespacedPodLog(namespace, podName, container, null, tailLines, timestamps, follow);
copy(is, response.getOutputStream());
} catch (Exception ex) {
logger.error("exception", ex);
}
}
public static void copy(InputStream in, OutputStream out) throws IOException {
try {
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = in.read(buffer)) != -1) {
String str = new String(buffer, 0, bytesRead);
out.write(buffer, 0, bytesRead);
out.flush();
}
} catch (IOException ex) {
ex.printStackTrace();
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
}

:

<html>
<head>
<title>Test</title>
<script type="text/javascript">

var xhr = null;

function setConnected(connected) {

document.getElementById('connect').disabled = connected;
document.getElementById('disconnect').disabled = !connected;
document.getElementById('response').innerHTML = '';
}

function connect() {

xhr = new XMLHttpRequest()
xhr.open('GET', `http://proxy.k8s.io/backend/api/pod-0/log?container=pod&tailLines=500&timestamps=true&follow=true`, true)
xhr.onreadystatechange = () => {
setConnected(true);
if (xhr.readyState >= 3 && xhr.status === 200) {
console.log("###")
console.log(xhr.responseText)
document.getElementById('response').innerHTML = document.getElementById('response').innerHTML + xhr.responseText;
}
}
xhr.send()
}
function disconnect() {

if(xhr != null) {
xhr.abort();
}

setConnected(false);
console.log("Disconnected");
}
</script>

</head>

<body  onload="disconnect()">
<div>
<button id="connect" onclick="connect();">Connect</button>
<button id="disconnect" disabled="disabled" onclick="disconnect();">Disconnect</button>
</div>
<div id="response"></div>
</body>
</html>

我可以得到完整的日志,并观看新生成的日志没有nginx.但是,使用以下配置的nginx无法从后端获取完整的日志。

upstream backend {
server backend:8080;
keepalive_timeout 60s;
keepalive 60;
keepalive_requests 100;
}
server {
listen       80;
location / {
root   /usr/share/nginx/html;
index  index.html index.htm;
}
location /backend/api/ {
proxy_pass http://backend/;
}
}

期望的日志示例:

2021-08-02T02:51:04.857054753Z  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
2021-08-02T02:51:04.857057468Z  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
2021-08-02T02:51:04.857060467Z  at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
2021-08-02T02:51:04.857063232Z  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

我得到的示例日志:

2021-08-02T02:51:04.857054753Z  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
2021-08-02T02:51:04.857057468Z  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
2021-08-02T02:51:04.857060467Z  at org.springframework.web.filter.HiddenHt

看起来最后的数据块丢失了。当我用tailLines=200访问它时,仍然有一些数据丢失。当新的日志生成时,没有新的数据可以显示在首页。

根本原因是proxy_http_version错误。默认情况下,使用1.0版本。1.1版本应该用于keepalive连接。

文档说:

语法:proxy_http_version 1.0 | 1.1;
默认值:
proxy_http_version 1.0;
上下文:http, server, location
该指令出现在1.1.4版本。

设置代理使用的HTTP协议版本。默认版本为1.0使用。1.1版本推荐用于keepalive连接和NTLM认证。


在位置块下添加proxy_http_version 1.1;
location /backend/api/ {
proxy_http_version    1.1;
proxy_pass http://backend/;
}

这个问题可能是由于在每次迭代中刷新输出,而Nginx只转发第一个块。

这是在规格:

当缓冲区被禁用时,响应被同步地立即传递给客户端。Nginx不会尝试从代理服务器读取整个响应。

尝试在nginx中proxy_buffering on;,或头X-Accel-Buffering: yes。或者在Java中缓冲整个响应。

参见https://stackoverflow.com/a/23171377/7362396

相关内容

  • 没有找到相关文章

最新更新