io.fabric8.kubernetes.client.KuberneteClientException:70000毫



从AKS Jenkins Agent pod获取此错误。知道这个错误的原因是什么吗?我做的故障排除步骤。

将Jenkins恢复到旧版本=>导致相同的错误将Jenkins升级到所有新版本,包括使用中的插件=>导致相同的错误。根据github中的一些建议,将Jenkins K8s和K8s API插件降级到稳定版本=>相同错误创建了全新的集群并安装Jenkins和Job pod,开始给出相同的错误=>相同错误

如何解决此问题?

18:23:33  [Pipeline] // podTemplate
18:23:33  [Pipeline] End of Pipeline
18:23:33  io.fabric8.kubernetes.client.KubernetesClientException: not ready after 5000 MILLISECONDS
18:23:33    at io.fabric8.kubernetes.client.utils.Utils.waitUntilReadyOrFail(Utils.java:176)
18:23:33    at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:322)
18:23:33    at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.exec(PodOperationsImpl.java:84)
18:23:33    at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:413)
18:23:33    at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:330)
18:23:33    at hudson.Launcher$ProcStarter.start(Launcher.java:507)
18:23:33    at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:176)
18:23:33    at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:132)
18:23:33    at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:324)
18:23:33    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:319)
18:23:33    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:193)
18:23:33    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
18:23:33    at jdk.internal.reflect.GeneratedMethodAccessor6588.invoke(Unknown Source)
18:23:33    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
18:23:33    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
18:23:33    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
18:23:33    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
18:23:33    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
18:23:33    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
18:23:33    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
18:23:33    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
18:23:33    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
18:23:33    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
18:23:33    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
18:23:33    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
18:23:33    at WorkflowScript.run(WorkflowScript:114)
18:23:33    at ___cps.transform___(Native Method)
18:23:33    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
18:23:33    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
18:23:33    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
18:23:33    at jdk.internal.reflect.GeneratedMethodAccessor210.invoke(Unknown Source)
18:23:33    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
18:23:33    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
18:23:33    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
18:23:33    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
18:23:33    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
18:23:33    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
18:23:33    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
18:23:33    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
18:23:33    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
18:23:33    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
18:23:33    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
18:23:33    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
18:23:33    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)

编辑:似乎没有任何区别。仍然有5000毫秒的超时,所以不确定这个方法是否有效(至少使用环境变量(。如果你真的能够改变超时,它可能会起作用,但我还没有弄清楚。


在仅更新Jenkins(和插件(后开始看到相同的问题,而不是K8S集群。

但我得到的是5000毫秒,而不是7000毫秒。

io.fabric8.kubernetes.client.KubernetesClientException: not ready after 5000 MILLISECONDS

深入研究Github上的堆栈和源代码会导致默认超时(6年来一直没有改变(,所以不知何故,你似乎有一个非默认超时。

public static final Long DEFAULT_WEBSOCKET_TIMEOUT = 5 * 1000L;

它似乎可以被pod上的环境变量KUBERNETES_WEBSOCKET_TIMEOUT_SYSTEM_PROPERTY覆盖。我只是试着把我的时间提高到10秒,看看这是否有什么不同。

也许值得一试。如果是这样的话,这可能表明集群API服务器在某种程度上比预期的响应慢。在我的情况下,在升级期间,我不知道有什么会影响集群性能,而且由于默认超时多年来一直没有改变,这似乎很奇怪。也许有些代码是以某种方式重构的——不再忽略超时/重试等——只是猜测。

编辑:我正在裸机集群上运行

我的解决方案是在控制器(Jenkins(服务行命令中添加-Dkubernetes.websocket.timeout=60000 -Dorg.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.websocketConnectionTimeout=60000这项工作为插件的最后一个版本也是

最新更新