为什么即使主机上有可用内存,kubernete pod也会报告"内存不足"



我正在MacOS上运行minikube v1.15.1并安装了helm v3.4.1。我运行helm install elasticsearch elastic/elasticsearch --set resources.requests.memory=2Gi --set resources.limits.memory=4Gi --set replicas=1在k8s集群上安装elasticsearch。吊舱elasticsearch-master-0已部署,但处于pending状态。

当我运行kubectl describe pod elasticsearch-master-0时,它会给我以下警告:


Warning  FailedScheduling  61s (x2 over 2m30s)  default-scheduler  0/1 nodes are available: 1 Insufficient memory.

它采用Insufficient memory,但我的主机至少有4GB的可用内存。内存问题是否意味着minikube内存不足?如果是,我如何增加它的记忆力?

我在minikube中增加了内存,并重新启动了minikube,但仍然有同样的问题。

我确实运行了minikube delete,然后是minikube start。你可以看到下面的输出,它使用4个CPU和8GB内存

minikube v1.15.1 on Darwin 11.0.1
✨  Automatically selected the docker driver. Other choices: hyperkit, virtualbox
👍  Starting control plane node minikube in cluster minikube
🔥  Creating docker container (CPUs=4, Memory=8096MB) ...
🐳  Preparing Kubernetes v1.19.4 on Docker 19.03.13 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

下面是从配置中获取cpu和内存的代码。

$ minikube config get cpus
4
$ minikube config get memory
8096

以下是metrics-server的输出。

$ kubectl top node
NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
minikube   466m         5%     737Mi           37%
$ kubectl top pod
W1125 20:11:23.232025   46719 top_pod.go:265] Metrics not available for pod default/elasticsearch-master-0, age: 34m3.231199s
error: Metrics not available for pod default/elasticsearch-master-0, age: 34m3.231199s

kubectl describe pod的全部输出为:

$ kubectl describe pod elasticsearch-master-0
Name:           elasticsearch-master-0
Namespace:      default
Priority:       0
Node:           <none>
Labels:         app=elasticsearch-master
chart=elasticsearch
controller-revision-hash=elasticsearch-master-677c65788d
release=elasticsearch
statefulset.kubernetes.io/pod-name=elasticsearch-master-0
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  StatefulSet/elasticsearch-master
Init Containers:
configure-sysctl:
Image:      docker.elastic.co/elasticsearch/elasticsearch:7.10.0
Port:       <none>
Host Port:  <none>
Command:
sysctl
-w
vm.max_map_count=262144
Environment:  <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-kthrd (ro)
Containers:
elasticsearch:
Image:       docker.elastic.co/elasticsearch/elasticsearch:7.10.0
Ports:       9200/TCP, 9300/TCP
Host Ports:  0/TCP, 0/TCP
Limits:
cpu:     1
memory:  4Gi
Requests:
cpu:      1
memory:   2Gi
Readiness:  exec [sh -c #!/usr/bin/env bash -e
# If the node is starting up wait for the cluster to be ready (request params: "wait_for_status=green&timeout=1s" )
# Once it has started only check that the node itself is responding
START_FILE=/tmp/.es_start_file
# Disable nss cache to avoid filling dentry cache when calling curl
# This is required with Elasticsearch Docker using nss < 3.52
export NSS_SDB_USE_CACHE=no
http () {
local path="${1}"
local args="${2}"
set -- -XGET -s
if [ "$args" != "" ]; then
set -- "$@" $args
fi
if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
set -- "$@" -u "${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
fi
curl --output /dev/null -k "$@" "http://127.0.0.1:9200${path}"
}
if [ -f "${START_FILE}" ]; then
echo 'Elasticsearch is already running, lets check the node is healthy'
HTTP_CODE=$(http "/" "-w %{http_code}")
RC=$?
if [[ ${RC} -ne 0 ]]; then
echo "curl --output /dev/null -k -XGET -s -w '%{http_code}' ${BASIC_AUTH} http://127.0.0.1:9200/ failed with RC ${RC}"
exit ${RC}
fi
# ready if HTTP code 200, 503 is tolerable if ES version is 6.x
if [[ ${HTTP_CODE} == "200" ]]; then
exit 0
elif [[ ${HTTP_CODE} == "503" && "7" == "6" ]]; then
exit 0
else
echo "curl --output /dev/null -k -XGET -s -w '%{http_code}' ${BASIC_AUTH} http://127.0.0.1:9200/ failed with HTTP code ${HTTP_CODE}"
exit 1
fi
else
echo 'Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" )'
if http "/_cluster/health?wait_for_status=green&timeout=1s" "--fail" ; then
touch ${START_FILE}
exit 0
else
echo 'Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" )'
exit 1
fi
fi
] delay=10s timeout=5s period=10s #success=3 #failure=3
Environment:
node.name:                     elasticsearch-master-0 (v1:metadata.name)
cluster.initial_master_nodes:  elasticsearch-master-0,
discovery.seed_hosts:          elasticsearch-master-headless
cluster.name:                  elasticsearch
network.host:                  0.0.0.0
ES_JAVA_OPTS:                  -Xmx1g -Xms1g
node.data:                     true
node.ingest:                   true
node.master:                   true
node.remote_cluster_client:    true
Mounts:
/usr/share/elasticsearch/data from elasticsearch-master (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-kthrd (ro)
Conditions:
Type           Status
PodScheduled   False
Volumes:
elasticsearch-master:
Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName:  elasticsearch-master-elasticsearch-master-0
ReadOnly:   false
default-token-kthrd:
Type:        Secret (a volume populated by a Secret)
SecretName:  default-token-kthrd
Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type     Reason            Age                From               Message
----     ------            ----               ----               -------
Warning  FailedScheduling  76s (x2 over 77s)  default-scheduler  0/1 nodes are available: 1 Insufficient memory.

mac上的Minikube使用虚拟机来托管kubernetes。这与主机是分开的,并限制了单节点集群的可用内存。

您可以使用为虚拟机配置更多内存

minikube start --memory=4096

Minikube将在第一次启动时获取内存设置,但如果您之前启动时没有该选项,则需要执行minikube delete并重新启动。

要检查吊舱/节点正在使用的资源,可以使用minikube addons:启用metrics-server

➜  ~ minikube addons enable metrics-server 
🌟  The 'metrics-server' addon is enabled

您将不得不等待一段时间,等待指标出现:

➜  ~ kubectl top node 
NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
minikube   186m         4%     2344Mi          15%       
➜  ~ kubectl top pod 
NAME                     CPU(cores)   MEMORY(bytes)   
elasticsearch-master-0   6m           1272Mi   

最新更新