来自自定义导出程序的Prometheus度量显示在/metrics中,但不显示在/graph(k8s)中



我在golang中编写了一个名为"我的节点导出器";与一些收集器一起显示度量。在我的集群中,我可以通过以下方式查看我的指标:

kubectl port-forward my-node-exporter-999b5fd99-bvc2c 9090:8080 -n kube-system
localhost:9090/metrics

然而,当我试图在prometheus仪表板中查看我的指标时

kubectl port-forward prometheus-prometheus-operator-158978-prometheus-0 9090
localhost:9090/graph

我的指标找不到,只能看到默认指标。我是否错过了在图表上获取指标的步骤?


以下是我默认命名空间中的pod,其中包含我的prometheus内容。

pod/alertmanager-prometheus-operator-158978-alertmanager-0            2/2     Running   0          85d
pod/grafana-1589787858-fd7b847f9-sxxpr                                1/1     Running   0          85d
pod/prometheus-operator-158978-operator-75f4d57f5b-btwk9              2/2     Running   0          85d
pod/prometheus-operator-1589787700-grafana-5fb7fd9d8d-2kptx           2/2     Running   0          85d
pod/prometheus-operator-1589787700-kube-state-metrics-765d4b7bvtdhj   1/1     Running   0          85d
pod/prometheus-operator-1589787700-prometheus-node-exporter-bwljh     1/1     Running   0          85d
pod/prometheus-operator-1589787700-prometheus-node-exporter-nb4fv     1/1     Running   0          85d
pod/prometheus-operator-1589787700-prometheus-node-exporter-rmw2f     1/1     Running   0          85d
pod/prometheus-prometheus-operator-158978-prometheus-0                3/3     Running   1          85d

我用舵杆安装普罗米修斯操纵器。

编辑:添加我的yaml文件

# Configuration to deploy
#
# example usage: kubectl create -f <this_file>
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-node-exporter-sa
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: my-node-exporter-binding
subjects:
- kind: ServiceAccount
name: my-node-exporter-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: my-node-exporter-role
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: my-node-exporter-role
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
---
#####################################################
############  Service ############
#####################################################
kind: Service
apiVersion: v1
metadata:
name: my-node-exporter-svc
namespace: kube-system
labels:
app: my-node-exporter
spec:
ports:
- name: my-node-exporter
port: 8080
targetPort: metrics
protocol: TCP
selector:
app: my-node-exporter
---
#########################################################
############   Deployment  ############
#########################################################
kind: Deployment
apiVersion: apps/v1
metadata:
name: my-node-exporter
namespace: kube-system
spec:
selector:
matchLabels:
app: my-node-exporter
replicas: 1
template:
metadata:
labels:
app: my-node-exporter
spec:
serviceAccount: my-node-exporter-sa
containers:
- name: my-node-exporter
image: locationofmyimagehere
args:
- "--telemetry.addr=8080"
- "--telemetry.path=/metrics"
imagePullPolicy: Always
ports:
- containerPort: 8080
volumeMounts:
- name: log-dir
mountPath: /var/log
volumes:
- name: log-dir
hostPath:
path: /var/log

服务监视器yaml

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: my-node-exporter-service-monitor
labels:
app: my-node-exporter-service-monitor
spec:
selector:
matchLabels:
app: my-node-exporter
matchExpressions:
- {key: app, operator: Exists}
endpoints:
- port: my-node-exporter
namespaceSelector:
matchNames:
- default
- kube-system

普罗米修斯山芋

# Prometheus will use selected ServiceMonitor
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: my-node-exporter
labels:
team: frontend
spec:
serviceMonitorSelector:
matchLabels:
app: my-node-exporter
matchExpressions:
- key: app
operator: Exists

您需要明确地告诉Prometheus要收集什么指标,以及从哪里收集——首先创建一个指向您的my-node-exporter吊舱的Service(如果您还没有(,然后创建一个ServiceMonitor,如PrometheusOperator文档中所述——搜索短语";这个服务对象是由ServiceMonitor"发现的;。

PrometheusOperator中使用Deployment/Service/ServiceMonitor/PrometheusRule需要非常小心
所以我创建了一个helm chart repo kehao95/helm prometheus exporter来安装任何prometheus出口商,包括您的客户出口商,您可以试用它
它不仅会为您创建导出器Deployment,还会为您创建Service/ServiceMonitor/PrometheusRule

  • 安装图表
helm repo add kehao95 https://kehao95.github.io/helm-prometheus-exporter/
  • kehao95/prometheus-exporter创建值文件my-exporter.yaml
exporter: 
image: your-exporter
tag: latest
port: 8080
args:
- "--telemetry.addr=8080"
- "--telemetry.path=/metrics"
  • 安装带舵
helm install --namespace yourns my-exporter kehao95/prometheus-exporter -f my-exporter.yaml

然后你应该在普罗米修斯中看到你的度量。

最新更新