尝试在aks中使用V1版本执行时，添加CustomResourceDefinition引发错误

Question

我正在尝试在aks中添加customresourcedeployment。早些时候，我们使用了V1beta1版本，我认为该版本已弃用。现在我正在尝试v1版本，它抛出了模式验证错误。V1beta1版本如下。

{
'apiVersion': 'apiextensions.k8s.io/v1beta1',
'kind': 'CustomResourceDefinition',
'metadata': {
'name': 'azureassignedidentities.aadpodidentity.k8s.io'
},
'spec': {
'group': 'aadpodidentity.k8s.io',
'version': 'v1',
'names': {
'kind': 'AzureAssignedIdentity',
'plural': 'azureassignedidentities'
},
'scope': 'Namespaced'
}
}

现在我正在尝试使用以下模板。

let collection = {
'apiVersion': 'apiextensions.k8s.io/v1',
'kind': 'CustomResourceDefinition',
'metadata': {
"annotations": {
"api-approved.kubernetes.io": "unapproved",
"controller-gen.kubebuilder.io/version": "v0.5.0"
},
'name': 'azureassignedidentities.aadpodidentity.k8s.io'
},
'spec': {
'group': 'aadpodidentity.k8s.io',
'names': {
'kind': 'AzureAssignedIdentity',
'listKind': 'AzureAssignedIdentityList',
'plural': 'azureassignedidentities',
'singular': 'azureassignedidentity'
},
'scope': 'Namespaced',
'versions': {
'name': 'v1',
'schema': {
'openAPIV3Schema': {
'description': 'AzureAssignedIdentity contains the identity <-> pod mapping which is matched.',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureAssignedIdentitySpec contains the relationship between an AzureIdentity and an AzureIdentityBinding.',
'properties': {
'azureBindingRef': {
'description': 'AzureBindingRef is an embedded resource referencing the AzureIdentityBinding used by the AzureAssignedIdentity, which requires x-kubernetes-embedded-resource fields to be true',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureIdentityBindingSpec matches the pod with the Identity. Used to indicate the potential matches to look for between the pod/deployment and the identities present.',
'properties': {
'azureIdentity': {
'type': 'string'
},
'metadata': {
'type': 'object'
},
'selector': {
'type': 'string'
},
'weight': {
'description': 'Weight is used to figure out which of the matching identities would be selected.',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureIdentityBindingStatus contains the status of an AzureIdentityBinding.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
}
},
'type': 'object'
}
},
'type': 'object',
'x-kubernetes-embedded-resource': 'true'
},
'azureIdentityRef': {
'description': 'AzureIdentityRef is an embedded resource referencing the AzureIdentity used by the AzureAssignedIdentity, which requires x-kubernetes-embedded-resource fields to be true',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureIdentitySpec describes the credential specifications of an identity on Azure.',
'properties': {
'adEndpoint': {
'type': 'string'
},
'adResourceID': {
'description': 'For service principal. Option param for specifying the  AD details.',
'type': 'string'
},
'auxiliaryTenantIDs': {
'description': 'Service principal auxiliary tenant ids',
'items': {
'type': 'string'
},
'nullable': 'true',
'type': 'array'
},
'clientID': {
'description': 'Both User Assigned MSI and SP can use this field.',
'type': 'string'
},
'clientPassword': {
'description': 'Used for service principal',
'properties': {
'name': {
'description': 'Name is unique within a namespace to reference a secret resource.',
"type": "string"
},
'namespace': {
'description': 'Namespace defines the space within which the secret name must be unique.',
'type': 'string'
}
},
'type': 'object'
},
'metadata': {
'type': 'object'
},
'replicas': {
'format': 'int32',
'nullable': 'true',
'type': 'integer'
},
'resourceID': {
'description': 'User assigned MSI resource id.',
'type': 'string'
},
'tenantID': {
'description': 'Service principal primary tenant id.',
'type': 'string'
},
'type': {
'description': 'UserAssignedMSI or Service Principal',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureIdentityStatus contains the replica status of the resource.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
}
},
'type': 'object'
}
},
'type': 'object',
'x-kubernetes-embedded-resource': 'true'
},
'metadata': {
'type': 'object'
},
'nodename': {
'type': 'string'
},
'pod': {
'type': 'string'
},
'podNamespace': {
'type': 'string'
},
'replicas': {
'format': 'int32',
'nullable': 'true',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureAssignedIdentityStatus contains the replica status of the resource.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
},
'status': {
'type': 'string'
}
},
'type': 'object'
}
},
'type': 'object'
}
},
'served': 'true',
'storage': 'true',

}
},
'status': {
'acceptedNames': {
'kind': "",
'plural': ""
},
'conditions': [],
'storedVersions': []
}
}

我正在尝试使用以下代码进行部署。

const client = this.kubeConfig.makeApiClient(k8s.ApiextensionsV1Api);
return new Promise<string>((resolve, reject) => {
client.createCustomResourceDefinition(collection).then(
(response) => {
resolve(response);
},
(err) => {
reject(err.response && err.response.body ? err.response.body : err);
},
);
});

错误如下。

*{"消息"："自定义资源定义.apiextensions.k8s.io"azurelegatedentity.aadpodentity.k8s.io"；无效：[spec.versions[0]。schema.openAPIV3Schema:必需值：必需架构，spec.versions[1]。schema.openAPIV3Schema：必需值：必须架构，spec-versions[2]。schema.open APIV3Schema:必需值：必要架构，spec.versions[3]。schema.Open APIV3Schema：必需值，spec.visions[0]。名称：无效值："quot；：DNS-1035标签必须由小写字母数字字符或"-"组成，以字母字符开头，以字母数字字符结尾(例如"my name"或"abc-123"，用于验证的正则表达式为"a-z？")，规范版本[1]。名称：无效值："quot；：DNS-1035标签必须由小写字母数字字符或"-"组成，以字母字符开头，以字母数字字符结尾(例如"my name"或"abc-123"，用于验证的正则表达式为"a-z？")，规范版本[2]。名称：无效值："quot；：DNS-1035标签必须由小写字母数字字符或"-"组成，以字母字符开头，以字母数字字符结尾(例如"my name"或"abc-123"，用于验证的正则表达式为"a-z？")，规范版本[3]。名称：无效值："quot；：DNS-1035标签必须由小写字母数字字符或"-"组成，以字母字符开头，以字母数字字符结尾(例如"my name"或"abc-123"，用于验证的正则表达式为"a-z？")，spec.versions：无效值：[]apiextensions.CustomResourceDefinitionVersion｛apiextensions.CustomResourceDefinitionVersion｝名称："，服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensionsCustomResourceValidation)apiextensions。CustomResourceColumnDefinition(nil)｝，apiextensions。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation)。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation)。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation，spec.versions：无效值：[]apiextensions.CustomResourceDefinitionVersion｛apiextensions.CustomResourceDefinitionVersion｝名称："，服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensionsCustomResourceValidation)apiextensions。CustomResourceColumnDefinition(nil)｝，apiextensions。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation)。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation)。CustomResourceDefinitionVersion｛名称："，已服务：false，存储：false，已弃用：false，弃用警告：(*string)(nil)，架构：(*apiextensions.CustomResourceValidation，status.storedVersions:无效值：[]字符串(nil)：必须至少有一个存储的版本]"}

请帮助我解决此问题

Answer 1

请检查是否可能的原因是版本(至少1个)未存储在存储版本状态中。

查看显示的错误

status.storedVersions:无效值：[]字符串(nil)：必须在至少一个存储版本]"}

在弃用版本时，请选择存储升级过程。从CustomResourceDefinition status.storedVersions字段中删除旧版本，并确保存在新版本(至少一个)。

手动将现有对象升级到新的存储版本：编写升级过程以列出所有现有对象，并使用相同的内容编写这些对象。这迫使后端在当前存储版本中写入对象，即v1

步骤

1.首先使用以前的版本创建了资源crd1.yaml。

versions:
- name: v1beta1
served: true
storage: true

使用kubectl应用它。

kubectl apply -f crd1.yaml

2.用更高版本的定义更新资源，比如v1。将v1设置为CustomResourceDefinition文件crd12.yaml.中的存储

versions:
- name: v1
served: true
storage: true

并使用kubectl进行应用。

kubectl apply -f crd12.yaml

3. 确认存在两个版本

kubectl获取自定义资源定义$resource_name-o yaml

storedVersions: [v1beta1 ,v1]

4. 对于spec.versions列表中的旧版本，检查并设置为false

(注意：我们完全可以选择在CRD中保留v1beta1版本。如果使用serve:false指定，则会阻止对v1alpha1资源的任何获取/创建/更新操作。)

验证CustomResourceDefinition中spec.versions列表中新版本的storage is set to true。

您可以将CustomResourceDefinition保存在YAML文件中，然后使用kubectl apply创建它

示例：

spec:
group: example.com
# list of versions supported by this CustomResourceDefinition
versions:
- name: v1
served: true
storage: true     # One and only one version must be marked as the storage version.
schema:
openAPIV3Schema:
type: object
properties:
host:
type: string
port:
type: string
........
- name: v1beta1
served: false     # Each version can be enabled/disabled by Served flag.
storage: false
# A schema is required
schema:
openAPIV3Schema:
type: object
properties:
host:
type: string
port:
type: string
......
conversion:
strategy: None
# None conversion assumes the same schema for all versions and only sets the apiVersion
field of custom resources to the proper value

然后删除旧版本。状态下的字段通过状态子源进行修改

例如：

curl -d '[{ "op": "replace", "path":"/status/storedVersions", "value": ["v1"] }]' 
-H "Content-Type: application/json-patch+json" 
-X PATCH  http://localhost:8080  /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/$resource_name/status

验证旧版本是否不再列在CustomResourceDefinition状态中。storedVersions

---

请参考这些：

1. 从status.storedVersions中删除版本(kubernetesquestions.com)或如何从status/storedVersions(github.com)中删除版本
2. 自定义资源定义版本控制示例
3. CustomResourceDefinitions中的版本-Kubernetes