我正在尝试通过Google Cloud Build在jfrog存储库中安装私有包。我可以通过https:<USER>:<API_KEY>@<artifactory_url>访问jfrog存储库，也可以进行pip install <package_name_and_version> https:<USER>:<API_KEY>@<artifactory_url>
我想在云构建中集成这一步骤，使用云KMS在pip安装期间解密API_KEY。我使用以下命令加密了API_KEY字符串
# Create a local file with the secret
echo "MyAPIKEY" > plain_pwd.txt
# To encrypt a secret using KMS
gcloud kms encrypt
--plaintext-file=plain_pwd.txt
--ciphertext-file=cipher_pwd.enc.txt
--location=global
--keyring=<keyring>
--key=<key>
# Encode the binary encoded secret as base64 string
base64 cipher_pwd.enc.txt -w 0 > cipher_pw.enc.64.txt
cloudbuild.yaml中指定的机密为：
secrets:
- kmsKeyName: projects/<project_id>/locations/global/keyRings/<keyring>/cryptoKeys/<key>
secretEnv:
APIKEY: <base64 encrypted string from cloud kms encrypt command>
这是我安装pip的cloudbuild.yaml步骤：
-
args:
- "-m"
- pip
- install
- "-t"
- /workspace/lib
- "schema-registry-client==0.8.14.dev0"
- "--extra-index-url"
- 'https://adminuser:$$APIKEY@<artifactory_url>'
entrypoint: python3
secretEnv: ['APIKEY']
id: INSTALL_SCHEMA_REGISTRY
name: "python:3.7"
在执行这一步骤时，我在云构建中得到了"EOFError:EOF when reading a line"：
正在查找索引：https://pypi.org/simple，
https://onpat:****@artifactory.build.****.****.com/artifactory/api/pypi/dpfw-pypi-dev-local/simple
ERROR: Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
status = self.run(options, args)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
return func(self, options, args)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
reqs, check_supported_wheels=not options.target_dir
File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 339, in _get_abstract_dist_for
self._populate_link(req)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 305, in _populate_link
req.link = self._find_requirement_link(req)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 270, in _find_requirement_link
best_candidate = self.finder.find_requirement(req, upgrade)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 899, in find_requirement
req.name, specifier=req.specifier, hashes=hashes,
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 881, in find_best_candidate
candidates = self.find_all_candidates(project_name)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 826, in find_all_candidates
project_url, link_evaluator=link_evaluator,
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 790, in process_project_url
html_page = self._link_collector.fetch_page(project_url)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 643, in fetch_page
return _get_html_page(location, session=self.session)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 455, in _get_html_page
resp = _get_html_response(url, session=session)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 169, in _get_html_response
"Cache-Control": "max-age=0",
File "/usr/local/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/network/session.py", line 421, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 650, in send
r = dispatch_hook('response', hooks, r, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
_hook_data = hook(hook_data, **kwargs)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
username, password, save = self._prompt_for_password(parsed.netloc)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
username = ask_input("User for {}: ".format(netloc))
File "/usr/local/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 259, in ask_input
return input(message)
EOFError: EOF when reading a line
Finished Step #2 - "INSTALL_SCHEMA_REGISTRY"
ERROR
ERROR: build step 2 "python:3.7" failed: step exited with non-zero status: 2
此外，我尝试对相同的密文进行gcloud kms解密，并找回了原始的API密钥。所以，我不认为加密/解密是一个问题。我还为云kms提供了对云构建服务代理的必要访问权限。
关于如何解决这个问题，有什么建议/帮助吗？
我找到了解决这个问题的另一种方法，显然，pip-cli在扩展参数和env变量时遇到了问题。您可以使用Google Cloud Secret管理器来存储凭据(本例中为JFrog API_KEY(。使用gcloud secrets versions access <version_id> --secret=<secret_key>在运行时检索机密。在构建步骤中使用--extra-index-url=https://user:<api_key_secret_value>@<artifactory_url>在~/.pip/pip.conf创建pip.conf文件，在随后的构建步骤中只执行pip install <package_name>。唯一的缺点是它在构建过程中增加了一个额外的步骤。我创建了bash脚本，在构建步骤中将机密读取到文件中，并在下一个构建步骤中使用该机密值创建pip.conf值。使用云构建卷在步骤之间传递文件。
#read_secret.sh
#!/bin/bash
gcloud secrets versions access 1 --secret=artifactory-api-key > "/data/key.txt"
在下一步中使用以下bash脚本安装包
#install_module.sh
#!/bin/bash
export API_KEY=$(cat "/data/key.txt")
echo $API_KEY
mkdir -p ~/.pip
echo -e "[global]n--extra-index-url=https://myuser:$API_KEY@<artifactory_url>" > ~/.pip/pip.conf
pip install -t "/workspace/lib" private-package-lib==0.1.2