我使用Serverless v2.23.0将一组端点部署到AWS Lambda函数。Lambda函数运行python 3.8。我想在部署到AWS Lambda时自动安装一个依赖项。
例如，我希望自动安装程序包pycurl。我正在上学习教程https://www.serverless.com/blog/serverless-python-packaging
我添加了一个requirements.txt文件，它只有一行：
pycurl
我在serverless.yml中添加了：
plugins:
- serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: non-linux
当我通过无服务器部署时，我会得到这个怪物：
STDERR: ERROR: Command errored out with exit status 1:
command: /var/lang/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-jv_aqotm
cwd: /tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/
Complete output (22 lines):
Traceback (most recent call last):
File "/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py", line 236, in configure_unix
p = subprocess.Popen((self.curl_config(), '--version'),
File "/var/lang/lib/python3.8/subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/var/lang/lib/python3.8/subprocess.py", line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'curl-config'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py", line 988, in <module>
ext = get_extension(sys.argv, split_extension_source=split_extension_source)
File "/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py", line 649, in get_extension
ext_config = ExtensionConfiguration(argv)
File "/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py", line 101, in __init__
self.configure()
File "/tmp/pip-install-_p_d5byz/pycurl_960a4c2e1a5b4bf1b382d3930bd2fba3/setup.py", line 241, in configure_unix
raise ConfigurationError(msg)
__main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config'
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/50/1a/35b1d8b8e4e23a234f1b17a8a40299fd550940b16866c9a1f2d47a04b969/pycurl-7.43.0.6.tar.gz#sha256=8301518689daefa53726b59ded6b48f33751c383cf987b0ccfbbc4ed40281325 (from https://pypi.org/simple/pycurl/) (requires-python:>=3.5). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1:
command: /var/lang/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-6g6vfjsr
cwd: /tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/
Complete output (22 lines):
Traceback (most recent call last):
File "/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py", line 233, in configure_unix
p = subprocess.Popen((self.curl_config(), '--version'),
File "/var/lang/lib/python3.8/subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/var/lang/lib/python3.8/subprocess.py", line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'curl-config'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py", line 961, in <module>
ext = get_extension(sys.argv, split_extension_source=split_extension_source)
File "/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py", line 623, in get_extension
ext_config = ExtensionConfiguration(argv)
File "/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py", line 101, in __init__
self.configure()
File "/tmp/pip-install-_p_d5byz/pycurl_fe88f280458f4a83807016624ffa0f80/setup.py", line 238, in configure_unix
raise ConfigurationError(msg)
__main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config'
----------------------------------------
...
...
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/08/79/576eaecd19f11a265cd8d10f339ecfd4b1033436e56e6f2625153a4b79aa/pycurl-7.19.0.2.tar.gz#sha256=7a9e793b9181654d5eef3f6d22c244c57d2b51d38feb4c1b71d68efda99b0547 (from https://pypi.org/simple/pycurl/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1:
command: /var/lang/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_p_d5byz/pycurl_94d64ac946c44540bb8e097906871c69/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_p_d5byz/pycurl_94d64ac946c44540bb8e097906871c69/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-1gyrq4uv
cwd: /tmp/pip-install-_p_d5byz/pycurl_94d64ac946c44540bb8e097906871c69/
Complete output (6 lines):
/bin/sh: curl-config: command not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-_p_d5byz/pycurl_94d64ac946c44540bb8e097906871c69/setup.py", line 90, in <module>
raise Exception("`%s' not found -- please install the libcurl development files or specify --curl-config=/path/to/curl-config" % CURL_CONFIG)
Exception: `curl-config' not found -- please install the libcurl development files or specify --curl-config=/path/to/curl-config
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/17/11/79e2187232ffdb96c9c200f8fc82452f8631f659fd9bdd2acf4bf7991905/pycurl-7.19.0.1.tar.gz#sha256=8b14d91d7453c5d9a4cc8cd4e61ab9389298813028d10661bf2079ee06d676c8 (from https://pypi.org/simple/pycurl/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1:
command: /var/lang/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_p_d5byz/pycurl_a3d77a6dfa274613baa311998e8830a2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_p_d5byz/pycurl_a3d77a6dfa274613baa311998e8830a2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-skxkhoag
cwd: /tmp/pip-install-_p_d5byz/pycurl_a3d77a6dfa274613baa311998e8830a2/
Complete output (6 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-_p_d5byz/pycurl_a3d77a6dfa274613baa311998e8830a2/setup.py", line 58
print "FATAL: bad directory %s in environment variable %s" % (dir, envvar)
^
SyntaxError: invalid syntax
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/11/73/abcfbbb6e1dd7087fa53042c301c056c11264e8a737a4688f834162d731e/pycurl-7.19.0.tar.gz#sha256=8ef0bf1c0e48f0d193ef8ac4ae4b3596a1e91b6cbf1ff1b25de769fe3782aeb6 (from https://pypi.org/simple/pycurl/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1:
command: /var/lang/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_p_d5byz/pycurl_c9a9d43d5c9a4fb4b41908f9f1558869/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_p_d5byz/pycurl_c9a9d43d5c9a4fb4b41908f9f1558869/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-mnx6l7bp
cwd: /tmp/pip-install-_p_d5byz/pycurl_c9a9d43d5c9a4fb4b41908f9f1558869/
Complete output (6 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-_p_d5byz/pycurl_c9a9d43d5c9a4fb4b41908f9f1558869/setup.py", line 58
print "FATAL: bad directory %s in environment variable %s" % (dir, envvar)
^
SyntaxError: invalid syntax
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/fe/8f/711d436c76938093dd10dae465ccee316c6241cfdada06c5cc3b5c1f74b8/pycurl-7.18.2.tar.gz#sha256=cda1085b7911ca802746d816f37d541165ad1e20c241d22e4c304ddb2bc81ffe (from https://pypi.org/simple/pycurl/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pycurl
ERROR: No matching distribution found for pycurl
at /Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/serverless-python-requirements/lib/pip.js:325:13
at Array.forEach (<anonymous>)
at installRequirements (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/serverless-python-requirements/lib/pip.js:312:28)
at installRequirementsIfNeeded (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/serverless-python-requirements/lib/pip.js:556:3)
at ServerlessPythonRequirements.installAllRequirements (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/serverless-python-requirements/lib/pip.js:635:29)
at ServerlessPythonRequirements.tryCatcher (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/promise.js:729:18)
at _drainQueueStep (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/Users/markkelly/RedMatter/repos/new_version/RedMatterAppBackend/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:439:21)
at process.topLevelDomainCallback (domain.js:130:23)
For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: darwin
Node Version: 12.14.1
Framework Version: 2.23.0
Plugin Version: 4.4.3
SDK Version: 2.3.2
Components Version: 3.6.2
从碎片中挑选，我看到错误，例如：
Exception: `curl-config' not found -- please install the libcurl development files or specify --curl-config=/path/to/curl-config
和：
ERROR: Could not find a version that satisfies the requirement pycurl
ERROR: No matching distribution found for pycurl
我使用这种方法尝试了其他依赖项，但没有任何问题。为什么没有安装pycurl？
编辑
我在这里实现了答案，并且它部署正确。但当我对端点进行CURL时，我得到：
[ERROR] Runtime.ImportModuleError: Unable to import module 'handler': No module named 'pycurl'
处理程序为：
import os
import json
import uuid
import time
import boto3
from boto3.dynamodb.conditions import Key
import pycurl
from io import BytesIO
def events(event, context):
b_obj = BytesIO()
crl = pycurl.Curl()
# Set URL value
crl.setopt(crl.URL, '[url to fetch]')
# Write bytes that are utf-8 encoded
crl.setopt(crl.WRITEDATA, b_obj)
# Perform a file transfer
crl.perform()
# End curl session
crl.close()
# Get the content stored in the BytesIO object (in byte characters)
get_body = b_obj.getvalue()
print(get_body)
response = {
"statusCode": "200",
"headers": {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json"
},
"body": json.dumps(get_body)
}
return response
serverless.yml:
service: rmappbackend
frameworkVersion: "2"
package:
exclude:
- "frontend/**"
plugins:
- serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: true
dockerFile: ./dockerfile
provider:
name: aws
runtime: python3.8
stage: ${opt:stage}
lambdaHashingVersion: 20201221
apiGateway:
shouldStartNameWithService: true
environment:
STAGE: ${opt:stage}
EVENTS_TABLE: ${opt:stage}-events
GRAPH_TABLE: ${opt:stage}-graph
DATA_TABLE: ${opt:stage}-data
STORAGE_BUCKET: ${opt:stage}-storage-bucket-01392334
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:DescribeTable
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:us-east-1:*:*"
functions:
events:
handler: handler.events
events:
- http:
method: GET
path: /events
resources:
Resources:
events:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:provider.environment.EVENTS_TABLE}
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
storage:
Type: AWS::S3::Bucket
Properties:
BucketName: ${self:provider.environment.STORAGE_BUCKET}
libcurl-devel&缺少openssl-devel包，请使用其中一个lambci/lambda Docker映像进行构建，并确保安装了所有依赖项。
将serverless.yml更改为：
provider:
runtime: python2.7
[...]
custom:
pythonRequirements:
dockerizePip: true
dockerFile: ./Dockerfile
然后添加一个Dockerfile，内容如下：
FROM lambci/lambda:build-python2.7
ENV PYCURL_SSL_LIBRARY=openssl
RUN yum install -y libcurl-devel openssl-devel