使用Maven管理AWS Java Lambda依赖JAR大小

Question

我有一个Lambda函数，使用Java-11启动大约需要10秒。我在谷歌上搜索了一下，发现了一些帖子，建议降低包的JAR大小可能有助于更快的启动时间(减少加载的冗余库等…)。

我也在一些帖子中读到使用下面可能会有所帮助，并尝试添加

<scope>provided</scope>

在AWS相关的依赖认为，好吧…AWS Lambda会有AWS特定的库吗?事实并非如此!当尝试执行函数时，添加提供的作用域不起作用。

我当前的pom.xml如下:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>REMOVED</groupId>
<artifactId>REMOVED</artifactId>
<version>REMOVED</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>30.1-jre</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.20</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.7.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.7.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20220320</version>
</dependency>
<dependency>
<groupId>software.amazon.kinesis</groupId>
<artifactId>amazon-kinesis-client</artifactId>
<version>2.4.1</version>
</dependency><dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kinesis</artifactId>
<version>1.12.228</version>
</dependency>
<!-- <dependency><groupId>software.amazon.awssdk</groupId><artifactId>firehose</artifactId><version>2.17.198</version></dependency> -->
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/kinesis -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>kinesis</artifactId>
<version>2.17.201</version>
</dependency>
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/secretsmanager -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
<version>2.17.204</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

AWS Kinesis的使用只是为了创建KinesisClient，使用PutRecordRequest和PutRecordResponse，与AWS Secret Manager一起，只是为了访问特定的秘密。

我并不精通AWS API，根据我的理解，我已经包含了基于依赖项执行这些任务所需的有限数量的库。

JAR文件大约65mb大。我是否能够进一步优化依赖项负载?我猜降低JAR文件会增加Lambda函数的启动时间吗?

谢谢你,

Answer 1

这篇文章并没有减少Lambda的大小(这也是我所遇到的问题)，但是它有助于在没有任何代码更改的情况下进行冷启动。在他们的例子中，它将开始时间从大约5秒改变为2秒。

只需添加一个lambda环境变量:

Key: JAVA_TOOL_OPTIONS
Value: -XX:+TieredCompilation -XX:TieredStopAtLevel=1

还有一篇文章描述了如何删除Netty和Apache客户端，以支持内置的JDK客户端。

<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>url-connection-client</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<exclusions>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>netty-nio-client</artifactId>
</exclusion>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>apache-client</artifactId>
</exclusion>
</exclusions>
</dependency>

使用UrlConnectionHttpClient封装Java的HTTP客户端。

S3Client client = S3Client.builder()
.region(Region.US_WEST_2)
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
.httpClient(UrlConnectionHttpClient.builder().build())
.build();

最后你可以使用Jackson-jr，这是一个+- 100kb的Jackson版本。

Answer 2

我认为您不小心包含了同一依赖项的三个副本。你只想要最新的2.x。x版本的Kinesis库。

你当然不需要1.x。x. x SDK和2.x. x. x。x SDK。

Answer 3

您应该能够使用Java编写的AWS lambda的新SnapStart特性来解决冷启动问题。它基本上创建了一个容器的快照，用于在初始化/首次冷启动后启动java lambda并保存它。下次调用lambda时，它将"恢复"快照，这通常需要几百毫秒，而不是完全冷启动，这需要几秒钟。

如果您正在使用AWS CDK，那么这里有一个示例，说明如何在API网关后面的lambda上配置snapstart。基本上，您需要创建一个带有别名的新lambda版本，并将snapstart应用于lambda的已发布版本。

package com.snapstart.example;
import software.amazon.awscdk.Stack;
import software.amazon.awscdk.StackProps;
import software.amazon.awscdk.services.apigateway.LambdaIntegration;
import software.amazon.awscdk.services.apigateway.RestApi;
import software.amazon.awscdk.services.lambda.Runtime;
import software.amazon.awscdk.services.lambda.*;
import software.constructs.Construct;
import java.util.Map;

public class MyStack extends Stack {
public MyStack(final Construct scope, final String id, final StackProps props) {
super(scope, id, props);

FunctionProps functionProps = FunctionProps.builder().
handler("com.snapstart.example.MyHandler::handleRequest")
.code(Code.fromAsset("../software/target/software-1.0-SNAPSHOT.jar"))
.runtime(Runtime.JAVA_11).build();
Function function = new Function(this, "lambda-with-snapstart", functionProps);
// configure SnapStart on published versions of this lambda.
((CfnFunction) function.getNode().getDefaultChild())
.addPropertyOverride("SnapStart", Map.of("ApplyOn", "PublishedVersions"));
// create a new lambda version with alias.
Version lambdaVersion = Version.Builder.create(this, "snapstart-lambda-version")
.lambda(function).build();
Alias alias = Alias.Builder.create(this, "snapstart-lambda-alias")
.aliasName("snapstart-alias-name").version(lambdaVersion).build();
// Use the lambda alias when creating the lambda integration rather than the function its self.
LambdaIntegration lambdaIntegration = LambdaIntegration.Builder.create(alias).build();
RestApi api = RestApi.Builder.create(this, "java-with-snapstart-api")
.description("An API for testing java SnapStart.").build();
api.getRoot().resourceForPath("mypath").addMethod("POST", lambdaIntegration);
}