Skip to content

Commit f4363f4

Browse files
maxdaymaxday
authored
feat: remove vendored aws-lambda-cpp dependency (#633)
1 parent a32b1e4 commit f4363f4

71 files changed

Lines changed: 217 additions & 34539 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/runtime-interface-client_pr.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,61 @@ jobs:
106106
env:
107107
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
108108

109+
integration-test-matrix:
110+
runs-on: ${{ matrix.arch.runner }}
111+
strategy:
112+
# Run every OS/arch combination to completion so one failure doesn't mask the others.
113+
fail-fast: false
114+
matrix:
115+
buildspec:
116+
- buildspec.os.alpine.yml
117+
- buildspec.os.amazoncorretto.yml
118+
- buildspec.os.amazonlinux.1.yml
119+
- buildspec.os.amazonlinux.2.yml
120+
- buildspec.os.debian.yml
121+
- buildspec.os.ubuntu.yml
122+
arch:
123+
- label: x64
124+
runner: ubuntu-latest
125+
platform: linux/amd64
126+
- label: arm64
127+
runner: ubuntu-24.04-arm
128+
platform: linux/arm64/v8
129+
exclude:
130+
# Amazon Linux 1 was never published for ARM64 (x86_64 only), so
131+
# public.ecr.aws/amazonlinux/amazonlinux:1 has no arm64 manifest.
132+
- buildspec: buildspec.os.amazonlinux.1.yml
133+
arch:
134+
label: arm64
135+
runner: ubuntu-24.04-arm
136+
platform: linux/arm64/v8
137+
name: "integration-test (${{ matrix.buildspec }} / ${{ matrix.arch.label }})"
138+
steps:
139+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
140+
141+
- name: Set up Docker Buildx
142+
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
143+
with:
144+
install: true
145+
146+
- name: Run OS integration test - 'test-integ' target
147+
working-directory: ./aws-lambda-java-runtime-interface-client
148+
run: make test-integ BUILDSPEC=test/integration/codebuild/${{ matrix.buildspec }}
149+
env:
150+
PLATFORM_FILTER: ${{ matrix.arch.platform }}
151+
152+
integration-test:
153+
needs: integration-test-matrix
154+
if: always()
155+
runs-on: ubuntu-latest
156+
steps:
157+
- name: Check integration-test results
158+
run: |
159+
if [ "${{ needs.integration-test-matrix.result }}" != "success" ]; then
160+
echo "Integration tests failed on one or more OS/arch combinations"
161+
exit 1
162+
fi
163+
109164
smoke-test:
110165
needs: smoke-test-arch
111166
if: always()

‎aws-lambda-java-runtime-interface-client/.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
11
compile-flags.txt
22
ric-dev-environment/codeartifact-properties.mk
3+
4+
# aws-lambda-cpp prebuilt lib + headers, fetched and staged at build time
5+
src/main/jni/deps/aws-lambda-cpp/

‎aws-lambda-java-runtime-interface-client/Makefile‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,10 @@ test:
2626

2727
.PHONY: setup-codebuild-agent
2828
setup-codebuild-agent:
29-
docker build -t codebuild-agent \
29+
test/integration/codebuild-local/docker-retry.sh docker build --load -t codebuild-agent \
3030
--build-arg ARCHITECTURE=$(ARCHITECTURE_ALIAS) \
31-
- < test/integration/codebuild-local/Dockerfile.agent
31+
-f test/integration/codebuild-local/Dockerfile.agent \
32+
test/integration/codebuild-local
3233

3334
# Smoke tests are split per-architecture so CI can run each set on a native
3435
# runner. Running the linux/arm64/v8 combos under QEMU on an x86_64 host makes
@@ -46,9 +47,13 @@ test-smoke-aarch64: setup-codebuild-agent
4647
CODEBUILD_IMAGE_TAG=codebuild-agent test/integration/codebuild-local/test_one.sh test/integration/codebuild/buildspec.os.alpine.yml alpine 3.15 corretto11 linux/arm64/v8
4748
CODEBUILD_IMAGE_TAG=codebuild-agent test/integration/codebuild-local/test_one.sh test/integration/codebuild/buildspec.os.amazoncorretto.yml amazoncorretto amazoncorretto 11 linux/arm64/v8
4849

50+
# BUILDSPEC can point to the buildspec directory (default, runs every OS) or to a
51+
# single buildspec file, which is how CI parallelizes the run across OSes.
52+
BUILDSPEC ?= test/integration/codebuild
53+
4954
.PHONY: test-integ
5055
test-integ: setup-codebuild-agent
51-
CODEBUILD_IMAGE_TAG=codebuild-agent test/integration/codebuild-local/test_all.sh test/integration/codebuild
56+
CODEBUILD_IMAGE_TAG=codebuild-agent test/integration/codebuild-local/test_all.sh $(BUILDSPEC)
5257

5358
# Command to run everytime you make changes to verify everything works
5459
.PHONY: dev

‎aws-lambda-java-runtime-interface-client/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ You can include this package in your preferred base image to make that base imag
1111

1212
### Creating a Docker Image for Lambda with the Runtime Interface Client
1313

14-
Choose a preferred base image. The Runtime Interface Client is tested on Amazon Linux, Alpine, Ubuntu, Debian, and CentOS. The requirements are that the image is:
14+
Choose a preferred base image. The Runtime Interface Client is tested on Amazon Linux, Alpine, Ubuntu, and Debian. The requirements are that the image is:
1515

1616
* built for x86_64 and ARM64
1717
* contains Java >= 8

‎aws-lambda-java-runtime-interface-client/src/main/jni/Dockerfile.glibc‎

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@ ARG AWS_REGION
77
RUN if [ -n "${AWS_REGION}" ]; then echo "${AWS_REGION}" > /etc/yum/vars/awsregion; fi
88

99
RUN yum install -y \
10-
cmake3 \
1110
tar \
1211
gzip \
1312
make \
@@ -33,18 +32,12 @@ RUN ./configure \
3332
make && \
3433
make install
3534

36-
# Install aws-lambda-cpp dependency
37-
ADD ./deps/aws-lambda-cpp-* /src/deps/aws-lambda-cpp
38-
RUN mkdir -p /src/deps/aws-lambda-cpp/build
39-
WORKDIR /src/deps/aws-lambda-cpp/build
40-
RUN cmake3 .. \
41-
-DENABLE_LTO=OFF \
42-
-DCMAKE_CXX_FLAGS="-fPIC -DBACKWARD_SYSTEM_UNKNOWN" \
43-
-DCMAKE_CXX_STANDARD=11 \
44-
-DCMAKE_INSTALL_PREFIX=$(pwd)/../../artifacts \
45-
-DCMAKE_MODULE_PATH=$(pwd)/../../artifacts/lib/pkgconfig && \
46-
make && \
47-
make install
35+
# Install prebuilt aws-lambda-cpp dependency. The static library and headers
36+
# were fetched and GPG-verified on the host by build-jni-lib.sh; here we only
37+
# COPY them into the artifacts tree the native client links against (the build
38+
# container never reaches the network).
39+
COPY ./deps/aws-lambda-cpp/include /src/deps/artifacts/include
40+
COPY ./deps/aws-lambda-cpp/lib/libaws-lambda-runtime.a /src/deps/artifacts/lib/
4841

4942
# Build native client
5043
ADD *.cpp *.h /src/

‎aws-lambda-java-runtime-interface-client/src/main/jni/Dockerfile.musl‎

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ ARG CURL_VERSION
66
RUN apk update && \
77
apk add \
88
openjdk11 \
9-
cmake \
109
file \
1110
g++ \
1211
gcc \
@@ -31,17 +30,8 @@ RUN ./configure \
3130
make && \
3231
make install
3332

34-
# Install aws-lambda-cpp dependency
35-
ADD ./deps/aws-lambda-cpp-* /src/deps/aws-lambda-cpp
36-
RUN mkdir -p /src/deps/aws-lambda-cpp/build
37-
WORKDIR /src/deps/aws-lambda-cpp/build
38-
RUN cmake .. \
39-
-DCMAKE_CXX_FLAGS="-fPIC -DBACKWARD_SYSTEM_UNKNOWN" \
40-
-DCMAKE_CXX_STANDARD=11 \
41-
-DCMAKE_INSTALL_PREFIX=$(pwd)/../../artifacts\
42-
-DCMAKE_MODULE_PATH=$(pwd)/../../artifacts/lib/pkgconfig && \
43-
make && \
44-
make install
33+
COPY ./deps/aws-lambda-cpp/include /src/deps/artifacts/include
34+
COPY ./deps/aws-lambda-cpp/lib/libaws-lambda-runtime.a /src/deps/artifacts/lib/
4535

4636
# Build native client
4737
ADD *.cpp *.h /src/

‎aws-lambda-java-runtime-interface-client/src/main/jni/build-jni-lib.sh‎

Lines changed: 64 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,67 @@ MULTI_ARCH=${2}
99
BUILD_OS=${3}
1010
BUILD_ARCH=${4}
1111
CURL_VERSION=7.83.1
12-
# Registry hosting the base images. Defaults to public.ecr.aws for local and
13-
# GitHub-hosted builds; the release workflow overrides it with the ECR
14-
# pull-through cache so egress-locked runners don't hit public.ecr.aws.
12+
1513
BASE_REGISTRY="${BASE_REGISTRY:-public.ecr.aws}"
1614
AWS_REGION="${AWS_REGION:-${AWS_DEFAULT_REGION:-}}"
1715

16+
# aws-lambda-cpp is consumed as the prebuilt static library published on the
17+
# upstream GitHub release rather than being compiled from a vendored source
18+
# tree. We fetch and GPG-verify it
19+
ALC_VERSION="1.0.1"
20+
ALC_TAG="v${ALC_VERSION}"
21+
ALC_REPO_URL="https://github.com/awslabs/aws-lambda-cpp"
22+
ALC_RELEASE_URL="${ALC_REPO_URL}/releases/download/${ALC_TAG}"
23+
ALC_SIGNING_KEY_URL="https://raw.githubusercontent.com/awslabs/aws-lambda-cpp/${ALC_TAG}/signing-public-key.asc"
24+
ALC_STAGE_DIR="${SRC_DIR}/deps/aws-lambda-cpp"
25+
26+
function fetch_aws_lambda_cpp() {
27+
arch=$1
28+
29+
release_arch="${arch/aarch_64/aarch64}"
30+
31+
if [ -f "${ALC_STAGE_DIR}/.staged-arch" ] && \
32+
[ "$(cat "${ALC_STAGE_DIR}/.staged-arch")" == "${release_arch}" ]; then
33+
echo "aws-lambda-cpp ${ALC_VERSION} (${release_arch}) already staged, skipping fetch"
34+
return
35+
fi
36+
37+
echo "Fetching prebuilt aws-lambda-cpp ${ALC_VERSION} for ${release_arch}"
38+
rm -rf "${ALC_STAGE_DIR}"
39+
mkdir -p "${ALC_STAGE_DIR}/lib" "${ALC_STAGE_DIR}/include"
40+
41+
local workdir
42+
workdir=$(mktemp -d)
43+
local lib_asset="libaws-lambda-runtime-${release_arch}.a"
44+
45+
curl -fsSL -o "${workdir}/${lib_asset}" "${ALC_RELEASE_URL}/${lib_asset}"
46+
curl -fsSL -o "${workdir}/${lib_asset}.asc" "${ALC_RELEASE_URL}/${lib_asset}.asc"
47+
curl -fsSL -o "${workdir}/SHA256SUMS" "${ALC_RELEASE_URL}/SHA256SUMS"
48+
curl -fsSL -o "${workdir}/SHA256SUMS.asc" "${ALC_RELEASE_URL}/SHA256SUMS.asc"
49+
curl -fsSL -o "${workdir}/signing-key.asc" "${ALC_SIGNING_KEY_URL}"
50+
51+
local gnupghome
52+
gnupghome=$(mktemp -d)
53+
gpg --homedir "${gnupghome}" --batch --quiet --import "${workdir}/signing-key.asc"
54+
gpg --homedir "${gnupghome}" --batch --verify "${workdir}/${lib_asset}.asc" "${workdir}/${lib_asset}"
55+
gpg --homedir "${gnupghome}" --batch --verify "${workdir}/SHA256SUMS.asc" "${workdir}/SHA256SUMS"
56+
rm -rf "${gnupghome}"
57+
58+
# Cross-check the checksum too (defence in depth; SHA256SUMS is itself signed).
59+
( cd "${workdir}" && grep "${lib_asset}\$" SHA256SUMS | sha256sum -c - )
60+
61+
cp "${workdir}/${lib_asset}" "${ALC_STAGE_DIR}/lib/libaws-lambda-runtime.a"
62+
63+
# Headers aren't a release asset, so take them from the source at the same
64+
# tag. They are declarations only -- every symbol lives in the prebuilt lib.
65+
curl -fsSL -o "${workdir}/src.tar.gz" "${ALC_REPO_URL}/archive/refs/tags/${ALC_TAG}.tar.gz"
66+
tar -xzf "${workdir}/src.tar.gz" -C "${workdir}" "aws-lambda-cpp-${ALC_VERSION}/include"
67+
cp -R "${workdir}/aws-lambda-cpp-${ALC_VERSION}/include/." "${ALC_STAGE_DIR}/include/"
68+
69+
echo "${release_arch}" > "${ALC_STAGE_DIR}/.staged-arch"
70+
rm -rf "${workdir}"
71+
}
72+
1873
function get_docker_platform() {
1974
arch=$1
2075

@@ -44,6 +99,8 @@ function build_for_libc_arch() {
4499
arch=$2
45100
artifact=$3
46101

102+
fetch_aws_lambda_cpp "${arch}"
103+
47104
docker_platform=$(get_docker_platform ${arch})
48105

49106
echo "Compiling the native library with libc implementation \`${libc_impl}\` on architecture \`${arch}\` using Docker platform \`${docker_platform}\`"
@@ -118,12 +175,10 @@ else
118175
declare -a ARCHITECTURES=("x86_64" "aarch_64")
119176
declare -a LIBC_IMPLS=("glibc" "musl")
120177

121-
# `arch` reports the host as `aarch64`, but we use Maven's classifier
122-
# spelling `aarch_64` in ARCHITECTURES, so normalize before comparing.
123-
host_arch=$(arch)
124-
if [ "${host_arch}" == "aarch64" ]; then
125-
host_arch="aarch_64"
126-
fi
178+
host_arch="$(arch)"
179+
case "${host_arch}" in
180+
aarch64|arm64) host_arch="aarch_64" ;;
181+
esac
127182

128183
for arch in "${ARCHITECTURES[@]}"; do
129184

‎aws-lambda-java-runtime-interface-client/src/main/jni/deps/aws-lambda-cpp-0.2.7/.clang-format‎

Lines changed: 0 additions & 61 deletions
This file was deleted.

‎aws-lambda-java-runtime-interface-client/src/main/jni/deps/aws-lambda-cpp-0.2.7/.clang-tidy‎

Lines changed: 0 additions & 41 deletions
This file was deleted.

‎aws-lambda-java-runtime-interface-client/src/main/jni/deps/aws-lambda-cpp-0.2.7/.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 0 additions & 6 deletions
This file was deleted.

0 commit comments

Comments
 (0)