Repository navigation
Conversation
sam build: - SamFunctionProvider extracts AWS::Serverless::WebFunction resources when include_web_functions is set, which only BuildContext does, so sam local commands don't treat Web Functions as invokable functions. EntryPoint (default index.js) takes the handler's place. - The built template points the WebFunction CodeUri at the build output. sam package / sam deploy: - WebFunction CodeUri is a packageable local path, zipped and uploaded like a Function's CodeUri. Managed bucket: - The managed stack's bucket policy lets lambda.amazonaws.com read objects (s3:GetObject, s3:GetObjectVersion) for this account's Web Function revisions, which Web Functions need to read their code. - sam deploy and sam sync update an existing managed stack that lacks the statement when the template has a WebFunction.
…ed stacks - When the template, including nested stacks, has an AWS::Serverless::WebFunction with local code, packaging warns if the target bucket isn't versioned, if its policy doesn't let lambda.amazonaws.com read the uploaded code (s3:GetObject, s3:GetObjectVersion), or if --kms-key-id is an AWS managed key or a key whose policy doesn't let Lambda call kms:Decrypt. The checks are warnings and never fail packaging. - The managed stack upgrade looks for Web Functions in nested stacks too, and only for Web Functions with local code.
| import click | ||
| from botocore.exceptions import ClientError | ||
|
|
||
| from samcli.lib.providers.provider import Stack |
There was a problem hiding this comment.
Is the circular import error mentioned in the other files because you're importing Stack here?
Here you're only using Stack for type checking, so the best way to just solve this so you don't have to deal with circular imports in the other files is to use if TYPE_CHECKING before this import
Like
aws-sam-cli/samcli/lib/providers/exceptions.py
Lines 5 to 8 in 7bf8efb
| from samcli.lib.providers.provider import Stack | |
| from typing import TYPE_CHECKING | |
| if TYPE_CHECKING: # pragma: no cover | |
| from samcli.lib.providers.provider import Stack |
…naged bucket Web Functions read their code from S3 for as long as a revision exists, so the code no longer goes to the shared managed bucket, which users can empty or delete without affecting Event Functions. The managed stack and its bucket are unchanged. - When sam deploy (--resolve-s3 or guided) or sam sync uses the managed bucket and the template, including nested stacks, has an AWS::Serverless::WebFunction with local code, SAM CLI gets or creates a <stack>-<hash>-WebFunctionCode stack with a versioned, SSE-S3 bucket whose policy lets lambda.amazonaws.com read the code for this account's Web Function revisions. Only Web Function code is uploaded there. - sam delete deletes that stack's artifacts other than Web Function code as before, deletes the stack, and only then empties and deletes the code bucket stack. - With --s3-bucket, Web Function code goes to that bucket as before, with the access warnings. - Removes the managed stack bucket policy statement and upgrade check.
| "Runtime": resource_properties.get("Runtime"), | ||
| "Timeout": resource_properties.get("Timeout"), | ||
| "FunctionName": resource_properties.get("FunctionName", name), | ||
| } |
There was a problem hiding this comment.
[BUG] Web Function Architectures is dropped, so arm64 Web Functions are built for x86_64
_convert_sam_web_function_resource passes only an allow-listed build_properties dict to _build_function_configuration. That function reads resource_properties.get("Architectures"), which here is always None, so Function.architecture falls back to X86_64.
ApplicationBuilder passes function.architecture into the build definition, so a Web Function with Architectures: [arm64] gets an x86_64 build image and native dependencies, and fails at runtime. The wrong architecture is also written to build.toml. Passing Architectures through fixes it.
| } | |
| "FunctionName": resource_properties.get("FunctionName", name), | |
| "Architectures": resource_properties.get("Architectures"), |
Five other candidates were checked by verifiers and scored 7, below the posting threshold of 8, so they're left out:
- the code bucket is deleted even when a WebFunction has
DeletionPolicy: Retain - the delete waiter and
delete_objectserrors aren't handled - every
describe_stackserror is treated as "no stack" - esbuild entry points come out wrong for file-path
EntryPointvalues like./server.js - the built template keeps the source
EntryPointfor esbuild Web Functions
The PR description doesn't match the code. It says the shared managed bucket's policy gets an AllowLambdaWebFunctionCodeAccess statement and existing managed stacks are upgraded. The diff instead creates a separate code bucket stack for each deployed stack, and sam delete empties and deletes it. I reviewed the code as written.
Both earlier findings were dismissed, so neither was posted again.
…while in use
The code bucket stack exports its bucket name, and packaging writes a Web
Function's CodeUri as {Bucket: !ImportValue <export>, Key: <key>} (keeping
the Version, if any) when the code goes to that bucket. CloudFormation then
refuses to delete the code bucket stack while a stack that reads code from
it exists. sam delete is unaffected: it deletes the importing stack first.
sam package has no stack to create a Web Function code bucket for, and the code can't go to the shared managed bucket. When the template, including nested stacks, has an AWS::Serverless::WebFunction with local code, sam package --resolve-s3 now fails before creating or using the managed bucket, and points to sam deploy or --s3-bucket.
The warnings went to stdout, which corrupted --output json and packaged templates that sam package writes to stdout.
… Web Function code Lambda always rejects a Web Function revision whose code is in an unversioned bucket or encrypted with an AWS managed KMS key, so packaging now fails before uploading in those cases (WebFunctionCodeBucketError), listing every such problem. Bucket and key policy problems and checks that couldn't run stay warnings.
….update The attribute is always set in __init__, so the getattr fallback isn't needed.
…ilds Web Functions were built as Function objects whose handler held the entry point, with an include_web_functions flag on SamFunctionProvider to keep them out of sam local. - WebFunction (provider.py) holds only what a Web Function has: runtime, code, entry point, metadata and build info. Its package type is always Zip and it builds for x86_64. - SamWebFunctionProvider extracts them from the stacks; SamFunctionProvider is back to Lambda functions only, so sam local can't see Web Functions. - The build accepts BuildableFunction (Function or WebFunction). get_build_entry_point returns a function's handler or a Web Function's entry point for the builders; only the build graph reads Lambda-only fields, and it skips them for Web Functions. - BuildContext collects Web Functions from SamWebFunctionProvider for all builds, single-resource builds and the not-found message.
Language extensions aren't supported for Web Functions yet: a Fn::ForEach whose items have different code would put the Web Function's code location, which can import the code bucket's name, into generated Mappings, which CloudFormation doesn't allow. When SAM CLI expands language extensions locally and a Fn::ForEach (including nested ones and ones in nested stacks) generates an AWS::Serverless::WebFunction with local code, sam build, sam package, sam deploy and sam sync now fail with WebFunctionInForEachError. Web Functions outside Fn::ForEach, and templates SAM CLI doesn't expand, are unaffected.
…pted stack name In guided mode the deployed stack is the one from the Stack Name prompt, not --stack-name. Using --stack-name left the code bucket off when it wasn't set, sending Web Function code to the managed bucket, and named the bucket stack after the wrong stack when it differed, so sam delete couldn't find it.
upload_local_artifacts returns an s3:// CodeUri unchanged, and the export then rewrote it to import the code bucket, pointing it at an object that was never uploaded there. Only rewrite CodeUri for code this export uploaded.
Every Web Function code change creates a new revision, so continuously syncing it doesn't make sense, and watch mode never watched Web Function code anyway: code changes were skipped and only deployed with the next template change. sam sync --watch now fails before doing anything when the template, including nested stacks, has an AWS::Serverless::WebFunction with local code, and points to sam sync without --watch or sam deploy.
…ode isn't watched Reverts the sam sync --watch error for Web Functions (6c9b004). Watch mode has no code watcher for AWS::Serverless::WebFunction, so a code change never creates a revision by itself; the code is deployed with the next infrastructure sync. sam sync --watch now prints a note saying so when the template, including nested stacks, has a Web Function with local code.
…ndency layer sam sync builds with combine_dependencies=False so that function dependencies go to a separate directory for the auto dependency layer. Web Functions can't use layers and the dependency layer stack only covers Lambda functions, so their dependencies were left out and the deployed Web Function couldn't load them. Web Function builds now always combine dependencies into the artifact.
Web Function code was zipped with make_zip, which keeps each file's mode, so files Lambda couldn't read (for example 0600) were uploaded as is. Add AWS::Serverless::WebFunction to LAMBDA_LOCAL_RESOURCES so it's zipped with make_zip_with_lambda_permissions, like Lambda functions. The list's only other use is that zip-method choice.
delete_web_function_code_bucket treated every describe_stacks error as "no code bucket stack", so access, throttling and credential errors left the code bucket, its stack and its export behind with no warning. Only a missing stack is treated that way now; other errors are raised. Re-running sam delete couldn't recover either, because it stops when the deployed stack is gone. It now still deletes a leftover code bucket stack in that case.
With BuildMethod: esbuild, a Web Function's EntryPoint (for example src/server.ts) was bundled to server.js at the artifact root, but the built template kept the source path, and Web Functions didn't count toward esbuild detection, so source map pre-processing was skipped for them. - esbuild_configured() also checks Web Functions. - The built template's EntryPoint is rewritten to the bundled file when the configured one isn't in the artifact, like a Lambda function's handler. - Sourcemap handling reads and writes NODE_OPTIONS in a Web Function's EnvironmentVariables.
| dependencies_dir, | ||
| download_dependencies, | ||
| self._combine_dependencies, | ||
| self._combine_dependencies or self._is_web_function(function_name), |
There was a problem hiding this comment.
[BUG] A Web Function that shares a build definition with a Lambda function is built without its dependencies
_is_web_function(function_name) only checks the name passed to _build_function. The build strategy passes build_definition.get_function_name(), which is the first function in the definition, and then copies that artifact to every other function in the definition.
FunctionBuildDefinition.__eq__ doesn't compare resource type, and this PR makes Web Functions match on the fields it does compare: imageuri is None, the architecture is x86_64, and the env vars are the same. So an AWS::Serverless::Function and an AWS::Serverless::WebFunction with the same CodeUri, runtime and metadata go into one definition. collect_all_build_resources adds Lambda functions first, so the Lambda function becomes functions[0] and _is_web_function returns False.
With sam sync's auto dependency layer (combine_dependencies=False), the shared artifact is built without dependencies and copied to the Web Function. Web Functions can't use layers, so the deployed Web Function has no dependencies at runtime. Two possible fixes:
- Make the resource type part of build-definition equality, so Web Functions never share a definition with Lambda functions.
- Decide
combine_dependenciesbased on whether any function in the build definition is aWebFunction.
The Fn::ForEach check counted any dict CodeUri without a literal Bucket as
local code, so a per-item S3 location such as !Sub "s3://bucket/${Name}.zip"
was rejected even though packaging never uploads dict values. Treat every
dict CodeUri (an S3 object or an intrinsic) as remote, consistently with
Resource.export and stacks_have_web_functions.
sam delete always emptied and deleted the code bucket after deleting the stack, even when a WebFunction had DeletionPolicy: Retain (or RetainExceptOnCreate) or was in retain_resources. A retained Web Function keeps reading its code from the bucket, so deleting it broke the function. - sam delete keeps the code bucket stack when the deleted template has a retained WebFunction, turns on its termination protection, and says how to delete it later. - Deleting a code bucket stack skips stacks with termination protection on, so re-running sam delete after the stack is gone leaves it in place too.
A Web Function's EntryPoint was split on the first '.' like a Lambda handler, so ./server.ts, src/v1.2/server.ts and server.config.ts gave esbuild the wrong entry file.
Which issue(s) does this change fix?
N/A
Why is this change necessary?
AWS::Serverless::WebFunction(aws-sam-translator 1.115.0) only worked with code that was already in S3.sam buildskipped it, andsam package/sam deployleft a localCodeUriunchanged.Web Function code also can't simply go to the managed bucket. A Web Function reads its code from S3 with the
lambda.amazonaws.comservice principal for as long as a revision exists, not just at deploy time. If the object, or the pinned version, is deleted, the revision fails permanently. The managed bucket is shared across the account and lacks a Lambda read policy. Users can also empty or delete it today without affecting Event Functions, andsam deleteremoves artifacts before the stack.How does it address the issue?
sam buildWebFunction(insamcli/lib/providers/provider.py) models a buildable Web Function: runtime, code, entry point, metadata and build info. Its package type is always Zip, and it builds for x86_64.SamWebFunctionProviderextracts Web Functions from the stacks, including nested stacks, and skips ones whose code is already in S3.SamFunctionProvideris unchanged, sosam localcommands never see Web Functions.BuildableFunction(FunctionorWebFunction):get_build_entry_point()gives the builders a Lambda function's handler, or a Web Function'sEntryPoint(defaultindex.js).imageuri) for Web Functions.BuildContextcollects Web Functions for full builds, single-resource builds (sam build MyWebFn),--exclude, and the "not found" message.Metadata.BuildMethodis supported. The built template points the WebFunctionCodeUriat the build output.BuildMethod: esbuild, the built template'sEntryPointpoints at the bundled file, for examplesrc/server.tsbecomesserver.js, the same way a Lambda function's handler is updated. esbuild gets theEntryPointfile path as is (not split on the first.like a Lambda handler), so paths like./src/v1.2/server.config.tswork and bundle toserver.config.js. Source maps setNODE_OPTIONSin the Web Function'sEnvironmentVariables.sam syncseparates function dependencies for the auto dependency layer, which only covers Lambda functions.Packaging
CodeUriis a packageable local path (ServerlessWebFunctionResource), uploaded through a newDestination.WEB_FUNCTION_S3. That destination falls back to the normal S3 uploader when no separate bucket is set.Per-stack code bucket
sam deploy(--resolve-s3or guided) orsam syncis using the managed bucket, and the template, including nested stacks, has a WebFunction with local code.<stack>-<hash>-WebFunctionCodestack (samcli/lib/bootstrap/web_function_code_bucket.py) and uploads only the Web Function code there.lambda.amazonaws.comcalls3:GetObject/s3:GetObjectVersion, scoped withaws:SourceAccountandaws:SourceArnmatching…:web-function/*/revision/*CodeUrias{Bucket: !ImportValue <export>, Key: <key>}(plusVersion, if any). CloudFormation then refuses to delete the code bucket stack while any stack, including nested stacks, reads code from it.sam deletedeletes the other artifacts and the stack as before, and only after the stack is deleted empties the code bucket (all versions) and deletes the code bucket stack.ServerlessWebFunctionResource.deleteis a no-op, so code is never deleted while revisions might still read it.ManagedStackSource: AwsSamClitag are left alone.sam deleteleft the code bucket stack behind after deleting the stack, runningsam deleteagain deletes it.DeletionPolicy: RetainorRetainExceptOnCreate, or inretain_resources), its revisions keep reading code from the bucket. Sosam deletekeeps the code bucket stack, turns on its termination protection, and tells the user how to delete it later.sam deletenever deletes a code bucket stack that has termination protection on, including on a re-run.sam package --resolve-s3sam packagehas no stack name, so it can't create a per-stack code bucket, and Web Function code can't go to the shared managed bucket. When the template, including nested stacks, has a WebFunction with local code,sam package --resolve-s3fails before creating or using the managed bucket. The error says to usesam deploy(per-stack code bucket) or--s3-bucketwith a versioned bucket whose policy lets Lambda read the code.sam package --s3-bucketworks as below.--s3-bucket--kms-key-idis an AWS managed keylambda.amazonaws.comread the uploaded objects (prefix and partition are considered; conditions aren't evaluated)kms:DecryptstatementLanguage extensions (
Fn::ForEach)--language-extensions), and aFn::ForEachgenerates anAWS::Serverless::WebFunctionwith local code,sam build,sam package,sam deployandsam syncfail withWebFunctionInForEachError. This includes nestedFn::ForEachblocks and nested stacks.Fn::ForEachwhose items have different code puts each item's code location into generatedMappings. The location imports the code bucket's name (Fn::ImportValue), and CloudFormation doesn't allow intrinsics inMappings.Fn::ForEach, and templates SAM CLI doesn't expand locally, are unaffected.sam sync --watchCodeTriggerFactory.GENERATOR_MAPPING), so a code change never triggers a sync by itself. The code is deployed with the next infrastructure sync, for example when the template changes.sam sync --watchprints a note at startup: "Web Function code isn't watched; code changes are deployed with the next infrastructure sync."Nested stacks
What side effects does this change have?
sam deleteremoves.--s3-bucketmakes up to two extra S3 calls, plus two KMS calls with--kms-key-id.--kms-key-idisn't applied to the code bucket uploads, which use the bucket's SSE-S3 default.sam deleteprints the existing "Cannot resolve s3 bucket information" warning and leaves the packaged template file in the managed bucket, as it does today for templates with no other artifacts.sam syncflow for Web Functions.sam syncdeploys them with an infrastructure sync, andsam sync --watchdoesn't watch their code (see above).sam localsupport.Fn::ForEach(rejected, see above).sam package --resolve-s3work for Web Functions. It would add a helper Lambda function and bucket-cleanup resources to every stack.Testing
WebFunction,SamWebFunctionProvider, theBuildContextcollection and the build graph entry pointPackageContextsam deleteorder (the code bucket is deleted after the stack)--s3-bucketcheckspytest tests/unit(excludingcfn_language_extensions): 94.07% coverage. One existing test,test_template.py::Test_update_sam_mappings_relative_paths::test_updates_imageuri_when_pointing_to_local_archive, fails locally both with and without this change. It writes../../my-image.tar.gzrelative to the checkout, which isn't writable in my environment.black,ruffandmypypass.CodeUriand an npm dependency:sam buildinstalled the dependency.sam deploy --resolve-s3created the…-WebFunctionCodestack and uploaded the code there. The deployed template'sCodeUripoints at the code bucket, and the endpoint returned 200 from the dependency-using code. The managed stack'sLastUpdatedTimedidn't change.CodeUriusesFn::ImportValue: <stack>-<hash>-WebFunctionCode-Bucket, andlist-importsshows the app stack importing it.aws cloudformation delete-stackon the code bucket stack was refused ("Delete canceled. Cannot delete export … as it is in use by …"). The stack stayedCREATE_COMPLETE, and the endpoint kept returning 200.sam delete --no-promptsdeleted the stack, then the code bucket stack. The bucket and the export no longer exist.sam deleteremoved the leftover code bucket stack.DeletionPolicy: Retainon the WebFunction:sam deletekept the code bucket stack with termination protection on, a secondsam deletekept it, a directaws cloudformation delete-stackwas blocked, and the retained function kept returning 200.AWS::Serverless::Applicationwhose child template has the WebFunction: built, deployed, 200 from the child endpoint, deleted in order.--language-extensions,sam buildandsam packageon aFn::ForEachthat generates Web Functions failed with the new error. A template with a Web Function outsideFn::ForEach, next to aFn::ForEachof SNS topics, built normally.sam sync(auto dependency layer on) on a template with a WebFunction and a Node.js Lambda function: the WebFunction artifact keptnode_moduleswhile the Lambda function's dependencies went to the layer. The synced WebFunction returned 200 using its dependency, andsam deletecleaned up.EntryPoint: src/server.tsbuilt toserver.jswithEntryPoint: server.jsin the built template, deployed, and returned 200 using its bundled dependency.EntryPoint: ./src/v1.2/server.config.tsbuilt toserver.config.js(it failed with "entry point ... does not exist" before the fix).sam sync --watchon the WebFunction project printed the note before the sync confirmation prompt.sam package --resolve-s3on a template with a local-code WebFunction, and on a nested template whose child has one, failed with the new error, without creating any AWS resources.--s3-bucketchecks withsam packageagainst new buckets:--kms-key-id alias/aws/s3: failed with the AWS managed key error.Mandatory Checklist
PRs will only be reviewed after checklist is complete
make prpassesmake update-reproducible-reqsif dependencies were changedBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.