Durable execution fixes - #2585
Open
dscpinheiro wants to merge 3 commits into
Open
dscpinheiro wants to merge 3 commits into
dscpinheiro wants to merge 3 commits into
Conversation
The service rejects a stale token with "Invalid checkpoint token", but IsTerminalCheckpointError matched "Invalid Checkpoint Token" with an ordinal comparison. A stale token was classified as terminal and failed the whole execution instead of letting Lambda retry the invocation. Match case-insensitively and test with the service's real message. Fixes #2571
A checkpoint response without a CheckpointToken means the service will accept no more checkpoints from this invocation. Previously the next flush sent an empty token, got a 400, and failed the execution. CheckpointBatcher now stops flushing, leaves pending awaiters suspended, and terminates with CheckpointTokenMissing, so the invocation returns Pending and the execution continues in the next invocation. Add TestRunnerOptions.OmitCheckpointToken so workflows can be tested against this path. Fixes #2572
Steps and callbacks always recorded "Step"/"Callback", so operations inside a composed child context could only be told apart by name. StepConfig.SubType and CallbackConfig.SubType override the default on every checkpoint; WaitForCallbackConfig applies it to its inner callback. Validate all user-supplied subtypes, including ChildContextConfig.SubType, against the service constraint (1-32 chars from [a-zA-Z0-9-_]) and throw ArgumentException before checkpointing, instead of failing the whole checkpoint request with a 400. Null or empty selects the default. Fixes #2583
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The concurrency-sensitive suspension behavior and multi-package API changes warrant final human validation.
Review effort: Balanced
Findings: None
What changed in this PR
Addresses durable execution issues #2571, #2572, and #2583.
Changes:
- Corrects stale checkpoint-token error classification and handles missing response tokens as suspension.
- Adds validated custom subtypes for steps, callbacks, and child contexts.
- Extends the testing framework and coverage for these behaviors.
| File | Description |
|---|---|
SubTypeTests.cs |
Tests subtype defaults, propagation, and validation. |
MockLambdaClient.cs |
Simulates omitted checkpoint tokens. |
DurableFunctionTests.cs |
Tests stale and missing token handling. |
CheckpointBatcherTests.cs |
Tests suspension after missing tokens. |
RunAsyncTests.cs |
Tests local-runner resumption. |
WaitForCallbackConfig.cs |
Documents callback subtype behavior. |
StepConfig.cs |
Adds step subtype configuration. |
TerminationManager.cs |
Adds missing-token termination reason. |
SubTypeValidator.cs |
Validates and resolves subtypes. |
StepOperation.cs |
Propagates configured step subtypes. |
ChildContextOperation.cs |
Validates child-context subtypes. |
CheckpointBatcher.cs |
Suspends flushing when tokens are omitted. |
CallbackOperation.cs |
Propagates callback subtypes. |
DurableFunction.cs |
Returns pending and fixes stale-token matching. |
DurableContext.cs |
Forwards composite callback subtypes. |
ChildContextConfig.cs |
Documents subtype constraints. |
CallbackConfig.cs |
Adds callback subtype configuration. |
TestRunnerOptions.cs |
Adds omitted-token simulation. |
README.md |
Documents the testing option. |
InMemoryDurableServiceClient.cs |
Implements token omission. |
FunctionRegistry.cs |
Isolates nested-runner simulation. |
DurableTestRunner.cs |
Wires token omission into the client. |
durable-step-callback-subtype.json |
Records subtype feature release. |
durable-stale-checkpoint-token.json |
Records stale-token fix. |
durable-missing-checkpoint-token.json |
Records missing-token changes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
jnunn-aws
approved these changes
Oct 2, 2026
jnunn-aws
left a comment
jnunn-aws
left a comment
Contributor
There was a problem hiding this comment.
The new SubType option isn't in the docs yet. Can we add it to the StepConfig block in docs/core/steps.md and the CallbackConfig block in docs/core/callbacks.md, with the 1-32 chars [a-zA-Z0-9-_] rule? For WaitForCallbackConfig, it'd help to say it's applied to the inner callback, not the wrapping context.
Contributor
|
There is also the durable execution docs in some other repo you should update too |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: #2571, #2572, and #2583 (one commit for each issue)
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.