Skip to content

Add schema-serde streaming JSON/XML deserializers, ec2Query, Codec, and ClientProtocol - #3910

Open
pulimsr wants to merge 6 commits into
mainfrom
schema-serde
Open

pulimsr wants to merge 6 commits into
mainfrom
schema-serde

Conversation

@pulimsr

@pulimsr pulimsr commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Adds response deserialization and the protocol-selection layer to the schema-serde runtime.

  • ShapeDeserializer reworked to a push/consumer model (ReadStruct/ReadList/ReadMap), the dual of ShapeSerializer; scalars take const Schema&.
  • JsonShapeDeserializer and XmlShapeDeserializer (new), CborShapeDeserializer updated to match.
  • QueryShapeSerializer: full nesting + ec2Query flavor (new Ec2QueryNameTrait).
  • Codec (Json/Xml/Cbor) pairs serializer+deserializer; ClientProtocol picks codec + content-type per protocol (restJson1, awsJson1.0/1.1, rpcv2Cbor, restXml, awsQuery, ec2Query),
    including the asymmetric query request/XML response case.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@pulimsr
pulimsr marked this pull request as ready for review September 1, 2026 16:51
Comment thread src/aws-cpp-sdk-core/include/smithy/client/schema/Codec.h Outdated
Comment thread src/aws-cpp-sdk-core/source/smithy/client/schema/SerializableStruct.cpp Outdated

namespace {

class Person : public SerializableStruct {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at my desk i was going "yes, yes, yes, yes, yes, exactly" 💯

@pulimsr
pulimsr force-pushed the schema-serde branch 2 times, most recently from a885d30 to 755dd92 Compare September 8, 2026 20:33
@pulimsr
pulimsr force-pushed the schema-serde branch 3 times, most recently from a810118 to b877f00 Compare September 18, 2026 14:20
Aws::String GetProtocolId() const override;
Aws::String GetContentType() const override;
SerializerOutcome SerializeInput(const Schema& schema, const SerializableStruct& input) const override;
Aws::UniquePtr<ShapeDeserializer> CreateOutputDeserializer(const unsigned char* data, size_t length) const override;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not absolutely needed on this, but lets talk about the API

const unsigned char* data, size_t length

this specific very common and normal api has caused a substantial amount of pain. so much so that c++ introduced a library type called std::span. The CRT also has the same idea as aws_byte_cursor. it might be worth seeing if you can use ByteCursor instead of decomposition of it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switched to Aws::Crt::ByteCursor throughout. Codec::CreateDeserializer/DeserializeShape, ClientProtocol::CreateOutputDeserializer/DeserializeResponse, and the three {Json,Xml,Cbor}ShapeDeserializer ctors now take a single ByteCursor instead of (const unsigned char*, size_t)


// Sketch of a generated, schema-driven shape: it owns its Schema, writes its
// members (push), and populates itself from per-member deserialize callbacks.
class Widget : public SerializableStruct {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a test like this for XML? i didnt see one, and i think theres def value in having the full "fake codegen expierence tests" for each deserializer

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya XML and CBOR didn't have one, only JSON did, I've added DeserializesNestedIntoClass to both XmlShapeDeserializerTest and CborShapeDeserializerTest

if (!token.has_value()) {
return {};
}
DateTime parsed(*token, DateFormat::ISO_8601);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for both read and write timestamps on all codecs does hardcoding ISO_8601 work? i think this would break both in the read and write path where timestampFormat is a trait on the shape. is there protocol tests around this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was hardcoded and would break any shape overriding the format. TimestampFormatTrait existed but nothing read it. Fixed in 9caf95f

Aws::Crt::Optional<Aws::String> ReadString(const Schema& schema) override;
Aws::Crt::Optional<Aws::Utils::DateTime> ReadTimestamp(const Schema& schema) override;
Aws::Crt::Optional<Aws::Utils::ByteBuffer> ReadBlob(const Schema& schema) override;
Aws::Crt::Optional<int> ReadEnum(const Schema& schema) override;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is late in the game, but should ReadEnum return a int? also should ReadEnum exist? smithy java doesnt have it. since enums are represented as strings on the wire. shouldnt we keep them as string, and allow enums to interpret that value?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing WriteEnum/ReadEnum

…fake-codegen deser tests, remove enum methods

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants