-
Notifications
You must be signed in to change notification settings - Fork 872
Generate PutBucketMetricsConfiguration and add required check for query string parameters #4101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: development
Are you sure you want to change the base?
Conversation
…equired query string params are set. Remove visitor pattern and move custom logic for putlifecycleconfiguration marshaller to custom partial method
| @@ -282,7 +282,8 @@ public void AbortMultipartUploadTest() | |||
| var request = new AbortMultipartUploadRequest | |||
| { | |||
| BucketName = accessPointArn, | |||
| Key = "foo.txt" | |||
| Key = "foo.txt", | |||
| UploadId = "testing" | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since we are validating query string params now we must populate UploadId. Also, this test was only validating the resource path and not that the operation worked. If this was actually to be called, it would've turned into a DeleteObjects call.
| { | ||
| #> | ||
| if (string.IsNullOrEmpty(<#=variableName#>.<#=member.PropertyName#>)) | ||
| throw new <#=this.Config.BaseException#>("Request object does not have required field <#=member.PropertyName#> set"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Throwing the base exception to be consistent with what we do for resource path members
| if (member.Shape.IsString) | ||
| { | ||
| #> | ||
| if (string.IsNullOrEmpty(<#=variableName#>.<#=member.PropertyName#>)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can you make it consistent with having { for the if statements. im having a hard time figuring out where the if (<#=variableName#>.<#=member.PropertyName#> == null) case is applying. if its inside an if or else block i cant tell easily
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the code we will generate and not the actual logic, so I kept the brackets off to be consistent with other places where we throw exceptions
| if (member.Shape.IsString) | ||
| { | ||
| #> | ||
| if (string.IsNullOrEmpty(<#=variableName#>.<#=member.PropertyName#>)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there any reason to not ship this change (query string required field) as its own thing? in case there is some issue that arises.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, PutBucketMetricsConfiguration has a required query string so even this operation uses this logic. I do understand the concern though. We do really need to ship the change for AbortMPU though since that is the one where the operation can actually change, so I'd rather just ship it in this PR😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i was thinking if its not strictly required, then could just do it separate to make it easier to review and rollback if ever required in the future. will let david/muhammad decide though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Who's David? 🤔
I prefer the smaller PRs, they're easier to review and make me more confident we're not shipping anything we shouldn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lol woops daniel :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gotcha, just to be clear we're talking about shipping it for S3 then all other services separately?
I can make the change but I would have to
- add a condition in the generator to only do this for S3
- Have a separate PR that just removes that condition.
If we're okay with that I'll go ahead and do it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR generates code for S3's PutBucketMetricsConfiguration operation and adds validation for required query string parameters across the AWS SDK for .NET. The changes include:
- Generating the
PutBucketMetricsConfigurationoperation for S3 - Replacing hand-written marshaller/unmarshaller code with generated code
- Adding validation to ensure required query string parameters are set before making requests
- Refactoring the code generation approach from a visitor pattern to an injection-based approach for custom marshalling logic
Reviewed Changes
Copilot reviewed 12 out of 23 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
sdk/test/Services/S3/UnitTests/Custom/S3AccessPointTests.cs |
Adds missing UploadId property to test request |
sdk/src/Services/S3/Generated/Model/PutBucketMetricsConfiguration*.cs |
Generated request/response models with updated documentation and field naming conventions |
sdk/src/Services/S3/Generated/Model/MetricsConfiguration.cs |
Generated model with updated documentation and field naming |
sdk/src/Services/S3/Generated/Model/Internal/MarshallTransformations/*.cs |
Generated marshallers with required field validation for query string parameters |
sdk/src/Services/S3/Custom/Model/Internal/MarshallTransformations/PutLifecycleConfigurationRequestMarshaller.cs |
Extracts filter marshalling logic to a custom method |
sdk/src/Services/S3/Custom/Model/Internal/MarshallTransformations/PutBucketMetricsConfiguration*.cs |
Custom marshaller/unmarshaller with filter marshalling and checksum logic |
generator/ServiceModels/s3/s3.customizations.json |
Configuration updates for code generation, removes visitor pattern |
generator/ServiceClientGeneratorLib/*.cs |
Updates code generator to use injection-based approach instead of visitor pattern |
generator/.DevConfigs/*.json |
Dev config files documenting the changes |
generator/ServiceClientGeneratorLib/Generators/Marshallers/RestXmlRequestMarshaller.tt
Show resolved
Hide resolved
|
I'm pasting the results of the AI analysis done on the larger PR here The added validation isn't a breaking change because the request will fail and throw Now I have all the information needed to provide a comprehensive analysis. Let me compare the deleted custom files with the generated ones and examine all the marshaller changes: Task Completed Breaking Changes Analysis - S3 Custom to Generated MigrationAfter analyzing all deleted custom files, their generated replacements, and modified marshallers, I have identified the following breaking changes: BREAKING CHANGES IDENTIFIED:1. AbortMultipartUploadRequestMarshaller.cs - NEW VALIDATION ADDEDIssue: Added new required field validation that didn't exist before: if (string.IsNullOrEmpty(publicRequest.UploadId))
throw new AmazonS3Exception("Request object does not have required field UploadId set");Impact: Customer code that previously could make AbortMultipartUpload requests without UploadId set will now throw an exception. 2. CopyPartRequestMarshaller.cs - NEW VALIDATIONS ADDEDIssue: Added two new required field validations: if (publicRequest.PartNumber == null)
throw new AmazonS3Exception("Request object does not have required field PartNumber set");
if (string.IsNullOrEmpty(publicRequest.UploadId))
throw new AmazonS3Exception("Request object does not have required field UploadId set");Impact: Customer code that previously could make CopyPart requests without PartNumber or UploadId set will now throw exceptions. 3. ListPartsRequestMarshaller.cs - NEW VALIDATION ADDEDIssue: Added new required field validation: if (string.IsNullOrEmpty(publicRequest.UploadId))
throw new AmazonS3Exception("Request object does not have required field UploadId set");Impact: Customer code that previously could make ListParts requests without UploadId set will now throw an exception. NO BREAKING CHANGES FOUND IN:Model Files Migration (✅ SAFE)
Differences are non-breaking:
Marshaller Logic Preservation (✅ SAFE)
SUMMARY:
The breaking changes are all related to new validation logic being added to existing marshallers rather than logic being lost during migration. These validations enforce that required fields are set before making API calls, which technically makes the SDK more strict than before. |
…equired query string params are set. Remove visitor pattern and move custom logic for putlifecycleconfiguration marshaller to custom partial method
Description
This PR generates the S3 operation PutBucketMetricsConfiguration. It also adds a check to see if a required query string member is set in the marshallers, which also affects other services as well. This is necessary so that a different operation isn't called if a required query string parameter isn't set.
This also moves some code to partial methods for PutLifecycleConfigurationRequest, since I removed the ImplementsVisitorPattern customization hook since generating PutBucketMetricsConfiguration using that customization hook logic wasn't possible. I decided to just move that logic to a custom partial so we can have an exact 1:1 mapping of that previous logic.
Best way to review is by pulling up the custom version of the file on one side and the generated version of the file on the other side and compare.
Motivation and Context
Testing
#4090 dry run passed (contains 4 generated operations and this branch is branched off of this PR)
Screenshots (if appropriate)
fuzz-tests for the larger PR all passed.
See output of larger PR's AI prompt (all passed)
Types of changes
Checklist
License