HDDS-16039. Introduce S3StorageClass mapping to OzoneStoragePolicy.

[devmadhuu]

What changes were proposed in this pull request?

This PR is to introduce S3StorageClass — a small enum that maps AWS S3 storage class strings (STANDARD, STANDARD_IA, GLACIER) to Ozone storage policies (HOT, WARM, COLD).

Two helper methods on the enum:

• fromS3StorageClass(String) — parse the value of an incoming x-amz-storage-class header.
• fromStoragePolicy(StoragePolicy) — go the other way, for building responses.

This PR is infrastructure only — no caller uses S3StorageClass yet. The S3 write path (PutObject) and read path (GetObject) will start calling it in follow-up PRs.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16039

How was this patch tested?

New unit test TestS3StorageClass covers:

• S3 string → enum for all three values, plus rejects null and unknown values.
• Ozone StoragePolicy → enum for all three, plus rejects null.
• Round-trip consistency across all enum values.

[Gargi-jais11]

Thanks @devmadhuu for this PR.

[Gargi-jais11]

I don't know whether u are aware or not but S3StorageType.java in the S3 Gateway has similar enums means: “this S3 storage class string picks how many copies / EC layout the object gets” (STANDARD → 3-way Ratis, STANDARD_IA → EC, etc.). Almost all upload/download code uses that today.

This PR adds S3StorageClass, which means: “this string picks Hot / Warm / Cold (which disk tier).”

Those are two different ideas with similar names. That will confuse people and cause wrong imports later.

We should look into these and check how they both can be used independently.

[xichen01]

AWS's StorageClass https://aws.amazon.com/s3/storage-classes does not explicitly specify the replica types or storage media types for each StorageClass; instead, it defines concepts such as durability, SLA, zones, and latency , etc for different StorageClasses

Ozone’s S3 StorageClasses can have custom definitions; users can specify the storage medium (storage policy) and replica type through S3 StorageClass parameter, which does not conflict with AWS S3’s StorageClasses.

[devmadhuu]

@xichen01 so what is the way to go forward ? should we combine both and adapt the concept of durability, SLA, zones, and latency , etc as below in existing S3StorageType

Add a second constructor arg. GLACIER gets null replication, meaning "no replication opinion, archive tier only" — AWS does not pin storage class to a replica count,
and inventing one for GLACIER would make it indistinguishable from STANDARD_IA on the reverse map.

   REDUCED_REDUNDANCY(RatisReplicationConfig.getInstance(ReplicationFactor.ONE),   HOT),                                                                                                                   
   STANDARD(RatisReplicationConfig.getInstance(ReplicationFactor.THREE),           HOT),                                                                                                                   
   STANDARD_IA(new ECReplicationConfig(3, 2),                                      WARM),                                                                                                                  
   GLACIER(null,                                                                   COLD);   

[devmadhuu]

@xichen01 @ivandika3 @F64116045 @greenwich Kindly advise.

[F64116045]

@devmadhuu Thanks for proposing a way to combine the two mappings
If we combine them, would the reverse mapping then use both replication config and storage policy? I think it needs both to distinguish REDUCED_REDUNDANCY from STANDARD, since both are HOT. If so, is null still needed for GLACIER? COLD already distinguishes it from STANDARD_IA.

[devmadhuu]

Kindly have a look and we can discuss more if needed or good to go with this design ?
cc: @ChenSammi

[xichen01]

@devmadhuu

@Gargi-jais11 Agree that fields should be optional. On your question "Where does the reverse direction get the policy?", this patch #11377 keys can override the bucket default.
The reverse direction reads KeyInfo.storagePolicy and never fills a missing value from the bucket. So SetBucketProperty to COLD won't relabel existing objects as GLACIER. KeyInfo.storagePolicy is only set when a policy was explicitly requested on the write, so a key that inherited its policy has null, and null falls back to the replication-based mapping.

Once the storage policy feature for OM is launched, every Key will have a storagePolicy. The priority order is as follows:

1. Explicitly specified when the key is created;
2. If not specified for the key, the bucket’s storage policy is used (if not null);
3. If not specified when the key is created and the bucket’s storage policy is also null, OzoneStoragePolicy.getDefaultPolicy() is used;

can see:
https://github.com/apache/ozone/pull/11377/changes#diff-2841a017f8d700d802af1b1c49fea9ca281098728796384f5b2e79c23f09ab65R998

Thus, for keys created before the storage policy feature was launched, the storagePolicy is null; after the feature is launched, every Key will have a storagePolicy.

It can be understood that once a storage policy is specified, it will not change (unless you set it manually).

• The storagePolicy attribute of a Key is independent and will not change based on changes to the bucket’s storagePolicy.
• Even if a fallback occurs during key write (when allowFallbackStoragePolicy=true), this will not change the key’s storagePolicy. Fallback is an SCM-side behavior and does not affect the key’s storagePolicy

Therefore, the reverse map simply retrieves the storage policy directly based on the key (the same applies to replication type).

[SaketaChalamchala]

@devmadhuu +1 to your proposal to add a gateway setting for the storage policy for STANDARD/REDUCED_REDUNDANCY storage classes.
IMO whether to strictly enforce storage policy for these classes and how can be scoped for a future phase.

[Gargi-jais11]

@devmadhuu +1 to the proposal.

[devmadhuu]

Thanks @xichen01, you're right, and I had it wrong. OM resolves the policy in OMKeyCreateRequest.preExecute (key, then bucket, then OzoneStoragePolicy.getDefaultPolicy()) and writes it back into KeyArgs, so every new key stores a policy. That includes keys in buckets with no policy, which get the default, WARM. Only keys written before the feature have null.

Thanks @Gargi-jais11 and @SaketaChalamchala for +1 to proposal. So the reverse direction just reads the key's own replication and storage policy. The rule stays lenient: replication decides the class, and only COLD changes it. null falls back to replication, as today.

This means mapping STANDARD to HOT through the gateway setting doesn't change what any object reports. Old keys (RATIS/THREE + null) and new ones (RATIS/THREE + HOT or WARM) all report STANDARD. It only changes where new STANDARD writes go. To keep that true, the setting will accept HOT, WARM or unset for STANDARD/REDUCED_REDUNDANCY, but not COLD.

[Gargi-jais11]

IllegalArgumentException is fine for a util with no callers yet but PutObject must map unknown classes to OS3Exception / InvalidStorageClass, not a 500.

[greenwich]

One thing worth settling before the enum shape: for S3 buckets that are links, which bucket supplies the policy?
A bucket created through the S3 API is a real bucket in /s3v, so inheriting from the bucket works fine there. But where an existing Ozone bucket is exposed via ozone sh bucket link, the gateway reads the link row's properties: getBucketInfo doesn't resolve links (OzoneManagerUtils:63), while the key write is rewritten to the resolved target (OMKeyRequest:141, bucket.update(keyArgs)).

Today that already means replication comes from the link rather than the target; if the tier defaults from the bucket policy, it hits the same split, with the link row unset and the real bucket carrying the intended policy.

[devmadhuu]

Thanks @greenwich for raising the linked-bucket case. I traced the current HDDS-11233 flow. Although the low-level OzoneManagerUtils.getBucketInfo() returns the raw link row, the S3 client path goes through
OzoneManager.getBucketInfo(), which calls enrichLinkBucketInfo(). That method resolves the real target and withOperationalPropertiesFrom() copies both storagePolicy and defaultReplicationConfig from the target while preserving the link identity. The OM key-write path then resolves to the same target bucket. Therefore, the target bucket is the source of both policy and replication defaults. I’ll add an S3 linked-bucket regression test when the storage-class mapping is connected to the write path in the follow-up PR.

[devmadhuu]

@SaketaChalamchala @xichen01 Kindly have a relook. This is only a framework in this PR. read and write paths will follow in future PR