test: run the S3 suites against an available MinIO image

[dwsmith1983]

Which issue does this PR close?

Closes #6884.

Rationale for this change

Docker Hub no longer serves minio/minio, so every suite built on CometS3TestBase aborts in beforeAll before a test runs. Once the image pulls, two more gaps show up that only stayed hidden because these suites ran on machines with an AWS region configured:

• the client createBucketIfNotExists builds has no region, so the AWS SDK's region chain fails;
• the REST catalog tests vend s3.region, which iceberg-rust reads, but Iceberg's JVM S3FileIO reads client.region, so the INSERT that writes the table fails the same way.

What changes are included in this PR?

• CometS3TestBase starts cgr.dev/chainguard/minio:latest, declared with asCompatibleSubstituteFor("minio/minio") so MinIOContainer accepts it. Chainguard's free images are only published as latest, and the old reference was minio/minio:latest too.
• The bucket setup client sets Region.US_EAST_1, the region the suites already pass to the catalogs.
• The REST catalog vended properties in IcebergReadFromS3Suite and CometS3CredentialBridgeSuite add client.region, as CometIcebergWriteDetectionSuite already does. s3.region stays for the native scan.

How are these changes tested?

No workflow runs these suites, so I ran them in a Linux container with no AWS configuration. On main, ParquetReadFromS3Suite aborts with pull access denied for minio/minio. With this change, ParquetReadFromS3Suite, IcebergReadFromS3Suite, CometS3CredentialBridgeSuite and HadoopS3ACredentialProviderAdapterBridgeSuite pass (27 tests), and the -Pspark-3.5 -Pstrict-warnings test compile passes.

REST catalog credential vending rejects wrong credentials now gets the 403 it checks for. Without a region it failed on a region error instead.

[sunchao]

Reviewed the full three-file diff from 019f3be9d106a8076e8ab22ba31466195bdf3e5b to 986c87287ba111a3a474626a456d5ac9c411977f. The PR is not a draft. No introduced P1/P2 issues found within this review.

Summary

• Prior state and problem: The S3 suites depended on the unavailable minio/minio image. Bucket setup and REST-catalog writes also depended on an externally configured AWS region.
• Design approach: Replace the shared fixture’s image with cgr.dev/chainguard/minio:latest and configure the region explicitly where each client consumes it.
• Correctness: Testcontainers 1.21.4 supports asCompatibleSubstituteFor, and its launch command matches Chainguard’s documented MinIO entrypoint. AWS SDK 2.31.51 uses an explicitly supplied region before its default chain. The existing wrong-credentials test still requires a 403, and scan assertions remain unchanged.
• Compatibility analysis: Checked Spark catalog initialization in the configured Spark 3.4.3, 3.5.9, 4.0.4, 4.1.3, and 4.2.0 sources. Checked client.region handling in Iceberg 1.5.2, 1.8.1, 1.10.0, and 1.11.0. The existing Spark 4.2 Iceberg exclusion remains intact. Comparison with branch-1.1 shows the intended fixture changes without a production behavior change attributable to this PR.
• Key design decisions: Keep both region properties at us-east-1. Iceberg’s JVM client reads client.region, and the pinned native implementation accepts it with precedence over s3.region. Their identical values preserve native behavior.
• Implementation sketch: CometS3TestBase changes the image declaration and sets Region.US_EAST_1 on bucket creation. Three REST credential maps gain client.region.
• Performance: Changes occur during test setup and client configuration. No query execution overhead is introduced. Explicit region selection removes the affected clients’ need for region discovery. Container download and startup performance were not measured.
• Design: The fix stays in the shared fixture and affected REST configurations. It preserves container lifecycle, endpoint selection, credentials, and path-style access.
• Abstraction & complexity: Existing Testcontainers and AWS SDK APIs express the change directly. No new wrapper, version branch, or production configuration is needed.
• Behavioral changes worth calling out: Tests use a different image registry and no longer require an ambient region for the affected clients. The mutable latest tag policy already existed. Production Spark semantics and fallback decisions are unchanged.
• Suggested improvements: No additional code change meeting the P1/P2 reporting bar was identified.

Routed skills: Read AGENTS.md and used .ai/skills/review-comet-pr/SKILL.md. No sibling subsystem skill applies to these fixture-only changes. The snapshot and live discussion checks contained no reviews, issue comments, inline comments, or review threads.

Exact-head CI: Only label passed. Comet CI, CodeQL, and Check PR Title report action_required. Comet CI has no executed jobs, so there is no build or test verdict. The four affected S3 suites are explicitly designated manual suites.

Validation limits: git diff --check passed. Inspected surrounding fixtures, Testcontainers and AWS SDK sources, supported Spark/Iceberg sources, and Chainguard’s published image configuration. A bounded Docker pull failed because this environment blocks access to cgr.dev. No JVM/native build or S3 suite ran locally. The author reports 27 passing tests and strict Spark 3.5 test compilation, but those results were not independently reproduced. Project files were unchanged.

[dwsmith1983]

#5365 needs this too. Its Delta S3 job builds on CometS3TestBase and fails on the minio/minio pull (https://github.com/apache/datafusion-comet/actions/runs/38233957666/job/114789734253), so that job stays red until this merges and main is merged into that branch.

[andygrove]

The Chainguard image looks like the right replacement to me. Upstream MinIO is archived and no longer publishes a public community image, this one is built from Chainguard's maintained fork, and Trino and Apache Hudi have already switched to it.

Nit, optional: Which Spark profile did you run the four suites on? On 3.4 and 3.5, S3A is Hadoop 3.3.4 on the v1 SDK, a different client from the 4.x builds, and this server is stricter about SigV4 than the old image was. Your Delta S3 run on 3.5 in #5365 is a good sign. A run of these four on -Pspark-3.5 would confirm they work everywhere we use them.

[andygrove]

Nit, optional: Could we pin this by digest rather than latest, the way Trino does (cgr.dev/chainguard/minio@sha256:... with a comment naming the MinIO release)? Chainguard rebuilds latest from its fork regularly, and the fork is still tightening request checks. Its 2026-09-22 release started rejecting unsigned x-amz-* headers in SigV4 verification. Once #5365's Delta S3 job runs on this fixture, a rebuild could turn it red with no change on our side. A digest also means everyone runs these suites against the same server. It should be the multi-arch index digest so it still works on arm64.

[dwsmith1983]

Pinned in b71e80e to the multi-arch index digest of RELEASE.2026-09-22T19-25-18Z. The 3.5 and 4.1 runs against it are in #6887 (comment).

[dwsmith1983]

Which Spark profile did you run the four suites on?

The first run was on 4.1. With the pinned image below, all four pass on -Pspark-3.5 and on 4.1 (27 tests each), and the strict warnings build passes.

Could we pin this by digest rather than latest

Pinned to the multi-arch index digest sha256:dc455cdd4db6e390b1b26514f15ff37681a777bf8dab41aac445a0259f751e56, which is MinIO RELEASE.2026-09-22T19-25-18Z, and the comment names that release. It's the release that started rejecting unsigned x-amz-* headers, so the 3.5 run covers the stricter check.