Native Iceberg writes use an iceberg-rust revision missing partition-path escaping #5636

Describe the bug

The native Iceberg write path currently pins iceberg-rust revision:

3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7

This revision predates apache/iceberg-rust#2875, which fixed PartitionSpec::partition_to_path to URL-encode partition field names and values.

As a result, Comet's native Iceberg writer can generate unescaped partition paths for values containing special characters such as #, ?, or /.

This is not only a directory-layout difference. For example, an S3 partition value containing # can produce a committed location such as:

s3://bucket/table/data/p=a#b/file.parquet

Iceberg Java's S3FileIO treats # and ? as URI fragment/query delimiters when resolving the object key, so a location produced by the older iceberg-rust behavior can resolve to a different key when later read through the standard Iceberg Java path.

Upstream iceberg-rust already fixed this in:

• URL encode field names with special characters for partition paths iceberg-rust#2874
• feat(writer): URL encode field names for partition paths iceberg-rust#2875

Steps to reproduce

No response

Expected behavior

Native Iceberg writes should generate partition paths compatible with Iceberg Java's PartitionSpec.partitionToPath behavior.

Comet should consume an iceberg-rust revision containing apache/iceberg-rust#2875 and include a native-write → JVM-read interoperability regression test for escaped partition paths.

Additional context

This was found during follow-up review of #5361.

The upstream issue has already been fixed, so no new iceberg-rust issue is required. The remaining work is for Comet to update its pinned iceberg-rust revision and verify interoperability.

Upstream fix:

• URL encode field names with special characters for partition paths iceberg-rust#2874
• feat(writer): URL encode field names for partition paths iceberg-rust#2875