You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reconcile the two config namespaces for native writes #5306
One is spark.comet.write.<format>.<feature>, the other spark.comet.<format>.write.<feature>. Both are CATEGORY_TESTING and default to false, so nothing user-facing is broken yet — which makes this the cheap moment to pick one shape, before either graduates and the key has to be kept as a deprecated alias.
Worth settling alongside it: the operator opt-in for native Parquet writes is spark.comet.operator.WriteFilesExec.allowIncompatible (with spark.comet.operator.DataWritingCommandExec.allowIncompatible retained as an alternative), so a user enabling native Parquet writes currently sets two configs while the Iceberg feature needs one.
Describe the potential solution
Pick one namespace and move the other with withAlternative, per the rename checklist in docs/source/contributor-guide/config_conventions.md. spark.comet.write.* reads better as writes grow more formats and features, but either is fine as long as it is one.
Activity
chore: move native write configs under spark.comet.write (apache#6494)