Skip to content

Core: Validate implementation classes loaded from properties - #3951

Open
akashchamp wants to merge 2 commits into
apache:mainfrom
akashchamp:fix/validate-implementation-types
Open

akashchamp wants to merge 2 commits into
apache:mainfrom
akashchamp:fix/validate-implementation-types

Conversation

@akashchamp

@akashchamp akashchamp commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Closes #3932

Rationale for this change

Table and catalog properties can name implementation classes that are imported and instantiated. Those values can come from table metadata or catalog configuration, so loading an arbitrary importable class can pass properties to an unrelated constructor. Each loader now verifies the expected base class before instantiation.

Covered loaders:

  • py-io-impl must resolve to a FileIO subclass (pyiceberg/io/__init__.py)
  • write.py-location-provider.impl must resolve to a LocationProvider subclass (pyiceberg/table/locations.py)
  • s3.retry-strategy-impl must resolve to an S3RetryStrategy subclass (pyiceberg/io/pyarrow.py)
  • py-catalog-impl must resolve to a Catalog subclass (pyiceberg/catalog/__init__.py)
  • auth.impl must resolve to an AuthManager subclass (pyiceberg/catalog/rest/auth.py, AuthManagerFactory.create)

Are these changes tested?

  • Focused regression tests for each loader: a wrong-but-importable class is rejected in every path, and existing tests confirm valid custom implementations still load (SqlCatalog via py-catalog-impl, BasicAuthManager via auth.impl).
  • make lint passed.
  • Unit tests for tests/io, tests/table/test_locations.py, and tests/catalog pass locally.

Are there any user-facing changes?

Yes. Invalid py-io-impl, write.py-location-provider.impl, s3.retry-strategy-impl, py-catalog-impl, and auth.impl values now raise a clear ValueError before an unrelated importable class is instantiated. Valid custom implementations continue to load normally.

@rambleraptor rambleraptor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some grepping around looking for other places in the codebase where this pattern occurs (taking some form of outside text and attempting a python import based off of it)

The other two places are in catalog/__init__.py for importing a catalog and for AuthManagers.

I don't see any harm is doing a similar check there. Importing a Catalog should conform with the Catalog class and AuthManager should conform with the AuthManager class.

I know they're not mentioned in the bug, but do you mind doing those? (I'm still approving since this looks great as-is)

Apply the same base-class check to the two remaining loaders that import
a class named by a property: py-catalog-impl must resolve to a Catalog
subclass and auth.impl must resolve to an AuthManager subclass. A
wrong-but-importable class now raises a clear ValueError instead of
having catalog or auth configuration passed to an unrelated constructor.
@akashchamp

Copy link
Copy Markdown
Contributor Author

Thanks for the review and for tracking down the other two loaders. I added the same base-class check to _import_catalog in pyiceberg/catalog/__init__.py (py-catalog-impl must be a Catalog subclass) and to AuthManagerFactory.create in pyiceberg/catalog/rest/auth.py (auth.impl must be an AuthManager subclass), each raising a clear ValueError before anything is instantiated. Regression tests for both are in tests/catalog/test_base.py, tests/catalog/test_rest.py, and tests/catalog/test_rest_auth.py, and the existing custom SqlCatalog and BasicAuthManager tests confirm valid implementations still load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implementation classes named in table properties are imported and instantiated without a type check

2 participants