Core: Validate implementation classes loaded from properties - #3951
akashchamp wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
I did some grepping around looking for other places in the codebase where this pattern occurs (taking some form of outside text and attempting a python import based off of it)
The other two places are in catalog/__init__.py for importing a catalog and for AuthManagers.
I don't see any harm is doing a similar check there. Importing a Catalog should conform with the Catalog class and AuthManager should conform with the AuthManager class.
I know they're not mentioned in the bug, but do you mind doing those? (I'm still approving since this looks great as-is)
Apply the same base-class check to the two remaining loaders that import a class named by a property: py-catalog-impl must resolve to a Catalog subclass and auth.impl must resolve to an AuthManager subclass. A wrong-but-importable class now raises a clear ValueError instead of having catalog or auth configuration passed to an unrelated constructor.
|
Thanks for the review and for tracking down the other two loaders. I added the same base-class check to |
Closes #3932
Rationale for this change
Table and catalog properties can name implementation classes that are imported and instantiated. Those values can come from table metadata or catalog configuration, so loading an arbitrary importable class can pass properties to an unrelated constructor. Each loader now verifies the expected base class before instantiation.
Covered loaders:
py-io-implmust resolve to aFileIOsubclass (pyiceberg/io/__init__.py)write.py-location-provider.implmust resolve to aLocationProvidersubclass (pyiceberg/table/locations.py)s3.retry-strategy-implmust resolve to anS3RetryStrategysubclass (pyiceberg/io/pyarrow.py)py-catalog-implmust resolve to aCatalogsubclass (pyiceberg/catalog/__init__.py)auth.implmust resolve to anAuthManagersubclass (pyiceberg/catalog/rest/auth.py,AuthManagerFactory.create)Are these changes tested?
SqlCatalogviapy-catalog-impl,BasicAuthManagerviaauth.impl).make lintpassed.tests/io,tests/table/test_locations.py, andtests/catalogpass locally.Are there any user-facing changes?
Yes. Invalid
py-io-impl,write.py-location-provider.impl,s3.retry-strategy-impl,py-catalog-impl, andauth.implvalues now raise a clearValueErrorbefore an unrelated importable class is instantiated. Valid custom implementations continue to load normally.