CrossFittedTransformer(Preprocessor) relearns column types and category codes per fold, unlike transform #79

Description

CrossFittedTransformer builds each fold's model by cloning the wrapped estimator and refitting it from scratch on the fold's training rows. For a wrapped Preprocessor, that refit redoes everything that does not depend on the target:

• column-type detection, whose unique-ratio cat_cutoff depends on the row count;
• the category vocabularies (sorted integer codes, one-hot columns).

The out-of-fold training features therefore use a different encoding from transform, which uses the all-data estimator_. Several things follow:

• A category missing from a fold's training rows shifts the codes of every category that sorts after it in that fold.
• An integer column can be categorical on the full data but numerical on a fold.
• One-hot widths change between folds, which crashes the fit with a message about adaptive sizing.

CrossFittedTransformer also converts a DataFrame to an array before fitting (#58). A mixed-dtype frame becomes an object array, so every column is detected as categorical and every column's codes are relearned per fold. The reproduction below uses single-dtype frames, whose column types survive that conversion.

Affected code

• pretab/core/supervised.py:181-184: fold = clone(self.transformer), fold.fit(...) and fold.transform(...) for every fold.
• pretab/core/supervised.py:185-190: a width mismatch is reported as "Disable adaptive sizing on the wrapped transformer."
• pretab/preprocessor.py:539-544 and :577-583: every Preprocessor.fit re-detects column types and relearns the categorical encoders.
• pretab/compose/feature_detection.py:72-73: the float cat_cutoff compares n_unique / n_rows, so the same column can be classified differently on fewer rows.

Reproduction

import numpy as np, pandas as pd
from pretab import CrossFittedTransformer, Preprocessor

rng = np.random.default_rng(0)
X = pd.DataFrame({"city": rng.choice(["Berlin", "Paris"], 100)})
X.loc[7, "city"] = "Amsterdam"                      # one rare category
y = rng.normal(size=100)

cf = CrossFittedTransformer(Preprocessor(random_state=0), n_folds=5, random_state=0)
out_of_fold = cf.fit_transform(X, y)
print(pd.crosstab(X["city"], out_of_fold[:, 0]))
print("transform() codes for Amsterdam, Berlin, Paris:",
      cf.transform(pd.DataFrame({"city": ["Amsterdam", "Berlin", "Paris"]}))[:, 0])
try:
    CrossFittedTransformer(Preprocessor(categorical_method="one-hot", random_state=0), n_folds=5, random_state=0).fit_transform(X, y)
except Exception as e:
    print(type(e).__name__, e)

# 25 integer levels: categorical on 1000 rows (25/1000 < cat_cutoff=0.03), numerical on a fold's 800 rows
Xk = pd.DataFrame({"k": rng.integers(0, 25, 1000)})
cf = CrossFittedTransformer(Preprocessor(numerical_method="minmax", random_state=0), n_folds=5, random_state=0)
out_of_fold = cf.fit_transform(Xk, rng.normal(size=1000))
print("out-of-fold values:", np.unique(out_of_fold)[:4].round(3), "| transform() values:", np.unique(cf.transform(Xk))[:4])

Output (PreTab 1.0.0 (6bacaba), Python 3.12, numpy 2.5.3, pandas 2.3.3, scikit-learn 1.9.1, scipy 1.18.1):

col_0      0.0  1.0  2.0  3.0
city                         
Amsterdam    1    0    0    0
Berlin       0    5   38    0
Paris        0    0   14   42
transform() codes for Amsterdam, Berlin, Paris: [1 2 3]
IncompatibleParamsError Cross-fitting requires a fixed output width across folds; expected 3, got 2. Disable adaptive sizing on the wrapped transformer.
out-of-fold values: [0.    0.042 0.083 0.125] | transform() values: [1 2 3 4]

• In the training features, code 2 means Berlin for 38 rows and Paris for 14 rows, while transform maps Berlin to 2 and Paris to 3. Amsterdam gets the "unknown" code 0 out of fold.
• The integer column is categorical for estimator_ but numerical in every fold, so the training features are min-max scaled values while transform returns category codes. With the default numerical_method ("ple") the widths differ instead, and the fit raises "expected 1, got 7".

Expected behavior

Cross-fitting exists so that "every training row is encoded by a model that never saw its own target", while transform uses one all-data model (pretab/core/supervised.py:93-96). For that to hold, the out-of-fold features must use the same encoding as transform. Only the target-aware parts should differ per fold. Category codes, column types and blocks that do not use y involve no target leakage, and they should match transform.

Actual behavior

• Integer codes, and the numerical/categorical split itself, differ between the out-of-fold training features and transform.
• Changing widths (one-hot, or a column that switches type) crash the fit with a misleading message about adaptive sizing.

Impact

A downstream model trained on the out-of-fold features learns category effects under one coding and is then applied under another, with no error. This happens with the default categorical encoding as soon as a category is missing from some fold's training rows, which is always the case for a category seen once.

Root cause

The fold models are refit from scratch instead of being derived from the all-data fit.

Suggested fix direction

For a wrapped Preprocessor, derive each fold model from estimator_ and refit only the blocks that use the target on the fold's rows. Keep the clone-and-refit path for other estimators. Also make the width-mismatch message generic.

Related issues

• CrossFittedTransformer converts DataFrames to ndarray; a wrapped Preprocessor treats every column as categorical #58: CrossFittedTransformer converts DataFrames to arrays, so a wrapped Preprocessor sees an object array; on mixed-dtype frames this per-fold refit then affects every column.
• CrossFittedTransformer(Preprocessor(add_missing_indicator=True)) crashes when a fold's training rows contain no missing value #89: the per-fold refit also relearns the missing indicators, which crashes when a fold's training rows have no missing value; deriving folds from estimator_ must keep the indicators from the all-data fit.
• CrossFittedTransformer.fit_transform forces a dense float64 array: string and sparse outputs crash #90: fit_transform forces a dense float64 output in the same fold loop.
• Preprocessor(target_aware=False) still fits registered supervision="optional" representations on y #96: registered supervision="optional" representations ignore target_aware; refitting only the target-using blocks per fold depends on identifying those blocks correctly.