Make AttentionModuleMixin.set_attention_slice callable - #14751
Open
ayo0la wants to merge 1 commit into
Open
Conversation
set_attention_slice on the mixin referenced two names that did not exist, so it raised AttributeError for every argument: default_processor_cls (the attribute is _default_processor_cls) and _get_compatible_processor (never defined). Nothing routes into the method today, so it was latent, but any mixin-based model wired into enable_attention_slicing would have hit it. Fix the attribute name, add _get_compatible_processor which instantiates the first entry in _available_processors whose class name matches the requested type, and warn instead of silently no-oping when slicing is requested on a module that lists no sliced processor. Fixes huggingface#14729
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
AttentionModuleMixin.set_attention_slicereferred to two names the mixin never defined, so it raisedAttributeErrorfor every argument:self.default_processor_cls()— the class attribute is_default_processor_cls.self._get_compatible_processor("sliced")— no such method existed anywhere insrc/.Nothing routes into the method today (the models that recurse into submodule
set_attention_sliceall holdattention_processor.Attention, which has its own working implementation), so this is latent. It stops being latent as soon as a model built onAttentionModuleMixinis wired intoenable_attention_slicing, which theTODOinmodeling_utils.pypoints toward.This PR keeps the shape the method already described in its own comments rather than inventing a new one:
set_attention_slice(None)restores_default_processor_cls, matching the legacyAttention.set_attention_slice._get_compatible_processor(processor_type, **init_kwargs)on the mixin. It looks through_available_processorsfor a class whose name contains the requested type ("sliced"matchesSlicedAttnProcessor-style names) and instantiates it with the given kwargs, or returnsNone.enable_attention_slicing()no longer silently becomes a no-op for that model. NoAttentionModuleMixinsubclass in the tree currently ships a sliced processor, so today every mixin-based module takes the warning path; the lookup means one that adds a sliced processor gets picked up without touching the mixin again.Raising instead of falling back was the other option. I did not take it because
DiffusionPipeline.enable_attention_slicingcallsset_attention_sliceon every submodule that has the method, so a raise would make that pipeline-level call crash for any pipeline holding a mixin-based model. Happy to switch to a raise if you would rather have the loud failure.Tests added in
tests/models/test_attention_processor.pyusing a tiny in-testAttentionModuleMixinsubclass (no checkpoints, CPU only):Nonerestores the default, an available sliced processor is used with the rightslice_size, the no-sliced-processor path falls back and warns, thesliceable_head_dimguard still raises, and_get_compatible_processorreturns the instance orNone.Fixes #14729
Before submitting
self-reviewskill on the diff?Who can review?
@DN6 @yiyixuxu
Self-review notes
Reviewed the diff against
.ai/references/review-rules.mdandmodels.md/testing.md._get_compatible_processormatches on class name substring. This mirrors the existing"Added" in processor.__class__.__name__check infuse_qkv_projections, so it is consistent with how the mixin already classifies processors, but a registry attribute would be more explicit if you want that._get_compatible_processorhas one caller (set_attention_slice) and is exercised directly bytest_get_compatible_processor.set_attention_slicewas already documented; its docstring still describes the behavior.make qualityclean (ruff, doc-builder style, check_doc_toc, check_ai),tests/models/test_attention_processor.py7 passed on CPU (torch 2.14.0+cpu, Python 3.12), and the reproduction from the issue now printsAnimaTextConditionerAttnProcessorfor bothNoneand1, with the warning on the1path.