Skip to content

Make AttentionModuleMixin.set_attention_slice callable - #14751

Open
ayo0la wants to merge 1 commit into
huggingface:mainfrom
ayo0la:fix/attention-mixin-set-attention-slice
Open

Make AttentionModuleMixin.set_attention_slice callable#14751
ayo0la wants to merge 1 commit into
huggingface:mainfrom
ayo0la:fix/attention-mixin-set-attention-slice

Conversation

@ayo0la

@ayo0la ayo0la commented Sep 10, 2026

Copy link
Copy Markdown

What does this PR do?

AttentionModuleMixin.set_attention_slice referred to two names the mixin never defined, so it raised AttributeError for every argument:

  • self.default_processor_cls() — the class attribute is _default_processor_cls.
  • self._get_compatible_processor("sliced") — no such method existed anywhere in src/.

Nothing routes into the method today (the models that recurse into submodule set_attention_slice all hold attention_processor.Attention, which has its own working implementation), so this is latent. It stops being latent as soon as a model built on AttentionModuleMixin is wired into enable_attention_slicing, which the TODO in modeling_utils.py points toward.

This PR keeps the shape the method already described in its own comments rather than inventing a new one:

  • Fixes the typo so set_attention_slice(None) restores _default_processor_cls, matching the legacy Attention.set_attention_slice.
  • Adds _get_compatible_processor(processor_type, **init_kwargs) on the mixin. It looks through _available_processors for a class whose name contains the requested type ("sliced" matches SlicedAttnProcessor-style names) and instantiates it with the given kwargs, or returns None.
  • When slicing is requested but the module lists no sliced processor, it falls back to the default processor as the original comment intended, but now logs a warning naming the module class, so enable_attention_slicing() no longer silently becomes a no-op for that model. No AttentionModuleMixin subclass in the tree currently ships a sliced processor, so today every mixin-based module takes the warning path; the lookup means one that adds a sliced processor gets picked up without touching the mixin again.

Raising instead of falling back was the other option. I did not take it because DiffusionPipeline.enable_attention_slicing calls set_attention_slice on every submodule that has the method, so a raise would make that pipeline-level call crash for any pipeline holding a mixin-based model. Happy to switch to a raise if you would rather have the loud failure.

Tests added in tests/models/test_attention_processor.py using a tiny in-test AttentionModuleMixin subclass (no checkpoints, CPU only): None restores the default, an available sliced processor is used with the right slice_size, the no-sliced-processor path falls back and warns, the sliceable_head_dim guard still raises, and _get_compatible_processor returns the instance or None.

Fixes #14729

Before submitting

Who can review?

@DN6 @yiyixuxu

Self-review notes

Reviewed the diff against .ai/references/review-rules.md and models.md/testing.md.

  • Blocking: none found.
  • Non-blocking, left for review: the fallback-and-warn choice when a module has no sliced processor (versus raising). Deliberate, reasoning in the description above.
  • Non-blocking, left for review: _get_compatible_processor matches on class name substring. This mirrors the existing "Added" in processor.__class__.__name__ check in fuse_qkv_projections, so it is consistent with how the mixin already classifies processors, but a registry attribute would be more explicit if you want that.
  • Dead code: none added. _get_compatible_processor has one caller (set_attention_slice) and is exercised directly by test_get_compatible_processor.
  • Docs: no public API surface changed. set_attention_slice was already documented; its docstring still describes the behavior.
  • Verified: make quality clean (ruff, doc-builder style, check_doc_toc, check_ai), tests/models/test_attention_processor.py 7 passed on CPU (torch 2.14.0+cpu, Python 3.12), and the reproduction from the issue now prints AnimaTextConditionerAttnProcessor for both None and 1, with the warning on the 1 path.

set_attention_slice on the mixin referenced two names that did not exist,
so it raised AttributeError for every argument: default_processor_cls (the
attribute is _default_processor_cls) and _get_compatible_processor (never
defined). Nothing routes into the method today, so it was latent, but any
mixin-based model wired into enable_attention_slicing would have hit it.

Fix the attribute name, add _get_compatible_processor which instantiates
the first entry in _available_processors whose class name matches the
requested type, and warn instead of silently no-oping when slicing is
requested on a module that lists no sliced processor.

Fixes huggingface#14729
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AttentionModuleMixin.set_attention_slice references two attributes that do not exist

1 participant