Skip to content

Suppress unified diff when token comparators ignore all changes - #2936

Merged
tobiasmelcher merged 1 commit into
eclipse-platform:masterfrom
tobiasmelcher:d031119/token-comparator-suppresses-unified-diff
Sep 24, 2026
Merged

tobiasmelcher merged 1 commit into
eclipse-platform:masterfrom
tobiasmelcher:d031119/token-comparator-suppresses-unified-diff

Conversation

@tobiasmelcher

@tobiasmelcher tobiasmelcher commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

The unified diff feature uses two levels of comparison: a line-level diff
to find changed regions, and a token-level diff (via ITokenComparator) to
highlight the exact words that changed within each region.

A custom ITokenComparator can normalise text before comparing — for example
treating "line ONE" and "line one" as equal. In that case the line-level
diff reports a change, but the token diff finds nothing: all results are
NOCHANGE. Before this fix, the parent UnifiedDiff was created and shown to
the user anyway, producing a highlighted region with no inline delta. The
annotation was visually present but carried no information, which was
confusing.

The fix moves the token diff computation ahead of the UnifiedDiff
construction. If hasDetailedChanges() returns false the change is skipped
entirely, so only diffs with at least one real token difference are ever
presented to the user.

Helped by Claude Code (Anthropic).

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

    54 files  ±0      54 suites  ±0   59m 27s ⏱️ + 3m 7s
 4 826 tests +3   4 804 ✅ +3   22 💤 ±0  0 ❌ ±0 
12 369 runs  +9  12 216 ✅ +9  153 💤 ±0  0 ❌ ±0 

Results for commit 1c68738. ± Comparison against base commit e5de473.

♻️ This comment has been updated with latest results.

@tobiasmelcher

Copy link
Copy Markdown
Contributor Author

Are there any objections? I plan to merge this change tomorrow.

@vogella

vogella commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

In REPLACE_MODE the document is rewritten from unifiedDiffs, so a suppressed region is never applied.

With the case-insensitive comparator from the test, replacing "line one" with "LINE ONE" leaves "line one". Could you limit the suppression to the overlay modes (or still apply the text without the annotation) and add a REPLACE_MODE test?

If line-level diffing detects a change but the token comparators report
no actual differences (e.g. a case-insensitive comparator on identical
text), no parent diff is added to the list. Previously a unified diff
was shown to the user without any detailed delta, which was confusing.
@tobiasmelcher
tobiasmelcher force-pushed the d031119/token-comparator-suppresses-unified-diff branch from 04c75e8 to 1c68738 Compare September 23, 2026 11:42
@tobiasmelcher

Copy link
Copy Markdown
Contributor Author

Thanks for the review. I see it differently: the token comparator defines what counts as a difference. If the caller plugs in a case-insensitive comparator, "line one" and "LINE ONE" are the same content, so there is nothing to apply in any mode - including REPLACE_MODE. Rewriting the text to a form the comparator already treats as identical would be wrong, so leaving it untouched is intended, not a lost edit.

I added a short comment at the suppression to make this explicit, plus two REPLACE_MODE tests: one with the case-insensitive comparator (document stays unchanged) and one with the default comparator (the case change is a real diff and gets applied). Rebased on latest master.

@vogella

vogella commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Thanks for the review. I see it differently: the token comparator defines what counts as a difference. If the caller plugs in a case-insensitive comparator, "line one" and "LINE ONE" are the same content, so there is nothing to apply in any mode - including REPLACE_MODE. Rewriting the text to a form the comparator already treats as identical would be wrong, so leaving it untouched is intended, not a lost edit.

Ok, fine for me. :-)

I added a short comment at the suppression to make this explicit, plus two REPLACE_MODE tests: one with the case-insensitive comparator (document stays unchanged) and one with the default comparator (the case change is a real diff and gets applied). Rebased on latest master.

Thanks.

@vogella

vogella commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

LGTM

@tobiasmelcher
tobiasmelcher merged commit 40e0e8c into eclipse-platform:master Sep 24, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants