Skip to content

test(ios): cover purge restore timeout, commit and reset orderings - #647

Merged
sunnylqm merged 2 commits into
masterfrom
test/tvos-purge-restore-orderings
Sep 26, 2026
Merged

sunnylqm merged 2 commits into
masterfrom
test/tvos-purge-restore-orderings

Conversation

@sunnylqm

@sunnylqm sunnylqm commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #646: turn the three suggested timing regressions into deterministic native tests. This PR changes tests and CI only; it does not change production activation policy or public APIs.

Coverage

Case Controlled ordering Required outcome
late_commit Close the restore window before releasing an already-started request's response Keep the packaged bundle selected; allow response/metadata caching for JS, but do not activate B or label it purgeRestore
commit_before_signal Commit B, hold the completion signal, report a launch-wait timeout, then deliver the signal Re-resolve and launch B, keeping running identity and first-load protection consistent
reset_wins Pause reset while holding the production state lock, start the stale commit, then release reset Reject all stale-generation writes: activation, version metadata and response cache
commit_wins Pause the commit while holding the production state lock, start reset, then release the commit Reset clears the successful commit's state, metadata and response cache; the late signal cannot undo reset
complete_in_time Commit and completion signal both arrive within the window Preserve the ordinary successful restore path

Implementation

  • Build a small macOS Foundation-only Objective-C++ test executable. No React Native installation, CocoaPods, simulator, network requests or sleeps.
  • Extract the actual, unchanged RCTPushy.mm method/helper/global definitions at build time and compile them with the real state_core.cpp. Generated implementation copies are not checked in. The extractor rejects missing or ambiguous definitions and preserves source locations.
  • Inject only collaborators such as network delivery, app paths, defaults storage, post-reset cleanup and the launch wait/completion boundary. The production lock and state-changing bodies execute unchanged. Defaults writes assert ownership of the real os_unfair_lock.
  • Use real GCD workers and semaphore handshakes; five-second waits are deadlock guards, not timing assumptions.
  • Add an ASan/UBSan-enabled macOS job to the existing test workflow.
  • Add three generated-only negative controls: remove the activation veto, skip post-timeout re-resolution, or remove the reset-generation guard. Each must compile and fail its specific regression assertion (unrelated failures do not count).

Validation

  • Extractor unit tests: 8 passed locally and in macOS CI.
  • Python syntax, shell syntax and workflow YAML structure checked locally.
  • All 5 native Foundation ordering tests passed in macOS CI with ASan/UBSan.
  • All 3 negative controls compiled and failed the exact expected assertions, confirming that the original regressions are detected.

Verified native job for head e72197353e55cedca5eb929e9adfa6ab522822c8: Native purge-restore ordering — successful run and logs.

[PASS] late_commit
[PASS] commit_before_signal
[PASS] reset_wins
[PASS] commit_wins
[PASS] complete_in_time
5 native purge-restore ordering tests passed
[PASS] negative control: late-activation is detected by late_commit
[PASS] negative control: skip-reresolve is detected by commit_before_signal
[PASS] negative control: ignore-reset is detected by reset_wins

Run on macOS:

SANITIZE=1 VERIFY_REGRESSIONS=1 bash scripts/test-ios-purge-restore.sh

The editing environment is Linux; native execution was validated through the actual macOS CI job above, not claimed as a local run. See scripts/tests/ios-purge-restore/README.md for individual case commands, test seams and limitations. These are orchestration regression tests, not a replacement for tvOS device E2E, NSURLSession behavior, cache-purge or download/diff validation.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Summary by CodeRabbit

  • Tests

    • Added automated macOS checks for iOS purge-and-restore behavior, including timing, ordering, state persistence, and reset scenarios.
    • Added optional memory and undefined-behavior checks, along with regression checks for known failure cases and unsupported test interactions.
    • Added checks for deadlocks and unexpected test outcomes.
  • Documentation

    • Added guidance for running the test suite, its coverage, and its limitations.

Compile the production startup, restore, commit and reset methods in a
Foundation-only native host. Use semaphore-controlled GCD workers to cover
late commits, commits whose completion signal is delayed, and both reset
lock orders without sleeps or network traffic.

Add sanitizer-enabled macOS CI and three generated-only negative controls
that prove the corresponding regressions fail their specific assertions.
No production runtime code or public API changes.
@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: b67aead9-b132-4188-81d4-2065cd41f9a8

📥 Commits

Reviewing files that changed from the base of the PR and between e721973 and ebfa1c2.

📒 Files selected for processing (3)
  • scripts/test-ios-purge-restore.sh
  • scripts/tests/ios-purge-restore/README.md
  • scripts/tests/ios-purge-restore/purge_restore_test.mm
🚧 Files skipped from review as they are similar to previous changes (1)
  • scripts/tests/ios-purge-restore/README.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

The pull request adds a host-based Objective-C++ suite for purge-restore ordering tests. It extracts selected production code, checks five ordering cases, and supports sanitizer and regression-mutation runs in a macOS workflow.

Changes

Purge-Restore Test Suite

Layer / File(s) Summary
Extract production code for the test host
scripts/tests/ios-purge-restore/extract.py, scripts/tests/ios-purge-restore/test_extract.py, scripts/tests/ios-purge-restore/README.md
The extractor generates source-mapped includes from selected production definitions and declarations. Unit tests cover extraction behavior, and the README describes the extraction process.
Exercise purge-restore ordering
scripts/tests/ios-purge-restore/purge_restore_test.mm, scripts/tests/ios-purge-restore/README.md
The native test host uses test collaborators and coordination controls to exercise five orderings. Assertions check commit behavior, reset state, bundle selection, and timing.
Run baseline and mutation checks
scripts/test-ios-purge-restore.sh, .github/workflows/test.yml, scripts/tests/ios-purge-restore/README.md
The runner executes extraction tests and native tests, with optional sanitizer and mutation checks. The macOS workflow enables sanitizers and regression verification.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to ebfa1

The added regression tests and CI checks are mergeable after normal checks; no concrete issue requiring a pre-merge fix was identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.79% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 19 functions across 3 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: adding iOS purge/restore tests for timeout, commit, and reset orderings. It is concise and specific.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 15.79% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 19 functions across 3 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sunnylqm
sunnylqm marked this pull request as ready for review September 26, 2026 04:34
Keep ordinary compiler warnings visible without a blanket -Werror for
extracted product code. Retain objc-method-access as a targeted test-host
contract check.

Replace the NSUserDefaults subclass with an NSObject-backed in-memory
collaborator and substitute the defaults type only in the test translation
unit. Unsupported typed selectors fail compilation; dynamic sends fail
fast instead of reaching inherited real preferences or being swallowed by
product exception handlers.

Add instance-isolation, warning-tolerance, and unsupported read/write
compile/runtime probes. Keep the five ordering cases and three regression
mutants unchanged. Document the harness contracts; no production changes.

Copy link
Copy Markdown
Contributor Author

已在 ebfa1c2 修正这两项测试基础设施问题。

1. 不再把普通产品代码警告升级为测试失败

移除全局 -Werror,保留 -Wall -Wextra。仅将 objc-method-access 设为错误,用于发现测试宿主没有实现的 Objective-C 接口,而不是给抽取出的产品代码增加一套更严的通用告警门槛。新增编译探针,验证普通 warning 确实不导致构建失败。

2. TestDefaults 不再继承 NSUserDefaults

改为 NSObject + 私有、每实例独立的内存字典,显式声明并实现当前需要的六个访问器;写入仍经过 ObserveWrite(),保留真实状态锁的 owner 断言和时序控制。

仅在测试 translation unit 中、Foundation 导入后进行 NSUserDefaults → TestDefaults 类型替换,因此产品方法正文不用修改。将来使用 boolForKey: / setBool:forKey: 等尚未实现的接口时会明确编译失败,不会继承到真实 preferences 实现。即使通过 id 动态派发绕过静态类型,doesNotRecognizeSelector: 也会直接报告测试失败,不能被产品代码的异常处理吞掉。

新增实例隔离/快照独立性检查,以及未覆盖读写访问器的编译期、运行期反向验证。

验证

已经读取 macOS CI 的成功日志,运行 SANITIZE=1 VERIFY_REGRESSIONS=1 bash scripts/test-ios-purge-restore.sh:

  • 8 个提取器测试通过。
  • 原有 5 个原生时序测试和 3 个故障注入反向验证通过。
  • defaults_isolation 通过。
  • 普通 warning 保持非致命。
  • 未实现的 boolForKey:、setBool:forKey: 分别在编译期和运行期被预期诊断拦截。

上述原生测试启用 ASan/UBSan。本次只改测试脚本、测试宿主与说明文档,没有修改生产代码;这里确认的是该原生 job,不代表其他仍在执行的工作流已全部通过。

@sunnylqm
sunnylqm merged commit 91cadfe into master Sep 26, 2026
12 checks passed
sunnylqm added a commit that referenced this pull request Sep 26, 2026
Bump npm and Harmony package manifests together for the tvOS support
release, including the purge-restore fixes and deterministic native
regression coverage from #646 and #647.

Validated with RELEASE_VERSION=v10.58.0 node scripts/check-release-version.js.
No runtime or publishing-workflow changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant