Repository navigation
feat: support Duo hinge control and secondary display capture - #1287
Conversation
|
(Changed to draft as still working/testing the basic behavior) |
| W3C coordinate actions require the separate display-aware actions support tracked | ||
| in [#1269](https://github.com/appium/WebDriverAgent/pull/1269). |
|
Interesting, very curious regarding whether the I don’t have a physical iPhone Duo as of right now, but will be happy to review it and to run some simulator tests on Monday once I have access to my testing environment. |
| #import "XCUIScreen.h" | ||
| #import <objc/message.h> | ||
|
|
||
| static NSArray<XCUIScreen *> *FBScreensForDevice(id device, NSError **error) |
There was a problem hiding this comment.
It probably makes sense to split display-related stuff and the hinge one
There was a problem hiding this comment.
If you meant screensOrError, it is not hinge related for Duo. It exists since around Xcode 14 or 15 technically, so can be used for multiple screens over USB as well
ff009cc to
b03e7d8
Compare
| `GET /wda/device/hingeAngle` and `POST /wda/device/hingeAngle`. They operate on | ||
| the device and do not require creating an Appium session first. | ||
|
|
||
| ## Validation and feedback |
There was a problem hiding this comment.
Addressed appium/appium-xcuitest-driver#3001 side as well about this feedback
| } | ||
| if (!XCUIDevice.sharedDevice.fb_canAttemptSimulatedHingeAngleInjection) { | ||
| return FBResponseWithStatus([FBCommandStatus unsupportedOperationErrorWithMessage: | ||
| @"Hinge angle injection requires an available hinge and the IOKit HID APIs" traceback:nil]); |
There was a problem hiding this comment.
nit: this message is too technical. Can we instead change it to something like, requires a compatible/foldable device?
| * Easy development cycle as it can be launched & debugged directly via Xcode | ||
| * Use [Mac2Driver](https://github.com/appium/appium-mac2-driver) to automate macOS apps | ||
|
|
||
| ## Hinge angle |
There was a problem hiding this comment.
I think this part could be merged to "multiple displays" topic as they are interconnected
|
@Dan-Maor I'm going to merge this to proceed Duo testable. It is appreciated to test out something for Duo to find out potential issues 🙏 |
## [16.14.0](v16.13.6...v16.14.0) (2026-10-04) ### Features * support Duo hinge control and secondary display capture ([#1287](#1287)) ([76d6aa0](76d6aa0))
|
🎉 This PR is included in version 16.14.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
3 participants
On an open iPhone Duo simulator, selecting the inner
currentDisplayIdstill allowed the main-screen point lookup to choose the outer SpringBoard, while MJPEG and XCTest recordings remained fixed to the main display. This change makes secondary-display sessions usable for app discovery and capture, and adds hinge angle reading and simulated hinge events. Availability is checked at runtime on both simulators and physical devices; there is no fixed model or Simulator-build restriction. We have tested Duo-specific behavior only on the simulator, not on a real Duo device, and are awaiting feedback on whether these features work on real Duo devices as well.currentDisplayIdfor MJPEG frames and newly started XCTest recordings. An existing stream follows setting changes; a recording keeps its initial target. Invalid selections fail capture/recording and pause MJPEG delivery./session/:sessionId/wda/device/hingeAngle, also available without a session as/wda/device/hingeAngle. GET checks for an available hinge and the CoreMotion reading APIs. POST accepts finite angles in [0, 180], checks for an available hinge and the required IOKit functions, and sends an IOCF-serialized vendor HID payload. Those checks permit an injection attempt; they do not guarantee that the device accepts the event.Companion driver command: appium/appium-xcuitest-driver#3001 (draft until the WDA release is available).
Based on current master (
9d1d17dd). Complements #1269 without incorporating or duplicating its W3C action changes. UnsetcurrentDisplayIdstill means the main display; automatic selection after folding is not introduced.Validation
Xcode 27.1 / iOS 27.1 Duo:
defaultActiveApplication=autoon the inner display; source/window size become 951×669. Closed/main display is 466×678.mobile: setSimulatedHingeAngleexecutes 0 → 90 → 180 → 0 → 90.5 degrees. Independentdevicectl device motion hinge-anglereadings confirm every target, with matching Safari layout changes and screenshots. No UI actions are used for this sequence.iPhone 17 / iOS 26.5:
WDA JavaScript: build, lint, and all 77 tests pass. iOS and tvOS simulator runner builds pass; the final iOS build has no newly introduced source warnings.
Limitations: Duo display behavior and hinge operations have been tested on the iPhone Duo simulator. We have not had access to a physical Duo, so hinge readings, event acceptance, and display/layout behavior on that hardware remain unverified. CoreMotion loading and the presence of
CMAngleManagerwere verified on a physical iPad mini, which reported no available hinge; this does not validate physical hinge operations. Event dispatch does not wait for the angle/layout transition. W3C coordinate actions still require #1269. The existing one-pixel inner screenshot rounding was observed and is unchanged.We are awaiting real-device feedback to confirm whether hinge readings, hinge event injection, and secondary-display capture work on physical Duo devices as well. If you have access to a real Duo device, we would appreciate help testing these endpoints and reporting the observed behavior. Please include the device model, iOS/Xcode and WDA versions, requests and responses, and any angle or display/layout changes. Unsupported operations and successful responses with no observable change are useful reports too.