Taking control of a Bot’s live browser works for ordinary letters, but two common input paths are broken: pressing . does not insert a period, and Ctrl+V / Cmd+V does not paste the local clipboard into the remote page.
What it looks like
- Letters and most navigation keys reach the focused remote field.
- Pressing
. leaves the field unchanged.
- Ctrl+V on Windows/Linux and Cmd+V on macOS also leave it unchanged.
- Other remote shortcuts such as Ctrl+A still need to be forwarded to the remote page.
This is especially visible when entering email addresses, domains, decimal values, filenames, and credentials during human takeover.
Why it happens
There are two independent causes in the live-screen input path.
First, agent-computer/src/screencast.ts derives windowsVirtualKeyCode from the printable character when the surface does not provide one. For . this produces decimal 46. CDP interprets virtual key 46 as Delete; the browser virtual key for Period is 190. The event therefore does not insert the character.
Second, app/src/components/computer/live-screen.tsx calls preventDefault() for every local keydown while a person holds the wheel. That includes Ctrl/Cmd+V, so the local browser never emits the paste event whose clipboard data the existing handler is designed to forward through Input.insertText. Forwarding the shortcut itself cannot work because the remote Chromium process does not share the local machine’s clipboard.
Reproduction
- Open a Bot with computer access and navigate it to a page containing a text field.
- Take control of the live screen.
- Focus the field and type
a.b.
- Observe that the result is
ab, without the period.
- Copy some text locally and press Ctrl+V or Cmd+V while the field is focused.
- Observe that no clipboard text is inserted.
Expected behaviour
Printable punctuation should use the physical browser key code and be inserted normally. Paste shortcuts should remain local long enough to produce a clipboard event; the clipboard text should then be forwarded through the existing text-input message. Other shortcuts and ordinary keys should continue to control the remote page.
Proposed fix
The attached pull request forwards the browser-provided Windows virtual key code, keeps a punctuation fallback for older surfaces, leaves Ctrl/Cmd+V in the local page, and suppresses the matching V keyup even if the modifier is released first. It includes component coverage and a real Chromium/WebSocket regression test.
Implemented in #422.
Taking control of a Bot’s live browser works for ordinary letters, but two common input paths are broken: pressing
.does not insert a period, and Ctrl+V / Cmd+V does not paste the local clipboard into the remote page.What it looks like
.leaves the field unchanged.This is especially visible when entering email addresses, domains, decimal values, filenames, and credentials during human takeover.
Why it happens
There are two independent causes in the live-screen input path.
First,
agent-computer/src/screencast.tsderiveswindowsVirtualKeyCodefrom the printable character when the surface does not provide one. For.this produces decimal 46. CDP interprets virtual key 46 as Delete; the browser virtual key forPeriodis 190. The event therefore does not insert the character.Second,
app/src/components/computer/live-screen.tsxcallspreventDefault()for every local keydown while a person holds the wheel. That includes Ctrl/Cmd+V, so the local browser never emits thepasteevent whose clipboard data the existing handler is designed to forward throughInput.insertText. Forwarding the shortcut itself cannot work because the remote Chromium process does not share the local machine’s clipboard.Reproduction
a.b.ab, without the period.Expected behaviour
Printable punctuation should use the physical browser key code and be inserted normally. Paste shortcuts should remain local long enough to produce a clipboard event; the clipboard text should then be forwarded through the existing text-input message. Other shortcuts and ordinary keys should continue to control the remote page.
Proposed fix
The attached pull request forwards the browser-provided Windows virtual key code, keeps a punctuation fallback for older surfaces, leaves Ctrl/Cmd+V in the local page, and suppresses the matching V keyup even if the modifier is released first. It includes component coverage and a real Chromium/WebSocket regression test.
Implemented in #422.