Author SHA1 Message Date
郑博文 06bb7bcd1c feat(android): 控件浮层化——statusView/sensBar/controlBar 移入 stage 底部叠加层
三个控件原是 stage 的垂直兄弟节点,显示即抢高度,于是 surfaceView 的
letterbox 几何跟着变:横屏实测控件可见时画面 1216x684、隐藏时 1920x1080,
即用户感知的「一按按钮画面就被挤小」。

改为 stage 内的 chromeOverlay(LinearLayout,Gravity.BOTTOM,
WRAP_CONTENT,底色 0x99000000 半透明):画面尺寸现在与控件显隐无关。
setControlsVisible 仍统一 GONE 三个子视图,容器随之塌成 0 高,隐藏态与
改动前逐字节等价,4a 的黑边唤出逻辑一行未动。

不碰解码、触摸映射、协议与 CursorOverlayView。竖屏实测 stage 高 2069
(等于内容区高度)、画面恒定 1080x607,连续 4 次采样无差异;横屏可见态与
隐藏态画面均为 1920x1080;R7 绝对映射三点误差 1px/0px/1px;JVM 60 例全绿。
2026-11-02 16:53:27 +08:00
郑博文 3b219a442d test(tools): cursor_proof + proof_common 抽取 + rel_mouse 适配 v5/双屏与污染修复
4b 光标覆盖层验收(V-D1/D3/D6/D7)+ 4a 建议 2 的公共库沉淀:

- proof_common.ps1:抽 landscape/rel_mouse 已验证的观测面 helper
  (唤醒只从证明过的隐藏态、正向配对判隐藏、N25 唯一临时文件、按文案取按钮、
  静置门控 + SetCursorPos 读回自校验、内存预检、R-7 新鲜守卫)。
  老 proof 本轮**不迁移**(改 4 个已验证脚本而不全重跑=假绿风险,登记下轮卫生项)
- cursor_proof.ps1:真机判据
  V-D1 截图青簇质心 vs CursorMarker 映射 ±6px(含近原点 (48,48),Some(0,0)≠None 由单测+golden 锁)
  V-D3 静止桌面跳转后 900ms 内标记跟随(覆盖 500ms 保活重发带新鲜 cursor)
  V-D7 指针进左侧副屏负 X → agent 折叠 None → 无青簇(本机双屏白送的真实判据)
  reject 阶段 V-D6:旧 v4 APK 连 v5 agent → 屏幕出「协议版本不一致」+ detail client=4,server=5
  实测全绿:4 点偏差 ≤0.6px、跟随 0.5px、副屏无簇、拒绝判据命中
- rel_mouse_phone_proof.ps1:适配 v5 + 修两处 harness 缺陷(都是本轮回归实测逼出的)
  ① HUD v4 断言→v5(PROTO_LABEL 配对);② V-B2 R7 期望值从写死 1920 派生
     改为按实时桌面 / 动态算(桌面已漂到 2560x1440,旧写死值假报 575px,
     违反 AGENTS.md 坐标期望按当前分辨率重算)
  ③ 模式守卫改为无条件 Set-Mode relative(旧 HUD.Contains(绝对) 在 HUD 读空时
     静默跳过,上一轮 throw 残留的 relative_mode=false 污染了整个 step2——
     绝对模式 DOWN 发绝对移动是正确产品行为,被 2c 误读成 teleport)
  ④ Set-Mode 唤醒 miss 并入 try 预算(原来单次重读即 throw,烧掉整脚本)+ 失败截图证据
  实测:R7 误差 1px、2c maxJump=0、长按右键 0→1、全链路绿
2026-11-02 12:39:23 +08:00
郑博文 45daba5419 chore(android): 4b 版本落账(HUD v5 标签 + APK versionCode 5)
- PROTO_LABEL 4→5:与 common::PROTO_VERSION 的人工同步义务(R5 显示要求),
  rel_mouse_phone_proof 的 HUD v5 断言与之配对(R-5 同源注释写明)
- versionCode 4→5、versionName 0.1-phase4b:真机验收跑的就是这个包,
  dumpsys 版本号是 proof 判「装没装对」的唯一依据(R-7 的 APK 侧对应物)
2026-11-02 12:38:40 +08:00
郑博文 efcaeaad26 test(android): 光标映射 8 例 + 帧头拆包 golden bytes 6 例
4b/V-D1 的单元侧判据(跨端契约 + 映射语义):
- CursorMarkerTest:中心/满量程/(0,0) 合法/1:1 恒等映射;null、尺寸未就绪、
  越界全分支返回 null。origin_maps_not_treated_as_none 锁最危险的错——
  把左上角误当无光标(协议用 Option 的全部理由)
- RustBridgeTest:goldenHeader 逐字节手抄(与 android-ffi 写端测试同一组常数对拍),
  锁 [25]/[26..28]/[28..30] 三个新偏移 + 有符号 i16 + 截断缓冲防御 + 空载荷边界
- R-8 反向验证:读偏移 26→27 后 4 例坐标测试确实红(已改回);
  CursorMarker 边界 >= 改 <= 后 origin 用例确实红(已改回)
JVM 全量 60 passed / 0 failed(--rerun-tasks 新鲜跑,XML 时间戳核对)。
2026-11-02 11:05:24 +08:00
郑博文 9a6f6962d2 feat(android): v5 光标标记层(拆包迁移 FrameCodec + CursorMarker + Overlay + 接线)
4b 手机端「看得见远端鼠标」的实现(协议 v5 的消费端):

- RustBridge:Frame 增 cursor: Pair<Int,Int>?(可空 Pair 忠实映射 Option<(i16,i16)>,
  杜绝「x 有 y 无」非法态);拆包逻辑连同字节 reader 迁到顶层 FrameCodec——
  RustBridge object 带 init{loadLibrary},测试里不可达,抽成无 native 的顶层 object
  才能把跨端偏移放进 JVM 单测(核过 Rust 一端≠核过 Kotlin 另一端)
- CursorMarker.mapToView:桌面坐标→视图坐标等比映射(letterbox 下线性等比,
  与 RelPointer 同一套比例);null/尺寸未就绪/越界一律返回 null(不画),
  绝不用 (0,0) 当缺省——那会把「无光标」和「光标在左上角」抹平成一样
- CursorOverlayView:SurfaceView 的上层兄弟视图(surface 合成在窗口之下,
  同视图无法自叠加);双描边圆环+青色中心点(兼作 proof V-D1 的颜色特征码,
  UI 层绘制不经 H.264,screencap 里颜色精确);volatile + postInvalidate 跨线程;
  不消费触摸(覆盖整个 stage,若吃事件会同时废掉远端操作和黑边唤出)
- MainActivity:解码循环每帧喂 cursorOverlay.updateCursor(f.cursor)(放解码前,
  光标是帧头字段与解码成败无关);cleanupSession 与 disconnect 两条收尾路径同改
  清标记(R-5 同源,旧会话光标留着会冒充新会话)

assembleDebug BUILD SUCCESSFUL(Canvas API 编译过);JVM 60 passed。
2026-11-02 11:05:09 +08:00
郑博文 4379b01d41 refactor(android-ffi): 抽 pack_frame_header 并加写端字面量对拍单测
步骤 4 的跨端契约闭环:把 30 字节头构造从 nativeNextFrame 内联提取成纯函数,
写端只有一条路径(消除内联/函数两处并存 = R-5 同源漂移)。

新增 host 可跑的 3 条单测(#[cfg(test)],进 --workspace --exclude linux-agent 口径):
- header_bytes_match_the_declared_layout:逐字节手抄常数,与 Kotlin 侧
  RustBridgeTest.goldenHeader 是同一组——两侧各握一份手抄,改任一偏移另一侧必红
  (只锁读端有盲区:Kotlin 字面量若本身抄错 Rust 布局会跟着一起错还全绿)
- none_cursor_writes_zero_flag_and_zero_coords:None 分支 has=0 且坐标字节不残留脏值
- negative_coords_keep_bit_pattern:i16 负坐标位模式忠实
android-ffi 在 host(gnu) cargo test 可编译可跑,实测 3 passed。
2026-11-02 11:04:51 +08:00
郑博文 8eed1e20e1 feat(viewer-core,android-ffi): v5 光标字段透传到 FFI(帧头 25→30 字节)
主控链路把协议帧头的光标坐标交给 Kotlin:
- viewer-core::Frame 增 cursor: Option<(i16,i16)>,From 实现透传;
  契约测试改为 Some((640,480)) 并断言透传——丢它=标记永不显示且丢法是静默的
- android-ffi 定长头 25→30:尾部追加 [has_cursor 1][x i16 LE 2][y i16 LE 2],
  放末尾保证 Kotlin 既有偏移量(0/1/9/17/21)全部不移动
- None → has=0 且坐标字节恒 0(无意义);Some 分支显式写两坐标,
  杜绝「Some 但忘写坐标」的编辑事故(文件头注释同步 30 字节布局)
- R-5 同源检查:RustBridge.kt 的 FRAME_HEADER_LEN=25 与拆头偏移是本次协议变更的
  同步改动点,Kotlin 侧在下一提交完成(同一循环、相邻提交)
2026-11-02 10:39:19 +08:00
郑博文 ad46fc805b feat(agent-core): ScreenCapturer 新增 pointer_state 默认实现并接入帧头
v5 光标字段在平台无关会话层的接缝(4b/Q4=A:Linux 实装另立循环):

- trait 新增 pointer_state(),默认实现返回 None + 会话期一次性日志——
  与 InputInjector::text() 未实现先例同款纪律(异常路径必须可见,且不逐帧刷屏)
- encode_send 增 cursor 参数,两条发送路径(新帧/静止强制 IDR)都传入
- 静止保活重发分支:split 出的旧帧头沿用会让标记冻住(画面静止≠指针静止),
  与 seq/ts_ms 同规重建 cursor
- 新增单测锁默认实现契约(None 且 panic-free,Linux/X11 端真实形态)
- linux-agent 不强制实现(cargo build --workspace 复验编译通过)
2026-11-02 10:36:47 +08:00
郑博文 b1ac32175c feat(agent): v5 光标采样接入帧头(GetCursorInfo + 折叠规则 + dump 观测列)
采集端把远端光标位置填进每帧帧头,供主控端画标记(4b 解决看不见远端鼠标)。

- fold_cursor 纯函数承载全部折叠判定:隐藏(CURSOR_SHOWING 未置位)/
  越出采集画面 [0,w)x[0,h)(副屏负坐标区,Q5=A)/超 i16 范围(防截断成
  看似合法的错坐标)→ None;坐标语义单一来源在被控端
- sample_cursor 只负责 Win32 读数(cbSize 契约注释)与失败/越界留痕
  (会话期各只打一条日志,可见后果但不逐帧刷屏)
- 采样点在 emit! 宏(唯一发送入口):新帧/静止 500ms 保活重发/强制 IDR
  三条路径天然全覆盖,帧头 cursor 与 ts_ms 同刻
- 前提:agent 的 per-monitor DPI 感知(3b/D2)保证 ptScreenPos 是物理像素
- agent crate 首批单测 4 例(界内含原点/隐藏/五类越界/i16 上界),
  mutation 反向验证(>= 改 > 后越界用例确实红)
- dump_stream 帧表加光标列,Windows 端实测(本机同回环):
  界内 Some(1000,600) 精确、副屏 -960 折叠 None、None->Some(300,250) 恢复
  全部 PASS;期间发现有人操控真鼠标,探针已改静置门控+读回自校验设计
2026-11-02 10:32:09 +08:00
郑博文 98b4f03cff feat(common): 协议 v5 帧头新增光标字段 cursor
4b 光标覆盖层的协议地基:VideoFrameHeader 尾部追加 Option<(i16,i16)>,
携带本帧发送时刻的远端光标物理像素坐标(None=隐藏/离屏/未实现三态折叠)。
解决「看不见远端鼠标」——相对模式下用户不知道指针在哪、轻点会落在哪。

- 字段追加改变 bincode 线格式,PROTO_VERSION 4→5,v4 旧端由握手显式拒绝(既有机制)
- v4 守卫按 v3 先例降为下界锚点,最新一代用精确等号守卫
- 新增 3 测试:Option 往返(含 i16 边界与负值)、尾部追加不扰动既有字段、
  None 紧凑性(Some 比 None 恰多 4 字节,证明光标未被固定宽度白占)
- agent/agent-core/viewer-core 构造点补占位 cursor: None(真实采样下一步接入),
  保证本提交全 workspace 可编译
2026-11-02 10:11:23 +08:00
郑博文 e96502f888 test(tools): 4a 横屏验收脚本 + 既有 proof 适配自动隐藏与同源缺陷修复
新增 landscape_proof.ps1:V-C1~C7 判据(自动横屏 <=2s、旋转零中断、
动画桌面下 >=45fps、控制条自动隐藏、黑边唤出零远端泄漏、四按钮可点、
方向四态循环、表单竖屏)。方向判定只解析 Display: mDisplayId=0 块内
cur=WxH——这台 vivo 无 mRotation= 行,且 dumpsys 里有常驻 1080x2400 的
虚拟屏,匹配首个 cur= 会在已横屏时误报未旋转(探针实测踩过)。

横屏自动隐藏与既有 proof 的判据自冲突(HUD 随控制条 GONE,RTT= 读不到
被误判断线/未连接),三个 proof 统一加「确认隐藏态才走黑边唤醒」:
phone_chain_proof、ime_chinese_proof、rel_mouse_phone_proof。

修 rel_mouse_phone_proof 两处:
- 2c 基线判据:横屏滑动 768px 的残余相对事件成批延迟落地,旧的「两次
  读数相等」被突发间隙骗过(实测假失败 351/1418px)。改为 6 读数沉降 +
  4 次采样判漂移 + 绝对泄漏点特征比对。
- step 4 靶窗启动缺 3 个 Mandatory 参数,子进程瞬间失败退出(3b 起一直
  如此,右键落在裸屏幕中心,靠近违反 R-11「不得注入他人窗口」)。现按
  契约启动并把指针停在自家窗口内。

R-5 同源排查:靶窗结果目录固定路径 + 启动时删自己的 stop.flag,会让
上一轮失败遗留的靶窗永远收不到退休信号、继续抢前台并写脏 received.txt
(实测收到 abc\nabc)。5 个 proof 统一改每轮唯一目录(\47512 后缀)。

ime_chinese_proof 的切换键标定失效:设备 IME 已换成搜狗(原为讯飞),
旧坐标 (146,2096) 静默点空。IME 按键视图对 uiautomator 不可见、无法机械
标定,按 R-9 转人工确认,不硬凑自动化。
2026-11-01 21:58:19 +08:00
郑博文 cff7fc8ab3 feat(android): 横屏远控(自动方向 + 控制条自动隐藏 + 黑边唤出)
竖屏看 16:9 桌面太小(用户 2026-11-01 反馈)。探针实测(docs/output/
2026-11-01-landscape-probe/):App 级 setRequestedOrientation 在这台 vivo
上有效(761ms),且旋转只触发 surfaceChanged、不销毁 surface、不断会话、
横屏 fps=58~60——所以不需要 TextureView(探针另测 TextureView 掉 30~37%
帧率)、不需要任何解码器重建配合,旧 S3 作废。

- OrientationPolicy(AUTO/LANDSCAPE/PORTRAIT/SYSTEM)纯逻辑 + 10 单测,
  常量值经 javap 对 android-34 核实;已做 mutation 反向验证(AUTO 判定
  > 改 >= 时方屏用例确实红)。
- 表单页强制竖屏;连接成功按策略设方向(默认 AUTO:远端宽>高即横屏)。
- 横屏 immersive + 状态行压 36dp + 控制条 3s 无操作自动隐藏。
- 唤出区 = letterbox 黑边(stage 在隐藏态消费画面外触摸)。刻意不用
  底部边缘带:R7 回归实测那种带会吃掉远端桌面底部 ~11%(Windows 任务栏
  就在那里),点任务栏会变成唤出菜单。黑边不属于 SurfaceView,零冲突。
- 竖屏永不隐藏(裁决只针对横屏空间不足)。
2026-11-01 21:57:49 +08:00
郑博文 1a6c3f15a2 test(tools): 3b 真机验收脚本 + 修正 IME proof 的按钮定位
rel_mouse_phone_proof.ps1 覆盖 V-B1a(相对位移方向/符号/不跳变)、V-B2(切绝对
模式后 R7 重测)、V-B3(长按出右键、拖拽不出)。

三处 harness 教训(都已写进脚本注释,避免下次重踩):
1. uiautomator 不暴露 SurfaceView,裸 SurfaceView 塌成 android.view.View;
   按 16:9 宽高比识别视频视图,并断言比例,避免静默抓错节点。
2. 上一段滑动的相对事件可能在 SetCursorPos 之后才落地,制造出 15px/298px 的
   假跳变。判据前必须先等链路静默、并要求基线两次读数一致。
3. ime_chinese_proof 用「第一个 Button」当键盘按钮——3b 在按钮栏上方插入了
   灵敏度行,第一个 Button 变成了减号,proof 报「IME not shown」。
   改为按文案匹配,与位置无关。
2026-10-31 17:35:18 +08:00
郑博文 aede7b6c3c test(android): 相对鼠标/长按右键纯逻辑单测 17 例
覆盖 V-B1 判据对应的行为,不是「跑通就算」:
- 慢速小幅不丢位移(10 次 0.3px 必须发出 3px)
- 负值向远离零取整(-0.7 取 -1,向零截断会让反向慢滑丢位移)
- 撞边不积债(回滑立即发出,不因补偿而空转)
- i16 越界 clamp 而非回绕
- 长按三条判据:够久、没动、非双指

已做 mutation 反向验证:把期望 3 改成 4 后测试确实红,改回后绿(断言非空转)。
2026-10-31 17:35:11 +08:00
郑博文 7c3b6b3e93 feat(android): 相对鼠标模式 + 灵敏度 + 长按右键 + 修复表单 IME 残留
相对模式(默认):手指滑动按增量移动远端指针,DOWN 不发绝对移动——轻点落在
远端指针当前位置,不再「手指落下指针跳到脚下」。换算基准为视图全宽=桌面全宽
(灵敏度 1.0),慢速滑动用小数累积取整,逐次 toInt() 会静默吞掉位移。

长按右键(Q5=A):>=500ms 且位移 <12dp;触发后抑制本次 ACTION_UP 的左键,
否则一次长按会「先右键再左键」把菜单立刻点掉。超过 slop 立即转拖拽并取消计时,
所以拖窗口不会弹菜单。双指右键路径保持不变。

真机暴露并修复(N19):切到远控面板后软键盘仍绑在已隐藏的表单 EditText 上
(dumpsys 里 mServedView 指向 EditText、mInputShown=true),键盘盖住面板底部
按钮栏,**所有按钮点不动**——3b 新增的模式/灵敏度按钮把它暴露出来。改为对每个
表单输入框显式 hideSoftInputFromWindow。

新增右键计数器:长按后紧跟的 ACTION_UP 会覆盖一行式诊断文本,验收脚本读不到
「长按右键」;计数器持久且能区分长按/双指来源。
2026-10-31 17:35:11 +08:00
郑博文 3773a0851c feat(viewer-core,android-ffi): 相对位移上行链路 + 桌面像素级 proof
viewer-core 新增 send_mouse_rel(零位移不占消息:手指静止时 ACTION_MOVE
每帧都发,空事件只会白占 QUIC 帧)。
android-ffi 新增 nativeSendMouseRel,JNI 符号 13 -> 14,Int 入参 clamp 到 i16
而不是回绕(回绕会让指针跳到屏幕另一侧)。

新增 examples/rel_mouse_proof:走完整链路(viewer-core -> QUIC -> agent 注入
-> GetCursorPos 读回)验证 V-B1a/b/c。判据是「位移恰好等于请求值」,不是
「大概动了」——因为被推翻的 MOUSEEVENTF_MOVE 方案正是「看起来能动但比例错」。
运行前后记录并还原光标位置,不污染桌面状态。
2026-10-31 17:35:02 +08:00
郑博文 2022bbcca0 feat(agent-core): InputInjector 新增 mouse_move_rel 默认实现与分发
默认实现不注入但打印日志(与 v3 的 text() 同一条纪律):Linux/X11 后端本轮
未实现相对移动,用户用相对模式操作 Linux 被控端时「指针纹丝不动」不能是静默的。

相对事件不得被降级成绝对移动——单测显式断言默认实现不产生任何注入调用
(否则 Linux 端会把「相对增量」当成「绝对坐标」解释,指针乱跳)。
2026-10-31 17:35:02 +08:00
郑博文 78ac9d9972 feat(agent): 相对鼠标注入(读光标+增量+clamp 绝对注入)+ 显式 DPI 感知
注入实现刻意**不用** SendInput(MOUSEEVENTF_MOVE):实测该路径的 dx 不是
像素而是 mickey,会被「指针加速」与「DPI 缩放」双重放大——本机 dx=1 走 0px、
dx=20 走 41px、dx=100 走 309px,且倍率随用户系统设置与机器变化,无法标定。
改为「GetCursorPos + 增量 + clamp + 绝对注入」,实测 1:1 精确(±1/2/3/5/17/100
四次重复全部等于请求值),撞边后回滑立即响应、不积步数债。

同时显式设 PER_MONITOR_AWARE_V2:实测本机 agent.exe 虽是 per-monitor,
但那是注册表 AppCompatFlags\\Layers 里一条**路径绑定**的 HIGHDPIAWARE 覆盖
造成的(同一 exe 复制到 %TEMP% 启动即变 UNAWARE),换机器/换构建路径即失效。
UNAWARE 时 GetCursorPos 返回逻辑坐标(1536x864)而采集是物理像素(1920x1080),
clamp 上界与增量尺度都会错。设置失败时区分「已被外部设定」与「真正降级」并分别报日志。
2026-10-31 17:34:54 +08:00
郑博文 39af42e1b3 feat(common): 协议 v4 新增 MouseMoveRel 相对鼠标事件
v4 新增 InputEvent::MouseMoveRel{dx,dy},单位为被控端桌面物理像素,
语义是「把远端指针从当前位置移动 (dx,dy)」,越界由被控端 clamp。

为什么必须升版本:bincode 按位置编码枚举判别符,新增变体会改变其后
变体的编号,旧端会静默错解成别的输入事件。v4 后旧 v3 端由 Auth 握手
显式拒绝(已用 v3 二进制实测:client=3, server=4)。

为什么单位是像素而不是归一化增量:AuthOk 已下发桌面宽高,主控端换算
一次即可;被控端拿到像素可直接与 GetCursorPos 同尺度相加,少一次取整。

单测锁住:版本号、负值往返(i16 全范围)、与绝对 MouseMove 不混淆。
2026-10-31 17:34:47 +08:00
郑博文 114e8431a6 test(android-3a): 连接生命周期 proof 入仓 + 四个 proof 的主机默认值跟上环境
新增 tools\connect_lifecycle_proof.ps1(V-A1a/A1b/A2/A3/A4/A5 + 一键重连,
含 keep-awake、tap 前唤醒、期望文案码点从 Kotlin 源提取)。
四个既有 proof 的 -PhoneIp 默认值 .180 -> .110(2026-10-31 办公网实测;
stale 默认值曾静默吃掉一次 proof 运行,注释写明 DHCP 需按需传参)。
capture_evidence 的 192.168.1.199 保持不变——那是 V5 故意不可达的路径。
2026-10-31 15:57:57 +08:00
郑博文 3891c970eb fix(android): 等待中的取消入口落在面板按钮上(连接后表单被隐藏)
真机 V-A1b 实测发现:startConnect() 切到远控面板后,表单上的按钮不可见,
「点取消」没有可点的目标。把底部按钮在 connecting 态显示为「取消」并共享
cancelConnect() 一条路径(两个入口一个语义),成功后复位为「断开」。
APK 重装后 proof step 3/4 通过(连接已取消文案 + 3.9s 回表单)。
2026-10-31 15:54:48 +08:00
郑博文 bb3cf6b09b test(android): JUnit 测试基础设施 + 19 个纯逻辑单测(3a/U7,Q3=A 批准)
仅 testImplementation junit:4.13.2(测试作用域,不进 APK,runtime deps 仍空);
versionName→0.1-phase3a。覆盖:WinScancode 映射表(右 Shift 非扩展/UNKNOWN 落 null/
F 键值钉死待 U6)、ConnectErrorText(每类中文非空、编号契约、未知不猜)、
planBackspaces((0,0)/(after>0)/截断 64)。含 mutation 反向验证:断言写反即红。
2026-10-31 13:30:07 +08:00
郑博文 718b66617f feat(android): 连接状态机 + 取消/断线回表单 + 错误中文映射 + HUD fps(3a)
- ConnectFailure(kind,detail) + ConnectErrorText:编号→中文可执行提示,原始英文第二行
- sessionGeneration 代际:取消/断开/退出作废在途回调;等待中按钮变「取消」即时回表单
- 断线可见:取帧 null 回表单显示原因 + 「重新连接」(修「冻结在已连接」);destroy 只在 UI 线程
- BACK 语义补连接中:取消而非关 Activity(修悬挂会话)
- 键盘按钮可见时收起(补唯一收起入口)
- HUD 1s 窗口 fps;触摸/按键诊断去坐标去内容(V-A5,proof 不依赖这些字符串)
- RemoteInputConnection 退格决策抽纯函数 planBackspaces(为 JVM 单测,语义不变)
2026-10-31 13:30:07 +08:00
郑博文 f28db050fb feat(android-ffi): nativeLastErrorKind 出口带错误分类(3a)
LAST_ERROR 槽位改存 (kind, detail),新增 nativeLastErrorKind();FFI 契约
kind 先取、message 后取(消息读走清空槽位)。kind 编号与 viewer_core::
ConnectErrorKind::code 一致,跨 FFI 只传整数。JNI 符号 12→13 全部导出(字节扫描核)。
2026-10-31 13:29:48 +08:00
郑博文 4eb82f9779 feat(viewer-core): 连接失败分类 + 短预算×有限重拨(3a/U8)
ConnectErrorKind(Resolve/Timeout/Refused/Auth/Version/Other)在 Rust 侧分类,
Kotlin 只做编号→文案映射。分级预算:第一拨握手 5s(快速判不可达)、重拨拨 8s
(接住 agent 串行 accept 的僵尸释放 ~10.2s)、认证 11s、全程 14s 封顶。
is_retryable 只对 Timeout/Refused 重拨;Resolve/Auth/Version 确定性失败不拖慢。

判据演进(偏差 D2/D3,用户 2026-10-31 裁决方案①):单次 5s 总预算在串行 accept
架构下把合法排队压断(text proof 实爆),故改重拨。不可达 32s→~13s 且全程可取消;
僵尸重连第一拨 5s 失败第二拨 10.2s 自动成功(桌面 e2e 实测)。9 单测锁分类/预算/重拨。
2026-10-31 13:29:48 +08:00
郑博文 99ffffa393 test(android-2b): 真机 emoji 人工验收工具 + 链路 proof 参数化主机地址
V7 设备侧此前转人工,原因是 IME 的 emoji 面板对 uiautomator 不可见、adb 无法
驱动。工具只负责准备接收窗口并读回**实际到达的码元**,点 emoji 由人完成——
判据仍是真实字节,不是截图。

真机实测通过:靶窗收到 U+D83D U+DE02 -> U+1F602,两个 code unit 相邻成对、
无 U+FFFD。这证明手机 IME 把增补平面字符作为一次 commitText 送出,协议 v3
保住了代理对相邻性。至此 V7 双侧都有证据。

同时修掉 phone_chain_proof 里硬编码的 192.168.1.180:本机 LAN 地址变成
192.168.1.110 后,表单连接打向旧地址超时,看起来像产品缺陷,实为脚本过期。
改为从 -PhoneIp 参数逐字符生成 keyevent,并在断言里比对实际输入值。
2026-10-31 11:08:16 +08:00
郑博文 8993a38658 fix(android): BACK 键改用自测键盘可见性——isAcceptingText 会让会话永远断不掉
真机实测(vivo S6 + 讯飞,2026-10-30):软键盘已隐藏、mInputShown=false,
但 imm.isAcceptingText 仍返回 true——只要 InputConnection 还绑着它就是 true。
原先用它判断「先关键盘还是先断会话」,后果是第一次 BACK 空转,第二次也进不来
(我们已 return true),会话再也断不掉。这不是推测,是完整链路 proof 跑到
step 5 时实测失败的。

改为自测:比较根窗口可见区域与窗口高度,被 IME 吃掉超过 15% 视为可见;
用 OnGlobalLayoutListener 被动跟踪(没有公开 API 能直接查询,API 34 上
getInputMethodWindowVisibleHeight 也不是公开接口)。阈值用比例不用固定像素。

验证:diag 依次显示「键盘可见」→「键盘隐藏」→ 会话断开回表单;
随后完整链路 proof 与中文 proof 均复跑通过。

同时修正 phone_chain_proof 的 V4 断言:uiautomator 属性顺序是 text= 在
class= 之前,原正则要求反序,导致打印空表单行仍判 PASS(假绿)。
空 EditText 的 text 会回显 hint,所以判据改为比对实际输入值。
2026-10-30 13:31:28 +08:00
郑博文 9792662181 test(android-2b): 真机中文候选词 proof——拼音组词经 commitText 落到 Windows
V6 的判据是「手机中文输入法能把汉字送进远端」,此前只能靠人手点候选词。
实测发现两条关键事实,决定了 proof 的可行形态:
1. adb keyevent 相当于物理键盘输入,中文 IME **不组词**,字母原样透传到
   Activity.onKeyDown——所以驱动候选词必须用 adb input text(走 IME)。
2. IME 自己的按键视图对 uiautomator 不可见(dump 里只有我们 App 的 15 个节点),
   中/英切换按钮读不到坐标,只能按截图标定一次点击位;且该按钮**有状态**,
   于是写成幂等重试:先试,只收到字母才翻转模式再来一次。

跑通结果:讯飞拼音 nihao + 空格 -> commitText 2u -> 靶窗收到 U+4F60 U+597D。
这补上了方案风险 R1(vivo ROM + 第三方中文 IME 行为未观测)的核心一环。
2026-10-30 13:15:27 +08:00
郑博文 0089ae86e1 test(android-2b): 真机 IME 提交 proof 与字节安全证据采集
ime_commit_proof 补上 phone_chain_proof 没覆盖的那半段:IME **显示**状态下
按键被输入法消费,只有 RemoteInputConnection.commitText 能到达我们——这正是
方案风险 R1(vivo ROM + 第三方中文 IME 行为未观测)此前唯一没有证据的点。
真机实测:mInputShown=true、mHaveConnection=true、状态栏 commit 2u、
Windows 靶窗收到 Hi(讯飞把首字母自动大写,属输入法行为,不是链路缺陷)。
back 键第一次只关键盘不断会话,也在这支脚本里断言。

shot.ps1 / capture_evidence.ps1:PS 5.1 的 '>' 以 UTF-16 写文件,
'adb exec-out screencap -p > x.png' 产出的 9 张证据图首字节全是 FF FE,
全部不可用;改为设备内 screencap 落地再 pull,并校验 PNG 签名。

V5 等待预算 30s→50s:不可达 IP 实测 32s 才回表单报 timed out,
预算过小会把「仍在超时中」误判成验收失败。
2026-10-30 13:06:08 +08:00
郑博文 9f0b0742a8 feat(android): 键盘事件双通道 + 手机到 Windows 全链路 proof
真机暴露出 IME/按键通道的缺口,一并补上并证明:
1. RemoteInputConnection.sendKeyEvent 与 Activity.onKeyDown 都先处理**可打印**
   unicodeChar(走 v3 Text 通道),再退到扫描码查表。原先只查表:软键盘发的是
   keyCode=UNKNOWN + unicodeChar,查不到表就被当未映射键吞掉,字整个丢了。
   门槛排除控制值(Enter/Tab)——远端应用要的是按键事件而非 WM_CHAR。
2. onKeyDown/onKeyUp 补齐硬键与物理键盘路径(阶段 2 验收里的软键盘按键),
   repeatCount>0 只发一次 down,避免长按把远端消息队列刷爆。
   isAutorepeat() 是 @hide,改用公开的 getRepeatCount()(javap 核实)。
3. getUnicodeChar() 返回的是单个 UTF-16 code unit 而非码点,注释按此改正:
   BMP 外字符由 IME commitText 成对送达,不经这条路。

tools/phone_chain_proof.ps1:手机按键 -> v3 Text -> agent SendInput -> 自有
Windows 靶窗自报内容,全链路一次跑通(真机 vivo S6 实测 abc 到位、ENTER 走
scan 28、BACK 断开回表单、表单 UI 重连成功)。要点:
- 靶窗必须自报窗口句柄(pid+hwnd 两行),Get-Process 取 MainWindowHandle 有竞态;
- 不能用 keybd_event(VK_MENU) 破前台锁:ALT 之后的首个字符被当菜单助记键吞掉,
  实测三个字符只剩 b;改用 AttachThreadInput + SetForegroundWindow,不合成按键;
- 退役遗留靶窗只能 touch 已存在的 stop.flag:新建会命中即将启动那次的停止路径,
  靶窗刚起就自杀(实测 pid 报了又消失);
- adb 进度信息走 stderr,PS 5.1 + ErrorActionPreference=Stop 会当终止错误,
  按 build-android.ps1 同款办法临时放宽并只看退出码。
- 表单填字用 keyevent 而非 adb input text:设备只有第三方中文 IME,text 注入会被
  组词(ASCII 句点变 U+3002)。跑前记录并禁用原 IME,finally 精确还原。
2026-10-30 12:58:36 +08:00
郑博文 a93430dd3f chore(tools): 2b 注入 proof 路径随仓库位置推导,缺失参数报错文案改正
proof 脚本原先硬编码 D 盘绝对路径,换机器就失效;仓库内已有同约定
(build-android.ps1 用 \ 推导),此处对齐。
text_check 的报错文案说「为空」,实际触发条件是 --units 缺失,改正。
2026-10-30 12:21:19 +08:00
郑博文 79e3e77c97 fix(android): windowSoftInputMode=adjustPan——软键盘弹出不再踢掉会话
默认 adjustResize 会缩小窗口高度,SurfaceView 随之收到 surfaceDestroyed,
而现有回调把 surfaceDestroyed 当作「断开」处理——等于用户一点键盘输入,
2a 验收过的会话就自己断了。真机首验前先做这道防御;adjustPan 只平移窗口,
不改变 surface 尺寸。合并后的 manifest 已核对(android:windowSoftInputMode 存在)。
2026-10-30 12:17:20 +08:00
郑博文 8ffe19cd5f test(agent-core): v3 分发路径单测——默认后端不 panic、截断契约、空输入
新功能必须带测试,agent-core 此前无测试模块,v3 新增的 text 默认实现与
截断分支正好落在盲区里:

- 后端不实现 text() 时(Linux/X11 的真实形态)必须被正常调用且不 panic;
- 超 MAX_TEXT_UNITS 必须以截断后的长度到注入侧,未超限原样;
- 空 Text 仍会下发到后端——被控端不假设对端守规矩,线格式输入按不可信处理。
2026-10-30 12:11:28 +08:00
郑博文 936986a940 test(android-2b): 文本注入真边界工具——viewer-core 直发 Text,靶窗自报收到内容
text_check 走的是产品同一条路径(Session::send_text → v3 线格式 → agent
SendInput KEYEVENTF_UNICODE),手机掉线也能先把 Windows 半段验实。
参数是十进制 code unit 列表而不是自然语言字符串:命令行到 Rust 的编码往返
会引入与本轮无关的变量,由 PowerShell 从 .NET 字符串取 char[] 更忠实。

proof 脚本刻意用自有 WinForms 靶窗而非记事本:Win11 记事本所有标签页同进程,
本机当时开着用户文档,往里注入等于改用户文件。靶窗自报文本(写文件),
判据不依赖截图和 OCR。注入前后各断言一次前台窗口——SendInput 打的是前台,
靶窗不在前台时的注入会进别的应用,宁可失败也不误注入。
2026-10-30 12:09:48 +08:00
郑博文 6c28a3f7e3 feat(android): 阶段 2b——连接界面 + IME 文本通道 + 控制键扫描码
两态 UI:连接态是 host/port/code 表单(原生 View 零依赖,intent extras 保留且
优先,构建脚本不受影响),成功后切远控态。只存 host/port;验证码不落盘(用户裁决)。
返回键分态:IME 显示时先收键盘,否则断开回表单——否则一次返回关键盘又杀会话。

IME 走 P1:onCheckIsTextEditor + BaseInputConnection 子类。commitText 整段
经 v3 Text 通道上行(char[]→GetCharArrayRegion,UTF-16 无损);控制键映射
Set-1 扫描码(值与 viewer/keymap.rs 同源,右 Shift 按官方文档不置 extended)。
composing 不上远端(U4),getText* 回空串——本地没有也不伪造远端缓冲区。

键盘按钮显式 showSoftInput,不自动弹(vivo 行为不一致 + 挡画面)。
RemoteSurfaceView 补 isFocusableInTouchMode:没有它触摸态拿不到焦点,
InputConnection 根本不会建立(2a 弹不出键盘的根因之一)。
2026-10-30 11:51:01 +08:00
郑博文 55cc8edbc1 fix(agent): 文本注入改为单次 SendInput 批量——按官方文档消除事件被打散的风险
实读 Keyboard Input Overview 后修正两处凭记忆的实现:
1. 原文承诺「单次调用的事件不会被其他输入插入」,此前拆成 down 批 + up 批
   两次调用,两次之间可能夹进真实键鼠事件,代理对存在被拆开的风险。合并为
   一次提交(全部 down 紧接全部 up)。
2. KEYBDINPUT 文档「只能与 KEYEVENTF_KEYUP 组合」那句按推断写成结论;
   改为如实标注为推断,并写明真机失败时的退路(只发 UNICODE|KEYUP 单事件)。
2026-10-30 11:41:54 +08:00
郑博文 174a3f8ac0 feat(viewer-core,android-ffi): 主控端文本上行与 JNI 入口
Session::send_text 收 &[u16] 而不是 &str:整条链(Java 字符串 → JNI → 线格式 →
Windows 注入)的原生表示都是 UTF-16,任何一次 UTF-8 往返都有损,故中间层不做转换。
空输入与超限在本层拦掉,超限走 clamp_text_units 并打日志。

nativeSendText 的参数用 char[](JObject 收再转 JCharArray),经 GetCharArrayRegion
按 u16 直接拷贝。不用 JString + get_string:jni 0.21 那条路走 Modified-UTF-8,
IME 提交边界的落单代理项会在往返里丢。JNI 取数失败必须 eprintln,不能静默——
用户对着远端打字最坏的情况是「看起来发送了」。
2026-10-30 11:41:53 +08:00
郑博文 5075888861 feat(agent): v3 文本注入用 KEYEVENTF_UNICODE,一次批量提交成对代理项
Windows 侧按官方 KEYBDINPUT 语义实现:置 KEYEVENTF_UNICODE 时 wVk 必须为 0、
wScan 即要送出的字符,系统合成 VK_PACKET 经 TranslateMessage 产出 WM_CHAR。
down 批与 up 批分两次 SendInput,且让同一段文本的 code unit 在输入队列里相邻,
使代理对能被远端合成一个字符(最终由真机 emoji 验收定论)。

agent-core 的 InputInjector::text 用默认实现记日志而非静默忽略:Linux 后端
本轮不做文本注入(X11 需 keysym/XKB 映射,另立循环),但用户对着 Linux
被控端打字不能「什么都没发生还没痕迹」。Windows 端走自己的 handle(),
不经 agent-core,故超限截断日志在两侧同源补齐。
日志只打长度,不打用户输入内容。
2026-10-30 11:29:25 +08:00
郑博文 38df106fb6 feat(common): 协议 v3——新增 InputEvent::Text 文本注入通道
安卓软键盘的中文候选词结果不是扫描码,扫描码通道对中文天然不通,必须
另开一条按字符注入的通道。载体选 Vec<u16>(UTF-16 code unit)而非 String:
Windows 注入侧与 Java 字符串原生都是 UTF-16,中间任何一次 UTF-8 往返都会
把 IME 提交边界的落单代理项变成替换字符。

InputEvent 因携带 Vec 不再是 Copy,调用方改用 clone;三端必须同步升级,
旧端由 Auth 握手显式拒绝(这正是 PROTO_VERSION 递增的目的)。
新增 6 个单测覆盖 roundtrip、落单代理项、上限边界与 1MiB 长度闸共存。
2026-10-30 11:29:13 +08:00
45 changed files with 8040 additions and 182 deletions

No files matched your search

Generated
+1
View File
@@ -1434,6 +1434,7 @@ dependencies = [
"quinn", "quinn",
"rustls", "rustls",
"tokio", "tokio",
"windows",
] ]
[[package]] [[package]]
+270 -2
View File
@@ -40,6 +40,22 @@ pub trait ScreenCapturer: Send {
/// 取出最近一帧,BGRA 紧凑排列(已去掉 row pitch),供 openh264 直接编码。 /// 取出最近一帧,BGRA 紧凑排列(已去掉 row pitch),供 openh264 直接编码。
fn copy_frame(&mut self) -> Result<Vec<u8>>; fn copy_frame(&mut self) -> Result<Vec<u8>>;
/// 协议 v5 的光标采样:当前指针在**采集画面内**的物理像素坐标。
/// 语义与折叠规则(隐藏/离屏 → None)见 `common::VideoFrameHeader::cursor`。
///
/// **默认实现刻意返回 None,但必须留下可见痕迹**:Linux 后端本轮未实现光标采样
/// (X11 `query_pointer` 与 Windows `GetCursorInfo` 不同构,且本机无 X11 无法给
/// proof——4b/Q4=A 裁决另立循环)。全局规则要求异常路径有可见后果——安卓用户对着
/// Linux 被控端看不到标记不能是静默的,日志必须说明是没实现。一次性日志防刷屏
/// (本方法每帧都会被调,最高 60Hz)。
fn pointer_state(&self) -> Option<(i16, i16)> {
static LOGGED_UNIMPL: AtomicBool = AtomicBool::new(false);
if !LOGGED_UNIMPL.swap(true, Ordering::Relaxed) {
eprintln!("[agent-core] 光标采样后端未实现:帧头 cursor 将恒为 None");
}
None
}
} }
/// 输入注入后端。实现需可在异步运行时中被多任务共享,故要求 `Send + Sync`。 /// 输入注入后端。实现需可在异步运行时中被多任务共享,故要求 `Send + Sync`。
@@ -48,6 +64,31 @@ pub trait InputInjector: Send + Sync + 'static {
fn mouse_button(&self, b: MouseButton, down: bool); fn mouse_button(&self, b: MouseButton, down: bool);
fn wheel(&self, dy: i32); fn wheel(&self, dy: i32);
fn key(&self, scancode: u16, extended: bool, down: bool); fn key(&self, scancode: u16, extended: bool, down: bool);
/// 协议 v3 的文本注入(UTF-16 code unit)。
///
/// **默认实现刻意不注入任何东西,但必须留下可见痕迹**:Linux 后端本轮未实现
/// 文本注入(X11 需要 keysym/XKB 映射,与 Windows 的 `KEYEVENTF_UNICODE`
/// 不同构,另立循环)。全局规则要求异常路径有可见后果——用户对着 Linux 被控端
/// 打字时「什么都没发生」不能是静默的,日志必须说明是没实现。
///
/// 日志纪律:只打印长度,**不打印用户输入内容**。
fn text(&self, units: &[u16]) {
eprintln!(
"[agent-core] 文本注入未实现(忽略 {} 个 UTF-16 code unit)",
units.len()
);
}
/// 协议 v4 的相对鼠标移动(单位:被控端桌面物理像素)。
///
/// 与 [`InputInjector::text`] 同一条纪律:**默认实现不注入,但必须留下可见痕迹**。
/// Linux/X11 后端本轮未实现相对移动(XTest 的相对语义要自己维护指针状态,
/// 与 Windows 的 `GetCursorPos`+绝对注入不同构,另立循环)。用户用相对模式
/// 操作 Linux 被控端时「指针纹丝不动」不能是静默的,日志必须说明是没实现。
fn mouse_move_rel(&self, dx: i16, dy: i16) {
eprintln!("[agent-core] 相对鼠标注入未实现(忽略 dx={dx} dy={dy})");
}
} }
/// 把一条协议输入事件分发到注入后端。 /// 把一条协议输入事件分发到注入后端。
@@ -59,6 +100,20 @@ pub fn dispatch_input(inj: &dyn InputInjector, ev: InputEvent) {
InputEvent::Wheel { dy } => inj.wheel(dy), InputEvent::Wheel { dy } => inj.wheel(dy),
InputEvent::KeyDown { scancode, extended } => inj.key(scancode, extended, true), InputEvent::KeyDown { scancode, extended } => inj.key(scancode, extended, true),
InputEvent::KeyUp { scancode, extended } => inj.key(scancode, extended, false), InputEvent::KeyUp { scancode, extended } => inj.key(scancode, extended, false),
InputEvent::Text { units } => {
// 上限在前端截断,截断必须可见(否则用户以为长文本已送达)。
let (units, truncated) = common::clamp_text_units(units);
if truncated {
eprintln!(
"[agent-core] 文本超过上限,已截断到 {} 个 code unit",
common::MAX_TEXT_UNITS
);
}
inj.text(&units);
}
// v4:相对移动原样下发(含负值)。clamp 与像素换算由注入后端负责——
// 只有它知道被控端桌面的真实像素尺寸(`agent-core` 是平台无关层)。
InputEvent::MouseMoveRel { dx, dy } => inj.mouse_move_rel(dx, dy),
} }
} }
@@ -137,6 +192,7 @@ fn encode_send(
(w, h): (u32, u32), (w, h): (u32, u32),
ts_ms: u64, ts_ms: u64,
seq: u64, seq: u64,
cursor: Option<(i16, i16)>,
tx: &watch::Sender<Option<Vec<u8>>>, tx: &watch::Sender<Option<Vec<u8>>>,
last_frame: &mut Option<Vec<u8>>, last_frame: &mut Option<Vec<u8>>,
) -> bool { ) -> bool {
@@ -150,6 +206,7 @@ fn encode_send(
ts_ms, ts_ms,
seq, seq,
keyframe: h264_contains_idr(&payload), keyframe: h264_contains_idr(&payload),
cursor,
}; };
let frame = encode_video_frame(&header, &payload); let frame = encode_video_frame(&header, &payload);
if tx.send(Some(frame.clone())).is_err() { if tx.send(Some(frame.clone())).is_err() {
@@ -208,7 +265,16 @@ fn capture_session(
encoder.force_intra_frame(); encoder.force_intra_frame();
} }
let ts = start.elapsed().as_millis() as u64; let ts = start.elapsed().as_millis() as u64;
if !encode_send(&mut encoder, &raw, (w, h), ts, seq, tx, &mut last_frame) { if !encode_send(
&mut encoder,
&raw,
(w, h),
ts,
seq,
cap.pointer_state(),
tx,
&mut last_frame,
) {
return LoopEnd::ChannelClosed; // 会话结束 return LoopEnd::ChannelClosed; // 会话结束
} }
seq = seq.wrapping_add(1); seq = seq.wrapping_add(1);
@@ -247,7 +313,16 @@ fn capture_session(
encoder.force_intra_frame(); encoder.force_intra_frame();
let raw = last_raw.as_ref().expect("can_key 已保证"); let raw = last_raw.as_ref().expect("can_key 已保证");
let ts = start.elapsed().as_millis() as u64; let ts = start.elapsed().as_millis() as u64;
if !encode_send(&mut encoder, raw, (w, h), ts, seq, tx, &mut last_frame) { if !encode_send(
&mut encoder,
raw,
(w, h),
ts,
seq,
cap.pointer_state(),
tx,
&mut last_frame,
) {
return LoopEnd::ChannelClosed; return LoopEnd::ChannelClosed;
} }
seq = seq.wrapping_add(1); seq = seq.wrapping_add(1);
@@ -261,6 +336,9 @@ fn capture_session(
Ok((mut hdr, payload)) => { Ok((mut hdr, payload)) => {
hdr.seq = seq; hdr.seq = seq;
hdr.ts_ms = start.elapsed().as_millis() as u64; hdr.ts_ms = start.elapsed().as_millis() as u64;
// v5 与 seq/ts_ms 同理:重发帧的 cursor 若沿用旧值会让标记
// 冻住(画面静止≠指针静止,触控板用户尤甚)
hdr.cursor = cap.pointer_state();
let frame = encode_video_frame(&hdr, payload); let frame = encode_video_frame(&hdr, payload);
if tx.send(Some(frame)).is_err() { if tx.send(Some(frame)).is_err() {
return LoopEnd::ChannelClosed; return LoopEnd::ChannelClosed;
@@ -483,3 +561,193 @@ where
println!("[agent] session closed"); println!("[agent] session closed");
Ok(()) Ok(())
} }
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
/// 记录调用轨迹的注入后端:验证分发与截断,不碰任何真实输入。
#[derive(Default)]
struct MockInjector {
calls: Mutex<Vec<String>>,
}
impl MockInjector {
fn take(&self) -> Vec<String> {
let mut g = self.calls.lock().unwrap();
std::mem::take(&mut g)
}
}
impl InputInjector for MockInjector {
fn mouse_move(&self, x: u16, y: u16) {
self.calls.lock().unwrap().push(format!("move {x},{y}"));
}
fn mouse_button(&self, b: MouseButton, down: bool) {
self.calls.lock().unwrap().push(format!("btn {b:?} {down}"));
}
fn wheel(&self, dy: i32) {
self.calls.lock().unwrap().push(format!("wheel {dy}"));
}
fn key(&self, scancode: u16, extended: bool, down: bool) {
self.calls
.lock()
.unwrap()
.push(format!("key {scancode:#x} ext={extended} {down}"));
}
// 刻意**不**实现 text():让默认实现进入测试面,验证「后端没做文本注入
// 时也必须被 dispatch_input 正常调用、且不 panic」——Linux 端正是这个形态。
}
/// 实现了 text() 的后端,用来验证截断后的长度真的传给了注入侧。
struct TextCapableInjector {
received: Mutex<Vec<usize>>,
}
impl InputInjector for TextCapableInjector {
fn mouse_move(&self, _x: u16, _y: u16) {}
fn mouse_button(&self, _b: MouseButton, _down: bool) {}
fn wheel(&self, _dy: i32) {}
fn key(&self, _scancode: u16, _extended: bool, _down: bool) {}
fn text(&self, units: &[u16]) {
self.received.lock().unwrap().push(units.len());
}
}
#[test]
fn dispatch_routes_each_event_kind() {
let inj = MockInjector::default();
dispatch_input(&inj, InputEvent::MouseMove { x: 1, y: 2 });
dispatch_input(&inj, InputEvent::MouseDown(MouseButton::Right));
dispatch_input(&inj, InputEvent::Wheel { dy: -120 });
dispatch_input(
&inj,
InputEvent::KeyDown {
scancode: 0x48,
extended: true,
},
);
let calls = inj.take();
assert_eq!(
calls,
vec![
"move 1,2",
"btn Right true",
"wheel -120",
"key 0x48 ext=true true",
]
);
}
/// v4:相对移动必须被分发,**负值原样到达后端**(负值是相对语义的核心,
/// 绝对坐标是无符号的,这条路径是新引入的有符号通道)。
#[test]
fn rel_move_reaches_backend_with_signed_deltas() {
struct RelInjector {
got: Mutex<Vec<(i16, i16)>>,
}
impl InputInjector for RelInjector {
fn mouse_move(&self, _x: u16, _y: u16) {}
fn mouse_button(&self, _b: MouseButton, _down: bool) {}
fn wheel(&self, _dy: i32) {}
fn key(&self, _s: u16, _e: bool, _d: bool) {}
fn mouse_move_rel(&self, dx: i16, dy: i16) {
self.got.lock().unwrap().push((dx, dy));
}
}
let inj = RelInjector {
got: Mutex::new(Vec::new()),
};
dispatch_input(&inj, InputEvent::MouseMoveRel { dx: 12, dy: -34 });
dispatch_input(&inj, InputEvent::MouseMoveRel { dx: -1, dy: 0 });
assert_eq!(
*inj.got.lock().unwrap(),
vec![(12, -34), (-1, 0)],
"相对增量(含负值)必须原样到达注入后端"
);
}
/// 后端**没有**实现 mouse_move_rel 时不得 panic,且相对事件不得被误路由成
/// 绝对移动(Linux/X11 后端的真实形态:默认实现 + 日志)。
#[test]
fn rel_move_default_backend_does_not_panic() {
let inj = MockInjector::default();
dispatch_input(&inj, InputEvent::MouseMoveRel { dx: 5, dy: 5 });
// 默认实现只打日志,不应产生任何注入调用(尤其不能变成 move/绝对移动)
assert!(
inj.take().is_empty(),
"默认实现不得把相对移动降级成绝对移动"
);
}
/// v3 关键回归:`Text` 必须被分发,且**后端没有实现 text() 时不能 panic**。
/// 这是 Linux/X11 后端的真实形态——用户对着 Linux 被控端打字会走到这里。
#[test]
fn text_event_reaches_default_backend_without_panic() {
let inj = MockInjector::default();
dispatch_input(
&inj,
InputEvent::Text {
units: vec![0x4F60, 0x597D],
},
);
// 默认实现只记日志,不产生调用记录;此处断言"没炸"即为通过。
assert!(inj.take().is_empty());
}
/// 截断契约:超过 `MAX_TEXT_UNITS` 的输入必须以截断后的长度到达注入侧。
/// 不截断会让长粘贴把远端注入队列刷爆;截断不告知则是静默丢数据。
#[test]
fn text_is_clamped_to_protocol_cap() {
let inj = TextCapableInjector {
received: Mutex::new(vec![]),
};
dispatch_input(
&inj,
InputEvent::Text {
units: vec![0x61; common::MAX_TEXT_UNITS + 500],
},
);
dispatch_input(
&inj,
InputEvent::Text {
units: vec![0x62; 10],
},
);
let got = inj.received.lock().unwrap().clone();
assert_eq!(got, vec![common::MAX_TEXT_UNITS, 10], "超限截断、未超限原样");
}
/// 空 Text 仍会调用后端(是否忽略由后端决定):主控侧 `send_text` 已拦掉空输入,
/// 但被控端不能假设对端守规矩——线格式输入一律按不可信处理。
#[test]
fn empty_text_still_dispatches() {
let inj = TextCapableInjector {
received: Mutex::new(vec![]),
};
dispatch_input(&inj, InputEvent::Text { units: vec![] });
assert_eq!(inj.received.lock().unwrap().clone(), vec![0]);
}
/// v5:采集后端未实现 `pointer_state` 时必须返回 None 且不 panic——Linux/X11 端
/// 本轮的真实形态(4b/Q4=A:默认 None + 一次性日志,实装另立循环)。
/// 锁的是「默认实现存在且语义为 None」这条契约:若有人把默认实现改成 panic 或
/// 伪造坐标,这条会红。
#[test]
fn pointer_state_default_returns_none_without_panic() {
struct NoPointerCapturer;
impl ScreenCapturer for NoPointerCapturer {
fn size(&self) -> (u32, u32) {
(100, 100)
}
fn acquire(&mut self, _timeout_ms: u32) -> Result<bool> {
Ok(false)
}
fn copy_frame(&mut self) -> Result<Vec<u8>> {
Ok(Vec::new())
}
}
assert_eq!(NoPointerCapturer.pointer_state(), None);
}
}
+3
View File
@@ -24,4 +24,7 @@ windows = { workspace = true, features = [
"Win32_Graphics_Dxgi_Common", "Win32_Graphics_Dxgi_Common",
"Win32_UI_Input_KeyboardAndMouse", "Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging", "Win32_UI_WindowsAndMessaging",
# SetProcessDpiAwarenessContext:相对鼠标的 GetCursorPos 必须与采集同为物理像素,
# 不能依赖注册表 AppCompat 覆盖(见 3b 开工实测 §4)。
"Win32_UI_HiDpi",
] } ] }
+15 -5
View File
@@ -114,6 +114,14 @@ fn arg(name: &str) -> Option<String> {
.cloned() .cloned()
} }
/// v5 观测列:帧头光标字段的稳定文本形态(proof 脚本按 `cursor=(x,y)` / `cursor=None` 解析)
fn cursor_cell(c: &Option<(i16, i16)>) -> String {
match c {
Some((x, y)) => format!("cursor=({x},{y})"),
None => "cursor=None".into(),
}
}
#[tokio::main] #[tokio::main]
async fn main() -> Result<()> { async fn main() -> Result<()> {
rustls::crypto::ring::default_provider() rustls::crypto::ring::default_provider()
@@ -184,8 +192,8 @@ async fn main() -> Result<()> {
// 累积裸码流,用于 --save // 累积裸码流,用于 --save
let mut saved: Vec<u8> = Vec::new(); let mut saved: Vec<u8> = Vec::new();
println!("\n序号 seq kf 载荷字节 NAL 结构 解码结果"); println!("\n序号 seq kf 载荷字节 NAL 结构 光标 解码结果");
println!("{}", "-".repeat(72)); println!("{}", "-".repeat(86));
for i in 0..total { for i in 0..total {
let (hdr, payload) = match tokio::time::timeout( let (hdr, payload) = match tokio::time::timeout(
@@ -221,11 +229,12 @@ async fn main() -> Result<()> {
// 模拟主控端积压跳帧:读出但不解码 // 模拟主控端积压跳帧:读出但不解码
if Some(i) == skip_at { if Some(i) == skip_at {
println!( println!(
"[{i:2}] {:3} {:<5} {:8} {:<14} ⏭ 跳过解码(模拟积压跳帧)", "[{i:2}] {:3} {:<5} {:8} {:<14} {:14} ⏭ 跳过解码(模拟积压跳帧)",
hdr.seq, hdr.seq,
hdr.keyframe, hdr.keyframe,
payload.len(), payload.len(),
describe(&types) describe(&types),
cursor_cell(&hdr.cursor)
); );
continue; continue;
} }
@@ -237,11 +246,12 @@ async fn main() -> Result<()> {
}; };
println!( println!(
"[{i:2}] {:3} {:<5} {:8} {:<14} {}", "[{i:2}] {:3} {:<5} {:8} {:<14} {:14} {}",
hdr.seq, hdr.seq,
hdr.keyframe, hdr.keyframe,
payload.len(), payload.len(),
describe(&types), describe(&types),
cursor_cell(&hdr.cursor),
res res
); );
+125
View File
@@ -36,6 +36,7 @@ use windows::Win32::Graphics::Dxgi::{
IDXGIDevice, IDXGIOutput1, IDXGIOutputDuplication, IDXGIResource, DXGI_ERROR_WAIT_TIMEOUT, IDXGIDevice, IDXGIOutput1, IDXGIOutputDuplication, IDXGIResource, DXGI_ERROR_WAIT_TIMEOUT,
DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO, DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
}; };
use windows::Win32::UI::WindowsAndMessaging::{GetCursorInfo, CURSORINFO, CURSOR_SHOWING};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -50,6 +51,31 @@ use std::time::{Duration, Instant};
/// 又不会因过密的整帧内编码抬高码率。 /// 又不会因过密的整帧内编码抬高码率。
pub const INTRA_PERIOD_FRAMES: u32 = 60; pub const INTRA_PERIOD_FRAMES: u32 = 60;
/// v5:光标可见性 + 采集边界折叠的**纯逻辑**(无 Win32 调用,可单测)。
///
/// 输入是 `GetCursorInfo` 的原始读数与采集画面尺寸;输出协议 `cursor` 字段语义
/// (见 `common::VideoFrameHeader::cursor`):
/// - 未 `CURSOR_SHOWING`(指针隐藏或被触控抑制)→ None
/// - 越出 `[0,w)×[0,h)`(副屏负坐标区、主屏外右侧等)→ None
/// ——采集画面里没有它,主控端画了位置也是错的(4b/Q5=A:语义单一来源在被控端)
/// - 越出 i16 正范围 → None 且第二返回 `true`(调用方打日志):协议用 i16 省字节,
/// 依赖「现实桌面 << 32767」前提;前提破了就折叠,**绝不截断出一个看起来合法的错坐标**
///
/// 独立成函数的原因:这套判定是 4b 折叠规则的全部智力内容,而 Win32 读数在
/// 有人用真鼠标的机器上不可复现——逻辑必须与采样解耦地被测试锁死。
fn fold_cursor(showing: bool, x: i32, y: i32, w: i32, h: i32) -> (Option<(i16, i16)>, bool) {
if !showing {
return (None, false);
}
if x < 0 || y < 0 || x >= w || y >= h {
return (None, false);
}
if x > i16::MAX as i32 || y > i16::MAX as i32 {
return (None, true);
}
(Some((x as i16, y as i16)), false)
}
pub struct Duplication { pub struct Duplication {
pub device: ID3D11Device, pub device: ID3D11Device,
context: ID3D11DeviceContext, context: ID3D11DeviceContext,
@@ -182,6 +208,42 @@ impl Duplication {
Ok(r) Ok(r)
} }
} }
/// v5:采样当前远端光标位置(**采集显示器内**的物理像素,(0,0)=画面左上角)。
///
/// 折叠规则(隐藏/离屏/i16 越界 → None)的唯一真源是 [`fold_cursor`]——
/// 那里是可单测的纯逻辑;本函数只负责 Win32 读数与失败留痕。
///
/// `GetCursorInfo` 失败 → None + 会话期只打一条日志(可见后果,但不逐帧刷屏)。
///
/// 前提:进程必须 per-monitor DPI 感知(`main.rs::init_dpi_awareness`,3b/D2 教训),
/// 否则 `ptScreenPos` 是逻辑坐标,与采集像素不同尺度。
pub fn sample_cursor(&self) -> Option<(i16, i16)> {
static LOGGED_FAIL: AtomicBool = AtomicBool::new(false);
static LOGGED_OVERFLOW: AtomicBool = AtomicBool::new(false);
unsafe {
let mut ci = CURSORINFO::default();
// Win32 契约:调用方必须先填 cbSize,否则 API 直接拒绝(MSDN GetCursorInfo)
ci.cbSize = std::mem::size_of::<CURSORINFO>() as u32;
if GetCursorInfo(&mut ci).is_err() {
if !LOGGED_FAIL.swap(true, Ordering::Relaxed) {
eprintln!("[capture] GetCursorInfo 失败:帧头 cursor 将恒为 None");
}
return None;
}
let (pos, overflow) = fold_cursor(
ci.flags.0 & CURSOR_SHOWING.0 != 0,
ci.ptScreenPos.x,
ci.ptScreenPos.y,
self.width as i32,
self.height as i32,
);
if overflow && !LOGGED_OVERFLOW.swap(true, Ordering::Relaxed) {
eprintln!("[capture] 光标坐标超出 i16 协议范围,已折叠为 None(桌面尺寸异常?)");
}
pos
}
}
} }
/// NVENC 硬编封装:H.264 High、CBR 码率控制、UltraLowLatency 调优、无 B 帧。 /// NVENC 硬编封装:H.264 High、CBR 码率控制、UltraLowLatency 调优、无 B 帧。
@@ -364,6 +426,10 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
// `$kf` 由调用点显式给出:NVENC 路径知道 `force_key`,软编路径同理。 // `$kf` 由调用点显式给出:NVENC 路径知道 `force_key`,软编路径同理。
// 不要在这里猜——首帧和 KeyFrameReq 都会产生 IDR,猜错会让解码端错过重同步点。 // 不要在这里猜——首帧和 KeyFrameReq 都会产生 IDR,猜错会让解码端错过重同步点。
//
// `cursor` 在**这里**采样(不在循环顶部):帧头语义是「本帧发送时刻的位置」。
// emit! 是唯一的发送入口,静止保活重发也走它(500ms 分支),所以新帧 /
// 静止重发 / 强制 IDR 三种发送路径天然全覆盖,无需各自补采样。
macro_rules! emit { macro_rules! emit {
($payload:expr, $kf:expr) => {{ ($payload:expr, $kf:expr) => {{
let header = VideoFrameHeader { let header = VideoFrameHeader {
@@ -372,6 +438,7 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
ts_ms: start.elapsed().as_millis() as u64, ts_ms: start.elapsed().as_millis() as u64,
seq, seq,
keyframe: $kf, keyframe: $kf,
cursor: dup.sample_cursor(),
}; };
let payload_vec: Vec<u8> = $payload; let payload_vec: Vec<u8> = $payload;
let frame = encode_video_frame(&header, &payload_vec); let frame = encode_video_frame(&header, &payload_vec);
@@ -585,3 +652,61 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
std::thread::sleep(Duration::from_millis(300)); std::thread::sleep(Duration::from_millis(300));
} }
} }
#[cfg(test)]
mod tests {
use super::fold_cursor;
const W: i32 = 1920;
const H: i32 = 1080;
/// 界内可见 → 原样返回。中心与 (0,0) 都要覆盖:(0,0) 与 None 在协议里是两种语义,
/// 若实现把「左上角」误折叠成 None,光标到左上角就「消失」了(这是最容易写错的一条)。
#[test]
fn inside_bounds_pass_through_including_origin() {
assert_eq!(fold_cursor(true, 960, 540, W, H), (Some((960, 540)), false));
assert_eq!(fold_cursor(true, 0, 0, W, H), (Some((0, 0)), false));
assert_eq!(
fold_cursor(true, W - 1, H - 1, W, H),
(Some((1919, 1079)), false)
);
}
/// 隐藏/触控抑制(CURSOR_SHOWING 未置位)→ None,**即使坐标界内**。
/// 顺序很关键:坐标合法但不可见时也不能画标记。
#[test]
fn hidden_cursor_folds_to_none_even_when_in_bounds() {
assert_eq!(fold_cursor(false, 100, 100, W, H), (None, false));
assert_eq!(fold_cursor(false, 0, 0, W, H), (None, false));
}
/// 越出采集画面 → None:负坐标(副屏在左,本机实测 -1920..-1)、
/// 右/下边界外、以及**恰好在边界上**(坐标 w 对应第 w+1 列,画面里没有)。
#[test]
fn out_of_capture_bounds_folds_to_none() {
assert_eq!(fold_cursor(true, -960, 500, W, H), (None, false)); // 副屏(负 X)
assert_eq!(fold_cursor(true, 100, -1, W, H), (None, false)); // 负 Y
assert_eq!(fold_cursor(true, W, 500, W, H), (None, false)); // 右边界外一列
assert_eq!(fold_cursor(true, 500, H, W, H), (None, false)); // 下边界外一行
assert_eq!(fold_cursor(true, 2610, 1300, W, H), (None, false)); // 主屏外右侧
}
/// 越出 i16 正范围 → None + 第二返回 true(调用方据此打日志)。
/// 注意:常规采集尺寸(≤1920)下越界 x 永远先被边界检查拦下,i16 分支不可达——
/// 必须用超宽采集(假想带鱼屏)才能测到;它证明「界内但超 i16」会被折叠
/// 而**不是截断成看似合法的错坐标**。i16::MAX 恰好合法,不误报。
#[test]
fn beyond_i16_folds_with_overflow_flag() {
let huge_w = 40_000; // 假想的超宽采集,越过 i16::MAX
assert_eq!(
fold_cursor(true, i16::MAX as i32, 0, huge_w, H),
(Some((i16::MAX, 0)), false),
"i16::MAX 恰好界内合法,不该误报 overflow"
);
assert_eq!(
fold_cursor(true, 33_000, 10, huge_w, H),
(None, true),
"界内但超 i16 → overflow=true 的 None,而非截断成错误坐标"
);
}
}
+139 -6
View File
@@ -1,16 +1,18 @@
//! 输入注入:SendInput 实现(鼠标归一化绝对坐标 + 扫描码键盘)。 //! 输入注入:SendInput 实现(鼠标归一化绝对坐标 + 扫描码键盘 + v3 文本注入)。
use common::{InputEvent, MouseButton}; use common::{InputEvent, MouseButton};
use windows::Win32::UI::Input::KeyboardAndMouse::{ use windows::Win32::UI::Input::KeyboardAndMouse::{
SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY,
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_MIDDLEDOWN, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, KEYEVENTF_UNICODE, MOUSEEVENTF_ABSOLUTE,
MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN,
MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEINPUT, VIRTUAL_KEY, MOUSE_EVENT_FLAGS, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEINPUT, VIRTUAL_KEY,
MOUSE_EVENT_FLAGS,
}; };
use windows::Win32::UI::WindowsAndMessaging::{ use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, GetCursorPos, GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_CXSCREEN,
SM_YVIRTUALSCREEN, SM_CYSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN,
}; };
use windows::Win32::Foundation::POINT;
// windows 0.58 未导出 LEFT 常量,按 WinUser.h 原值定义 // windows 0.58 未导出 LEFT 常量,按 WinUser.h 原值定义
const MOUSEEVENTF_LEFTDOWN: u32 = 0x0002; const MOUSEEVENTF_LEFTDOWN: u32 = 0x0002;
@@ -35,6 +37,62 @@ fn send_mouse(flags: u32, dx: i32, dy: i32, data: u32) {
} }
} }
/// 主显示器物理尺寸(像素)。相对移动的 clamp 上界必须与**采集空间**一致
/// (capture 用 `EnumOutputs(0)`;AuthOk 下发的 width/height 也来自它)。
fn output_size() -> (i32, i32) {
unsafe {
(
GetSystemMetrics(SM_CXSCREEN),
GetSystemMetrics(SM_CYSCREEN),
)
}
}
/// v4 相对鼠标:把远端指针从当前位置移动 `(dx, dy)` 像素。
///
/// **为什么不用 `SendInput(MOUSEEVENTF_MOVE)`**(协议 v4 的语义最容易在这里跑偏):
/// 实测(2026-10-31,见个人层 `docs/output/2026-10-31-3b-开工前实测.md`)该路径的
/// `dx` 不是像素,而是「mickey」,会被两件事放大:
/// 1. **指针加速**(「提高指针精确度」)。本机 `SPI_GETMOUSE=6/10/1` 下
/// dx=20 实际走 41px、dx=100 走 309px,且倍率是非线性的;
/// 2. **DPI 缩放**。关掉加速后倍率恒为 1.25 = 本机 125% 缩放。
/// 两者都随用户系统设置与机器变化,无法标定,因此**不能**把协议里的像素增量直接透传。
///
/// 改为「读光标 → 加增量 → clamp → 绝对注入」:
/// - 语义仍是**相对**(主控端只说移动多少,不说去哪里),远端指针始终权威;
/// - 实测 1:1 精确(dx=±1/2/3/5/17/100 四次重复全部等于请求值);
/// - 撞边后回滑立即响应,不积累「步数债」(clamp 丢掉的部分不回补)。
///
/// 已知限制(不隐藏):读光标与注入之间若用户同时移动真鼠标,会有一次增量被叠加。
/// 本机联调场景可接受;彻底消除需要 `SendInput` 的 mickey 路径,而那条路不可标定。
fn send_mouse_rel(dx: i16, dy: i16) {
if dx == 0 && dy == 0 {
return; // 零位移没有注入语义,也不该产生一次系统调用
}
let (w, h) = output_size();
if w <= 1 || h <= 1 {
eprintln!("[input] 相对移动失败:输出尺寸异常 {w}x{h}");
return;
}
let mut cur = POINT::default();
// GetCursorPos 失败时不能猜一个坐标继续注入——那会把指针扔到屏幕角落
if let Err(e) = unsafe { GetCursorPos(&mut cur) } {
eprintln!("[input] GetCursorPos 失败,相对移动被丢弃: {e}");
return;
}
let tx = (cur.x + dx as i32).clamp(0, w - 1);
let ty = (cur.y + dy as i32).clamp(0, h - 1);
// 归一化到 0..=65535(绝对注入空间 = 主显示器,不带 VIRTUALDESK,见 MouseMove 分支)
let nx = ((tx as i64 * 65535 + (w as i64 - 1) / 2) / (w as i64 - 1)) as i32;
let ny = ((ty as i64 * 65535 + (h as i64 - 1) / 2) / (h as i64 - 1)) as i32;
send_mouse(
MOUSEEVENTF_ABSOLUTE.0 | MOUSEEVENTF_MOVE.0,
nx,
ny,
0,
);
}
fn send_key(scancode: u16, extended: bool, up: bool) { fn send_key(scancode: u16, extended: bool, up: bool) {
let mut flags = KEYEVENTF_SCANCODE; let mut flags = KEYEVENTF_SCANCODE;
if extended { if extended {
@@ -73,6 +131,67 @@ pub fn virtual_desktop() -> (i32, i32, i32, i32) {
} }
} }
/// v3 文本注入:把一段 UTF-16 code unit 作为键盘事件送入前台应用。
///
/// 语义来源(实读,不凭记忆):
/// [KEYBDINPUT (winuser.h)](https://learn.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-keybdinput)
/// - `KEYEVENTF_UNICODE`(0x0004):`wVk` **必须为 0**,`wScan` 即要送出的 Unicode 字符;
/// 系统合成 `VK_PACKET` 按键事件,经 `TranslateMessage` 转为带该字符的 `WM_CHAR`。
/// - 文档原文是 "This flag can only be combined with the KEYEVENTF_KEYUP flag"。
/// 本项目**仍先发不带 KEYUP 的 down**(与现有 `send_key` 的 down/up 成对形态一致),
/// 把那句读作「不能与 SCANCODE/EXTENDEDKEY 混用」是**我的推断,文档没有明说**。
/// 若真机验收发现 down 事件被系统丢弃(表现为一个字符进两次或不进),
/// 退路是只发 `KEYEVENTF_UNICODE | KEYEVENTF_KEYUP` 单事件——改一处 `mk` 即可。
///
/// 为什么一次性批量提交(而非逐字符一对 down/up):
/// 代理对(U+1F600 = D83D DE00)的两个 code unit 必须在输入队列里**相邻**,
/// 远端才会把它们合成成一个字符。逐字符成对(down,up,down,up)会在两个
/// down 之间插入 up 事件;Windows 对 VK_PACKET 孤立代理项的缓冲规则文档
/// 没有描述,所以这里选择让两个 down 严格相邻的形态,由真机 emoji 验收(V7)定论。
///
/// 为什么是**一次** `SendInput` 调用(而不是 down 批 + up 批两次):
/// [Keyboard Input Overview](https://learn.microsoft.com/en-us/windows/win32/inputdev/about-keyboard-input)
/// 的 Simulating Input 一节明确——单调用注入的一批事件"系统确保不会被其他输入事件
/// 打散"(the system ensures that no other input events intermingle with the
/// simulated events),两次调用之间则可能插入真实键盘/鼠标事件,代理对就有被打散风险。
/// 于是把 N 个 down + N 个 up 拼成一个数组一次提交,顺序仍是全部 down 先于全部 up。
///
/// 上限由调用方(`common::clamp_text_units`,4096)保证;这里对空输入直接返回,
/// 因为 `SendInput(0)` 会返回 0(失败)且没有任何注入语义。
fn send_text(units: &[u16]) {
if units.is_empty() {
return;
}
let mk = |scan: u16, up: bool| INPUT {
r#type: INPUT_KEYBOARD,
Anonymous: INPUT_0 {
ki: KEYBDINPUT {
wVk: VIRTUAL_KEY(0), // KEYEVENTF_UNICODE 要求 wVk=0(官方文档)
wScan: scan,
dwFlags: if up {
KEYEVENTF_UNICODE | KEYEVENTF_KEYUP
} else {
KEYEVENTF_UNICODE
},
time: 0,
dwExtraInfo: 0,
},
},
};
// 一次提交:全部 down,紧接全部 up。
let inputs: Vec<INPUT> = units
.iter()
.map(|&u| mk(u, false))
.chain(units.iter().map(|&u| mk(u, true)))
.collect();
let n = unsafe { SendInput(&inputs, std::mem::size_of::<INPUT>() as i32) };
// 可见失败后果:SendInput 返回「实际插入的事件数」,不等于请求数即失败
// (常见原因:桌面被 Secure UI 锁定、被 BlockInput 拦截)。
if n as usize != inputs.len() {
eprintln!("[input] SendInput 文本注入不完整: {n}/{}", inputs.len());
}
}
pub fn handle(ev: InputEvent) { pub fn handle(ev: InputEvent) {
match ev { match ev {
InputEvent::MouseMove { x, y } => { InputEvent::MouseMove { x, y } => {
@@ -105,5 +224,19 @@ pub fn handle(ev: InputEvent) {
} }
InputEvent::KeyDown { scancode, extended } => send_key(scancode, extended, false), InputEvent::KeyDown { scancode, extended } => send_key(scancode, extended, false),
InputEvent::KeyUp { scancode, extended } => send_key(scancode, extended, true), InputEvent::KeyUp { scancode, extended } => send_key(scancode, extended, true),
InputEvent::MouseMoveRel { dx, dy } => send_mouse_rel(dx, dy),
InputEvent::Text { units } => {
// 上限截断必须可见(与 agent-core::dispatch_input 同口径;Windows 端
// 走自己的 handle(),不经过 agent-core,所以这里同源补齐——R-3 教训)。
let (units, truncated) = common::clamp_text_units(units);
if truncated {
eprintln!(
"[input] 文本超过上限,已截断到 {} 个 code unit",
common::MAX_TEXT_UNITS
);
}
// 只打印长度:注入日志不得含用户输入内容(安全红线)
send_text(&units);
}
} }
} }
+52
View File
@@ -32,8 +32,60 @@ fn parse_args() -> Result<(u16, String)> {
Ok((port, code)) Ok((port, code))
} }
/// 显式声明进程 DPI 感知(**必须在任何采集/注入调用之前**,越早越好)。
///
/// 为什么不能省(3b 开工实测,见个人层 `docs/output/2026-10-31-3b-开工前实测.md` §4):
/// 相对鼠标走「`GetCursorPos` + 增量」,而 `GetCursorPos` 的坐标空间取决于进程的
/// DPI 感知模式——UNAWARE 进程拿到的是**逻辑坐标**(本机 125% 缩放下是 1536x864),
/// 而 DXGI 采集与 `AuthOk` 下发的是**物理像素**(1920x1080)。两者混用会让
/// clamp 上界偏小、增量尺度偏小:指针到不了屏幕右/下边缘,且移动距离只有 80%。
///
/// 为什么不能依赖现状:本机 `agent.exe` 实测确实是 `PER_MONITOR_AWARE(2)`,但那是
/// `HKCU\...\AppCompatFlags\Layers` 里一条**路径绑定**的 `HIGHDPIAWARE` 覆盖造成的
/// ——同一个 exe 复制到 `%TEMP%` 启动就变成 `UNAWARE(0)`。换机器、换构建路径、
/// CI 产物、用户机器上都不成立,所以必须由代码显式设置。
///
/// 失败时只告警不退出:DPI 感知不可用是老系统(< Win10 1703)的情况,
/// 此时相对鼠标会退化(精度下降),但绝对注入与其余功能不受影响——
/// 让整个被控端起不来是更差的选择。
fn init_dpi_awareness() {
use windows::Win32::UI::HiDpi::{
GetThreadDpiAwarenessContext, SetProcessDpiAwarenessContext, GetAwarenessFromDpiAwarenessContext,
DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
match unsafe { SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) } {
Ok(()) => println!("[agent] DPI awareness: PER_MONITOR_AWARE_V2 (set by this process)"),
Err(e) => {
// 失败有两种完全不同的原因,**必须分开报告**,否则日志会误导排查方向:
// a) 进程的感知模式已被别处设定(注册表 AppCompat 覆盖、清单、父进程)——
// ERROR_ACCESS_DENIED(0x80070005),此时若已经是 per-monitor 就完全没问题;
// b) 系统过旧不支持该 API——这才是真正的降级。
// 本机实测是 (a):HKCU 的 AppCompatFlags\Layers 里有一条路径绑定的 HIGHDPIAWARE。
let actual = unsafe {
GetAwarenessFromDpiAwarenessContext(GetThreadDpiAwarenessContext())
};
// DPI_AWARENESS(WinUser.h):INVALID=-1、UNAWARE=0、SYSTEM_AWARE=1、
// PER_MONITOR_AWARE=2。注意 PER_MONITOR 是 **2** 而不是 3。
let is_per_monitor = actual.0 == 2;
if is_per_monitor {
println!(
"[agent] DPI awareness: 已由外部设定为 PER_MONITOR_AWARE(无需本进程设置;{e})"
);
} else {
eprintln!(
"[agent] 警告:DPI 感知为 {}(非 per-monitor),且本进程设置失败({e})。\
相对鼠标的位移比例会与被采集桌面不一致;绝对触摸不受影响。",
actual.0
);
}
}
}
}
#[tokio::main] #[tokio::main]
async fn main() -> Result<()> { async fn main() -> Result<()> {
// 第一件事:定下 DPI 感知,后续所有坐标运算都依赖它
init_dpi_awareness();
let (port, code) = parse_args()?; let (port, code) = parse_args()?;
rustls::crypto::ring::default_provider() rustls::crypto::ring::default_provider()
.install_default() .install_default()
+192 -30
View File
@@ -15,26 +15,116 @@
//! 句柄约定:`nativeConnect` 返回 Box 裸指针(永不为 0)作为 jlong; //! 句柄约定:`nativeConnect` 返回 Box 裸指针(永不为 0)作为 jlong;
//! `nativeDestroy` 释放。句柄释放后不得再使用,Kotlin 侧用置 0 保护。 //! `nativeDestroy` 释放。句柄释放后不得再使用,Kotlin 侧用置 0 保护。
//! //!
//! 帧返回约定:byte[],布局 = [1 标志][8 seq LE][8 ts LE][4 w LE][4 h LE][payload]。 //! 帧返回约定:byte[],布局 = [1 标志][8 seq LE][8 ts LE][4 w LE][4 h LE]
//! 定长头 25 字节;Kotlin 读 buf[0] 判断 keyframe(标志位复用为:0=P 帧, 1=IDR 帧, //! [1 has_cursor][2 cursorX i16 LE][2 cursorY i16 LE][payload]。
//! 会话结束则整个返回 null)。 //! 定长头 30 字节(v5 起;此前 25);Kotlin 读 buf[0] 判断 keyframe(标志位复用为:
//! 0=P 帧, 1=IDR 帧,会话结束则整个返回 null)。
//! has_cursor=0 时后 4 字节恒为 0 且**无意义**(指针隐藏/离屏/后端未实现,
//! 语义单一来源在被控端——见 common::VideoFrameHeader::cursor);=1 时坐标为
//! 采集画面内物理像素(0 ≤ x < w、0 ≤ y < h,被控端已折叠越界值)。
use common::InputEvent; use common::InputEvent;
use jni::objects::{JClass, JString}; use jni::objects::{JCharArray, JClass, JObject, JString};
use jni::sys::{jbyteArray, jint, jlong, jstring}; use jni::sys::{jbyteArray, jint, jlong, jstring};
use jni::JNIEnv; use jni::JNIEnv;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use viewer_core::{ConnectConfig, Session}; use viewer_core::{ConnectConfig, Session};
/// 帧头长度:keyframe 标志(1) + seq(8) + ts(8) + w(4) + h(4) /// 帧头长度:keyframe 标志(1) + seq(8) + ts(8) + w(4) + h(4) + has_cursor(1) + x(2) + y(2)
const FRAME_HEADER_LEN: usize = 25; /// (v5:25→30,尾部三字段是光标;放末尾保证 Kotlin 旧偏移量全部不移动)
const FRAME_HEADER_LEN: usize = 30;
/// 把一个帧的元数据打包成定长头字节(**不含载荷**)。
///
/// 独立成纯函数并被单测钉住的原因:Kotlin 侧 `FrameCodec` 按同样的偏移读这些字节,
/// 任一侧挪一位,两端都编译得过、真机也连得上,只是标记画错位置——静默失败。
/// 单测用**手写字面量**断言(不是"算出来的期望值"),这样 RustBridgeTest 的
/// golden bytes 与这里是同一组常数对拍:改了写入布局,两边至少一边必红。
fn pack_frame_header(f: &viewer_core::Frame) -> [u8; FRAME_HEADER_LEN] {
let mut head = [0u8; FRAME_HEADER_LEN];
head[0] = f.keyframe as u8;
head[1..9].copy_from_slice(&f.seq.to_le_bytes());
head[9..17].copy_from_slice(&f.ts_ms.to_le_bytes());
head[17..21].copy_from_slice(&f.width.to_le_bytes());
head[21..25].copy_from_slice(&f.height.to_le_bytes());
// v5 光标三字段(偏移 25/26/28):None → has=0 且坐标字节保持 0(无意义)。
// 用显式 match 而不是 `f.cursor.is_some() as u8`——两处读同一个 Option,
// 任何「Some 但忘写坐标」的编辑错误都会在这里暴露。
match f.cursor {
Some((x, y)) => {
head[25] = 1;
head[26..28].copy_from_slice(&x.to_le_bytes());
head[28..30].copy_from_slice(&y.to_le_bytes());
}
None => head[25] = 0,
}
head
}
#[cfg(test)]
mod tests {
use super::*;
/// 构造最小 Frame(载荷只放一字节,本测试只关心定长头)。
fn frame(seq: u64, cursor: Option<(i16, i16)>) -> viewer_core::Frame {
viewer_core::Frame {
width: 1920,
height: 1080,
ts_ms: 0x0011_2233_4455_6677,
seq,
keyframe: true,
cursor,
payload: vec![0xAB],
}
}
/// **写端**字面量对拍:期望值逐字节手写,与 Kotlin 侧 RustBridgeTest 的
/// goldenHeader 是同一组常数。两侧各握一份手抄常数,改任何一侧的偏移,
/// 另一侧必红——这才是跨端契约的闭环(golden 测试只锁读端时有盲区:
/// 若 Kotlin 的字面量本身抄错了 Rust 布局,读端测试会跟着一起错还全绿)。
#[test]
fn header_bytes_match_the_declared_layout() {
let head = pack_frame_header(&frame(0x0102_0304_0506_0708, Some((960, 540))));
assert_eq!(
head,
[
0x01, // [0] keyframe=true
0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, // [1..9] seq=0x0102..0708 LE
0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00, // [9..17] ts LE
0x80, 0x07, 0x00, 0x00, // [17..21] w=1920=0x0780 LE
0x38, 0x04, 0x00, 0x00, // [21..25] h=1080=0x0438 LE
0x01, // [25] has_cursor
0xC0, 0x03, // [26..28] x=960=0x03C0 LE
0x1C, 0x02, // [28..30] y=540=0x021C LE
]
);
}
/// None 分支:has=0 且**坐标字节必须为 0**(vec 初始化保证),Kotlin 侧
/// 测试锁「has=0 时无条件 null」,这里锁「写端不残留脏坐标」。
#[test]
fn none_cursor_writes_zero_flag_and_zero_coords() {
let head = pack_frame_header(&frame(1, None));
assert_eq!(head[25], 0);
assert_eq!(&head[26..30], &[0u8; 4]);
}
/// 负坐标(协议层不该到达,但写端要忠实 i16 位模式——符号截断是静默错位)。
#[test]
fn negative_coords_keep_bit_pattern() {
let head = pack_frame_header(&frame(1, Some((-1, -32768))));
assert_eq!(&head[26..28], &[0xFF, 0xFF]);
assert_eq!(&head[28..30], &[0x00, 0x80]);
}
}
struct SessionBox { struct SessionBox {
session: Arc<Session>, session: Arc<Session>,
runtime: tokio::runtime::Runtime, runtime: tokio::runtime::Runtime,
} }
/// 连接被控端。成功返回句柄(>0),失败返回 0;错误消息经 `nativeLastError()` 取。 /// 连接被控端。成功返回句柄(>0),失败返回 0;错误消息经 `nativeLastError()` 取,
/// **错误分类**经 `nativeLastErrorKind()` 取(3a:界面按分类显示中文文案,原始英文作第二行)。
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>( pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>(
mut env: JNIEnv<'local>, mut env: JNIEnv<'local>,
@@ -46,14 +136,14 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>(
let host_s = match env.get_string(&host) { let host_s = match env.get_string(&host) {
Ok(s) => s.to_string_lossy().into_owned(), Ok(s) => s.to_string_lossy().into_owned(),
Err(_) => { Err(_) => {
set_last_error("host 参数不是合法字符串"); set_last_error(-1, "host 参数不是合法字符串");
return 0; return 0;
} }
}; };
let code_s = match env.get_string(&code) { let code_s = match env.get_string(&code) {
Ok(s) => s.to_string_lossy().into_owned(), Ok(s) => s.to_string_lossy().into_owned(),
Err(_) => { Err(_) => {
set_last_error("code 参数不是合法字符串"); set_last_error(-1, "code 参数不是合法字符串");
return 0; return 0;
} }
}; };
@@ -64,7 +154,7 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>(
.build() .build()
{ {
Ok(r) => r, Ok(r) => r,
Err(e) => return Err(format!("runtime init: {e}")), Err(e) => return Err((-1i32, format!("runtime init: {e}"))),
}; };
let cfg = ConnectConfig::direct(host_s, port as u16, code_s); let cfg = ConnectConfig::direct(host_s, port as u16, code_s);
match runtime.block_on(Session::connect(cfg)) { match runtime.block_on(Session::connect(cfg)) {
@@ -72,53 +162,73 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>(
session: Arc::new(session), session: Arc::new(session),
runtime, runtime,
}), }),
Err(e) => Err(format!("{e:#}")), // ConnectError 的 Display 即原始 detail;kind.code() 与
// viewer_core::ConnectErrorKind::code 的映射表一致(FFI 契约)。
Err(e) => Err((e.kind.code(), e.detail)),
} }
}); });
match built { match built {
Ok(Ok(boxed)) => { Ok(Ok(boxed)) => {
set_last_error(""); set_last_error(0, "");
Box::into_raw(Box::new(Mutex::new(boxed))) as jlong Box::into_raw(Box::new(Mutex::new(boxed))) as jlong
} }
Ok(Err(msg)) => { Ok(Err((kind, msg))) => {
set_last_error(&msg); set_last_error(kind, &msg);
0 0
} }
Err(_) => { Err(_) => {
set_last_error("panic in nativeConnect"); set_last_error(-1, "panic in nativeConnect");
0 0
} }
} }
} }
fn set_last_error(msg: &str) { /// kind = 0 表示清除;-1 表示 FFI 层自身错误(无 viewer-core 分类,Kotlin 走 Other 文案)。
fn set_last_error(kind: i32, msg: &str) {
let mut slot = LAST_ERROR.lock().unwrap(); let mut slot = LAST_ERROR.lock().unwrap();
if msg.is_empty() { if msg.is_empty() {
*slot = None; *slot = None;
} else { } else {
*slot = Some(msg.to_string()); *slot = Some((kind, msg.to_string()));
} }
} }
static LAST_ERROR: Mutex<Option<String>> = Mutex::new(None); static LAST_ERROR: Mutex<Option<(i32, String)>> = Mutex::new(None);
/// 取最近一次 nativeConnect 的错误消息(无错误返回 null)。 /// 取最近一次 nativeConnect 的错误消息(无错误返回 null)。
/// FFI 契约:调用方必须先 `nativeLastErrorKind()` 再 `nativeLastError()`——
/// 本函数读走消息时**清空整个槽位**(kind 一起失效),颠倒顺序会拿到 kind=0。
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeLastError<'local>( pub extern "system" fn Java_com_verify_remote_RustBridge_nativeLastError<'local>(
env: JNIEnv<'local>, env: JNIEnv<'local>,
_class: JClass<'local>, _class: JClass<'local>,
) -> jstring { ) -> jstring {
let msg = LAST_ERROR.lock().unwrap().take(); let mut slot = LAST_ERROR.lock().unwrap();
let msg = slot.as_ref().map(|(_, m)| m.clone());
match msg { match msg {
Some(m) => env Some(m) => {
.new_string(m) *slot = None;
.map(|s| s.into_raw()) env.new_string(m)
.unwrap_or(std::ptr::null_mut()), .map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
None => std::ptr::null_mut(), None => std::ptr::null_mut(),
} }
} }
/// 取最近一次 nativeConnect 的错误分类码(viewer_core::ConnectErrorKind::code)。
/// 只读不清空;0 = 无错误,-1 = FFI 层自身错误(参数非法/panic/runtime 初始化失败,
/// 无 viewer-core 分类,Kotlin 按 Other 文案 + 原始消息显示)。
#[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeLastErrorKind(
_env: JNIEnv,
_class: JClass,
) -> jint {
let slot = LAST_ERROR.lock().unwrap();
slot.as_ref().map(|(k, _)| *k).unwrap_or(0)
}
/// 取下一帧(阻塞直到有帧或会话结束)。返回 null = 会话结束。 /// 取下一帧(阻塞直到有帧或会话结束)。返回 null = 会话结束。
/// 仅允许解码线程调用;destroy 前必须先停止本线程(见文件头锁纪律)。 /// 仅允许解码线程调用;destroy 前必须先停止本线程(见文件头锁纪律)。
#[no_mangle] #[no_mangle]
@@ -141,13 +251,11 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeNextFrame<'local>
match rt.block_on(session.next_frame()) { match rt.block_on(session.next_frame()) {
None => std::ptr::null_mut(), None => std::ptr::null_mut(),
Some(f) => { Some(f) => {
let mut buf = vec![0u8; FRAME_HEADER_LEN + f.payload.len()]; // 布局单一真源是 pack_frame_header(那里有字面量对拍单测)
buf[0] = f.keyframe as u8; let head = pack_frame_header(&f);
buf[1..9].copy_from_slice(&f.seq.to_le_bytes()); let mut buf = Vec::with_capacity(FRAME_HEADER_LEN + f.payload.len());
buf[9..17].copy_from_slice(&f.ts_ms.to_le_bytes()); buf.extend_from_slice(head.as_slice());
buf[17..21].copy_from_slice(&f.width.to_le_bytes()); buf.extend_from_slice(&f.payload);
buf[21..25].copy_from_slice(&f.height.to_le_bytes());
buf[FRAME_HEADER_LEN..].copy_from_slice(&f.payload);
match env.byte_array_from_slice(&buf) { match env.byte_array_from_slice(&buf) {
Ok(a) => a.into_raw(), Ok(a) => a.into_raw(),
Err(_) => std::ptr::null_mut(), Err(_) => std::ptr::null_mut(),
@@ -204,6 +312,21 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseMove(
}); });
} }
#[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseRel(
_env: JNIEnv,
_class: JClass,
handle: jlong,
dx: jint,
dy: jint,
) {
// 协议 v4 的字段是 i16:Kotlin 侧传 Int,越界一律 clamp 而不是静默回绕。
// 越界值只可能来自主控端 bug 或恶意对端;clamp 让行为有界且可见(不 panic、不 UB)。
let dx = dx.clamp(i16::MIN as i32, i16::MAX as i32) as i16;
let dy = dy.clamp(i16::MIN as i32, i16::MAX as i32) as i16;
with_session(handle, |s| s.send_mouse_rel(dx, dy));
}
/// button: 0=左 1=右 其他=中;down: 1=按下 0=抬起 /// button: 0=左 1=右 其他=中;down: 1=按下 0=抬起
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseButton( pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseButton(
@@ -262,6 +385,45 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendKey(
}); });
} }
/// 协议 v3:文本上行。`units` 是 Kotlin 侧的 `CharArray`(UTF-16 code unit)。
///
/// 为什么不用 `JString`:jni 0.21 的 `get_string` 走 Modified-UTF-8 → Rust `String`,
/// 落单代理项(IME 提交边界常见)会在这次往返里丢失。`char[]` 经
/// `GetCharArrayRegion` 按 u16 直接拷贝,零转换(`jchar = u16`,见 jni-sys-0.3.1/lib.rs:15)。
///
/// 截断与空输入的判断在 `viewer_core::Session::send_text` 里做,与被控端同一口径。
/// 失败必须有可见后果:JNI 取数失败走 eprintln(Android 上落 stderr→logcat),
/// 不静默返回成功——用户对着远端打字却没字,最坏的就是「看起来发送了」。
#[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendText<'local>(
// 不需要 `mut`:jni 0.21.1 的 `get_array_length` / `get_char_array_region`
// 都是 `&self`(jnienv.rs:1652 / :1887),与 `new_string` 之类的 `&mut self` 不同。
env: JNIEnv<'local>,
_class: JClass<'local>,
handle: jlong,
units: JObject<'local>,
) {
if units.as_raw().is_null() {
// Kotlin 侧不会传 null,但 JNI 入口的对端数据一律不信任(全局安全规则)
return;
}
let arr = JCharArray::from(units);
let len = match env.get_array_length(&arr) {
Ok(n) if n > 0 => n as usize,
Ok(_) => return,
Err(e) => {
eprintln!("[android-ffi] get_array_length failed: {e}");
return;
}
};
let mut buf = vec![0u16; len];
if let Err(e) = env.get_char_array_region(&arr, 0, &mut buf) {
eprintln!("[android-ffi] get_char_array_region failed: {e}");
return;
}
with_session(handle, |s| s.send_text(&buf));
}
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeRequestKeyframe( pub extern "system" fn Java_com_verify_remote_RustBridge_nativeRequestKeyframe(
_env: JNIEnv, _env: JNIEnv,
+13 -2
View File
@@ -11,8 +11,8 @@ android {
applicationId = "com.verify.remote" applicationId = "com.verify.remote"
minSdk = 26 minSdk = 26
targetSdk = 34 targetSdk = 34
versionCode = 1 versionCode = 5
versionName = "0.1-phase1" versionName = "0.1-phase4b"
// 阶段 1 只测真机 arm64;x86_64 留给模拟器回归 // 阶段 1 只测真机 arm64;x86_64 留给模拟器回归
ndk { ndk {
abiFilters += listOf("arm64-v8a") abiFilters += listOf("arm64-v8a")
@@ -25,6 +25,15 @@ android {
} }
} }
// 3a/U7:安卓测试基础设施(Q3=A,用户 2026-10-31 批准)。
// **仅测试作用域**:junit 不进 APK,runtime dependencies 保持为空。
testOptions {
unitTests {
// android.jar 桩方法抛 "not mocked"——本工程的单测刻意只测纯逻辑
// (不构造 Android 对象),此开关只兜常量引用类的类加载副作用。
isReturnDefaultValues = true
}
}
compileOptions { compileOptions {
sourceCompatibility = JavaVersion.VERSION_11 sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11 targetCompatibility = JavaVersion.VERSION_11
@@ -53,4 +62,6 @@ android {
dependencies { dependencies {
// 阶段 1 刻意零第三方 UI 依赖:纯 View 代码布局,减少变量。 // 阶段 1 刻意零第三方 UI 依赖:纯 View 代码布局,减少变量。
// AndroidX 在阶段 2 做正式 UI 时再引入。 // AndroidX 在阶段 2 做正式 UI 时再引入。
// 3a/U7(Q3=A 批准):仅 JUnit 测试作用域,不进 APK。
testImplementation("junit:junit:4.13.2")
} }
@@ -11,6 +11,7 @@
android:name=".MainActivity" android:name=".MainActivity"
android:exported="true" android:exported="true"
android:keepScreenOn="true" android:keepScreenOn="true"
android:windowSoftInputMode="adjustPan"
android:configChanges="orientation|screenSize|keyboardHidden"> android:configChanges="orientation|screenSize|keyboardHidden">
<intent-filter> <intent-filter>
<action android:name="android.intent.action.MAIN" /> <action android:name="android.intent.action.MAIN" />
@@ -0,0 +1,38 @@
package com.verify.remote
/**
* 连接失败分类 → 用户可执行的中文提示(阶段 3a,U8/N6 闭环)。
*
* 分类在 Rust 侧做(`viewer_core::ConnectErrorKind`),这里只做编号→文案映射。
* 编号契约与 Rust 的 `code()` 一一对应(`android-ffi/src/lib.rs` 文件头);
* Kotlin 不 import Rust 类型,跨 FFI 只传整数,所以本对象是纯逻辑、可 JVM 单测。
*
* 文案纪律:每条都必须**告诉用户下一步做什么**(方案 §3.4:不能停在「连接中」),
* 原始英文错误由 UI 作第二行展示(排障用,两者不互相替代)。
*/
object ConnectErrorText {
/** FFI 编号。与 viewer_core::ConnectErrorKind::code 同步,改动必须两侧一起改 */
const val KIND_NONE = 0
const val KIND_FFI_SELF = -1 // FFI 层自身错误(参数/panic/runtime),无业务分类
const val KIND_RESOLVE = 1
const val KIND_TIMEOUT = 2
const val KIND_REFUSED = 3
const val KIND_AUTH = 4
const val KIND_VERSION = 5
const val KIND_OTHER = 9
/** kind 未知(编号漂移)时返回 null,调用方必须落到带原始错误的兜底文案,不得静默 */
fun summary(kind: Int): String? = when (kind) {
KIND_RESOLVE -> "无法解析这个地址:检查 IP 是否写对"
KIND_TIMEOUT -> "连接超时:被控端不在线或入站被拦(检查 agent 是否在跑;本机联调需防火墙放行 47800-47810)"
KIND_REFUSED -> "被控端应答异常:确认这个端口上跑的是 agent"
KIND_AUTH -> "验证码错误:请向被控端核对验证码"
KIND_VERSION -> "协议版本不一致:两端都要更新到同一版本"
KIND_FFI_SELF, KIND_OTHER -> "连接失败"
else -> null
}
/** 完整首行:有分类文案用分类,未知编号不猜语义 */
fun firstLine(kind: Int): String =
summary(kind) ?: "连接失败(未知错误类型 $kind)"
}
@@ -0,0 +1,50 @@
package com.verify.remote
/**
* 光标标记的**纯逻辑**:把协议帧头里的远端光标坐标(采集画面内物理像素)映射到
* 手机视图上的绘制位置。抽成纯 Kotlin 是为了能被 JVM 单测钉死(V-D1 判据依赖它)。
*
* 为什么映射就是简单的等比缩放:[RemoteSurfaceView] 是 letterbox——视图宽高比恒等于
* 远端桌面宽高比(`videoAspectRatio = remoteW / remoteH`),且视图左上角对应桌面 (0,0)。
* 于是「视图内像素 ↔ 桌面像素」是一个线性等比,与 [RelPointer.toDesktopDelta] 用的是
* **同一套比例**(那里是反向:视图像素增量→桌面像素增量)。
*
* 这里的边界校验**不是**对被控端 `fold_cursor` 的重复防御:那条在 Windows 被控端,
* 而本协议把对端字节一律按**不可信输入**处理(见 viewer-core 纪律、[RustBridge.nextFrame] 的长度闸)。
* 跨信任边界的输入必须自证——尤其 Linux 后端当前恒发 None,未来实装时若折叠写错,
* 这里不兜住就会把标记画到画面外。
*/
object CursorMarker {
/** 映射后的绘制点(视图内浮点像素)。绘制侧([CursorOverlayView])按此画圆环中心。 */
data class OnView(val x: Float, val y: Float)
/**
* @param desktopW/desktopH 远端采集画面尺寸(物理像素,来自 AuthOk/desktopSize)
* @param cursor 帧头光标坐标(桌面物理像素),null = 隐藏/离屏/未实现
* @param viewW/viewH SurfaceView 当前测量尺寸(像素)
* @return 视图内绘制点;任一维度非法或坐标越界 → null(不画标记)
*
* null 的所有分支都必须走「不画」而不是「画在 (0,0)」:(0,0) 是一个**合法**的光标位置
* (左上角),把它当成默认值会让「没有光标」和「光标在左上角」看起来一样——
* 这正是协议用 Option 而不用 0 的原因,Kotlin 侧不能把这层区分抹平。
*/
fun mapToView(
desktopW: Int,
desktopH: Int,
cursor: Pair<Int, Int>?,
viewW: Int,
viewH: Int,
): OnView? {
if (cursor == null) return null
// 尺寸未就绪(视图尚未测量 / desktopSize 还没取到):不画,而不是按 0 除
if (desktopW <= 0 || desktopH <= 0 || viewW <= 0 || viewH <= 0) return null
val (cx, cy) = cursor
// 采集画面 [0,w)×[0,h) 之外不画(越界 = 后端折叠漏网或被篡改)
if (cx < 0 || cy < 0 || cx >= desktopW || cy >= desktopH) return null
return OnView(
x = cx.toFloat() * viewW.toFloat() / desktopW.toFloat(),
y = cy.toFloat() * viewH.toFloat() / desktopH.toFloat(),
)
}
}
@@ -0,0 +1,95 @@
package com.verify.remote
import android.content.Context
import android.graphics.Canvas
import android.graphics.Color
import android.graphics.Paint
import android.view.View
/**
* 远端光标标记层(4b/Q1=A、Q2=A、Q6=A):画在 [RemoteSurfaceView] **之上**的透明兄弟视图。
*
* 为什么是兄弟视图而不是 SurfaceView 自己画:SurfaceView 的内容合成在窗口之下
* (窗口在其位置打透明洞),同一视图内无法给自己的 surface 叠加绘制。兄弟 View 盖在
* 洞上方是标准做法(视频播放器控件同理)。[RemoteSurfaceView] 保持纯采集/触摸职责不变。
*
* 坐标换算全部委托 [CursorMarker.mapToView](纯逻辑、JVM 单测钉死);本类只负责:
* 接收解码线程写来的光标 → invalidate → 在 UI 线程把「桌面坐标」画成「屏幕坐标」。
* 锚点几何(surfaceView 的 x/y/width/height)在 onDraw 现取——旋转/重布局后自动跟新,
* 无需订阅布局回调。
*
* 线程纪律:[updateCursor] 由**解码线程**每帧调用(与 stopDecoding 同款的跨线程写入),
* 字段 volatile + [postInvalidate](安卓文档指定的跨线程失效 API)。绘制侧只在 UI 线程读。
*
* 触摸:本视图**不消费任何触摸**(不可点击、不可聚焦)——它覆盖整个 stage,若吃掉事件
* 会同时废掉远端操作和黑边唤出(R-7/4a 的判据会当场全红)。事件按 FrameLayout 规则
* 下沉到 SurfaceView,行为与 4a 完全一致。
*/
class CursorOverlayView(context: Context) : View(context) {
/** 标记直径 22dp(Q2=A)。横屏 480dpi 下 ≈66px ≈ 桌面 33px 等比,够看清又不挡目标。 */
private val markerRadiusPx = 11f * context.resources.displayMetrics.density
/**
* 中心点兼作 proof 颜色特征码(青色非纯白):真机判据 V-D1 用 screencap 找这个
* 颜色的像素簇并断言簇心 == 期望映射点。覆盖层绘制发生在解码**之后**的 UI 层,
* 不经过 H.264 有损链路,screencap 里颜色精确可匹配。圆环仍按 Q2 双描边。
*/
private val dotColor = Color.rgb(0x00, 0xE5, 0xFF)
private val ringPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.STROKE
color = Color.WHITE
strokeWidth = 2f * context.resources.displayMetrics.density
}
private val outerRingPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.STROKE
color = Color.BLACK
strokeWidth = 4f * context.resources.displayMetrics.density
}
private val dotPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.FILL
color = dotColor
}
/**
* 当前帧头光标(桌面物理像素)。null = 无标记:还没收到带光标的帧、
* 指针隐藏/离屏、或被控端后端未实现——三种情况都必须**什么都不画**。
*/
@Volatile
private var cursor: Pair<Int, Int>? = null
/** 解码线程调用;null 也走这里(清除标记同样必须触发重绘,否则标记会留在原位)。 */
fun updateCursor(newCursor: Pair<Int, Int>?) {
cursor = newCursor
postInvalidate()
}
/** 由 Activity 接线:几何锚点就是 SurfaceView(letterbox 后的实际画面矩形)。 */
var anchor: RemoteSurfaceView? = null
init {
isClickable = false
isFocusable = false
}
override fun onDraw(canvas: Canvas) {
super.onDraw(canvas)
val a = anchor ?: return
val c = cursor ?: return
val p = CursorMarker.mapToView(
desktopW = a.remoteWidth,
desktopH = a.remoteHeight,
cursor = c,
viewW = a.width,
viewH = a.height,
) ?: return
// 视图内坐标 + 画面在 stage 里的原点偏移(黑边存在时不为 0)
val sx = a.x + p.x
val sy = a.y + p.y
// 双描边:先粗黑再细白——白环两侧各露 1dp 黑边,深浅背景都保底可见(Q2)
canvas.drawCircle(sx, sy, markerRadiusPx, outerRingPaint)
canvas.drawCircle(sx, sy, markerRadiusPx, ringPaint)
canvas.drawCircle(sx, sy, markerRadiusPx * 0.24f, dotPaint)
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,60 @@
package com.verify.remote
/** 4a:指针方向策略(Q1=A:默认 AUTO)。 */
enum class OrientationPolicy { AUTO, LANDSCAPE, PORTRAIT, SYSTEM }
/**
* 方向策略 → `Activity.requestedOrientation` 的纯映射(4a)。
*
* 抽成纯 Kotlin 的原因(与 WinScancode/ConnectErrorText 同一模式):
* 映射表必须能在 JVM 单测里钉死,而 `ActivityInfo` 常量在 android.jar 桩上
* 引用类加载有坑,这里直接落整数常量。
*
* 数值来源(不凭记忆,2026-11-01 `javap -constants` 对 android-34 android.jar 核实):
* - SCREEN_ORIENTATION_LANDSCAPE = 0
* - SCREEN_ORIENTATION_PORTRAIT = 1
* - SCREEN_ORIENTATION_UNSPECIFIED = -1
* 刻意只用这三个:探针在真机验证过 LANDSCAPE(0) 与 PORTRAIT 行为;
* SENSOR_LANDSCAPE(6)(允许左右横)未在本机验证过,不引入——固定横屏对
* 「看 16:9 桌面」的场景足够,翻转手机方向时系统不会再把画面翻到另一侧,
* 这反而是更稳定的行为。
*/
object Orientation {
const val REQUEST_LANDSCAPE = 0
const val REQUEST_PORTRAIT = 1
const val REQUEST_UNSPECIFIED = -1
/**
* AUTO 的语义:远端画面是横的(宽>高)就用横屏,否则竖屏。
* 尺寸未知(0,连接前/断开后)按竖屏兜底——表单页本来就是竖屏。
*/
fun requestFor(policy: OrientationPolicy, remoteW: Int, remoteH: Int): Int = when (policy) {
OrientationPolicy.AUTO ->
if (remoteW > remoteH) REQUEST_LANDSCAPE else REQUEST_PORTRAIT
OrientationPolicy.LANDSCAPE -> REQUEST_LANDSCAPE
OrientationPolicy.PORTRAIT -> REQUEST_PORTRAIT
OrientationPolicy.SYSTEM -> REQUEST_UNSPECIFIED
}
/** 方向按钮的循环次序(Q4=A):AUTO→横→竖→系统→AUTO。 */
fun next(policy: OrientationPolicy): OrientationPolicy = when (policy) {
OrientationPolicy.AUTO -> OrientationPolicy.LANDSCAPE
OrientationPolicy.LANDSCAPE -> OrientationPolicy.PORTRAIT
OrientationPolicy.PORTRAIT -> OrientationPolicy.SYSTEM
OrientationPolicy.SYSTEM -> OrientationPolicy.AUTO
}
/** 按钮文案。每个策略都必须有非空、互不相同的文案(单测锁)。 */
fun label(policy: OrientationPolicy): String = when (policy) {
OrientationPolicy.AUTO -> "方向:自动"
OrientationPolicy.LANDSCAPE -> "方向:横屏"
OrientationPolicy.PORTRAIT -> "方向:竖屏"
OrientationPolicy.SYSTEM -> "方向:系统"
}
/** prefs 读写:未知字符串(版本回退/手改)必须落回 AUTO,不得抛。 */
fun fromName(name: String?): OrientationPolicy =
runCatching { OrientationPolicy.valueOf(name.orEmpty().uppercase()) }
.getOrDefault(OrientationPolicy.AUTO)
}
@@ -0,0 +1,119 @@
package com.verify.remote
import kotlin.math.abs
import kotlin.math.roundToInt
/**
* 相对鼠标的**纯逻辑**:把「手指在视图上移动了多少像素」换算成「远端指针该走多少桌面像素」,
* 并做小数累积取整。抽成纯 Kotlin 是为了能在 JVM 单测里验证(V-B1 的判据依赖它)。
*
* 为什么不直接发 `deltaViewPx`:
* - 视图尺寸 ≠ 桌面尺寸(letterbox 后视图是 1080x607,桌面是 1920x1080);
* - `sensitivity != 1.0` 时还要缩放。
* 两步合并成一次乘法,避免分两次取整(每多一次取整就多一个累积误差源)。
*
* 为什么不直接 `toInt()` 就发:
* 手指慢速滑动时每次 `ACTION_MOVE` 的增量可能只有 0.3 个桌面像素,`toInt()` 逐次截断会让
* **位移被静默吞掉**(V-B1「慢速小幅滑动不丢位移」直接不成立)。必须把余数累积到下一次。
*/
object RelPointer {
/** 灵敏度允许范围(Q4=A:默认 1.0,可调 0.25–3.0) */
const val MIN_SENSITIVITY = 0.25f
const val MAX_SENSITIVITY = 3.0f
const val DEFAULT_SENSITIVITY = 1.0f
fun clampSensitivity(v: Float): Float = v.coerceIn(MIN_SENSITIVITY, MAX_SENSITIVITY)
/**
* 视图像素增量 → 桌面像素增量(**浮点,未取整**)。
*
* `sensitivity = 1.0` 的语义:手指扫过视图整个宽度 = 指针走完整个桌面宽度。
* 这样相对模式与绝对模式的手感一致(绝对模式下 10% 视图宽 = 10% 桌面宽)。
*
* 任一维度非法(<=0,例如视图尚未测量、远端尺寸未取到)时返回 0:
* 宁可不移动,也不要按错误比例乱飞。
*/
fun toDesktopDelta(
viewDeltaPx: Float,
viewExtentPx: Int,
remoteExtentPx: Int,
sensitivity: Float,
): Float {
if (viewExtentPx <= 0 || remoteExtentPx <= 0) return 0f
if (viewDeltaPx == 0f) return 0f
val scale = remoteExtentPx.toFloat() / viewExtentPx.toFloat()
return viewDeltaPx * scale * clampSensitivity(sensitivity)
}
}
/**
* 小数累积器:把连续的浮点位移攒成整数像素再发送,余数留到下一次。
*
* 保证:连续输入之和与单独一次输入等价(不丢位移);每次发出的都是整数(协议是 i16)。
* 撞边等外部原因丢弃的位移**不回补**——累积器只负责「取整」,不负责「补偿」,
* 否则撞边后回滑会出现一段"空转"(V-B1 第三条判据要求撞边后立即响应)。
*/
class RelAccumulator {
private var accX = 0f
private var accY = 0f
/** 会话/手势结束时清空,避免上一段手势的余数漏到下一段 */
fun reset() {
accX = 0f
accY = 0f
}
/**
* 累积一段位移,返回**本次可以发送的整数增量**;不足 1 像素时返回 null(不发)。
*
* X/Y 独立累积:斜向慢速滑动时两个轴各自攒够才发,不会因为一个轴没到 1 就把另一个轴也拖住。
*/
fun take(dxDesktop: Float, dyDesktop: Float): Pair<Int, Int>? {
accX += dxDesktop
accY += dyDesktop
// roundToInt 而非 toInt:-0.7 应取 -1 而不是 0(向零截断会让反向慢滑丢位移)
val ix = accX.roundToInt()
val iy = accY.roundToInt()
if (ix == 0 && iy == 0) return null
accX -= ix
accY -= iy
// 协议是 i16;单次事件不可能接近这个范围,但显式 clamp 让越界行为有界
return ix.coerceIn(-32768, 32767) to iy.coerceIn(-32768, 32767)
}
}
/**
* 长按右键的判定(Q5=A:≥500ms 且位移 < slop)。
*
* 抽成纯逻辑的原因:真实触发依赖 Handler 计时与 MotionEvent,JVM 里不可构造;
* 但「什么条件下该弹右键」是纯判断,必须能被单测钉住(尤其"拖窗口不弹菜单"这条)。
*/
object LongPressPolicy {
/** 长按触发阈值(毫秒) */
const val LONG_PRESS_MS = 500L
/**
* 是否应触发长按右键。
*
* @param elapsedMs 从 ACTION_DOWN 起经过的时间
* @param movedPx 期间的最大位移(欧氏距离)
* @param slopPx 拖动阈值(12dp),超过即视为拖拽而非长按
* @param twoFinger 是否出现过第二根手指(双指手势有自己的右键路径,不能叠加)
*
* 三条都必须满足:够久、没怎么动、不是双指。
* 「拖窗口不弹菜单」靠 movedPx 这条——超过 slop 时 [RemoteSurfaceView] 会切到 DRAG
* 并取消计时任务,所以这里通常压根不会被调用;但作为纯逻辑仍要自洽。
*/
fun shouldFire(
elapsedMs: Long,
movedPx: Float,
slopPx: Float,
twoFinger: Boolean,
): Boolean {
if (twoFinger) return false
if (elapsedMs < LONG_PRESS_MS) return false
return abs(movedPx) < slopPx
}
}
@@ -0,0 +1,154 @@
package com.verify.remote
import android.view.KeyEvent
import android.view.View
import android.view.inputmethod.BaseInputConnection
import android.view.inputmethod.EditorInfo
/** 单次退格请求上限:防 IME 异常大值刷爆消息队列(与 agent 侧 WHEEL_MAX_CLICKS 同思路) */
const val MAX_BACKSPACE = 64
/**
* 退格决策(纯函数 [planBackspaces] 的输出)。类型刻意放在类外:
* `RemoteInputConnection` 构造链到 `BaseInputConnection`(android.jar 桩,JVM 单测
* 里抛 "not mocked"),决策本身无 Android 依赖——抽出来才能被 3a 的 JVM 单测覆盖(U7)。
*/
sealed class BackspaceAction {
/** (0,0):IME 没有要删的东西 */
object Noop : BackspaceAction()
/** 退格 [count] 次(已截断到 [MAX_BACKSPACE]) */
data class Backspaces(val count: Int) : BackspaceAction()
/** `after>0` / 双向删除:本地无远端真值,不可执行(U3) */
data class Unsupported(val before: Int, val after: Int) : BackspaceAction()
}
/**
* `deleteSurroundingText(before, after)` 的决策逻辑,从 [RemoteInputConnection] 抽出。
* 语义与原实现一致(3a 只提取不改动):
* - `(0,0)` → Noop;
* - `after>0` → Unsupported(forward-delete 形态本地无法执行,U3);
* - `before>0 && after==0` → 退格 min(before, [MAX_BACKSPACE]) 次
* (上限防 IME 异常大值刷爆消息队列,与 agent 侧 WHEEL_MAX_CLICKS 同思路)。
*/
fun planBackspaces(before: Int, after: Int, maxBackspace: Int = MAX_BACKSPACE): BackspaceAction =
when {
before == 0 && after == 0 -> BackspaceAction.Noop
after != 0 -> BackspaceAction.Unsupported(before, after)
else -> BackspaceAction.Backspaces(minOf(before, maxBackspace))
}
/**
* P1 IME 通道(loop-2b-plan §5,用户已裁决):**本地没有编辑缓冲,远端才是权威**。
*
* 三条约定(与方案一致):
* 1. `commitText` = 用户确认的整段文本 → 直接走协议 v3 `Text` 通道(布局无关、支持中文);
* 2. `sendKeyEvent` = 硬键盘/控制键 → 既有扫描码通道(Enter/Backspace/方向键…);
* 3. `deleteSurroundingText`:本地不知道远端内容,`before>0 && after==0`
* 是 IME 退格最常见形态 → 发对应次数的 Backspace;其余形态记录后放弃(U3,
* 决策逻辑抽到纯函数 [planBackspaces],3a 起可 JVM 单测)。
*
* composing(拼音未确认串)不显示到远端(U4):`setComposingText` 吞掉但返回 true,
* 让 IME 维持本地组合状态,候选确认后仍会走 `commitText`。
*
* 日志纪律:debug 通道只报**长度与事件类型**,不报用户输入内容(安全红线)。
*/
class RemoteInputConnection(
target: View,
private val sendText: (CharArray) -> Unit,
private val sendKey: (WinKey, Boolean) -> Unit,
private val debug: (String) -> Unit,
) : BaseInputConnection(target, false) {
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
val units = text.toString().toCharArray()
if (units.isEmpty()) return true
sendText(units)
debug("commit ${units.size}u")
return true
}
override fun sendKeyEvent(event: KeyEvent): Boolean {
if (event.action == KeyEvent.ACTION_MULTIPLE) {
val chars: String? = event.characters
if (!chars.isNullOrEmpty()) {
sendText(chars.toCharArray())
debug("multiple ${chars.length}u")
}
return true
}
// 字符事件优先于扫描码:多数软键盘发的是 keyCode=KEYCODE_UNKNOWN +
// unicodeChar(adb `input text` 同款形态)。这一分支必须排在 WinScancode
// 查表**之前**——否则 UNKNOWN(0) 查不到表会被当「未映射」吞掉,字就丢了。
// 注意 `getUnicodeChar()` 返回的是单个 UTF-16 code unit(不是码点),
// 所以这条路只覆盖 BMP;增补平面字符由 `commitText` 成对送达。
// 只收**可打印**字符(空格起、排除 0x7F):控制值(Enter/Tab)必须走扫描码,
// 远端应用期待的是按键事件而非 WM_CHAR。见 MainActivity.onKeyDown 同款门槛。
val cp = event.unicodeChar
if (event.action == KeyEvent.ACTION_DOWN && cp >= 0x20 && cp != 0x7F) {
sendText(charArrayOf(cp.toChar()))
debug("keychar 1u")
return true
}
val key = WinScancode.of(event.keyCode)
if (key == null) {
debug("IME 未映射键 code=${event.keyCode}")
return false
}
sendKey(key, event.action == KeyEvent.ACTION_DOWN)
return true
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean =
applyBackspaces(planBackspaces(beforeLength, afterLength))
override fun deleteSurroundingTextInCodePoints(beforeLength: Int, afterLength: Int): Boolean =
applyBackspaces(planBackspaces(beforeLength, afterLength))
/** 决策 → 注入。日志纪律:只报数量与形态,不报内容。 */
private fun applyBackspaces(action: BackspaceAction): Boolean = when (action) {
is BackspaceAction.Noop -> true
is BackspaceAction.Unsupported -> {
// U3:forward-delete 与「先删 after 再删 before」的形态本地无远端真值可依
debug("delete(${action.before},${action.after}):after>0 形态未支持")
false
}
is BackspaceAction.Backspaces -> {
val bk = WinScancode.of(KeyEvent.KEYCODE_DEL) ?: return false
repeat(action.count) {
sendKey(bk, true)
sendKey(bk, false)
}
debug("backspace x${action.count}")
true
}
}
/** 吞掉 composing 但返回 true:IME 保持本地组合流程,候选确认仍会 commitText。 */
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean = true
override fun setComposingRegion(start: Int, end: Int): Boolean = true
override fun finishComposingText(): Boolean = true
// 本地无可信缓冲区(远端才是权威),一律回空串——比 null 温和,多数 IME 兼容。
override fun getTextBeforeCursor(n: Int, flags: Int): CharSequence = ""
override fun getTextAfterCursor(n: Int, flags: Int): CharSequence = ""
override fun getSelectedText(flags: Int): CharSequence = ""
override fun getCursorCapsMode(reqModes: Int): Int = 0
override fun performEditorAction(actionCode: Int): Boolean {
if (actionCode == EditorInfo.IME_ACTION_DONE) {
// 软键盘「完成/回车」在远控语义里就是 Enter
WinScancode.of(KeyEvent.KEYCODE_ENTER)?.let {
sendKey(it, true)
sendKey(it, false)
}
debug("IME_ACTION_DONE→Enter")
return true
}
debug("editorAction $actionCode 忽略")
return true
}
}
@@ -1,13 +1,19 @@
package com.verify.remote package com.verify.remote
import android.content.Context import android.content.Context
import android.os.Handler
import android.os.Looper
import android.text.InputType
import android.view.MotionEvent import android.view.MotionEvent
import android.view.SurfaceView import android.view.SurfaceView
import android.view.inputmethod.EditorInfo
import android.view.inputmethod.InputConnection
import kotlin.math.abs import kotlin.math.abs
import kotlin.math.hypot import kotlin.math.hypot
/** /**
* 保持视频宽高比的 SurfaceView(letterbox),并把触摸手势映射为远端鼠标事件。 * 保持视频宽高比的 SurfaceView(letterbox),把触摸手势映射为远端鼠标事件,
* 并作为软键盘的宿主(协议 v3 文本通道 + 扫描码控制键)。
* *
* 为什么视图尺寸必须等于画面尺寸:触摸坐标按视图宽高归一化到远端 0..=65535。 * 为什么视图尺寸必须等于画面尺寸:触摸坐标按视图宽高归一化到远端 0..=65535。
* 若视图被拉伸(阶段 1 用 MATCH_PARENT),画面与视图不再一一对应,手指位置与 * 若视图被拉伸(阶段 1 用 MATCH_PARENT),画面与视图不再一一对应,手指位置与
@@ -17,22 +23,71 @@ import kotlin.math.hypot
* 实现为本项目独立编写): * 实现为本项目独立编写):
* - 单指轻点 = 左键单击 * - 单指轻点 = 左键单击
* - 单指按下并移动 = 左键拖拽(窗口拖动、文字选中) * - 单指按下并移动 = 左键拖拽(窗口拖动、文字选中)
* - 单指长按不动 = 右键(协议 v4 起;双指轻点仍可用)
* - 双指轻点 = 右键 * - 双指轻点 = 右键
* - 双指上下滑动 = 滚轮 * - 双指上下滑动 = 滚轮
* *
* 刻意不做「手指即光标」的绝对定位:手指会完全挡住目标,桌面精细操作不可用。 * **两种指针模式(协议 v4)**:
* - 相对模式(默认):手指滑动 = 远端指针按**增量**移动,手指只扫一小段就能走很远,
* 且不遮挡目标。DOWN 不发送绝对移动——轻点落在**远端指针当前位置**。
* - 绝对模式:手指位置直接映射到远端坐标(2a/3a 行为,DOWN 先发绝对 `MouseMove`)。
* 两种模式由 [relativeMode] 切换;R7 绝对触摸回归判据依赖绝对模式仍可用。
*
* **IME 接线(阶段 2b)**:本视图是文本编辑器(`onCheckIsTextEditor`),
* 软键盘的提交经 `RemoteInputConnection` 上行。`isFocusableInTouchMode` 必须为
* true,否则触摸态拿不到焦点、系统不建立 InputConnection(阶段 2a 只有
* `isFocusable`,这就是当时软键盘弹不出来的原因之一)。
*/ */
class RemoteSurfaceView(context: Context) : SurfaceView(context) { class RemoteSurfaceView(context: Context) : SurfaceView(context) {
/** 由 Activity 实现,转发到 RustBridge */ /** 由 Activity 实现,转发到 RustBridge */
interface Listener { interface Listener {
fun sendMouseMove(x: Int, y: Int) fun sendMouseMove(x: Int, y: Int)
/** 协议 v4:相对位移(被控端桌面物理像素) */
fun sendMouseRel(dx: Int, dy: Int)
fun sendMouseButton(button: Int, down: Boolean) fun sendMouseButton(button: Int, down: Boolean)
fun sendWheel(dy: Int) fun sendWheel(dy: Int)
/** 协议 v3:一段 UTF-16 code unit(IME 提交结果) */
fun sendText(units: CharArray)
/** 控制键(Enter/Backspace/方向键等)走扫描码通道 */
fun sendKey(key: WinKey, down: Boolean)
/** 连接态:只有已连接才声明自己是文本编辑器,避免未连接时误弹键盘 */
fun isReadyForInput(): Boolean
/**
* 右键注入发生了(3b)。用于把「长按是否真的发了右键」变成**可持久观测**的状态。
*
* 为什么需要它:`onDebug` 是覆盖式的一行文本,长按后紧跟着的 ACTION_UP 会
* 立刻把它改写成「长按结束」——真机验收脚本因此看不到「长按右键」,
* 判据被后续事件冲掉。计数器不会丢。
*/
fun onRightClickInjected(source: String)
} }
var listener: Listener? = null var listener: Listener? = null
/**
* 相对指针模式(协议 v4)。默认 true:桌面远控的通行形态,手指不遮目标。
* 切换时清空累积器——上一模式的余数不该漏进新模式。
*/
var relativeMode: Boolean = true
set(value) {
if (field != value) {
field = value
relAcc.reset()
}
}
/** 灵敏度(仅相对模式生效)。默认 1.0 = 手指扫过视图全宽,指针走完桌面全宽。 */
var sensitivity: Float = RelPointer.DEFAULT_SENSITIVITY
set(value) {
field = RelPointer.clampSensitivity(value)
relAcc.reset()
}
/** 远端桌面像素尺寸(相对模式换算用);未连接时为 0,此时相对移动不发送 */
var remoteWidth: Int = 0
var remoteHeight: Int = 0
/** 视频宽高比(w/h)。>0 时视图按此比例测量。 */ /** 视频宽高比(w/h)。>0 时视图按此比例测量。 */
var videoAspectRatio = 0f var videoAspectRatio = 0f
set(value) { set(value) {
@@ -55,11 +110,71 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
private var twoFingerMoved = false private var twoFingerMoved = false
private var wheelAccum = 0f private var wheelAccum = 0f
/** 相对模式的小数累积器(慢速滑动不丢位移) */
private val relAcc = RelAccumulator()
/** 长按右键:计时任务 + 是否已消费本次手势的 UP(防「先右键再左键」) */
private val ui = Handler(Looper.getMainLooper())
private var longPressFired = false
private var longPressTwoFinger = false
/** 判定「拖拽 vs 点击」的位移阈值:12dp */ /** 判定「拖拽 vs 点击」的位移阈值:12dp */
private val dragSlopPx = resources.displayMetrics.density * 12f private val dragSlopPx = resources.displayMetrics.density * 12f
init { init {
isFocusable = true isFocusable = true
// IME 需要触摸态焦点;没有这行,点「键盘」按钮系统不会建立输入连接
isFocusableInTouchMode = true
}
/**
* 长按计时任务(Q5=A:≥500ms 且位移 < slop)。
* 命名刻意避开 `cancelLongPress`——`View` 已有同名方法,会被要求 override。
*/
private val longPressTask = Runnable {
val l = listener ?: return@Runnable
if (mode != Mode.TAP) return@Runnable // 已转拖拽/双指
val elapsed = System.currentTimeMillis() - downTime
val movedPx = hypot((lastX - downX).toDouble(), (lastY - downY).toDouble()).toFloat()
if (!LongPressPolicy.shouldFire(elapsed, movedPx, dragSlopPx, longPressTwoFinger)) {
return@Runnable
}
// 相对模式:在远端指针**当前位置**点右键,不发绝对移动
if (!relativeMode) l.sendMouseMove(normX(lastX), normY(lastY))
l.sendMouseButton(BTN_RIGHT, true)
l.sendMouseButton(BTN_RIGHT, false)
longPressFired = true
performClick()
onDebug?.invoke("长按右键")
listener?.onRightClickInjected("长按")
}
private fun cancelLongPressTimer() {
ui.removeCallbacks(longPressTask)
}
// ---- IME 通道 ----
override fun onCheckIsTextEditor(): Boolean = listener?.isReadyForInput() == true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
val l = listener ?: return null
if (!l.isReadyForInput()) return null
// TYPE_CLASS_TEXT:让输入法给出正常键盘(含中文候选)。
// 常量从 InputType 引用而非 EditorInfo:Kotlin 不保证经实现类名访问接口常量。
// 刻意不加 TYPE_TEXT_FLAG_NO_SUGGESTIONS——中文输入依赖候选词。
outAttrs.inputType = InputType.TYPE_CLASS_TEXT
// 抑制全屏/extract 模式抢走画面(API 29 上两个标志都还在,具体效果真机验证,见 U4/U5)
outAttrs.imeOptions =
EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
// 右下动作键文案。字段名是 actionLabel(javap android-34 核实,不是 imeActionLabel)
outAttrs.actionLabel = "发送"
return RemoteInputConnection(
target = this,
sendText = { l.sendText(it) },
sendKey = { k, down -> l.sendKey(k, down) },
debug = { msg -> onDebug?.invoke(msg) },
)
} }
/** /**
@@ -94,6 +209,25 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
private fun normX(x: Float) = (x / width * 65535).toInt().coerceIn(0, 65535) private fun normX(x: Float) = (x / width * 65535).toInt().coerceIn(0, 65535)
private fun normY(y: Float) = (y / height * 65535).toInt().coerceIn(0, 65535) private fun normY(y: Float) = (y / height * 65535).toInt().coerceIn(0, 65535)
/**
* 相对模式下把视图像素增量发出去(经累积取整)。
*
* 为什么 TAP/DRAG 都走这一个函数:两个分支都必须用**同一个**累积器,
* 分成两套会让 TAP→DRAG 切换时丢一次余数。
*/
private fun sendRelative(l: Listener, fromX: Float, fromY: Float, toX: Float, toY: Float) {
val dx = RelPointer.toDesktopDelta(toX - fromX, width, remoteWidth, sensitivity)
val dy = RelPointer.toDesktopDelta(toY - fromY, height, remoteHeight, sensitivity)
if (dx == 0f && dy == 0f) return
val d = relAcc.take(dx, dy) ?: return
l.sendMouseRel(d.first, d.second)
}
/** 绝对模式下按当前手指位置发归一化坐标 */
private fun sendAbsolute(l: Listener, x: Float, y: Float) {
l.sendMouseMove(normX(x), normY(y))
}
private fun moved(fromX: Float, fromY: Float, toX: Float, toY: Float) = private fun moved(fromX: Float, fromY: Float, toX: Float, toY: Float) =
hypot((toX - fromX).toDouble(), (toY - fromY).toDouble()) > dragSlopPx hypot((toX - fromX).toDouble(), (toY - fromY).toDouble()) > dragSlopPx
@@ -103,28 +237,37 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
private fun midY(e: MotionEvent) = private fun midY(e: MotionEvent) =
if (e.pointerCount >= 2) (e.getY(0) + e.getY(1)) / 2f else e.y if (e.pointerCount >= 2) (e.getY(0) + e.getY(1)) / 2f else e.y
/** 诊断回调:把触摸情况显示到状态栏(本机 logcat 被系统限流,日志不可靠) */ /**
* 诊断回调:把触摸情况显示到状态栏(本机 logcat 被系统限流,日志不可靠)。
*
* 3a 降噪(V-A5):只显示**事件语义**(单击/拖拽/右键/滚轮),不再显示
* 坐标与归一化值——那 20 列噪声挤满状态栏,且对操作者没有决策价值
* (坐标类验证由 proof 脚本在 PC 侧读光标闭环,不经屏幕)。
*/
var onDebug: ((String) -> Unit)? = null var onDebug: ((String) -> Unit)? = null
/**
* 4a/Q3=A:任何触摸都回调一次(Activity 用它重置自动隐藏计时)。
*
* 唤出区刻意**不在本视图内**(原底部 40dp 边缘带的设计已废弃):那条带会吃掉
* 远端桌面底部约 11% 的高度——Windows 任务栏就在那里,点任务栏会变成唤出菜单
* 而不是点击任务栏(2026-11-01 R7 回归实测暴露:frac=0.9 的 tap 被边缘带吞掉)。
* 改为由 Activity 在 stage 上消费「画面黑边」的触摸:黑边不属于本视图,
* 天然与远端操作零冲突。
*/
var onAnyTouch: (() -> Unit)? = null
override fun dispatchTouchEvent(event: MotionEvent): Boolean {
onAnyTouch?.invoke()
return super.dispatchTouchEvent(event)
}
override fun onTouchEvent(event: MotionEvent): Boolean { override fun onTouchEvent(event: MotionEvent): Boolean {
val l = listener ?: run { val l = listener ?: run {
onDebug?.invoke("touch: listener=null") onDebug?.invoke("触摸(未接线)")
return false return false
} }
if (width == 0 || height == 0) return false if (width == 0 || height == 0) return false
val act = when (event.actionMasked) {
MotionEvent.ACTION_DOWN -> "DOWN"
MotionEvent.ACTION_MOVE -> "MOVE"
MotionEvent.ACTION_UP -> "UP"
MotionEvent.ACTION_POINTER_DOWN -> "P2DOWN"
MotionEvent.ACTION_POINTER_UP -> "P2UP"
MotionEvent.ACTION_CANCEL -> "CANCEL"
else -> event.actionMasked.toString()
}
onDebug?.invoke(
"$act view=(${event.x.toInt()},${event.y.toInt()}) " +
"norm=(${normX(event.x)},${normY(event.y)}) ${width}x$height"
)
when (event.actionMasked) { when (event.actionMasked) {
MotionEvent.ACTION_DOWN -> { MotionEvent.ACTION_DOWN -> {
@@ -132,8 +275,16 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
lastX = event.x; lastY = event.y lastX = event.x; lastY = event.y
downTime = System.currentTimeMillis() downTime = System.currentTimeMillis()
mode = Mode.TAP mode = Mode.TAP
// 先把远端指针移到位,点击/拖拽都以此为基准 longPressFired = false
l.sendMouseMove(normX(event.x), normY(event.y)) longPressTwoFinger = false
relAcc.reset()
// 相对模式:**不发**绝对移动。手指落下不该让远端指针跳到自己脚下
// (那正是相对鼠标要解决的问题);轻点落在指针当前位置。
if (!relativeMode) sendAbsolute(l, event.x, event.y)
// 长按右键计时(Q5=A:500ms 且不动)
cancelLongPressTimer()
ui.postDelayed(longPressTask, LongPressPolicy.LONG_PRESS_MS)
onDebug?.invoke("按下")
} }
MotionEvent.ACTION_POINTER_DOWN -> { MotionEvent.ACTION_POINTER_DOWN -> {
@@ -141,24 +292,40 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
// 必须先抬起,否则远端会留下一个被按住未释放的左键。 // 必须先抬起,否则远端会留下一个被按住未释放的左键。
if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false) if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false)
mode = Mode.TWO_FINGER mode = Mode.TWO_FINGER
longPressTwoFinger = true
cancelLongPressTimer()
twoFingerMoved = false twoFingerMoved = false
wheelAccum = 0f wheelAccum = 0f
twoFingerStartY = midY(event) twoFingerStartY = midY(event)
lastX = midX(event); lastY = midY(event) lastX = midX(event); lastY = midY(event)
twoFingerDownTime = System.currentTimeMillis() twoFingerDownTime = System.currentTimeMillis()
onDebug?.invoke("双指按下")
} }
MotionEvent.ACTION_MOVE -> when (mode) { MotionEvent.ACTION_MOVE -> when (mode) {
Mode.TAP -> if (moved(downX, downY, event.x, event.y)) { Mode.TAP -> if (moved(downX, downY, event.x, event.y)) {
// 超过 slop = 拖拽,长按不再可能(先取消计时,避免拖到一半弹菜单)
cancelLongPressTimer()
mode = Mode.DRAG mode = Mode.DRAG
l.sendMouseButton(BTN_LEFT, true) l.sendMouseButton(BTN_LEFT, true)
if (relativeMode) {
// 相对拖拽:从落点开始按增量移动(不先跳到手指位置)
relAcc.reset()
sendRelative(l, downX, downY, event.x, event.y)
} else {
sendAbsolute(l, event.x, event.y)
}
lastX = event.x; lastY = event.y lastX = event.x; lastY = event.y
l.sendMouseMove(normX(event.x), normY(event.y)) onDebug?.invoke("拖拽中")
} }
Mode.DRAG -> { Mode.DRAG -> {
if (relativeMode) {
sendRelative(l, lastX, lastY, event.x, event.y)
} else {
sendAbsolute(l, event.x, event.y)
}
lastX = event.x; lastY = event.y lastX = event.x; lastY = event.y
l.sendMouseMove(normX(event.x), normY(event.y))
} }
Mode.TWO_FINGER -> { Mode.TWO_FINGER -> {
@@ -172,6 +339,7 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
if (notches != 0) { if (notches != 0) {
l.sendWheel(-notches * WHEEL_DELTA) l.sendWheel(-notches * WHEEL_DELTA)
wheelAccum -= notches * WHEEL_PX_PER_NOTCH wheelAccum -= notches * WHEEL_PX_PER_NOTCH
onDebug?.invoke(if (notches > 0) "滚轮↓" else "滚轮↑")
} }
lastX = cx; lastY = cy lastX = cx; lastY = cy
} }
@@ -182,42 +350,65 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
MotionEvent.ACTION_POINTER_UP -> { MotionEvent.ACTION_POINTER_UP -> {
// 双指之一抬起:先判定右键,再把剩余手指的移动视为已结束手势 // 双指之一抬起:先判定右键,再把剩余手指的移动视为已结束手势
if (mode == Mode.TWO_FINGER) { if (mode == Mode.TWO_FINGER) {
maybeRightClick(l) if (maybeRightClick(l)) onDebug?.invoke("右键")
mode = Mode.IDLE mode = Mode.IDLE
} }
} }
MotionEvent.ACTION_UP -> { MotionEvent.ACTION_UP -> {
cancelLongPressTimer()
when (mode) { when (mode) {
Mode.TAP -> { Mode.TAP -> {
// 轻点 = 左键单击(位置已在 DOWN 时发送) if (longPressFired) {
l.sendMouseButton(BTN_LEFT, true) // 长按已弹右键:本次 UP 不能再发左键单击,
l.sendMouseButton(BTN_LEFT, false) // 否则一次长按会「先右键再左键」,右键菜单立刻被左键点掉
performClick() onDebug?.invoke("长按结束")
} else {
// 轻点 = 左键单击。绝对模式位置已在 DOWN 时发送;
// 相对模式点击**远端指针当前位置**,不发任何移动
if (!relativeMode) sendAbsolute(l, lastX, lastY)
l.sendMouseButton(BTN_LEFT, true)
l.sendMouseButton(BTN_LEFT, false)
performClick()
onDebug?.invoke("单击")
}
}
Mode.DRAG -> {
l.sendMouseButton(BTN_LEFT, false)
onDebug?.invoke("拖拽结束")
}
Mode.TWO_FINGER -> {
if (maybeRightClick(l)) onDebug?.invoke("右键")
} }
Mode.DRAG -> l.sendMouseButton(BTN_LEFT, false)
Mode.TWO_FINGER -> maybeRightClick(l)
Mode.IDLE -> {} Mode.IDLE -> {}
} }
mode = Mode.IDLE mode = Mode.IDLE
longPressFired = false
} }
MotionEvent.ACTION_CANCEL -> { MotionEvent.ACTION_CANCEL -> {
cancelLongPressTimer()
if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false) if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false)
mode = Mode.IDLE mode = Mode.IDLE
longPressFired = false
onDebug?.invoke("手势取消")
} }
} }
return true return true
} }
/** 双指未移动且按住时间短 → 右键 */ /** 双指未移动且按住时间短 → 右键。返回是否真的注入了右键(供诊断显示)。 */
private fun maybeRightClick(l: Listener) { private fun maybeRightClick(l: Listener): Boolean {
if (twoFingerMoved) return if (twoFingerMoved) return false
if (System.currentTimeMillis() - twoFingerDownTime > TWO_FINGER_TAP_MAX_MS) return if (System.currentTimeMillis() - twoFingerDownTime > TWO_FINGER_TAP_MAX_MS) return false
l.sendMouseMove(normX(lastX), normY(lastY)) // 双指右键沿用「先定位再点」:双指落点本身就是用户的定位意图,
// 与单指相对模式「不动指针」的语义不同(2a 起就是这个行为,回归判据依赖它)
sendAbsolute(l, lastX, lastY)
l.sendMouseButton(BTN_RIGHT, true) l.sendMouseButton(BTN_RIGHT, true)
l.sendMouseButton(BTN_RIGHT, false) l.sendMouseButton(BTN_RIGHT, false)
performClick() performClick()
listener?.onRightClickInjected("双指")
return true
} }
@Suppress("ClickableViewAccessibility") @Suppress("ClickableViewAccessibility")
@@ -1,5 +1,11 @@
package com.verify.remote package com.verify.remote
/**
* 连接失败:Rust 侧分类([kind],`ConnectErrorText.KIND_*`)+ 原始错误文本([detail])。
* 界面第一行按分类给中文可执行提示,第二行显示 detail(排障用,见 3a 错误显示纪律)。
*/
class ConnectFailure(val kind: Int, val detail: String) : Exception(detail)
/** /**
* viewer-core(Rust)的 JNI 声明。符号约定:Java_com_verify_remote_RustBridge_*, * viewer-core(Rust)的 JNI 声明。符号约定:Java_com_verify_remote_RustBridge_*,
* 与 android-ffi/src/lib.rs 一一对应。 * 与 android-ffi/src/lib.rs 一一对应。
@@ -10,30 +16,51 @@ package com.verify.remote
*/ */
object RustBridge { object RustBridge {
private const val FRAME_HEADER_LEN = 25
init { init {
System.loadLibrary("androidffi") System.loadLibrary("androidffi")
} }
private external fun nativeConnect(host: String, port: Int, code: String): Long private external fun nativeConnect(host: String, port: Int, code: String): Long
external fun nativeLastError(): String? external fun nativeLastError(): String?
/** 返回 null = 会话结束;否则 [FRAME_HEADER_LEN] 字节头 + Annex-B 载荷 */
/**
* 最近一次 nativeConnect 失败的业务分类(`ConnectErrorText.KIND_*` 编号)。
* FFI 契约(android-ffi/src/lib.rs):**必须在 nativeLastError() 之前调用**——
* 消息读走时槽位整体清空,颠倒顺序拿到 kind=0。
*/
external fun nativeLastErrorKind(): Int
/** 返回 null = 会话结束;否则 [FrameCodec.FRAME_HEADER_LEN] 字节头 + Annex-B 载荷 */
external fun nativeNextFrame(handle: Long): ByteArray? external fun nativeNextFrame(handle: Long): ByteArray?
external fun nativeStats(handle: Long): Long external fun nativeStats(handle: Long): Long
external fun nativeDesktopSize(handle: Long): Long external fun nativeDesktopSize(handle: Long): Long
external fun nativeSendMouseMove(handle: Long, x: Int, y: Int) external fun nativeSendMouseMove(handle: Long, x: Int, y: Int)
/**
* 协议 v4:相对移动(单位 = 被控端桌面物理像素)。
* Rust 侧把 Int clamp 到 i16,越界不会回绕;零位移在 viewer-core 被拦掉。
*/
external fun nativeSendMouseRel(handle: Long, dx: Int, dy: Int)
external fun nativeSendMouseButton(handle: Long, button: Int, down: Int) external fun nativeSendMouseButton(handle: Long, button: Int, down: Int)
external fun nativeSendWheel(handle: Long, dy: Int) external fun nativeSendWheel(handle: Long, dy: Int)
external fun nativeSendKey(handle: Long, scancode: Int, extended: Int, down: Int) external fun nativeSendKey(handle: Long, scancode: Int, extended: Int, down: Int)
/**
* 协议 v3 文本上行。参数刻意用 `CharArray`(UTF-16 code unit)而不是 `String`:
* jni 侧 `get_string` 走 Modified-UTF-8 往返,IME 提交边界的落单代理项会在
* 往返中丢失;`char[]` 经 `GetCharArrayRegion` 按 u16 原样拷贝。
*/
external fun nativeSendText(handle: Long, units: CharArray)
external fun nativeRequestKeyframe(handle: Long) external fun nativeRequestKeyframe(handle: Long)
external fun nativeDestroy(handle: Long) external fun nativeDestroy(handle: Long)
/** 连接。失败抛 IllegalStateException(带 Rust 侧错误消息)。 */ /**
* 连接。失败抛 [ConnectFailure](分类 + Rust 侧原始消息)。
* 取序遵守 FFI 契约:先 kind 后 message。
*/
fun connect(host: String, port: Int, code: String): Long { fun connect(host: String, port: Int, code: String): Long {
val h = nativeConnect(host, port, code) val h = nativeConnect(host, port, code)
if (h == 0L) { if (h == 0L) {
throw IllegalStateException(nativeLastError() ?: "connect failed") val kind = nativeLastErrorKind()
val msg = nativeLastError() ?: "connect failed"
throw ConnectFailure(kind, msg)
} }
return h return h
} }
@@ -49,23 +76,21 @@ object RustBridge {
val tsMs: Long, val tsMs: Long,
val width: Int, val width: Int,
val height: Int, val height: Int,
/**
* v5:本帧发送时刻的远端光标(采集画面内物理像素)。
* `null` = 隐藏/离屏/后端未实现(语义单一来源在被控端,见 common 协议注释)。
* 用一个可空 Pair 而非两个可空 Int:忠实对应 `Option<(i16,i16)>`,
* 杜绝「x 有 y 无」这种协议里不存在的状态。
*/
val cursor: Pair<Int, Int>?,
val payload: ByteArray, val payload: ByteArray,
) )
/** 取帧 + 拆头。null = 会话结束。 */ /** 取帧 + 拆头。null = 会话结束。 */
fun nextFrame(handle: Long): Frame? { fun nextFrame(handle: Long): Frame? {
val raw = nativeNextFrame(handle) ?: return null val raw = nativeNextFrame(handle) ?: return null
if (raw.size < FRAME_HEADER_LEN) return null return FrameCodec.parse(raw)
return Frame(
keyframe = raw[0] != 0.toByte(),
seq = readLong(raw, 1),
tsMs = readLong(raw, 9),
width = readInt(raw, 17),
height = readInt(raw, 21),
payload = raw.copyOfRange(FRAME_HEADER_LEN, raw.size),
)
} }
fun rttMs(handle: Long): Long { fun rttMs(handle: Long): Long {
val packed = nativeStats(handle) val packed = nativeStats(handle)
return if (packed < 0) 0 else (packed ushr 32) return if (packed < 0) 0 else (packed ushr 32)
@@ -87,6 +112,41 @@ object RustBridge {
val h = (packed and 0xFFFF_FFFF).toInt() val h = (packed and 0xFFFF_FFFF).toInt()
return w to h return w to h
} }
}
/**
* 帧头拆包(v5)。从 [RustBridge] 里独立出来的原因:**这个文件里的字节偏移必须能在
* JVM 单测里被验证**,而 `RustBridge` object 带 `init { System.loadLibrary }`,
* 测试里一碰它就抛 `UnsatisfiedLinkError`(本机无 .so)。顶层 object 无 native、无 init,
* 构造嵌套类 [RustBridge.Frame] 也不触发外层 object 初始化——拆包逻辑因此可测。
*
* 跨端契约:android-ffi/src/lib.rs 按 buf[0]/[1..9]/[9..17]/[17..21]/[21..25]/[25]/
* [26..28]/[28..30] 写字节,这里按**同样的偏移**读。任一侧挪一位,两端都编译得过、
* 真机也连得上——只是标记画错位置或永不显示(静默失败)。golden bytes 对拍锁死它,
* 见 RustBridgeTest。
*/
internal object FrameCodec {
// v5:[1 kf][8 seq][8 ts][4 w][4 h][1 has_cursor][2 x i16][2 y i16] = 30
// (与 android-ffi/src/lib.rs 的 FRAME_HEADER_LEN 同步;新增字段一律追加尾部,
// 旧偏移 0/1/9/17/21 不移动——R-5 同源检查过:两处必须同改)
const val FRAME_HEADER_LEN = 30
/** 拆一帧。短于定长头 → null(截断/旧布局防御)。 */
fun parse(raw: ByteArray): RustBridge.Frame? {
if (raw.size < FRAME_HEADER_LEN) return null
val hasCursor = raw[25] != 0.toByte()
return RustBridge.Frame(
keyframe = raw[0] != 0.toByte(),
seq = readLong(raw, 1),
tsMs = readLong(raw, 9),
width = readInt(raw, 17),
height = readInt(raw, 21),
// has=0 时坐标字节恒为 0 且无意义,绝不能读成 (0,0)——那会被画到左上角
cursor = if (hasCursor) Pair(readI16(raw, 26), readI16(raw, 28)) else null,
payload = raw.copyOfRange(FRAME_HEADER_LEN, raw.size),
)
}
private fun readLong(b: ByteArray, off: Int): Long { private fun readLong(b: ByteArray, off: Int): Long {
var v = 0L var v = 0L
@@ -100,4 +160,9 @@ object RustBridge {
((b[off + 2].toInt() and 0xFF) shl 16) or ((b[off + 2].toInt() and 0xFF) shl 16) or
((b[off + 3].toInt() and 0xFF) shl 24) ((b[off + 3].toInt() and 0xFF) shl 24)
} }
/** 按 little-endian **有符号** 16 位读(协议 cursor 是 i16;读成无符号会让负坐标穿透) */
private fun readI16(b: ByteArray, off: Int): Int {
return ((b[off].toInt() and 0xFF) or ((b[off + 1].toInt() and 0xFF) shl 8)).toShort().toInt()
}
} }
@@ -0,0 +1,126 @@
package com.verify.remote
import android.view.KeyEvent
/**
* Android KeyEvent.KEYCODE_* → Windows PS/2 Set-1 扫描码 + extended 标志。
*
* **扫描码值直接取自桌面 viewer 自己的映射(`viewer/src/keymap.rs`,项目内部事实)**,
* 不是凭记忆写的;Android 侧一律用命名常量 `KeyEvent.KEYCODE_*` 引用,
* 数值由编译器解析(已用 `javap -constants android.jar` 验证这些常量在 API 34 存在)。
*
* 分工(协议 v3 之后):
* - 可打印字符优先走 **Text 通道**(`KeyEvent.unicodeChar` / IME `commitText`),
* 布局无关、中文可用;
* - 本表服务控制键(Enter/Tab/Backspace/方向键/功能键)与物理键盘,
* 走 `KeyDown/KeyUp` 扫描码通道。
*
* 不做 Shift/Ctrl 组合状态跟踪:软键盘的大写字母由 IME 直接 commit 字符,
* 不需要本地修饰键语义。物理键盘组合键(Ctrl+C 等)登记为后续事项。
*/
data class WinKey(val scancode: Int, val extended: Boolean)
object WinScancode {
fun of(keyCode: Int): WinKey? = when (keyCode) {
// 字母区(Set-1 make code,同 keymap.rs)
KeyEvent.KEYCODE_A -> WinKey(0x1E, false)
KeyEvent.KEYCODE_B -> WinKey(0x30, false)
KeyEvent.KEYCODE_C -> WinKey(0x2E, false)
KeyEvent.KEYCODE_D -> WinKey(0x20, false)
KeyEvent.KEYCODE_E -> WinKey(0x12, false)
KeyEvent.KEYCODE_F -> WinKey(0x21, false)
KeyEvent.KEYCODE_G -> WinKey(0x22, false)
KeyEvent.KEYCODE_H -> WinKey(0x23, false)
KeyEvent.KEYCODE_I -> WinKey(0x17, false)
KeyEvent.KEYCODE_J -> WinKey(0x24, false)
KeyEvent.KEYCODE_K -> WinKey(0x25, false)
KeyEvent.KEYCODE_L -> WinKey(0x26, false)
KeyEvent.KEYCODE_M -> WinKey(0x32, false)
KeyEvent.KEYCODE_N -> WinKey(0x31, false)
KeyEvent.KEYCODE_O -> WinKey(0x18, false)
KeyEvent.KEYCODE_P -> WinKey(0x19, false)
KeyEvent.KEYCODE_Q -> WinKey(0x10, false)
KeyEvent.KEYCODE_R -> WinKey(0x13, false)
KeyEvent.KEYCODE_S -> WinKey(0x1F, false)
KeyEvent.KEYCODE_T -> WinKey(0x14, false)
KeyEvent.KEYCODE_U -> WinKey(0x16, false)
KeyEvent.KEYCODE_V -> WinKey(0x2F, false)
KeyEvent.KEYCODE_W -> WinKey(0x11, false)
KeyEvent.KEYCODE_X -> WinKey(0x2D, false)
KeyEvent.KEYCODE_Y -> WinKey(0x15, false)
KeyEvent.KEYCODE_Z -> WinKey(0x2C, false)
// 数字主键区(1..9, 0;Android keycode 不连续,逐个写)
KeyEvent.KEYCODE_1 -> WinKey(0x02, false)
KeyEvent.KEYCODE_2 -> WinKey(0x03, false)
KeyEvent.KEYCODE_3 -> WinKey(0x04, false)
KeyEvent.KEYCODE_4 -> WinKey(0x05, false)
KeyEvent.KEYCODE_5 -> WinKey(0x06, false)
KeyEvent.KEYCODE_6 -> WinKey(0x07, false)
KeyEvent.KEYCODE_7 -> WinKey(0x08, false)
KeyEvent.KEYCODE_8 -> WinKey(0x09, false)
KeyEvent.KEYCODE_9 -> WinKey(0x0A, false)
KeyEvent.KEYCODE_0 -> WinKey(0x0B, false)
// 符号区(US 布局物理位置,同 keymap.rs)
KeyEvent.KEYCODE_SPACE -> WinKey(0x39, false)
KeyEvent.KEYCODE_ENTER -> WinKey(0x1C, false)
KeyEvent.KEYCODE_DEL -> WinKey(0x0E, false) // Backspace
KeyEvent.KEYCODE_FORWARD_DEL -> WinKey(0x53, true) // Delete(扩展)
KeyEvent.KEYCODE_TAB -> WinKey(0x0F, false)
KeyEvent.KEYCODE_ESCAPE -> WinKey(0x01, false)
KeyEvent.KEYCODE_CAPS_LOCK -> WinKey(0x3A, false)
KeyEvent.KEYCODE_GRAVE -> WinKey(0x29, false)
KeyEvent.KEYCODE_MINUS -> WinKey(0x0C, false)
KeyEvent.KEYCODE_EQUALS -> WinKey(0x0D, false)
KeyEvent.KEYCODE_LEFT_BRACKET -> WinKey(0x1A, false)
KeyEvent.KEYCODE_RIGHT_BRACKET -> WinKey(0x1B, false)
KeyEvent.KEYCODE_BACKSLASH -> WinKey(0x2B, false)
KeyEvent.KEYCODE_SEMICOLON -> WinKey(0x27, false)
KeyEvent.KEYCODE_APOSTROPHE -> WinKey(0x28, false)
KeyEvent.KEYCODE_COMMA -> WinKey(0x33, false)
KeyEvent.KEYCODE_PERIOD -> WinKey(0x34, false)
KeyEvent.KEYCODE_SLASH -> WinKey(0x35, false)
// 修饰键
KeyEvent.KEYCODE_SHIFT_LEFT -> WinKey(0x2A, false)
// 右 Shift **不是**扩展键:微软 Keyboard Input Overview 明确
// "The right-hand Shift key is not considered an extended key either;
// it has a separate scan code (0x36) instead."(本会话实读该页核实)
KeyEvent.KEYCODE_SHIFT_RIGHT -> WinKey(0x36, false)
KeyEvent.KEYCODE_CTRL_LEFT -> WinKey(0x1D, false)
KeyEvent.KEYCODE_CTRL_RIGHT -> WinKey(0x1D, true)
KeyEvent.KEYCODE_ALT_LEFT -> WinKey(0x38, false)
KeyEvent.KEYCODE_ALT_RIGHT -> WinKey(0x38, true)
KeyEvent.KEYCODE_META_LEFT -> WinKey(0x5B, true)
KeyEvent.KEYCODE_META_RIGHT -> WinKey(0x5C, true)
// 导航区(全部扩展键)
KeyEvent.KEYCODE_DPAD_UP -> WinKey(0x48, true)
KeyEvent.KEYCODE_DPAD_DOWN -> WinKey(0x50, true)
KeyEvent.KEYCODE_DPAD_LEFT -> WinKey(0x4B, true)
KeyEvent.KEYCODE_DPAD_RIGHT -> WinKey(0x4D, true)
KeyEvent.KEYCODE_MOVE_HOME -> WinKey(0x47, true)
KeyEvent.KEYCODE_MOVE_END -> WinKey(0x4F, true)
KeyEvent.KEYCODE_PAGE_UP -> WinKey(0x49, true)
KeyEvent.KEYCODE_PAGE_DOWN -> WinKey(0x51, true)
KeyEvent.KEYCODE_INSERT -> WinKey(0x52, true)
// 功能键(F11/F12 的 0x57/0x58 与 linux-agent keycode_of 注释同源)
KeyEvent.KEYCODE_F1 -> WinKey(0x3B, false)
KeyEvent.KEYCODE_F2 -> WinKey(0x3C, false)
KeyEvent.KEYCODE_F3 -> WinKey(0x3D, false)
KeyEvent.KEYCODE_F4 -> WinKey(0x3E, false)
KeyEvent.KEYCODE_F5 -> WinKey(0x3F, false)
KeyEvent.KEYCODE_F6 -> WinKey(0x40, false)
KeyEvent.KEYCODE_F7 -> WinKey(0x41, false)
KeyEvent.KEYCODE_F8 -> WinKey(0x42, false)
KeyEvent.KEYCODE_F9 -> WinKey(0x43, false)
KeyEvent.KEYCODE_F10 -> WinKey(0x44, false)
KeyEvent.KEYCODE_F11 -> WinKey(0x57, false)
KeyEvent.KEYCODE_F12 -> WinKey(0x58, false)
else -> null
}
}
@@ -0,0 +1,80 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* 3a/U7:连接错误编号→文案映射。
* 编号契约的另一端是 Rust(`viewer_core::ConnectErrorKind::code`,
* 由 `cargo test -p viewer-core ffi_error_codes_are_stable_and_unique` 锁死),
* 这里锁 Kotlin 侧的完整覆盖与文案质量——两端一致才算契约成立。
*/
class ConnectErrorTextTest {
@Test
fun every_known_kind_has_actionable_chinese_text() {
val kinds = listOf(
ConnectErrorText.KIND_RESOLVE,
ConnectErrorText.KIND_TIMEOUT,
ConnectErrorText.KIND_REFUSED,
ConnectErrorText.KIND_AUTH,
ConnectErrorText.KIND_VERSION,
ConnectErrorText.KIND_OTHER,
ConnectErrorText.KIND_FFI_SELF,
)
for (k in kinds) {
val s = ConnectErrorText.summary(k)
assertTrue("kind=$k 必须有文案", !s.isNullOrEmpty())
// 项目纪律:用户可见输出中文(文案里至少含一个 CJK 字符)
assertTrue("kind=$k 文案必须中文", s!!.any { c -> c.code in 0x4E00..0x9FFF })
}
}
@Test
fun codes_match_ffi_contract() {
// 期望值 = android-ffi / viewer-core 的 code() 表(改动必须三处同步)
assertEquals(1, ConnectErrorText.KIND_RESOLVE)
assertEquals(2, ConnectErrorText.KIND_TIMEOUT)
assertEquals(3, ConnectErrorText.KIND_REFUSED)
assertEquals(4, ConnectErrorText.KIND_AUTH)
assertEquals(5, ConnectErrorText.KIND_VERSION)
assertEquals(9, ConnectErrorText.KIND_OTHER)
assertEquals(-1, ConnectErrorText.KIND_FFI_SELF)
assertEquals(0, ConnectErrorText.KIND_NONE)
}
@Test
fun texts_are_distinct_per_failure_mode() {
// 五种可区分失败不能给同一句文案(V-A2 的映射前提)
val texts = listOf(
ConnectErrorText.KIND_RESOLVE,
ConnectErrorText.KIND_TIMEOUT,
ConnectErrorText.KIND_REFUSED,
ConnectErrorText.KIND_AUTH,
ConnectErrorText.KIND_VERSION,
).map { ConnectErrorText.summary(it)!! }
assertEquals(texts.size, texts.distinct().size)
for (t in texts) {
assertNotEquals("连接失败", t) // 可分类的不得落兜底文案
}
}
@Test
fun unknown_kind_falls_back_without_claiming_a_cause() {
// 编号漂移时不猜语义:兜底文案 + 带未知编号(R-8:宁可少说不可说错)
assertNull(ConnectErrorText.summary(42))
val line = ConnectErrorText.firstLine(42)
assertTrue("实际: $line", line.contains("42"))
assertTrue("实际: $line", line.startsWith("连接失败"))
}
@Test
fun firstLine_uses_summary_for_known_kinds() {
for (k in listOf(1, 2, 3, 4, 5, 9, -1)) {
assertEquals(ConnectErrorText.summary(k), ConnectErrorText.firstLine(k))
}
}
}
@@ -0,0 +1,115 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotNull
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* 4b/V-D1:光标标记映射的纯逻辑(协议 v5)。
*
* 这些断言对应验收判据,不是「跑通就算」:
* - V-D1「标记位置=映射值」→ [center_maps_to_view_center] / [full_extent_linear_map]
* - 「(0,0) 是合法位置」→ [origin_maps_not_treated_as_none](协议用 Option 的全部理由)
* - 跨信任边界自证 → 越界与负值一律 null(Linux 后端未实现/被篡改时的防线)
*/
class CursorMarkerTest {
// ---- 核心等比映射 ----
/**
* 横屏典型几何:桌面 1920x1080,视图 1080x607(满宽 letterbox)。
* 桌面中心应映射到视图中心。这是 V-D1 的锚点用例——真机 proof 就是把
* SetCursorPos 打到桌面中心,再在截图里断言标记簇心在视图中心 ±6px。
*/
@Test
fun center_maps_to_view_center() {
val p = CursorMarker.mapToView(
desktopW = 1920, desktopH = 1080,
cursor = 960 to 540,
viewW = 1080, viewH = 607,
)
assertNotNull(p)
assertEquals(540f, p!!.x, 1f)
assertEquals(303.5f, p.y, 1f)
}
/**
* 满量程线性映射:桌面右下角 (1919,1079) → 视图右下角附近,
* 且不越过视图边界(映射后必须仍在画面内可画)。
*/
@Test
fun full_extent_linear_map() {
val p = CursorMarker.mapToView(1920, 1080, 1919 to 1079, 1080, 607)
assertNotNull(p)
assertEquals(1080f * 1919 / 1920f, p!!.x, 0.5f)
assertEquals(607f * 1079 / 1080f, p.y, 0.5f)
assertTrue("映射 x 不得越过视图右边界", p.x < 1080f)
assertTrue("映射 y 不得越过视图下边界", p.y < 607f)
}
/**
* (0,0) 必须映射成视图左上角,**而不是 null**。
* 这条锁住最危险的错误:实现把「左上角」和「没有光标」混为一谈
* (比如用 (0,0) 当缺省值再判 ==0 返回 null)。协议专门用 Option 区分二者,
* 这层区分在 Kotlin 侧不能被抹平。
*/
@Test
fun origin_maps_not_treated_as_none() {
val p = CursorMarker.mapToView(1920, 1080, 0 to 0, 1080, 607)
assertNotNull("左上角是合法光标位置,不得当成无光标丢弃")
assertEquals(0f, p!!.x, 0.001f)
assertEquals(0f, p.y, 0.001f)
}
/** 1:1 场景(视图尺寸恰等于桌面):坐标原样透传。 */
@Test
fun identity_when_sizes_match() {
val p = CursorMarker.mapToView(1280, 720, 640 to 360, 1280, 720)
assertEquals(640f, p!!.x, 0.001f)
assertEquals(360f, p.y, 0.001f)
}
// ---- null 分支:一律「不画」而非画到别处 ----
/** cursor=null(隐藏/离屏/后端未实现)→ null。 */
@Test
fun null_cursor_returns_null() {
assertNull(CursorMarker.mapToView(1920, 1080, null, 1080, 607))
}
/**
* 任一尺寸未就绪(0 或负)→ null,绝不按 0 除得到 NaN/Infinity。
* 覆盖四种:视图未测量(width 0,letterbox onMeasure 前)、
* 桌面尺寸还没从 AuthOk 取到、以及负值这种异常。
*/
@Test
fun not_ready_or_degenerate_dimensions_return_null() {
assertNull(CursorMarker.mapToView(0, 1080, 10 to 10, 1080, 607)) // 桌面宽未就绪
assertNull(CursorMarker.mapToView(1920, 0, 10 to 10, 1080, 607)) // 桌面高未就绪
assertNull(CursorMarker.mapToView(1920, 1080, 10 to 10, 0, 607)) // 视图未测量
assertNull(CursorMarker.mapToView(1920, 1080, 10 to 10, 1080, 0)) // 视图高 0
assertNull(CursorMarker.mapToView(-5, 1080, 10 to 10, 1080, 607)) // 异常负尺寸
}
/**
* 越界坐标 → null(不画)。这是跨信任边界的自证:被控端 `fold_cursor` 本应折叠掉
* 越界值,但协议把对端字节按不可信输入处理——若折叠漏网或被篡改,这里兜住,
* 否则标记会画到画面外(甚至映射出一个「看似合理」的错位置)。
*/
@Test
fun out_of_desktop_bounds_returns_null() {
assertNull(CursorMarker.mapToView(1920, 1080, 1920 to 500, 1080, 607)) // 右边界外一列
assertNull(CursorMarker.mapToView(1920, 1080, 500 to 1080, 1080, 607)) // 下边界外一行
assertNull(CursorMarker.mapToView(1920, 1080, -1 to 500, 1080, 607)) // 负 X(副屏穿透)
assertNull(CursorMarker.mapToView(1920, 1080, 500 to -1, 1080, 607)) // 负 Y
assertNull(CursorMarker.mapToView(1920, 1080, 3000 to 500, 1080, 607)) // 远超桌面
}
/** 恰好界内 (desktopW-1, desktopH-1) 必须画(与上一条配对,钉死 >= 还是 > 的边界)。 */
@Test
fun exactly_in_bounds_draws() {
assertNotNull(CursorMarker.mapToView(1920, 1080, 1919 to 1079, 1080, 607))
}
}
@@ -0,0 +1,100 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotEquals
import org.junit.Test
/**
* 4a:方向策略纯逻辑。
*
* 常量数值不是随手写的——`javap -constants` 对 android-34 android.jar 核实过
* (LANDSCAPE=0、PORTRAIT=1、UNSPECIFIED=-1)。这里把它们钉成断言:
* 若有人「顺手改成 SENSOR_LANDSCAPE=6」或调换数值,测试先红。
*/
class OrientationPolicyTest {
@Test
fun android_constants_pinned_to_verified_values() {
assertEquals(0, Orientation.REQUEST_LANDSCAPE)
assertEquals(1, Orientation.REQUEST_PORTRAIT)
assertEquals(-1, Orientation.REQUEST_UNSPECIFIED)
}
// ---- AUTO 的判定 ----
@Test
fun auto_picks_landscape_for_wider_remote() {
// 本机被控端就是 1920x1080 —— 这正是 4a 需求的来源
assertEquals(Orientation.REQUEST_LANDSCAPE, Orientation.requestFor(OrientationPolicy.AUTO, 1920, 1080))
}
@Test
fun auto_picks_portrait_for_taller_remote() {
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 1080, 1920))
}
@Test
fun auto_picks_portrait_for_square_and_unknown() {
// 方屏没有「更宽」的优势,按竖屏兜底
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 1080, 1080))
// 尺寸未知(连接前/断开后)必须竖屏,不能让表单页横掉
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 0, 0))
}
@Test
fun forced_policies_ignore_remote_geometry() {
for (wh in listOf(1920 to 1080, 1080 to 1920, 0 to 0)) {
assertEquals(Orientation.REQUEST_LANDSCAPE,
Orientation.requestFor(OrientationPolicy.LANDSCAPE, wh.first, wh.second))
assertEquals(Orientation.REQUEST_PORTRAIT,
Orientation.requestFor(OrientationPolicy.PORTRAIT, wh.first, wh.second))
assertEquals(Orientation.REQUEST_UNSPECIFIED,
Orientation.requestFor(OrientationPolicy.SYSTEM, wh.first, wh.second))
}
}
// ---- 按钮循环(Q4=A)----
@Test
fun button_cycles_all_four_in_order() {
assertEquals(OrientationPolicy.LANDSCAPE, Orientation.next(OrientationPolicy.AUTO))
assertEquals(OrientationPolicy.PORTRAIT, Orientation.next(OrientationPolicy.LANDSCAPE))
assertEquals(OrientationPolicy.SYSTEM, Orientation.next(OrientationPolicy.PORTRAIT))
assertEquals(OrientationPolicy.AUTO, Orientation.next(OrientationPolicy.SYSTEM))
}
@Test
fun four_clicks_return_to_start() {
var p = OrientationPolicy.AUTO
repeat(4) { p = Orientation.next(p) }
assertEquals(OrientationPolicy.AUTO, p)
}
@Test
fun every_policy_has_distinct_nonempty_label() {
val labels = OrientationPolicy.values().map { Orientation.label(it) }
labels.forEach { assertNotEquals("", it) }
assertEquals("四个策略文案必须互不相同", labels.size, labels.toSet().size)
// 文案要让用户看出当前状态(含「方向:」前缀),不能只写「自动/横/竖」
labels.forEach { assertEquals(true, it.startsWith("方向:")) }
}
// ---- prefs 反序列化 ----
@Test
fun from_name_roundtrips_all() {
OrientationPolicy.values().forEach {
assertEquals(it, Orientation.fromName(it.name))
}
}
@Test
fun from_name_falls_back_to_auto_for_garbage() {
// 未知值不得抛异常(会让 Activity 起不来),必须落回默认 AUTO
assertEquals(OrientationPolicy.AUTO, Orientation.fromName(null))
assertEquals(OrientationPolicy.AUTO, Orientation.fromName(""))
assertEquals(OrientationPolicy.AUTO, Orientation.fromName("DIAGONAL"))
// 大小写宽容:早期版本可能存过小写
assertEquals(OrientationPolicy.LANDSCAPE, Orientation.fromName("landscape"))
}
}
@@ -0,0 +1,46 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Test
/**
* 3a/U7:退格决策纯函数。原逻辑在 `RemoteInputConnection.deleteBackspaces`
* (构造链依赖 android.jar 桩,JVM 里不可测),3a 提取为 [planBackspaces] 后
* 首次可单测。期望值对照 2b 真机行为(语义不变,见 RemoteInputConnection 注释)。
*/
class PlanBackspacesTest {
@Test
fun zero_zero_is_noop() {
assertEquals(BackspaceAction.Noop, planBackspaces(0, 0))
}
@Test
fun after_positive_is_unsupported_and_reports_both_counts() {
// U3:forward-delete 形态本地无远端真值,必须显式 Unsupported(可见后果),
// 不得静默当退格执行——before 值要随形态带出去供 debug 通道显示
assertEquals(BackspaceAction.Unsupported(2, 1), planBackspaces(2, 1))
assertEquals(BackspaceAction.Unsupported(0, 3), planBackspaces(0, 3))
}
@Test
fun before_only_maps_to_backspace_count() {
// IME 退格最常见形态(R-9 实测)
assertEquals(BackspaceAction.Backspaces(3), planBackspaces(3, 0))
assertEquals(BackspaceAction.Backspaces(1), planBackspaces(1, 0))
}
@Test
fun count_is_capped_at_limit() {
// 防 IME 异常大值刷爆消息队列;截断语义必须发生在计划层(执行层无感知上限)
assertEquals(BackspaceAction.Backspaces(64), planBackspaces(64, 0))
assertEquals(BackspaceAction.Backspaces(64), planBackspaces(999, 0))
assertEquals(BackspaceAction.Backspaces(MAX_BACKSPACE), planBackspaces(1_000_000, 0))
}
@Test
fun MAX_BACKSPACE_constant_is_pinned() {
// 与 RemoteInputConnection 的 companion 原值一致(提取时不得改数)
assertEquals(64, MAX_BACKSPACE)
}
}
@@ -0,0 +1,172 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* 3b/V-B1:相对鼠标的纯逻辑(协议 v4)。
*
* 这些断言对应验收判据,不是「跑通就算」:
* - V-B1b「慢速小幅滑动不丢位移」→ [accumulatorKeepsSubPixelRemainders]
* - V-B1c「撞边后回滑立即响应」→ 累积器只做取整、不做补偿([accumulatorDoesNotBankLostSteps])
*/
class RelPointerTest {
// ---- 换算:视图像素 → 桌面像素 ----
@Test
fun sensitivity_one_maps_full_view_width_to_full_desktop_width() {
// Q4=A 的语义定义:1.0 = 手指扫过视图全宽 = 指针走完桌面全宽
val d = RelPointer.toDesktopDelta(
viewDeltaPx = 1080f, // 视图全宽(1080x607 letterbox)
viewExtentPx = 1080,
remoteExtentPx = 1920, // 远端桌面宽
sensitivity = 1.0f,
)
assertEquals(1920f, d, 0.01f)
}
@Test
fun sensitivity_scales_linearly() {
val base = RelPointer.toDesktopDelta(100f, 1080, 1920, 1.0f)
val twice = RelPointer.toDesktopDelta(100f, 1080, 1920, 2.0f)
assertEquals(base * 2f, twice, 0.01f)
}
@Test
fun negative_delta_stays_negative() {
// 向左滑必须得到负增量(绝对坐标是无符号的,相对通道是唯一有符号路径)
val d = RelPointer.toDesktopDelta(-54f, 1080, 1920, 1.0f)
assertTrue("向左滑应得到负增量,实际 $d", d < 0f)
}
@Test
fun invalid_extents_yield_zero_not_garbage() {
// 视图未测量 / 远端尺寸未取到时必须不动,而不是按错误比例乱飞
assertEquals(0f, RelPointer.toDesktopDelta(50f, 0, 1920, 1.0f), 0f)
assertEquals(0f, RelPointer.toDesktopDelta(50f, 1080, 0, 1.0f), 0f)
assertEquals(0f, RelPointer.toDesktopDelta(0f, 1080, 1920, 1.0f), 0f)
}
@Test
fun sensitivity_is_clamped_to_documented_range() {
assertEquals(RelPointer.MIN_SENSITIVITY, RelPointer.clampSensitivity(0.01f), 0f)
assertEquals(RelPointer.MAX_SENSITIVITY, RelPointer.clampSensitivity(99f), 0f)
assertEquals(1.5f, RelPointer.clampSensitivity(1.5f), 0f)
// 边界常量本身必须与方案 Q4=A 一致
assertEquals(0.25f, RelPointer.MIN_SENSITIVITY, 0f)
assertEquals(3.0f, RelPointer.MAX_SENSITIVITY, 0f)
assertEquals(1.0f, RelPointer.DEFAULT_SENSITIVITY, 0f)
}
// ---- 累积取整:V-B1b ----
@Test
fun accumulatorKeepsSubPixelRemainders() {
// 每次只移动 0.3 桌面像素:逐次 toInt() 会全被吞掉(位移静默丢失)。
// 累积器必须在攒够 1 像素时发出,且总量守恒。
val acc = RelAccumulator()
var total = 0
repeat(10) {
acc.take(0.3f, 0f)?.let { total += it.first }
}
// 10 * 0.3 = 3.0 → 必须发出 3 像素,而不是 0
assertEquals("慢速滑动不得丢位移", 3, total)
}
@Test
fun accumulatorDoesNotEmitBelowHalfPixel() {
// 小于 0.5 的余量四舍五入后是 0 → 不发(否则会产生大量零位移消息)。
// 注意阈值是 0.5 而不是 1.0:累积器用 roundToInt 而非 toInt,
// 0.8 会先发出 1 并把 -0.2 留作余数(自我修正,长期总量守恒)。
val acc = RelAccumulator()
assertNull("0.4 四舍五入为 0,不得发送", acc.take(0.4f, 0.4f))
// 再攒 0.4 → 合计 0.8,四舍五入为 1,此时必须发出
val d = acc.take(0.4f, 0.4f)
assertEquals(1, d!!.first)
assertEquals(1, d.second)
// 余数 -0.2:紧接着的 0.2 会被抵消成 0,不再发送(证明余数被正确保留)
assertNull(acc.take(0.2f, 0.2f))
}
@Test
fun accumulatorRoundsNegativeValuesAwayFromZero() {
// -0.7 应取 -1(向零截断会得到 0,反向慢滑就丢了)
val acc = RelAccumulator()
val d = acc.take(-0.7f, 0f)
assertEquals(-1, d!!.first)
}
@Test
fun accumulatorAxesAreIndependent() {
// 斜向慢滑:X 攒够时 Y 还没攒够,不能因为 Y 不够就把 X 也拖住
val acc = RelAccumulator()
val d = acc.take(1.0f, 0.2f)
assertEquals(1, d!!.first)
assertEquals(0, d.second)
}
@Test
fun accumulatorResetDropsRemainder() {
// 手势/会话结束时清空:上一段的余数不该漏进下一段
val acc = RelAccumulator()
acc.take(0.6f, 0.6f)
acc.reset()
assertNull(acc.take(0.3f, 0.3f))
}
@Test
fun accumulatorDoesNotBankLostSteps() {
// V-B1c:撞边时被 clamp 丢掉的位移**不回补**。累积器只负责取整,
// 不负责补偿——否则撞边后回滑会先"空转"一段,违反「立即响应」。
val acc = RelAccumulator()
acc.take(1.0f, 0f) // 发出 +1
val back = acc.take(-1.0f, 0f)
assertEquals("回滑必须立即发出,不得因补偿而空转", -1, back!!.first)
}
@Test
fun accumulatorClampsToProtocolRange() {
// 协议字段是 i16;越界必须被 clamp 而不是回绕(回绕会让指针跳到屏幕另一侧)
val acc = RelAccumulator()
val d = acc.take(100_000f, -100_000f)
assertEquals(32767, d!!.first)
assertEquals(-32768, d.second)
}
// ---- 长按右键:V-B3 ----
@Test
fun longPress_fires_at_threshold_when_still() {
assertTrue(LongPressPolicy.shouldFire(500L, 0f, 30f, false))
assertTrue(LongPressPolicy.shouldFire(900L, 2f, 30f, false))
}
@Test
fun longPress_does_not_fire_too_early() {
assertFalse(LongPressPolicy.shouldFire(499L, 0f, 30f, false))
assertFalse(LongPressPolicy.shouldFire(0L, 0f, 30f, false))
}
@Test
fun longPress_does_not_fire_after_moving() {
// V-B3「拖动窗口时不弹菜单」:超过 slop 一律不算长按
assertFalse(LongPressPolicy.shouldFire(800L, 31f, 30f, false))
assertFalse(LongPressPolicy.shouldFire(800L, 200f, 30f, false))
}
@Test
fun longPress_does_not_fire_with_two_fingers() {
// 双指有自己的右键路径,长按不能叠加(否则一次双指轻点会出两个右键)
assertFalse(LongPressPolicy.shouldFire(900L, 0f, 30f, true))
}
@Test
fun long_press_threshold_is_pinned_to_decision() {
// Q5=A:500ms,改动必须显式经过这条断言
assertEquals(500L, LongPressPolicy.LONG_PRESS_MS)
}
}
@@ -0,0 +1,106 @@
package com.verify.remote
import org.junit.Assert.assertArrayEquals
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Test
/**
* 4b:[FrameCodec] 帧头拆包的**跨端字节契约**测试(golden bytes)。
*
* 为什么值这个钱:android-ffi/src/lib.rs 按 buf[0]/[1..9]/[9..17]/[17..21]/[21..25]/
* [25]/[26..28]/[28..30] 写字节,Kotlin 按同样的偏移读。**任何一侧挪一位**(尤其 v5
* 从 25→30 这次),Rust 编译过、Kotlin 也编译过、真机能连上——只是标记画到错误位置
* 或永远不显示,静默失败(全局规则:核过 Rust 一端不等于核过 Kotlin 另一端)。
*
* 期望值用 goldenHeader 按 Rust 侧写入布局**手算**生成(`to_le_bytes()` 的逐字节结果)。
* Rust 侧若改偏移或改字节序,这里的对拍就红。这是唯一能在 JVM 里锁住 JNI 二进制布局的
* 手段(真机跑不了单测)。拆包逻辑之所以住在顶层 [FrameCodec] 而不是 RustBridge object,
* 正因为后者带 loadLibrary 的 init,测试里不可达。
*/
class RustBridgeTest {
/**
* 「IDR、seq=0x0102030405060708、ts=0x0011223344556677、桌面 1920x1080、
* 光标 has=x/y」的 30 字节头,逐字节 = android-ffi 的写入布局。
*/
private fun goldenHeader(hasCursor: Int, x: Int, y: Int): ByteArray {
val bytes = listOf(
0x01, // [0] keyframe = true
// [1..9] seq = 0x0102030405060708 LE
0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01,
// [9..17] ts = 0x0011223344556677 LE
0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00,
// [17..21] width = 1920 (0x0780) LE
0x80, 0x07, 0x00, 0x00,
// [21..25] height = 1080 (0x0438) LE
0x38, 0x04, 0x00, 0x00,
// [25] has_cursor
hasCursor,
// [26..28] cursor.x i16 LE
x and 0xFF, (x shr 8) and 0xFF,
// [28..30] cursor.y i16 LE
y and 0xFF, (y shr 8) and 0xFF,
)
return bytes.map { it.toByte() }.toByteArray()
}
/** 光标 Some((960,540)):逐字段核对,锁死 [25]/[26..28]/[28..30] 三个新偏移。 */
@Test
fun parses_v5_header_with_cursor() {
val payload = byteArrayOf(0xAA.toInt().toByte(), 0xBB.toInt().toByte(), 0xCC.toInt().toByte())
val raw = goldenHeader(hasCursor = 1, x = 960, y = 540) + payload
val f = FrameCodec.parse(raw)!!
assertEquals(true, f.keyframe)
assertEquals(0x0102030405060708L, f.seq)
assertEquals(0x0011223344556677L, f.tsMs)
assertEquals(1920, f.width)
assertEquals(1080, f.height)
assertEquals(960 to 540, f.cursor)
assertArrayEquals(payload, f.payload)
}
/**
* has_cursor=0 时**即便坐标字节是全 1 位模式(0xFFFF = -1)也必须读成 null**。
* 这条锁住 `if (hasCursor)` 的判断:若误判成「非零即有」,(-1,-1) 会穿透成一个
* 看似合法的错位置——正是协议用独立 has 字节而非「0 表示无」的原因。
*/
@Test
fun has_cursor_zero_means_none_regardless_of_coord_bytes() {
// 故意把 x/y 字节填成 -1 的位模式(0xFFFF)
val raw = goldenHeader(hasCursor = 0, x = 0xFFFF, y = 0xFFFF) + byteArrayOf(0x00)
val f = FrameCodec.parse(raw)!!
assertNull("has=0 时坐标字节无意义,必须 null", f.cursor)
}
/** 负光标坐标(有符号 i16)必须正确读回——agent 折叠后不该出现,但拆包层要忠实。 */
@Test
fun reads_signed_cursor_coords() {
val raw = goldenHeader(hasCursor = 1, x = (-1) and 0xFFFF, y = (-32768) and 0xFFFF) + byteArrayOf(0)
val f = FrameCodec.parse(raw)!!
assertEquals(-1 to -32768, f.cursor)
}
/** 光标坐标取 i16 两端边界 (32767, -32768) 往返,证明 readI16 符号扩展正确。 */
@Test
fun cursor_i16_extremes() {
val raw = goldenHeader(hasCursor = 1, x = 32767, y = -32768) + byteArrayOf(0)
val f = FrameCodec.parse(raw)!!
assertEquals(32767 to -32768, f.cursor)
}
/** 短于 30 字节的缓冲(旧 25 字节布局或截断帧)必须返回 null,不能切片越界崩溃。 */
@Test
fun rejects_truncated_buffer() {
assertNull(FrameCodec.parse(ByteArray(29)))
}
/** 恰好 30 字节(空载荷)是合法的最小帧;载荷长度为 0 而非 null。 */
@Test
fun accepts_exact_header_with_empty_payload() {
val raw = goldenHeader(hasCursor = 1, x = 100, y = 200)
val f = FrameCodec.parse(raw)!!
assertEquals(100 to 200, f.cursor)
assertEquals(0, f.payload.size)
}
}
@@ -0,0 +1,82 @@
package com.verify.remote
import android.view.KeyEvent
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Test
/**
* 3a/U7 首批 JVM 单测:扫描码映射表(此前只能真机点验)。
*
* `KeyEvent.KEYCODE_*` 是 Java 编译期常量(static final int),引用会被内联进
* 测试字节码——JVM 单测不触发 android.jar 桩类加载,所以这里不需要 Robolectric。
* 期望值来自项目内部权威源 `viewer/src/keymap.rs`(同 R-1:不信记忆,对照项目已有用法)。
*/
class WinScancodeTest {
@Test
fun enter_is_make_0x1C_not_extended() {
assertEquals(WinKey(0x1C, false), WinScancode.of(KeyEvent.KEYCODE_ENTER))
}
@Test
fun backspace_is_0x0E() {
assertEquals(WinKey(0x0E, false), WinScancode.of(KeyEvent.KEYCODE_DEL))
}
@Test
fun forward_delete_is_0x53_extended() {
assertEquals(WinKey(0x53, true), WinScancode.of(KeyEvent.KEYCODE_FORWARD_DEL))
}
@Test
fun arrows_are_extended() {
assertEquals(WinKey(0x48, true), WinScancode.of(KeyEvent.KEYCODE_DPAD_UP))
assertEquals(WinKey(0x50, true), WinScancode.of(KeyEvent.KEYCODE_DPAD_DOWN))
assertEquals(WinKey(0x4B, true), WinScancode.of(KeyEvent.KEYCODE_DPAD_LEFT))
assertEquals(WinKey(0x4D, true), WinScancode.of(KeyEvent.KEYCODE_DPAD_RIGHT))
}
@Test
fun right_shift_is_NOT_extended_with_its_own_scancode() {
// 微软 Keyboard Input Overview:右 Shift 不是扩展键,它有独立扫描码 0x36
// (2b 的 D4 偏差修正,回归锁死)
assertEquals(WinKey(0x36, false), WinScancode.of(KeyEvent.KEYCODE_SHIFT_RIGHT))
assertEquals(WinKey(0x2A, false), WinScancode.of(KeyEvent.KEYCODE_SHIFT_LEFT))
}
@Test
fun right_ctrl_and_alt_are_extended_shared_scancodes() {
assertEquals(WinKey(0x1D, true), WinScancode.of(KeyEvent.KEYCODE_CTRL_RIGHT))
assertEquals(WinKey(0x1D, false), WinScancode.of(KeyEvent.KEYCODE_CTRL_LEFT))
assertEquals(WinKey(0x38, true), WinScancode.of(KeyEvent.KEYCODE_ALT_RIGHT))
}
@Test
fun letters_and_digits_match_keymap_rs() {
// 抽查与 keymap.rs 交集的字母/数字(Set-1 make code)
assertEquals(WinKey(0x1E, false), WinScancode.of(KeyEvent.KEYCODE_A))
assertEquals(WinKey(0x2C, false), WinScancode.of(KeyEvent.KEYCODE_Z))
assertEquals(WinKey(0x02, false), WinScancode.of(KeyEvent.KEYCODE_1))
assertEquals(WinKey(0x0B, false), WinScancode.of(KeyEvent.KEYCODE_0))
assertEquals(WinKey(0x39, false), WinScancode.of(KeyEvent.KEYCODE_SPACE))
}
@Test
fun unmapped_and_unknown_keycodes_return_null() {
// KEYCODE_UNKNOWN=0:软键盘可打印字符的形态,必须落 null 让调用方走 Text 通道,
// 不能兜底成任意扫描码(2b D13 的根因之一)
assertNull(WinScancode.of(KeyEvent.KEYCODE_UNKNOWN))
assertNull(WinScancode.of(0))
assertNull(WinScancode.of(-1))
assertNull(WinScancode.of(9999))
}
@Test
fun f_keys_pinned_until_u6_physical_keyboard_verification() {
// N5/U6:F1-F10 来自 Set-1 常识与 keymap.rs 交集,未在 Windows 逐一对账;
// 这里锁「当前值不变」,防止无声漂移——真机接物理键盘验证后(U6)才允许改期望值
assertEquals(WinKey(0x3B, false), WinScancode.of(KeyEvent.KEYCODE_F1))
assertEquals(WinKey(0x44, false), WinScancode.of(KeyEvent.KEYCODE_F10))
}
}
+259 -3
View File
@@ -16,7 +16,16 @@ use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// ///
/// - v1:初版(无版本字段,隐式) /// - v1:初版(无版本字段,隐式)
/// - v2:`Auth`/`AuthOk` 携带版本;帧头增加 `seq`/`keyframe` /// - v2:`Auth`/`AuthOk` 携带版本;帧头增加 `seq`/`keyframe`
pub const PROTO_VERSION: u16 = 2; /// - v3:新增 `InputEvent::Text`(文本注入通道,UTF-16 code unit)。
/// `InputEvent` 因携带 `Vec<u16>` 不再是 `Copy`;三端必须同步升级,
/// 旧端由 `Auth` 握手显式拒绝。
/// - v4:新增 `InputEvent::MouseMoveRel`(相对鼠标,单位=被控端桌面物理像素)。
/// bincode 按位置编码枚举判别序号,新增变体会改变其后变体的编号,
/// 故同样必须三端同步;旧端由 `Auth` 握手显式拒绝。
/// - v5:`VideoFrameHeader` 尾部新增 `cursor`(本帧发送时刻的远端光标位置)。
/// 字段追加同样改变线格式(旧端会在反序列化时少读/多读而错位),故必须三端同步;
/// 旧端由 `Auth` 握手显式拒绝。语义见 [`VideoFrameHeader::cursor`]。
pub const PROTO_VERSION: u16 = 5;
/// 控制通道消息(双向) /// 控制通道消息(双向)
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -47,9 +56,12 @@ pub enum MouseButton {
Middle, Middle,
} }
/// 输入事件。鼠标坐标为虚拟桌面归一化 0..=65535; /// 输入事件。鼠标坐标为**被采集显示器**(主屏)归一化 0..=65535
/// (协议 v2 语义,v3 未改;`agent/src/input.rs` 的注入空间与此一致,见个人层 lessons R-4);
/// 按键为 Set-1 扫描码,extended 标记扩展键(方向键/Home 等,需要 EXTENDEDKEY 标志)。 /// 按键为 Set-1 扫描码,extended 标记扩展键(方向键/Home 等,需要 EXTENDEDKEY 标志)。
#[derive(Debug, Clone, Copy, Serialize, Deserialize)] ///
/// v3 起本类型不再 `Copy`:`Text` 携带堆上 `Vec<u16>`。需要复制的地方用 `.clone()`。
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum InputEvent { pub enum InputEvent {
MouseMove { x: u16, y: u16 }, MouseMove { x: u16, y: u16 },
MouseDown(MouseButton), MouseDown(MouseButton),
@@ -57,6 +69,61 @@ pub enum InputEvent {
Wheel { dy: i32 }, Wheel { dy: i32 },
KeyDown { scancode: u16, extended: bool }, KeyDown { scancode: u16, extended: bool },
KeyUp { scancode: u16, extended: bool }, KeyUp { scancode: u16, extended: bool },
/// v3 新增:**文本注入通道**。`units` 为 UTF-16 code unit 序列。
///
/// 为什么不是 `String`:Windows 注入侧(`KEYEVENTF_UNICODE`)与 Java 字符串的
/// 原生表示都是 UTF-16 code unit;用 `String` 要走 UTF-8 往返,遇到 IME 提交边界
/// 产生的落单代理项会**有损**。`Vec<u16>` 忠实透传,由接收端按 Win32 语义处理。
///
/// 为什么整段携带、而不是逐 code unit 一条消息:一次 IME 候选确认常是多个汉字;
/// 代理对(emoji 等增补平面字符)**必须在同一次注入批次里成对提交**才能被目标
/// 应用合成成一个字符,拆开注入必现乱码。
///
/// 上限 [`MAX_TEXT_UNITS`]:超长输入截断且两侧打警告日志(不静默丢弃)。
/// 另一道防线是 [`read_control`] 的 1 MiB 消息长度闸。
///
/// 日志纪律:中间层与注入端只允许打印 `units.len()`,**不得**打印用户输入内容。
Text { units: Vec<u16> },
/// v4 新增:**相对**鼠标位移,单位是**被控端桌面的物理像素**(不是归一化值)。
///
/// 语义:把远端指针从**当前位置**移动 `(dx, dy)`,越界部分由被控端 clamp 到桌面内。
/// 主控端只说「移动多少」,不说「去哪里」——远端指针位置始终是唯一权威,
/// 本机真鼠标同时移动也不会造成跳变。
///
/// 为什么单位是像素而不是归一化增量:`AuthOk` 已下发桌面 `width`/`height`,
/// 主控端换算一次即可;被控端拿到像素可直接与 `GetCursorPos` 同尺度相加,
/// 少一次取整(每多一次取整就多一个累积误差源)。
///
/// 为什么是 `i16`:单次触摸事件的位移远小于 32767;用定长整数让线格式保持紧凑。
///
/// **与 [`InputEvent::MouseMove`] 互斥**:绝对模式发 `MouseMove`,相对模式发本变体,
/// 由主控端的模式开关决定,同一会话内不应混用(混用不会出错,但手感会跳)。
///
/// 注入实现注意:Windows 上**不能**直接 `SendInput(MOUSEEVENTF_MOVE)` 透传
/// ——实测该路径被「指针加速」与「DPI 缩放」双重放大(本机 dx=100 实际走了 309px,
/// 且倍率随系统设置变化)。正确做法见 `agent/src/input.rs::send_mouse_rel`。
MouseMoveRel { dx: i16, dy: i16 },
}
/// [`InputEvent::Text`] 单次携带的 code unit 上限。
///
/// 与 `read_control` 的 1 MiB 消息闸是两道独立防线:前者防恶意对端把接收端分配打爆,
/// 后者防合法但异常长的 IME 提交把注入队列刷爆。4096 unit ≈ 8 KiB,
/// 远超正常 IME 一次提交(数十 unit),又足够小使单次 `SendInput` 批量仍可接受。
pub const MAX_TEXT_UNITS: usize = 4096;
/// 把文本输入截断到 [`MAX_TEXT_UNITS`],返回 `(可注入的 units, 是否发生截断)`。
///
/// 调用方**必须**在截断为真时打印日志(可见后果);本函数只负责机械截断,
/// 不自己打印——因为 `common` 同时被主控端与被控端复用,日志口径由调用方决定。
pub fn clamp_text_units(units: Vec<u16>) -> (Vec<u16>, bool) {
if units.len() > MAX_TEXT_UNITS {
let mut trimmed = units;
trimmed.truncate(MAX_TEXT_UNITS);
(trimmed, true)
} else {
(units, false)
}
} }
/// 视频帧头。载荷为 H.264 Annex-B 码流(含 00 00 00 01 起始码)。 /// 视频帧头。载荷为 H.264 Annex-B 码流(含 00 00 00 01 起始码)。
@@ -68,6 +135,15 @@ pub enum InputEvent {
/// 用于丢帧检测与弱网统计;注意它统计的是"发送序",不是"显示序" /// 用于丢帧检测与弱网统计;注意它统计的是"发送序",不是"显示序"
/// - `keyframe`:本帧是否 IDR。解码端(尤其 MediaCodec)需要它决定是否重置解码器, /// - `keyframe`:本帧是否 IDR。解码端(尤其 MediaCodec)需要它决定是否重置解码器,
/// 不必再自己扫 NAL 起始码判断 /// 不必再自己扫 NAL 起始码判断
/// - `cursor`(v5):本帧**发送时刻**的远端光标位置,被采集显示器内的**物理像素**
/// ((0,0)=采集画面左上角,与 `width`/`height` 同尺度同原点)。
/// `None` = 不可用,含三种情况:指针被隐藏(系统未显示光标)、指针在采集显示器
/// **之外**(多显示器负坐标区等——采集画面里没有它,主控端画了也是错的)、
/// 或被控端平台未实现光标采样。
/// 用途:主控端据此在画面上标记指针位置(「看不见远端鼠标」修复);它与 `ts_ms`
/// 同刻、同帧到达,不存在标记与画面的相对错位。
/// 为什么 `i16`:采集原点固定 (0,0),现实桌面尺寸远小于 32767;被控端已把
/// 越界值折叠为 `None`,到达值恒在 `[0, width)×[0, height)` 内。
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VideoFrameHeader { pub struct VideoFrameHeader {
pub width: u32, pub width: u32,
@@ -75,6 +151,7 @@ pub struct VideoFrameHeader {
pub ts_ms: u64, pub ts_ms: u64,
pub seq: u64, pub seq: u64,
pub keyframe: bool, pub keyframe: bool,
pub cursor: Option<(i16, i16)>,
} }
async fn read_u32le<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<u32> { async fn read_u32le<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<u32> {
@@ -215,6 +292,7 @@ mod tests {
ts_ms: 12345, ts_ms: 12345,
seq, seq,
keyframe, keyframe,
cursor: Some((640, 480)),
} }
} }
@@ -269,6 +347,184 @@ mod tests {
} }
} }
// ---- v3 文本注入通道 ----
/// 版本守卫:v3 是文本通道的登记点。v4 之后此断言随版本推进,
/// 保留它的作用是「任何一次版本变更都必须显式经过这里」。
#[test]
fn proto_version_is_three_after_text_event() {
assert!(
PROTO_VERSION >= 3,
"Text 通道自 v3 起存在,版本不得回退(当前 {PROTO_VERSION})"
);
}
// ---- v4 相对鼠标 ----
/// 版本守卫:v4 是相对鼠标的登记点。v5 之后此断言随版本推进为下界,
/// 与 v3 守卫同规(保留它是「任何一次版本变更都必须显式经过这里」的锚点)。
#[test]
fn proto_version_is_four_after_rel_move() {
assert!(
PROTO_VERSION >= 4,
"MouseMoveRel 自 v4 起存在,版本不得回退(当前 {PROTO_VERSION})"
);
}
/// 版本守卫:v5 是光标字段的登记点(最新一代用精确等号)。
#[test]
fn proto_version_is_five_after_cursor_field() {
assert_eq!(
PROTO_VERSION, 5,
"新增 VideoFrameHeader::cursor 后必须递增到 5"
);
}
/// 光标字段必须原样往返,含 `None`(隐藏/离屏/未实现三态都折叠为它)与负值边界。
/// `None` 是本变体的核心语义:缺它主控端就无法区分「指针在 (0,0)」与「没有指针」。
#[test]
fn cursor_field_roundtrips_option_and_i16() {
for c in [
None,
Some((0i16, 0i16)),
Some((1919, 1079)),
Some((-32768, 32767)),
Some((i16::MIN, i16::MAX)),
] {
let mut h = hdr(1, false);
h.cursor = c;
let payload = b"\x00\x00\x00\x01\x65x";
let bytes = encode_video_frame(&h, payload);
let hlen = u32::from_le_bytes(bytes[4..8].try_into().unwrap()) as usize;
let back: VideoFrameHeader = bincode::deserialize(&bytes[8..8 + hlen]).unwrap();
assert_eq!(back.cursor, c, "光标字段往返失真: {c:?}");
}
}
/// cursor 是**追加在尾部**的字段:既有字段(width/height/ts/seq/keyframe)的
/// 解码不受影响,且 `None` 变体让帧头显著变短(证明它没被塞进固定宽度里白占字节)。
#[test]
fn cursor_is_trailing_field_and_none_is_compact() {
let mut with = hdr(9, true);
with.cursor = Some((10, 20));
let mut without = hdr(9, true);
without.cursor = None;
let wb = bincode::serialize(&with).unwrap();
let wob = bincode::serialize(&without).unwrap();
// Some((i16,i16)) 比 None 多 4 字节(两个 i16),1 字节 tag 两边都有
assert_eq!(wb.len(), wob.len() + 4, "光标坐标应为 2×i16 定长附加");
// 前缀不变:去掉尾部光标字段,其余字节的解码结果完全一致(追加语义)
let h: VideoFrameHeader = bincode::deserialize(&wb).unwrap();
assert_eq!((h.width, h.height, h.seq, h.keyframe), (1920, 1080, 9, true));
}
/// 相对位移必须原样往返,**包含负值**(向左/向上移动)。
/// 负值是本变体的核心语义:绝对坐标是无符号的,相对增量必须有符号。
#[test]
fn mouse_move_rel_roundtrips_signed_deltas() {
for (dx, dy) in [(0i16, 0i16), (1, -1), (-32768, 32767), (100, -250)] {
let m = ControlMsg::Input(InputEvent::MouseMoveRel { dx, dy });
let bytes = encode_control(&m);
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
match back {
ControlMsg::Input(InputEvent::MouseMoveRel { dx: gx, dy: gy }) => {
assert_eq!((gx, gy), (dx, dy), "相对增量往返失真");
}
other => panic!("期望 Input::MouseMoveRel,实际 {other:?}"),
}
}
}
/// 判别序号守卫:`MouseMoveRel` 必须与既有变体**互不混淆**。
/// bincode 按位置编码枚举判别符,若新变体插在中间会让旧端静默错解——
/// 这条测试锁住「新增变体后各变体仍各自往返」。
#[test]
fn rel_move_does_not_alias_other_input_events() {
let rel = ControlMsg::Input(InputEvent::MouseMoveRel { dx: 7, dy: -9 });
let abs = ControlMsg::Input(InputEvent::MouseMove { x: 7, y: 9 });
let rb = encode_control(&rel);
let ab = encode_control(&abs);
assert_ne!(rb, ab, "相对与绝对移动的线格式不得相同");
match bincode::deserialize::<ControlMsg>(&rb[4..]).unwrap() {
ControlMsg::Input(InputEvent::MouseMoveRel { .. }) => {}
other => panic!("相对移动被错解为 {other:?}"),
}
}
#[test]
fn text_event_roundtrips_utf16_units() {
// 「你好」+ 一个 emoji 的代理对(U+1F600 = D83D DE00)
let units: Vec<u16> = vec![0x4F60, 0x597D, 0xD83D, 0xDE00];
let m = ControlMsg::Input(InputEvent::Text {
units: units.clone(),
});
let bytes = encode_control(&m);
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
match back {
ControlMsg::Input(InputEvent::Text { units: got }) => assert_eq!(got, units),
other => panic!("期望 Input::Text,实际 {other:?}"),
}
}
/// 落单代理项必须原样往返:IME 提交边界可能给出孤立高/低代理项,
/// 用 `String` 承载会在这里有损(这正是选 `Vec<u16>` 的理由)。
#[test]
fn text_event_keeps_lone_surrogate() {
let units = vec![0xD83Du16];
let m = ControlMsg::Input(InputEvent::Text { units: units.clone() });
let bytes = encode_control(&m);
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
match back {
ControlMsg::Input(InputEvent::Text { units: got }) => assert_eq!(got, units),
other => panic!("期望 Input::Text,实际 {other:?}"),
}
}
#[test]
fn text_units_clamped_at_boundary() {
// 边界:恰好上限不截断;超一个才截断
let exact = vec![0x61u16; MAX_TEXT_UNITS];
let (kept, truncated) = clamp_text_units(exact);
assert_eq!(kept.len(), MAX_TEXT_UNITS);
assert!(!truncated);
let over = vec![0x61u16; MAX_TEXT_UNITS + 1];
let (kept, truncated) = clamp_text_units(over);
assert_eq!(kept.len(), MAX_TEXT_UNITS);
assert!(truncated, "超限必须返回截断标志,让调用方打印警告");
let (empty, truncated) = clamp_text_units(vec![]);
assert!(empty.is_empty());
assert!(!truncated);
}
/// 空 `Text` 也必须能往返(IME 可能提交空串;调用方负责决定忽略还是报错)。
#[test]
fn text_event_empty_roundtrips() {
let m = ControlMsg::Input(InputEvent::Text { units: vec![] });
let bytes = encode_control(&m);
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
match back {
ControlMsg::Input(InputEvent::Text { units }) => assert!(units.is_empty()),
other => panic!("期望 Input::Text,实际 {other:?}"),
}
}
/// 上限内的 `Text` 必须能通过 `read_control` 的长度闸;
/// 反向验证:伪造一个超限的声明长度必须在分配前被拒。
#[tokio::test]
async fn text_within_cap_passes_control_length_gate() {
let (mut a, mut b) = duplex(1 << 20);
let msg = ControlMsg::Input(InputEvent::Text {
units: vec![0x4F60; MAX_TEXT_UNITS],
});
write_control(&mut a, &msg).await.unwrap();
match read_control(&mut b).await.unwrap() {
ControlMsg::Input(InputEvent::Text { units }) => assert_eq!(units.len(), MAX_TEXT_UNITS),
other => panic!("期望 Input::Text,实际 {other:?}"),
}
}
// ---- 视频帧头 ---- // ---- 视频帧头 ----
#[test] #[test]
+147
View File
@@ -0,0 +1,147 @@
# 2b evidence capture: byte-safe device screenshots of each acceptance state.
# ASCII-only source. Screenshots go through tools\shot.ps1 because PS 5.1 `>`
# writes UTF-16 and corrupts PNG bytes (see that file's header).
# Text/state assertions use uiautomator (lessons R-6: vivo logcat is throttled).
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$ev = "D:\working\projects\yuankong\evidence\2b"
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
# PC (controlled side) LAN address; DHCP-assigned. 2026-10-31: the stale
# .180 default silently cost proof runs after the office network moved to
# .110 - pass -PhoneIp (param) or update this line when it changes again.
# The 192.168.1.199 in step 2 is INTENTIONALLY unreachable (V5 error path).
$phoneIp = "192.168.1.110"
function Adb-Shell {
param([Parameter(Mandatory)][string[]]$Cmd)
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
$out = & $adb shell @Cmd 2>$null
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
}
finally { $ErrorActionPreference = $old }
}
function Get-Ui {
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try { & $adb pull /sdcard/ui.xml "$env:TEMP\cap-ui.xml" 2>$null | Out-Null }
finally { $ErrorActionPreference = $old }
if (-not (Test-Path "$env:TEMP\cap-ui.xml")) { return "" }
return [System.IO.File]::ReadAllText("$env:TEMP\cap-ui.xml", [System.Text.Encoding]::UTF8)
}
function Wait-Live([int]$Seconds) {
$deadline = (Get-Date).AddSeconds($Seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900
$x = Get-Ui
if ($x -match "RTT=") { return $x }
}
return ""
}
function Shot([string]$name) {
$out = Join-Path $ev $name
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out $out
}
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# drop the earlier broken captures (PS 5.1 '>' wrote them as UTF-16)
Get-ChildItem $ev -Filter "*.png" | ForEach-Object {
$b = [System.IO.File]::ReadAllBytes($_.FullName)
if ($b.Length -lt 4 -or ($b[0] -ne 0x89)) { Remove-Item $_.FullName -Force }
}
Write-Host "=== ensure fresh v3 agent ==="
$agentExe = "$repo\target\release\agent.exe"
$ag = Get-Process agent -ErrorAction SilentlyContinue
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = $null
}
if (-not $ag) {
Start-Process $agentExe -ArgumentList "--port","47800","--code","t" -WindowStyle Hidden
Start-Sleep -Seconds 1
}
Write-Host "=== 1. connect form (V4 entry state) ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity"))
Start-Sleep -Seconds 3
$ui = Get-Ui
if ($ui -notmatch "android\.widget\.EditText") { throw "form fields missing" }
Shot "4b-form.png"
Write-Host "=== 2. failure path (V5): unreachable host via intent ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host","192.168.1.199","--ei","port","47800","--es","code","t"))
# Measured 2026-10-30: an unreachable host returns to the form with a concrete
# reason after ~32s (quinn's default connect timeout). Budget 50s, not 30s -
# the earlier 30s budget made this step look like a failure when it was only
# still inside the timeout. UX follow-up (logged in the result record): during
# those 30s the panel says only "connecting" with no cancel affordance.
$deadline = (Get-Date).AddSeconds(50)
$err = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 1000
$ui = Get-Ui
if ($ui -match "android\.widget\.EditText") {
$shown = [regex]::Matches($ui, 'text="([^"]+)"') | ForEach-Object { $_.Groups[1].Value }
$err = $shown -join " | "
break
}
}
if ($err) { Shot "4b-v5-error.png" } else { throw "V5: never returned to form with an error" }
Write-Host " error state visible: yes"
Write-Host "=== 3. live session (V4 via intent) ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host",$phoneIp,"--ei","port","47800","--es","code","t"))
$ui = Wait-Live 25
if (-not $ui) { throw "session never became live" }
Shot "4b-connected-stream.png"
Write-Host " $([regex]::Match($ui,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
Write-Host "=== 4. keyboard shown (real IME bound to our InputConnection) ==="
$btns = [regex]::Matches($ui, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($btns.Count -lt 1) { throw "no keyboard button" }
$b0 = $btns[0]
$kx = [int](([int]$b0.Groups[1].Value + [int]$b0.Groups[3].Value) / 2)
$ky = [int](([int]$b0.Groups[2].Value + [int]$b0.Groups[4].Value) / 2)
[void](Adb-Shell @("input","tap","$kx","$ky"))
Start-Sleep -Seconds 3
$ime = (Adb-Shell @("dumpsys","input_method")).out
if ($ime -notmatch "mInputShown=true") { throw "IME not shown" }
if ($ime -notmatch "mHaveConnection=true") { throw "no InputConnection bound" }
Shot "4b-ime-shown.png"
Write-Host "=== 5. IME commit reaches the pipeline (diagnostic shows commit) ==="
[void](Adb-Shell @("input","text","hello"))
Start-Sleep -Milliseconds 1200
[void](Adb-Shell @("input","keyevent","62")) # SPACE: accept any pending candidate
Start-Sleep -Seconds 2
$ui = Get-Ui
$diag = [regex]::Match($ui, 'text="([^"]*RTT=[^"]*)"').Groups[1].Value
Write-Host " phone status: $diag"
Shot "4b-ime-committed.png"
Write-Host "=== 6. disconnect (back key) ==="
[void](Adb-Shell @("input","keyevent","4")) # dismiss keyboard
Start-Sleep -Milliseconds 800
[void](Adb-Shell @("input","keyevent","4")) # disconnect
Start-Sleep -Seconds 2
$ui = Get-Ui
if ($ui -match "RTT=") { throw "still connected after two BACK presses" }
Shot "4b-disconnected.png"
Write-Host "=== CAPTURE COMPLETE ==="
Get-ChildItem $ev -Filter "4b-*.png" | ForEach-Object {
$b = [System.IO.File]::ReadAllBytes($_.FullName)
$sig = ($b[0..3] | ForEach-Object { $_.ToString("X2") }) -join " "
"{0,-28} {1,9} {2}" -f $_.Name, $b.Length, $sig
}
+432
View File
@@ -0,0 +1,432 @@
# 3a connect-lifecycle proof (phone side): U8 timeout/cancel, error classes, BACK-cancel, HUD.
#
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
# Expected Chinese UI strings are matched as CODE-POINT fragments extracted from
# the Kotlin sources at authoring time (R-8: assert the actual value, and print
# what the parser really read so an empty result cannot pass silently).
#
# Judgements (V-A1 rewritten per loop-3-plan D3 redial ruling, user 2026-10-31):
# V-A2a wrong code -> AUTH class text + original-error detail line
# V-A1a unresolvable host -> RESOLVE text fast (<3s; Resolve is NOT retryable)
# V-A1b blackhole IP -> connecting shows CANCEL; tap -> actionable form <1.5s
# V-A4 BACK while connecting -> cancel to form, activity NOT finished
# V-A5 live session -> HUD has fps= and no coordinate noise (norm=/view=)
#
# The IME is disabled for the whole run (form fields then receive raw key events,
# no soft keyboard covering the button; same pattern as phone_chain_proof step 6)
# and restored in `finally`.
#
# Prereqs: vivo device online via adb, 3a APK installed, v3 agent build present.
# powershell -ExecutionPolicy Bypass -File tools\connect_lifecycle_proof.ps1 -Host_ 192.168.1.110
param(
[string]$Host_ = "192.168.1.110",
[int]$Port = 47800,
[string]$Code = "t",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
$ev = Join-Path (Split-Path $repo -Parent) "evidence\3a"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- code-point fragments extracted from Kotlin sources THIS authoring session
# (grep-verified present in ConnectErrorText.kt / MainActivity.kt): ----
# NB: build with `-join (...)`, NOT `[string](a,b)` - the latter stringifies the
# array with spaces ("连 接", len 3) and silently never matches (measured
# 2026-10-31; visibleTexts comparison then finds 'no button' for a button that
# is plainly on screen).
$F_RESOLVE = -join ([char]0x65E0,[char]0x6CD5,[char]0x89E3,[char]0x6790,[char]0x8FD9,[char]0x4E2A,[char]0x5730,[char]0x5740) # "cannot resolve this address"
$F_AUTH = -join ([char]0x9A8C,[char]0x8BC1,[char]0x7801,[char]0x9519,[char]0x8BEF) # "verification code wrong"
$F_CANCELED = -join ([char]0x8FDE,[char]0x63A5,[char]0x5DF2,[char]0x53D6,[char]0x6D88) # "connection cancelled"
$F_RECONNECT = -join ([char]0x91CD,[char]0x65B0,[char]0x8FDE,[char]0x63A5) # "reconnect"
$F_CANCEL = -join ([char]0x53D6,[char]0x6D88) # "cancel"
$F_CONNECT = -join ([char]0x8FDE,[char]0x63A5) # "connect"
$F_KILLMSG = -join ([char]0x8FDE,[char]0x63A5,[char]0x5DF2,[char]0x65AD,[char]0x5F00) # "connection lost"
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try { $out = & $adb @AdbArgs 2>$null; @{ out = ($out -join "`n"); code = $LASTEXITCODE } }
finally { $ErrorActionPreference = $old }
}
# NB: the concat must be parenthesized. `Adb @("shell") + $Cmd` parses `+` as a
# literal extra argument (measured 2026-10-31: every shell call silently ran
# `adb shell + ...` on the device, returning empty output - the form 'vanished'
# because uiautomator never actually ran).
function AdbShell {
param([string[]]$Cmd)
(Adb -AdbArgs (@("shell") + $Cmd)).out
}
function Get-UiText {
$d = AdbShell @("uiautomator","dump","/sdcard/cl.xml")
$r = Adb @("pull","/sdcard/cl.xml","$env:TEMP\cl.xml")
if ($r.code -ne 0 -or -not (Test-Path "$env:TEMP\cl.xml")) {
Write-Host " [ui] dump=[$($d -replace "`n",' ')] pull_rc=$($r.code) -> empty"
return ""
}
$txt = [System.IO.File]::ReadAllText("$env:TEMP\cl.xml", [System.Text.Encoding]::UTF8)
if ($txt.Length -lt 200) {
# dump can 'succeed' yet hand back a stale stub; show what we really read
Write-Host " [ui] short xml ($($txt.Length) chars): $($txt.Substring(0,[Math]::Min(120,$txt.Length)))"
}
return $txt
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml,'text="([^"]+)"') | ForEach-Object { [System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value) }
}
function FormFields([string]$xml) {
[regex]::Matches($xml,'<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
}
function Center($m) {
# NB: cast to [int] BEFORE adding. A regex group .Value is a STRING and PS
# "60" + "1020" CONCATENATES ("601020") -> /2 then lands the tap ~300000px
# off-screen, silently. Measured 2026-10-31; phone_chain_proof's Center is
# the proven form - copy it, don't re-derive.
$a=[int]$m.Groups[1].Value; $b=[int]$m.Groups[2].Value
$c=[int]$m.Groups[3].Value; $d=[int]$m.Groups[4].Value
@([int](($a+$c)/2), [int](($b+$d)/2))
}
function ButtonCenter([string]$xml, [string]$want) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t -eq $want) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
return @([int](($x1+$x2)/2), [int](($y1+$y2)/2))
}
}
}
return $null
}
# Physical-key input with the IME disabled. Key codes from javap against
# android-34 (same table phone_chain_proof uses): KEYCODE_0=7..9=16,
# KEYCODE_A=29..Z=54 contiguous (T=48 verified there), PERIOD=56.
# NB1: [int]'A' converts the STRING and throws; letter anchor is [int][char]'A'.
# NB2: [int][string]'0' == 0 (parse), so digits map 7+digit (an earlier draft
# got this wrong twice: `-48`, then MINUS=14 - which is KEYCODE_7!).
# MINUS (69) is deliberately NOT supported: throw instead of typing a wrong
# key (measured 2026-10-31: a wrong MINUS mapping silently typed '7' into
# the code field and corrupted the whole fill sequence).
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 }
else { throw "unsupported char: $ch" }
[void](AdbShell @("input","keyevent","$k"))
# 35ms lost a digit on the live device (port arrived as "4780");
# 70ms gap + per-field verify-and-retry below is the honest fix.
Start-Sleep -Milliseconds 70
}
}
# which field (0..2) currently holds focus? "" if none.
function FocusedIndex([string]$xml) {
$i = -1
foreach ($n in [regex]::Matches($xml,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$i++
if ($n.Value -match 'focused="true"') { return $i }
}
}
return -1
}
# Tap a field's center and VERIFY focus landed there (2b N-series discipline:
# automation must assert its own preconditions - adb taps are not guaranteed).
function Focus-Field($c, [int]$wantIdx) {
for ($try=0; $try -lt 3; $try++) {
[void](AdbShell @("input","tap","$($c[0])","$($c[1])"))
$deadline = (Get-Date).AddSeconds(1.5)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
if ((FocusedIndex (Get-UiText)) -eq $wantIdx) { return }
}
}
throw "tap did not focus field $wantIdx after 3 tries (focused=$(FocusedIndex (Get-UiText)))"
}
function Clear-Field($c, [int]$idx) {
Focus-Field $c $idx
for ($i=0; $i -lt 40; $i++) { [void](AdbShell @("input","keyevent","67")) }
Start-Sleep -Milliseconds 150
}
function Restart-AppForm {
# vivo keeps re-sleeping mid-run (measured 2026-10-31: battery-saver + a 7s
# screen-off timeout re-locked the keyguard between steps, so uiautomator saw
# the always-on-display clock instead of the app and 'form did not appear').
# Wake + dismiss is idempotent and keeps the harness unattended-safe; if the
# device is PIN-locked a bare wake is not enough and this will fail loudly.
[void](AdbShell @("input","keyevent","224")) # WAKEUP
Start-Sleep -Milliseconds 300
[void](AdbShell @("wm","dismiss-keyguard"))
Start-Sleep -Milliseconds 500
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity")) # no extras -> form
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-UiText
if ((FormFields $x).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
$diag = (VisibleTexts (Get-UiText)) -join " || "
throw "connect form did not appear (expected >=3 EditText). screen shows: [$diag]"
}
# Fill host/port/code into the 3 form fields, asserting every precondition
# (R-8 + 2026-10-31 lesson: an adb tap does NOT guarantee focus - one failed run
# left all three type-ins in field 0 and the single 'vals[0] -ne host' check
# caught it only as a corruption symptom). Now: focus is verified per field, and
# every field's final text must equal its intended value. Returns tap time.
function FillAndConnect([string]$xml, [string]$host_, [string]$port_, [string]$code_) {
$f = FormFields $xml
if ($f.Count -lt 3) { throw "form has $($f.Count) fields" }
# NB: do NOT gather the three centers into @(a,b,c) - PowerShell array
# literals FLATTEN nested arrays, silently turning 3 coord-pairs into 6 ints;
# $cs[0] then holds only the x of field 0, every tap lands at a bogus point,
# focus never moves (measured 2026-10-31: 'focused=-1' was the honest symptom).
# Pass each call's array result directly as ONE argument.
Clear-Field (Center $f[0]) 0; TypeAscii $host_
Clear-Field (Center $f[1]) 1; TypeAscii $port_
Clear-Field (Center $f[2]) 2; TypeAscii $code_
# Verify ALL THREE fields hold their intended value; on mismatch, retype the
# offending field and re-check (max 2 retries) - a dropped key event on the
# live device is a known flake (measured: port typed as "4780"), and the
# final judgement must still be 'equals expected', never 'close enough'.
$expect = @($host_, $port_, $code_)
for ($retry=0; $retry -le 2; $retry++) {
$x2 = Get-UiText
$vals = @()
foreach ($n in [regex]::Matches($x2,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
}
}
Write-Host " form now: $($vals -join ' | ') (try $retry)"
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
$bad = -1
for ($i=0; $i -lt 3; $i++) { if ($vals[$i] -ne $expect[$i]) { $bad = $i; break } }
if ($bad -ge 0) {
$f = FormFields $x2
if ($f.Count -lt 3) { throw "form lost fields during retry" }
Clear-Field (Center $f[$bad]) $bad
TypeAscii $expect[$bad]
continue
}
break
}
if ($vals[0] -ne $host_) { throw "form input FAIL: host field is [$($vals[0])], expected [$host_]" }
if ($vals[1] -ne $port_) { throw "form input FAIL: port field is [$($vals[1])], expected [$port_]" }
if ($vals[2] -ne $code_) { throw "form input FAIL: code field is [$($vals[2])], expected [$code_]" }
$btn = $null
foreach ($want in @($F_CONNECT, $F_RECONNECT)) { $btn = ButtonCenter $x2 $want; if ($btn) { break } }
if (-not $btn) { throw "no primary connect/reconnect button found" }
$t = Get-Date
[void](AdbShell @("input","tap","$($btn[0])","$($btn[1])"))
return $t
}
function Wait-Connecting([string]$blackhole, [double]$seconds) {
# connecting state = remote panel up, bottom bar shows the CANCEL button
$deadline = (Get-Date).AddSeconds($seconds)
while ((Get-Date) -lt $deadline) {
$x = Get-UiText
if (ButtonCenter $x $F_CANCEL) { return $x }
Start-Sleep -Milliseconds 250
}
throw "never entered connecting state (no CANCEL button within ${seconds}s)"
}
Write-Host "=== 0. stale-agent guard (R-7) + IME disable (restore in finally) ==="
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and (Test-Path $AgentExe) -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " running agent predates current build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe (cargo build --release -p agent)" }
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 1; Write-Host " agent started (v3)"
} else { Write-Host " agent running and current" }
$origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim()
Write-Host " original IME: [$origIme] (N13 discipline: recorded, restored in finally)"
if ($origIme -and $origIme -ne "null") { [void](AdbShell @("ime","disable",$origIme)) }
# keep the screen awake for the whole run (measured 2026-10-31: mid-run screen
# timeout turned every later tap/keyevent into a no-op against the AOD - the
# 'cancel did not return to form' FAIL was the sleep, not the app; a second
# probe reproduced it with dump texts showing clock+battery instead of the app)
[void](AdbShell @("svc","power","stayon","true"))
try {
# =================================================================
Write-Host "=== 1. V-A2a: wrong code -> AUTH class text + detail line ==="
$x = Restart-AppForm
[void](FillAndConnect $x $Host_ "$Port" "wrongcode3a")
$deadline = (Get-Date).AddSeconds(8); $seenAuth = $false; $seenDetail = $false; $shown = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 400
$shown = ((VisibleTexts (Get-UiText)) -join " || ")
if ($shown.Contains($F_AUTH)) { $seenAuth = $true }
if ($shown.Contains("detail:")) { $seenDetail = $true }
if ($seenAuth) { break }
}
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a2-auth.png" | Out-Null
Write-Host " screen: $shown"
if (-not $seenAuth) { throw "V-A2a FAIL: no AUTH text ('$F_AUTH'); got [$shown]" }
if (-not $seenDetail) { throw "V-A2a FAIL: AUTH shown but no 'detail:' original-error line; got [$shown]" }
Write-Host " PASS: wrong code -> '$F_AUTH' + detail line"
# =================================================================
Write-Host "=== 2. V-A1a: unresolvable host -> RESOLVE fast (NOT redialed) ==="
$x = Restart-AppForm
$t = FillAndConnect $x "999.999.999.999" "$Port" $Code
$deadline = $t.AddSeconds(3); $seen = $false; $shown = ""; $elapsed = -1
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 200
$shown = ((VisibleTexts (Get-UiText)) -join " || ")
if ($shown.Contains($F_RESOLVE)) { $seen = $true; $elapsed = ((Get-Date) - $t).TotalSeconds; break }
}
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a1-resolve.png" | Out-Null
Write-Host " screen: $shown (after ${elapsed}s)"
if (-not $seen) { throw "V-A1a FAIL: no RESOLVE text ('$F_RESOLVE') within 3s; got [$shown]" }
Write-Host " PASS: unresolvable host -> '$F_RESOLVE' in ${elapsed}s (<3s proves no redial)"
# =================================================================
Write-Host "=== 3. V-A1b: blackhole IP -> CANCEL tap -> 'cancelled' form ==="
# Intent path (2026-10-31 lesson): with the form path, taps during connecting
# proved unreliable in THIS app - the EditText still holds focus when the
# panel swaps (the form's own click chain leaves focus where it was), and the
# first tap only defocuses: manual form-path probe lost the tap 4 rounds in a
# row, while the intent path (no EditText ever focused) took ONE tap and
# returned '连接已取消' instantly. Intent auto-connect reaches the very same
# connecting state (same startConnect code), so it isolates the product
# behaviour from keyboard/focus noise.
[void](AdbShell @("input","keyevent","224"))
[void](AdbShell @("wm","dismiss-keyguard"))
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host","192.0.2.1","--ei","port","$Port","--es","code",$Code))
$xc = Wait-Connecting "192.0.2.1" 6
$cancel = ButtonCenter $xc $F_CANCEL
if (-not $cancel) { throw "V-A1b FAIL: no CANCEL button in intent connecting state" }
$tc = Get-Date
[void](AdbShell @("input","tap","$($cancel[0])","$($cancel[1])"))
$deadline = $tc.AddSeconds(3); $back = $false; $shown = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
$x2 = Get-UiText
$shown = ((VisibleTexts $x2) -join " || ")
# judgement = the CANCEL-triggered text specifically (F_CANCELED), not
# merely 'back at the form' - the 8s budget would also return a form
# (with the timeout text) and that must not masquerade as a cancel PASS.
if ((FormFields $x2).Count -ge 3 -and $shown.Contains($F_CANCELED)) { $back = $true; break }
}
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a1-cancel.png" | Out-Null
Write-Host " after cancel: $shown"
if (-not $back) { throw "V-A1b FAIL: cancel tap did not surface '$F_CANCELED' within 3s; got [$shown]" }
$ms = [int]((Get-Date).Subtract($tc).TotalMilliseconds)
Write-Host " PASS: connecting -> CANCEL -> '$F_CANCELED' form in ${ms}ms"
# =================================================================
Write-Host "=== 4. V-A4: BACK while connecting -> cancel, activity survives ==="
# same intent-path rationale as step 3 (BACK also needs the app window to
# own input focus; a focused EditText would eat the first gesture instead)
[void](AdbShell @("input","keyevent","224"))
[void](AdbShell @("wm","dismiss-keyguard"))
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host","192.0.2.1","--ei","port","$Port","--es","code",$Code))
[void](Wait-Connecting "192.0.2.1" 6)
[void](AdbShell @("input","keyevent","4")) # BACK
$deadline = (Get-Date).AddSeconds(3); $atForm = $false; $shown = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
$x2 = Get-UiText
$shown = ((VisibleTexts $x2) -join " || ")
if ((FormFields $x2).Count -ge 3 -and $shown.Contains($F_CANCELED)) { $atForm = $true; break }
}
$focusLine = ((AdbShell @("dumpsys","window")) -join "`n")
$alive = $focusLine -match "com\.verify\.remote"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a4-back.png" | Out-Null
Write-Host " after BACK: activityAlive=$alive atForm=$atForm"
Write-Host " screen: $shown"
if (-not $alive) { throw "V-A4 FAIL: BACK during connect finished the activity (old super.onKeyDown regression)" }
if (-not $atForm) { throw "V-A4 FAIL: BACK during connect did not surface '$F_CANCELED' on the form" }
Write-Host " PASS: BACK during connect -> cancel + form, activity kept"
# =================================================================
Write-Host "=== 5. V-A5: live session HUD: fps= present, no coordinate noise ==="
$x = Restart-AppForm
[void](FillAndConnect $x $Host_ "$Port" $Code)
$deadline = (Get-Date).AddSeconds(12); $live = $false; $hud = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$m = [regex]::Match((Get-UiText),'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { $hud = [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value); $live = $true; break }
}
$hasFps = $hud.Contains("fps=")
$noNoise = -not ($hud.Contains("norm=") -or $hud.Contains("view="))
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a5-hud.png" | Out-Null
Write-Host " HUD: $hud"
if (-not $live) { throw "V-A5 FAIL: session never reached live state" }
if (-not $hasFps) { throw "V-A5 FAIL: HUD missing fps= ; got [$hud]" }
if (-not $noNoise) { throw "V-A5 FAIL: HUD carries coordinate noise (norm=/view=)" }
Write-Host " PASS: live HUD shows fps, no coordinate noise"
Write-Host "=== 5b. V-A3 re-check: kill agent -> 'lost' form within ~16s ==="
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
$deadline = (Get-Date).AddSeconds(16); $lost = $false; $shown = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$x3 = Get-UiText
$shown = ((VisibleTexts $x3) -join " || ")
if ($shown.Contains($F_KILLMSG) -and (FormFields $x3).Count -ge 3) { $lost = $true; break }
}
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-a3-lost.png" | Out-Null
Write-Host " after kill: $shown"
if (-not $lost) { throw "V-A3 FAIL: kill did not surface '$F_KILLMSG' form within 16s" }
Write-Host " PASS: agent killed -> lost-session surfaces on form (idle-timeout bound)"
Write-Host "=== 6. reconnect to a fresh agent (self-heal path) ==="
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 2
$x = Get-UiText
$btn = ButtonCenter $x $F_RECONNECT
if (-not $btn) { $x = Restart-AppForm; $btn = $null }
if ($btn) {
[void](AdbShell @("input","tap","$($btn[0])","$($btn[1])"))
$deadline = (Get-Date).AddSeconds(14); $live2 = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
if ((Get-UiText) -match "RTT=") { $live2 = $true; break }
}
if (-not $live2) { throw "step 6 FAIL: reconnect button did not reach a live session" }
Write-Host " PASS: one-tap reconnect after agent restart works"
[void](AdbShell @("input","keyevent","4")) # disconnect, free the slot
Start-Sleep -Milliseconds 600
} else {
Write-Host " (no live reconnect path exercised this run - covered by manual run)"
}
}
finally {
[void](AdbShell @("svc","power","stayon","false")) # undo the keep-awake above
if ($origIme -and $origIme -ne "null") {
[void](AdbShell @("ime","enable",$origIme))
[void](AdbShell @("ime","set",$origIme))
Write-Host " IME restored: $(((AdbShell @('settings','get','secure','default_input_method')) -join '').Trim())"
}
}
Write-Host "=== CONNECT LIFECYCLE PROOF COMPLETE ==="
+280
View File
@@ -0,0 +1,280 @@
# 4b cursor-overlay proof: marker position, off-screen fold, refresh, old-client reject.
#
# ASCII-only (PS 5.1 reads BOM-less .ps1 as GBK; repo lessons). Chinese expected
# texts are built from code points (R-8: never trust an encoding round trip).
#
# Judgements (loop-4b-plan.md section 9):
# V-D6 OLD v4 APK + new v5 agent -> form shows the version-mismatch text and
# the session never goes live. RUN THIS PHASE BEFORE INSTALLING THE NEW
# APK: -Phase reject
# V-D1 screencap cyan cluster centroid == CursorMarker expectation +/- 6 px
# for 4 desktop points incl. (48,48) ~ origin (Some(0,0) vs None: the
# marker MUST be painted at the corner, not skipped)
# V-D3 after moving the PC cursor on a quiet desktop the marker reaches the
# new spot within ~1.2s (covers the 500ms idle-resend path; the frame
# header rebuild with fresh cursor was the whole point of R-A)
# V-D7 cursor into the LEFT SECOND MONITOR (negative X, machine is dual-screen
# since 2026-11-02) -> agent folds to None -> cyan cluster GONE
#
# Prereqs: v5 release agent.exe built; new (v5) APK installed for -Phase main;
# second monitor actually present (V-D7 skips loudly if not - see probe output);
# nobody touching the real mouse during a step (gates retry, loud fail otherwise).
#
# powershell -ExecutionPolicy Bypass -File tools\cursor_proof.ps1 -Host_ 192.168.1.181 -Phase reject
# powershell -ExecutionPolicy Bypass -File tools\cursor_proof.ps1 -Host_ 192.168.1.181 -Phase main
param(
[Parameter(Mandatory)][string]$Host_,
[int]$Port = 47800,
[string]$Code = "t",
[ValidateSet("reject", "main")][string]$Phase = "main",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
. "$PSScriptRoot\proof_common.ps1"
# ---- expected Chinese fragments as code points (R-8) ----
# "connect" button label
$F_CONNECT = -join ([char]0x8FDE, [char]0x63A5)
# "reconnect"
$F_RECONNECT = -join ([char]0x91CD, [char]0x65B0, [char]0x8FDE, [char]0x63A5)
# "protocol version mismatch" error summary (ConnectErrorText KIND_VERSION)
$F_VERSION = -join ([char]0x534F, [char]0x8BAE, [char]0x7248, [char]0x672C,
[char]0x4E0D, [char]0x4E00, [char]0x81F4)
# "live" HUD prefix (yi-lian-jie = connected)
$F_LIVE = -join ([char]0x5DF2, [char]0x8FDE, [char]0x63A5)
$ev = Join-Path (Split-Path $repo -Parent) "evidence\4b"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- cyan marker pixel detector (LockBits scan, full centroid) ----
Add-Type -AssemblyName System.Windows.Forms # VirtualScreen for V-D7
if (-not ('PccCyan' -as [type])) {
Add-Type -ReferencedAssemblies System.Drawing -TypeDefinition @"
using System; using System.Drawing; using System.Drawing.Imaging;
using System.Runtime.InteropServices; using System.Collections.Generic;
public class PccCyan {
public static string Find(string path) {
using (var bmp = new Bitmap(path)) {
var rect = new Rectangle(0,0,bmp.Width,bmp.Height);
var data = bmp.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int n = data.Stride * data.Height; byte[] buf = new byte[n];
Marshal.Copy(data.Scan0, buf, 0, n);
bmp.UnlockBits(data);
// dot color = RGB(0x00,0xE5,0xFF); BGRA in memory. Loose tolerance:
// the ring is white/black; a desktop with a big cyan region would poison the
// centroid, so the total-count guard below rejects it instead of averaging.
int total = 0;
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) total++;
}
}
if (total == 0) return "none";
if (total > 4000) return "patch:" + total; // suspicious giant color field
// Centroid over ALL cyan pixels would be corrupted by a few stray ones far
// away (measured 2026-11-02: n=181 at the right spot but spread=955 - the
// desktop itself had a handful of cyan pixels elsewhere). Cluster properly:
// bin the hits on an 8px grid, take the DENSEST bin as the marker seed,
// then centroid/spread over hits within 40px of that seed only.
var bins = new Dictionary<long,int>();
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) {
long key = (long)(x/8)*1000000 + (y/8);
bins[key] = bins.ContainsKey(key) ? bins[key]+1 : 1;
}
}
}
long bestK = -1; int bestN = 0; int secondN = 0;
foreach (var kv in bins) {
if (kv.Value > bestN) { secondN = bestN; bestN = kv.Value; bestK = kv.Key; }
else if (kv.Value > secondN) { secondN = kv.Value; }
}
int bx = (int)(bestK/1000000)*8 + 4, by = (int)(bestK%1000000)*8 + 4;
long sx=0, sy=0; int cnt=0; int minx=99999999,miny=99999999,maxx=-1,maxy=-1;
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) {
if (Math.Abs(x-bx)<=40 && Math.Abs(y-by)<=40) {
sx+=x; sy+=y; cnt++;
if(x<minx)minx=x; if(x>maxx)maxx=x; if(y<miny)miny=y; if(y>maxy)maxy=y;
}
}
}
}
if (cnt == 0) return "none";
int spread = Math.Max(maxx-minx, maxy-miny);
// report seed too: harness prints it when a check fails (diagnosis, not judgement)
return string.Format("{0}|{1}|{2}|{3}|{4}|{5}|{6}", sx/(double)cnt, sy/(double)cnt, cnt, spread, total, bx, secondN);
}
}
}
"@
}
function Find-Marker([string]$pngPath) {
$r = [PccCyan]::Find($pngPath)
if ($r -eq "none") { return $null }
if ($r -like "patch:*") { throw "screenshot contains a huge cyan-ish color field ($($r.Substring(6)) px) - clear the desktop (wallpaper/browser) and re-run; refusing to average a fake marker (R-8)" }
$p = $r.Split('|')
@{ x = [double]$p[0]; y = [double]$p[1]; n = [int]$p[2]; spread = [int]$p[3]; total = [int]$p[4]; seed = "$($p[5]),$($p[6])" }
}
function Take-Shot([string]$tag) {
$dev = "/sdcard/pcc_$PID.png"
$local = Join-Path $ev "$tag.png"
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try {
[void](AdbShell @("screencap", "-p", $dev))
$r = Adb -AdbArgs @("pull", $dev, $local)
} finally { $ErrorActionPreference = $old }
if ($r.code -ne 0 -or -not (Test-Path $local)) { throw "screencap/pull failed for $tag" }
return $local
}
Write-Host "=== 0. preflight (memory / fresh v5 agent) ==="
Memory-Precheck
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
Ensure-FreshAgent $AgentExe $Port $Code
$installed = ((AdbShell @("dumpsys", "package", "com.verify.remote"))) -join "`n"
$vn = [regex]::Match($installed, 'versionName=(\S+)').Groups[1].Value
Write-Host " installed APK: $vn"
# ======================================================================
if ($Phase -eq "reject") {
Write-Host "=== V-D6: OLD (v4) APK against the NEW v5 agent must be rejected ==="
if ($vn -like "*phase4a*") {
Write-Host " (running against the v4 client - this phase MUST come before installing 4b)"
} else {
Write-Host " WARNING: installed APK is $vn - the v4-rejection test is only meaningful against a real v4 client"
}
# NOTE (first-run lesson 2026-11-02): launching via `am start --es host ...`
# does NOT dial: the intent only prefill-prefers stale SharedPreferences and
# the form just sits there ("未连接"). Rejection must be observed through the
# REAL dial path - type the form, tap connect, read the failure screen.
$x = Restart-AppForm
FillAndConnect $x $Host_ "$Port" $Code $F_CONNECT $F_RECONNECT
$deadline = (Get-Date).AddSeconds(25); $shown = $null
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 800
$ui = Get-UiRetry
$texts = (VisibleTexts $ui) -join " / "
if ($texts -match "version mismatch" -or $texts.Contains($F_VERSION)) { $shown = $texts; break }
if ($texts.Contains($F_LIVE)) { throw "V-D6 FAIL: v4 client CONNECTED to the v5 agent ('$F_LIVE' on screen) - version gate broken" }
}
if (-not $shown) { throw "V-D6 FAIL: no version-mismatch text within 25s; last screen: [$((VisibleTexts (Get-Ui)) -join ' / ')]" }
Write-Host " screen: $shown"
Write-Host " PASS: v5 agent refused the v4 client with the actionable Chinese hint"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-d6-reject.png" | Out-Null
# pollution discipline (4a): leave no live session for the next proof
[void](AdbShell @("am", "force-stop", "com.verify.remote"))
Write-Host "=== CURSOR PROOF (reject phase) COMPLETE ==="
exit 0
}
# ======================================================================
# ---- main phase: needs the NEW apk; go live once, then geometry checks ----
Write-Host "=== 1. go live (new v5 APK), capture HUD (v5 pin + desktop size) ==="
if ($vn -notlike "*phase4b*") { throw "main phase needs the 4b APK installed (found $vn)" }
$x = Restart-AppForm
FillAndConnect $x $Host_ "$Port" $Code $F_CONNECT $F_RECONNECT
$hud = Wait-LiveHud 20
if (-not $hud) { throw "never reached live: [$((VisibleTexts (Get-Ui)) -join ' / ')]" }
Write-Host " HUD: $($hud -replace "`n", ' | ')"
if (-not $hud.Contains("v5")) { throw "V-D5x FAIL: HUD does not report v5 (PROTO_LABEL out of sync?): [$hud]" }
$m = [regex]::Match($hud, '(\d+)x(\d+)')
if (-not $m.Success) { throw "cannot parse desktop size from HUD: [$hud]" }
$dW = [int]$m.Groups[1].Value; $dH = [int]$m.Groups[2].Value
Write-Host " desktop from HUD: ${dW}x${dH}"
# Marker geometry works in EITHER orientation (letterbox mapping is proportional),
# but landscape is the 4a headline case. AUTO rotates for a wide desktop - give the
# rotation a moment to land, then proceed on whatever orientation is actually up.
$rot = (Get-Date).AddSeconds(3); $d = LocalDisplay
while ((Get-Date) -lt $rot) {
$d = LocalDisplay
if ($d.w -gt $d.h) { break }
Start-Sleep -Milliseconds 250
}
Write-Host (" orientation: {0}x{1} ({2})" -f $d.w, $d.h, $(if ($d.w -gt $d.h) { "landscape" } else { "portrait" }))
Start-Sleep -Seconds 2 # let the hide timer run so the video fills its final rect
$ui = Get-UiRetry
$vb = VideoBounds $ui
if (-not $vb) { throw "video (16:9 View) not found in the tree - every coordinate expectation below would be meaningless (N21)" }
Write-Host (" video bounds: l={0} t={1} w={2} h={3} ratio err {4:N4}" -f $vb.l, $vb.t, $vb.w, $vb.h, $vb.err)
if ($vb.err -gt 0.02) { throw "video not 16:9 - geometry changed since 4a, re-derive the mapping here" }
function Expect-OnScreen([int]$cx, [int]$cy) {
# CursorMarker on-screen anchor = video view origin + mapped point.
# PS parses `a / [double]b, c` inside @() as an array construction (the type
# literal + comma is a known pitfall - measured 2026-11-02: this threw
# op_Division on Object[] / silently produced 3 elements). Parenthesize each
# element so each division is evaluated as a scalar first.
@( ($vb.l + $cx * $vb.w / [double]$dW), ($vb.t + $cy * $vb.h / [double]$dH) )
}
function Check-MarkerAt([int]$cx, [int]$cy, [string]$tag, [double]$tol = 6.0) {
Set-CursorQuiet $cx $cy
Start-Sleep -Milliseconds 700 # >= one 500ms idle frame + render
$png = Take-Shot $tag
$mk = Find-Marker $png
if (-not $mk) { throw "V-D1 FAIL [$tag]: no marker at desktop ($cx,$cy) - cyan cluster absent" }
$e = Expect-OnScreen $cx $cy
$dx = [Math]::Abs($mk.x - $e[0]); $dy = [Math]::Abs($mk.y - $e[1])
Write-Host (" [{0}] marker ({1:N0},{2:N0}) n={3} spread={4} expect ({5:N0},{6:N0}) dev=({7:N1},{8:N1})" -f `
$tag, $mk.x, $mk.y, $mk.n, $mk.spread, $e[0], $e[1], $dx, $dy)
if ($dx -gt $tol -or $dy -gt $tol) { throw "V-D1 FAIL [$tag]: deviation ($dx,$dy) > $tol px (shot: $png)" }
if ($mk.spread -gt 90) { throw "V-D1 FAIL [$tag]: cyan spread $($mk.spread)px - too wide to be the marker dot (false positive path)" }
return $mk
}
Write-Host "=== 2. V-D1: four desktop points (incl. near-origin) ==="
# (48,48) instead of (0,0): the dot radius would clip at the exact corner and
# bias the centroid; Some(0,0)-vs-None semantics are already locked by unit +
# golden tests. Corner-proximity error is bounded well below tol here.
Check-MarkerAt 48 48 "v-d1-origin" | Out-Null
Check-MarkerAt 1280 720 "v-d1-center" | Out-Null
Check-MarkerAt 2400 1300 "v-d1-lowright" | Out-Null
Check-MarkerAt 2000 300 "v-d1-upperright" | Out-Null
Write-Host " PASS: marker centroid within 6 px for all 4 points"
Write-Host "=== 3. V-D3: idle-desktop move refreshes the marker <= ~1.2s ==="
# Quiet gate, then one decisive jump; 700ms above already covers 500ms resend.
$from = Check-MarkerAt 300 1200 "v-d3-from"
$e2 = Expect-OnScreen 1600 400
Set-CursorQuiet 1600 400
Start-Sleep -Milliseconds 900
$mk3 = Find-Marker (Take-Shot "v-d3-to")
if (-not $mk3) { throw "V-D3 FAIL: marker vanished after moving the cursor on a quiet desktop" }
$dx3 = [Math]::Abs($mk3.x - $e2[0]); $dy3 = [Math]::Abs($mk3.y - $e2[1])
Write-Host (" after jump: marker ({0:N0},{1:N0}) expect ({2:N0},{3:N0}) dev=({4:N1},{5:N1})" -f `
$mk3.x, $mk3.y, $e2[0], $e2[1], $dx3, $dy3)
if ($dx3 -gt 6 -or $dy3 -gt 6) { throw "V-D3 FAIL: marker did not follow within 900ms (stale cursor field?)" }
Write-Host " PASS: marker refreshes within one idle-resend window"
Write-Host "=== 4. V-D7: cursor into the LEFT second monitor folds to None ==="
$vd = [System.Windows.Forms.SystemInformation]::VirtualScreen
if ($vd.X -ge 0) {
Write-Host " SKIP: no negative-X monitor on this machine - V-D7 (off-screen fold) cannot be tested here; the fold rule is locked by fold_cursor unit tests"
} else {
$ox = [int]($vd.X + 100) # deep into the second display (negative X)
$oy = 500
Set-CursorQuiet $ox $oy
Start-Sleep -Milliseconds 900
$mk7 = Find-Marker (Take-Shot "v-d7-second-monitor")
if ($mk7) { throw "V-D7 FAIL: marker still drawn with cursor at ($ox,$oy) outside the captured display (n=$($mk7.n) at {x=$($mk7.x);y=$($mk7.y)})" }
Write-Host " PASS: no cyan cluster with the pointer at ($ox,$oy) - folded to None end to end"
}
Write-Host "=== 5. tidy: park the pointer inside the primary screen ==="
Set-CursorQuiet ([int]($dW/2)) ([int]($dH/2))
Write-Host "=== CURSOR PROOF COMPLETE ==="
+197
View File
@@ -0,0 +1,197 @@
# 2b V7 (device side): manual emoji acceptance helper.
# ASCII-only source (PS 5.1 + BOM-less = GBK corruption; see personal lessons).
#
# WHY MANUAL: the IME's own emoji panel is invisible to uiautomator (dump shows
# only our app's nodes), so adb cannot drive emoji selection. The human taps the
# emoji; this script only prepares the receiving window and reads back what
# actually arrived - the judgement is still made against real received bytes.
#
# What this proves: the phone IME commits a supplementary-plane character
# (surrogate pair) as ONE commitText, and protocol v3 delivers both code units
# adjacently so Windows synthesises a single character.
# The Windows half is already proven mechanically (V7' in text_injection_proof).
#
# Usage:
# powershell -ExecutionPolicy Bypass -File tools\emoji_manual_proof.ps1
# ... then, when prompted, tap an emoji on the phone's keyboard.
param(
# PC (controlled side) LAN address; DHCP-assigned - pass -PhoneIp when it
# moves (see ime_chinese_proof.ps1 for the 2026-10-31 stale-default lesson).
[string]$PhoneIp = "192.168.1.110",
[int]$Port = 47800,
[string]$Code = "t",
[int]$WaitSeconds = 120
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
# UNIQUE per run: a fixed path lets the startup Remove-Item below delete the
# stop.flag that leftover targets from a previous FAILED run are watching, so
# they never retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-emoji-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid"
$stopFile = "$tmp\stop.flag"
$resultFile = "$tmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public static class FgEmoji {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
}
"@
function Adb-Shell {
param([Parameter(Mandatory)][string[]]$Cmd)
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
$out = & $adb shell @Cmd 2>$null
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
}
finally { $ErrorActionPreference = $old }
}
function Wait-File {
param([string]$Path, [int]$TimeoutMs = 20000)
$t = 0
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
}
function Read-Received {
if (-not (Test-Path $resultFile)) { return "" }
$b = [System.IO.File]::ReadAllBytes($resultFile)
$s = [System.Text.Encoding]::UTF8.GetString($b)
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
return $s
}
function Show-Cps([string]$s) {
($s.ToCharArray() | ForEach-Object { "U+{0:X4}" -f [int]$_ }) -join " "
}
Write-Host "=== 1. retire earlier targets, ensure fresh agent ==="
$flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -like "*remote*proof*" })
foreach ($f in $flags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
if ($flags.Count -gt 0) { Start-Sleep -Seconds 3 }
$agentExe = "$repo\target\release\agent.exe"
$ag = Get-Process agent -ErrorAction SilentlyContinue
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = $null
}
if (-not $ag) {
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
Start-Sleep -Seconds 1
Write-Host " agent started"
}
Write-Host "=== 2. target window (receives the injected text) ==="
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
Wait-File $pidFile
$parts = @((Get-Content $pidFile -Raw) -split "`n")
$tid = [int]$parts[0].Trim()
$hwnd = [IntPtr][long]$parts[1].Trim()
Start-Sleep -Milliseconds 800
$fg = [FgEmoji]::GetForegroundWindow()
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64()) foreground=$($fg -eq $hwnd)"
Write-Host "=== 3. phone session + show keyboard ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host",$PhoneIp,"--ei","port","$Port","--es","code",$Code))
$deadline = (Get-Date).AddSeconds(25)
$ui = ""
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900
[void](Adb-Shell @("uiautomator","dump","/sdcard/ui.xml"))
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try { & $adb pull /sdcard/ui.xml "$tmp\ui.xml" 2>$null | Out-Null } finally { $ErrorActionPreference = $old }
if (Test-Path "$tmp\ui.xml") { $ui = [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8) }
if ($ui -match "RTT=") { break }
}
if ($ui -notmatch "RTT=") { throw "phone did not connect" }
$btns = [regex]::Matches($ui, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($btns.Count -lt 1) { throw "keyboard button not found" }
$b0 = $btns[0]
$kx = [int](([int]$b0.Groups[1].Value + [int]$b0.Groups[3].Value) / 2)
$ky = [int](([int]$b0.Groups[2].Value + [int]$b0.Groups[4].Value) / 2)
[void](Adb-Shell @("input","tap","$kx","$ky"))
Start-Sleep -Seconds 3
$ime = (Adb-Shell @("dumpsys","input_method")).out
if ($ime -notmatch "mInputShown=true") { throw "IME not shown" }
if ($ime -notmatch "mHaveConnection=true") { throw "IME not bound to our InputConnection" }
Write-Host " IME shown and bound: $([regex]::Match($ime,'mCurId=(\S+)').Groups[1].Value)"
$baseline = Read-Received
Write-Host ""
Write-Host " ------------------------------------------------------------"
Write-Host " NOW ON THE PHONE: tap the emoji key, then tap ONE emoji."
Write-Host " (Keyboard is up; the Windows target window is receiving.)"
Write-Host " Waiting up to $WaitSeconds s for a supplementary-plane character..."
Write-Host " ------------------------------------------------------------"
Write-Host ""
$deadline = (Get-Date).AddSeconds($WaitSeconds)
$got = ""
$found = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$got = Read-Received
if ($got.Length -gt $baseline.Length) {
$new = $got.Substring($baseline.Length)
# any code unit in the surrogate range means an emoji (or other
# supplementary-plane char) reached us
foreach ($ch in $new.ToCharArray()) {
$cp = [int]$ch
if ($cp -ge 0xD800 -and $cp -le 0xDFFF) { $found = $true; break }
}
if ($found) { break }
}
}
$final = Read-Received
$newFinal = if ($final.Length -ge $baseline.Length) { $final.Substring($baseline.Length) } else { "" }
Write-Host "=== 4. result ==="
Write-Host " baseline : [$(Show-Cps $baseline)]"
Write-Host " final : [$(Show-Cps $final)]"
Write-Host " new chars: [$(Show-Cps $newFinal)]"
Write-Host " new units: $($newFinal.Length)"
if ($found) {
# verify the pair is adjacent and forms a real code point
$units = $newFinal.ToCharArray() | ForEach-Object { [int]$_ }
$paired = $false
for ($i = 0; $i -lt $units.Count - 1; $i++) {
if ($units[$i] -ge 0xD800 -and $units[$i] -le 0xDBFF -and
$units[$i+1] -ge 0xDC00 -and $units[$i+1] -le 0xDFFF) {
$cp = 0x10000 + (($units[$i] - 0xD800) * 0x400) + ($units[$i+1] - 0xDC00)
Write-Host (" V7 PASS: adjacent surrogate pair -> U+{0:X4} ({1} code units committed as one)" -f $cp, $newFinal.Length)
$paired = $true
break
}
}
if (-not $paired) { throw "V7 FAIL: surrogate units present but not an adjacent valid pair" }
if ($newFinal.Contains([char]0xFFFD)) { throw "V7 FAIL: U+FFFD present (pair was split)" }
}
else {
Write-Host " V7 INCONCLUSIVE: no supplementary-plane character arrived within $WaitSeconds s"
Write-Host " (If you tapped an emoji and nothing arrived, that IS the finding - record it.)"
}
Write-Host "=== 5. cleanup ==="
[void](Adb-Shell @("input","keyevent","4"))
New-Item -ItemType File -Force -Path $stopFile | Out-Null
Start-Sleep -Milliseconds 500
Write-Host "=== done ==="
+302
View File
@@ -0,0 +1,302 @@
# 2b V6 on-device CHINESE: drive the IME through pinyin + candidate selection.
# ASCII-only source. Non-ASCII expectations are built from code points.
#
# Measured path differences (2026-10-30, vivo S6 + iFlytek IME):
# - `adb input keyevent <letter>` behaves like a HARDWARE keyboard: the IME
# does not compose, the event lands in Activity.onKeyDown and our printable
# unicodeChar branch sends it as text. So keyevents CANNOT exercise Chinese
# candidate selection.
# - `adb input text <latin>` is delivered to the IME, which composes it;
# SPACE (keyevent 62) then accepts the highlighted candidate, and the Hanzi
# arrive at RemoteInputConnection.commitText.
# This script therefore types pinyin with `input text` and commits with SPACE.
#
# Acceptance: the Windows target window receives the exact Han characters the
# IME committed - Chinese input works end to end over protocol v3.
# Not covered: swipe/handwriting input, browsing the candidate list.
param(
# PC (controlled side) LAN address. DHCP-assigned and DRIFTED twice
# (.180 -> .110 -> .112); a stale default silently dials a dead host
# (measured 2026-11-01: this run failed "phone did not connect" because the
# caller passed -Host_, which does NOT bind to this parameter, so the
# default was used). Always pass -PhoneIp explicitly and check the real IP.
[string]$PhoneIp = "192.168.1.112",
[int]$Port = 47800,
[string]$Code = "t"
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
# UNIQUE per-run dir: with a fixed path, the startup Remove-Item deletes the
# stop.flag that leftover targets from a previous failed run are watching, so
# they can never retire and they pollute the shared received.txt
# (same class of bug measured 2026-11-01 in phone_chain_proof).
$tmp = "$env:TEMP\remote-cjk2-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid"
$stopFile = "$tmp\stop.flag"
$resultFile = "$tmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
using System.Text;
public static class Fg5 {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr h, out uint pid);
[DllImport("user32.dll", CharSet=CharSet.Auto)] public static extern int GetWindowText(IntPtr h, StringBuilder s, int n);
public static string Desc() {
IntPtr h = GetForegroundWindow(); uint pid; GetWindowThreadProcessId(h, out pid);
var sb = new StringBuilder(256); GetWindowText(h, sb, 256);
return h.ToInt64() + " [" + sb.ToString() + "] pid=" + pid;
}
}
"@
function Adb-Shell {
param([Parameter(Mandatory)][string[]]$Cmd)
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
$out = & $adb shell @Cmd 2>$null
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
}
finally { $ErrorActionPreference = $old }
}
function Wait-File {
param([string]$Path, [int]$TimeoutMs = 20000)
$t = 0
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
}
function Assert-Foreground([IntPtr]$hwnd, [string]$where) {
$deadline = (Get-Date).AddMilliseconds(2500)
while ((Get-Date) -lt $deadline) {
if ([Fg5]::GetForegroundWindow() -eq $hwnd) { return }
Start-Sleep -Milliseconds 50
}
throw "$where : foreground is $([Fg5]::Desc()), want target $($hwnd.ToInt64()) - refusing to inject"
}
function Get-Ui {
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try { & $adb pull /sdcard/ui.xml "$tmp\ui.xml" 2>$null | Out-Null }
finally { $ErrorActionPreference = $old }
if (-not (Test-Path "$tmp\ui.xml")) { return "" }
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
}
function Read-Received {
if (-not (Test-Path $resultFile)) { return "" }
$b = [System.IO.File]::ReadAllBytes($resultFile)
$s = [System.Text.Encoding]::UTF8.GetString($b)
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
return $s
}
function Show-Cps([string]$s) {
($s.ToCharArray() | ForEach-Object { "U+{0:X4}" -f [int]$_ }) -join " "
}
# ---- 4a landscape adaptations ----
# In landscape the HUD TextView hides WITH the auto-hiding control bar, so a
# plain "RTT=" read returns nothing while the session is alive, and the
# keyboard button is not in the tree at all. Both reads must wake through the
# letterbox black bar first (that tap is consumed by the app; it never reaches
# the remote). NB: this script's Adb-Shell returns @{out;code}, not a string.
function Phone-DisplaySize {
$d = (Adb-Shell @("dumpsys", "window", "displays")).out
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 1080; h = 2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 1080; h = 2400 } }
return @{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake-Controls-If-Hidden([string]$xml) {
$hidden = ($xml -match 'class="android\.view\.View"') -and ($xml -notmatch 'class="android\.widget\.Button"')
if (-not $hidden) { return $false }
$pd = Phone-DisplaySize
[void](Adb-Shell @("input", "tap", "100", "$([int]($pd.h / 2))"))
Start-Sleep -Milliseconds 400
return $true
}
# Read the tree until it shows a live session, waking the controls as needed.
function Get-Ui-Live([int]$Seconds) {
$deadline = (Get-Date).AddSeconds($Seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900
$x = Get-Ui
if ($x -match "RTT=") { return $x }
if (Wake-Controls-If-Hidden $x) {
$x = Get-Ui
if ($x -match "RTT=") { return $x }
}
}
return (Get-Ui)
}
# Find a Button by label prefix in a tree that may have auto-hidden controls.
function Find-ButtonByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
return @(
[int]((([int]$b.Groups[1].Value) + ([int]$b.Groups[3].Value)) / 2),
[int]((([int]$b.Groups[2].Value) + ([int]$b.Groups[4].Value)) / 2),
$t)
}
}
}
return $null
}
function Find-ButtonByPrefix-Live([string]$prefix) {
for ($t=0; $t -lt 4; $t++) {
$x = Get-Ui
$b = Find-ButtonByPrefix $x $prefix
if ($b) { return $b }
if (-not (Wake-Controls-If-Hidden $x)) { Start-Sleep -Milliseconds 400 }
}
return $null
}
# Force PORTRAIT for this proof. Why: the IME's ab/zh language key is NOT visible
# to uiautomator (its key views are separate windows), so the toggle is reached by
# a CALIBRATED tap that is portrait-specific (146,2096 on 1080x2400). In landscape
# that y is off-screen, so the retry path could not work. The text pipeline itself
# is orientation-independent (landscape 'abc' is already covered by
# phone_chain_proof), so running V6 in portrait costs nothing in coverage.
function Set-Orientation-Portrait {
$F_ORIENT = -join ([char]0x65B9,[char]0x5411,[char]0x3A) # "fangxiang:"
$F_PORTRAIT = $F_ORIENT + -join ([char]0x7AD6,[char]0x5C4F) # "...:portrait"
for ($t=0; $t -lt 8; $t++) {
$b = Find-ButtonByPrefix-Live $F_ORIENT
if (-not $b) { throw "orientation button not reachable" }
if ($b[2] -eq $F_PORTRAIT) { Write-Host " orientation: already portrait"; return }
[void](Adb-Shell @("input","tap","$($b[0])","$($b[1])"))
Start-Sleep -Milliseconds 1300
}
throw "could not reach portrait orientation after 8 taps"
}
Write-Host "=== 0. retire ALL earlier targets, verify none holds foreground ==="
$flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -like "*remote*cjk*" -or $_.FullName -like "*remote*proof*" })
foreach ($f in $flags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
if ($flags.Count -gt 0) { Start-Sleep -Seconds 3 }
$hosts = @(Get-CimInstance Win32_Process -Filter "Name='powershell.exe'" |
Where-Object { $_.CommandLine -match 'text_target\.ps1' -and $_.ProcessId -ne $PID })
foreach ($hx in $hosts) { Stop-Process -Id $hx.ProcessId -Force -ErrorAction SilentlyContinue }
if ($hosts.Count -gt 0) { Start-Sleep -Seconds 1 }
Write-Host " retired $($flags.Count) flag(s), killed $($hosts.Count) host(s); fg now: $([Fg5]::Desc())"
Write-Host "=== 1. agent freshness ==="
$agentExe = "$repo\target\release\agent.exe"
$ag = Get-Process agent -ErrorAction SilentlyContinue
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = $null
}
if (-not $ag) {
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
Start-Sleep -Seconds 1
}
Write-Host "=== 2. fresh target window (empty field) ==="
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
Wait-File $pidFile
$parts = @((Get-Content $pidFile -Raw) -split "`n")
$tid = [int]$parts[0].Trim()
$hwnd = [IntPtr][long]$parts[1].Trim()
Assert-Foreground $hwnd "after launch"
$base0 = Read-Received
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64()) baseline=[$(Show-Cps $base0)]"
Write-Host "=== 3. phone session ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host",$PhoneIp,"--ei","port","$Port","--es","code",$Code))
$ui = Get-Ui-Live 25
if ($ui -notmatch "RTT=") { throw "phone did not connect" }
Write-Host " live: $([regex]::Match($ui,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
# 4a: pin PORTRAIT before driving the IME (calibrated toggle tap is portrait-only)
Set-Orientation-Portrait
Write-Host "=== 4. show IME (tap keyboard button) ==="
# Select the keyboard button BY ITS LABEL, not by position.
# 3b added a sensitivity row (- / +) above the button bar, so "the first Button"
# is now the minus sign - the old positional assumption silently tapped the wrong
# control and the proof reported "IME not shown" (a harness defect, not a product
# regression; measured 2026-10-31). Label matching is order-independent.
$F_KEYBOARD = -join ([char]0x952E,[char]0x76D8) # "keyboard"
$kb = Find-ButtonByPrefix-Live $F_KEYBOARD
if (-not $kb) { throw "keyboard button not found by label" }
[void](Adb-Shell @("input","tap","$($kb[0])","$($kb[1])"))
Start-Sleep -Seconds 3
$ime = (Adb-Shell @("dumpsys","input_method")).out
if ($ime -notmatch "mInputShown=true") { throw "IME not shown" }
if ($ime -notmatch "mHaveConnection=true") { throw "IME not bound to our InputConnection" }
Write-Host " IME: $([regex]::Match($ime,'mCurId=(\S+)').Groups[1].Value)"
Write-Host "=== 4b/5. pinyin -> IME candidate -> commitText (mode-tolerant) ==="
# The IME may remember an ENGLISH mode, in which case letters pass through
# without composing - which looks identical to "our pipeline lost the Hanzi"
# unless you check. The IME's key views are NOT visible to uiautomator (its
# dump contains only our app's nodes), so the zh/en toggle is reached by a
# calibrated tap. A stray tap here changes a keyboard setting, nothing
# destructive. The loop below is idempotent by design (R-9): flip only if
# letters-without-Hanzi were seen, and the judgement is the received text.
#
# Toggle calibration (2026-11-01): the device IME is now SOGOU
# (com.sohu.inputmethod.sogou.vivo), not iFlytek - the old (146,2096) iFlytek
# coordinate silently missed the key and both attempts stayed in English mode.
# Re-measured from screencap: bottom row, the language key right of '.'
# (labelled zh/en) at device px (838,2282) on 1080x2400 portrait.
$modeToggle = @(838, 2282)
$expect = -join @([char]0x4F60, [char]0x597D) # NI HAO
$attempt = 0
$new = ""
while ($attempt -lt 2) {
$attempt++
$base = Read-Received
Assert-Foreground $hwnd "attempt ${attempt}: before pinyin"
[void](Adb-Shell @("input","text","nihao"))
Start-Sleep -Milliseconds 1500
$mid = Read-Received
Write-Host " attempt ${attempt} after pinyin (composing): [$(Show-Cps $mid)]"
Assert-Foreground $hwnd "attempt ${attempt}: before SPACE"
[void](Adb-Shell @("input","keyevent","62"))
Start-Sleep -Seconds 2
$got = Read-Received
$new = if ($got.Length -ge $base.Length) { $got.Substring($base.Length) } else { "" }
Write-Host " attempt ${attempt} newly received: [$(Show-Cps $new)]"
if ($new -match [regex]::Escape($expect)) { break }
if ($attempt -eq 1) {
Write-Host " no Hanzi yet -> flipping IME language mode and retrying once"
[void](Adb-Shell @("input","tap","$($modeToggle[0])","$($modeToggle[1])"))
Start-Sleep -Seconds 2
}
}
$diag = [regex]::Match((Get-Ui), 'text="([^"]*RTT=[^"]*)"').Groups[1].Value
Write-Host " phone status: $diag"
Write-Host " expected to contain: $(Show-Cps $expect)"
if ($new -notmatch [regex]::Escape($expect)) {
Write-Host " GROUND TRUTH: [$(Show-Cps $new)]"
Write-Host " U+006E.. letters only => IME stayed in English mode (re-calibrate the toggle tap)"
throw "V6 CHINESE FAIL: IME did not deliver Hanzi through commitText"
}
Write-Host " V6 PASS: pinyin -> IME candidate -> commitText -> v3 Text -> Windows Hanzi"
Write-Host "=== 6. cleanup ==="
[void](Adb-Shell @("input","keyevent","4"))
New-Item -ItemType File -Force -Path $stopFile | Out-Null
Start-Sleep -Milliseconds 500
Write-Host "=== CHINESE IME PROOF COMPLETE ==="
+188
View File
@@ -0,0 +1,188 @@
# 2b V6/V7 on-device IME path: does a real IME's commitText reach the remote
# desktop? ASCII-only source (see personal lessons on PS 5.1 + BOM-less .ps1).
#
# Difference from phone_chain_proof.ps1: that one drives Activity.onKeyDown with
# adb keyevents (IME hidden). Here the IME is SHOWN, so keys are consumed by the
# input method and reach us only through RemoteInputConnection.commitText /
# sendKeyEvent. That is the path Chinese candidates take, and it is the risk R1
# item in the plan (vivo ROM + third-party IME behaviour was never observed).
#
# What this proves mechanically: IME is bound to our InputConnection and its
# committed text lands in a Windows window we own, via protocol v3.
# What it cannot prove: human candidate-list selection (needs a finger); see
# the result record - V6/V7 full judgement is listed as manual acceptance.
#
# Usage:
# powershell -ExecutionPolicy Bypass -File tools\ime_commit_proof.ps1
param(
# PC (controlled side) LAN address; DHCP-assigned - pass -PhoneIp when it
# moves (see ime_chinese_proof.ps1 for the 2026-10-31 stale-default lesson).
[string]$PhoneIp = "192.168.1.110",
[int]$Port = 47800,
[string]$Code = "t",
[string]$Text = "hi"
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
# UNIQUE per run: a fixed path lets the startup Remove-Item below delete the
# stop.flag that leftover targets from a previous FAILED run are watching, so
# they never retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-ime-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid"
$stopFile = "$tmp\stop.flag"
$resultFile = "$tmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public static class FgApi2 {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
}
"@
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
$out = & $adb @AdbArgs 2>$null
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
}
finally { $ErrorActionPreference = $old }
}
function Adb-Shell { param([Parameter(Mandatory)][string[]]$Cmd) return Adb -AdbArgs (@("shell") + $Cmd) }
function Wait-File {
param([string]$Path, [int]$TimeoutMs = 20000)
$t = 0
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
}
function Assert-Foreground([IntPtr]$hwnd, [string]$where) {
$deadline = (Get-Date).AddMilliseconds(2000)
while ((Get-Date) -lt $deadline) {
if ([FgApi2]::GetForegroundWindow() -eq $hwnd) { return }
Start-Sleep -Milliseconds 30
}
throw "$where : foreground=$([FgApi2]::GetForegroundWindow()) want=$($hwnd.ToInt64()) - refusing to inject"
}
function Get-PhoneUiText {
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
$r = Adb -AdbArgs @("pull", "/sdcard/ui.xml", "$tmp\ui.xml")
if ($r.code -ne 0 -or -not (Test-Path "$tmp\ui.xml")) { return "" }
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
}
function Read-Received {
if (-not (Test-Path $resultFile)) { return "" }
$b = [System.IO.File]::ReadAllBytes($resultFile)
$s = [System.Text.Encoding]::UTF8.GetString($b)
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
return $s
}
Write-Host "=== 0. retire earlier targets (touch only existing stop flags) ==="
$flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -like "*remote*proof*" })
if ($flags.Count -gt 0) {
Write-Host " retiring $($flags.Count)"
foreach ($f in $flags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
Start-Sleep -Seconds 3
}
Write-Host "=== 1. fresh target window ==="
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
Wait-File $pidFile
$parts = @((Get-Content $pidFile -Raw) -split "`n")
$tid = [int]$parts[0].Trim()
$hwnd = [IntPtr][long]$parts[1].Trim()
Assert-Foreground $hwnd "after launch"
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64())"
Write-Host "=== 2. agent freshness + phone session ==="
$agentExe = "$repo\target\release\agent.exe"
$ag = Get-Process agent -ErrorAction SilentlyContinue
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = $null
}
if (-not $ag) {
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
Start-Sleep -Seconds 1
}
[void](Adb-Shell @("am", "force-stop", "com.verify.remote"))
[void](Adb-Shell @("am", "start", "-n", "com.verify.remote/.MainActivity",
"--es", "host", $PhoneIp, "--ei", "port", "$Port", "--es", "code", $Code))
$deadline = (Get-Date).AddSeconds(25)
$xml = ""
while ((Get-Date) -lt $deadline) { Start-Sleep -Milliseconds 900; $xml = Get-PhoneUiText; if ($xml -match "RTT=") { break } }
if ($xml -notmatch "RTT=") { throw "phone did not connect" }
Write-Host " $([regex]::Match($xml,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
Write-Host "=== 3. show the soft keyboard (tap the app's keyboard button) ==="
# Button coordinates come from the live hierarchy. Our remote-state bar adds
# "keyboard" before "disconnect", so index 0 is the keyboard button - asserted
# rather than assumed by checking the count and printing what we tapped.
$btns = [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($btns.Count -lt 1) { throw "no buttons found in remote state" }
$b0 = $btns[0]
$kx = [int](([int]$b0.Groups[1].Value + [int]$b0.Groups[3].Value) / 2)
$ky = [int](([int]$b0.Groups[2].Value + [int]$b0.Groups[4].Value) / 2)
Write-Host " tapping first button ($kx,$ky)"
[void](Adb-Shell @("input", "tap", "$kx", "$ky"))
Start-Sleep -Seconds 3
$ime = (Adb-Shell @("dumpsys", "input_method")).out
$shown = ($ime -match "mInputShown=true")
$bound = ($ime -match "mHaveConnection=true")
$curId = [regex]::Match($ime, "mCurId=(\S+)").Groups[1].Value
Write-Host " IME: shown=$shown boundToUs=$bound cur=$curId"
if (-not $shown) { throw "soft keyboard did not appear" }
if (-not $bound) { throw "no InputConnection bound - our onCheckIsTextEditor/onCreateInputConnection path failed" }
Write-Host "=== 4. type through the IME, expect commitText -> v3 Text -> Windows ==="
$before = Read-Received
Assert-Foreground $hwnd "before IME text"
[void](Adb-Shell @("input", "text", $Text))
Start-Sleep -Milliseconds 1500
# Latin input in most Chinese IMEs commits without a candidate pick; if the IME
# instead left it composing, nudge with SPACE to force a commit.
$got = Read-Received
if ($got -notmatch [regex]::Escape($Text)) {
Write-Host " not committed yet (got [$got]); sending SPACE to accept candidate"
[void](Adb-Shell @("input", "keyevent", "62")) # KEYCODE_SPACE
Start-Sleep -Milliseconds 1500
$got = Read-Received
}
Write-Host " target received: [$got] (expected to contain [$Text])"
$diag = [regex]::Match((Get-PhoneUiText), 'text="([^"]*RTT=[^"]*)"').Groups[1].Value
Write-Host " phone status: $diag"
if ($got -notmatch [regex]::Escape($Text)) {
throw "IME COMMIT FAIL: nothing from the IME reached the Windows target"
}
Write-Host " PASS: real IME -> RemoteInputConnection -> v3 Text -> Windows window"
Write-Host "=== 5. back key must close keyboard first, not the session ==="
[void](Adb-Shell @("input", "keyevent", "4"))
Start-Sleep -Seconds 2
$ime2 = (Adb-Shell @("dumpsys", "input_method")).out
$xml2 = Get-PhoneUiText
$stillLive = ($xml2 -match "RTT=")
$kbHidden = ($ime2 -match "mInputShown=false")
Write-Host " after BACK: session live=$stillLive keyboard hidden=$kbHidden"
if (-not ($kbHidden -and $stillLive)) { throw "BACK did not behave as keyboard-first-then-session" }
Write-Host " PASS: first BACK dismissed the keyboard, session kept"
Write-Host "=== 6. cleanup ==="
New-Item -ItemType File -Force -Path $stopFile | Out-Null
Start-Sleep -Milliseconds 500
Write-Host "=== IME COMMIT PROOF COMPLETE ==="
+486
View File
@@ -0,0 +1,486 @@
# 4a landscape proof: rotation, full-height video, auto-hide controls, regressions.
#
# ASCII-only (PS 5.1 reads BOM-less .ps1 as GBK; personal lessons).
#
# Judgements (loop-4a-landscape-plan.md section 6):
# V-C1 connect to 1920x1080 -> app rotates landscape <=2s after live
# (LOCAL display cur= only: this vivo ROM has a permanent 1080x2400
# virtual display and prints no mRotation= line - probe lesson 2026-11-01)
# V-C2 with controls hidden, video view is 16:9 (err <2%) and >=90% screen height
# V-C3 frame counter strictly increases across rotation (session survives)
# V-C4 >=45 fps over a 2.5s window with an ACTIVE desktop animator
# (static desktop is source-limited ~25fps - measuring that would be a
# category error, probe lesson)
# V-C5 controls auto-hide ~3s idle; bottom-edge tap wakes them back
# V-C5b edge-band tap does NOT reach the remote (PC cursor delta <=1px)
# V-C5c all four buttons present after wake; mode tap changes and restores
# V-C6 form page is portrait
# V-C7 orientation button cycles AUTO->L->P->SYS->AUTO, display follows
#
# powershell -ExecutionPolicy Bypass -File tools\landscape_proof.ps1 -Host_ 192.168.1.112
param(
[string]$Host_ = "192.168.1.112",
[int]$Port = 47800,
[string]$Code = "t",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
$ev = Join-Path (Split-Path $repo -Parent) "evidence\4a"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- Chinese fragments as code points (R-8) ----
$F_ORIENT = -join ([char]0x65B9,[char]0x5411,[char]0x3A) # "fangxiang:" (orientation:)
$F_AUTO = $F_ORIENT + -join ([char]0x81EA,[char]0x52A8) # label: orientation AUTO
$F_LAND = $F_ORIENT + -join ([char]0x6A2A,[char]0x5C4F) # label: orientation LANDSCAPE
$F_PORT = $F_ORIENT + -join ([char]0x7AD6,[char]0x5C4F) # label: orientation PORTRAIT
$F_SYS = $F_ORIENT + -join ([char]0x7CFB,[char]0x7EDF) # label: orientation SYSTEM
$F_CONNECT = -join ([char]0x8FDE,[char]0x63A5) # label: connect
$F_RECONNECT = -join ([char]0x91CD,[char]0x65B0,[char]0x8FDE,[char]0x63A5) # label: reconnect
$F_KB = -join ([char]0x952E,[char]0x76D8) # label: keyboard
$F_MODE = -join ([char]0x6A21,[char]0x5F0F,[char]0x3A) # "moshi:" (mode:)
$F_DISC = -join ([char]0x65AD,[char]0x5F00) # label: disconnect
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try { $out = & $adb @AdbArgs 2>$null; @{ out = ($out -join "`n"); code = $LASTEXITCODE } }
finally { $ErrorActionPreference = $old }
}
function AdbShell { param([string[]]$Cmd) (Adb -AdbArgs (@("shell") + $Cmd)).out }
function Get-Ui {
[void](AdbShell @("uiautomator","dump","/sdcard/ls.xml"))
$r = Adb -AdbArgs @("pull","/sdcard/ls.xml","$env:TEMP\ls.xml")
if ($r.code -ne 0 -or -not (Test-Path "$env:TEMP\ls.xml")) { return "" }
return [System.IO.File]::ReadAllText("$env:TEMP\ls.xml", [System.Text.Encoding]::UTF8)
}
# uiautomator intermittently returns an empty/failed dump (measured 2026-11-01:
# a tap lookup got "" and the null-button tap then crashed with an empty-string
# argument). Retry until the tree is plausibly real; callers still assert on
# their own node - this only removes the transient, never masks a real absence.
function Get-UiRetry([int]$tries=4) {
$x = ""
for ($i=0; $i -lt $tries; $i++) {
$x = Get-Ui
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
Write-Host " [ui] retry exhausted, last dump len=$($x.Length)"
return $x
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml,'text="([^"]+)"') | ForEach-Object { [System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value) }
}
function ButtonByName([string]$xml, [string]$want, [bool]$prefix=$false) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
$hit = if ($prefix) { $t.StartsWith($want) } else { $t -eq $want }
if ($hit) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
return @{ cx=[int](($x1+$x2)/2); cy=[int](($y1+$y2)/2) }
}
}
}
return $null
}
function ButtonLabelByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) { return $t }
}
return $null
}
# 4a race (measured 2026-11-01): each uiautomator dump costs ~2s, comparable to
# the 3s auto-hide timer - find-then-read sequences straddle expiry and labels
# come back empty. Every observation goes through the helpers below.
#
# CRITICAL SAFETY: the wake tap (screen bottom center) is only consumed by the
# app's edge band WHEN CONTROLS ARE HIDDEN. If controls are visible, that point
# lands on the control bar (mode/orientation buttons!) and would fire a real
# action. So a wake is allowed only after positively proving the hidden state:
# video View node present AND zero Button nodes.
function ControlsHidden([string]$ui) {
$anyBtn = ($ui -match 'class="android\.widget\.Button"')
$video = ($ui -match 'class="android\.view\.View"')
return ($video -and -not $anyBtn)
}
function Wake-Band {
# 4a design change (measured 2026-11-01): the wake zone is the LETTERBOX
# BLACK BAR, not a bottom edge band. With controls hidden the video is
# full-height, so bottom-center lies INSIDE the picture and would fire a
# real remote click. x=100 is outside the video in both states.
$d = LocalDisplay
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 400
}
function Find-VisibleButton([string]$want, [bool]$prefix=$false) {
for ($try=0; $try -lt 5; $try++) {
$ui = Get-UiRetry
$b = ButtonByName $ui $want $prefix
if ($b) { return $b }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null } # real tree, genuine absence
Start-Sleep -Milliseconds 400 # empty-dump transient
}
return $null
}
function Tap-VisibleButton([string]$want, [bool]$prefix=$false) {
$b = Find-VisibleButton $want $prefix
if (-not $b) { return $null }
[void](AdbShell @("input","tap","$($b.cx)","$($b.cy)"))
return $b
}
# Label read that survives the hide race: wake only from a proven-hidden state.
function Read-ButtonLabel([string]$want, [bool]$prefix=$false) {
for ($try=0; $try -lt 6; $try++) {
$ui = Get-UiRetry
$l = ButtonLabelByPrefix $ui $want
if ($l) { return $l }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null }
Start-Sleep -Milliseconds 400
}
return $null
}
function FrameCount {
$m = [regex]::Match((Get-Ui), ([char]0x5E27 + '=(\d+)'))
if ($m.Success) { return [long]$m.Groups[1].Value }
return -1
}
# 4a self-conflict (measured 2026-11-01): once the controls auto-hide, the HUD
# TextView is GONE too - uiautomator's tree has NO text nodes and FrameCount
# returns -1 even though the session is alive (the video View is right there).
# Wake through the bottom edge band first: that tap is consumed by the app
# (zero remote events), so it cannot perturb the fps window.
function WakeControls {
$d = LocalDisplay
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 500
}
function FrameCountOrWake {
$f = FrameCount
if ($f -ge 0) { return $f }
WakeControls
return FrameCount
}
function LocalDisplay {
$d = (AdbShell @("dumpsys","window","displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w=0; h=0 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w=0; h=0 } }
return @{ w=[int]$m.Groups[1].Value; h=[int]$m.Groups[2].Value }
}
function Wake {
[void](AdbShell @("input","keyevent","224"))
Start-Sleep -Milliseconds 250
[void](AdbShell @("wm","dismiss-keyguard"))
Start-Sleep -Milliseconds 350
}
function Restart-AppForm {
Wake
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity"))
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-Ui
if (([regex]::Matches($x,'class="android\.widget\.EditText"')).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
throw "form did not appear: [$((VisibleTexts (Get-Ui)) -join ' || ')]"
}
function CenterOf($m) {
$a=[int]$m.Groups[1].Value; $b=[int]$m.Groups[2].Value
$c=[int]$m.Groups[3].Value; $e=[int]$m.Groups[4].Value
@([int](($a+$c)/2), [int](($b+$e)/2))
}
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 } else { throw "unsupported char $ch" }
[void](AdbShell @("input","keyevent","$k"))
Start-Sleep -Milliseconds 70
}
}
function ClearField($c) {
for ($try=0; $try -lt 3; $try++) {
[void](AdbShell @("input","tap","$($c[0])","$($c[1])"))
Start-Sleep -Milliseconds 250
for ($i=0; $i -lt 40; $i++) { [void](AdbShell @("input","keyevent","67")) }
Start-Sleep -Milliseconds 120
$fx = Get-Ui
$focused = $false
$fi = -1
foreach ($n in [regex]::Matches($fx,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$fi++
if ($n.Value -match 'focused="true"') { $focused = $true }
}
}
if ($focused) { return }
}
throw "field focus failed"
}
# PC-side cursor (V-C5b) + desktop animator (V-C4)
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class Cur4a {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr c);
public static void Dpi(){ SetProcessDpiAwarenessContext(new IntPtr(-4)); }
public static int[] Get(){ POINT p; GetCursorPos(out p); return new int[]{p.X,p.Y}; }
public static void Set(int x,int y){ SetCursorPos(x,y); }
}
"@
[Cur4a]::Dpi()
Write-Host "=== 0. memory precheck (probe lesson: OOM fakes failures) ==="
$os = Get-CimInstance Win32_OperatingSystem
$freeGB = $os.FreeVirtualMemory/1MB
Write-Host (" commit free: {0:N1} GB" -f $freeGB)
if ($freeGB -lt 0.5) { throw "commit memory critically low - free RAM first (gradle --stop)" }
Write-Host "=== 1. stale-agent guard (R-7) ==="
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and (Test-Path $AgentExe) -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " agent predates build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 2; Write-Host " agent started"
} else { Write-Host " agent running and current" }
$origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim()
if ($origIme -and $origIme -ne "null") { [void](AdbShell @("ime","disable",$origIme)) }
[void](AdbShell @("svc","power","stayon","true"))
try {
# =====================================================================
Write-Host "=== 2. V-C6: form is portrait ==="
$x = Restart-AppForm
$d = LocalDisplay
Write-Host (" form display: {0}x{1}" -f $d.w, $d.h)
if (-not ($d.w -lt $d.h)) { throw "V-C6 FAIL: form not portrait ($($d.w)x$($d.h))" }
Write-Host " PASS: form portrait"
# =====================================================================
Write-Host "=== 3. V-C1: connect -> AUTO rotates landscape <=2s after live ==="
$F_edits = [regex]::Matches($x,'<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
ClearField (CenterOf $F_edits[0]); TypeAscii $Host_
ClearField (CenterOf $F_edits[1]); TypeAscii "$Port"
ClearField (CenterOf $F_edits[2]); TypeAscii $Code
# Read back until 3 EditText nodes are actually present: uiautomator
# intermittently returns a truncated/empty tree (measured 2026-11-01, worse
# under memory pressure) and a single Get-Ui then reads "form now: " empty
# and fails the fill check as a false negative.
$vals = @()
for ($rd=0; $rd -lt 5; $rd++) {
$x2 = Get-UiRetry
$vals = @()
foreach ($n in [regex]::Matches($x2,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
}
}
if ($vals.Count -ge 3) { break }
Write-Host " form readback got $($vals.Count) fields, retry $rd"
Start-Sleep -Milliseconds 500
}
Write-Host " form now: $($vals -join ' | ')"
if ($vals.Count -lt 3) { throw "form fill FAIL: only $($vals.Count) fields after retries" }
if ($vals[0] -ne $Host_ -or $vals[1] -ne "$Port" -or $vals[2] -ne $Code) { throw "form fill FAIL" }
$btn = ButtonByName $x2 $F_CONNECT
if (-not $btn) { $btn = ButtonByName $x2 $F_RECONNECT }
if (-not $btn) { $btn = Find-VisibleButton $F_CONNECT; if (-not $btn) { $btn = Find-VisibleButton $F_RECONNECT } }
if (-not $btn) { throw "connect button not found" }
[void](AdbShell @("input","tap","$($btn.cx)","$($btn.cy)"))
$deadline = (Get-Date).AddSeconds(20); $live = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
if ((Get-Ui) -match "RTT=") { $live = $true; break }
}
if (-not $live) { throw "never reached live: [$((VisibleTexts (Get-Ui)) -join ' || ')]" }
$tRot = Get-Date
$deadline = $tRot.AddSeconds(2); $rotated = $false
while ((Get-Date) -lt $deadline) {
$d = LocalDisplay
if ($d.w -gt $d.h) { $rotated = $true; break }
Start-Sleep -Milliseconds 150
}
$ms = ((Get-Date) - $tRot).TotalMilliseconds
Write-Host (" display {0}x{1}, landscape within {2:N0}ms of live" -f $d.w,$d.h,$ms)
if (-not $rotated) { throw "V-C1 FAIL: still portrait 2s after live" }
Write-Host " PASS: AUTO rotated to landscape"
# =====================================================================
Write-Host "=== 4. V-C3 + V-C4: counter monotonic; >=45fps with animated desktop ==="
# Static desktop is source-limited (~25fps): the honest window needs motion.
$anim = Start-Job -ScriptBlock {
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class An { [DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y); }
"@
$end=(Get-Date).AddSeconds(14); $i=0
while ((Get-Date) -lt $end) {
$a=$i*0.18
[An]::SetCursorPos([int](960+320*[Math]::Cos($a)), [int](540+320*[Math]::Sin($a))) | Out-Null
Start-Sleep -Milliseconds 12; $i++
}
}
Start-Sleep -Seconds 2 # let the encoder see motion
$fA = FrameCountOrWake; $ta = Get-Date
Start-Sleep -Milliseconds 2500
$fB = FrameCountOrWake; $tb = Get-Date
$fps = if ($fA -ge 0 -and $fB -gt $fA) { [Math]::Round(($fB-$fA)/(($tb-$ta).TotalSeconds),1) } else { -1 }
Write-Host " frames $fA -> $fB fps=$fps (animated)"
if ($fB -le $fA) { throw "V-C3 FAIL: counter stalled across rotation" }
Write-Host " PASS: V-C3 session survived rotation"
if ($fps -lt 45) { throw "V-C4 FAIL: $fps fps < 45 with animated desktop" }
Write-Host " PASS: V-C4 fps >= 45"
# =====================================================================
Write-Host "=== 5. V-C5: wake -> visible -> idle -> hidden ==="
# R-8 discipline: "button absent" is NOT a valid judgement on its own - an
# empty/broken dump would pass it trivially (measured 2026-11-01: after the
# hide, the HUD TextView is GONE too, so the tree has zero text nodes).
# Positive pair: keyboard button absent WHILE the video View node is present.
#
# Sequence is explicit (first run's implicit precondition raced: the fps
# step's wake expired its 3s timer before we checked, so controls were
# already hidden - correct product behavior, wrong harness assumption):
# wake -> assert visible -> idle -> assert hidden.
WakeControls
$x5 = Get-Ui
if (-not (ButtonByName $x5 $F_KB)) { throw "V-C5 FAIL: edge wake did not bring controls back" }
Write-Host " visible after wake: OK"
$deadline = (Get-Date).AddSeconds(6); $hidden = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 600
$x5 = Get-Ui
$videoSeen = ($x5 -match 'class="android\.view\.View"')
$kbGone = -not (ButtonByName $x5 $F_KB)
if ($videoSeen -and $kbGone) { $hidden = $true; break }
}
if (-not $hidden) {
throw "V-C5 FAIL: no (kb-gone AND video-present) state within 6s; last dump len=$($x5.Length)" }
Write-Host " PASS: controls auto-hidden (video View still in tree - real hide, not empty dump)"
Write-Host "=== 5b. V-C2: video 16:9 and >=90% screen height (controls hidden) ==="
$ui = Get-UiRetry
$best = $null; $bestErr = 1e9
foreach ($n in [regex]::Matches($ui,'<node[^>]*class="android\.view\.View"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')) {
$x1=[int]$n.Groups[1].Value; $y1=[int]$n.Groups[2].Value
$x2i=[int]$n.Groups[3].Value; $y2=[int]$n.Groups[4].Value
$w=$x2i-$x1; $h=$y2-$y1
if ($w -lt 300 -or $h -lt 100) { continue }
$err = [Math]::Abs($w/[double]$h - 16.0/9.0)
if ($err -lt $bestErr) { $bestErr=$err; $best=@{w=$w;h=$h} }
}
$d = LocalDisplay
$hFrac = $best.h / [double]$d.h
Write-Host (" video {0}x{1} ratio err {2:N4} height {3:N1}% of {4}x{5}" -f `
$best.w,$best.h,$bestErr,($hFrac*100),$d.w,$d.h)
if ($bestErr -gt 0.02) { throw "V-C2 FAIL: video not 16:9 (err $bestErr)" }
if ($hFrac -lt 0.90) { throw ("V-C2 FAIL: video height {0:N1}% < 90%" -f ($hFrac*100)) }
Write-Host " PASS: full-height 16:9 video (the tiny-video fix)"
Stop-Job $anim -ErrorAction SilentlyContinue; Remove-Job $anim -Force -ErrorAction SilentlyContinue
Write-Host "=== 6. V-C5b: black-bar tap wakes controls WITHOUT touching the remote ==="
Start-Sleep -Seconds 2 # animator fully stopped
# The wake zone IS the letterbox black bar now (4a design change), so this
# is the honest target. Also assert a video-interior tap is NOT consumed as
# a wake - otherwise the black bar would have eaten real clicks too.
$pc0 = [Cur4a]::Get()
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 600
$pc1 = [Cur4a]::Get()
$ui = Get-UiRetry
$kb = ButtonByName $ui $F_KB
$jump = [Math]::Max([Math]::Abs($pc1[0]-$pc0[0]), [Math]::Abs($pc1[1]-$pc0[1]))
Write-Host (" black-bar tap: cursor jump=$jump px ; keyboard back: {0}" -f ($null -ne $kb))
if (-not $kb) { throw "V-C5 FAIL: black-bar tap did not wake controls" }
if ($jump -gt 1) { throw "V-C5b FAIL: black-bar tap leaked $jump px to remote" }
Write-Host " PASS: wake works, zero remote leakage"
Write-Host "=== 6c. four buttons present; mode tap changes and restores ==="
$kb2 = Find-VisibleButton $F_KB
$mode = Find-VisibleButton $F_MODE $true
$disc = Find-VisibleButton $F_DISC
$orient = Find-VisibleButton $F_ORIENT $true
if (-not ($kb2 -and $mode -and $disc -and $orient)) {
throw ("V-C5 FAIL: missing buttons kb={0} mode={1} disc={2} orient={3}" -f `
($null -ne $kb2),($null -ne $mode),($null -ne $disc),($null -ne $orient)) }
$beforeLabel = Read-ButtonLabel $F_MODE $true
if (-not $beforeLabel) { throw "V-C5 precondition FAIL: mode label unreadable" }
[void](Tap-VisibleButton $F_MODE $true)
Start-Sleep -Milliseconds 600
$afterLabel = Read-ButtonLabel $F_MODE $true
Write-Host " mode label: [$beforeLabel] -> [$afterLabel]"
# null would trivially "change" from a real label - assert non-empty too (R-8)
if (-not $afterLabel -or $beforeLabel -eq $afterLabel) { throw "V-C5 FAIL: mode tap changed nothing" }
# converge back to relative (the product default the later proofs assume),
# regardless of which way the flip above landed - tap until it reads relative.
$F_MODE_REL = $F_MODE + -join ([char]0x76F8,[char]0x5BF9) # mode: relative
$converged = $false
for ($i=0; $i -lt 3 -and -not $converged; $i++) {
$cur = Read-ButtonLabel $F_MODE $true
if ($cur -eq $F_MODE_REL) { $converged = $true; break }
[void](Tap-VisibleButton $F_MODE $true)
Start-Sleep -Milliseconds 600
}
$restored = Read-ButtonLabel $F_MODE $true
Write-Host " mode converged to relative: [$restored]"
if (-not $converged -or $restored -ne $F_MODE_REL) {
throw "V-C5 FAIL: mode not converged to relative (pollutes later proofs)" }
Write-Host " PASS: buttons work, state restored"
# =====================================================================
Write-Host "=== 7. V-C7: orientation cycle AUTO->L->P->SYS->AUTO ==="
$seq = @($F_LAND, $F_PORT, $F_SYS, $F_AUTO)
$wantLand = @($true, $false, $null, $true) # null = don't assert SYSTEM policy
for ($i=0; $i -lt 4; $i++) {
$orient = Find-VisibleButton $F_ORIENT $true
if (-not $orient) { throw "V-C7 FAIL: orientation button unreachable after 5 wakes" }
[void](Tap-VisibleButton $F_ORIENT $true)
Start-Sleep -Milliseconds 1400
$label = Read-ButtonLabel $F_ORIENT $true
$d = LocalDisplay
$land = $d.w -gt $d.h
Write-Host (" step {0}: label=[{1}] want=[{2}] display={3}x{4} landscape={5}" -f ($i+1),$label,$seq[$i],$d.w,$d.h,$land)
if ($label -ne $seq[$i]) { throw ("V-C7 FAIL: step {0} label [{1}] != [{2}]" -f ($i+1),$label,$seq[$i]) }
if ($null -ne $wantLand[$i] -and $land -ne $wantLand[$i]) {
throw ("V-C7 FAIL: step {0} landscape={1} want={2}" -f ($i+1),$land,$wantLand[$i]) }
}
Write-Host " PASS: full cycle with display following each policy"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-c7-final.png" | Out-Null
}
finally {
# 4a pollution discipline: leave no live session behind - the next proof's
# Restart-AppForm force-stops the app mid-session, which strands the agent's
# serial accept in a zombie (N11) and the run after that fails to connect
# (measured 2026-11-01: agent alive but no longer serving handshakes).
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 500
Get-Job | Stop-Job -ErrorAction SilentlyContinue
Get-Job | Remove-Job -Force -ErrorAction SilentlyContinue
[void](AdbShell @("svc","power","stayon","false"))
if ($origIme -and $origIme -ne "null") {
[void](AdbShell @("ime","enable",$origIme))
[void](AdbShell @("ime","set",$origIme))
}
}
Write-Host "=== LANDSCAPE PROOF COMPLETE ==="
+315
View File
@@ -0,0 +1,315 @@
# 2b full-chain proof (phone side): Android key -> v3 Text channel -> agent
# SendInput -> dedicated Windows target window, read back from that window.
# ASCII-only source (PS 5.1 + BOM-less = GBK corruption; see personal lessons).
# State checks use ASCII markers ("RTT=" / EditText presence) rather than
# matching the app's Chinese labels, for the same reason.
#
# Why ONE atomic script: SendInput lands on the FOREGROUND window, and Windows'
# foreground lock means a background process cannot simply reclaim it. This
# box's interactive GUI steals foreground between separate tool calls, so
# launch -> assert foreground -> drive phone -> read back must run without
# yielding. Every injection is preceded by a foreground assertion: if the target
# lost foreground we ABORT rather than inject into some other application.
#
# Prereqs: vivo device online via adb, app installed, v3 agent build present.
# Usage:
# powershell -ExecutionPolicy Bypass -File tools\phone_chain_proof.ps1
param(
[string]$PhoneIp = "192.168.1.180",
[int]$Port = 47800,
[string]$Code = "t"
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
# UNIQUE per-run dir (measured 2026-11-01): with a fixed path, this script's
# own startup Remove-Item deleted the stop.flag that LEFTOVER targets from a
# previous failed run were watching - they could never retire, kept grabbing
# foreground every 60ms, and wrote their textbox content into the shared
# received.txt ("abc\nabc" = old run's abc+ENTER + this run's abc).
$tmp = "$env:TEMP\remote-phone-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid"
$stopFile = "$tmp\stop.flag"
$resultFile = "$tmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public static class FgApi {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
}
"@
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
# adb writes progress ("1 file pulled...") to stderr; PS 5.1 with
# $ErrorActionPreference=Stop turns every stderr line into a terminating
# error. Same workaround build-android.ps1 documents: relax for native
# calls, judge by exit code and captured stdout.
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
$out = & $adb @AdbArgs 2>$null
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
}
finally { $ErrorActionPreference = $old }
}
function Adb-Shell {
param([Parameter(Mandatory)][string[]]$Cmd)
return Adb -AdbArgs (@("shell") + $Cmd)
}
function Wait-File {
param([string]$Path, [int]$TimeoutMs = 20000)
$t = 0
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
}
function Assert-Foreground([IntPtr]$hwnd, [string]$where) {
# Bounded wait, not a single instant check: the target re-asserts itself
# every 60ms in proof mode, so a momentary steal by the interactive desktop
# is recoverable. Still refuse to inject if it never comes back - a stray
# injection lands in whatever window DOES hold foreground.
$deadline = (Get-Date).AddMilliseconds(1500)
while ((Get-Date) -lt $deadline) {
if ([FgApi]::GetForegroundWindow() -eq $hwnd) { return }
Start-Sleep -Milliseconds 30
}
throw "$where : foreground=$([FgApi]::GetForegroundWindow()) want=$hwnd after 1.5s - refusing to inject"
}
function Read-Received {
Wait-File $resultFile 8000
$b = [System.IO.File]::ReadAllBytes($resultFile)
$s = [System.Text.Encoding]::UTF8.GetString($b)
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
return $s
}
function Get-PhoneUiText {
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
$r = Adb -AdbArgs @("pull", "/sdcard/ui.xml", "$tmp\ui.xml")
if ($r.code -ne 0 -or -not (Test-Path "$tmp\ui.xml")) { return "" }
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
}
# 4a adaptation: in landscape the HUD TextView hides WITH the auto-hiding
# control bar, so "RTT=" disappears while the session is perfectly alive
# (measured 2026-11-01: this proof died at step 4 "session died after ENTER"
# on a healthy 22fps stream). Detect the hidden state (video View node present,
# zero Button nodes) and wake via the letterbox black bar - that tap is
# consumed by the app and never reaches the remote.
function Phone-DisplaySize {
# NB: this script's Adb-Shell returns @{out;code}, not a bare string.
$d = (Adb-Shell @("dumpsys", "window", "displays")).out
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 1080; h = 2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 1080; h = 2400 } }
return @{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake-Controls-If-Hidden([string]$xml) {
$hidden = ($xml -match 'class="android\.view\.View"') -and ($xml -notmatch 'class="android\.widget\.Button"')
if (-not $hidden) { return $false }
$pd = Phone-DisplaySize
[void](Adb-Shell @("input", "tap", "100", "$([int]($pd.h / 2))"))
Start-Sleep -Milliseconds 400
return $true
}
function Wait-For-LiveSession([int]$Seconds, [ref]$outXml) {
$deadline = (Get-Date).AddSeconds($Seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900
$x = Get-PhoneUiText
if ($x -match "RTT=") { $outXml.Value = $x; return $true }
if (Wake-Controls-If-Hidden $x) {
$x = Get-PhoneUiText
if ($x -match "RTT=") { $outXml.Value = $x; return $true }
}
}
$outXml.Value = (Get-PhoneUiText)
return $false
}
Write-Host "=== 0. stale-agent guard (same rule as the desktop proof) ==="
$agentExe = "$repo\target\release\agent.exe"
$ag = Get-Process agent -ErrorAction SilentlyContinue
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Write-Host " running agent predates current build; restarting it"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800
$ag = $null
}
if (-not $ag) {
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
Start-Sleep -Seconds 1
Write-Host " agent started (v3)"
}
Write-Host "=== 1. launch target window (own process, empty field) ==="
# Retire leftovers from earlier runs WITHOUT killing them. Killing a foreground
# window hands foreground to the next app (this box's interactive GUI), and the
# foreground lock then refuses a brand-new background process any claim - the
# new target could never take foreground back.
# Signal only stop files that ALREADY EXIST: creating a fresh one here would
# hit the stop path of the run we are about to start, and that target would
# close itself the moment it launched (seen 2026-10-30: pid reported then gone).
$existingFlags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -like "*remote*proof*" })
if ($existingFlags.Count -gt 0) {
Write-Host " retiring $($existingFlags.Count) earlier target(s) via their own stop file"
foreach ($f in $existingFlags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
Start-Sleep -Seconds 3
}
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
Wait-File $pidFile
# pid file carries "pid<n><hwnd>": the target writes its own handle once the
# form is realized, which removes the Get-Process MainWindowHandle race.
$parts = @((Get-Content $pidFile -Raw) -split "`n")
$tid = [int]$parts[0].Trim()
$hwnd = [IntPtr][long]$parts[1].Trim()
if ($hwnd -eq 0) { throw "target reported a null window handle" }
Assert-Foreground $hwnd "after launch"
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64()) foreground=yes"
Write-Host "=== 2. start app, connect via intent extras ==="
[void](Adb-Shell @("am", "force-stop", "com.verify.remote"))
[void](Adb-Shell @("am", "start", "-n", "com.verify.remote/.MainActivity",
"--es", "host", $PhoneIp, "--ei", "port", "$Port", "--es", "code", $Code))
$xmlRef = ""
if (-not (Wait-For-LiveSession 25 ([ref]$xmlRef))) { throw "phone never reported a live session (no RTT marker)" }
$script:xml = $xmlRef
Write-Host " phone: $([regex]::Match($script:xml,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
Assert-Foreground $hwnd "after connect"
Write-Host "=== 3. phone keys A B C -> Activity.onKeyDown -> v3 Text channel ==="
# 29/30/31 = KEYCODE_A/B/C. adb-injected keys arrive with a printable
# unicodeChar, so the app routes them to nativeSendText (IME hidden). That is
# the product path introduced in 2b, not a stand-in for it.
foreach ($k in 29,30,31) {
Assert-Foreground $hwnd "before keyevent $k"
[void](Adb-Shell @("input", "keyevent", "$k"))
Start-Sleep -Milliseconds 250
}
$deadline = (Get-Date).AddSeconds(10)
$got = ""
while ((Get-Date) -lt $deadline) {
$got = Read-Received
if ($got -eq "abc") { break }
Start-Sleep -Milliseconds 300
}
Write-Host " target received: [$got]"
if ($got -ne "abc") {
Write-Host " foreground now: $([FgApi]::GetForegroundWindow()) target: $hwnd"
throw "FULL-CHAIN FAIL: expected [abc] got [$got]"
}
Write-Host " PASS: Android key -> v3 Text -> agent KEYEVENTF_UNICODE -> Windows window"
Write-Host "=== 4. phone ENTER (66): control key uses the scancode channel ==="
# Enter carries no printable unicodeChar, so it must go down the WinScancode
# path. Observable acceptance: session survives and keeps streaming (a
# single-line TextBox swallows Enter, so glyph-level proof is not possible).
Assert-Foreground $hwnd "before ENTER"
[void](Adb-Shell @("input", "keyevent", "66"))
Start-Sleep -Milliseconds 900
$xmlRef = ""
if (-not (Wait-For-LiveSession 6 ([ref]$xmlRef))) { throw "session died after ENTER" }
Write-Host " session alive: $([regex]::Match($xmlRef,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
Write-Host "=== 5. phone BACK (4): disconnect and return to form ==="
[void](Adb-Shell @("input", "keyevent", "4"))
Start-Sleep -Seconds 2
$xml2 = Get-PhoneUiText
if ($xml2 -match "RTT=") { throw "BACK did not disconnect (still live session)" }
if ($xml2 -notmatch "android\.widget\.EditText") { throw "BACK did not return to the form" }
Write-Host " PASS: disconnected, form visible again"
Write-Host "=== 6. connect again FROM THE FORM (V4 through real UI, not intent) ==="
# Form typing uses keyevent, not 'adb input text': with the Chinese IME active,
# text injection gets composed (ASCII '.' becomes U+3002). We disable the IME,
# restore it in finally, and never assume a Chinese IME is absent.
$origIme = (Adb-Shell @("settings", "get", "secure", "default_input_method")).out.Trim()
Write-Host " original IME: [$origIme]"
if ($origIme -and $origIme -ne "null") { [void](Adb-Shell @("ime", "disable", $origIme)) }
try {
$xml3 = Get-PhoneUiText
$fields = [regex]::Matches($xml3, '<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($fields.Count -lt 3) { throw "expected 3 form fields, found $($fields.Count)" }
function Center($m) {
$a = [int]$m.Groups[1].Value; $b = [int]$m.Groups[2].Value
$c = [int]$m.Groups[3].Value; $d = [int]$m.Groups[4].Value
return @(([int](($a + $c) / 2)), ([int](($b + $d) / 2)))
}
$ipC = Center $fields[0]; $codeC = Center $fields[2]
# KEYCODE_0=7..KEYCODE_9=16, PERIOD=56, DEL=67, T=48 (A=29 -> T=29+19;
# all from javap against android-34, not memory)
[void](Adb-Shell @("input", "tap", "$($ipC[0])", "$($ipC[1])"))
Start-Sleep -Milliseconds 400
for ($i = 0; $i -lt 24; $i++) { [void](Adb-Shell @("input", "keyevent", "67")) }
# Type the SAME host this run was given. A hardcoded literal broke this
# proof on 2026-10-31 when the dev box's LAN address changed
# (192.168.1.180 -> 192.168.1.110): the form dialed the stale address,
# timed out, and it looked like a product failure when the harness was
# simply out of date. Digits are KEYCODE_0..9 = 7..16, '.' = 56.
$ipKeys = @()
foreach ($ch in $PhoneIp.ToCharArray()) {
if ($ch -eq '.') { $ipKeys += 56 }
elseif ($ch -ge '0' -and $ch -le '9') { $ipKeys += (7 + [int][string]$ch) }
else { throw "unsupported character in PhoneIp: $ch" }
}
foreach ($k in $ipKeys) { [void](Adb-Shell @("input", "keyevent", "$k")) }
[void](Adb-Shell @("input", "tap", "$($codeC[0])", "$($codeC[1])"))
Start-Sleep -Milliseconds 400
for ($i = 0; $i -lt 6; $i++) { [void](Adb-Shell @("input", "keyevent", "67")) }
[void](Adb-Shell @("input", "keyevent", "48"))
$xml3 = Get-PhoneUiText
# Attribute order in uiautomator dumps is text= ... class= ... bounds=;
# matching "class=...text=" silently finds nothing (seen 2026-10-30: printed
# an empty form line). Parse per-node and pull attributes independently.
$vals = @()
foreach ($n in [regex]::Matches($xml3, '<node[^>]*>')) {
$v = $n.Value
if ($v -match 'class="android\.widget\.EditText"') {
$vals += [regex]::Match($v, 'text="([^"]*)"').Groups[1].Value
}
}
# NOTE: an EMPTY EditText reports its hint as text (verified on device), so
# "field non-empty" is NOT a usable judgement - the assertion is always
# "field text == typed value". (ASCII-only comments: PS 5.1 decodes
# BOM-less .ps1 as GBK.)
Write-Host " form now: $($vals -join ' | ')"
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
if ($vals[0] -ne $PhoneIp) { throw "V4 FAIL: IP field is [$($vals[0])], expected [$PhoneIp]" }
$btn = [regex]::Match($xml3, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if (-not $btn.Success) { throw "connect button not found" }
$bc = Center $btn
Assert-Foreground $hwnd "before form connect"
[void](Adb-Shell @("input", "tap", "$($bc[0])", "$($bc[1])"))
$xmlRef = ""
if (-not (Wait-For-LiveSession 25 ([ref]$xmlRef))) {
$shown = [regex]::Matches($xmlRef, 'text="([^"]+)"') | ForEach-Object { $_.Groups[1].Value }
throw "V4 FAIL: form connect did not reach a live session. screen=[$($shown -join ' | ')]"
}
Write-Host " V4 PASS via form UI: $([regex]::Match($xmlRef,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
}
finally {
if ($origIme -and $origIme -ne "null") {
[void](Adb-Shell @("ime", "enable", $origIme))
[void](Adb-Shell @("ime", "set", $origIme))
Write-Host " IME restored: $((Adb-Shell @("settings", "get", "secure", "default_input_method")).out.Trim())"
}
}
Write-Host "=== 7. stop target ==="
New-Item -ItemType File -Force -Path $stopFile | Out-Null
Start-Sleep -Milliseconds 600
Write-Host "=== PHONE CHAIN PROOF COMPLETE ==="
+349
View File
@@ -0,0 +1,349 @@
# proof_common.ps1 - shared device-side proof helpers (4b, personal docs loop-4b S6).
#
# ASCII-only file (PS 5.1 reads BOM-less .ps1 as GBK; repo lessons).
# Dot-source it: . "$PSScriptRoot\proof_common.ps1"
#
# Provenance: these are the battle-tested helpers from landscape_proof.ps1 (4a:
# wake-only-from-proven-hidden, positive-pair hide check, N25-style unique temp
# files, uiautomator empty-dump retry) plus rel_mouse_phone_proof's focus-assert
# field clearing. Older proofs are NOT migrated to this file in loop 4b (changing
# four validated scripts without re-running all four is itself a false-green
# risk) - migration is registered as a harness hygiene item for the next loop.
# ---- adb wrappers (never let stderr trip ErrorActionPreference=Stop) ----
if (-not (Get-Variable -Name PccAdb -Scope Global -ErrorAction SilentlyContinue)) {
$global:PccAdb = "D:\working\android\sdk\platform-tools\adb.exe"
}
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try {
$out = & $global:PccAdb @AdbArgs 2>$null
@{ out = ($out -join "`n"); code = $LASTEXITCODE }
} finally { $ErrorActionPreference = $old }
}
function AdbShell { param([string[]]$Cmd) (Adb -AdbArgs (@("shell") + $Cmd)).out }
# ---- UI tree ----
# Unique temp per process (N25 lesson applied to dump pulls, cheap insurance).
$script:PccUiTmp = "$env:TEMP\pcc_ui_$PID.xml"
$script:PccDevDump = "/sdcard/pcc_$PID.xml"
function Get-Ui {
[void](AdbShell @("uiautomator", "dump", $script:PccDevDump))
$r = Adb -AdbArgs @("pull", $script:PccDevDump, $script:PccUiTmp)
if ($r.code -ne 0 -or -not (Test-Path $script:PccUiTmp)) { return "" }
return [System.IO.File]::ReadAllText($script:PccUiTmp, [System.Text.Encoding]::UTF8)
}
# uiautomator intermittently dumps empty (measured repeatedly in 4a). Retry until
# the tree is plausibly real. Callers still assert on their own node - this only
# removes the transient, never masks a genuine absence.
function Get-UiRetry([int]$tries = 4) {
$x = ""
for ($i = 0; $i -lt $tries; $i++) {
$x = Get-Ui
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
Write-Host " [ui] retry exhausted, last dump len=$($x.Length)"
return $x
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml, 'text="([^"]+)"') | ForEach-Object {
[System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value)
}
}
# ---- buttons: match by LABEL, never by position (3b/N23 lesson) ----
function ButtonNode([string]$xml, [string]$want, [bool]$prefix = $false) {
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
$hit = if ($prefix) { $t.StartsWith($want) } else { $t -eq $want }
if ($hit) { return $n.Value }
}
return $null
}
function ButtonCenter([string]$xml, [string]$want, [bool]$prefix = $false) {
$v = ButtonNode $xml $want $prefix
if (-not $v) { return $null }
$b = [regex]::Match($v, 'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if (-not $b.Success) { return $null }
@{ cx = [int](([int]$b.Groups[1].Value + [int]$b.Groups[3].Value) / 2)
cy = [int](([int]$b.Groups[2].Value + [int]$b.Groups[4].Value) / 2) }
}
function ButtonLabelByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) { return $t }
}
return $null
}
# ---- 4a observation-surface discipline: controls auto-hide in landscape ----
# Hidden state = POSITIVE PAIR (video View node present AND zero Button nodes).
# An empty dump would otherwise pass "no buttons" trivially (N24 false-fails).
function ControlsHidden([string]$ui) {
$anyBtn = ($ui -match 'class="android\.widget\.Button"')
$video = ($ui -match 'class="android\.view\.View"')
return ($video -and -not $anyBtn)
}
# Wake tap lives in the letterbox black bar (x=100): outside the video in both
# states, consumed by the app only when hidden - zero remote leakage (4a/D1).
function Wake-Band {
$d = LocalDisplay
[void](AdbShell @("input", "tap", "100", "$([int]($d.h / 2))"))
Start-Sleep -Milliseconds 400
}
function Find-VisibleButton([string]$want, [bool]$prefix = $false) {
for ($try = 0; $try -lt 5; $try++) {
$ui = Get-UiRetry
$b = ButtonCenter $ui $want $prefix
if ($b) { return $b }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null } # real tree, genuine absence
Start-Sleep -Milliseconds 400 # empty-dump transient
}
return $null
}
function Tap-VisibleButton([string]$want, [bool]$prefix = $false) {
$b = Find-VisibleButton $want $prefix
if (-not $b) { return $null }
[void](AdbShell @("input", "tap", "$($b.cx)", "$($b.cy)"))
return $b
}
function Read-ButtonLabel([string]$want, [bool]$prefix = $false) {
for ($try = 0; $try -lt 6; $try++) {
$ui = Get-UiRetry
$l = ButtonLabelByPrefix $ui $want
if ($l) { return $l }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null }
Start-Sleep -Milliseconds 400
}
return $null
}
# ---- HUD / session liveness ----
# "zhen=" frame counter (char U+5E27 as code point, ASCII-safe source).
function FrameCount {
$m = [regex]::Match((Get-Ui), ([char]0x5E27 + '=(\d+)'))
if ($m.Success) { return [long]$m.Groups[1].Value }
return -1
}
function WakeControls {
$d = LocalDisplay
[void](AdbShell @("input", "tap", "100", "$([int]($d.h / 2))"))
Start-Sleep -Milliseconds 500
}
function FrameCountOrWake {
$f = FrameCount
if ($f -ge 0) { return $f }
WakeControls
return FrameCount
}
# Live HUD line (RTT= only exists when connected; hidden-controls caveat above).
function Wait-LiveHud([int]$seconds = 20) {
$deadline = (Get-Date).AddSeconds($seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$ui = Get-Ui
if ($ui -notmatch "RTT=") {
if (ControlsHidden $ui) { WakeControls }
continue
}
$m = [regex]::Match($ui, 'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { return [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value) }
}
return $null
}
# ---- display / app lifecycle ----
# vivo ROM prints NO mRotation= line and keeps a permanent 1080x2400 virtual
# display (N30): orientation MUST be read from cur= inside the mDisplayId=0 block.
function LocalDisplay {
$d = (AdbShell @("dumpsys", "window", "displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 0; h = 0 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 0; h = 0 } }
@{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake {
[void](AdbShell @("input", "keyevent", "224"))
Start-Sleep -Milliseconds 250
[void](AdbShell @("wm", "dismiss-keyguard"))
Start-Sleep -Milliseconds 350
}
function Restart-AppForm {
Wake
[void](AdbShell @("am", "force-stop", "com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am", "start", "-n", "com.verify.remote/.MainActivity"))
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-Ui
if (([regex]::Matches($x, 'class="android\.widget\.EditText"')).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
throw "form did not appear: [$(VisibleTexts (Get-Ui) -join ' || ')]"
}
# ---- connect form: focus-assert typing (rel_mouse_proof grade, R-8) ----
function CenterOf($m) {
@([int](([int]$m.Groups[1].Value + [int]$m.Groups[3].Value) / 2),
[int](([int]$m.Groups[2].Value + [int]$m.Groups[4].Value) / 2))
}
function FormFields([string]$xml) {
[regex]::Matches($xml, '<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
}
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 } else { throw "unsupported char $ch" }
[void](AdbShell @("input", "keyevent", "$k"))
Start-Sleep -Milliseconds 70
}
}
function FocusedIndex([string]$xml) {
$i = -1
foreach ($n in [regex]::Matches($xml, '<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$i++
if ($n.Value -match 'focused="true"') { return $i }
}
}
return -1
}
function Clear-Field($c, [int]$wantIdx) {
for ($try = 0; $try -lt 3; $try++) {
[void](AdbShell @("input", "tap", "$($c[0])", "$($c[1])"))
$deadline = (Get-Date).AddSeconds(1.5)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
if ((FocusedIndex (Get-Ui)) -eq $wantIdx) {
for ($i = 0; $i -lt 40; $i++) { [void](AdbShell @("input", "keyevent", "67")) }
Start-Sleep -Milliseconds 120
return
}
}
}
throw "field $wantIdx never took focus"
}
function FillAndConnect([string]$xml, [string]$host_, [string]$port_, [string]$code_,
[string]$fConnect, [string]$fReconnect) {
$f = FormFields $xml
if ($f.Count -lt 3) { throw "form has $($f.Count) fields" }
Clear-Field (CenterOf $f[0]) 0; TypeAscii $host_
Clear-Field (CenterOf $f[1]) 1; TypeAscii $port_
Clear-Field (CenterOf $f[2]) 2; TypeAscii $code_
$expect = @($host_, $port_, $code_)
$vals = @()
for ($retry = 0; $retry -le 2; $retry++) {
$x2 = Get-UiRetry
$vals = @()
foreach ($n in [regex]::Matches($x2, '<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
}
}
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
$bad = -1
for ($i = 0; $i -lt 3; $i++) { if ($vals[$i] -ne $expect[$i]) { $bad = $i; break } }
if ($bad -ge 0) {
$f = FormFields $x2
Clear-Field (CenterOf $f[$bad]) $bad
TypeAscii $expect[$bad]
continue
}
$btn = ButtonCenter $x2 $fConnect
if (-not $btn) { $btn = ButtonCenter $x2 $fReconnect }
if (-not $btn) { $btn = Find-VisibleButton $fConnect
if (-not $btn) { $btn = Find-VisibleButton $fReconnect } }
if (-not $btn) { throw "connect button not found" }
[void](AdbShell @("input", "tap", "$($btn.cx)", "$($btn.cy)"))
return
}
throw "form input never converged: [$($vals -join ' | ')]"
}
# ---- video geometry (uiautomator has no SurfaceView - match 16:9 View, N21) ----
function VideoBounds([string]$xml) {
$best = $null; $bestErr = 1e9
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.view\.View"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')) {
$x1 = [int]$n.Groups[1].Value; $y1 = [int]$n.Groups[2].Value
$x2 = [int]$n.Groups[3].Value; $y2 = [int]$n.Groups[4].Value
$w = $x2 - $x1; $h = $y2 - $y1
if ($w -lt 300 -or $h -lt 100) { continue }
$err = [Math]::Abs($w / [double]$h - 16.0 / 9.0)
if ($err -lt $bestErr) { $bestErr = $err; $best = @{ l = $x1; t = $y1; w = $w; h = $h } }
}
if ($best) { $best.err = $bestErr }
return $best
}
# ---- PC-side cursor: DPI-aware set with hijack gates ----
if (-not ('PccCur' -as [type])) {
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class PccCur {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr c);
public static void Dpi(){ SetProcessDpiAwarenessContext(new IntPtr(-4)); }
public static int[] Get(){ POINT p; GetCursorPos(out p); return new int[]{p.X,p.Y}; }
public static void Set(int x,int y){ SetCursorPos(x,y); }
}
"@
}
[PccCur]::Dpi()
# SetCursorQuiet: the machine may be shared (2026-11-02 probe: a human used the
# real mouse mid-probe and faked every reading). Gate: two reads 350ms apart must
# agree (nobody is moving), then Set + read-back must equal the request; retries
# on hijack; loud failure instead of a silently wrong sample (R-8).
function Set-CursorQuiet([int]$x, [int]$y, [int]$tries = 8) {
for ($i = 0; $i -lt $tries; $i++) {
$a = [PccCur]::Get()
Start-Sleep -Milliseconds 350
$b = [PccCur]::Get()
if (-not ($a[0] -eq $b[0] -and $a[1] -eq $b[1])) {
Write-Host " [cursor] busy ($($a -join ',')->$($b -join ',')), retry $($i+1)"
Start-Sleep -Seconds 1
continue
}
[PccCur]::Set($x, $y)
Start-Sleep -Milliseconds 120
$g = [PccCur]::Get()
if ($g[0] -eq $x -and $g[1] -eq $y) { return }
Write-Host " [cursor] readback ($($g -join ',')) != ($x,$y), retry $($i+1)"
Start-Sleep -Milliseconds 500
}
throw "cursor never settled at ($x,$y) - someone is using the real mouse; retry the proof when the PC is idle"
}
# ---- memory precheck (N20/N29: OOM fakes build/test failures) ----
function Memory-Precheck([double]$minGB = 0.5) {
$os = Get-CimInstance Win32_OperatingSystem
$freeGB = $os.FreeVirtualMemory / 1MB
Write-Host (" commit free: {0:N1} GB" -f $freeGB)
if ($freeGB -lt $minGB) { throw "commit memory critically low - free RAM first (gradle --stop)" }
}
# ---- R-7 stale-agent guard: binary newer than process => restart ----
function Ensure-FreshAgent([string]$AgentExe, [int]$Port, [string]$Code) {
if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe (cargo build --release -p agent)" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " agent predates build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
Start-Process $AgentExe -ArgumentList "--port", "$Port", "--code", $Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 2
Write-Host " agent started (fresh)"
} else { Write-Host " agent running and current" }
}
+698
View File
@@ -0,0 +1,698 @@
# 3b phone-side proof: relative mouse (protocol v4) + long-press right click + R7 regression.
#
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
# Expected Chinese UI strings are matched as CODE-POINT fragments (R-8: assert the
# actual value, and print what the parser really read so an empty result cannot pass).
#
# Judgements:
# V-B1a relative mode moves the PC pointer by the expected amount (direction+sign)
# V-B2 absolute mode still maps finger -> desktop 1:1 (R7 re-run, <=8px error)
# V-B3 long press (>=500ms, still) opens a context menu; a drag does NOT
# V-B7 R7 + Chinese IME + full chain regressions stay green
#
# Prereqs: vivo device online via adb, 3b APK installed, v4 agent running.
# powershell -ExecutionPolicy Bypass -File tools\rel_mouse_phone_proof.ps1 -Host_ 192.168.1.110
param(
[string]$Host_ = "192.168.1.110",
[int]$Port = 47800,
[string]$Code = "t",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
$ev = Join-Path (Split-Path $repo -Parent) "evidence\3b"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- code-point fragments (grep-verified present in the Kotlin sources) ----
$F_CONNECT = -join ([char]0x8FDE,[char]0x63A5) # "connect"
$F_RECONNECT = -join ([char]0x91CD,[char]0x65B0,[char]0x8FDE,[char]0x63A5) # "reconnect"
$F_RELATIVE = -join ([char]0x76F8,[char]0x5BF9) # "relative"
$F_ABSOLUTE = -join ([char]0x7EDD,[char]0x5BF9) # "absolute"
$F_MODE_TAG = -join ([char]0x6A21,[char]0x5F0F) # "mode"
$F_LONGPRESS = -join ([char]0x957F,[char]0x6309,[char]0x53F3,[char]0x952E) # "long-press right click"
$F_RCLICK_TAG = -join ([char]0x53F3,[char]0x952E,[char]0x3D) # "right-click="
$F_CLICK = -join ([char]0x5355,[char]0x51FB) # "single click"
# Read the persistent right-click counter from the HUD ("right-click=N(source)").
# Returns -1 when absent. A counter is used instead of the one-line debug text
# because ACTION_UP immediately overwrites the debug line after a long press
# (measured 2026-10-31: screen showed the long-press-END label, the firing evidence was gone).
function RightClickCount {
for ($t=0; $t -lt 3; $t++) {
$x = Get-UiText
$m = [regex]::Match($x, 'text="([^"]*RTT=[^"]*)"')
if ($m.Success) {
$hud = [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value)
$c = [regex]::Match($hud, "$F_RCLICK_TAG(\d+)")
if ($c.Success) { return [int]$c.Groups[1].Value }
return 0
}
# 4a: in landscape the HUD hides WITH the controls (GONE, not empty text)
Wake-Controls $x
Start-Sleep -Milliseconds 400
}
return -1
}
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try { $out = & $adb @AdbArgs 2>$null; @{ out = ($out -join "`n"); code = $LASTEXITCODE } }
finally { $ErrorActionPreference = $old }
}
function AdbShell {
param([string[]]$Cmd)
(Adb -AdbArgs (@("shell") + $Cmd)).out
}
function Get-UiText {
$d = AdbShell @("uiautomator","dump","/sdcard/rm.xml")
$r = Adb @("pull","/sdcard/rm.xml","$env:TEMP\rm.xml")
if ($r.code -ne 0 -or -not (Test-Path "$env:TEMP\rm.xml")) {
Write-Host " [ui] dump=[$($d -replace "`n",' ')] pull_rc=$($r.code) -> empty"
return ""
}
$txt = [System.IO.File]::ReadAllText("$env:TEMP\rm.xml", [System.Text.Encoding]::UTF8)
if ($txt.Length -lt 200) {
Write-Host " [ui] short xml ($($txt.Length) chars): $($txt.Substring(0,[Math]::Min(120,$txt.Length)))"
}
return $txt
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml,'text="([^"]+)"') | ForEach-Object { [System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value) }
}
function FormFields([string]$xml) {
[regex]::Matches($xml,'<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
}
function Center($m) {
$a=[int]$m.Groups[1].Value; $b=[int]$m.Groups[2].Value
$c=[int]$m.Groups[3].Value; $d=[int]$m.Groups[4].Value
@([int](($a+$c)/2), [int](($b+$d)/2))
}
function ButtonCenter([string]$xml, [string]$want) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t -eq $want) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
return @([int](($x1+$x2)/2), [int](($y1+$y2)/2))
}
}
}
return $null
}
function ButtonStartingWith([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
return @([int](($x1+$x2)/2), [int](($y1+$y2)/2))
}
}
}
return $null
}
# 4a self-adaptation: in landscape the control bar auto-hides after 3s idle
# (and the HUD TextView hides with it). The wake tap goes to the LEFT BLACK BAR
# (x=100, outside the letterboxed video) - the app consumes black-bar touches
# only in the hidden state, so this never reaches the remote. Bottom-center is
# NOT usable: with the controls hidden the video is full-height, so the bottom
# edge lies INSIDE the picture and would fire a real remote click.
function PhoneDisplay {
$d = (AdbShell @("dumpsys","window","displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w=1080; h=2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w=1080; h=2400 } }
return @{ w=[int]$m.Groups[1].Value; h=[int]$m.Groups[2].Value }
}
function Wake-Controls([string]$xml) {
if ($xml -match 'class="android\.widget\.Button"') { return } # visible already
$pd = PhoneDisplay
[void](AdbShell @("input","tap","100","$([int]($pd.h/2))"))
Start-Sleep -Milliseconds 500
}
# HUD text with wake-retry (4a: in landscape the HUD TextView hides WITH the
# control bar - a plain read returns "" once the 3s timer expired, and each
# dump itself costs ~2s so reads routinely straddle the expiry).
function Get-HudText {
for ($t=0; $t -lt 3; $t++) {
$x = Get-UiText
$m = [regex]::Match($x, 'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { return [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value) }
Wake-Controls $x
Start-Sleep -Milliseconds 400
}
return ""
}
# Toggle the pointer mode and PROVE it flipped by re-reading the button label.
# The bottom bar sits at y=2256..2324, right against the system navigation bar;
# a single tap there is not reliable (measured 2026-10-31: the tap landed but the
# mode did not change). So: tap, wait, re-read, retry; and judge on the BUTTON
# label as well as the HUD (two independent observations of the same state).
function Get-UiTextRetry([int]$tries = 4) {
# landscape_proof Get-UiRetry pattern (N24-era lesson): uiautomator
# intermittently returns short/empty trees; a single miss inside a
# wake-then-read sequence used to burn the whole mode-switch budget
# (measured 2026-11-02 step 3: full run died at "mode button not found"
# although the very next steps would have re-dumped fine).
$x = ""
for ($i = 0; $i -lt $tries; $i++) {
$x = Get-UiText
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
return $x
}
function Set-Mode([string]$want) {
# $want is "relative" or "absolute"
$wantTag = if ($want -eq "relative") { $F_RELATIVE } else { $F_ABSOLUTE }
for ($try=0; $try -lt 6; $try++) {
$xml = Get-UiTextRetry
$cur = $null
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.Contains($F_MODE_TAG)) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
$cur = @{ label=$t; cx=[int](($x1+$x2)/2); cy=[int](($y1+$y2)/2) }
}
}
}
if (-not $cur) {
# 4a: controls may be auto-hidden (landscape). Wake and fold the miss
# into the SAME try budget instead of a single re-read + instant throw.
Wake-Controls $xml
continue
}
Write-Host " mode button label now: [$($cur.label)] (try $try)"
if ($cur.label.Contains($wantTag)) { return $true }
[void](AdbShell @("input","tap","$($cur.cx)","$($cur.cy)"))
Start-Sleep -Milliseconds 700
}
# Fail LOUD with evidence: a bare text dump cannot show what the screen
# actually looked like (this is how the 2026-11-02 step-3 death became a
# guessing game; one PNG makes the next occurrence self-evidencing).
$evShot = Join-Path $ev "mode-switch-fail-$PID.png"
try { & powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out $evShot | Out-Null } catch { }
$final = ((VisibleTexts (Get-UiTextRetry)) -join " || ")
throw "mode did not switch to $want after 6 tries; screen: [$final] shot: $evShot"
}
# The video SurfaceView bounds - touch coordinates must land inside it.
#
# uiautomator does NOT expose SurfaceView as "android.view.SurfaceView"; a bare
# SurfaceView with no accessibility node collapses to a plain "android.view.View"
# (measured 2026-10-31 on the live device: the only candidate is 1080x607, which
# is exactly the 16:9 letterbox of the 1920x1080 desktop inside a 1080-wide stage).
# So identify it structurally: the widest View whose aspect ratio matches 16:9.
# Assert the ratio instead of assuming it - a silent wrong node would make every
# downstream coordinate assertion meaningless.
function SurfaceBounds([string]$xml) {
$best = $null; $bestErr = [double]::MaxValue
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.view\.View"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')) {
$x1=[int]$n.Groups[1].Value; $y1=[int]$n.Groups[2].Value
$x2=[int]$n.Groups[3].Value; $y2=[int]$n.Groups[4].Value
$w = $x2-$x1; $h = $y2-$y1
if ($w -lt 300 -or $h -lt 100) { continue }
$ratio = $w / [double]$h
$err = [Math]::Abs($ratio - (16.0/9.0))
if ($err -lt $bestErr) { $bestErr = $err; $best = @($x1,$y1,$x2,$y2) }
}
if (-not $best) { return $null }
# sanity: the chosen node must really be ~16:9, else we grabbed something else
$w = $best[2]-$best[0]; $h = $best[3]-$best[1]
if ($bestErr -gt 0.25) {
throw "no 16:9 view found (best candidate ${w}x${h}, ratio $([Math]::Round($w/[double]$h,3)))"
}
Write-Host " video view ${w}x${h} (ratio $([Math]::Round($w/[double]$h,3)), 16:9 error $([Math]::Round($bestErr,4)))"
return $best
}
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 }
else { throw "unsupported char: $ch" }
[void](AdbShell @("input","keyevent","$k"))
Start-Sleep -Milliseconds 70
}
}
function FocusedIndex([string]$xml) {
$i = -1
foreach ($n in [regex]::Matches($xml,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$i++
if ($n.Value -match 'focused="true"') { return $i }
}
}
return -1
}
function Focus-Field($c, [int]$wantIdx) {
for ($try=0; $try -lt 3; $try++) {
[void](AdbShell @("input","tap","$($c[0])","$($c[1])"))
$deadline = (Get-Date).AddSeconds(1.5)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
if ((FocusedIndex (Get-UiText)) -eq $wantIdx) { return }
}
}
throw "tap did not focus field $wantIdx after 3 tries"
}
function Clear-Field($c, [int]$idx) {
Focus-Field $c $idx
for ($i=0; $i -lt 40; $i++) { [void](AdbShell @("input","keyevent","67")) }
Start-Sleep -Milliseconds 150
}
function Restart-AppForm {
[void](AdbShell @("input","keyevent","224"))
Start-Sleep -Milliseconds 300
[void](AdbShell @("wm","dismiss-keyguard"))
Start-Sleep -Milliseconds 500
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity"))
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-UiText
if ((FormFields $x).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
$diag = (VisibleTexts (Get-UiText)) -join " || "
throw "connect form did not appear. screen: [$diag]"
}
function FillAndConnect([string]$xml, [string]$host_, [string]$port_, [string]$code_) {
$f = FormFields $xml
if ($f.Count -lt 3) { throw "form has $($f.Count) fields" }
Clear-Field (Center $f[0]) 0; TypeAscii $host_
Clear-Field (Center $f[1]) 1; TypeAscii $port_
Clear-Field (Center $f[2]) 2; TypeAscii $code_
$expect = @($host_, $port_, $code_)
for ($retry=0; $retry -le 2; $retry++) {
$x2 = Get-UiText
$vals = @()
foreach ($n in [regex]::Matches($x2,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
}
}
Write-Host " form now: $($vals -join ' | ') (try $retry)"
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
$bad = -1
for ($i=0; $i -lt 3; $i++) { if ($vals[$i] -ne $expect[$i]) { $bad = $i; break } }
if ($bad -ge 0) {
$f = FormFields $x2
Clear-Field (Center $f[$bad]) $bad
TypeAscii $expect[$bad]
continue
}
break
}
if ($vals[0] -ne $host_) { throw "form input FAIL: host is [$($vals[0])], expected [$host_]" }
if ($vals[1] -ne $port_) { throw "form input FAIL: port is [$($vals[1])], expected [$port_]" }
if ($vals[2] -ne $code_) { throw "form input FAIL: code is [$($vals[2])], expected [$code_]" }
$btn = $null
foreach ($want in @($F_CONNECT, $F_RECONNECT)) { $btn = ButtonCenter $x2 $want; if ($btn) { break } }
if (-not $btn) { throw "no primary connect/reconnect button found" }
[void](AdbShell @("input","tap","$($btn[0])","$($btn[1])"))
}
function Wait-Live([int]$seconds, [ref]$hud) {
$deadline = (Get-Date).AddSeconds($seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$m = [regex]::Match((Get-UiText),'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { $hud.Value = [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value); return $true }
}
return $false
}
# ---- PC-side cursor readback (the only honest judge of "pointer moved N px") ----
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public class PC {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr ctx);
[DllImport("user32.dll")] public static extern int GetSystemMetrics(int i);
public static void Dpi() { SetProcessDpiAwarenessContext(new IntPtr(-4)); }
public static int[] Get() { POINT p; GetCursorPos(out p); return new int[]{p.X,p.Y}; }
public static void Set(int x,int y) { SetCursorPos(x,y); }
public static int W() { return GetSystemMetrics(0); }
public static int H() { return GetSystemMetrics(1); }
}
"@
[void][PC]::Dpi()
$SW = [PC]::W(); $SH = [PC]::H()
Write-Host " PC screen ${SW}x${SH} (physical, DPI-aware)"
Write-Host "=== 0. stale-artifact guard (R-7) + IME disable + keep-awake ==="
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and (Test-Path $AgentExe) -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " running agent predates current build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe" }
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 1; Write-Host " agent started (v5)"
} else { Write-Host " agent running and current" }
$origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim()
Write-Host " original IME: [$origIme] (recorded, restored in finally)"
if ($origIme -and $origIme -ne "null") { [void](AdbShell @("ime","disable",$origIme)) }
[void](AdbShell @("svc","power","stayon","true"))
$origCursor = [PC]::Get()
Write-Host " PC cursor origin ($($origCursor[0]),$($origCursor[1])) (restored in finally)"
try {
# =================================================================
Write-Host "=== 1. connect (protocol v4) and confirm HUD shows v4 ==="
$x = Restart-AppForm
[void](FillAndConnect $x $Host_ "$Port" $Code)
$hud = ""
if (-not (Wait-Live 14 ([ref]$hud))) { throw "never reached a live session" }
Write-Host " HUD: $hud"
# HUD version pin (3b/V-B4). PROTO bump MUST sync this string -
# see MainActivity PROTO_LABEL comment (4b R-5 pair).
if (-not $hud.Contains("v5")) { throw "V-B4 FAIL: HUD does not report protocol v5; got [$hud]" }
Write-Host " PASS: live session on protocol v5"
$xml = Get-UiText
$sb = SurfaceBounds $xml
if (-not $sb) { throw "cannot locate SurfaceView bounds" }
$vx1=$sb[0]; $vy1=$sb[1]; $vx2=$sb[2]; $vy2=$sb[3]
$vcx=[int](($vx1+$vx2)/2); $vcy=[int](($vy1+$vy2)/2)
Write-Host " surface bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-b1-live.png" | Out-Null
# 4a pollution guard: the mode PERSISTS in prefs across runs, and a run that
# threw between Set-Mode "absolute" and its restore left absolute installed
# (measured 2026-11-01: step 2 ran in absolute mode, the DOWN absolute move
# "teleported" 1441px - correct product behavior, wrong precondition).
# Force relative, then settle back into the hidden steady state: while the
# controls are hidden, gestures never re-show them, so the geometry stays
# stable for steps 2/2b/2c.
$hudNow = Get-HudText
Write-Host " HUD at start: $hudNow"
# Converge to relative UNCONDITIONALLY via Set-Mode, which reads the BUTTON
# label and returns early if already relative (idempotent, authoritative).
# The old guard `if ($hudNow.Contains(absolute))` silently skipped when the
# HUD read came back empty or straddled the auto-hide (N24 observation-surface
# trap) - a mode left polluted by a previous run that threw after Set-Mode
# absolute then poisoned step 2c: absolute mode's DOWN DOES send an absolute
# move (correct product behavior), which the leak-point check read as a
# teleport (measured again 2026-11-02: prefs relative_mode=false at start,
# V-B1a "passed" on direction alone because absolute swipes also have
# direction - the whole step 2 block was running in the wrong mode).
[void](Set-Mode "relative")
Start-Sleep -Milliseconds 3800 # let the 3s auto-hide re-engage
$xh = Get-UiText
if ($xh -match 'class="android\.widget\.Button"') { throw "precondition FAIL: controls did not re-hide after 3.8s" }
$hsb = SurfaceBounds $xh
# NB: PowerShell variable names are CASE-INSENSITIVE - a local `$sh` here
# silently clobbers `$SH` (screen height) and every later [PC]::Set($SW/2,$SH/2)
# dies with "Object[] has no op_Division" (measured 2026-11-01). Prefix locals
# distinctly ($hsb = hidden-state bounds).
if (-not $hsb) { throw "precondition FAIL: no video bounds in hidden state" }
$vx1=$hsb[0]; $vy1=$hsb[1]; $vx2=$hsb[2]; $vy2=$hsb[3]
$vcx=[int](($vx1+$vx2)/2); $vcy=[int](($vy1+$vy2)/2)
Write-Host " hidden steady-state bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
# =================================================================
Write-Host "=== 2. V-B1a: relative mode moves PC pointer (swipe right) ==="
# Park the PC cursor at a known point so the delta is unambiguous.
# Let the link drain first (same discipline as 2c): residual events from an
# earlier gesture would otherwise be attributed to this one.
Start-Sleep -Milliseconds 1000
[PC]::Set([int]($SW/2), [int]($SH/2))
Start-Sleep -Milliseconds 500
$before = [PC]::Get()
# A horizontal swipe of ~40% of the view width. With sensitivity 1.0 and
# view width mapping to desktop width, expected delta = 0.4 * SW.
$swipeFromX = [int]($vx1 + ($vx2-$vx1)*0.30)
$swipeToX = [int]($vx1 + ($vx2-$vx1)*0.70)
[void](AdbShell @("input","swipe","$swipeFromX","$vcy","$swipeToX","$vcy","400"))
Start-Sleep -Milliseconds 500
$after = [PC]::Get()
$moved = $after[0] - $before[0]
Write-Host " swipe dx_device=$($swipeToX-$swipeFromX)px -> PC pointer dx=$moved px (before x=$($before[0]), after x=$($after[0]))"
if ($moved -le 0) { throw "V-B1a FAIL: rightward swipe did not move the pointer right (dx=$moved)" }
# Direction check is the hard judgement; exact magnitude depends on how many
# ACTION_MOVE samples the ROM delivers for an `input swipe` (not equal to the
# finger travel), so magnitude is reported, not asserted.
Write-Host " PASS: relative mode moves the pointer in the finger's direction"
Write-Host "=== 2b. V-B1a negative: swipe left moves pointer left ==="
Start-Sleep -Milliseconds 1000
[PC]::Set([int]($SW/2), [int]($SH/2))
Start-Sleep -Milliseconds 500
$before2 = [PC]::Get()
[void](AdbShell @("input","swipe","$swipeToX","$vcy","$swipeFromX","$vcy","400"))
Start-Sleep -Milliseconds 500
$after2 = [PC]::Get()
$moved2 = $after2[0] - $before2[0]
Write-Host " PC pointer dx=$moved2 px (before x=$($before2[0]), after x=$($after2[0]))"
if ($moved2 -ge 0) { throw "V-B1a FAIL: leftward swipe did not move the pointer left (dx=$moved2)" }
Write-Host " PASS: negative direction correct (sign preserved)"
Write-Host "=== 2c. relative mode does NOT teleport the pointer on touch-down ==="
# The defining property of relative mode: ACTION_DOWN must not send an
# absolute move. Tap at the far LEFT of the view; the pointer must not jump
# to the left edge of the desktop.
#
# DISCRIMINATION (2026-11-01, landscape): a real absolute-leak has an exact
# signature - the pointer lands at the tap's desktop mapping
# (0.05*1920=96, 540) and STAYS. Residual swipe events instead keep the
# cursor DRIFTING for seconds (observed: stable-looking 4-read windows
# followed by -890/-1418px surges - bursts with >1.2s gaps between them).
# So: settle (6 equal reads), tap, then sample 4x over 2s.
# all samples == before -> PASS
# samples drift -> harness residue: retry (max 3)
# settled AT leak point -> product FAIL (no retry will change it)
# Absolute-leak landing point for this tap: view fraction 0.05 maps to
# desktop x = 0.05*(SW-1); y = view center row -> desktop mid row.
$leakX = [int][Math]::Round(0.05 * ($SW-1))
$leakY = [int][Math]::Round((($vcy-$vy1) / [double]($vy2-$vy1)) * ($SH-1))
$verdict = "retry"
for ($att=1; $att -le 3 -and $verdict -eq "retry"; $att++) {
Start-Sleep -Milliseconds 2000
[PC]::Set([int]($SW*0.8), [int]($SH/2))
Start-Sleep -Milliseconds 1500
$stable = $false
for ($i=0; $i -lt 8; $i++) {
$runs = @()
for ($k=0; $k -lt 6; $k++) { $runs += ,([PC]::Get()); Start-Sleep -Milliseconds 250 }
$eq = $true
foreach ($r in $runs) { if ($r[0] -ne $runs[0][0] -or $r[1] -ne $runs[0][1]) { $eq = $false } }
if ($eq) { $stable = $true; break }
Write-Host (" att{0}: cursor still settling (last {1},{2}); re-parking" -f $att,$runs[-1][0],$runs[-1][1])
[PC]::Set([int]($SW*0.8), [int]($SH/2))
Start-Sleep -Milliseconds 800
}
if (-not $stable) { throw "cannot obtain a stable cursor baseline (residual events never drained)" }
$beforeTap = [PC]::Get()
$tapX = [int]($vx1 + ($vx2-$vx1)*0.05)
[void](AdbShell @("input","tap","$tapX","$vcy"))
$samples = @()
for ($k=0; $k -lt 4; $k++) { Start-Sleep -Milliseconds 500; $samples += ,([PC]::Get()) }
$dbgTap = ((VisibleTexts (Get-UiText)) -join " || ")
$maxJump = 0
foreach ($s in $samples) {
$j = [Math]::Max([Math]::Abs($s[0]-$beforeTap[0]), [Math]::Abs($s[1]-$beforeTap[1]))
if ($j -gt $maxJump) { $maxJump = $j }
}
$last = $samples[-1]
$drift = $false
for ($k=1; $k -lt 4; $k++) {
if ($samples[$k][0] -ne $samples[0][0] -or $samples[$k][1] -ne $samples[0][1]) { $drift = $true }
}
$atLeak = ([Math]::Abs($last[0]-$leakX) -le 8) -and ([Math]::Abs($last[1]-$leakY) -le 8)
# NB: "$_[0]" inside a double-quoted string expands only `$_` and prints
# "[0]" literally (PS 5.1) - use explicit sub-expressions.
$sampleStr = ($samples | ForEach-Object { "(" + $_[0] + "," + $_[1] + ")" }) -join " "
Write-Host (" att{0}: tap ({1},{2}) -> samples {3} maxJump={4} drift={5} atLeakPoint={6} (leak=({7},{8}))" -f `
$att,$beforeTap[0],$beforeTap[1],$sampleStr,$maxJump,$drift,$atLeak,$leakX,$leakY)
if ($dbgTap.Contains($F_LONGPRESS)) { throw "tap was interpreted as a long press; harness timing is wrong" }
if ($atLeak) { throw ("V-B1a FAIL: pointer settled at the absolute-leak point ({0},{1}) - DOWN sent an absolute move" -f $last[0],$last[1]) }
if ($maxJump -le 8 -and -not $drift) { $verdict = "pass"; break }
if (-not $drift) {
# settled far away but NOT at the leak point: some other single event
# moved it - that IS a product-relevant teleport, retry once more
# before condemning (a queued burst can also settle far away).
Write-Host " att${att}: settled far (jump=$maxJump) but not at leak point - treating as residue, retrying"
}
}
if ($verdict -ne "pass") { throw "V-B1a FAIL: pointer moved on touch-down in relative mode across 3 attempts (residual or teleport - see samples above)" }
Write-Host " PASS: relative mode leaves the pointer where it was (no teleport)"
# =================================================================
Write-Host "=== 3. V-B2: switch to ABSOLUTE mode, re-run R7 touch mapping ==="
[void](Set-Mode "absolute")
# 4a geometry race (measured 2026-11-01): Set-Mode WOK the controls, so the
# video shrank to 1216x684 - but each dump costs ~2s and the auto-hide timer
# is 3s, so by tap time the controls had hidden again and the video was back
# to full height. Tapping with the stale visible-state geometry produced
# "errors" of 281-357px that were pure harness error (y landed at the old
# center row 341, not 540). Wait for the hidden steady state and sample
# THOSE bounds - in hidden state a video tap only re-arms the timer, it never
# re-shows the controls, so the geometry is stable for the whole R7 loop.
Start-Sleep -Milliseconds 3800
$hud2 = Get-HudText
$sb = SurfaceBounds (Get-UiText)
if (-not $sb) { throw "V-B2 FAIL: no video bounds in hidden steady state" }
$vx1=$sb[0];$vy1=$sb[1];$vx2=$sb[2];$vy2=$sb[3]
$vcx=[int](($vx1+$vx2)/2); $vcy=[int](($vy1+$vy2)/2)
Write-Host " hidden steady-state bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
Write-Host " HUD: $hud2"
if (-not $hud2.Contains($F_ABSOLUTE)) { throw "V-B2 FAIL: HUD still does not say absolute; got [$hud2]" }
Write-Host " PASS: mode switched to absolute (button label and HUD agree)"
# R7 expectations derived from the LIVE desktop size (physical pixels; 125%
# scaling does NOT apply - the agent injects into the physical desktop, and
# normX=frac*65535 lands at frac*(SW-1)).
# Fixed 1920-based literals broke the moment the capture display became
# 2560x1440 (2026-11-02: reported 575px error = 0.9*(2560-1920), a harness
# staleness, not a mapping regression). AGENTS.md: coordinate expectations
# MUST be recomputed for the current resolution. Same script already uses
# dynamic $SW/$SH for V-B1a and the leak point - this block was the outlier.
$r7 = @(
@{ frac=0.10 },
@{ frac=0.90 },
@{ frac=0.50 }
)
$worst = 0
foreach ($c in $r7) {
$tx = [int]($vx1 + ($vx2-$vx1)*$c.frac)
$ty = [int]($vy1 + ($vy2-$vy1)*$c.frac)
$c.ex = [int][Math]::Round($c.frac * ($SW-1))
$c.ey = [int][Math]::Round($c.frac * ($SH-1))
[void](AdbShell @("input","tap","$tx","$ty"))
Start-Sleep -Milliseconds 400
$p = [PC]::Get()
$errX = [Math]::Abs($p[0] - $c.ex)
$errY = [Math]::Abs($p[1] - $c.ey)
$e = [Math]::Max($errX,$errY)
if ($e -gt $worst) { $worst = $e }
Write-Host " frac=$($c.frac) expect=($($c.ex),$($c.ey)) actual=($($p[0]),$($p[1])) err=($errX,$errY)"
}
Write-Host " worst R7 error = $worst px"
if ($worst -gt 8) { throw "V-B2 FAIL: R7 absolute mapping error $worst px > 8px" }
Write-Host " PASS: absolute mode still maps 1:1 (R7 regression green)"
# =================================================================
Write-Host "=== 4. V-B3: long press opens a context menu; drag does NOT ==="
# Switch back to relative for the long-press test (that is the default mode
# users get, and the gesture must work there).
[void](Set-Mode "relative")
# Use a desktop of our own: the proof's WinForms target window (R-11: never
# inject into a window we do not own - it would modify user data).
# NB: the 3b version launched this WITHOUT its Mandatory params, so the
# child died instantly and the right-click landed at raw screen center
# (possible user window - near-miss on R-11). Launch properly now and park
# the pointer INSIDE our own window: topmost form at (80,80) 420x140.
$target = Join-Path $repo "tools\text_target.ps1"
if (-not (Test-Path $target)) { throw "missing $target" }
# UNIQUE per-run dir (same orphan-stop-flag trap as phone_chain_proof,
# fixed 2026-11-01): startup Remove-Item must not delete the retirement
# channel of targets left by a previous failed run.
$ttmp = Join-Path $env:TEMP "vbx4a-$PID"
New-Item -ItemType Directory -Force -Path $ttmp | Out-Null
$pidFile = "$ttmp\target.pid"; $stopFile = "$ttmp\stop.flag"; $resultFile = "$ttmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
$tp = Start-Process powershell -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass","-File",$target, `
"-PidFile",$pidFile,"-ResultFile",$resultFile,"-StopFile",$stopFile -PassThru -WindowStyle Hidden
$deadline = (Get-Date).AddSeconds(8)
while (-not (Test-Path $pidFile) -and (Get-Date) -lt $deadline) { Start-Sleep -Milliseconds 200 }
if (-not (Test-Path $pidFile)) { throw "target window did not start (no pid file)" }
Write-Host " target window started (pid $(Get-Content $pidFile))"
# Long-press and drag run in the HIDDEN steady state: geometry is stable
# there (video taps only re-arm the hide timer, they never re-show the
# controls), and the counter read wakes the HUD afterwards.
function Enter-HiddenSteady {
Start-Sleep -Milliseconds 3800
$x = Get-UiText
if ($x -match 'class="android\.widget\.Button"') { throw "precondition FAIL: controls did not hide" }
$s = SurfaceBounds $x
if (-not $s) { throw "precondition FAIL: no hidden-state video bounds" }
return $s
}
$rcBefore = RightClickCount # may wake the controls; re-enter hidden after
$hsb4 = Enter-HiddenSteady
$hx1=$hsb4[0];$hy1=$hsb4[1];$hx2=$hsb4[2];$hy2=$hsb4[3]
$hcx=[int](($hx1+$hx2)/2); $hcy=[int](($hy1+$hy2)/2)
Write-Host " right-click counter before: $rcBefore ; hidden center=($hcx,$hcy)"
# The long press right-clicks AT THE REMOTE POINTER, so park the pointer on
# our own target window first - the menu then opens over a window we own.
[PC]::Set(290, 150)
Start-Sleep -Milliseconds 300
[void](AdbShell @("input","swipe","$hcx","$hcy","$hcx","$hcy","900")) # press & hold ~900ms, no movement
Start-Sleep -Milliseconds 700
$rcAfter = RightClickCount
$debug = ((VisibleTexts (Get-UiText)) -join " || ")
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-b3-longpress.png" | Out-Null
Write-Host " right-click counter after : $rcAfter (source tag in HUD)"
Write-Host " screen after long press: $debug"
if ($rcBefore -lt 0 -or $rcAfter -lt 0) { throw "V-B3 FAIL: could not read the right-click counter from the HUD" }
if ($rcAfter -le $rcBefore) {
throw "V-B3 FAIL: long press did not inject a right click (counter stayed $rcAfter; screen: $debug)"
}
Write-Host " PASS: long press injected a right click (counter $rcBefore -> $rcAfter)"
Write-Host "=== 4b. V-B3 negative: a DRAG must not fire long press ==="
$rcBeforeDrag = RightClickCount
Enter-HiddenSteady | Out-Null
[void](AdbShell @("input","swipe","$hcx","$hcy",$([int]($hcx+120)),$([int]($hcy+80)),900))
Start-Sleep -Milliseconds 700
$rcAfterDrag = RightClickCount
$debug2 = ((VisibleTexts (Get-UiText)) -join " || ")
Write-Host " right-click counter after drag: $rcAfterDrag (was $rcBeforeDrag)"
Write-Host " screen after drag: $debug2"
if ($rcAfterDrag -gt $rcBeforeDrag) {
throw "V-B3 FAIL: drag injected a right click (counter $rcBeforeDrag -> $rcAfterDrag); long press must be suppressed once the finger moves"
}
Write-Host " PASS: drag did not fire long press"
Write-Host "=== 4c. V-B3: two-finger right click still works ==="
$rcBeforeTwo = RightClickCount
# adb cannot synthesize a two-finger tap (lessons R-6): this is a MANUAL item.
Write-Host " (two-finger tap is not drivable via adb - recorded as manual, not asserted here)"
}
finally {
# 4a pollution discipline: end with no live session - the agent's 10s idle
# reclaim then happens BETWEEN runs, not in the middle of the next run's
# connect (measured 2026-11-01: a stranded zombie left the next proof unable
# to reach a live session at all).
[void](AdbShell @("am","force-stop","com.verify.remote"))
# target window exits via its stop flag (text_target contract); kill is the
# fallback. In finally so a throw never leaves a topmost window on the user's desktop.
if ($stopFile) { [void](New-Item -ItemType File -Force -Path $stopFile -ErrorAction SilentlyContinue) }
Start-Sleep -Milliseconds 400
if ($tp -and -not $tp.HasExited) { Stop-Process -Id $tp.Id -Force -ErrorAction SilentlyContinue }
[void][PC]::Set($origCursor[0], $origCursor[1])
[void](AdbShell @("svc","power","stayon","false"))
if ($origIme -and $origIme -ne "null") {
[void](AdbShell @("ime","enable",$origIme))
[void](AdbShell @("ime","set",$origIme))
Write-Host " IME restored: $(((AdbShell @('settings','get','secure','default_input_method')) -join '').Trim())"
}
Write-Host " PC cursor restored to ($($origCursor[0]),$($origCursor[1]))"
}
Write-Host "=== REL MOUSE PHONE PROOF COMPLETE ==="
+23
View File
@@ -0,0 +1,23 @@
# Byte-safe device screenshot helper.
# ASCII-only. PS 5.1 `>` redirection writes UTF-16, which corrupts PNG bytes
# (measured 2026-10-30: `adb exec-out screencap -p > x.png` produced a file
# starting with FF FE). Capture on-device and pull instead.
param(
[Parameter(Mandatory = $true)][string]$Out,
[string]$Device = "/sdcard/shot.png"
)
$ErrorActionPreference = "Stop"
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
$old = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
& $adb shell screencap -p $Device | Out-Null
if ($LASTEXITCODE -ne 0) { throw "screencap failed ($LASTEXITCODE)" }
& $adb pull $Device $Out | Out-Null
if ($LASTEXITCODE -ne 0) { throw "pull failed ($LASTEXITCODE)" }
}
finally { $ErrorActionPreference = $old }
$b = [System.IO.File]::ReadAllBytes($Out)
$sig = ($b[0..3] | ForEach-Object { $_.ToString("X2") }) -join " "
if ($sig -ne "89 50 4E 47") { throw "not a PNG (sig=$sig): $Out" }
Write-Host ("OK {0} ({1:N0} bytes)" -f (Split-Path $Out -Leaf), $b.Length)
+175
View File
@@ -0,0 +1,175 @@
# 2b V6/V7 proof: v3 text injection into a dedicated WinForms target window,
# read back from that process itself.
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
# Non-ASCII payloads are built from explicit code points, never literals.
#
# Usage:
# powershell -ExecutionPolicy Bypass -File tools\text_injection_proof.ps1
#
# Chain: viewer-core text_check --units <utf16> -> v3 agent SendInput
# KEYEVENTF_UNICODE -> foreground WinForms TextBox -> self-reported text.
#
# Why NOT Notepad (2026-10-30): Windows 11 Notepad hosts all tabs in ONE
# process and this dev box had a user document open in it - injecting "into
# Notepad" would mean injecting into the user's real file. The WinForms target
# is a separate process we own, opened on an empty field.
#
# Foreground handling (measured, not assumed): a form with TopMost=true that
# is activated at startup KEEPS foreground for seconds; AppActivate /
# SetForegroundWindow called from here actually STEALS it away. So we assert
# the target is foreground before injecting, and refuse to inject otherwise -
# a stray injection would land in whatever window was foreground.
$ErrorActionPreference = "Stop"
# Repo root derived from this script's location (tools/ is one level under the
# repo root) so the proof runs on any clone, not just the author's D: drive.
$repo = (Resolve-Path "$PSScriptRoot\..").Path
# UNIQUE per run: a fixed path lets the startup Remove-Item delete the stop.flag
# that leftover targets from a previous FAILED run are watching, so they never
# retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-text-proof-$PID"
$base = "D:\working\environment"
$env:RUSTUP_HOME = "$base\rust"
$env:CARGO_HOME = "$base\cargo"
$env:PATH = "$base\mingw64\bin;$base\cargo\bin;" + $env:PATH
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public static class WinFg {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
}
"@
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid"
$stopFile = "$tmp\stop.flag"
$resultFile = "$tmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
function Wait-File {
param([string]$Path, [int]$TimeoutMs = 15000)
$tries = 0
while (-not (Test-Path $Path) -and $tries -lt ($TimeoutMs / 150)) {
Start-Sleep -Milliseconds 150; $tries++
}
if (-not (Test-Path $Path)) { throw "file did not appear: $Path" }
}
function Read-Received {
Wait-File $resultFile
$bytes = [System.IO.File]::ReadAllBytes($resultFile)
$text = [System.Text.Encoding]::UTF8.GetString($bytes)
if ($text.Length -gt 0 -and $text[0] -eq [char]0xFEFF) { $text = $text.Substring(1) }
return $text
}
function CodePointsToString([int[]]$cps) {
-join ($cps | ForEach-Object { [char]$_ })
}
function Show-Text([string]$s) {
# ASCII-safe console preview: non-ASCII rendered as U+XXXX
($s.ToCharArray() | ForEach-Object {
if ([int]$_ -ge 32 -and [int]$_ -lt 127) { $_ } else { "U+{0:X4}" -f [int]$_ }
}) -join ""
}
function Assert-TargetForeground([IntPtr]$hwnd) {
$fg = [WinFg]::GetForegroundWindow()
if ($fg -ne $hwnd) {
throw "foreground is $fg, not target $hwnd - refusing to inject (would hit another app)"
}
}
function Send-Units([int[]]$cps, [IntPtr]$hwnd, [int]$delayMs = 1500) {
$units = $cps -join ","
$exe = "$repo\target\debug\examples\text_check.exe"
if (-not (Test-Path $exe)) { throw "missing $exe (cargo build -p viewer-core --example text_check)" }
Assert-TargetForeground $hwnd
& $exe --port 47800 --code t --units $units --delay $delayMs --hold 800
$code = $LASTEXITCODE
Assert-TargetForeground $hwnd # 注入窗口在 --delay 期间必须仍是靶窗
if ($code -ne 0) { throw "text_check exited $code" }
Start-Sleep -Milliseconds 500
}
Write-Host "=== 0. ensure CURRENT v3 agent is the one running ==="
$ag = Get-Process agent -ErrorAction SilentlyContinue
$agentExe = "$repo\target\release\agent.exe"
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
Write-Host " running agent predates current build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800
$ag = $null
}
if (-not $ag) {
Start-Process -FilePath $agentExe -ArgumentList "--port","47800","--code","t" -WindowStyle Hidden
Start-Sleep -Seconds 1
Write-Host " agent started (v3)"
}
Write-Host "=== 1. launch WinForms target (own process, empty field) ==="
# NO -WindowStyle Hidden: the hidden host makes the form invisible, and an
# invisible window can never be foreground (measured 2026-10-30).
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
"-ResultFile",$resultFile,"-StopFile",$stopFile
Wait-File $pidFile
# target.pid holds "pid<n><hwnd>" (see text_target.ps1 Shown handler); take line 1.
$parts = @((Get-Content $pidFile -Raw) -split "`n")
$tid = [int]$parts[0].Trim()
$tp = Get-Process -Id $tid
Start-Sleep -Milliseconds 1200
$hwnd = [IntPtr][long]$parts[1].Trim()
if ($hwnd -eq 0) { throw "target reported a null window handle" }
Write-Host " target pid=$tid hwnd=$hwnd title=[$($tp.MainWindowTitle)]"
Assert-TargetForeground $hwnd
Write-Host " target is foreground: yes"
Write-Host "=== 2. V6: four CJK BMP characters (Ni Hao Shi Jie) ==="
$cjk = @(0x4F60, 0x597D, 0x4E16, 0x754C)
Send-Units $cjk $hwnd
$got6 = Read-Received
$exp6 = CodePointsToString $cjk
Write-Host (" received [{0}] expected [{1}]" -f (Show-Text $got6), (Show-Text $exp6))
if ($got6 -ne $exp6) { throw "V6 FAIL" }
Write-Host " V6 PASS - BMP CJK arrived intact through v3 Text + KEYEVENTF_UNICODE"
Write-Host "=== 3. V7: exclaim + emoji U+1F600 (UTF-16 surrogate pair) appended ==="
$mix = @(0x0021, 0xD83D, 0xDE00)
Send-Units $mix $hwnd
$got7 = Read-Received
$exp7 = $exp6 + (CodePointsToString $mix)
Write-Host (" received len={0} [{1}]" -f $got7.Length, (Show-Text $got7))
Write-Host (" expected len={0} [{1}]" -f $exp7.Length, (Show-Text $exp7))
if ($got7 -ne $exp7) { throw "V7 FAIL - mismatch" }
if ($got7.Contains([char]0xFFFD)) { throw "V7 FAIL - U+FFFD present: pair got split" }
Write-Host " V7 PASS - surrogate pair survived as one character, no replacement char"
Write-Host "=== 4. negative guard: missing --units must NOT inject anything ==="
$before = Read-Received
$exe = "$repo\target\debug\examples\text_check.exe"
# Omit --units entirely (passing an empty string would be eaten by PowerShell
# argument handling and parsed as the next flag). viewer-core's parse_units
# rejects the degenerate input before any session is even opened.
# No 2>&1 here: viewer-core logs go to stderr by design; merging them turns
# every normal log line into a PowerShell error record.
& $exe --port 47800 --code t --delay 100 --hold 100 | Out-Null
$emptyCode = $LASTEXITCODE
Write-Host " exit=$emptyCode (expected non-zero: tool rejects empty --units)"
if ($emptyCode -eq 0) { throw "FAIL - empty units should be rejected by the tool" }
Start-Sleep -Milliseconds 400
$after = Read-Received
if ($after -ne $before) { throw "FAIL - target text changed unexpectedly: [$after]" }
Write-Host " guard PASS - no injection happened, target text unchanged"
Write-Host "=== 5. stop target ==="
New-Item -ItemType File -Force -Path $stopFile | Out-Null
$tries = 0
while ((Get-Process -Id $tid -ErrorAction SilentlyContinue) -and $tries -lt 40) {
Start-Sleep -Milliseconds 150; $tries++
}
Write-Host "=== PROOF COMPLETE: final target text = [$(Show-Text (Read-Received))] ==="
+110
View File
@@ -0,0 +1,110 @@
# 2b V6/V7 proof target: a dedicated WinForms window we own.
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
#
# Launched by text_injection_proof.ps1. Shows a topmost empty TextBox, writes
# its current text to ResultFile (UTF-8 with BOM) on every change, exits when
# StopFile appears. The proof injects through the remote-control pipeline and
# polls ResultFile - so "what arrived on the controlled desktop" is reported
# by the receiving application itself, not by a screenshot or OCR.
param(
[Parameter(Mandatory = $true)][string]$PidFile,
[Parameter(Mandatory = $true)][string]$ResultFile,
[Parameter(Mandatory = $true)][string]$StopFile,
# SendInput lands on whatever is FOREGROUND. When the injector is driven
# from another machine (phone -> agent -> this desktop), nothing in this
# process keeps our window on top, so an unrelated window could eat the
# injection. With this switch the form re-asserts foreground every tick and
# the caller still asserts foreground before injecting.
[switch]$GrabForeground
)
$ErrorActionPreference = "Stop"
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
# Event handler scriptblocks resolve script-scoped vars; make that explicit
# instead of relying on the param locals staying on a visible scope chain.
$script:PidFile = $PidFile
$script:ResultFile = $ResultFile
$script:StopFile = $StopFile
$script:lastWritten = $null
$form = New-Object System.Windows.Forms.Form
$form.Text = "RemoteTextTarget"
$form.ClientSize = New-Object System.Drawing.Size(420, 140)
$form.TopMost = $true
$form.StartPosition = "Manual"
$form.Location = New-Object System.Drawing.Point(80, 80)
$tb = New-Object System.Windows.Forms.TextBox
$tb.Multiline = $true
$tb.Font = New-Object System.Drawing.Font("Segoe UI", 16.0)
$tb.Dock = "Fill"
$form.Controls.Add($tb)
$script:GrabForeground = $GrabForeground
if ($script:GrabForeground) {
Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;
public static class OwnForeground {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h);
[DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr h, IntPtr pid);
[DllImport("kernel32.dll")] public static extern uint GetCurrentThreadId();
[DllImport("user32.dll")] public static extern bool AttachThreadInput(uint a, uint b, bool f);
// Deliberately NO keybd_event(VK_MENU). The usual 'tap ALT to beat the
// foreground lock' trick corrupts exactly what we measure: after ALT, Windows
// treats the next character as a menu MNEMONIC, so the first injected
// character is eaten. Measured 2026-10-30: ALT re-asserting every 60ms turned
// three injected characters into only 'b' in the target.
// Attaching input queues is a lock workaround that synthesises no key input.
public static bool Claim(IntPtr h) {
uint fgThread = GetWindowThreadProcessId(GetForegroundWindow(), IntPtr.Zero);
uint myThread = GetCurrentThreadId();
AttachThreadInput(myThread, fgThread, true);
bool ok = SetForegroundWindow(h);
AttachThreadInput(myThread, fgThread, false);
return ok && GetForegroundWindow() == h;
}
public static IntPtr Now() { return GetForegroundWindow(); }
}
"@
}
$timer = New-Object System.Windows.Forms.Timer
$timer.Interval = 150
$timer.Add_Tick({
if (Test-Path $script:StopFile) {
$form.Close()
return
}
if ($script:GrabForeground) {
# Re-assert only when actually lost (unconditional claiming every tick
# would fight the user for the desktop outside a proof run).
if ([OwnForeground]::Now() -ne $form.Handle) {
[void]$form.Activate()
[void][OwnForeground]::Claim($form.Handle)
}
}
if ($tb.Text -ne $script:lastWritten) {
$script:lastWritten = $tb.Text
$enc = New-Object System.Text.UTF8Encoding($true)
[System.IO.File]::WriteAllText($script:ResultFile, $tb.Text, $enc)
}
if (-not $tb.Focused) { [void]$tb.Focus() }
})
$form.Add_Shown({
$form.Activate()
[void]$tb.Focus()
# Report BOTH pid and window handle: a proof caller resolving the handle via
# Get-Process races with window creation and can read 0. Handle is stable
# here because the form is realized by the time Shown fires.
[System.IO.File]::WriteAllText($script:PidFile, "$PID`n" + $form.Handle.ToInt64())
})
$timer.Start()
[void]$form.ShowDialog()
$timer.Stop()
+8
View File
@@ -20,3 +20,11 @@ rustls.workspace = true
# headless_check 用软解交叉验证收到的码流合法(库本身刻意不依赖 openh264, # headless_check 用软解交叉验证收到的码流合法(库本身刻意不依赖 openh264,
# 解码在 Android 侧由 MediaCodec 承担) # 解码在 Android 侧由 MediaCodec 承担)
openh264 = { version = "0.9.9", features = ["source"] } openh264 = { version = "0.9.9", features = ["source"] }
# rel_mouse_proof(3b/V-B1)需要读回 Windows 光标位置来闭环验证相对注入。
# 仅 examples 使用,不进库本体;库本身仍然平台无关。
[target.'cfg(windows)'.dev-dependencies]
windows = { workspace = true, features = [
"Win32_Foundation",
"Win32_UI_WindowsAndMessaging",
"Win32_UI_HiDpi",
] }
+185
View File
@@ -0,0 +1,185 @@
//! 3b 桌面验收:相对鼠标(协议 v4)的像素级闭环。
//!
//! 为什么必须单独写一个 proof:V-B1 的判据是「位移**恰好**等于请求值」,
//! 而 Android 侧的真机脚本只能看到 UI 文本,看不到远端指针实际走了多少像素。
//! 这里直接走完整链路:viewer-core 发 `MouseMoveRel` → QUIC → agent 注入 →
//! `GetCursorPos` 读回,逐项比对。
//!
//! 覆盖判据:
//! - V-B1a 方向与量值:dx=+50 → +50px,dx=-50 → -50px(误差 ≤1px)
//! - V-B1b 慢速不丢位移:连发 20 次 dx=+1 → 累计 +20px(误差 ≤2px)
//! - V-B1c 撞边不积债:右边界 +200 后 -50 → 立即 -50
//!
//! 用法(agent 需已启动):
//! cargo run --release -p viewer-core --example rel_mouse_proof -- --host 127.0.0.1 --port 47800 --code t
//!
//! 安全:运行前记录光标原位,结束时还原(不污染用户桌面状态)。
use std::time::Duration;
use viewer_core::{ConnectConfig, Session};
fn arg(name: &str) -> Option<String> {
let a: Vec<String> = std::env::args().collect();
a.iter()
.position(|x| x == name)
.and_then(|i| a.get(i + 1))
.cloned()
}
/// 读当前光标(物理像素)。用 windows crate 而不是 shell,避免进程 DPI 差异。
fn cursor() -> (i32, i32) {
use windows::Win32::Foundation::POINT;
use windows::Win32::UI::WindowsAndMessaging::GetCursorPos;
let mut p = POINT::default();
unsafe {
let _ = GetCursorPos(&mut p);
}
(p.x, p.y)
}
fn set_cursor(x: i32, y: i32) {
use windows::Win32::UI::WindowsAndMessaging::SetCursorPos;
unsafe {
let _ = SetCursorPos(x, y);
}
}
fn screen_size() -> (i32, i32) {
use windows::Win32::UI::WindowsAndMessaging::{GetSystemMetrics, SM_CXSCREEN, SM_CYSCREEN};
unsafe { (GetSystemMetrics(SM_CXSCREEN), GetSystemMetrics(SM_CYSCREEN)) }
}
fn enable_dpi_awareness() {
use windows::Win32::UI::HiDpi::{
SetProcessDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
unsafe {
let _ = SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2);
}
}
/// 发一串相对增量,每次之间留出注入完成的时间,然后读回光标。
fn apply_rel(session: &Session, steps: &[(i16, i16)], settle_ms: u64) {
for &(dx, dy) in steps {
session.send_mouse_rel(dx, dy);
// 让 QUIC 上行 + agent 注入 + 光标更新都完成;太小会读到中间态
std::thread::sleep(Duration::from_millis(settle_ms));
}
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
enable_dpi_awareness();
let port: u16 = arg("--port").map(|s| s.parse()).transpose()?.unwrap_or(47800);
let code = arg("--code").unwrap_or_else(|| "t".into());
let host = arg("--host").unwrap_or_else(|| "127.0.0.1".into());
let (sw, sh) = screen_size();
println!("[proof] screen {sw}x{sh} (physical, DPI-aware)");
let session = Session::connect(ConnectConfig::direct(&host, port, code)).await?;
let (dw, dh) = session.desktop_size();
println!("[proof] agent desktop {dw}x{dh}");
// 记录原位,结束时还原(不污染桌面状态)
let (ox, oy) = cursor();
println!("[proof] cursor origin ({ox},{oy})");
let mut failures = 0usize;
// ---- V-B1a 方向与量值 ----
println!("\n=== V-B1a direction & magnitude ===");
for &delta in &[50i16, -50i16, 120i16, -120i16] {
// 每次都从屏幕中心出发,避免被边界 clamp 干扰量值判定
set_cursor(sw / 2, sh / 2);
std::thread::sleep(Duration::from_millis(60));
let (bx, _) = cursor();
apply_rel(&session, &[(delta, 0)], 80);
let (ax, _) = cursor();
let applied = ax - bx;
let ok = (applied - delta as i32).abs() <= 1;
println!(
" dx={delta:+5} from x={bx:4} to x={ax:4} applied={applied:+5} {}",
if ok { "PASS" } else { "FAIL" }
);
if !ok {
failures += 1;
}
}
// 纵向同理(同一套代码路径,但轴不同,clamp 上界不同)
for &delta in &[40i16, -40i16] {
set_cursor(sw / 2, sh / 2);
std::thread::sleep(Duration::from_millis(60));
let (_, by) = cursor();
apply_rel(&session, &[(0, delta)], 80);
let (_, ay) = cursor();
let applied = ay - by;
let ok = (applied - delta as i32).abs() <= 1;
println!(
" dy={delta:+5} from y={by:4} to y={ay:4} applied={applied:+5} {}",
if ok { "PASS" } else { "FAIL" }
);
if !ok {
failures += 1;
}
}
// ---- V-B1b 慢速小幅不丢位移 ----
println!("\n=== V-B1b slow small steps must not be lost ===");
{
set_cursor(sw / 2, sh / 2);
std::thread::sleep(Duration::from_millis(60));
let (bx, _) = cursor();
// 20 次 dx=+1:若注入层按 mickey 透传,dx=1 会被加速吞成 0 像素(实测如此)
let steps: Vec<(i16, i16)> = (0..20).map(|_| (1i16, 0i16)).collect();
apply_rel(&session, &steps, 45);
std::thread::sleep(Duration::from_millis(120));
let (ax, _) = cursor();
let applied = ax - bx;
let ok = (applied - 20).abs() <= 2;
println!(" 20 x dx=+1 from x={bx} to x={ax} applied={applied:+} (expect +20) {}",
if ok { "PASS" } else { "FAIL" });
if !ok {
failures += 1;
}
}
// ---- V-B1c 撞边不积债 ----
println!("\n=== V-B1c edge clamp leaves no step debt ===");
{
set_cursor(sw - 1, sh / 2);
std::thread::sleep(Duration::from_millis(60));
let (bx, _) = cursor();
apply_rel(&session, &[(200, 0)], 80);
let (mx, _) = cursor();
let clamped = mx - bx;
// 回滑必须立即生效(若累积器"记住"了被 clamp 丢掉的 200px,这里会先空转)
apply_rel(&session, &[(-50, 0)], 80);
let (ax, _) = cursor();
let back = ax - mx;
let ok = clamped == 0 && (back + 50).abs() <= 1;
println!(
" at right edge x={bx}: +200 -> applied {clamped:+} (clamped); -50 -> applied {back:+} {}",
if ok { "PASS" } else { "FAIL" }
);
if !ok {
failures += 1;
}
}
// 还原光标
set_cursor(ox, oy);
println!("\n========== RESULT ==========");
if failures == 0 {
println!("REL MOUSE PROOF PASS (all V-B1a/b/c checks)");
} else {
println!("REL MOUSE PROOF FAIL ({failures} checks failed)");
}
session.close().await;
if failures > 0 {
std::process::exit(1);
}
Ok(())
}
+90
View File
@@ -0,0 +1,90 @@
//! 协议 v3 文本注入链路的桌面真边界验证工具(loop-2b,不依赖 Android)。
//!
//! 证明的事:`Session::send_text` 经线格式到被控端后,`agent` 的
//! `KEYEVENTF_UNICODE` 注入把**同一段 UTF-16 code unit** 送进了前台应用。
//! 判据由外部读取目标窗口的实际文本内容(调用方负责),本工具只负责发送。
//!
//! 用法:
//! 终端1: target\release\agent.exe --port 47800 --code t
//! 终端2: 把目标应用(记事本)切到前台,然后
//! cargo run -p viewer-core --example text_check -- \
//! --port 47800 --code t --units 20320,22909,55357,56832
//!
//! `--units` 是十进制的 UTF-16 code unit 列表(逗号分隔):
//! 「你好😀」= 20320,22909,55357,56832(😀 是代理对)。
//! 刻意不收自然语言字符串参数:命令行到 Rust 的编码往返会引入与本轮无关的变量,
//! 由调用方(PowerShell)从 .NET 字符串取 char[] 更忠实。
//!
//! `--delay`:连接后等待毫秒再注入。给调用方把目标窗口切到前台的时间——
//! SendInput 注入的是**前台**应用,窗口没在前台等于测了个寂寞。
//! `--hold`:注入后保持会话的毫秒数,等消息在远端消化完再关连接。
use anyhow::{bail, Result};
use std::time::Duration;
use viewer_core::{ConnectConfig, Session};
fn arg(name: &str) -> Option<String> {
let a: Vec<String> = std::env::args().collect();
a.iter()
.position(|x| x == name)
.and_then(|i| a.get(i + 1))
.cloned()
}
fn parse_units(spec: &str) -> Result<Vec<u16>> {
if spec.trim().is_empty() {
bail!(
"--units 缺失或为空:本工具要验证的是「有文本送出并被注入」,\
空输入没有验证价值(空 Text 在产品路径上由 Session::send_text 拦掉)"
);
}
spec.split(',')
.map(|s| s.trim())
.filter(|s| !s.is_empty())
.map(|s| {
s.parse::<u32>()
.ok()
.and_then(|v| u16::try_from(v).ok())
.ok_or_else(|| anyhow::anyhow!("--units 必须是 0..=65535 的十进制数,got {s:?}"))
})
.collect()
}
#[tokio::main]
async fn main() -> Result<()> {
let port: u16 = arg("--port")
.map(|s| s.parse())
.transpose()?
.unwrap_or(47800);
let code = arg("--code").unwrap_or_else(|| "123456".into());
let host = arg("--host").unwrap_or_else(|| "127.0.0.1".into());
let delay = arg("--delay")
.map(|s| s.parse())
.transpose()?
.unwrap_or(1500u64);
let hold = arg("--hold")
.map(|s| s.parse())
.transpose()?
.unwrap_or(800u64);
let units = parse_units(&arg("--units").unwrap_or_default())?;
let session = Session::connect(ConnectConfig::direct(&host, port, &code)).await?;
println!(
"[text_check] connected peer={} units={}",
session.peer_addr(),
units.len()
);
tokio::time::sleep(Duration::from_millis(delay)).await;
session.send_text(&units);
println!("[text_check] sent (只报长度,不打印内容)");
tokio::time::sleep(Duration::from_millis(hold)).await;
let st = session.stats();
println!(
"[text_check] frames={} rtt={}ms — 会话活着,注入已发给前台应用",
st.frames_in, st.rtt_ms
);
session.close().await;
Ok(())
}
+449 -22
View File
@@ -15,7 +15,6 @@
//! 仅限局域网测试)。产品化前必须换成信令下发的证书指纹钉扎(`signaling.rs` //! 仅限局域网测试)。产品化前必须换成信令下发的证书指纹钉扎(`signaling.rs`
//! 已有 `CertFingerprint`),该项登记在方案阶段 4。 //! 已有 `CertFingerprint`),该项登记在方案阶段 4。
use anyhow::{bail, Context, Result};
use common::{ use common::{
read_control, read_video_frame, write_control, ControlMsg, InputEvent, VideoFrameHeader, read_control, read_video_frame, write_control, ControlMsg, InputEvent, VideoFrameHeader,
PROTO_VERSION, PROTO_VERSION,
@@ -30,6 +29,30 @@ use std::sync::Arc;
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
use tokio::sync::mpsc; use tokio::sync::mpsc;
/// QUIC 握手单次尝试预算(第一拨):不可达/被拦路径的失败上限(阶段 3a U8)。
/// 与被控端 `agent-core` 的 max_idle_timeout(10s) 同思路——超时策略必须显式、短、可见。
pub const CONNECT_BUDGET: Duration = Duration::from_secs(5);
/// 第二拨握手预算(重拨专用,8s > 第一拨)。
/// 存在理由:agent 串行 accept,僵尸释放实测在「杀端后 ~10.2s + 残骸握手再排
/// 5s 档」≈13s 窗口内;第二拨 8s 让握手段天然长于僵尸窗口,一次重拨接住。
/// 真·不可达不受此影响——它由总预算封顶报 Timeout。
pub const CONNECT_RETRY_BUDGET: Duration = Duration::from_secs(8);
/// 认证应答预算:握手成功后等 `AuthOk` 的上限。agent 串行 accept(R-5),
/// 正常应答是毫秒级;11s 只是异常场景(agent 卡在旧会话收尾)的兜底。
pub const AUTH_BUDGET: Duration = Duration::from_secs(11);
/// connect() 全程硬上限(含重拨)。14s = 5 + 8 两拨握手预算 + 1s 余量:
/// 正常路径由**内层**各拨预算先触发(错误 detail 精确到段),外层只兜
/// 「重试组合出无界等待」这一种失控。真不可达 → ≈13s 报 Timeout
/// (2b 的 32s → 13s 且全程可取消,判据更新见 V-A1)。
pub const CONNECT_TOTAL_BUDGET: Duration = Duration::from_secs(14);
/// 可重拨失败(Timeout/Refused)的最大尝试次数。Resolve/Auth/Version 是
/// 确定性结果,重拨只会拖慢反馈,不重试。
pub const MAX_CONNECT_ATTEMPTS: usize = 2;
/// 连接目标。当前只有局域网直连;设备码 + 打洞在后续阶段替换 `connect()` 内部实现。 /// 连接目标。当前只有局域网直连;设备码 + 打洞在后续阶段替换 `connect()` 内部实现。
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub enum PeerTarget { pub enum PeerTarget {
@@ -62,6 +85,110 @@ impl ConnectConfig {
} }
} }
/// 连接建立失败的分类(阶段 3a)。
///
/// 为什么分类在 Rust 侧做而不是 Kotlin:错误来源(DNS / QUIC 握手 / 认证 / 版本)
/// 只有这一层看得全;Kotlin 只做 `code → 中文文案` 映射。
/// [`ConnectErrorKind::code`] 是 FFI 编号(android-ffi `nativeLastErrorKind` 与 Kotlin
/// `ConnectErrorText` 都按它映射),**改动必须三处同步**——FFI 只传整数不传 Rust 类型。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ConnectErrorKind {
/// 地址解析失败:host 写错或 DNS 不可达
Resolve,
/// 连接超时:被控端不在线、IP/端口不对、入站被拦
Timeout,
/// 被控端异常应答/拒绝:该端口上没有 QUIC 服务、握手被重置
Refused,
/// 认证失败:验证码错误
Auth,
/// 协议版本不一致:两端必须重编到同一版本
Version,
/// 未分类。**detail 原文必须展示**(第二行),不得静默归进某类文案
Other,
}
impl ConnectErrorKind {
/// FFI 稳定编号。新增项只追加,不复用旧编号(Kotlin `else ->` 兜住未知码)。
pub fn code(self) -> i32 {
match self {
ConnectErrorKind::Resolve => 1,
ConnectErrorKind::Timeout => 2,
ConnectErrorKind::Refused => 3,
ConnectErrorKind::Auth => 4,
ConnectErrorKind::Version => 5,
ConnectErrorKind::Other => 9,
}
}
}
/// 连接失败 = 分类 + 原始错误文本。
#[derive(Debug)]
pub struct ConnectError {
pub kind: ConnectErrorKind,
pub detail: String,
}
impl ConnectError {
fn new(kind: ConnectErrorKind, detail: impl Into<String>) -> Self {
Self { kind, detail: detail.into() }
}
}
impl std::fmt::Display for ConnectError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.detail)
}
}
impl std::error::Error for ConnectError {}
/// 未分类错误的统一包装:保留原文(Other 的 detail 会显示到界面第二行,不静默)。
fn other_err(e: impl std::fmt::Display) -> ConnectError {
ConnectError::new(ConnectErrorKind::Other, e.to_string())
}
/// 认证失败消息 → 分类。判据是 agent 侧的稳定前缀(`agent/src/main.rs` 与
/// `agent-core/src/lib.rs` 都以 `protocol version mismatch` 开头);
/// 命中不了的一律 `Auth`——宁可提示「验证码错误」也不把内部错误文案当分类依据。
fn classify_auth_fail(msg: &str) -> ConnectErrorKind {
if msg.contains("protocol version mismatch") {
ConnectErrorKind::Version
} else {
ConnectErrorKind::Auth
}
}
/// 同步 connect() 调用(本地配置面)的分类:全是 Other(endpoint 停摆/地址非法等),
/// 保留原文。单列函数是为了让分类面在测试里可枚举、改动可见。
fn classify_connect_error(_e: &quinn::ConnectError) -> ConnectErrorKind {
ConnectErrorKind::Other
}
/// 哪些失败值得重拨(3a 方案①)。
/// - Timeout/Refused:agent 串行 accept 的排队/残骸窗口是**暂时**的
/// (实测僵尸释放 ~10.2s、残骸阻塞握手 5~8.5s),第二拨能接住;
/// - Resolve/Auth/Version:确定性结果,重拨只拖慢反馈、不改变答案;
/// - Other:语义不明(含钉扎未实现、首消息异常),不猜、不重试。
fn is_retryable(kind: ConnectErrorKind) -> bool {
matches!(kind, ConnectErrorKind::Timeout | ConnectErrorKind::Refused)
}
/// 握手/连接期错误 → 分类。逐个变体对照 quinn 0.11.12 re-export 的
/// `quinn::ConnectionError`(quinn-proto 0.11.19 `connection/mod.rs:3906`,本会话实读)。
fn classify_connection_error(e: &quinn::ConnectionError) -> ConnectErrorKind {
use quinn::ConnectionError as E;
match e {
E::TimedOut => ConnectErrorKind::Timeout,
// 端口上没有 QUIC 服务(回收包/非法头)、被重置、握手后被关:
// 用户语境统一是「对端不应答」→ Refused
E::TransportError(_) | E::Reset | E::ConnectionClosed(_) | E::ApplicationClosed(_) => {
ConnectErrorKind::Refused
}
// QUIC 传输版本协商失败 ≠ 应用协议版本不一致,不要误导成 Version
E::VersionMismatch | E::LocallyClosed | E::CidsExhausted => ConnectErrorKind::Other,
}
}
/// 交付给消费方的一帧视频。字段语义与 `common::VideoFrameHeader` 一致, /// 交付给消费方的一帧视频。字段语义与 `common::VideoFrameHeader` 一致,
/// payload 为标准 Annex-B H.264(可直接喂 MediaCodec)。 /// payload 为标准 Annex-B H.264(可直接喂 MediaCodec)。
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@@ -71,6 +198,10 @@ pub struct Frame {
pub ts_ms: u64, pub ts_ms: u64,
pub seq: u64, pub seq: u64,
pub keyframe: bool, pub keyframe: bool,
/// v5:帧发送时刻的远端光标位置(采集画面内物理像素)。
/// `None` = 隐藏/离屏/后端未实现(语义单一来源在被控端,见 `VideoFrameHeader::cursor`)。
/// 桌面 viewer 不消费该字段(看着自己的鼠标),安卓据它画标记。
pub cursor: Option<(i16, i16)>,
pub payload: Vec<u8>, pub payload: Vec<u8>,
} }
@@ -82,6 +213,7 @@ impl From<(VideoFrameHeader, Vec<u8>)> for Frame {
ts_ms: h.ts_ms, ts_ms: h.ts_ms,
seq: h.seq, seq: h.seq,
keyframe: h.keyframe, keyframe: h.keyframe,
cursor: h.cursor,
payload, payload,
} }
} }
@@ -163,7 +295,66 @@ pub struct Session {
impl Session { impl Session {
/// 建立连接并完成认证握手 + 首帧关键帧请求。 /// 建立连接并完成认证握手 + 首帧关键帧请求。
pub async fn connect(cfg: ConnectConfig) -> Result<Self> { ///
/// **短预算 × 有限重拨**(3a/U8,2026-10-31 用户裁决方案①;演进证据链见
/// loop-3-plan 偏差 D2/D3:单次 5s 预算在 agent 串行 accept 架构下自相矛盾——
/// 不可达要快失败、僵尸排队要耐心等,同一数字满足不了;拆握手/认证两段也不够,
/// 实测阻塞点在握手本身):
/// - 每次尝试:握手第一拨 [`CONNECT_BUDGET`](5s)、重拨拨
/// [`CONNECT_RETRY_BUDGET`](8s)、认证 [`AUTH_BUDGET`](11s);
/// - [`is_retryable`] 的失败(Timeout/Refused)**自动重拨**,共
/// [`MAX_CONNECT_ATTEMPTS`](2) 次。存在理由(2026-10-31 两组对照实验):
/// agent 串行 accept 期间不泵 socket,新握手被残骸/僵尸阻塞实测 5~8.5s
/// (杀会话后必现、干净连发 8 次亦有 1 次);第一拨 5s 被切是预期成本,
/// 第二拨的窗口天然落在释放点之后。包级机制未逐行核实(R-15),此处只
/// 陈述实测:**第一拨 5s 超时 → 第二拨成功**(kill 场景 total ≈5-11s 内接住)。
/// - 全程 [`CONNECT_TOTAL_BUDGET`](13s) 硬上限 = 5+8 完整两拨,
/// 防重试组合出无界等待;
/// - 真不可达 ≈13s 报 Timeout(修复前 32s),**全程 UI 可取消**——
/// U8 判据形态由此更新(plan V-A1)。
///
/// 已知边界:DNS 解析走同步 `to_socket_addrs`,current-thread runtime 里阻塞调用
/// 不能被 timeout 打断;本机实测解析失败是亚秒级。极端 DNS 挂起属阶段 4 网络治理面。
pub async fn connect(cfg: ConnectConfig) -> Result<Self, ConnectError> {
let attempts = async {
let mut attempt = 0usize;
loop {
attempt += 1;
// 第一拨短握手(快速判不可达),重拨拨长握手(接住僵尸释放)
let handshake = if attempt == 1 {
CONNECT_BUDGET
} else {
CONNECT_RETRY_BUDGET
};
match Self::connect_once(cfg.clone(), handshake).await {
Ok(s) => return Ok(s),
Err(e) => {
if attempt >= MAX_CONNECT_ATTEMPTS || !is_retryable(e.kind) {
return Err(e);
}
log(&format!(
"connect 第 {attempt}/{MAX_CONNECT_ATTEMPTS} 拨失败: {}(重拨)",
e.detail
));
}
}
}
};
// 外层硬上限。Timeout::poll 先轮询内层——已成功返回的会话不会被
// 「刚好到点」的外层吞掉(D2 的教训:预算必须给合法路径留出完成空间)
match tokio::time::timeout(CONNECT_TOTAL_BUDGET, attempts).await {
Ok(r) => r,
Err(_) => Err(ConnectError::new(
ConnectErrorKind::Timeout,
format!(
"connect exceeded {}s total budget within {MAX_CONNECT_ATTEMPTS} attempts",
CONNECT_TOTAL_BUDGET.as_secs()
),
)),
}
}
async fn connect_once(cfg: ConnectConfig, handshake_budget: Duration) -> Result<Self, ConnectError> {
rustls::crypto::ring::default_provider() rustls::crypto::ring::default_provider()
.install_default() .install_default()
.ok(); .ok();
@@ -175,11 +366,14 @@ impl Session {
.with_no_client_auth() .with_no_client_auth()
} else { } else {
// 证书钉扎尚未接入(方案阶段 4)。这里明确报错而不是静默放行。 // 证书钉扎尚未接入(方案阶段 4)。这里明确报错而不是静默放行。
bail!("insecure_skip_verify=false 需要证书钉扎,尚未实现(见方案阶段 4)"); return Err(ConnectError::new(
ConnectErrorKind::Other,
"insecure_skip_verify=false 需要证书钉扎,尚未实现(见方案阶段 4)",
));
}; };
let qc: quinn::crypto::rustls::QuicClientConfig = Arc::new(tls) let qc: quinn::crypto::rustls::QuicClientConfig = Arc::new(tls)
.try_into() .try_into()
.context("build QUIC client config")?; .map_err(other_err)?;
// 接收窗口放大:视频下行大流量,quinn 默认 512KB 流窗口会卡吞吐 // 接收窗口放大:视频下行大流量,quinn 默认 512KB 流窗口会卡吞吐
// (与桌面 viewer / agent 的实测调优一致) // (与桌面 viewer / agent 的实测调优一致)
@@ -189,21 +383,54 @@ impl Session {
let mut client_config = quinn::ClientConfig::new(Arc::new(qc)); let mut client_config = quinn::ClientConfig::new(Arc::new(qc));
client_config.transport_config(Arc::new(transport)); client_config.transport_config(Arc::new(transport));
let mut endpoint = quinn::Endpoint::client("0.0.0.0:0".parse()?)?; let mut endpoint =
quinn::Endpoint::client("0.0.0.0:0".parse().map_err(other_err)?).map_err(other_err)?;
endpoint.set_default_client_config(client_config); endpoint.set_default_client_config(client_config);
let PeerTarget::Direct { host, port } = &cfg.target; let PeerTarget::Direct { host, port } = &cfg.target;
let addr = (host.as_str(), *port) let addr = (host.as_str(), *port)
.to_socket_addrs()? .to_socket_addrs()
.map_err(|e| {
ConnectError::new(
ConnectErrorKind::Resolve,
format!("resolve {host}:{port}: {e}"),
)
})?
.next() .next()
.with_context(|| format!("resolve {host}:{port}"))?; .ok_or_else(|| {
let conn = endpoint ConnectError::new(
.connect(addr, "localhost")? ConnectErrorKind::Resolve,
.await format!("resolve {host}:{port}: no record"),
.with_context(|| format!("QUIC connect {addr}"))?; )
})?;
let conn = {
let connect_fut = endpoint.connect(addr, "localhost").map_err(|e| {
ConnectError::new(classify_connect_error(&e), e.to_string())
})?;
// 单次尝试的握手预算(第一拨 5s、重拨拨 8s,见 connect() 注释)。
// 超时归类 Timeout(可重拨);真不可达由重拨打满总预算后报出。
match tokio::time::timeout(handshake_budget, connect_fut).await {
Ok(Ok(c)) => c,
Ok(Err(e)) => {
return Err(ConnectError::new(
classify_connection_error(&e),
e.to_string(),
))
}
Err(_) => {
return Err(ConnectError::new(
ConnectErrorKind::Timeout,
format!(
"QUIC handshake exceeded {}s budget",
handshake_budget.as_secs()
),
))
}
}
};
// ---- 流 1:认证 + 视频下行 ---- // ---- 流 1:认证 + 视频下行 ----
let (mut v_send, mut v_recv) = conn.open_bi().await?; let (mut v_send, mut v_recv) = conn.open_bi().await.map_err(other_err)?;
write_control( write_control(
&mut v_send, &mut v_send,
&ControlMsg::Auth { &ControlMsg::Auth {
@@ -212,30 +439,59 @@ impl Session {
}, },
) )
.await .await
.context("send Auth")?; .map_err(other_err)?;
let desktop: (u32, u32); let desktop: (u32, u32);
match read_control(&mut v_recv).await.context("read AuthOk")? { // 认证应答预算 11s:agent 串行 accept,僵尸会话最长 ~10s(R-5)才轮到本请求。
// 这一段超时的语义是「连上了但对端一直不应答」——同样归类 Timeout,detail 说清。
let first_msg = match tokio::time::timeout(AUTH_BUDGET, read_control(&mut v_recv)).await {
Ok(r) => r.map_err(other_err)?,
Err(_) => {
return Err(ConnectError::new(
ConnectErrorKind::Timeout,
format!(
"connected but auth reply exceeded {}s (agent busy?)",
AUTH_BUDGET.as_secs()
),
))
}
};
match first_msg {
ControlMsg::AuthOk { ControlMsg::AuthOk {
width, width,
height, height,
proto_version, proto_version,
} => { } => {
if proto_version != PROTO_VERSION { if proto_version != PROTO_VERSION {
bail!( return Err(ConnectError::new(
"protocol mismatch: agent={proto_version}, viewer-core={PROTO_VERSION}" ConnectErrorKind::Version,
); format!(
"protocol mismatch: agent={proto_version}, viewer-core={PROTO_VERSION}"
),
));
} }
desktop = (width, height); desktop = (width, height);
log(&format!("auth ok, remote desktop {width}x{height}")); log(&format!("auth ok, remote desktop {width}x{height}"));
} }
ControlMsg::AuthFail(e) => bail!("auth failed: {e}"), ControlMsg::AuthFail(e) => {
m => bail!("unexpected first message: {m:?}"), return Err(ConnectError::new(
classify_auth_fail(&e),
format!("auth failed: {e}"),
))
}
m => {
return Err(ConnectError::new(
ConnectErrorKind::Other,
format!("unexpected first message: {m:?}"),
))
}
} }
// ---- 流 2:输入 + 控制 + Ping/Pong ---- // ---- 流 2:输入 + 控制 + Ping/Pong ----
let (mut ctl_send, mut ctl_recv) = conn.open_bi().await?; let (mut ctl_send, mut ctl_recv) = conn.open_bi().await.map_err(other_err)?;
// 入会即请求 IDR:从 GOP 中间接入时裸 P 帧解不出来(桌面端同款教训) // 入会即请求 IDR:从 GOP 中间接入时裸 P 帧解不出来(桌面端同款教训)
write_control(&mut ctl_send, &ControlMsg::KeyFrameReq).await?; write_control(&mut ctl_send, &ControlMsg::KeyFrameReq)
.await
.map_err(other_err)?;
let stats = Arc::new(std::sync::Mutex::new(SessionStats::default())); let stats = Arc::new(std::sync::Mutex::new(SessionStats::default()));
@@ -344,6 +600,44 @@ impl Session {
let _ = self.tx.send(Command::Input(ev)); let _ = self.tx.send(Command::Input(ev));
} }
/// 协议 v3:发送一段文本(UTF-16 code unit)到被控端注入。
///
/// 为什么参数是 `&[u16]` 而不是 `&str`:被控端的注入通道(Windows
/// `KEYEVENTF_UNICODE`)与主控端来源(Java 字符串)都是 UTF-16 code unit,
/// 中间任何一次 UTF-8 往返都会把 IME 提交边界的落单代理项变成替换字符。
/// 见 `common::InputEvent::Text` 的取舍说明。
///
/// 超限处理:截断到 [`common::MAX_TEXT_UNITS`] 并打日志——静默丢弃会让用户
/// 以为长文本已送达。日志只报长度,不报内容。
pub fn send_text(&self, units: &[u16]) {
if units.is_empty() {
return; // 空提交没有注入语义,也不该占用一条消息
}
let (units, truncated) = common::clamp_text_units(units.to_vec());
if truncated {
log(&format!(
"文本超过上限,已截断到 {} 个 code unit",
common::MAX_TEXT_UNITS
));
}
self.send_input(InputEvent::Text { units });
}
/// 协议 v4:发送**相对**鼠标位移(单位:被控端桌面物理像素)。
///
/// 与 [`Session::send_text`] 同样的空值纪律:零位移不占一条消息——它没有注入语义,
/// 只会白占一个 QUIC 帧(触摸事件里 `dx==0 && dy==0` 是常态,手指静止时每帧都发)。
///
/// 语义是「从远端指针当前位置移动 (dx,dy)」,远端负责 clamp 到桌面内。
/// 主控端**不要**先算绝对坐标再发——那就退化成绝对模式,失去相对鼠标的意义
/// (本机真鼠标同时移动时会跳变)。
pub fn send_mouse_rel(&self, dx: i16, dy: i16) {
if dx == 0 && dy == 0 {
return;
}
self.send_input(InputEvent::MouseMoveRel { dx, dy });
}
/// 请求被控端强制关键帧(解码器错误/跳帧后的恢复路径)。 /// 请求被控端强制关键帧(解码器错误/跳帧后的恢复路径)。
pub fn request_keyframe(&self) { pub fn request_keyframe(&self) {
let _ = self.tx.send(Command::KeyFrameReq); let _ = self.tx.send(Command::KeyFrameReq);
@@ -389,12 +683,15 @@ mod tests {
ts_ms: 42, ts_ms: 42,
seq: 7, seq: 7,
keyframe: true, keyframe: true,
cursor: Some((640, 480)),
}; };
let payload = vec![0u8; 16]; let payload = vec![0u8; 16];
let f: Frame = (h.clone(), payload.clone()).into(); let f: Frame = (h.clone(), payload.clone()).into();
assert_eq!((f.width, f.height), (1920, 1080)); assert_eq!((f.width, f.height), (1920, 1080));
assert_eq!(f.seq, 7); assert_eq!(f.seq, 7);
assert!(f.keyframe); assert!(f.keyframe);
// v5:光标字段必须透传,丢它=标记永远不显示,且丢法是静默的
assert_eq!(f.cursor, Some((640, 480)));
assert_eq!(f.payload, payload); assert_eq!(f.payload, payload);
} }
@@ -424,8 +721,138 @@ mod tests {
insecure_skip_verify: false, insecure_skip_verify: false,
}; };
match Session::connect(cfg).await { match Session::connect(cfg).await {
Err(e) => assert!(e.to_string().contains("证书钉扎"), "实际错误: {e}"), Err(e) => {
assert_eq!(e.kind, ConnectErrorKind::Other);
assert!(e.detail.contains("证书钉扎"), "实际错误: {e}");
}
Ok(_) => panic!("insecure_skip_verify=false 必须明确报错,不得静默放行"), Ok(_) => panic!("insecure_skip_verify=false 必须明确报错,不得静默放行"),
} }
} }
#[test]
fn auth_fail_classification_uses_agent_message_prefix() {
// 判据来自 agent 侧的真实文案(agent/src/main.rs:111),不是自造样本
assert_eq!(
classify_auth_fail("protocol version mismatch: client=2, server=3"),
ConnectErrorKind::Version
);
assert_eq!(classify_auth_fail("bad code"), ConnectErrorKind::Auth);
// 未知形态宁可归 Auth 也不静默
assert_eq!(
classify_auth_fail("expected Auth, got Ping(0)"),
ConnectErrorKind::Auth
);
}
#[test]
fn ffi_error_codes_are_stable_and_unique() {
// FFI 编号是 Kotlin 映射表的契约面:改动=公共 API 变更。
// 期望值写死(R-8:判据是等于期望值,不是"没重复")。
let expect = [
(ConnectErrorKind::Resolve, 1),
(ConnectErrorKind::Timeout, 2),
(ConnectErrorKind::Refused, 3),
(ConnectErrorKind::Auth, 4),
(ConnectErrorKind::Version, 5),
(ConnectErrorKind::Other, 9),
];
for (kind, code) in expect {
assert_eq!(kind.code(), code);
}
}
#[tokio::test]
async fn unresolvable_host_is_classified_as_resolve() {
// 本机环境事实(2026-10-31 实测):这台办公机的 DNS 拦截层(Meta/198.18.0.1)
// 把 `.invalid` 直接**解析成功**并指向 198.18.0.x 假地址——RFC 6761 的
// NXDOMAIN 保证在这条链路上不成立,`.invalid` 样本会走成 Timeout。
// 含空格的 name 在任何 resolver 里都是非法输入、快速失败(实测 26ms),
// 判据不依赖本机 DNS 行为。
let cfg = ConnectConfig::direct("a b", 47800, "t");
let e = match Session::connect(cfg).await {
Ok(_) => panic!("非法主机名必须解析失败"),
Err(e) => e,
};
assert_eq!(e.kind, ConnectErrorKind::Resolve, "实际: {e:?}");
assert!(e.detail.contains("a b"), "detail 必须含原输入: {e}");
}
#[tokio::test]
async fn unreachable_peer_redials_then_reports_timeout() {
// TEST-NET-1(RFC 5737):无可达主机 → 每拨握手都无应答。
// 方案①(用户裁决):第一拨 5s 超时 → 重拨 → 第二拨 8s 超时 → 报 Timeout。
// 期望落点 [13s-1s, 14s+2s):
// - 下界 ≈ 两拨握手预算之和,证明**第二拨真的跑完**(重拨丢失就是回归);
// - 上界证明总预算封顶(不会滚向第三拨)。
// 修复前(quinn 默认)这里要 32s;U8 判据形态更新为「≤13s 且全程可取消」。
let cfg = ConnectConfig::direct("192.0.2.1", 47800, "t");
let t0 = Instant::now();
let e = match Session::connect(cfg).await {
Ok(_) => panic!("TEST-NET-1 无主机,必须失败"),
Err(e) => e,
};
let elapsed = t0.elapsed();
assert_eq!(e.kind, ConnectErrorKind::Timeout, "实际: {e:?}");
let two_dials = CONNECT_BUDGET + CONNECT_RETRY_BUDGET;
assert!(
elapsed >= two_dials - Duration::from_secs(1)
&& elapsed < CONNECT_TOTAL_BUDGET + Duration::from_secs(2),
"重拨后的等待 {elapsed:?} 不在 [{two_dials:?}-1s, {CONNECT_TOTAL_BUDGET:?}+2s)"
);
}
#[test]
fn budget_constants_match_redial_design() {
// 方案①常量关系固化(2026-10-31 用户裁决):
// - 第一拨握手 5s:不可达的快速探测
// - 第二拨握手 8s:第二拨起点在 ~5s,需盖住实测僵尸释放点 ~10.2s
// (R-5 idle 超时)+ 余量——比第一拨长是这个物理原因,不是随手加大
// - 总预算 14s:两拨之和 + 1s,让内层各拨 Timeout 先触发(错误 detail
// 精确到段),外层只封顶失控组合
// - 认证 11s:> 僵尸释放 10.2s(实测),握手成功后排队不被压断
assert_eq!(CONNECT_BUDGET, Duration::from_secs(5));
assert_eq!(CONNECT_RETRY_BUDGET, Duration::from_secs(8));
assert_eq!(AUTH_BUDGET, Duration::from_secs(11));
assert_eq!(CONNECT_TOTAL_BUDGET, Duration::from_secs(14));
assert_eq!(MAX_CONNECT_ATTEMPTS, 2);
assert!(CONNECT_RETRY_BUDGET > CONNECT_BUDGET, "第二拨必须更长才盖得住释放点");
assert!(CONNECT_TOTAL_BUDGET > CONNECT_BUDGET + CONNECT_RETRY_BUDGET);
assert!(AUTH_BUDGET > Duration::from_secs(10));
// 重拨名单是策略面:只有暂时性失败可重试,改动必须过这条测试
assert!(is_retryable(ConnectErrorKind::Timeout));
assert!(is_retryable(ConnectErrorKind::Refused));
for k in [
ConnectErrorKind::Resolve,
ConnectErrorKind::Auth,
ConnectErrorKind::Version,
ConnectErrorKind::Other,
] {
assert!(!is_retryable(k), "{k:?} 是确定性失败,不得重拨");
}
}
#[test]
fn connection_error_variants_map_to_declared_kinds() {
// 只测**可安全构造**的单元变体(unit variants):TransportError /
// ConnectionClosed / ApplicationClose 的字段是 `bytes::Bytes`,测试构造
// 需要把 bytes 提为直接依赖——为分类表测试引依赖不值当(偏差 D-R1)。
// 这三个结构变体的映射由 match 的穷尽性 + 上面的注释钉住;
// 真实归类靠 V-A2 的端到端(端口上跑非 QUIC 服务)覆盖。
assert_eq!(
classify_connection_error(&quinn::ConnectionError::TimedOut),
ConnectErrorKind::Timeout
);
assert_eq!(
classify_connection_error(&quinn::ConnectionError::Reset),
ConnectErrorKind::Refused
);
assert_eq!(
classify_connection_error(&quinn::ConnectionError::VersionMismatch),
ConnectErrorKind::Other
);
assert_eq!(
classify_connection_error(&quinn::ConnectionError::LocallyClosed),
ConnectErrorKind::Other
);
}
} }