Author SHA1 Message Date
郑博文 06bb7bcd1c feat(android): 控件浮层化——statusView/sensBar/controlBar 移入 stage 底部叠加层
三个控件原是 stage 的垂直兄弟节点,显示即抢高度,于是 surfaceView 的
letterbox 几何跟着变:横屏实测控件可见时画面 1216x684、隐藏时 1920x1080,
即用户感知的「一按按钮画面就被挤小」。

改为 stage 内的 chromeOverlay(LinearLayout,Gravity.BOTTOM,
WRAP_CONTENT,底色 0x99000000 半透明):画面尺寸现在与控件显隐无关。
setControlsVisible 仍统一 GONE 三个子视图,容器随之塌成 0 高,隐藏态与
改动前逐字节等价,4a 的黑边唤出逻辑一行未动。

不碰解码、触摸映射、协议与 CursorOverlayView。竖屏实测 stage 高 2069
(等于内容区高度)、画面恒定 1080x607,连续 4 次采样无差异;横屏可见态与
隐藏态画面均为 1920x1080;R7 绝对映射三点误差 1px/0px/1px;JVM 60 例全绿。
2026-11-02 16:53:27 +08:00
郑博文 3b219a442d test(tools): cursor_proof + proof_common 抽取 + rel_mouse 适配 v5/双屏与污染修复
4b 光标覆盖层验收(V-D1/D3/D6/D7)+ 4a 建议 2 的公共库沉淀:

- proof_common.ps1:抽 landscape/rel_mouse 已验证的观测面 helper
  (唤醒只从证明过的隐藏态、正向配对判隐藏、N25 唯一临时文件、按文案取按钮、
  静置门控 + SetCursorPos 读回自校验、内存预检、R-7 新鲜守卫)。
  老 proof 本轮**不迁移**(改 4 个已验证脚本而不全重跑=假绿风险,登记下轮卫生项)
- cursor_proof.ps1:真机判据
  V-D1 截图青簇质心 vs CursorMarker 映射 ±6px(含近原点 (48,48),Some(0,0)≠None 由单测+golden 锁)
  V-D3 静止桌面跳转后 900ms 内标记跟随(覆盖 500ms 保活重发带新鲜 cursor)
  V-D7 指针进左侧副屏负 X → agent 折叠 None → 无青簇(本机双屏白送的真实判据)
  reject 阶段 V-D6:旧 v4 APK 连 v5 agent → 屏幕出「协议版本不一致」+ detail client=4,server=5
  实测全绿:4 点偏差 ≤0.6px、跟随 0.5px、副屏无簇、拒绝判据命中
- rel_mouse_phone_proof.ps1:适配 v5 + 修两处 harness 缺陷(都是本轮回归实测逼出的)
  ① HUD v4 断言→v5(PROTO_LABEL 配对);② V-B2 R7 期望值从写死 1920 派生
     改为按实时桌面 / 动态算(桌面已漂到 2560x1440,旧写死值假报 575px,
     违反 AGENTS.md 坐标期望按当前分辨率重算)
  ③ 模式守卫改为无条件 Set-Mode relative(旧 HUD.Contains(绝对) 在 HUD 读空时
     静默跳过,上一轮 throw 残留的 relative_mode=false 污染了整个 step2——
     绝对模式 DOWN 发绝对移动是正确产品行为,被 2c 误读成 teleport)
  ④ Set-Mode 唤醒 miss 并入 try 预算(原来单次重读即 throw,烧掉整脚本)+ 失败截图证据
  实测:R7 误差 1px、2c maxJump=0、长按右键 0→1、全链路绿
2026-11-02 12:39:23 +08:00
郑博文 45daba5419 chore(android): 4b 版本落账(HUD v5 标签 + APK versionCode 5)
- PROTO_LABEL 4→5:与 common::PROTO_VERSION 的人工同步义务(R5 显示要求),
  rel_mouse_phone_proof 的 HUD v5 断言与之配对(R-5 同源注释写明)
- versionCode 4→5、versionName 0.1-phase4b:真机验收跑的就是这个包,
  dumpsys 版本号是 proof 判「装没装对」的唯一依据(R-7 的 APK 侧对应物)
2026-11-02 12:38:40 +08:00
郑博文 efcaeaad26 test(android): 光标映射 8 例 + 帧头拆包 golden bytes 6 例
4b/V-D1 的单元侧判据(跨端契约 + 映射语义):
- CursorMarkerTest:中心/满量程/(0,0) 合法/1:1 恒等映射;null、尺寸未就绪、
  越界全分支返回 null。origin_maps_not_treated_as_none 锁最危险的错——
  把左上角误当无光标(协议用 Option 的全部理由)
- RustBridgeTest:goldenHeader 逐字节手抄(与 android-ffi 写端测试同一组常数对拍),
  锁 [25]/[26..28]/[28..30] 三个新偏移 + 有符号 i16 + 截断缓冲防御 + 空载荷边界
- R-8 反向验证:读偏移 26→27 后 4 例坐标测试确实红(已改回);
  CursorMarker 边界 >= 改 <= 后 origin 用例确实红(已改回)
JVM 全量 60 passed / 0 failed(--rerun-tasks 新鲜跑,XML 时间戳核对)。
2026-11-02 11:05:24 +08:00
郑博文 9a6f6962d2 feat(android): v5 光标标记层(拆包迁移 FrameCodec + CursorMarker + Overlay + 接线)
4b 手机端「看得见远端鼠标」的实现(协议 v5 的消费端):

- RustBridge:Frame 增 cursor: Pair<Int,Int>?(可空 Pair 忠实映射 Option<(i16,i16)>,
  杜绝「x 有 y 无」非法态);拆包逻辑连同字节 reader 迁到顶层 FrameCodec——
  RustBridge object 带 init{loadLibrary},测试里不可达,抽成无 native 的顶层 object
  才能把跨端偏移放进 JVM 单测(核过 Rust 一端≠核过 Kotlin 另一端)
- CursorMarker.mapToView:桌面坐标→视图坐标等比映射(letterbox 下线性等比,
  与 RelPointer 同一套比例);null/尺寸未就绪/越界一律返回 null(不画),
  绝不用 (0,0) 当缺省——那会把「无光标」和「光标在左上角」抹平成一样
- CursorOverlayView:SurfaceView 的上层兄弟视图(surface 合成在窗口之下,
  同视图无法自叠加);双描边圆环+青色中心点(兼作 proof V-D1 的颜色特征码,
  UI 层绘制不经 H.264,screencap 里颜色精确);volatile + postInvalidate 跨线程;
  不消费触摸(覆盖整个 stage,若吃事件会同时废掉远端操作和黑边唤出)
- MainActivity:解码循环每帧喂 cursorOverlay.updateCursor(f.cursor)(放解码前,
  光标是帧头字段与解码成败无关);cleanupSession 与 disconnect 两条收尾路径同改
  清标记(R-5 同源,旧会话光标留着会冒充新会话)

assembleDebug BUILD SUCCESSFUL(Canvas API 编译过);JVM 60 passed。
2026-11-02 11:05:09 +08:00
郑博文 4379b01d41 refactor(android-ffi): 抽 pack_frame_header 并加写端字面量对拍单测
步骤 4 的跨端契约闭环:把 30 字节头构造从 nativeNextFrame 内联提取成纯函数,
写端只有一条路径(消除内联/函数两处并存 = R-5 同源漂移)。

新增 host 可跑的 3 条单测(#[cfg(test)],进 --workspace --exclude linux-agent 口径):
- header_bytes_match_the_declared_layout:逐字节手抄常数,与 Kotlin 侧
  RustBridgeTest.goldenHeader 是同一组——两侧各握一份手抄,改任一偏移另一侧必红
  (只锁读端有盲区:Kotlin 字面量若本身抄错 Rust 布局会跟着一起错还全绿)
- none_cursor_writes_zero_flag_and_zero_coords:None 分支 has=0 且坐标字节不残留脏值
- negative_coords_keep_bit_pattern:i16 负坐标位模式忠实
android-ffi 在 host(gnu) cargo test 可编译可跑,实测 3 passed。
2026-11-02 11:04:51 +08:00
郑博文 8eed1e20e1 feat(viewer-core,android-ffi): v5 光标字段透传到 FFI(帧头 25→30 字节)
主控链路把协议帧头的光标坐标交给 Kotlin:
- viewer-core::Frame 增 cursor: Option<(i16,i16)>,From 实现透传;
  契约测试改为 Some((640,480)) 并断言透传——丢它=标记永不显示且丢法是静默的
- android-ffi 定长头 25→30:尾部追加 [has_cursor 1][x i16 LE 2][y i16 LE 2],
  放末尾保证 Kotlin 既有偏移量(0/1/9/17/21)全部不移动
- None → has=0 且坐标字节恒 0(无意义);Some 分支显式写两坐标,
  杜绝「Some 但忘写坐标」的编辑事故(文件头注释同步 30 字节布局)
- R-5 同源检查:RustBridge.kt 的 FRAME_HEADER_LEN=25 与拆头偏移是本次协议变更的
  同步改动点,Kotlin 侧在下一提交完成(同一循环、相邻提交)
2026-11-02 10:39:19 +08:00
郑博文 ad46fc805b feat(agent-core): ScreenCapturer 新增 pointer_state 默认实现并接入帧头
v5 光标字段在平台无关会话层的接缝(4b/Q4=A:Linux 实装另立循环):

- trait 新增 pointer_state(),默认实现返回 None + 会话期一次性日志——
  与 InputInjector::text() 未实现先例同款纪律(异常路径必须可见,且不逐帧刷屏)
- encode_send 增 cursor 参数,两条发送路径(新帧/静止强制 IDR)都传入
- 静止保活重发分支:split 出的旧帧头沿用会让标记冻住(画面静止≠指针静止),
  与 seq/ts_ms 同规重建 cursor
- 新增单测锁默认实现契约(None 且 panic-free,Linux/X11 端真实形态)
- linux-agent 不强制实现(cargo build --workspace 复验编译通过)
2026-11-02 10:36:47 +08:00
郑博文 b1ac32175c feat(agent): v5 光标采样接入帧头(GetCursorInfo + 折叠规则 + dump 观测列)
采集端把远端光标位置填进每帧帧头,供主控端画标记(4b 解决看不见远端鼠标)。

- fold_cursor 纯函数承载全部折叠判定:隐藏(CURSOR_SHOWING 未置位)/
  越出采集画面 [0,w)x[0,h)(副屏负坐标区,Q5=A)/超 i16 范围(防截断成
  看似合法的错坐标)→ None;坐标语义单一来源在被控端
- sample_cursor 只负责 Win32 读数(cbSize 契约注释)与失败/越界留痕
  (会话期各只打一条日志,可见后果但不逐帧刷屏)
- 采样点在 emit! 宏(唯一发送入口):新帧/静止 500ms 保活重发/强制 IDR
  三条路径天然全覆盖,帧头 cursor 与 ts_ms 同刻
- 前提:agent 的 per-monitor DPI 感知(3b/D2)保证 ptScreenPos 是物理像素
- agent crate 首批单测 4 例(界内含原点/隐藏/五类越界/i16 上界),
  mutation 反向验证(>= 改 > 后越界用例确实红)
- dump_stream 帧表加光标列,Windows 端实测(本机同回环):
  界内 Some(1000,600) 精确、副屏 -960 折叠 None、None->Some(300,250) 恢复
  全部 PASS;期间发现有人操控真鼠标,探针已改静置门控+读回自校验设计
2026-11-02 10:32:09 +08:00
郑博文 98b4f03cff feat(common): 协议 v5 帧头新增光标字段 cursor
4b 光标覆盖层的协议地基:VideoFrameHeader 尾部追加 Option<(i16,i16)>,
携带本帧发送时刻的远端光标物理像素坐标(None=隐藏/离屏/未实现三态折叠)。
解决「看不见远端鼠标」——相对模式下用户不知道指针在哪、轻点会落在哪。

- 字段追加改变 bincode 线格式,PROTO_VERSION 4→5,v4 旧端由握手显式拒绝(既有机制)
- v4 守卫按 v3 先例降为下界锚点,最新一代用精确等号守卫
- 新增 3 测试:Option 往返(含 i16 边界与负值)、尾部追加不扰动既有字段、
  None 紧凑性(Some 比 None 恰多 4 字节,证明光标未被固定宽度白占)
- agent/agent-core/viewer-core 构造点补占位 cursor: None(真实采样下一步接入),
  保证本提交全 workspace 可编译
2026-11-02 10:11:23 +08:00
郑博文 e96502f888 test(tools): 4a 横屏验收脚本 + 既有 proof 适配自动隐藏与同源缺陷修复
新增 landscape_proof.ps1:V-C1~C7 判据(自动横屏 <=2s、旋转零中断、
动画桌面下 >=45fps、控制条自动隐藏、黑边唤出零远端泄漏、四按钮可点、
方向四态循环、表单竖屏)。方向判定只解析 Display: mDisplayId=0 块内
cur=WxH——这台 vivo 无 mRotation= 行,且 dumpsys 里有常驻 1080x2400 的
虚拟屏,匹配首个 cur= 会在已横屏时误报未旋转(探针实测踩过)。

横屏自动隐藏与既有 proof 的判据自冲突(HUD 随控制条 GONE,RTT= 读不到
被误判断线/未连接),三个 proof 统一加「确认隐藏态才走黑边唤醒」:
phone_chain_proof、ime_chinese_proof、rel_mouse_phone_proof。

修 rel_mouse_phone_proof 两处:
- 2c 基线判据:横屏滑动 768px 的残余相对事件成批延迟落地,旧的「两次
  读数相等」被突发间隙骗过(实测假失败 351/1418px)。改为 6 读数沉降 +
  4 次采样判漂移 + 绝对泄漏点特征比对。
- step 4 靶窗启动缺 3 个 Mandatory 参数,子进程瞬间失败退出(3b 起一直
  如此,右键落在裸屏幕中心,靠近违反 R-11「不得注入他人窗口」)。现按
  契约启动并把指针停在自家窗口内。

R-5 同源排查:靶窗结果目录固定路径 + 启动时删自己的 stop.flag,会让
上一轮失败遗留的靶窗永远收不到退休信号、继续抢前台并写脏 received.txt
(实测收到 abc\nabc)。5 个 proof 统一改每轮唯一目录(\47512 后缀)。

ime_chinese_proof 的切换键标定失效:设备 IME 已换成搜狗(原为讯飞),
旧坐标 (146,2096) 静默点空。IME 按键视图对 uiautomator 不可见、无法机械
标定,按 R-9 转人工确认,不硬凑自动化。
2026-11-01 21:58:19 +08:00
郑博文 cff7fc8ab3 feat(android): 横屏远控(自动方向 + 控制条自动隐藏 + 黑边唤出)
竖屏看 16:9 桌面太小(用户 2026-11-01 反馈)。探针实测(docs/output/
2026-11-01-landscape-probe/):App 级 setRequestedOrientation 在这台 vivo
上有效(761ms),且旋转只触发 surfaceChanged、不销毁 surface、不断会话、
横屏 fps=58~60——所以不需要 TextureView(探针另测 TextureView 掉 30~37%
帧率)、不需要任何解码器重建配合,旧 S3 作废。

- OrientationPolicy(AUTO/LANDSCAPE/PORTRAIT/SYSTEM)纯逻辑 + 10 单测,
  常量值经 javap 对 android-34 核实;已做 mutation 反向验证(AUTO 判定
  > 改 >= 时方屏用例确实红)。
- 表单页强制竖屏;连接成功按策略设方向(默认 AUTO:远端宽>高即横屏)。
- 横屏 immersive + 状态行压 36dp + 控制条 3s 无操作自动隐藏。
- 唤出区 = letterbox 黑边(stage 在隐藏态消费画面外触摸)。刻意不用
  底部边缘带:R7 回归实测那种带会吃掉远端桌面底部 ~11%(Windows 任务栏
  就在那里),点任务栏会变成唤出菜单。黑边不属于 SurfaceView,零冲突。
- 竖屏永不隐藏(裁决只针对横屏空间不足)。
2026-11-01 21:57:49 +08:00
25 changed files with 2756 additions and 150 deletions

No files matched your search

+62 -2
View File
@@ -40,6 +40,22 @@ pub trait ScreenCapturer: Send {
/// 取出最近一帧,BGRA 紧凑排列(已去掉 row pitch),供 openh264 直接编码。 /// 取出最近一帧,BGRA 紧凑排列(已去掉 row pitch),供 openh264 直接编码。
fn copy_frame(&mut self) -> Result<Vec<u8>>; fn copy_frame(&mut self) -> Result<Vec<u8>>;
/// 协议 v5 的光标采样:当前指针在**采集画面内**的物理像素坐标。
/// 语义与折叠规则(隐藏/离屏 → None)见 `common::VideoFrameHeader::cursor`。
///
/// **默认实现刻意返回 None,但必须留下可见痕迹**:Linux 后端本轮未实现光标采样
/// (X11 `query_pointer` 与 Windows `GetCursorInfo` 不同构,且本机无 X11 无法给
/// proof——4b/Q4=A 裁决另立循环)。全局规则要求异常路径有可见后果——安卓用户对着
/// Linux 被控端看不到标记不能是静默的,日志必须说明是没实现。一次性日志防刷屏
/// (本方法每帧都会被调,最高 60Hz)。
fn pointer_state(&self) -> Option<(i16, i16)> {
static LOGGED_UNIMPL: AtomicBool = AtomicBool::new(false);
if !LOGGED_UNIMPL.swap(true, Ordering::Relaxed) {
eprintln!("[agent-core] 光标采样后端未实现:帧头 cursor 将恒为 None");
}
None
}
} }
/// 输入注入后端。实现需可在异步运行时中被多任务共享,故要求 `Send + Sync`。 /// 输入注入后端。实现需可在异步运行时中被多任务共享,故要求 `Send + Sync`。
@@ -176,6 +192,7 @@ fn encode_send(
(w, h): (u32, u32), (w, h): (u32, u32),
ts_ms: u64, ts_ms: u64,
seq: u64, seq: u64,
cursor: Option<(i16, i16)>,
tx: &watch::Sender<Option<Vec<u8>>>, tx: &watch::Sender<Option<Vec<u8>>>,
last_frame: &mut Option<Vec<u8>>, last_frame: &mut Option<Vec<u8>>,
) -> bool { ) -> bool {
@@ -189,6 +206,7 @@ fn encode_send(
ts_ms, ts_ms,
seq, seq,
keyframe: h264_contains_idr(&payload), keyframe: h264_contains_idr(&payload),
cursor,
}; };
let frame = encode_video_frame(&header, &payload); let frame = encode_video_frame(&header, &payload);
if tx.send(Some(frame.clone())).is_err() { if tx.send(Some(frame.clone())).is_err() {
@@ -247,7 +265,16 @@ fn capture_session(
encoder.force_intra_frame(); encoder.force_intra_frame();
} }
let ts = start.elapsed().as_millis() as u64; let ts = start.elapsed().as_millis() as u64;
if !encode_send(&mut encoder, &raw, (w, h), ts, seq, tx, &mut last_frame) { if !encode_send(
&mut encoder,
&raw,
(w, h),
ts,
seq,
cap.pointer_state(),
tx,
&mut last_frame,
) {
return LoopEnd::ChannelClosed; // 会话结束 return LoopEnd::ChannelClosed; // 会话结束
} }
seq = seq.wrapping_add(1); seq = seq.wrapping_add(1);
@@ -286,7 +313,16 @@ fn capture_session(
encoder.force_intra_frame(); encoder.force_intra_frame();
let raw = last_raw.as_ref().expect("can_key 已保证"); let raw = last_raw.as_ref().expect("can_key 已保证");
let ts = start.elapsed().as_millis() as u64; let ts = start.elapsed().as_millis() as u64;
if !encode_send(&mut encoder, raw, (w, h), ts, seq, tx, &mut last_frame) { if !encode_send(
&mut encoder,
raw,
(w, h),
ts,
seq,
cap.pointer_state(),
tx,
&mut last_frame,
) {
return LoopEnd::ChannelClosed; return LoopEnd::ChannelClosed;
} }
seq = seq.wrapping_add(1); seq = seq.wrapping_add(1);
@@ -300,6 +336,9 @@ fn capture_session(
Ok((mut hdr, payload)) => { Ok((mut hdr, payload)) => {
hdr.seq = seq; hdr.seq = seq;
hdr.ts_ms = start.elapsed().as_millis() as u64; hdr.ts_ms = start.elapsed().as_millis() as u64;
// v5 与 seq/ts_ms 同理:重发帧的 cursor 若沿用旧值会让标记
// 冻住(画面静止≠指针静止,触控板用户尤甚)
hdr.cursor = cap.pointer_state();
let frame = encode_video_frame(&hdr, payload); let frame = encode_video_frame(&hdr, payload);
if tx.send(Some(frame)).is_err() { if tx.send(Some(frame)).is_err() {
return LoopEnd::ChannelClosed; return LoopEnd::ChannelClosed;
@@ -690,4 +729,25 @@ mod tests {
dispatch_input(&inj, InputEvent::Text { units: vec![] }); dispatch_input(&inj, InputEvent::Text { units: vec![] });
assert_eq!(inj.received.lock().unwrap().clone(), vec![0]); assert_eq!(inj.received.lock().unwrap().clone(), vec![0]);
} }
/// v5:采集后端未实现 `pointer_state` 时必须返回 None 且不 panic——Linux/X11 端
/// 本轮的真实形态(4b/Q4=A:默认 None + 一次性日志,实装另立循环)。
/// 锁的是「默认实现存在且语义为 None」这条契约:若有人把默认实现改成 panic 或
/// 伪造坐标,这条会红。
#[test]
fn pointer_state_default_returns_none_without_panic() {
struct NoPointerCapturer;
impl ScreenCapturer for NoPointerCapturer {
fn size(&self) -> (u32, u32) {
(100, 100)
}
fn acquire(&mut self, _timeout_ms: u32) -> Result<bool> {
Ok(false)
}
fn copy_frame(&mut self) -> Result<Vec<u8>> {
Ok(Vec::new())
}
}
assert_eq!(NoPointerCapturer.pointer_state(), None);
}
} }
+15 -5
View File
@@ -114,6 +114,14 @@ fn arg(name: &str) -> Option<String> {
.cloned() .cloned()
} }
/// v5 观测列:帧头光标字段的稳定文本形态(proof 脚本按 `cursor=(x,y)` / `cursor=None` 解析)
fn cursor_cell(c: &Option<(i16, i16)>) -> String {
match c {
Some((x, y)) => format!("cursor=({x},{y})"),
None => "cursor=None".into(),
}
}
#[tokio::main] #[tokio::main]
async fn main() -> Result<()> { async fn main() -> Result<()> {
rustls::crypto::ring::default_provider() rustls::crypto::ring::default_provider()
@@ -184,8 +192,8 @@ async fn main() -> Result<()> {
// 累积裸码流,用于 --save // 累积裸码流,用于 --save
let mut saved: Vec<u8> = Vec::new(); let mut saved: Vec<u8> = Vec::new();
println!("\n序号 seq kf 载荷字节 NAL 结构 解码结果"); println!("\n序号 seq kf 载荷字节 NAL 结构 光标 解码结果");
println!("{}", "-".repeat(72)); println!("{}", "-".repeat(86));
for i in 0..total { for i in 0..total {
let (hdr, payload) = match tokio::time::timeout( let (hdr, payload) = match tokio::time::timeout(
@@ -221,11 +229,12 @@ async fn main() -> Result<()> {
// 模拟主控端积压跳帧:读出但不解码 // 模拟主控端积压跳帧:读出但不解码
if Some(i) == skip_at { if Some(i) == skip_at {
println!( println!(
"[{i:2}] {:3} {:<5} {:8} {:<14} ⏭ 跳过解码(模拟积压跳帧)", "[{i:2}] {:3} {:<5} {:8} {:<14} {:14} ⏭ 跳过解码(模拟积压跳帧)",
hdr.seq, hdr.seq,
hdr.keyframe, hdr.keyframe,
payload.len(), payload.len(),
describe(&types) describe(&types),
cursor_cell(&hdr.cursor)
); );
continue; continue;
} }
@@ -237,11 +246,12 @@ async fn main() -> Result<()> {
}; };
println!( println!(
"[{i:2}] {:3} {:<5} {:8} {:<14} {}", "[{i:2}] {:3} {:<5} {:8} {:<14} {:14} {}",
hdr.seq, hdr.seq,
hdr.keyframe, hdr.keyframe,
payload.len(), payload.len(),
describe(&types), describe(&types),
cursor_cell(&hdr.cursor),
res res
); );
+125
View File
@@ -36,6 +36,7 @@ use windows::Win32::Graphics::Dxgi::{
IDXGIDevice, IDXGIOutput1, IDXGIOutputDuplication, IDXGIResource, DXGI_ERROR_WAIT_TIMEOUT, IDXGIDevice, IDXGIOutput1, IDXGIOutputDuplication, IDXGIResource, DXGI_ERROR_WAIT_TIMEOUT,
DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO, DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
}; };
use windows::Win32::UI::WindowsAndMessaging::{GetCursorInfo, CURSORINFO, CURSOR_SHOWING};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -50,6 +51,31 @@ use std::time::{Duration, Instant};
/// 又不会因过密的整帧内编码抬高码率。 /// 又不会因过密的整帧内编码抬高码率。
pub const INTRA_PERIOD_FRAMES: u32 = 60; pub const INTRA_PERIOD_FRAMES: u32 = 60;
/// v5:光标可见性 + 采集边界折叠的**纯逻辑**(无 Win32 调用,可单测)。
///
/// 输入是 `GetCursorInfo` 的原始读数与采集画面尺寸;输出协议 `cursor` 字段语义
/// (见 `common::VideoFrameHeader::cursor`):
/// - 未 `CURSOR_SHOWING`(指针隐藏或被触控抑制)→ None
/// - 越出 `[0,w)×[0,h)`(副屏负坐标区、主屏外右侧等)→ None
/// ——采集画面里没有它,主控端画了位置也是错的(4b/Q5=A:语义单一来源在被控端)
/// - 越出 i16 正范围 → None 且第二返回 `true`(调用方打日志):协议用 i16 省字节,
/// 依赖「现实桌面 << 32767」前提;前提破了就折叠,**绝不截断出一个看起来合法的错坐标**
///
/// 独立成函数的原因:这套判定是 4b 折叠规则的全部智力内容,而 Win32 读数在
/// 有人用真鼠标的机器上不可复现——逻辑必须与采样解耦地被测试锁死。
fn fold_cursor(showing: bool, x: i32, y: i32, w: i32, h: i32) -> (Option<(i16, i16)>, bool) {
if !showing {
return (None, false);
}
if x < 0 || y < 0 || x >= w || y >= h {
return (None, false);
}
if x > i16::MAX as i32 || y > i16::MAX as i32 {
return (None, true);
}
(Some((x as i16, y as i16)), false)
}
pub struct Duplication { pub struct Duplication {
pub device: ID3D11Device, pub device: ID3D11Device,
context: ID3D11DeviceContext, context: ID3D11DeviceContext,
@@ -182,6 +208,42 @@ impl Duplication {
Ok(r) Ok(r)
} }
} }
/// v5:采样当前远端光标位置(**采集显示器内**的物理像素,(0,0)=画面左上角)。
///
/// 折叠规则(隐藏/离屏/i16 越界 → None)的唯一真源是 [`fold_cursor`]——
/// 那里是可单测的纯逻辑;本函数只负责 Win32 读数与失败留痕。
///
/// `GetCursorInfo` 失败 → None + 会话期只打一条日志(可见后果,但不逐帧刷屏)。
///
/// 前提:进程必须 per-monitor DPI 感知(`main.rs::init_dpi_awareness`,3b/D2 教训),
/// 否则 `ptScreenPos` 是逻辑坐标,与采集像素不同尺度。
pub fn sample_cursor(&self) -> Option<(i16, i16)> {
static LOGGED_FAIL: AtomicBool = AtomicBool::new(false);
static LOGGED_OVERFLOW: AtomicBool = AtomicBool::new(false);
unsafe {
let mut ci = CURSORINFO::default();
// Win32 契约:调用方必须先填 cbSize,否则 API 直接拒绝(MSDN GetCursorInfo)
ci.cbSize = std::mem::size_of::<CURSORINFO>() as u32;
if GetCursorInfo(&mut ci).is_err() {
if !LOGGED_FAIL.swap(true, Ordering::Relaxed) {
eprintln!("[capture] GetCursorInfo 失败:帧头 cursor 将恒为 None");
}
return None;
}
let (pos, overflow) = fold_cursor(
ci.flags.0 & CURSOR_SHOWING.0 != 0,
ci.ptScreenPos.x,
ci.ptScreenPos.y,
self.width as i32,
self.height as i32,
);
if overflow && !LOGGED_OVERFLOW.swap(true, Ordering::Relaxed) {
eprintln!("[capture] 光标坐标超出 i16 协议范围,已折叠为 None(桌面尺寸异常?)");
}
pos
}
}
} }
/// NVENC 硬编封装:H.264 High、CBR 码率控制、UltraLowLatency 调优、无 B 帧。 /// NVENC 硬编封装:H.264 High、CBR 码率控制、UltraLowLatency 调优、无 B 帧。
@@ -364,6 +426,10 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
// `$kf` 由调用点显式给出:NVENC 路径知道 `force_key`,软编路径同理。 // `$kf` 由调用点显式给出:NVENC 路径知道 `force_key`,软编路径同理。
// 不要在这里猜——首帧和 KeyFrameReq 都会产生 IDR,猜错会让解码端错过重同步点。 // 不要在这里猜——首帧和 KeyFrameReq 都会产生 IDR,猜错会让解码端错过重同步点。
//
// `cursor` 在**这里**采样(不在循环顶部):帧头语义是「本帧发送时刻的位置」。
// emit! 是唯一的发送入口,静止保活重发也走它(500ms 分支),所以新帧 /
// 静止重发 / 强制 IDR 三种发送路径天然全覆盖,无需各自补采样。
macro_rules! emit { macro_rules! emit {
($payload:expr, $kf:expr) => {{ ($payload:expr, $kf:expr) => {{
let header = VideoFrameHeader { let header = VideoFrameHeader {
@@ -372,6 +438,7 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
ts_ms: start.elapsed().as_millis() as u64, ts_ms: start.elapsed().as_millis() as u64,
seq, seq,
keyframe: $kf, keyframe: $kf,
cursor: dup.sample_cursor(),
}; };
let payload_vec: Vec<u8> = $payload; let payload_vec: Vec<u8> = $payload;
let frame = encode_video_frame(&header, &payload_vec); let frame = encode_video_frame(&header, &payload_vec);
@@ -585,3 +652,61 @@ pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc
std::thread::sleep(Duration::from_millis(300)); std::thread::sleep(Duration::from_millis(300));
} }
} }
#[cfg(test)]
mod tests {
use super::fold_cursor;
const W: i32 = 1920;
const H: i32 = 1080;
/// 界内可见 → 原样返回。中心与 (0,0) 都要覆盖:(0,0) 与 None 在协议里是两种语义,
/// 若实现把「左上角」误折叠成 None,光标到左上角就「消失」了(这是最容易写错的一条)。
#[test]
fn inside_bounds_pass_through_including_origin() {
assert_eq!(fold_cursor(true, 960, 540, W, H), (Some((960, 540)), false));
assert_eq!(fold_cursor(true, 0, 0, W, H), (Some((0, 0)), false));
assert_eq!(
fold_cursor(true, W - 1, H - 1, W, H),
(Some((1919, 1079)), false)
);
}
/// 隐藏/触控抑制(CURSOR_SHOWING 未置位)→ None,**即使坐标界内**。
/// 顺序很关键:坐标合法但不可见时也不能画标记。
#[test]
fn hidden_cursor_folds_to_none_even_when_in_bounds() {
assert_eq!(fold_cursor(false, 100, 100, W, H), (None, false));
assert_eq!(fold_cursor(false, 0, 0, W, H), (None, false));
}
/// 越出采集画面 → None:负坐标(副屏在左,本机实测 -1920..-1)、
/// 右/下边界外、以及**恰好在边界上**(坐标 w 对应第 w+1 列,画面里没有)。
#[test]
fn out_of_capture_bounds_folds_to_none() {
assert_eq!(fold_cursor(true, -960, 500, W, H), (None, false)); // 副屏(负 X)
assert_eq!(fold_cursor(true, 100, -1, W, H), (None, false)); // 负 Y
assert_eq!(fold_cursor(true, W, 500, W, H), (None, false)); // 右边界外一列
assert_eq!(fold_cursor(true, 500, H, W, H), (None, false)); // 下边界外一行
assert_eq!(fold_cursor(true, 2610, 1300, W, H), (None, false)); // 主屏外右侧
}
/// 越出 i16 正范围 → None + 第二返回 true(调用方据此打日志)。
/// 注意:常规采集尺寸(≤1920)下越界 x 永远先被边界检查拦下,i16 分支不可达——
/// 必须用超宽采集(假想带鱼屏)才能测到;它证明「界内但超 i16」会被折叠
/// 而**不是截断成看似合法的错坐标**。i16::MAX 恰好合法,不误报。
#[test]
fn beyond_i16_folds_with_overflow_flag() {
let huge_w = 40_000; // 假想的超宽采集,越过 i16::MAX
assert_eq!(
fold_cursor(true, i16::MAX as i32, 0, huge_w, H),
(Some((i16::MAX, 0)), false),
"i16::MAX 恰好界内合法,不该误报 overflow"
);
assert_eq!(
fold_cursor(true, 33_000, 10, huge_w, H),
(None, true),
"界内但超 i16 → overflow=true 的 None,而非截断成错误坐标"
);
}
}
+99 -12
View File
@@ -15,9 +15,13 @@
//! 句柄约定:`nativeConnect` 返回 Box 裸指针(永不为 0)作为 jlong; //! 句柄约定:`nativeConnect` 返回 Box 裸指针(永不为 0)作为 jlong;
//! `nativeDestroy` 释放。句柄释放后不得再使用,Kotlin 侧用置 0 保护。 //! `nativeDestroy` 释放。句柄释放后不得再使用,Kotlin 侧用置 0 保护。
//! //!
//! 帧返回约定:byte[],布局 = [1 标志][8 seq LE][8 ts LE][4 w LE][4 h LE][payload]。 //! 帧返回约定:byte[],布局 = [1 标志][8 seq LE][8 ts LE][4 w LE][4 h LE]
//! 定长头 25 字节;Kotlin 读 buf[0] 判断 keyframe(标志位复用为:0=P 帧, 1=IDR 帧, //! [1 has_cursor][2 cursorX i16 LE][2 cursorY i16 LE][payload]。
//! 会话结束则整个返回 null)。 //! 定长头 30 字节(v5 起;此前 25);Kotlin 读 buf[0] 判断 keyframe(标志位复用为:
//! 0=P 帧, 1=IDR 帧,会话结束则整个返回 null)。
//! has_cursor=0 时后 4 字节恒为 0 且**无意义**(指针隐藏/离屏/后端未实现,
//! 语义单一来源在被控端——见 common::VideoFrameHeader::cursor);=1 时坐标为
//! 采集画面内物理像素(0 ≤ x < w、0 ≤ y < h,被控端已折叠越界值)。
use common::InputEvent; use common::InputEvent;
use jni::objects::{JCharArray, JClass, JObject, JString}; use jni::objects::{JCharArray, JClass, JObject, JString};
@@ -26,8 +30,93 @@ use jni::JNIEnv;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use viewer_core::{ConnectConfig, Session}; use viewer_core::{ConnectConfig, Session};
/// 帧头长度:keyframe 标志(1) + seq(8) + ts(8) + w(4) + h(4) /// 帧头长度:keyframe 标志(1) + seq(8) + ts(8) + w(4) + h(4) + has_cursor(1) + x(2) + y(2)
const FRAME_HEADER_LEN: usize = 25; /// (v5:25→30,尾部三字段是光标;放末尾保证 Kotlin 旧偏移量全部不移动)
const FRAME_HEADER_LEN: usize = 30;
/// 把一个帧的元数据打包成定长头字节(**不含载荷**)。
///
/// 独立成纯函数并被单测钉住的原因:Kotlin 侧 `FrameCodec` 按同样的偏移读这些字节,
/// 任一侧挪一位,两端都编译得过、真机也连得上,只是标记画错位置——静默失败。
/// 单测用**手写字面量**断言(不是"算出来的期望值"),这样 RustBridgeTest 的
/// golden bytes 与这里是同一组常数对拍:改了写入布局,两边至少一边必红。
fn pack_frame_header(f: &viewer_core::Frame) -> [u8; FRAME_HEADER_LEN] {
let mut head = [0u8; FRAME_HEADER_LEN];
head[0] = f.keyframe as u8;
head[1..9].copy_from_slice(&f.seq.to_le_bytes());
head[9..17].copy_from_slice(&f.ts_ms.to_le_bytes());
head[17..21].copy_from_slice(&f.width.to_le_bytes());
head[21..25].copy_from_slice(&f.height.to_le_bytes());
// v5 光标三字段(偏移 25/26/28):None → has=0 且坐标字节保持 0(无意义)。
// 用显式 match 而不是 `f.cursor.is_some() as u8`——两处读同一个 Option,
// 任何「Some 但忘写坐标」的编辑错误都会在这里暴露。
match f.cursor {
Some((x, y)) => {
head[25] = 1;
head[26..28].copy_from_slice(&x.to_le_bytes());
head[28..30].copy_from_slice(&y.to_le_bytes());
}
None => head[25] = 0,
}
head
}
#[cfg(test)]
mod tests {
use super::*;
/// 构造最小 Frame(载荷只放一字节,本测试只关心定长头)。
fn frame(seq: u64, cursor: Option<(i16, i16)>) -> viewer_core::Frame {
viewer_core::Frame {
width: 1920,
height: 1080,
ts_ms: 0x0011_2233_4455_6677,
seq,
keyframe: true,
cursor,
payload: vec![0xAB],
}
}
/// **写端**字面量对拍:期望值逐字节手写,与 Kotlin 侧 RustBridgeTest 的
/// goldenHeader 是同一组常数。两侧各握一份手抄常数,改任何一侧的偏移,
/// 另一侧必红——这才是跨端契约的闭环(golden 测试只锁读端时有盲区:
/// 若 Kotlin 的字面量本身抄错了 Rust 布局,读端测试会跟着一起错还全绿)。
#[test]
fn header_bytes_match_the_declared_layout() {
let head = pack_frame_header(&frame(0x0102_0304_0506_0708, Some((960, 540))));
assert_eq!(
head,
[
0x01, // [0] keyframe=true
0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, // [1..9] seq=0x0102..0708 LE
0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00, // [9..17] ts LE
0x80, 0x07, 0x00, 0x00, // [17..21] w=1920=0x0780 LE
0x38, 0x04, 0x00, 0x00, // [21..25] h=1080=0x0438 LE
0x01, // [25] has_cursor
0xC0, 0x03, // [26..28] x=960=0x03C0 LE
0x1C, 0x02, // [28..30] y=540=0x021C LE
]
);
}
/// None 分支:has=0 且**坐标字节必须为 0**(vec 初始化保证),Kotlin 侧
/// 测试锁「has=0 时无条件 null」,这里锁「写端不残留脏坐标」。
#[test]
fn none_cursor_writes_zero_flag_and_zero_coords() {
let head = pack_frame_header(&frame(1, None));
assert_eq!(head[25], 0);
assert_eq!(&head[26..30], &[0u8; 4]);
}
/// 负坐标(协议层不该到达,但写端要忠实 i16 位模式——符号截断是静默错位)。
#[test]
fn negative_coords_keep_bit_pattern() {
let head = pack_frame_header(&frame(1, Some((-1, -32768))));
assert_eq!(&head[26..28], &[0xFF, 0xFF]);
assert_eq!(&head[28..30], &[0x00, 0x80]);
}
}
struct SessionBox { struct SessionBox {
session: Arc<Session>, session: Arc<Session>,
@@ -162,13 +251,11 @@ pub extern "system" fn Java_com_verify_remote_RustBridge_nativeNextFrame<'local>
match rt.block_on(session.next_frame()) { match rt.block_on(session.next_frame()) {
None => std::ptr::null_mut(), None => std::ptr::null_mut(),
Some(f) => { Some(f) => {
let mut buf = vec![0u8; FRAME_HEADER_LEN + f.payload.len()]; // 布局单一真源是 pack_frame_header(那里有字面量对拍单测)
buf[0] = f.keyframe as u8; let head = pack_frame_header(&f);
buf[1..9].copy_from_slice(&f.seq.to_le_bytes()); let mut buf = Vec::with_capacity(FRAME_HEADER_LEN + f.payload.len());
buf[9..17].copy_from_slice(&f.ts_ms.to_le_bytes()); buf.extend_from_slice(head.as_slice());
buf[17..21].copy_from_slice(&f.width.to_le_bytes()); buf.extend_from_slice(&f.payload);
buf[21..25].copy_from_slice(&f.height.to_le_bytes());
buf[FRAME_HEADER_LEN..].copy_from_slice(&f.payload);
match env.byte_array_from_slice(&buf) { match env.byte_array_from_slice(&buf) {
Ok(a) => a.into_raw(), Ok(a) => a.into_raw(),
Err(_) => std::ptr::null_mut(), Err(_) => std::ptr::null_mut(),
+2 -2
View File
@@ -11,8 +11,8 @@ android {
applicationId = "com.verify.remote" applicationId = "com.verify.remote"
minSdk = 26 minSdk = 26
targetSdk = 34 targetSdk = 34
versionCode = 3 versionCode = 5
versionName = "0.1-phase3b" versionName = "0.1-phase4b"
// 阶段 1 只测真机 arm64;x86_64 留给模拟器回归 // 阶段 1 只测真机 arm64;x86_64 留给模拟器回归
ndk { ndk {
abiFilters += listOf("arm64-v8a") abiFilters += listOf("arm64-v8a")
@@ -0,0 +1,50 @@
package com.verify.remote
/**
* 光标标记的**纯逻辑**:把协议帧头里的远端光标坐标(采集画面内物理像素)映射到
* 手机视图上的绘制位置。抽成纯 Kotlin 是为了能被 JVM 单测钉死(V-D1 判据依赖它)。
*
* 为什么映射就是简单的等比缩放:[RemoteSurfaceView] 是 letterbox——视图宽高比恒等于
* 远端桌面宽高比(`videoAspectRatio = remoteW / remoteH`),且视图左上角对应桌面 (0,0)。
* 于是「视图内像素 ↔ 桌面像素」是一个线性等比,与 [RelPointer.toDesktopDelta] 用的是
* **同一套比例**(那里是反向:视图像素增量→桌面像素增量)。
*
* 这里的边界校验**不是**对被控端 `fold_cursor` 的重复防御:那条在 Windows 被控端,
* 而本协议把对端字节一律按**不可信输入**处理(见 viewer-core 纪律、[RustBridge.nextFrame] 的长度闸)。
* 跨信任边界的输入必须自证——尤其 Linux 后端当前恒发 None,未来实装时若折叠写错,
* 这里不兜住就会把标记画到画面外。
*/
object CursorMarker {
/** 映射后的绘制点(视图内浮点像素)。绘制侧([CursorOverlayView])按此画圆环中心。 */
data class OnView(val x: Float, val y: Float)
/**
* @param desktopW/desktopH 远端采集画面尺寸(物理像素,来自 AuthOk/desktopSize)
* @param cursor 帧头光标坐标(桌面物理像素),null = 隐藏/离屏/未实现
* @param viewW/viewH SurfaceView 当前测量尺寸(像素)
* @return 视图内绘制点;任一维度非法或坐标越界 → null(不画标记)
*
* null 的所有分支都必须走「不画」而不是「画在 (0,0)」:(0,0) 是一个**合法**的光标位置
* (左上角),把它当成默认值会让「没有光标」和「光标在左上角」看起来一样——
* 这正是协议用 Option 而不用 0 的原因,Kotlin 侧不能把这层区分抹平。
*/
fun mapToView(
desktopW: Int,
desktopH: Int,
cursor: Pair<Int, Int>?,
viewW: Int,
viewH: Int,
): OnView? {
if (cursor == null) return null
// 尺寸未就绪(视图尚未测量 / desktopSize 还没取到):不画,而不是按 0 除
if (desktopW <= 0 || desktopH <= 0 || viewW <= 0 || viewH <= 0) return null
val (cx, cy) = cursor
// 采集画面 [0,w)×[0,h) 之外不画(越界 = 后端折叠漏网或被篡改)
if (cx < 0 || cy < 0 || cx >= desktopW || cy >= desktopH) return null
return OnView(
x = cx.toFloat() * viewW.toFloat() / desktopW.toFloat(),
y = cy.toFloat() * viewH.toFloat() / desktopH.toFloat(),
)
}
}
@@ -0,0 +1,95 @@
package com.verify.remote
import android.content.Context
import android.graphics.Canvas
import android.graphics.Color
import android.graphics.Paint
import android.view.View
/**
* 远端光标标记层(4b/Q1=A、Q2=A、Q6=A):画在 [RemoteSurfaceView] **之上**的透明兄弟视图。
*
* 为什么是兄弟视图而不是 SurfaceView 自己画:SurfaceView 的内容合成在窗口之下
* (窗口在其位置打透明洞),同一视图内无法给自己的 surface 叠加绘制。兄弟 View 盖在
* 洞上方是标准做法(视频播放器控件同理)。[RemoteSurfaceView] 保持纯采集/触摸职责不变。
*
* 坐标换算全部委托 [CursorMarker.mapToView](纯逻辑、JVM 单测钉死);本类只负责:
* 接收解码线程写来的光标 → invalidate → 在 UI 线程把「桌面坐标」画成「屏幕坐标」。
* 锚点几何(surfaceView 的 x/y/width/height)在 onDraw 现取——旋转/重布局后自动跟新,
* 无需订阅布局回调。
*
* 线程纪律:[updateCursor] 由**解码线程**每帧调用(与 stopDecoding 同款的跨线程写入),
* 字段 volatile + [postInvalidate](安卓文档指定的跨线程失效 API)。绘制侧只在 UI 线程读。
*
* 触摸:本视图**不消费任何触摸**(不可点击、不可聚焦)——它覆盖整个 stage,若吃掉事件
* 会同时废掉远端操作和黑边唤出(R-7/4a 的判据会当场全红)。事件按 FrameLayout 规则
* 下沉到 SurfaceView,行为与 4a 完全一致。
*/
class CursorOverlayView(context: Context) : View(context) {
/** 标记直径 22dp(Q2=A)。横屏 480dpi 下 ≈66px ≈ 桌面 33px 等比,够看清又不挡目标。 */
private val markerRadiusPx = 11f * context.resources.displayMetrics.density
/**
* 中心点兼作 proof 颜色特征码(青色非纯白):真机判据 V-D1 用 screencap 找这个
* 颜色的像素簇并断言簇心 == 期望映射点。覆盖层绘制发生在解码**之后**的 UI 层,
* 不经过 H.264 有损链路,screencap 里颜色精确可匹配。圆环仍按 Q2 双描边。
*/
private val dotColor = Color.rgb(0x00, 0xE5, 0xFF)
private val ringPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.STROKE
color = Color.WHITE
strokeWidth = 2f * context.resources.displayMetrics.density
}
private val outerRingPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.STROKE
color = Color.BLACK
strokeWidth = 4f * context.resources.displayMetrics.density
}
private val dotPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply {
style = Paint.Style.FILL
color = dotColor
}
/**
* 当前帧头光标(桌面物理像素)。null = 无标记:还没收到带光标的帧、
* 指针隐藏/离屏、或被控端后端未实现——三种情况都必须**什么都不画**。
*/
@Volatile
private var cursor: Pair<Int, Int>? = null
/** 解码线程调用;null 也走这里(清除标记同样必须触发重绘,否则标记会留在原位)。 */
fun updateCursor(newCursor: Pair<Int, Int>?) {
cursor = newCursor
postInvalidate()
}
/** 由 Activity 接线:几何锚点就是 SurfaceView(letterbox 后的实际画面矩形)。 */
var anchor: RemoteSurfaceView? = null
init {
isClickable = false
isFocusable = false
}
override fun onDraw(canvas: Canvas) {
super.onDraw(canvas)
val a = anchor ?: return
val c = cursor ?: return
val p = CursorMarker.mapToView(
desktopW = a.remoteWidth,
desktopH = a.remoteHeight,
cursor = c,
viewW = a.width,
viewH = a.height,
) ?: return
// 视图内坐标 + 画面在 stage 里的原点偏移(黑边存在时不为 0)
val sx = a.x + p.x
val sy = a.y + p.y
// 双描边:先粗黑再细白——白环两侧各露 1dp 黑边,深浅背景都保底可见(Q2)
canvas.drawCircle(sx, sy, markerRadiusPx, outerRingPaint)
canvas.drawCircle(sx, sy, markerRadiusPx, ringPaint)
canvas.drawCircle(sx, sy, markerRadiusPx * 0.24f, dotPaint)
}
}
@@ -37,6 +37,8 @@ import kotlin.concurrent.thread
class MainActivity : Activity(), RemoteSurfaceView.Listener { class MainActivity : Activity(), RemoteSurfaceView.Listener {
private lateinit var surfaceView: RemoteSurfaceView private lateinit var surfaceView: RemoteSurfaceView
/** 4b:远端光标标记层(SurfaceView 的上层兄弟视图,常显无开关 Q6=A) */
private lateinit var cursorOverlay: CursorOverlayView
private lateinit var statusView: TextView private lateinit var statusView: TextView
private lateinit var connectPanel: LinearLayout private lateinit var connectPanel: LinearLayout
private lateinit var remotePanel: LinearLayout private lateinit var remotePanel: LinearLayout
@@ -46,10 +48,20 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
private lateinit var connectButton: Button private lateinit var connectButton: Button
private lateinit var disconnectButton: Button private lateinit var disconnectButton: Button
private lateinit var modeButton: Button private lateinit var modeButton: Button
private lateinit var orientationButton: Button
private lateinit var sensUpButton: Button private lateinit var sensUpButton: Button
private lateinit var sensDownButton: Button private lateinit var sensDownButton: Button
private lateinit var sensLabel: TextView private lateinit var sensLabel: TextView
private lateinit var connectStatus: TextView private lateinit var connectStatus: TextView
/** 4a:横屏自动隐藏的两个控制条(Q3=A)+ 状态行 */
private lateinit var controlBar: LinearLayout
private lateinit var sensBar: LinearLayout
/**
* 循环 5:三个控件的**浮层容器**,是 [stage] 的底部叠加子视图——不再是画面
* 的垂直兄弟。`setControlsVisible` 把三个视图一起 GONE,容器 WRAP_CONTENT
* 随之塌成 0 高,于是 stage / 画面的 letterbox 几何与控件显隐**无关**。
*/
private lateinit var chromeOverlay: LinearLayout
@Volatile @Volatile
private var handle = 0L private var handle = 0L
@@ -80,13 +92,119 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
@Volatile @Volatile
private var lastTouchDbg = "-" private var lastTouchDbg = "-"
/** 4a:方向策略(Q1=A 默认 AUTO,Q4=A 底栏按钮循环切换并落 prefs) */
private var orientationPolicy = OrientationPolicy.AUTO
/** 4a:首次进横屏提示一次「点底部唤出菜单」(Q7=A),只提示一次不骚扰 */
private var landscapeHintShown = false
// ---- 4a/Q3=A:横屏控制条自动隐藏(Q7=A:3s + 40dp 边缘带 + 首次 toast) ----
private val ui = android.os.Handler(android.os.Looper.getMainLooper())
private var controlsVisible = true
private val hideControlsTask = Runnable {
// 调度后可能已转回竖屏:执行时再判一次,竖屏永不隐藏(裁决只针对横屏)
if (isLandscape()) setControlsVisible(false)
}
private fun isLandscape(): Boolean =
resources.configuration.orientation == android.content.res.Configuration.ORIENTATION_LANDSCAPE
/**
* 控制条显隐的唯一入口(状态行 + 灵敏度行 + 底栏一起)。
* `controlsVisible` 同时是 stage 黑边唤出的判据,只有这一处写它。
*/
private fun setControlsVisible(v: Boolean) {
controlsVisible = v
val vis = if (v) View.VISIBLE else View.GONE
statusView.visibility = vis
sensBar.visibility = vis
controlBar.visibility = vis
}
/** 任何触摸/唤出都重新计时;只在横屏 + 已连接 + 非等待期才挂隐藏任务 */
private fun scheduleControlsHide() {
ui.removeCallbacks(hideControlsTask)
if (isLandscape() && handle != 0L && !connecting) {
ui.postDelayed(hideControlsTask, CONTROLS_HIDE_MS)
}
}
/**
* 横屏沉浸式:隐藏系统状态栏 + 导航栏(minSdk 26 → 用 legacy 标志;
* API 30+ 走 insetsController 分支,本机 API 29 实测走的是 legacy 路径)。
* IMMERSIVE_STICKY:滑动边缘临时唤出后自动收回,不打断远控操作。
*
* 名字刻意不叫 setImmersive——`Activity` 基类有同名废弃方法,
* Kotlin 会要求 override(真机编译实测 2026-11-01)。
*/
@Suppress("DEPRECATION")
private fun applyImmersive(on: Boolean) {
val flags = android.view.View.SYSTEM_UI_FLAG_HIDE_NAVIGATION or
android.view.View.SYSTEM_UI_FLAG_FULLSCREEN or
android.view.View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY or
android.view.View.SYSTEM_UI_FLAG_LAYOUT_STABLE or
android.view.View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION or
android.view.View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN
window.decorView.systemUiVisibility = if (on) flags else 0
}
/** 横屏时状态行压成 36dp 单行(纵向空间是稀缺资源),竖屏恢复 64dp 三行 */
private fun applyStatusLayout() {
val lp = statusView.layoutParams as LinearLayout.LayoutParams
lp.height = dp(if (isLandscape()) 36 else 64)
statusView.layoutParams = lp
statusView.maxLines = if (isLandscape()) 1 else 3
}
/**
* 按当前方向应用远控页形态(进面板、旋转各调一次)。
* 竖屏保持 3b 原样(常驻控制条)——本轮裁决只针对横屏空间不足的问题。
*/
private fun applyRemoteChrome() {
val land = isLandscape()
applyImmersive(land)
applyStatusLayout()
if (land) {
if (!landscapeHintShown) {
landscapeHintShown = true
android.widget.Toast.makeText(this, "无操作 3 秒隐藏菜单,点屏幕底部唤出",
android.widget.Toast.LENGTH_LONG).show()
}
setControlsVisible(true)
scheduleControlsHide()
} else {
setControlsVisible(true)
}
}
/**
* 按当前策略 + 已知远端尺寸应用屏幕方向(4a)。
*
* 探针实测(2026-11-01):App 级 requestedOrientation 在这台 vivo 上有效(761ms),
* 且旋转只触发 surfaceChanged、不销毁 surface、不断会话——所以这里**只设方向**,
* 不需要任何解码器/会话保护配套(旧 S3 作废的根据)。
*/
private fun applyOrientation() {
requestedOrientation = Orientation.requestFor(orientationPolicy, remoteW, remoteH)
}
override fun onCreate(savedInstanceState: Bundle?) { override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
Log.i(TAG, "onCreate start") Log.i(TAG, "onCreate start")
// 4a/Q6=A:表单页强制竖屏(输入框 + 软键盘在竖屏好用)。
// 放在 setContentView 前:首帧就定下方向,避免横屏手机打开 App 先闪一下。
orientationPolicy = Orientation.fromName(prefs().getString(PREF_ORIENTATION, null))
requestedOrientation = Orientation.REQUEST_PORTRAIT
surfaceView = RemoteSurfaceView(this).apply { surfaceView = RemoteSurfaceView(this).apply {
listener = this@MainActivity listener = this@MainActivity
onDebug = { msg -> lastTouchDbg = msg; refreshStatus() } onDebug = { msg -> lastTouchDbg = msg; refreshStatus() }
// 4a:画面内的任何触摸重置隐藏计时;黑边唤出在 stage 的
// dispatchTouchEvent 里做(那里能区分「落在画面外」)
onAnyTouch = { scheduleControlsHide() }
holder.addCallback(object : SurfaceHolder.Callback { holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {} override fun surfaceCreated(holder: SurfaceHolder) {}
override fun surfaceChanged(h: SurfaceHolder, f: Int, w: Int, ht: Int) {} override fun surfaceChanged(h: SurfaceHolder, f: Int, w: Int, ht: Int) {}
@@ -189,7 +307,9 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
statusView = TextView(this).apply { statusView = TextView(this).apply {
textSize = 12f textSize = 12f
setPadding(dp(12), dp(6), dp(12), dp(6)) setPadding(dp(12), dp(6), dp(12), dp(6))
setBackgroundColor(0xCC000000.toInt()) // 循环 5:底色统一移到浮层容器(chromeOverlay)上,这里保持透明——
// 两层半透明黑叠加会让浮层比设计值明显更黑。
setBackgroundColor(android.graphics.Color.TRANSPARENT)
setTextColor(0xFFFFFFFF.toInt()) setTextColor(0xFFFFFFFF.toInt())
maxLines = 3 maxLines = 3
} }
@@ -200,7 +320,32 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
// (2026-10-30 实测:视图被撑成 1080x2001,触摸按 2001 归一化 → 坐标全错)。 // (2026-10-30 实测:视图被撑成 1080x2001,触摸按 2001 归一化 → 坐标全错)。
// MATCH_PARENT + gravity=center 时父容器传 AT_MOST,onMeasure 的 // MATCH_PARENT + gravity=center 时父容器传 AT_MOST,onMeasure 的
// setMeasuredDimension 才真正决定尺寸,再被 gravity 居中摆放。 // setMeasuredDimension 才真正决定尺寸,再被 gravity 居中摆放。
val stage = FrameLayout(this).apply { //
// 4a:控制条隐藏时,落在**画面黑边**(SurfaceView 之外)的触摸唤出控制条。
// 为什么用黑边而不是底部边缘带(后者已废弃):letterbox 黑边本来就不承载
// 任何远端操作,拿它当唤出区是零冲突的;底部边缘带会吃掉远端桌面最下
// ~11% 的高度,而 Windows 任务栏就在那里(2026-11-01 R7 回归实测:
// 任务栏位置的 tap 被唤出带吞掉,映射误差 357px)。
// 落在画面内的触摸一律放行给 SurfaceView。
val stage = object : FrameLayout(this) {
override fun dispatchTouchEvent(e: android.view.MotionEvent): Boolean {
if (e.actionMasked == android.view.MotionEvent.ACTION_DOWN &&
!controlsVisible && remotePanel.visibility == View.VISIBLE
) {
val v = surfaceView
val inside = e.x >= v.x && e.x <= v.x + v.width &&
e.y >= v.y && e.y <= v.y + v.height
if (!inside) {
setControlsVisible(true)
scheduleControlsHide()
lastTouchDbg = "唤出菜单"
refreshStatus()
return true // 消费:黑边不产生远端事件
}
}
return super.dispatchTouchEvent(e)
}
}.apply {
setBackgroundColor(0xFF000000.toInt()) setBackgroundColor(0xFF000000.toInt())
addView( addView(
surfaceView, surfaceView,
@@ -210,6 +355,20 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
Gravity.CENTER, Gravity.CENTER,
), ),
) )
// 4b:光标标记层叠在 SurfaceView 之上(后 addView = 更高层)。
// 不能放进 SurfaceView 自己画:surface 合成在窗口之下,同视图无法自叠加。
// MATCH_PARENT + 透明背景 + 不消费触摸(见 CursorOverlayView):事件仍按
// FrameLayout 规则下沉到 SurfaceView,4a 的触摸与黑边唤出行为零变化。
cursorOverlay = CursorOverlayView(this@MainActivity).apply {
anchor = surfaceView
}
addView(
cursorOverlay,
FrameLayout.LayoutParams(
FrameLayout.LayoutParams.MATCH_PARENT,
FrameLayout.LayoutParams.MATCH_PARENT,
),
)
} }
val keyboardButton = Button(this).apply { val keyboardButton = Button(this).apply {
text = "键盘" text = "键盘"
@@ -247,27 +406,58 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
// 取消语义必须只有一条路径(V-A1 判据依赖取消后的固定状态)。 // 取消语义必须只有一条路径(V-A1 判据依赖取消后的固定状态)。
setOnClickListener { if (connecting) cancelConnect() else disconnect() } setOnClickListener { if (connecting) cancelConnect() else disconnect() }
} }
val bar = LinearLayout(this).apply { // 4a/Q4=A:方向循环按钮 AUTO→横→竖→系统
orientationButton = Button(this).apply {
text = Orientation.label(orientationPolicy)
setOnClickListener {
orientationPolicy = Orientation.next(orientationPolicy)
prefs().edit().putString(PREF_ORIENTATION, orientationPolicy.name).apply()
text = Orientation.label(orientationPolicy)
applyOrientation()
lastTouchDbg = Orientation.label(orientationPolicy)
refreshStatus()
}
}
controlBar = LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL orientation = LinearLayout.HORIZONTAL
addView(keyboardButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f)) addView(keyboardButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
addView(modeButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f)) addView(modeButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
addView(orientationButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
addView(disconnectButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f)) addView(disconnectButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
} }
val sensBar = LinearLayout(this).apply { sensBar = LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL orientation = LinearLayout.HORIZONTAL
addView(sensDownButton, LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT)) addView(sensDownButton, LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT))
addView(sensLabel, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f)) addView(sensLabel, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
addView(sensUpButton, LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT)) addView(sensUpButton, LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT))
} }
// 循环 5/Q2=A:三个控件收进浮层容器,作为 stage 的**底部叠加层**。
// 为什么必须这样:它们是 stage 的垂直兄弟时,显示即抢高度 → surfaceView
// 的 letterbox 几何跟着变(横屏实测可见态画面 1216x684、隐藏态 1920x1080,
// 即"一按按钮画面就被挤小")。改成 stage 的子视图后,画面尺寸与控件显隐无关。
//
// 半透明底放在**容器**上而不是各条上:一次绘制、一层叠色,且 GONE 时整层
// 一起消失(三个子视图仍由 setControlsVisible 统一 GONE,容器纵向塌成 0)。
chromeOverlay = LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
setBackgroundColor(CHROME_OVERLAY_BG)
addView(statusView, LayoutParams(LayoutParams.MATCH_PARENT, dp(64)))
addView(sensBar, LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT))
addView(controlBar, LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT))
}
stage.addView(
chromeOverlay,
FrameLayout.LayoutParams(
FrameLayout.LayoutParams.MATCH_PARENT,
FrameLayout.LayoutParams.WRAP_CONTENT,
Gravity.BOTTOM,
),
)
return LinearLayout(this).apply { return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL orientation = LinearLayout.VERTICAL
visibility = View.GONE visibility = View.GONE
// 画面撑满整个远控面板:控件的显隐不再参与垂直分配。
addView(stage, LayoutParams(LayoutParams.MATCH_PARENT, 0, 1f)) addView(stage, LayoutParams(LayoutParams.MATCH_PARENT, 0, 1f))
// 固定高度而非 WRAP_CONTENT:与 weight=1 的兄弟共存时,
// WRAP_CONTENT 可能被压成 0(实测 bounds=[0,0][0,0],状态栏不可见)
addView(statusView, LayoutParams(LayoutParams.MATCH_PARENT, dp(64)))
addView(sensBar, LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT))
addView(bar, LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT))
} }
} }
@@ -290,6 +480,13 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
connecting = false connecting = false
remotePanelVisible(false) remotePanelVisible(false)
connectStatus.text = msg connectStatus.text = msg
// 4a/Q6=A:回表单就恢复竖屏(AUTO 策略下 remoteW/H 已被清零,
// 但显式设 PORTRAIT 更稳:即使策略是 LANDSCAPE,表单也不该横着)。
requestedOrientation = Orientation.REQUEST_PORTRAIT
// 4a:表单页永远完整可见——收掉沉浸式、恢复控制条、撤掉隐藏计时
ui.removeCallbacks(hideControlsTask)
applyImmersive(false)
setControlsVisible(true)
} }
private fun showRemoteState() { private fun showRemoteState() {
@@ -299,6 +496,8 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
// 远控面板上的按钮第一下 tap 只被用来「丢焦点」而不触发点击——真机上表现 // 远控面板上的按钮第一下 tap 只被用来「丢焦点」而不触发点击——真机上表现
// 为「按钮要点两次」(3a 的 D8 已实测到 4 轮全丢 tap)。 // 为「按钮要点两次」(3a 的 D8 已实测到 4 轮全丢 tap)。
clearFormFocus() clearFormFocus()
// 4a:按当前方向应用形态(此刻可能还没转完,onConfigurationChanged 会再来一次)
applyRemoteChrome()
} }
/** /**
@@ -442,6 +641,20 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
override fun onConfigurationChanged(newConfig: android.content.res.Configuration) { override fun onConfigurationChanged(newConfig: android.content.res.Configuration) {
super.onConfigurationChanged(newConfig) super.onConfigurationChanged(newConfig)
updateKeyboardVisibility() updateKeyboardVisibility()
// 4a:旋转(含 AUTO 策略触发的)会走到这里——探针实测 Activity 不重建、
// surface 只 resize,所以这里只做形态切换,不碰会话。
if (remotePanel.visibility == View.VISIBLE) {
applyRemoteChrome()
}
}
/**
* 4a:任何用户交互都重置隐藏计时——系统栏临时唤出、面板按钮点击
* (按钮在 SurfaceView 之外,onAnyTouch 覆盖不到)。
*/
override fun onUserInteraction() {
super.onUserInteraction()
if (remotePanel.visibility == View.VISIBLE) scheduleControlsHide()
} }
/** /**
@@ -614,6 +827,9 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
if (w > 0 && ht > 0) { if (w > 0 && ht > 0) {
surfaceView.videoAspectRatio = w.toFloat() / ht surfaceView.videoAspectRatio = w.toFloat() / ht
} }
// 4a:拿到远端尺寸才谈得上 AUTO(Q1=A)。尺寸未知时 requestFor
// 落回竖屏,不会把表单尺寸错当远端尺寸用。
applyOrientation()
refreshStatus() refreshStatus()
startDecodeLoop(h, gen) startDecodeLoop(h, gen)
} }
@@ -686,6 +902,10 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
if (!stopDecoding) lostReason = "被控端不可达或会话已结束" if (!stopDecoding) lostReason = "被控端不可达或会话已结束"
break break
} }
// 4b:每帧把帧头光标交给标记层(含 null——隐藏/离屏时清除标记)。
// 放在解码入队之前:即使该帧解码排队失败(fed=false),画面内容仍是旧的,
// 但光标是帧头字段、与解码成败无关,标记跟帧走比跟画面解码走更正确。
cursorOverlay.updateCursor(f.cursor)
val fed = dec.queue(f) val fed = dec.queue(f)
if (!fed && !f.keyframe && System.currentTimeMillis() - lastKeyReq > 200) { if (!fed && !f.keyframe && System.currentTimeMillis() - lastKeyReq > 200) {
RustBridge.nativeRequestKeyframe(h) // 等 IDR 中:限频请求 RustBridge.nativeRequestKeyframe(h) // 等 IDR 中:限频请求
@@ -737,6 +957,9 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
// 断开后相对换算失去基准:不清零会让下一次会话在拿到新尺寸前按旧尺寸换算 // 断开后相对换算失去基准:不清零会让下一次会话在拿到新尺寸前按旧尺寸换算
surfaceView.remoteWidth = 0 surfaceView.remoteWidth = 0
surfaceView.remoteHeight = 0 surfaceView.remoteHeight = 0
// 4b:标记必须随会话清除——旧会话的光标留在原地会冒充新会话的状态
// (尤其「连 A 断开再连 B」,B 可能分辨率不同,旧坐标映射出的位置完全无意义)
cursorOverlay.updateCursor(null)
lastFrames = 0 lastFrames = 0
lastRtt = 0 lastRtt = 0
lastFps = 0 lastFps = 0
@@ -763,6 +986,8 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
remoteH = 0 remoteH = 0
surfaceView.remoteWidth = 0 surfaceView.remoteWidth = 0
surfaceView.remoteHeight = 0 surfaceView.remoteHeight = 0
// 4b 同源检查:本函数与 cleanupSession 是两条并行收尾路径,标记清除必须同改
cursorOverlay.updateCursor(null)
lastFrames = 0 lastFrames = 0
lastRtt = 0 lastRtt = 0
lastFps = 0 lastFps = 0
@@ -775,6 +1000,9 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
override fun onDestroy() { override fun onDestroy() {
super.onDestroy() super.onDestroy()
// 4a:撤掉隐藏计时,防止 Handler 持有已销毁 Activity(RemoteSurfaceView
// 的长按任务有自己的 cancel,这里只管本 Activity 的)
ui.removeCallbacks(hideControlsTask)
disconnect() disconnect()
} }
@@ -808,13 +1036,26 @@ class MainActivity : Activity(), RemoteSurfaceView.Listener {
private const val PREF_RELATIVE = "relative_mode" private const val PREF_RELATIVE = "relative_mode"
private const val PREF_SENSITIVITY = "sensitivity" private const val PREF_SENSITIVITY = "sensitivity"
private const val SENS_STEP = 0.25f private const val SENS_STEP = 0.25f
/** 4a/Q4=A:方向策略落盘(存枚举名,未知值 fromName 落回 AUTO) */
private const val PREF_ORIENTATION = "orientation_policy"
/** 4a/Q7=A:横屏无操作隐藏控制条的延时 */
private const val CONTROLS_HIDE_MS = 3000L
/**
* 循环 5/Q2=A:控件浮层底色——60% 黑。半透明是为了让被覆盖的画面仍然可辨
* (横屏画面占满 stage 高度,浮层必然盖住底部;透明度是可读性与不遮挡的折中)。
* 只用在这一处(容器上),各控件自身保持默认/透明底。
*/
private const val CHROME_OVERLAY_BG = 0x99000000.toInt()
/** /**
* 与协议版本保持一致:远控画面状态栏显示当前握手版本(方案 R5 要求)。 * 与协议版本保持一致:远控画面状态栏显示当前握手版本(方案 R5 要求)。
* 值随 `common::PROTO_VERSION` 人工同步——Android 侧不做线格式解析, * 值随 `common::PROTO_VERSION` 人工同步——Android 侧不做线格式解析,
* 版本由 Rust 握手校验,这里只作显示与排障用。 * 版本由 Rust 握手校验,这里只作显示与排障用。
* 3b:3 → 4(协议 v4 相对鼠标)。 * 3b:3 → 4(协议 v4 相对鼠标);4b:4 → 5(帧头光标字段)。
* **同步义务**:升 PROTO_VERSION 必须同改这里 + rel_mouse_phone_proof 的
* HUD `v<N>` 断言(4b R-5 同源排查发现的耦合点)。
*/ */
private const val PROTO_LABEL = "4" private const val PROTO_LABEL = "5"
} }
} }
@@ -0,0 +1,60 @@
package com.verify.remote
/** 4a:指针方向策略(Q1=A:默认 AUTO)。 */
enum class OrientationPolicy { AUTO, LANDSCAPE, PORTRAIT, SYSTEM }
/**
* 方向策略 → `Activity.requestedOrientation` 的纯映射(4a)。
*
* 抽成纯 Kotlin 的原因(与 WinScancode/ConnectErrorText 同一模式):
* 映射表必须能在 JVM 单测里钉死,而 `ActivityInfo` 常量在 android.jar 桩上
* 引用类加载有坑,这里直接落整数常量。
*
* 数值来源(不凭记忆,2026-11-01 `javap -constants` 对 android-34 android.jar 核实):
* - SCREEN_ORIENTATION_LANDSCAPE = 0
* - SCREEN_ORIENTATION_PORTRAIT = 1
* - SCREEN_ORIENTATION_UNSPECIFIED = -1
* 刻意只用这三个:探针在真机验证过 LANDSCAPE(0) 与 PORTRAIT 行为;
* SENSOR_LANDSCAPE(6)(允许左右横)未在本机验证过,不引入——固定横屏对
* 「看 16:9 桌面」的场景足够,翻转手机方向时系统不会再把画面翻到另一侧,
* 这反而是更稳定的行为。
*/
object Orientation {
const val REQUEST_LANDSCAPE = 0
const val REQUEST_PORTRAIT = 1
const val REQUEST_UNSPECIFIED = -1
/**
* AUTO 的语义:远端画面是横的(宽>高)就用横屏,否则竖屏。
* 尺寸未知(0,连接前/断开后)按竖屏兜底——表单页本来就是竖屏。
*/
fun requestFor(policy: OrientationPolicy, remoteW: Int, remoteH: Int): Int = when (policy) {
OrientationPolicy.AUTO ->
if (remoteW > remoteH) REQUEST_LANDSCAPE else REQUEST_PORTRAIT
OrientationPolicy.LANDSCAPE -> REQUEST_LANDSCAPE
OrientationPolicy.PORTRAIT -> REQUEST_PORTRAIT
OrientationPolicy.SYSTEM -> REQUEST_UNSPECIFIED
}
/** 方向按钮的循环次序(Q4=A):AUTO→横→竖→系统→AUTO。 */
fun next(policy: OrientationPolicy): OrientationPolicy = when (policy) {
OrientationPolicy.AUTO -> OrientationPolicy.LANDSCAPE
OrientationPolicy.LANDSCAPE -> OrientationPolicy.PORTRAIT
OrientationPolicy.PORTRAIT -> OrientationPolicy.SYSTEM
OrientationPolicy.SYSTEM -> OrientationPolicy.AUTO
}
/** 按钮文案。每个策略都必须有非空、互不相同的文案(单测锁)。 */
fun label(policy: OrientationPolicy): String = when (policy) {
OrientationPolicy.AUTO -> "方向:自动"
OrientationPolicy.LANDSCAPE -> "方向:横屏"
OrientationPolicy.PORTRAIT -> "方向:竖屏"
OrientationPolicy.SYSTEM -> "方向:系统"
}
/** prefs 读写:未知字符串(版本回退/手改)必须落回 AUTO,不得抛。 */
fun fromName(name: String?): OrientationPolicy =
runCatching { OrientationPolicy.valueOf(name.orEmpty().uppercase()) }
.getOrDefault(OrientationPolicy.AUTO)
}
@@ -246,6 +246,22 @@ class RemoteSurfaceView(context: Context) : SurfaceView(context) {
*/ */
var onDebug: ((String) -> Unit)? = null var onDebug: ((String) -> Unit)? = null
/**
* 4a/Q3=A:任何触摸都回调一次(Activity 用它重置自动隐藏计时)。
*
* 唤出区刻意**不在本视图内**(原底部 40dp 边缘带的设计已废弃):那条带会吃掉
* 远端桌面底部约 11% 的高度——Windows 任务栏就在那里,点任务栏会变成唤出菜单
* 而不是点击任务栏(2026-11-01 R7 回归实测暴露:frac=0.9 的 tap 被边缘带吞掉)。
* 改为由 Activity 在 stage 上消费「画面黑边」的触摸:黑边不属于本视图,
* 天然与远端操作零冲突。
*/
var onAnyTouch: (() -> Unit)? = null
override fun dispatchTouchEvent(event: MotionEvent): Boolean {
onAnyTouch?.invoke()
return super.dispatchTouchEvent(event)
}
override fun onTouchEvent(event: MotionEvent): Boolean { override fun onTouchEvent(event: MotionEvent): Boolean {
val l = listener ?: run { val l = listener ?: run {
onDebug?.invoke("触摸(未接线)") onDebug?.invoke("触摸(未接线)")
@@ -16,8 +16,6 @@ class ConnectFailure(val kind: Int, val detail: String) : Exception(detail)
*/ */
object RustBridge { object RustBridge {
private const val FRAME_HEADER_LEN = 25
init { init {
System.loadLibrary("androidffi") System.loadLibrary("androidffi")
} }
@@ -31,7 +29,7 @@ object RustBridge {
* 消息读走时槽位整体清空,颠倒顺序拿到 kind=0。 * 消息读走时槽位整体清空,颠倒顺序拿到 kind=0。
*/ */
external fun nativeLastErrorKind(): Int external fun nativeLastErrorKind(): Int
/** 返回 null = 会话结束;否则 [FRAME_HEADER_LEN] 字节头 + Annex-B 载荷 */ /** 返回 null = 会话结束;否则 [FrameCodec.FRAME_HEADER_LEN] 字节头 + Annex-B 载荷 */
external fun nativeNextFrame(handle: Long): ByteArray? external fun nativeNextFrame(handle: Long): ByteArray?
external fun nativeStats(handle: Long): Long external fun nativeStats(handle: Long): Long
external fun nativeDesktopSize(handle: Long): Long external fun nativeDesktopSize(handle: Long): Long
@@ -78,23 +76,21 @@ object RustBridge {
val tsMs: Long, val tsMs: Long,
val width: Int, val width: Int,
val height: Int, val height: Int,
/**
* v5:本帧发送时刻的远端光标(采集画面内物理像素)。
* `null` = 隐藏/离屏/后端未实现(语义单一来源在被控端,见 common 协议注释)。
* 用一个可空 Pair 而非两个可空 Int:忠实对应 `Option<(i16,i16)>`,
* 杜绝「x 有 y 无」这种协议里不存在的状态。
*/
val cursor: Pair<Int, Int>?,
val payload: ByteArray, val payload: ByteArray,
) )
/** 取帧 + 拆头。null = 会话结束。 */ /** 取帧 + 拆头。null = 会话结束。 */
fun nextFrame(handle: Long): Frame? { fun nextFrame(handle: Long): Frame? {
val raw = nativeNextFrame(handle) ?: return null val raw = nativeNextFrame(handle) ?: return null
if (raw.size < FRAME_HEADER_LEN) return null return FrameCodec.parse(raw)
return Frame(
keyframe = raw[0] != 0.toByte(),
seq = readLong(raw, 1),
tsMs = readLong(raw, 9),
width = readInt(raw, 17),
height = readInt(raw, 21),
payload = raw.copyOfRange(FRAME_HEADER_LEN, raw.size),
)
} }
fun rttMs(handle: Long): Long { fun rttMs(handle: Long): Long {
val packed = nativeStats(handle) val packed = nativeStats(handle)
return if (packed < 0) 0 else (packed ushr 32) return if (packed < 0) 0 else (packed ushr 32)
@@ -116,6 +112,41 @@ object RustBridge {
val h = (packed and 0xFFFF_FFFF).toInt() val h = (packed and 0xFFFF_FFFF).toInt()
return w to h return w to h
} }
}
/**
* 帧头拆包(v5)。从 [RustBridge] 里独立出来的原因:**这个文件里的字节偏移必须能在
* JVM 单测里被验证**,而 `RustBridge` object 带 `init { System.loadLibrary }`,
* 测试里一碰它就抛 `UnsatisfiedLinkError`(本机无 .so)。顶层 object 无 native、无 init,
* 构造嵌套类 [RustBridge.Frame] 也不触发外层 object 初始化——拆包逻辑因此可测。
*
* 跨端契约:android-ffi/src/lib.rs 按 buf[0]/[1..9]/[9..17]/[17..21]/[21..25]/[25]/
* [26..28]/[28..30] 写字节,这里按**同样的偏移**读。任一侧挪一位,两端都编译得过、
* 真机也连得上——只是标记画错位置或永不显示(静默失败)。golden bytes 对拍锁死它,
* 见 RustBridgeTest。
*/
internal object FrameCodec {
// v5:[1 kf][8 seq][8 ts][4 w][4 h][1 has_cursor][2 x i16][2 y i16] = 30
// (与 android-ffi/src/lib.rs 的 FRAME_HEADER_LEN 同步;新增字段一律追加尾部,
// 旧偏移 0/1/9/17/21 不移动——R-5 同源检查过:两处必须同改)
const val FRAME_HEADER_LEN = 30
/** 拆一帧。短于定长头 → null(截断/旧布局防御)。 */
fun parse(raw: ByteArray): RustBridge.Frame? {
if (raw.size < FRAME_HEADER_LEN) return null
val hasCursor = raw[25] != 0.toByte()
return RustBridge.Frame(
keyframe = raw[0] != 0.toByte(),
seq = readLong(raw, 1),
tsMs = readLong(raw, 9),
width = readInt(raw, 17),
height = readInt(raw, 21),
// has=0 时坐标字节恒为 0 且无意义,绝不能读成 (0,0)——那会被画到左上角
cursor = if (hasCursor) Pair(readI16(raw, 26), readI16(raw, 28)) else null,
payload = raw.copyOfRange(FRAME_HEADER_LEN, raw.size),
)
}
private fun readLong(b: ByteArray, off: Int): Long { private fun readLong(b: ByteArray, off: Int): Long {
var v = 0L var v = 0L
@@ -129,4 +160,9 @@ object RustBridge {
((b[off + 2].toInt() and 0xFF) shl 16) or ((b[off + 2].toInt() and 0xFF) shl 16) or
((b[off + 3].toInt() and 0xFF) shl 24) ((b[off + 3].toInt() and 0xFF) shl 24)
} }
/** 按 little-endian **有符号** 16 位读(协议 cursor 是 i16;读成无符号会让负坐标穿透) */
private fun readI16(b: ByteArray, off: Int): Int {
return ((b[off].toInt() and 0xFF) or ((b[off + 1].toInt() and 0xFF) shl 8)).toShort().toInt()
}
} }
@@ -0,0 +1,115 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotNull
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* 4b/V-D1:光标标记映射的纯逻辑(协议 v5)。
*
* 这些断言对应验收判据,不是「跑通就算」:
* - V-D1「标记位置=映射值」→ [center_maps_to_view_center] / [full_extent_linear_map]
* - 「(0,0) 是合法位置」→ [origin_maps_not_treated_as_none](协议用 Option 的全部理由)
* - 跨信任边界自证 → 越界与负值一律 null(Linux 后端未实现/被篡改时的防线)
*/
class CursorMarkerTest {
// ---- 核心等比映射 ----
/**
* 横屏典型几何:桌面 1920x1080,视图 1080x607(满宽 letterbox)。
* 桌面中心应映射到视图中心。这是 V-D1 的锚点用例——真机 proof 就是把
* SetCursorPos 打到桌面中心,再在截图里断言标记簇心在视图中心 ±6px。
*/
@Test
fun center_maps_to_view_center() {
val p = CursorMarker.mapToView(
desktopW = 1920, desktopH = 1080,
cursor = 960 to 540,
viewW = 1080, viewH = 607,
)
assertNotNull(p)
assertEquals(540f, p!!.x, 1f)
assertEquals(303.5f, p.y, 1f)
}
/**
* 满量程线性映射:桌面右下角 (1919,1079) → 视图右下角附近,
* 且不越过视图边界(映射后必须仍在画面内可画)。
*/
@Test
fun full_extent_linear_map() {
val p = CursorMarker.mapToView(1920, 1080, 1919 to 1079, 1080, 607)
assertNotNull(p)
assertEquals(1080f * 1919 / 1920f, p!!.x, 0.5f)
assertEquals(607f * 1079 / 1080f, p.y, 0.5f)
assertTrue("映射 x 不得越过视图右边界", p.x < 1080f)
assertTrue("映射 y 不得越过视图下边界", p.y < 607f)
}
/**
* (0,0) 必须映射成视图左上角,**而不是 null**。
* 这条锁住最危险的错误:实现把「左上角」和「没有光标」混为一谈
* (比如用 (0,0) 当缺省值再判 ==0 返回 null)。协议专门用 Option 区分二者,
* 这层区分在 Kotlin 侧不能被抹平。
*/
@Test
fun origin_maps_not_treated_as_none() {
val p = CursorMarker.mapToView(1920, 1080, 0 to 0, 1080, 607)
assertNotNull("左上角是合法光标位置,不得当成无光标丢弃")
assertEquals(0f, p!!.x, 0.001f)
assertEquals(0f, p.y, 0.001f)
}
/** 1:1 场景(视图尺寸恰等于桌面):坐标原样透传。 */
@Test
fun identity_when_sizes_match() {
val p = CursorMarker.mapToView(1280, 720, 640 to 360, 1280, 720)
assertEquals(640f, p!!.x, 0.001f)
assertEquals(360f, p.y, 0.001f)
}
// ---- null 分支:一律「不画」而非画到别处 ----
/** cursor=null(隐藏/离屏/后端未实现)→ null。 */
@Test
fun null_cursor_returns_null() {
assertNull(CursorMarker.mapToView(1920, 1080, null, 1080, 607))
}
/**
* 任一尺寸未就绪(0 或负)→ null,绝不按 0 除得到 NaN/Infinity。
* 覆盖四种:视图未测量(width 0,letterbox onMeasure 前)、
* 桌面尺寸还没从 AuthOk 取到、以及负值这种异常。
*/
@Test
fun not_ready_or_degenerate_dimensions_return_null() {
assertNull(CursorMarker.mapToView(0, 1080, 10 to 10, 1080, 607)) // 桌面宽未就绪
assertNull(CursorMarker.mapToView(1920, 0, 10 to 10, 1080, 607)) // 桌面高未就绪
assertNull(CursorMarker.mapToView(1920, 1080, 10 to 10, 0, 607)) // 视图未测量
assertNull(CursorMarker.mapToView(1920, 1080, 10 to 10, 1080, 0)) // 视图高 0
assertNull(CursorMarker.mapToView(-5, 1080, 10 to 10, 1080, 607)) // 异常负尺寸
}
/**
* 越界坐标 → null(不画)。这是跨信任边界的自证:被控端 `fold_cursor` 本应折叠掉
* 越界值,但协议把对端字节按不可信输入处理——若折叠漏网或被篡改,这里兜住,
* 否则标记会画到画面外(甚至映射出一个「看似合理」的错位置)。
*/
@Test
fun out_of_desktop_bounds_returns_null() {
assertNull(CursorMarker.mapToView(1920, 1080, 1920 to 500, 1080, 607)) // 右边界外一列
assertNull(CursorMarker.mapToView(1920, 1080, 500 to 1080, 1080, 607)) // 下边界外一行
assertNull(CursorMarker.mapToView(1920, 1080, -1 to 500, 1080, 607)) // 负 X(副屏穿透)
assertNull(CursorMarker.mapToView(1920, 1080, 500 to -1, 1080, 607)) // 负 Y
assertNull(CursorMarker.mapToView(1920, 1080, 3000 to 500, 1080, 607)) // 远超桌面
}
/** 恰好界内 (desktopW-1, desktopH-1) 必须画(与上一条配对,钉死 >= 还是 > 的边界)。 */
@Test
fun exactly_in_bounds_draws() {
assertNotNull(CursorMarker.mapToView(1920, 1080, 1919 to 1079, 1080, 607))
}
}
@@ -0,0 +1,100 @@
package com.verify.remote
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotEquals
import org.junit.Test
/**
* 4a:方向策略纯逻辑。
*
* 常量数值不是随手写的——`javap -constants` 对 android-34 android.jar 核实过
* (LANDSCAPE=0、PORTRAIT=1、UNSPECIFIED=-1)。这里把它们钉成断言:
* 若有人「顺手改成 SENSOR_LANDSCAPE=6」或调换数值,测试先红。
*/
class OrientationPolicyTest {
@Test
fun android_constants_pinned_to_verified_values() {
assertEquals(0, Orientation.REQUEST_LANDSCAPE)
assertEquals(1, Orientation.REQUEST_PORTRAIT)
assertEquals(-1, Orientation.REQUEST_UNSPECIFIED)
}
// ---- AUTO 的判定 ----
@Test
fun auto_picks_landscape_for_wider_remote() {
// 本机被控端就是 1920x1080 —— 这正是 4a 需求的来源
assertEquals(Orientation.REQUEST_LANDSCAPE, Orientation.requestFor(OrientationPolicy.AUTO, 1920, 1080))
}
@Test
fun auto_picks_portrait_for_taller_remote() {
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 1080, 1920))
}
@Test
fun auto_picks_portrait_for_square_and_unknown() {
// 方屏没有「更宽」的优势,按竖屏兜底
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 1080, 1080))
// 尺寸未知(连接前/断开后)必须竖屏,不能让表单页横掉
assertEquals(Orientation.REQUEST_PORTRAIT, Orientation.requestFor(OrientationPolicy.AUTO, 0, 0))
}
@Test
fun forced_policies_ignore_remote_geometry() {
for (wh in listOf(1920 to 1080, 1080 to 1920, 0 to 0)) {
assertEquals(Orientation.REQUEST_LANDSCAPE,
Orientation.requestFor(OrientationPolicy.LANDSCAPE, wh.first, wh.second))
assertEquals(Orientation.REQUEST_PORTRAIT,
Orientation.requestFor(OrientationPolicy.PORTRAIT, wh.first, wh.second))
assertEquals(Orientation.REQUEST_UNSPECIFIED,
Orientation.requestFor(OrientationPolicy.SYSTEM, wh.first, wh.second))
}
}
// ---- 按钮循环(Q4=A)----
@Test
fun button_cycles_all_four_in_order() {
assertEquals(OrientationPolicy.LANDSCAPE, Orientation.next(OrientationPolicy.AUTO))
assertEquals(OrientationPolicy.PORTRAIT, Orientation.next(OrientationPolicy.LANDSCAPE))
assertEquals(OrientationPolicy.SYSTEM, Orientation.next(OrientationPolicy.PORTRAIT))
assertEquals(OrientationPolicy.AUTO, Orientation.next(OrientationPolicy.SYSTEM))
}
@Test
fun four_clicks_return_to_start() {
var p = OrientationPolicy.AUTO
repeat(4) { p = Orientation.next(p) }
assertEquals(OrientationPolicy.AUTO, p)
}
@Test
fun every_policy_has_distinct_nonempty_label() {
val labels = OrientationPolicy.values().map { Orientation.label(it) }
labels.forEach { assertNotEquals("", it) }
assertEquals("四个策略文案必须互不相同", labels.size, labels.toSet().size)
// 文案要让用户看出当前状态(含「方向:」前缀),不能只写「自动/横/竖」
labels.forEach { assertEquals(true, it.startsWith("方向:")) }
}
// ---- prefs 反序列化 ----
@Test
fun from_name_roundtrips_all() {
OrientationPolicy.values().forEach {
assertEquals(it, Orientation.fromName(it.name))
}
}
@Test
fun from_name_falls_back_to_auto_for_garbage() {
// 未知值不得抛异常(会让 Activity 起不来),必须落回默认 AUTO
assertEquals(OrientationPolicy.AUTO, Orientation.fromName(null))
assertEquals(OrientationPolicy.AUTO, Orientation.fromName(""))
assertEquals(OrientationPolicy.AUTO, Orientation.fromName("DIAGONAL"))
// 大小写宽容:早期版本可能存过小写
assertEquals(OrientationPolicy.LANDSCAPE, Orientation.fromName("landscape"))
}
}
@@ -0,0 +1,106 @@
package com.verify.remote
import org.junit.Assert.assertArrayEquals
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Test
/**
* 4b:[FrameCodec] 帧头拆包的**跨端字节契约**测试(golden bytes)。
*
* 为什么值这个钱:android-ffi/src/lib.rs 按 buf[0]/[1..9]/[9..17]/[17..21]/[21..25]/
* [25]/[26..28]/[28..30] 写字节,Kotlin 按同样的偏移读。**任何一侧挪一位**(尤其 v5
* 从 25→30 这次),Rust 编译过、Kotlin 也编译过、真机能连上——只是标记画到错误位置
* 或永远不显示,静默失败(全局规则:核过 Rust 一端不等于核过 Kotlin 另一端)。
*
* 期望值用 goldenHeader 按 Rust 侧写入布局**手算**生成(`to_le_bytes()` 的逐字节结果)。
* Rust 侧若改偏移或改字节序,这里的对拍就红。这是唯一能在 JVM 里锁住 JNI 二进制布局的
* 手段(真机跑不了单测)。拆包逻辑之所以住在顶层 [FrameCodec] 而不是 RustBridge object,
* 正因为后者带 loadLibrary 的 init,测试里不可达。
*/
class RustBridgeTest {
/**
* 「IDR、seq=0x0102030405060708、ts=0x0011223344556677、桌面 1920x1080、
* 光标 has=x/y」的 30 字节头,逐字节 = android-ffi 的写入布局。
*/
private fun goldenHeader(hasCursor: Int, x: Int, y: Int): ByteArray {
val bytes = listOf(
0x01, // [0] keyframe = true
// [1..9] seq = 0x0102030405060708 LE
0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01,
// [9..17] ts = 0x0011223344556677 LE
0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00,
// [17..21] width = 1920 (0x0780) LE
0x80, 0x07, 0x00, 0x00,
// [21..25] height = 1080 (0x0438) LE
0x38, 0x04, 0x00, 0x00,
// [25] has_cursor
hasCursor,
// [26..28] cursor.x i16 LE
x and 0xFF, (x shr 8) and 0xFF,
// [28..30] cursor.y i16 LE
y and 0xFF, (y shr 8) and 0xFF,
)
return bytes.map { it.toByte() }.toByteArray()
}
/** 光标 Some((960,540)):逐字段核对,锁死 [25]/[26..28]/[28..30] 三个新偏移。 */
@Test
fun parses_v5_header_with_cursor() {
val payload = byteArrayOf(0xAA.toInt().toByte(), 0xBB.toInt().toByte(), 0xCC.toInt().toByte())
val raw = goldenHeader(hasCursor = 1, x = 960, y = 540) + payload
val f = FrameCodec.parse(raw)!!
assertEquals(true, f.keyframe)
assertEquals(0x0102030405060708L, f.seq)
assertEquals(0x0011223344556677L, f.tsMs)
assertEquals(1920, f.width)
assertEquals(1080, f.height)
assertEquals(960 to 540, f.cursor)
assertArrayEquals(payload, f.payload)
}
/**
* has_cursor=0 时**即便坐标字节是全 1 位模式(0xFFFF = -1)也必须读成 null**。
* 这条锁住 `if (hasCursor)` 的判断:若误判成「非零即有」,(-1,-1) 会穿透成一个
* 看似合法的错位置——正是协议用独立 has 字节而非「0 表示无」的原因。
*/
@Test
fun has_cursor_zero_means_none_regardless_of_coord_bytes() {
// 故意把 x/y 字节填成 -1 的位模式(0xFFFF)
val raw = goldenHeader(hasCursor = 0, x = 0xFFFF, y = 0xFFFF) + byteArrayOf(0x00)
val f = FrameCodec.parse(raw)!!
assertNull("has=0 时坐标字节无意义,必须 null", f.cursor)
}
/** 负光标坐标(有符号 i16)必须正确读回——agent 折叠后不该出现,但拆包层要忠实。 */
@Test
fun reads_signed_cursor_coords() {
val raw = goldenHeader(hasCursor = 1, x = (-1) and 0xFFFF, y = (-32768) and 0xFFFF) + byteArrayOf(0)
val f = FrameCodec.parse(raw)!!
assertEquals(-1 to -32768, f.cursor)
}
/** 光标坐标取 i16 两端边界 (32767, -32768) 往返,证明 readI16 符号扩展正确。 */
@Test
fun cursor_i16_extremes() {
val raw = goldenHeader(hasCursor = 1, x = 32767, y = -32768) + byteArrayOf(0)
val f = FrameCodec.parse(raw)!!
assertEquals(32767 to -32768, f.cursor)
}
/** 短于 30 字节的缓冲(旧 25 字节布局或截断帧)必须返回 null,不能切片越界崩溃。 */
@Test
fun rejects_truncated_buffer() {
assertNull(FrameCodec.parse(ByteArray(29)))
}
/** 恰好 30 字节(空载荷)是合法的最小帧;载荷长度为 0 而非 null。 */
@Test
fun accepts_exact_header_with_empty_payload() {
val raw = goldenHeader(hasCursor = 1, x = 100, y = 200)
val f = FrameCodec.parse(raw)!!
assertEquals(100 to 200, f.cursor)
assertEquals(0, f.payload.size)
}
}
+67 -5
View File
@@ -22,7 +22,10 @@ use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// - v4:新增 `InputEvent::MouseMoveRel`(相对鼠标,单位=被控端桌面物理像素)。 /// - v4:新增 `InputEvent::MouseMoveRel`(相对鼠标,单位=被控端桌面物理像素)。
/// bincode 按位置编码枚举判别序号,新增变体会改变其后变体的编号, /// bincode 按位置编码枚举判别序号,新增变体会改变其后变体的编号,
/// 故同样必须三端同步;旧端由 `Auth` 握手显式拒绝。 /// 故同样必须三端同步;旧端由 `Auth` 握手显式拒绝。
pub const PROTO_VERSION: u16 = 4; /// - v5:`VideoFrameHeader` 尾部新增 `cursor`(本帧发送时刻的远端光标位置)。
/// 字段追加同样改变线格式(旧端会在反序列化时少读/多读而错位),故必须三端同步;
/// 旧端由 `Auth` 握手显式拒绝。语义见 [`VideoFrameHeader::cursor`]。
pub const PROTO_VERSION: u16 = 5;
/// 控制通道消息(双向) /// 控制通道消息(双向)
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -132,6 +135,15 @@ pub fn clamp_text_units(units: Vec<u16>) -> (Vec<u16>, bool) {
/// 用于丢帧检测与弱网统计;注意它统计的是"发送序",不是"显示序" /// 用于丢帧检测与弱网统计;注意它统计的是"发送序",不是"显示序"
/// - `keyframe`:本帧是否 IDR。解码端(尤其 MediaCodec)需要它决定是否重置解码器, /// - `keyframe`:本帧是否 IDR。解码端(尤其 MediaCodec)需要它决定是否重置解码器,
/// 不必再自己扫 NAL 起始码判断 /// 不必再自己扫 NAL 起始码判断
/// - `cursor`(v5):本帧**发送时刻**的远端光标位置,被采集显示器内的**物理像素**
/// ((0,0)=采集画面左上角,与 `width`/`height` 同尺度同原点)。
/// `None` = 不可用,含三种情况:指针被隐藏(系统未显示光标)、指针在采集显示器
/// **之外**(多显示器负坐标区等——采集画面里没有它,主控端画了也是错的)、
/// 或被控端平台未实现光标采样。
/// 用途:主控端据此在画面上标记指针位置(「看不见远端鼠标」修复);它与 `ts_ms`
/// 同刻、同帧到达,不存在标记与画面的相对错位。
/// 为什么 `i16`:采集原点固定 (0,0),现实桌面尺寸远小于 32767;被控端已把
/// 越界值折叠为 `None`,到达值恒在 `[0, width)×[0, height)` 内。
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VideoFrameHeader { pub struct VideoFrameHeader {
pub width: u32, pub width: u32,
@@ -139,6 +151,7 @@ pub struct VideoFrameHeader {
pub ts_ms: u64, pub ts_ms: u64,
pub seq: u64, pub seq: u64,
pub keyframe: bool, pub keyframe: bool,
pub cursor: Option<(i16, i16)>,
} }
async fn read_u32le<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<u32> { async fn read_u32le<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<u32> {
@@ -279,6 +292,7 @@ mod tests {
ts_ms: 12345, ts_ms: 12345,
seq, seq,
keyframe, keyframe,
cursor: Some((640, 480)),
} }
} }
@@ -347,15 +361,63 @@ mod tests {
// ---- v4 相对鼠标 ---- // ---- v4 相对鼠标 ----
/// 版本守卫:v4 是相对鼠标的登记点。 /// 版本守卫:v4 是相对鼠标的登记点。v5 之后此断言随版本推进为下界,
/// 与 v3 守卫同规(保留它是「任何一次版本变更都必须显式经过这里」的锚点)。
#[test] #[test]
fn proto_version_is_four_after_rel_move() { fn proto_version_is_four_after_rel_move() {
assert_eq!( assert!(
PROTO_VERSION, 4, PROTO_VERSION >= 4,
"新增 InputEvent::MouseMoveRel 后必须递增到 4" "MouseMoveRel 自 v4 起存在,版本不得回退(当前 {PROTO_VERSION})"
); );
} }
/// 版本守卫:v5 是光标字段的登记点(最新一代用精确等号)。
#[test]
fn proto_version_is_five_after_cursor_field() {
assert_eq!(
PROTO_VERSION, 5,
"新增 VideoFrameHeader::cursor 后必须递增到 5"
);
}
/// 光标字段必须原样往返,含 `None`(隐藏/离屏/未实现三态都折叠为它)与负值边界。
/// `None` 是本变体的核心语义:缺它主控端就无法区分「指针在 (0,0)」与「没有指针」。
#[test]
fn cursor_field_roundtrips_option_and_i16() {
for c in [
None,
Some((0i16, 0i16)),
Some((1919, 1079)),
Some((-32768, 32767)),
Some((i16::MIN, i16::MAX)),
] {
let mut h = hdr(1, false);
h.cursor = c;
let payload = b"\x00\x00\x00\x01\x65x";
let bytes = encode_video_frame(&h, payload);
let hlen = u32::from_le_bytes(bytes[4..8].try_into().unwrap()) as usize;
let back: VideoFrameHeader = bincode::deserialize(&bytes[8..8 + hlen]).unwrap();
assert_eq!(back.cursor, c, "光标字段往返失真: {c:?}");
}
}
/// cursor 是**追加在尾部**的字段:既有字段(width/height/ts/seq/keyframe)的
/// 解码不受影响,且 `None` 变体让帧头显著变短(证明它没被塞进固定宽度里白占字节)。
#[test]
fn cursor_is_trailing_field_and_none_is_compact() {
let mut with = hdr(9, true);
with.cursor = Some((10, 20));
let mut without = hdr(9, true);
without.cursor = None;
let wb = bincode::serialize(&with).unwrap();
let wob = bincode::serialize(&without).unwrap();
// Some((i16,i16)) 比 None 多 4 字节(两个 i16),1 字节 tag 两边都有
assert_eq!(wb.len(), wob.len() + 4, "光标坐标应为 2×i16 定长附加");
// 前缀不变:去掉尾部光标字段,其余字节的解码结果完全一致(追加语义)
let h: VideoFrameHeader = bincode::deserialize(&wb).unwrap();
assert_eq!((h.width, h.height, h.seq, h.keyframe), (1920, 1080, 9, true));
}
/// 相对位移必须原样往返,**包含负值**(向左/向上移动)。 /// 相对位移必须原样往返,**包含负值**(向左/向上移动)。
/// 负值是本变体的核心语义:绝对坐标是无符号的,相对增量必须有符号。 /// 负值是本变体的核心语义:绝对坐标是无符号的,相对增量必须有符号。
#[test] #[test]
+280
View File
@@ -0,0 +1,280 @@
# 4b cursor-overlay proof: marker position, off-screen fold, refresh, old-client reject.
#
# ASCII-only (PS 5.1 reads BOM-less .ps1 as GBK; repo lessons). Chinese expected
# texts are built from code points (R-8: never trust an encoding round trip).
#
# Judgements (loop-4b-plan.md section 9):
# V-D6 OLD v4 APK + new v5 agent -> form shows the version-mismatch text and
# the session never goes live. RUN THIS PHASE BEFORE INSTALLING THE NEW
# APK: -Phase reject
# V-D1 screencap cyan cluster centroid == CursorMarker expectation +/- 6 px
# for 4 desktop points incl. (48,48) ~ origin (Some(0,0) vs None: the
# marker MUST be painted at the corner, not skipped)
# V-D3 after moving the PC cursor on a quiet desktop the marker reaches the
# new spot within ~1.2s (covers the 500ms idle-resend path; the frame
# header rebuild with fresh cursor was the whole point of R-A)
# V-D7 cursor into the LEFT SECOND MONITOR (negative X, machine is dual-screen
# since 2026-11-02) -> agent folds to None -> cyan cluster GONE
#
# Prereqs: v5 release agent.exe built; new (v5) APK installed for -Phase main;
# second monitor actually present (V-D7 skips loudly if not - see probe output);
# nobody touching the real mouse during a step (gates retry, loud fail otherwise).
#
# powershell -ExecutionPolicy Bypass -File tools\cursor_proof.ps1 -Host_ 192.168.1.181 -Phase reject
# powershell -ExecutionPolicy Bypass -File tools\cursor_proof.ps1 -Host_ 192.168.1.181 -Phase main
param(
[Parameter(Mandatory)][string]$Host_,
[int]$Port = 47800,
[string]$Code = "t",
[ValidateSet("reject", "main")][string]$Phase = "main",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
. "$PSScriptRoot\proof_common.ps1"
# ---- expected Chinese fragments as code points (R-8) ----
# "connect" button label
$F_CONNECT = -join ([char]0x8FDE, [char]0x63A5)
# "reconnect"
$F_RECONNECT = -join ([char]0x91CD, [char]0x65B0, [char]0x8FDE, [char]0x63A5)
# "protocol version mismatch" error summary (ConnectErrorText KIND_VERSION)
$F_VERSION = -join ([char]0x534F, [char]0x8BAE, [char]0x7248, [char]0x672C,
[char]0x4E0D, [char]0x4E00, [char]0x81F4)
# "live" HUD prefix (yi-lian-jie = connected)
$F_LIVE = -join ([char]0x5DF2, [char]0x8FDE, [char]0x63A5)
$ev = Join-Path (Split-Path $repo -Parent) "evidence\4b"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- cyan marker pixel detector (LockBits scan, full centroid) ----
Add-Type -AssemblyName System.Windows.Forms # VirtualScreen for V-D7
if (-not ('PccCyan' -as [type])) {
Add-Type -ReferencedAssemblies System.Drawing -TypeDefinition @"
using System; using System.Drawing; using System.Drawing.Imaging;
using System.Runtime.InteropServices; using System.Collections.Generic;
public class PccCyan {
public static string Find(string path) {
using (var bmp = new Bitmap(path)) {
var rect = new Rectangle(0,0,bmp.Width,bmp.Height);
var data = bmp.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int n = data.Stride * data.Height; byte[] buf = new byte[n];
Marshal.Copy(data.Scan0, buf, 0, n);
bmp.UnlockBits(data);
// dot color = RGB(0x00,0xE5,0xFF); BGRA in memory. Loose tolerance:
// the ring is white/black; a desktop with a big cyan region would poison the
// centroid, so the total-count guard below rejects it instead of averaging.
int total = 0;
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) total++;
}
}
if (total == 0) return "none";
if (total > 4000) return "patch:" + total; // suspicious giant color field
// Centroid over ALL cyan pixels would be corrupted by a few stray ones far
// away (measured 2026-11-02: n=181 at the right spot but spread=955 - the
// desktop itself had a handful of cyan pixels elsewhere). Cluster properly:
// bin the hits on an 8px grid, take the DENSEST bin as the marker seed,
// then centroid/spread over hits within 40px of that seed only.
var bins = new Dictionary<long,int>();
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) {
long key = (long)(x/8)*1000000 + (y/8);
bins[key] = bins.ContainsKey(key) ? bins[key]+1 : 1;
}
}
}
long bestK = -1; int bestN = 0; int secondN = 0;
foreach (var kv in bins) {
if (kv.Value > bestN) { secondN = bestN; bestN = kv.Value; bestK = kv.Key; }
else if (kv.Value > secondN) { secondN = kv.Value; }
}
int bx = (int)(bestK/1000000)*8 + 4, by = (int)(bestK%1000000)*8 + 4;
long sx=0, sy=0; int cnt=0; int minx=99999999,miny=99999999,maxx=-1,maxy=-1;
for (int y=0; y<data.Height; y++) {
int row = y*data.Stride;
for (int x=0; x<data.Width; x++) {
int i = row + x*4;
byte b=buf[i], g=buf[i+1], r=buf[i+2];
if (r < 40 && (g>=0xC0 && g<=0xFF) && b>=0xD0) {
if (Math.Abs(x-bx)<=40 && Math.Abs(y-by)<=40) {
sx+=x; sy+=y; cnt++;
if(x<minx)minx=x; if(x>maxx)maxx=x; if(y<miny)miny=y; if(y>maxy)maxy=y;
}
}
}
}
if (cnt == 0) return "none";
int spread = Math.Max(maxx-minx, maxy-miny);
// report seed too: harness prints it when a check fails (diagnosis, not judgement)
return string.Format("{0}|{1}|{2}|{3}|{4}|{5}|{6}", sx/(double)cnt, sy/(double)cnt, cnt, spread, total, bx, secondN);
}
}
}
"@
}
function Find-Marker([string]$pngPath) {
$r = [PccCyan]::Find($pngPath)
if ($r -eq "none") { return $null }
if ($r -like "patch:*") { throw "screenshot contains a huge cyan-ish color field ($($r.Substring(6)) px) - clear the desktop (wallpaper/browser) and re-run; refusing to average a fake marker (R-8)" }
$p = $r.Split('|')
@{ x = [double]$p[0]; y = [double]$p[1]; n = [int]$p[2]; spread = [int]$p[3]; total = [int]$p[4]; seed = "$($p[5]),$($p[6])" }
}
function Take-Shot([string]$tag) {
$dev = "/sdcard/pcc_$PID.png"
$local = Join-Path $ev "$tag.png"
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try {
[void](AdbShell @("screencap", "-p", $dev))
$r = Adb -AdbArgs @("pull", $dev, $local)
} finally { $ErrorActionPreference = $old }
if ($r.code -ne 0 -or -not (Test-Path $local)) { throw "screencap/pull failed for $tag" }
return $local
}
Write-Host "=== 0. preflight (memory / fresh v5 agent) ==="
Memory-Precheck
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
Ensure-FreshAgent $AgentExe $Port $Code
$installed = ((AdbShell @("dumpsys", "package", "com.verify.remote"))) -join "`n"
$vn = [regex]::Match($installed, 'versionName=(\S+)').Groups[1].Value
Write-Host " installed APK: $vn"
# ======================================================================
if ($Phase -eq "reject") {
Write-Host "=== V-D6: OLD (v4) APK against the NEW v5 agent must be rejected ==="
if ($vn -like "*phase4a*") {
Write-Host " (running against the v4 client - this phase MUST come before installing 4b)"
} else {
Write-Host " WARNING: installed APK is $vn - the v4-rejection test is only meaningful against a real v4 client"
}
# NOTE (first-run lesson 2026-11-02): launching via `am start --es host ...`
# does NOT dial: the intent only prefill-prefers stale SharedPreferences and
# the form just sits there ("未连接"). Rejection must be observed through the
# REAL dial path - type the form, tap connect, read the failure screen.
$x = Restart-AppForm
FillAndConnect $x $Host_ "$Port" $Code $F_CONNECT $F_RECONNECT
$deadline = (Get-Date).AddSeconds(25); $shown = $null
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 800
$ui = Get-UiRetry
$texts = (VisibleTexts $ui) -join " / "
if ($texts -match "version mismatch" -or $texts.Contains($F_VERSION)) { $shown = $texts; break }
if ($texts.Contains($F_LIVE)) { throw "V-D6 FAIL: v4 client CONNECTED to the v5 agent ('$F_LIVE' on screen) - version gate broken" }
}
if (-not $shown) { throw "V-D6 FAIL: no version-mismatch text within 25s; last screen: [$((VisibleTexts (Get-Ui)) -join ' / ')]" }
Write-Host " screen: $shown"
Write-Host " PASS: v5 agent refused the v4 client with the actionable Chinese hint"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-d6-reject.png" | Out-Null
# pollution discipline (4a): leave no live session for the next proof
[void](AdbShell @("am", "force-stop", "com.verify.remote"))
Write-Host "=== CURSOR PROOF (reject phase) COMPLETE ==="
exit 0
}
# ======================================================================
# ---- main phase: needs the NEW apk; go live once, then geometry checks ----
Write-Host "=== 1. go live (new v5 APK), capture HUD (v5 pin + desktop size) ==="
if ($vn -notlike "*phase4b*") { throw "main phase needs the 4b APK installed (found $vn)" }
$x = Restart-AppForm
FillAndConnect $x $Host_ "$Port" $Code $F_CONNECT $F_RECONNECT
$hud = Wait-LiveHud 20
if (-not $hud) { throw "never reached live: [$((VisibleTexts (Get-Ui)) -join ' / ')]" }
Write-Host " HUD: $($hud -replace "`n", ' | ')"
if (-not $hud.Contains("v5")) { throw "V-D5x FAIL: HUD does not report v5 (PROTO_LABEL out of sync?): [$hud]" }
$m = [regex]::Match($hud, '(\d+)x(\d+)')
if (-not $m.Success) { throw "cannot parse desktop size from HUD: [$hud]" }
$dW = [int]$m.Groups[1].Value; $dH = [int]$m.Groups[2].Value
Write-Host " desktop from HUD: ${dW}x${dH}"
# Marker geometry works in EITHER orientation (letterbox mapping is proportional),
# but landscape is the 4a headline case. AUTO rotates for a wide desktop - give the
# rotation a moment to land, then proceed on whatever orientation is actually up.
$rot = (Get-Date).AddSeconds(3); $d = LocalDisplay
while ((Get-Date) -lt $rot) {
$d = LocalDisplay
if ($d.w -gt $d.h) { break }
Start-Sleep -Milliseconds 250
}
Write-Host (" orientation: {0}x{1} ({2})" -f $d.w, $d.h, $(if ($d.w -gt $d.h) { "landscape" } else { "portrait" }))
Start-Sleep -Seconds 2 # let the hide timer run so the video fills its final rect
$ui = Get-UiRetry
$vb = VideoBounds $ui
if (-not $vb) { throw "video (16:9 View) not found in the tree - every coordinate expectation below would be meaningless (N21)" }
Write-Host (" video bounds: l={0} t={1} w={2} h={3} ratio err {4:N4}" -f $vb.l, $vb.t, $vb.w, $vb.h, $vb.err)
if ($vb.err -gt 0.02) { throw "video not 16:9 - geometry changed since 4a, re-derive the mapping here" }
function Expect-OnScreen([int]$cx, [int]$cy) {
# CursorMarker on-screen anchor = video view origin + mapped point.
# PS parses `a / [double]b, c` inside @() as an array construction (the type
# literal + comma is a known pitfall - measured 2026-11-02: this threw
# op_Division on Object[] / silently produced 3 elements). Parenthesize each
# element so each division is evaluated as a scalar first.
@( ($vb.l + $cx * $vb.w / [double]$dW), ($vb.t + $cy * $vb.h / [double]$dH) )
}
function Check-MarkerAt([int]$cx, [int]$cy, [string]$tag, [double]$tol = 6.0) {
Set-CursorQuiet $cx $cy
Start-Sleep -Milliseconds 700 # >= one 500ms idle frame + render
$png = Take-Shot $tag
$mk = Find-Marker $png
if (-not $mk) { throw "V-D1 FAIL [$tag]: no marker at desktop ($cx,$cy) - cyan cluster absent" }
$e = Expect-OnScreen $cx $cy
$dx = [Math]::Abs($mk.x - $e[0]); $dy = [Math]::Abs($mk.y - $e[1])
Write-Host (" [{0}] marker ({1:N0},{2:N0}) n={3} spread={4} expect ({5:N0},{6:N0}) dev=({7:N1},{8:N1})" -f `
$tag, $mk.x, $mk.y, $mk.n, $mk.spread, $e[0], $e[1], $dx, $dy)
if ($dx -gt $tol -or $dy -gt $tol) { throw "V-D1 FAIL [$tag]: deviation ($dx,$dy) > $tol px (shot: $png)" }
if ($mk.spread -gt 90) { throw "V-D1 FAIL [$tag]: cyan spread $($mk.spread)px - too wide to be the marker dot (false positive path)" }
return $mk
}
Write-Host "=== 2. V-D1: four desktop points (incl. near-origin) ==="
# (48,48) instead of (0,0): the dot radius would clip at the exact corner and
# bias the centroid; Some(0,0)-vs-None semantics are already locked by unit +
# golden tests. Corner-proximity error is bounded well below tol here.
Check-MarkerAt 48 48 "v-d1-origin" | Out-Null
Check-MarkerAt 1280 720 "v-d1-center" | Out-Null
Check-MarkerAt 2400 1300 "v-d1-lowright" | Out-Null
Check-MarkerAt 2000 300 "v-d1-upperright" | Out-Null
Write-Host " PASS: marker centroid within 6 px for all 4 points"
Write-Host "=== 3. V-D3: idle-desktop move refreshes the marker <= ~1.2s ==="
# Quiet gate, then one decisive jump; 700ms above already covers 500ms resend.
$from = Check-MarkerAt 300 1200 "v-d3-from"
$e2 = Expect-OnScreen 1600 400
Set-CursorQuiet 1600 400
Start-Sleep -Milliseconds 900
$mk3 = Find-Marker (Take-Shot "v-d3-to")
if (-not $mk3) { throw "V-D3 FAIL: marker vanished after moving the cursor on a quiet desktop" }
$dx3 = [Math]::Abs($mk3.x - $e2[0]); $dy3 = [Math]::Abs($mk3.y - $e2[1])
Write-Host (" after jump: marker ({0:N0},{1:N0}) expect ({2:N0},{3:N0}) dev=({4:N1},{5:N1})" -f `
$mk3.x, $mk3.y, $e2[0], $e2[1], $dx3, $dy3)
if ($dx3 -gt 6 -or $dy3 -gt 6) { throw "V-D3 FAIL: marker did not follow within 900ms (stale cursor field?)" }
Write-Host " PASS: marker refreshes within one idle-resend window"
Write-Host "=== 4. V-D7: cursor into the LEFT second monitor folds to None ==="
$vd = [System.Windows.Forms.SystemInformation]::VirtualScreen
if ($vd.X -ge 0) {
Write-Host " SKIP: no negative-X monitor on this machine - V-D7 (off-screen fold) cannot be tested here; the fold rule is locked by fold_cursor unit tests"
} else {
$ox = [int]($vd.X + 100) # deep into the second display (negative X)
$oy = 500
Set-CursorQuiet $ox $oy
Start-Sleep -Milliseconds 900
$mk7 = Find-Marker (Take-Shot "v-d7-second-monitor")
if ($mk7) { throw "V-D7 FAIL: marker still drawn with cursor at ($ox,$oy) outside the captured display (n=$($mk7.n) at {x=$($mk7.x);y=$($mk7.y)})" }
Write-Host " PASS: no cyan cluster with the pointer at ($ox,$oy) - folded to None end to end"
}
Write-Host "=== 5. tidy: park the pointer inside the primary screen ==="
Set-CursorQuiet ([int]($dW/2)) ([int]($dH/2))
Write-Host "=== CURSOR PROOF COMPLETE ==="
+5 -1
View File
@@ -26,7 +26,11 @@ param(
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path $repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe" $adb = "D:\working\android\sdk\platform-tools\adb.exe"
$tmp = "$env:TEMP\remote-emoji-proof" # UNIQUE per run: a fixed path lets the startup Remove-Item below delete the
# stop.flag that leftover targets from a previous FAILED run are watching, so
# they never retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-emoji-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid" $pidFile = "$tmp\target.pid"
+110 -31
View File
@@ -16,17 +16,23 @@
# Not covered: swipe/handwriting input, browsing the candidate list. # Not covered: swipe/handwriting input, browsing the candidate list.
param( param(
# PC (controlled side) LAN address. Default = the 2026-10-31 measured WLAN # PC (controlled side) LAN address. DHCP-assigned and DRIFTED twice
# address; it is DHCP-assigned, so pass -PhoneIp when the office network # (.180 -> .110 -> .112); a stale default silently dials a dead host
# moves (the stale .180 default silently cost a proof run on 2026-10-31). # (measured 2026-11-01: this run failed "phone did not connect" because the
[string]$PhoneIp = "192.168.1.110", # caller passed -Host_, which does NOT bind to this parameter, so the
# default was used). Always pass -PhoneIp explicitly and check the real IP.
[string]$PhoneIp = "192.168.1.112",
[int]$Port = 47800, [int]$Port = 47800,
[string]$Code = "t" [string]$Code = "t"
) )
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path $repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe" $adb = "D:\working\android\sdk\platform-tools\adb.exe"
$tmp = "$env:TEMP\remote-cjk2" # UNIQUE per-run dir: with a fixed path, the startup Remove-Item deletes the
# stop.flag that leftover targets from a previous failed run are watching, so
# they can never retire and they pollute the shared received.txt
# (same class of bug measured 2026-11-01 in phone_chain_proof).
$tmp = "$env:TEMP\remote-cjk2-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid" $pidFile = "$tmp\target.pid"
@@ -99,6 +105,86 @@ function Show-Cps([string]$s) {
($s.ToCharArray() | ForEach-Object { "U+{0:X4}" -f [int]$_ }) -join " " ($s.ToCharArray() | ForEach-Object { "U+{0:X4}" -f [int]$_ }) -join " "
} }
# ---- 4a landscape adaptations ----
# In landscape the HUD TextView hides WITH the auto-hiding control bar, so a
# plain "RTT=" read returns nothing while the session is alive, and the
# keyboard button is not in the tree at all. Both reads must wake through the
# letterbox black bar first (that tap is consumed by the app; it never reaches
# the remote). NB: this script's Adb-Shell returns @{out;code}, not a string.
function Phone-DisplaySize {
$d = (Adb-Shell @("dumpsys", "window", "displays")).out
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 1080; h = 2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 1080; h = 2400 } }
return @{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake-Controls-If-Hidden([string]$xml) {
$hidden = ($xml -match 'class="android\.view\.View"') -and ($xml -notmatch 'class="android\.widget\.Button"')
if (-not $hidden) { return $false }
$pd = Phone-DisplaySize
[void](Adb-Shell @("input", "tap", "100", "$([int]($pd.h / 2))"))
Start-Sleep -Milliseconds 400
return $true
}
# Read the tree until it shows a live session, waking the controls as needed.
function Get-Ui-Live([int]$Seconds) {
$deadline = (Get-Date).AddSeconds($Seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900
$x = Get-Ui
if ($x -match "RTT=") { return $x }
if (Wake-Controls-If-Hidden $x) {
$x = Get-Ui
if ($x -match "RTT=") { return $x }
}
}
return (Get-Ui)
}
# Find a Button by label prefix in a tree that may have auto-hidden controls.
function Find-ButtonByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
return @(
[int]((([int]$b.Groups[1].Value) + ([int]$b.Groups[3].Value)) / 2),
[int]((([int]$b.Groups[2].Value) + ([int]$b.Groups[4].Value)) / 2),
$t)
}
}
}
return $null
}
function Find-ButtonByPrefix-Live([string]$prefix) {
for ($t=0; $t -lt 4; $t++) {
$x = Get-Ui
$b = Find-ButtonByPrefix $x $prefix
if ($b) { return $b }
if (-not (Wake-Controls-If-Hidden $x)) { Start-Sleep -Milliseconds 400 }
}
return $null
}
# Force PORTRAIT for this proof. Why: the IME's ab/zh language key is NOT visible
# to uiautomator (its key views are separate windows), so the toggle is reached by
# a CALIBRATED tap that is portrait-specific (146,2096 on 1080x2400). In landscape
# that y is off-screen, so the retry path could not work. The text pipeline itself
# is orientation-independent (landscape 'abc' is already covered by
# phone_chain_proof), so running V6 in portrait costs nothing in coverage.
function Set-Orientation-Portrait {
$F_ORIENT = -join ([char]0x65B9,[char]0x5411,[char]0x3A) # "fangxiang:"
$F_PORTRAIT = $F_ORIENT + -join ([char]0x7AD6,[char]0x5C4F) # "...:portrait"
for ($t=0; $t -lt 8; $t++) {
$b = Find-ButtonByPrefix-Live $F_ORIENT
if (-not $b) { throw "orientation button not reachable" }
if ($b[2] -eq $F_PORTRAIT) { Write-Host " orientation: already portrait"; return }
[void](Adb-Shell @("input","tap","$($b[0])","$($b[1])"))
Start-Sleep -Milliseconds 1300
}
throw "could not reach portrait orientation after 8 taps"
}
Write-Host "=== 0. retire ALL earlier targets, verify none holds foreground ===" Write-Host "=== 0. retire ALL earlier targets, verify none holds foreground ==="
$flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue | $flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -like "*remote*cjk*" -or $_.FullName -like "*remote*proof*" }) Where-Object { $_.FullName -like "*remote*cjk*" -or $_.FullName -like "*remote*proof*" })
@@ -138,10 +224,11 @@ Write-Host "=== 3. phone session ==="
[void](Adb-Shell @("am","force-stop","com.verify.remote")) [void](Adb-Shell @("am","force-stop","com.verify.remote"))
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity", [void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
"--es","host",$PhoneIp,"--ei","port","$Port","--es","code",$Code)) "--es","host",$PhoneIp,"--ei","port","$Port","--es","code",$Code))
$deadline = (Get-Date).AddSeconds(25) $ui = Get-Ui-Live 25
$ui = ""
while ((Get-Date) -lt $deadline) { Start-Sleep -Milliseconds 900; $ui = Get-Ui; if ($ui -match "RTT=") { break } }
if ($ui -notmatch "RTT=") { throw "phone did not connect" } if ($ui -notmatch "RTT=") { throw "phone did not connect" }
Write-Host " live: $([regex]::Match($ui,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
# 4a: pin PORTRAIT before driving the IME (calibrated toggle tap is portrait-only)
Set-Orientation-Portrait
Write-Host "=== 4. show IME (tap keyboard button) ===" Write-Host "=== 4. show IME (tap keyboard button) ==="
# Select the keyboard button BY ITS LABEL, not by position. # Select the keyboard button BY ITS LABEL, not by position.
@@ -150,19 +237,7 @@ Write-Host "=== 4. show IME (tap keyboard button) ==="
# control and the proof reported "IME not shown" (a harness defect, not a product # control and the proof reported "IME not shown" (a harness defect, not a product
# regression; measured 2026-10-31). Label matching is order-independent. # regression; measured 2026-10-31). Label matching is order-independent.
$F_KEYBOARD = -join ([char]0x952E,[char]0x76D8) # "keyboard" $F_KEYBOARD = -join ([char]0x952E,[char]0x76D8) # "keyboard"
$kb = $null $kb = Find-ButtonByPrefix-Live $F_KEYBOARD
foreach ($n in [regex]::Matches($ui,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t -eq $F_KEYBOARD) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$kb = @(
[int](([int]$b.Groups[1].Value + [int]$b.Groups[3].Value) / 2),
[int](([int]$b.Groups[2].Value + [int]$b.Groups[4].Value) / 2)
)
}
}
}
if (-not $kb) { throw "keyboard button not found by label" } if (-not $kb) { throw "keyboard button not found by label" }
[void](Adb-Shell @("input","tap","$($kb[0])","$($kb[1])")) [void](Adb-Shell @("input","tap","$($kb[0])","$($kb[1])"))
Start-Sleep -Seconds 3 Start-Sleep -Seconds 3
@@ -172,16 +247,20 @@ if ($ime -notmatch "mHaveConnection=true") { throw "IME not bound to our InputCo
Write-Host " IME: $([regex]::Match($ime,'mCurId=(\S+)').Groups[1].Value)" Write-Host " IME: $([regex]::Match($ime,'mCurId=(\S+)').Groups[1].Value)"
Write-Host "=== 4b/5. pinyin -> IME candidate -> commitText (mode-tolerant) ===" Write-Host "=== 4b/5. pinyin -> IME candidate -> commitText (mode-tolerant) ==="
# Both IMEs on this device may remember an ENGLISH mode, in which case letters # The IME may remember an ENGLISH mode, in which case letters pass through
# pass through without composing - which looks identical to "our pipeline lost # without composing - which looks identical to "our pipeline lost the Hanzi"
# the Hanzi" unless you check. The IME's key views are NOT visible to # unless you check. The IME's key views are NOT visible to uiautomator (its
# uiautomator (its dump contains only our app's nodes), so the ab/zh toggle is # dump contains only our app's nodes), so the zh/en toggle is reached by a
# reached by a calibrated tap: iFlytek's key sits at display (73,1048) in a # calibrated tap. A stray tap here changes a keyboard setting, nothing
# 1080x2400 capture => device (146,2096) at 2x. Verified 2026-10-30. # destructive. The loop below is idempotent by design (R-9): flip only if
# The toggle is STATEFUL, so this is written as an idempotent attempt loop: # letters-without-Hanzi were seen, and the judgement is the received text.
# try, and only if we see letters-without-Hanzi do we flip the mode and retry. #
# A stray tap here changes a keyboard setting, nothing destructive. # Toggle calibration (2026-11-01): the device IME is now SOGOU
$modeToggle = @(146, 2096) # (com.sohu.inputmethod.sogou.vivo), not iFlytek - the old (146,2096) iFlytek
# coordinate silently missed the key and both attempts stayed in English mode.
# Re-measured from screencap: bottom row, the language key right of '.'
# (labelled zh/en) at device px (838,2282) on 1080x2400 portrait.
$modeToggle = @(838, 2282)
$expect = -join @([char]0x4F60, [char]0x597D) # NI HAO $expect = -join @([char]0x4F60, [char]0x597D) # NI HAO
$attempt = 0 $attempt = 0
$new = "" $new = ""
+5 -1
View File
@@ -26,7 +26,11 @@ param(
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path $repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe" $adb = "D:\working\android\sdk\platform-tools\adb.exe"
$tmp = "$env:TEMP\remote-ime-proof" # UNIQUE per run: a fixed path lets the startup Remove-Item below delete the
# stop.flag that leftover targets from a previous FAILED run are watching, so
# they never retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-ime-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid" $pidFile = "$tmp\target.pid"
+486
View File
@@ -0,0 +1,486 @@
# 4a landscape proof: rotation, full-height video, auto-hide controls, regressions.
#
# ASCII-only (PS 5.1 reads BOM-less .ps1 as GBK; personal lessons).
#
# Judgements (loop-4a-landscape-plan.md section 6):
# V-C1 connect to 1920x1080 -> app rotates landscape <=2s after live
# (LOCAL display cur= only: this vivo ROM has a permanent 1080x2400
# virtual display and prints no mRotation= line - probe lesson 2026-11-01)
# V-C2 with controls hidden, video view is 16:9 (err <2%) and >=90% screen height
# V-C3 frame counter strictly increases across rotation (session survives)
# V-C4 >=45 fps over a 2.5s window with an ACTIVE desktop animator
# (static desktop is source-limited ~25fps - measuring that would be a
# category error, probe lesson)
# V-C5 controls auto-hide ~3s idle; bottom-edge tap wakes them back
# V-C5b edge-band tap does NOT reach the remote (PC cursor delta <=1px)
# V-C5c all four buttons present after wake; mode tap changes and restores
# V-C6 form page is portrait
# V-C7 orientation button cycles AUTO->L->P->SYS->AUTO, display follows
#
# powershell -ExecutionPolicy Bypass -File tools\landscape_proof.ps1 -Host_ 192.168.1.112
param(
[string]$Host_ = "192.168.1.112",
[int]$Port = 47800,
[string]$Code = "t",
[string]$AgentExe = ""
)
$ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
$ev = Join-Path (Split-Path $repo -Parent) "evidence\4a"
New-Item -ItemType Directory -Force -Path $ev | Out-Null
# ---- Chinese fragments as code points (R-8) ----
$F_ORIENT = -join ([char]0x65B9,[char]0x5411,[char]0x3A) # "fangxiang:" (orientation:)
$F_AUTO = $F_ORIENT + -join ([char]0x81EA,[char]0x52A8) # label: orientation AUTO
$F_LAND = $F_ORIENT + -join ([char]0x6A2A,[char]0x5C4F) # label: orientation LANDSCAPE
$F_PORT = $F_ORIENT + -join ([char]0x7AD6,[char]0x5C4F) # label: orientation PORTRAIT
$F_SYS = $F_ORIENT + -join ([char]0x7CFB,[char]0x7EDF) # label: orientation SYSTEM
$F_CONNECT = -join ([char]0x8FDE,[char]0x63A5) # label: connect
$F_RECONNECT = -join ([char]0x91CD,[char]0x65B0,[char]0x8FDE,[char]0x63A5) # label: reconnect
$F_KB = -join ([char]0x952E,[char]0x76D8) # label: keyboard
$F_MODE = -join ([char]0x6A21,[char]0x5F0F,[char]0x3A) # "moshi:" (mode:)
$F_DISC = -join ([char]0x65AD,[char]0x5F00) # label: disconnect
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try { $out = & $adb @AdbArgs 2>$null; @{ out = ($out -join "`n"); code = $LASTEXITCODE } }
finally { $ErrorActionPreference = $old }
}
function AdbShell { param([string[]]$Cmd) (Adb -AdbArgs (@("shell") + $Cmd)).out }
function Get-Ui {
[void](AdbShell @("uiautomator","dump","/sdcard/ls.xml"))
$r = Adb -AdbArgs @("pull","/sdcard/ls.xml","$env:TEMP\ls.xml")
if ($r.code -ne 0 -or -not (Test-Path "$env:TEMP\ls.xml")) { return "" }
return [System.IO.File]::ReadAllText("$env:TEMP\ls.xml", [System.Text.Encoding]::UTF8)
}
# uiautomator intermittently returns an empty/failed dump (measured 2026-11-01:
# a tap lookup got "" and the null-button tap then crashed with an empty-string
# argument). Retry until the tree is plausibly real; callers still assert on
# their own node - this only removes the transient, never masks a real absence.
function Get-UiRetry([int]$tries=4) {
$x = ""
for ($i=0; $i -lt $tries; $i++) {
$x = Get-Ui
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
Write-Host " [ui] retry exhausted, last dump len=$($x.Length)"
return $x
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml,'text="([^"]+)"') | ForEach-Object { [System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value) }
}
function ButtonByName([string]$xml, [string]$want, [bool]$prefix=$false) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
$hit = if ($prefix) { $t.StartsWith($want) } else { $t -eq $want }
if ($hit) {
$b = [regex]::Match($n.Value,'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if ($b.Success) {
$x1=[int]$b.Groups[1].Value; $y1=[int]$b.Groups[2].Value
$x2=[int]$b.Groups[3].Value; $y2=[int]$b.Groups[4].Value
return @{ cx=[int](($x1+$x2)/2); cy=[int](($y1+$y2)/2) }
}
}
}
return $null
}
function ButtonLabelByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) { return $t }
}
return $null
}
# 4a race (measured 2026-11-01): each uiautomator dump costs ~2s, comparable to
# the 3s auto-hide timer - find-then-read sequences straddle expiry and labels
# come back empty. Every observation goes through the helpers below.
#
# CRITICAL SAFETY: the wake tap (screen bottom center) is only consumed by the
# app's edge band WHEN CONTROLS ARE HIDDEN. If controls are visible, that point
# lands on the control bar (mode/orientation buttons!) and would fire a real
# action. So a wake is allowed only after positively proving the hidden state:
# video View node present AND zero Button nodes.
function ControlsHidden([string]$ui) {
$anyBtn = ($ui -match 'class="android\.widget\.Button"')
$video = ($ui -match 'class="android\.view\.View"')
return ($video -and -not $anyBtn)
}
function Wake-Band {
# 4a design change (measured 2026-11-01): the wake zone is the LETTERBOX
# BLACK BAR, not a bottom edge band. With controls hidden the video is
# full-height, so bottom-center lies INSIDE the picture and would fire a
# real remote click. x=100 is outside the video in both states.
$d = LocalDisplay
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 400
}
function Find-VisibleButton([string]$want, [bool]$prefix=$false) {
for ($try=0; $try -lt 5; $try++) {
$ui = Get-UiRetry
$b = ButtonByName $ui $want $prefix
if ($b) { return $b }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null } # real tree, genuine absence
Start-Sleep -Milliseconds 400 # empty-dump transient
}
return $null
}
function Tap-VisibleButton([string]$want, [bool]$prefix=$false) {
$b = Find-VisibleButton $want $prefix
if (-not $b) { return $null }
[void](AdbShell @("input","tap","$($b.cx)","$($b.cy)"))
return $b
}
# Label read that survives the hide race: wake only from a proven-hidden state.
function Read-ButtonLabel([string]$want, [bool]$prefix=$false) {
for ($try=0; $try -lt 6; $try++) {
$ui = Get-UiRetry
$l = ButtonLabelByPrefix $ui $want
if ($l) { return $l }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null }
Start-Sleep -Milliseconds 400
}
return $null
}
function FrameCount {
$m = [regex]::Match((Get-Ui), ([char]0x5E27 + '=(\d+)'))
if ($m.Success) { return [long]$m.Groups[1].Value }
return -1
}
# 4a self-conflict (measured 2026-11-01): once the controls auto-hide, the HUD
# TextView is GONE too - uiautomator's tree has NO text nodes and FrameCount
# returns -1 even though the session is alive (the video View is right there).
# Wake through the bottom edge band first: that tap is consumed by the app
# (zero remote events), so it cannot perturb the fps window.
function WakeControls {
$d = LocalDisplay
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 500
}
function FrameCountOrWake {
$f = FrameCount
if ($f -ge 0) { return $f }
WakeControls
return FrameCount
}
function LocalDisplay {
$d = (AdbShell @("dumpsys","window","displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w=0; h=0 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w=0; h=0 } }
return @{ w=[int]$m.Groups[1].Value; h=[int]$m.Groups[2].Value }
}
function Wake {
[void](AdbShell @("input","keyevent","224"))
Start-Sleep -Milliseconds 250
[void](AdbShell @("wm","dismiss-keyguard"))
Start-Sleep -Milliseconds 350
}
function Restart-AppForm {
Wake
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am","start","-n","com.verify.remote/.MainActivity"))
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-Ui
if (([regex]::Matches($x,'class="android\.widget\.EditText"')).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
throw "form did not appear: [$((VisibleTexts (Get-Ui)) -join ' || ')]"
}
function CenterOf($m) {
$a=[int]$m.Groups[1].Value; $b=[int]$m.Groups[2].Value
$c=[int]$m.Groups[3].Value; $e=[int]$m.Groups[4].Value
@([int](($a+$c)/2), [int](($b+$e)/2))
}
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 } else { throw "unsupported char $ch" }
[void](AdbShell @("input","keyevent","$k"))
Start-Sleep -Milliseconds 70
}
}
function ClearField($c) {
for ($try=0; $try -lt 3; $try++) {
[void](AdbShell @("input","tap","$($c[0])","$($c[1])"))
Start-Sleep -Milliseconds 250
for ($i=0; $i -lt 40; $i++) { [void](AdbShell @("input","keyevent","67")) }
Start-Sleep -Milliseconds 120
$fx = Get-Ui
$focused = $false
$fi = -1
foreach ($n in [regex]::Matches($fx,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$fi++
if ($n.Value -match 'focused="true"') { $focused = $true }
}
}
if ($focused) { return }
}
throw "field focus failed"
}
# PC-side cursor (V-C5b) + desktop animator (V-C4)
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class Cur4a {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr c);
public static void Dpi(){ SetProcessDpiAwarenessContext(new IntPtr(-4)); }
public static int[] Get(){ POINT p; GetCursorPos(out p); return new int[]{p.X,p.Y}; }
public static void Set(int x,int y){ SetCursorPos(x,y); }
}
"@
[Cur4a]::Dpi()
Write-Host "=== 0. memory precheck (probe lesson: OOM fakes failures) ==="
$os = Get-CimInstance Win32_OperatingSystem
$freeGB = $os.FreeVirtualMemory/1MB
Write-Host (" commit free: {0:N1} GB" -f $freeGB)
if ($freeGB -lt 0.5) { throw "commit memory critically low - free RAM first (gradle --stop)" }
Write-Host "=== 1. stale-agent guard (R-7) ==="
if (-not $AgentExe) { $AgentExe = Join-Path $repo "target\release\agent.exe" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and (Test-Path $AgentExe) -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " agent predates build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 2; Write-Host " agent started"
} else { Write-Host " agent running and current" }
$origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim()
if ($origIme -and $origIme -ne "null") { [void](AdbShell @("ime","disable",$origIme)) }
[void](AdbShell @("svc","power","stayon","true"))
try {
# =====================================================================
Write-Host "=== 2. V-C6: form is portrait ==="
$x = Restart-AppForm
$d = LocalDisplay
Write-Host (" form display: {0}x{1}" -f $d.w, $d.h)
if (-not ($d.w -lt $d.h)) { throw "V-C6 FAIL: form not portrait ($($d.w)x$($d.h))" }
Write-Host " PASS: form portrait"
# =====================================================================
Write-Host "=== 3. V-C1: connect -> AUTO rotates landscape <=2s after live ==="
$F_edits = [regex]::Matches($x,'<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
ClearField (CenterOf $F_edits[0]); TypeAscii $Host_
ClearField (CenterOf $F_edits[1]); TypeAscii "$Port"
ClearField (CenterOf $F_edits[2]); TypeAscii $Code
# Read back until 3 EditText nodes are actually present: uiautomator
# intermittently returns a truncated/empty tree (measured 2026-11-01, worse
# under memory pressure) and a single Get-Ui then reads "form now: " empty
# and fails the fill check as a false negative.
$vals = @()
for ($rd=0; $rd -lt 5; $rd++) {
$x2 = Get-UiRetry
$vals = @()
foreach ($n in [regex]::Matches($x2,'<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
}
}
if ($vals.Count -ge 3) { break }
Write-Host " form readback got $($vals.Count) fields, retry $rd"
Start-Sleep -Milliseconds 500
}
Write-Host " form now: $($vals -join ' | ')"
if ($vals.Count -lt 3) { throw "form fill FAIL: only $($vals.Count) fields after retries" }
if ($vals[0] -ne $Host_ -or $vals[1] -ne "$Port" -or $vals[2] -ne $Code) { throw "form fill FAIL" }
$btn = ButtonByName $x2 $F_CONNECT
if (-not $btn) { $btn = ButtonByName $x2 $F_RECONNECT }
if (-not $btn) { $btn = Find-VisibleButton $F_CONNECT; if (-not $btn) { $btn = Find-VisibleButton $F_RECONNECT } }
if (-not $btn) { throw "connect button not found" }
[void](AdbShell @("input","tap","$($btn.cx)","$($btn.cy)"))
$deadline = (Get-Date).AddSeconds(20); $live = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
if ((Get-Ui) -match "RTT=") { $live = $true; break }
}
if (-not $live) { throw "never reached live: [$((VisibleTexts (Get-Ui)) -join ' || ')]" }
$tRot = Get-Date
$deadline = $tRot.AddSeconds(2); $rotated = $false
while ((Get-Date) -lt $deadline) {
$d = LocalDisplay
if ($d.w -gt $d.h) { $rotated = $true; break }
Start-Sleep -Milliseconds 150
}
$ms = ((Get-Date) - $tRot).TotalMilliseconds
Write-Host (" display {0}x{1}, landscape within {2:N0}ms of live" -f $d.w,$d.h,$ms)
if (-not $rotated) { throw "V-C1 FAIL: still portrait 2s after live" }
Write-Host " PASS: AUTO rotated to landscape"
# =====================================================================
Write-Host "=== 4. V-C3 + V-C4: counter monotonic; >=45fps with animated desktop ==="
# Static desktop is source-limited (~25fps): the honest window needs motion.
$anim = Start-Job -ScriptBlock {
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class An { [DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y); }
"@
$end=(Get-Date).AddSeconds(14); $i=0
while ((Get-Date) -lt $end) {
$a=$i*0.18
[An]::SetCursorPos([int](960+320*[Math]::Cos($a)), [int](540+320*[Math]::Sin($a))) | Out-Null
Start-Sleep -Milliseconds 12; $i++
}
}
Start-Sleep -Seconds 2 # let the encoder see motion
$fA = FrameCountOrWake; $ta = Get-Date
Start-Sleep -Milliseconds 2500
$fB = FrameCountOrWake; $tb = Get-Date
$fps = if ($fA -ge 0 -and $fB -gt $fA) { [Math]::Round(($fB-$fA)/(($tb-$ta).TotalSeconds),1) } else { -1 }
Write-Host " frames $fA -> $fB fps=$fps (animated)"
if ($fB -le $fA) { throw "V-C3 FAIL: counter stalled across rotation" }
Write-Host " PASS: V-C3 session survived rotation"
if ($fps -lt 45) { throw "V-C4 FAIL: $fps fps < 45 with animated desktop" }
Write-Host " PASS: V-C4 fps >= 45"
# =====================================================================
Write-Host "=== 5. V-C5: wake -> visible -> idle -> hidden ==="
# R-8 discipline: "button absent" is NOT a valid judgement on its own - an
# empty/broken dump would pass it trivially (measured 2026-11-01: after the
# hide, the HUD TextView is GONE too, so the tree has zero text nodes).
# Positive pair: keyboard button absent WHILE the video View node is present.
#
# Sequence is explicit (first run's implicit precondition raced: the fps
# step's wake expired its 3s timer before we checked, so controls were
# already hidden - correct product behavior, wrong harness assumption):
# wake -> assert visible -> idle -> assert hidden.
WakeControls
$x5 = Get-Ui
if (-not (ButtonByName $x5 $F_KB)) { throw "V-C5 FAIL: edge wake did not bring controls back" }
Write-Host " visible after wake: OK"
$deadline = (Get-Date).AddSeconds(6); $hidden = $false
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 600
$x5 = Get-Ui
$videoSeen = ($x5 -match 'class="android\.view\.View"')
$kbGone = -not (ButtonByName $x5 $F_KB)
if ($videoSeen -and $kbGone) { $hidden = $true; break }
}
if (-not $hidden) {
throw "V-C5 FAIL: no (kb-gone AND video-present) state within 6s; last dump len=$($x5.Length)" }
Write-Host " PASS: controls auto-hidden (video View still in tree - real hide, not empty dump)"
Write-Host "=== 5b. V-C2: video 16:9 and >=90% screen height (controls hidden) ==="
$ui = Get-UiRetry
$best = $null; $bestErr = 1e9
foreach ($n in [regex]::Matches($ui,'<node[^>]*class="android\.view\.View"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')) {
$x1=[int]$n.Groups[1].Value; $y1=[int]$n.Groups[2].Value
$x2i=[int]$n.Groups[3].Value; $y2=[int]$n.Groups[4].Value
$w=$x2i-$x1; $h=$y2-$y1
if ($w -lt 300 -or $h -lt 100) { continue }
$err = [Math]::Abs($w/[double]$h - 16.0/9.0)
if ($err -lt $bestErr) { $bestErr=$err; $best=@{w=$w;h=$h} }
}
$d = LocalDisplay
$hFrac = $best.h / [double]$d.h
Write-Host (" video {0}x{1} ratio err {2:N4} height {3:N1}% of {4}x{5}" -f `
$best.w,$best.h,$bestErr,($hFrac*100),$d.w,$d.h)
if ($bestErr -gt 0.02) { throw "V-C2 FAIL: video not 16:9 (err $bestErr)" }
if ($hFrac -lt 0.90) { throw ("V-C2 FAIL: video height {0:N1}% < 90%" -f ($hFrac*100)) }
Write-Host " PASS: full-height 16:9 video (the tiny-video fix)"
Stop-Job $anim -ErrorAction SilentlyContinue; Remove-Job $anim -Force -ErrorAction SilentlyContinue
Write-Host "=== 6. V-C5b: black-bar tap wakes controls WITHOUT touching the remote ==="
Start-Sleep -Seconds 2 # animator fully stopped
# The wake zone IS the letterbox black bar now (4a design change), so this
# is the honest target. Also assert a video-interior tap is NOT consumed as
# a wake - otherwise the black bar would have eaten real clicks too.
$pc0 = [Cur4a]::Get()
[void](AdbShell @("input","tap","100","$([int]($d.h/2))"))
Start-Sleep -Milliseconds 600
$pc1 = [Cur4a]::Get()
$ui = Get-UiRetry
$kb = ButtonByName $ui $F_KB
$jump = [Math]::Max([Math]::Abs($pc1[0]-$pc0[0]), [Math]::Abs($pc1[1]-$pc0[1]))
Write-Host (" black-bar tap: cursor jump=$jump px ; keyboard back: {0}" -f ($null -ne $kb))
if (-not $kb) { throw "V-C5 FAIL: black-bar tap did not wake controls" }
if ($jump -gt 1) { throw "V-C5b FAIL: black-bar tap leaked $jump px to remote" }
Write-Host " PASS: wake works, zero remote leakage"
Write-Host "=== 6c. four buttons present; mode tap changes and restores ==="
$kb2 = Find-VisibleButton $F_KB
$mode = Find-VisibleButton $F_MODE $true
$disc = Find-VisibleButton $F_DISC
$orient = Find-VisibleButton $F_ORIENT $true
if (-not ($kb2 -and $mode -and $disc -and $orient)) {
throw ("V-C5 FAIL: missing buttons kb={0} mode={1} disc={2} orient={3}" -f `
($null -ne $kb2),($null -ne $mode),($null -ne $disc),($null -ne $orient)) }
$beforeLabel = Read-ButtonLabel $F_MODE $true
if (-not $beforeLabel) { throw "V-C5 precondition FAIL: mode label unreadable" }
[void](Tap-VisibleButton $F_MODE $true)
Start-Sleep -Milliseconds 600
$afterLabel = Read-ButtonLabel $F_MODE $true
Write-Host " mode label: [$beforeLabel] -> [$afterLabel]"
# null would trivially "change" from a real label - assert non-empty too (R-8)
if (-not $afterLabel -or $beforeLabel -eq $afterLabel) { throw "V-C5 FAIL: mode tap changed nothing" }
# converge back to relative (the product default the later proofs assume),
# regardless of which way the flip above landed - tap until it reads relative.
$F_MODE_REL = $F_MODE + -join ([char]0x76F8,[char]0x5BF9) # mode: relative
$converged = $false
for ($i=0; $i -lt 3 -and -not $converged; $i++) {
$cur = Read-ButtonLabel $F_MODE $true
if ($cur -eq $F_MODE_REL) { $converged = $true; break }
[void](Tap-VisibleButton $F_MODE $true)
Start-Sleep -Milliseconds 600
}
$restored = Read-ButtonLabel $F_MODE $true
Write-Host " mode converged to relative: [$restored]"
if (-not $converged -or $restored -ne $F_MODE_REL) {
throw "V-C5 FAIL: mode not converged to relative (pollutes later proofs)" }
Write-Host " PASS: buttons work, state restored"
# =====================================================================
Write-Host "=== 7. V-C7: orientation cycle AUTO->L->P->SYS->AUTO ==="
$seq = @($F_LAND, $F_PORT, $F_SYS, $F_AUTO)
$wantLand = @($true, $false, $null, $true) # null = don't assert SYSTEM policy
for ($i=0; $i -lt 4; $i++) {
$orient = Find-VisibleButton $F_ORIENT $true
if (-not $orient) { throw "V-C7 FAIL: orientation button unreachable after 5 wakes" }
[void](Tap-VisibleButton $F_ORIENT $true)
Start-Sleep -Milliseconds 1400
$label = Read-ButtonLabel $F_ORIENT $true
$d = LocalDisplay
$land = $d.w -gt $d.h
Write-Host (" step {0}: label=[{1}] want=[{2}] display={3}x{4} landscape={5}" -f ($i+1),$label,$seq[$i],$d.w,$d.h,$land)
if ($label -ne $seq[$i]) { throw ("V-C7 FAIL: step {0} label [{1}] != [{2}]" -f ($i+1),$label,$seq[$i]) }
if ($null -ne $wantLand[$i] -and $land -ne $wantLand[$i]) {
throw ("V-C7 FAIL: step {0} landscape={1} want={2}" -f ($i+1),$land,$wantLand[$i]) }
}
Write-Host " PASS: full cycle with display following each policy"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-c7-final.png" | Out-Null
}
finally {
# 4a pollution discipline: leave no live session behind - the next proof's
# Restart-AppForm force-stops the app mid-session, which strands the agent's
# serial accept in a zombie (N11) and the run after that fails to connect
# (measured 2026-11-01: agent alive but no longer serving handshakes).
[void](AdbShell @("am","force-stop","com.verify.remote"))
Start-Sleep -Milliseconds 500
Get-Job | Stop-Job -ErrorAction SilentlyContinue
Get-Job | Remove-Job -Force -ErrorAction SilentlyContinue
[void](AdbShell @("svc","power","stayon","false"))
if ($origIme -and $origIme -ne "null") {
[void](AdbShell @("ime","enable",$origIme))
[void](AdbShell @("ime","set",$origIme))
}
}
Write-Host "=== LANDSCAPE PROOF COMPLETE ==="
+38 -3
View File
@@ -23,7 +23,12 @@ param(
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
$repo = (Resolve-Path "$PSScriptRoot\..").Path $repo = (Resolve-Path "$PSScriptRoot\..").Path
$adb = "D:\working\android\sdk\platform-tools\adb.exe" $adb = "D:\working\android\sdk\platform-tools\adb.exe"
$tmp = "$env:TEMP\remote-phone-proof" # UNIQUE per-run dir (measured 2026-11-01): with a fixed path, this script's
# own startup Remove-Item deleted the stop.flag that LEFTOVER targets from a
# previous failed run were watching - they could never retire, kept grabbing
# foreground every 60ms, and wrote their textbox content into the shared
# received.txt ("abc\nabc" = old run's abc+ENTER + this run's abc).
$tmp = "$env:TEMP\remote-phone-proof-$PID"
New-Item -ItemType Directory -Force -Path $tmp | Out-Null New-Item -ItemType Directory -Force -Path $tmp | Out-Null
$pidFile = "$tmp\target.pid" $pidFile = "$tmp\target.pid"
@@ -94,12 +99,40 @@ function Get-PhoneUiText {
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8) return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
} }
# 4a adaptation: in landscape the HUD TextView hides WITH the auto-hiding
# control bar, so "RTT=" disappears while the session is perfectly alive
# (measured 2026-11-01: this proof died at step 4 "session died after ENTER"
# on a healthy 22fps stream). Detect the hidden state (video View node present,
# zero Button nodes) and wake via the letterbox black bar - that tap is
# consumed by the app and never reaches the remote.
function Phone-DisplaySize {
# NB: this script's Adb-Shell returns @{out;code}, not a bare string.
$d = (Adb-Shell @("dumpsys", "window", "displays")).out
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 1080; h = 2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 1080; h = 2400 } }
return @{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake-Controls-If-Hidden([string]$xml) {
$hidden = ($xml -match 'class="android\.view\.View"') -and ($xml -notmatch 'class="android\.widget\.Button"')
if (-not $hidden) { return $false }
$pd = Phone-DisplaySize
[void](Adb-Shell @("input", "tap", "100", "$([int]($pd.h / 2))"))
Start-Sleep -Milliseconds 400
return $true
}
function Wait-For-LiveSession([int]$Seconds, [ref]$outXml) { function Wait-For-LiveSession([int]$Seconds, [ref]$outXml) {
$deadline = (Get-Date).AddSeconds($Seconds) $deadline = (Get-Date).AddSeconds($Seconds)
while ((Get-Date) -lt $deadline) { while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 900 Start-Sleep -Milliseconds 900
$x = Get-PhoneUiText $x = Get-PhoneUiText
if ($x -match "RTT=") { $outXml.Value = $x; return $true } if ($x -match "RTT=") { $outXml.Value = $x; return $true }
if (Wake-Controls-If-Hidden $x) {
$x = Get-PhoneUiText
if ($x -match "RTT=") { $outXml.Value = $x; return $true }
}
} }
$outXml.Value = (Get-PhoneUiText) $outXml.Value = (Get-PhoneUiText)
return $false return $false
@@ -249,8 +282,10 @@ try {
$vals += [regex]::Match($v, 'text="([^"]*)"').Groups[1].Value $vals += [regex]::Match($v, 'text="([^"]*)"').Groups[1].Value
} }
} }
# NOTE: an EMPTY EditText reports its hint as text (verified: text="被控端 IP"), # NOTE: an EMPTY EditText reports its hint as text (verified on device), so
# so "field text == typed value" is the assertion, not "field text non-empty". # "field non-empty" is NOT a usable judgement - the assertion is always
# "field text == typed value". (ASCII-only comments: PS 5.1 decodes
# BOM-less .ps1 as GBK.)
Write-Host " form now: $($vals -join ' | ')" Write-Host " form now: $($vals -join ' | ')"
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" } if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
if ($vals[0] -ne $PhoneIp) { throw "V4 FAIL: IP field is [$($vals[0])], expected [$PhoneIp]" } if ($vals[0] -ne $PhoneIp) { throw "V4 FAIL: IP field is [$($vals[0])], expected [$PhoneIp]" }
+349
View File
@@ -0,0 +1,349 @@
# proof_common.ps1 - shared device-side proof helpers (4b, personal docs loop-4b S6).
#
# ASCII-only file (PS 5.1 reads BOM-less .ps1 as GBK; repo lessons).
# Dot-source it: . "$PSScriptRoot\proof_common.ps1"
#
# Provenance: these are the battle-tested helpers from landscape_proof.ps1 (4a:
# wake-only-from-proven-hidden, positive-pair hide check, N25-style unique temp
# files, uiautomator empty-dump retry) plus rel_mouse_phone_proof's focus-assert
# field clearing. Older proofs are NOT migrated to this file in loop 4b (changing
# four validated scripts without re-running all four is itself a false-green
# risk) - migration is registered as a harness hygiene item for the next loop.
# ---- adb wrappers (never let stderr trip ErrorActionPreference=Stop) ----
if (-not (Get-Variable -Name PccAdb -Scope Global -ErrorAction SilentlyContinue)) {
$global:PccAdb = "D:\working\android\sdk\platform-tools\adb.exe"
}
function Adb {
param([Parameter(Mandatory)][string[]]$AdbArgs)
$old = $ErrorActionPreference; $ErrorActionPreference = "Continue"
try {
$out = & $global:PccAdb @AdbArgs 2>$null
@{ out = ($out -join "`n"); code = $LASTEXITCODE }
} finally { $ErrorActionPreference = $old }
}
function AdbShell { param([string[]]$Cmd) (Adb -AdbArgs (@("shell") + $Cmd)).out }
# ---- UI tree ----
# Unique temp per process (N25 lesson applied to dump pulls, cheap insurance).
$script:PccUiTmp = "$env:TEMP\pcc_ui_$PID.xml"
$script:PccDevDump = "/sdcard/pcc_$PID.xml"
function Get-Ui {
[void](AdbShell @("uiautomator", "dump", $script:PccDevDump))
$r = Adb -AdbArgs @("pull", $script:PccDevDump, $script:PccUiTmp)
if ($r.code -ne 0 -or -not (Test-Path $script:PccUiTmp)) { return "" }
return [System.IO.File]::ReadAllText($script:PccUiTmp, [System.Text.Encoding]::UTF8)
}
# uiautomator intermittently dumps empty (measured repeatedly in 4a). Retry until
# the tree is plausibly real. Callers still assert on their own node - this only
# removes the transient, never masks a genuine absence.
function Get-UiRetry([int]$tries = 4) {
$x = ""
for ($i = 0; $i -lt $tries; $i++) {
$x = Get-Ui
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
Write-Host " [ui] retry exhausted, last dump len=$($x.Length)"
return $x
}
function VisibleTexts([string]$xml) {
[regex]::Matches($xml, 'text="([^"]+)"') | ForEach-Object {
[System.Net.WebUtility]::HtmlDecode($_.Groups[1].Value)
}
}
# ---- buttons: match by LABEL, never by position (3b/N23 lesson) ----
function ButtonNode([string]$xml, [string]$want, [bool]$prefix = $false) {
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
$hit = if ($prefix) { $t.StartsWith($want) } else { $t -eq $want }
if ($hit) { return $n.Value }
}
return $null
}
function ButtonCenter([string]$xml, [string]$want, [bool]$prefix = $false) {
$v = ButtonNode $xml $want $prefix
if (-not $v) { return $null }
$b = [regex]::Match($v, 'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
if (-not $b.Success) { return $null }
@{ cx = [int](([int]$b.Groups[1].Value + [int]$b.Groups[3].Value) / 2)
cy = [int](([int]$b.Groups[2].Value + [int]$b.Groups[4].Value) / 2) }
}
function ButtonLabelByPrefix([string]$xml, [string]$prefix) {
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
if ($t.StartsWith($prefix)) { return $t }
}
return $null
}
# ---- 4a observation-surface discipline: controls auto-hide in landscape ----
# Hidden state = POSITIVE PAIR (video View node present AND zero Button nodes).
# An empty dump would otherwise pass "no buttons" trivially (N24 false-fails).
function ControlsHidden([string]$ui) {
$anyBtn = ($ui -match 'class="android\.widget\.Button"')
$video = ($ui -match 'class="android\.view\.View"')
return ($video -and -not $anyBtn)
}
# Wake tap lives in the letterbox black bar (x=100): outside the video in both
# states, consumed by the app only when hidden - zero remote leakage (4a/D1).
function Wake-Band {
$d = LocalDisplay
[void](AdbShell @("input", "tap", "100", "$([int]($d.h / 2))"))
Start-Sleep -Milliseconds 400
}
function Find-VisibleButton([string]$want, [bool]$prefix = $false) {
for ($try = 0; $try -lt 5; $try++) {
$ui = Get-UiRetry
$b = ButtonCenter $ui $want $prefix
if ($b) { return $b }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null } # real tree, genuine absence
Start-Sleep -Milliseconds 400 # empty-dump transient
}
return $null
}
function Tap-VisibleButton([string]$want, [bool]$prefix = $false) {
$b = Find-VisibleButton $want $prefix
if (-not $b) { return $null }
[void](AdbShell @("input", "tap", "$($b.cx)", "$($b.cy)"))
return $b
}
function Read-ButtonLabel([string]$want, [bool]$prefix = $false) {
for ($try = 0; $try -lt 6; $try++) {
$ui = Get-UiRetry
$l = ButtonLabelByPrefix $ui $want
if ($l) { return $l }
if (ControlsHidden $ui) { Wake-Band; continue }
if ($ui.Length -gt 500) { return $null }
Start-Sleep -Milliseconds 400
}
return $null
}
# ---- HUD / session liveness ----
# "zhen=" frame counter (char U+5E27 as code point, ASCII-safe source).
function FrameCount {
$m = [regex]::Match((Get-Ui), ([char]0x5E27 + '=(\d+)'))
if ($m.Success) { return [long]$m.Groups[1].Value }
return -1
}
function WakeControls {
$d = LocalDisplay
[void](AdbShell @("input", "tap", "100", "$([int]($d.h / 2))"))
Start-Sleep -Milliseconds 500
}
function FrameCountOrWake {
$f = FrameCount
if ($f -ge 0) { return $f }
WakeControls
return FrameCount
}
# Live HUD line (RTT= only exists when connected; hidden-controls caveat above).
function Wait-LiveHud([int]$seconds = 20) {
$deadline = (Get-Date).AddSeconds($seconds)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 500
$ui = Get-Ui
if ($ui -notmatch "RTT=") {
if (ControlsHidden $ui) { WakeControls }
continue
}
$m = [regex]::Match($ui, 'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { return [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value) }
}
return $null
}
# ---- display / app lifecycle ----
# vivo ROM prints NO mRotation= line and keeps a permanent 1080x2400 virtual
# display (N30): orientation MUST be read from cur= inside the mDisplayId=0 block.
function LocalDisplay {
$d = (AdbShell @("dumpsys", "window", "displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w = 0; h = 0 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w = 0; h = 0 } }
@{ w = [int]$m.Groups[1].Value; h = [int]$m.Groups[2].Value }
}
function Wake {
[void](AdbShell @("input", "keyevent", "224"))
Start-Sleep -Milliseconds 250
[void](AdbShell @("wm", "dismiss-keyguard"))
Start-Sleep -Milliseconds 350
}
function Restart-AppForm {
Wake
[void](AdbShell @("am", "force-stop", "com.verify.remote"))
Start-Sleep -Milliseconds 600
[void](AdbShell @("am", "start", "-n", "com.verify.remote/.MainActivity"))
$deadline = (Get-Date).AddSeconds(8)
while ((Get-Date) -lt $deadline) {
$x = Get-Ui
if (([regex]::Matches($x, 'class="android\.widget\.EditText"')).Count -ge 3) { return $x }
Start-Sleep -Milliseconds 300
}
throw "form did not appear: [$(VisibleTexts (Get-Ui) -join ' || ')]"
}
# ---- connect form: focus-assert typing (rel_mouse_proof grade, R-8) ----
function CenterOf($m) {
@([int](([int]$m.Groups[1].Value + [int]$m.Groups[3].Value) / 2),
[int](([int]$m.Groups[2].Value + [int]$m.Groups[4].Value) / 2))
}
function FormFields([string]$xml) {
[regex]::Matches($xml, '<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
}
function TypeAscii([string]$s) {
foreach ($ch in $s.ToUpper().ToCharArray()) {
$k = if ($ch -ge '0' -and $ch -le '9') { 7 + [int][string]$ch }
elseif ($ch -ge 'A' -and $ch -le 'Z') { 29 + ([int]$ch - [int][char]'A') }
elseif ($ch -eq '.') { 56 } else { throw "unsupported char $ch" }
[void](AdbShell @("input", "keyevent", "$k"))
Start-Sleep -Milliseconds 70
}
}
function FocusedIndex([string]$xml) {
$i = -1
foreach ($n in [regex]::Matches($xml, '<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$i++
if ($n.Value -match 'focused="true"') { return $i }
}
}
return -1
}
function Clear-Field($c, [int]$wantIdx) {
for ($try = 0; $try -lt 3; $try++) {
[void](AdbShell @("input", "tap", "$($c[0])", "$($c[1])"))
$deadline = (Get-Date).AddSeconds(1.5)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
if ((FocusedIndex (Get-Ui)) -eq $wantIdx) {
for ($i = 0; $i -lt 40; $i++) { [void](AdbShell @("input", "keyevent", "67")) }
Start-Sleep -Milliseconds 120
return
}
}
}
throw "field $wantIdx never took focus"
}
function FillAndConnect([string]$xml, [string]$host_, [string]$port_, [string]$code_,
[string]$fConnect, [string]$fReconnect) {
$f = FormFields $xml
if ($f.Count -lt 3) { throw "form has $($f.Count) fields" }
Clear-Field (CenterOf $f[0]) 0; TypeAscii $host_
Clear-Field (CenterOf $f[1]) 1; TypeAscii $port_
Clear-Field (CenterOf $f[2]) 2; TypeAscii $code_
$expect = @($host_, $port_, $code_)
$vals = @()
for ($retry = 0; $retry -le 2; $retry++) {
$x2 = Get-UiRetry
$vals = @()
foreach ($n in [regex]::Matches($x2, '<node[^>]*>')) {
if ($n.Value -match 'class="android\.widget\.EditText"') {
$vals += [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value, 'text="([^"]*)"').Groups[1].Value)
}
}
if ($vals.Count -lt 3) { throw "expected 3 fields after typing, got $($vals.Count)" }
$bad = -1
for ($i = 0; $i -lt 3; $i++) { if ($vals[$i] -ne $expect[$i]) { $bad = $i; break } }
if ($bad -ge 0) {
$f = FormFields $x2
Clear-Field (CenterOf $f[$bad]) $bad
TypeAscii $expect[$bad]
continue
}
$btn = ButtonCenter $x2 $fConnect
if (-not $btn) { $btn = ButtonCenter $x2 $fReconnect }
if (-not $btn) { $btn = Find-VisibleButton $fConnect
if (-not $btn) { $btn = Find-VisibleButton $fReconnect } }
if (-not $btn) { throw "connect button not found" }
[void](AdbShell @("input", "tap", "$($btn.cx)", "$($btn.cy)"))
return
}
throw "form input never converged: [$($vals -join ' | ')]"
}
# ---- video geometry (uiautomator has no SurfaceView - match 16:9 View, N21) ----
function VideoBounds([string]$xml) {
$best = $null; $bestErr = 1e9
foreach ($n in [regex]::Matches($xml, '<node[^>]*class="android\.view\.View"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')) {
$x1 = [int]$n.Groups[1].Value; $y1 = [int]$n.Groups[2].Value
$x2 = [int]$n.Groups[3].Value; $y2 = [int]$n.Groups[4].Value
$w = $x2 - $x1; $h = $y2 - $y1
if ($w -lt 300 -or $h -lt 100) { continue }
$err = [Math]::Abs($w / [double]$h - 16.0 / 9.0)
if ($err -lt $bestErr) { $bestErr = $err; $best = @{ l = $x1; t = $y1; w = $w; h = $h } }
}
if ($best) { $best.err = $bestErr }
return $best
}
# ---- PC-side cursor: DPI-aware set with hijack gates ----
if (-not ('PccCur' -as [type])) {
Add-Type -TypeDefinition @"
using System; using System.Runtime.InteropServices;
public class PccCur {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr c);
public static void Dpi(){ SetProcessDpiAwarenessContext(new IntPtr(-4)); }
public static int[] Get(){ POINT p; GetCursorPos(out p); return new int[]{p.X,p.Y}; }
public static void Set(int x,int y){ SetCursorPos(x,y); }
}
"@
}
[PccCur]::Dpi()
# SetCursorQuiet: the machine may be shared (2026-11-02 probe: a human used the
# real mouse mid-probe and faked every reading). Gate: two reads 350ms apart must
# agree (nobody is moving), then Set + read-back must equal the request; retries
# on hijack; loud failure instead of a silently wrong sample (R-8).
function Set-CursorQuiet([int]$x, [int]$y, [int]$tries = 8) {
for ($i = 0; $i -lt $tries; $i++) {
$a = [PccCur]::Get()
Start-Sleep -Milliseconds 350
$b = [PccCur]::Get()
if (-not ($a[0] -eq $b[0] -and $a[1] -eq $b[1])) {
Write-Host " [cursor] busy ($($a -join ',')->$($b -join ',')), retry $($i+1)"
Start-Sleep -Seconds 1
continue
}
[PccCur]::Set($x, $y)
Start-Sleep -Milliseconds 120
$g = [PccCur]::Get()
if ($g[0] -eq $x -and $g[1] -eq $y) { return }
Write-Host " [cursor] readback ($($g -join ',')) != ($x,$y), retry $($i+1)"
Start-Sleep -Milliseconds 500
}
throw "cursor never settled at ($x,$y) - someone is using the real mouse; retry the proof when the PC is idle"
}
# ---- memory precheck (N20/N29: OOM fakes build/test failures) ----
function Memory-Precheck([double]$minGB = 0.5) {
$os = Get-CimInstance Win32_OperatingSystem
$freeGB = $os.FreeVirtualMemory / 1MB
Write-Host (" commit free: {0:N1} GB" -f $freeGB)
if ($freeGB -lt $minGB) { throw "commit memory critically low - free RAM first (gradle --stop)" }
}
# ---- R-7 stale-agent guard: binary newer than process => restart ----
function Ensure-FreshAgent([string]$AgentExe, [int]$Port, [string]$Code) {
if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe (cargo build --release -p agent)" }
$ag = @(Get-Process agent -ErrorAction SilentlyContinue)
if ($ag.Count -gt 0 -and ((Get-Item $AgentExe).LastWriteTime -gt $ag[0].StartTime)) {
Write-Host " agent predates build; restarting"
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
Start-Sleep -Milliseconds 800; $ag = @()
}
if ($ag.Count -eq 0) {
Start-Process $AgentExe -ArgumentList "--port", "$Port", "--code", $Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 2
Write-Host " agent started (fresh)"
} else { Write-Host " agent running and current" }
}
+257 -63
View File
@@ -34,17 +34,25 @@ $F_LONGPRESS = -join ([char]0x957F,[char]0x6309,[char]0x53F3,[char]0x952E)
$F_RCLICK_TAG = -join ([char]0x53F3,[char]0x952E,[char]0x3D) # "right-click=" $F_RCLICK_TAG = -join ([char]0x53F3,[char]0x952E,[char]0x3D) # "right-click="
$F_CLICK = -join ([char]0x5355,[char]0x51FB) # "single click" $F_CLICK = -join ([char]0x5355,[char]0x51FB) # "single click"
# Read the persistent right-click counter from the HUD ("右键=N(source)"). # Read the persistent right-click counter from the HUD ("right-click=N(source)").
# Returns -1 when absent. A counter is used instead of the one-line debug text # Returns -1 when absent. A counter is used instead of the one-line debug text
# because ACTION_UP immediately overwrites the debug line after a long press # because ACTION_UP immediately overwrites the debug line after a long press
# (measured 2026-10-31: screen showed "长按结束", the firing evidence was gone). # (measured 2026-10-31: screen showed the long-press-END label, the firing evidence was gone).
function RightClickCount { function RightClickCount {
$m = [regex]::Match((Get-UiText), 'text="([^"]*RTT=[^"]*)"') for ($t=0; $t -lt 3; $t++) {
if (-not $m.Success) { return -1 } $x = Get-UiText
$hud = [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value) $m = [regex]::Match($x, 'text="([^"]*RTT=[^"]*)"')
$c = [regex]::Match($hud, "$F_RCLICK_TAG(\d+)") if ($m.Success) {
if ($c.Success) { return [int]$c.Groups[1].Value } $hud = [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value)
return 0 $c = [regex]::Match($hud, "$F_RCLICK_TAG(\d+)")
if ($c.Success) { return [int]$c.Groups[1].Value }
return 0
}
# 4a: in landscape the HUD hides WITH the controls (GONE, not empty text)
Wake-Controls $x
Start-Sleep -Milliseconds 400
}
return -1
} }
function Adb { function Adb {
@@ -110,16 +118,64 @@ function ButtonStartingWith([string]$xml, [string]$prefix) {
return $null return $null
} }
# 4a self-adaptation: in landscape the control bar auto-hides after 3s idle
# (and the HUD TextView hides with it). The wake tap goes to the LEFT BLACK BAR
# (x=100, outside the letterboxed video) - the app consumes black-bar touches
# only in the hidden state, so this never reaches the remote. Bottom-center is
# NOT usable: with the controls hidden the video is full-height, so the bottom
# edge lies INSIDE the picture and would fire a real remote click.
function PhoneDisplay {
$d = (AdbShell @("dumpsys","window","displays"))
$idx = $d.IndexOf("Display: mDisplayId=0")
if ($idx -lt 0) { return @{ w=1080; h=2400 } }
$m = [regex]::Match($d.Substring($idx), 'cur=(\d+)x(\d+)')
if (-not $m.Success) { return @{ w=1080; h=2400 } }
return @{ w=[int]$m.Groups[1].Value; h=[int]$m.Groups[2].Value }
}
function Wake-Controls([string]$xml) {
if ($xml -match 'class="android\.widget\.Button"') { return } # visible already
$pd = PhoneDisplay
[void](AdbShell @("input","tap","100","$([int]($pd.h/2))"))
Start-Sleep -Milliseconds 500
}
# HUD text with wake-retry (4a: in landscape the HUD TextView hides WITH the
# control bar - a plain read returns "" once the 3s timer expired, and each
# dump itself costs ~2s so reads routinely straddle the expiry).
function Get-HudText {
for ($t=0; $t -lt 3; $t++) {
$x = Get-UiText
$m = [regex]::Match($x, 'text="([^"]*RTT=[^"]*)"')
if ($m.Success) { return [System.Net.WebUtility]::HtmlDecode($m.Groups[1].Value) }
Wake-Controls $x
Start-Sleep -Milliseconds 400
}
return ""
}
# Toggle the pointer mode and PROVE it flipped by re-reading the button label. # Toggle the pointer mode and PROVE it flipped by re-reading the button label.
# The bottom bar sits at y=2256..2324, right against the system navigation bar; # The bottom bar sits at y=2256..2324, right against the system navigation bar;
# a single tap there is not reliable (measured 2026-10-31: the tap landed but the # a single tap there is not reliable (measured 2026-10-31: the tap landed but the
# mode did not change). So: tap, wait, re-read, retry; and judge on the BUTTON # mode did not change). So: tap, wait, re-read, retry; and judge on the BUTTON
# label as well as the HUD (two independent observations of the same state). # label as well as the HUD (two independent observations of the same state).
function Get-UiTextRetry([int]$tries = 4) {
# landscape_proof Get-UiRetry pattern (N24-era lesson): uiautomator
# intermittently returns short/empty trees; a single miss inside a
# wake-then-read sequence used to burn the whole mode-switch budget
# (measured 2026-11-02 step 3: full run died at "mode button not found"
# although the very next steps would have re-dumped fine).
$x = ""
for ($i = 0; $i -lt $tries; $i++) {
$x = Get-UiText
if ($x.Length -gt 500) { return $x }
Start-Sleep -Milliseconds 400
}
return $x
}
function Set-Mode([string]$want) { function Set-Mode([string]$want) {
# $want is "relative" or "absolute" # $want is "relative" or "absolute"
$wantTag = if ($want -eq "relative") { $F_RELATIVE } else { $F_ABSOLUTE } $wantTag = if ($want -eq "relative") { $F_RELATIVE } else { $F_ABSOLUTE }
for ($try=0; $try -lt 4; $try++) { for ($try=0; $try -lt 6; $try++) {
$xml = Get-UiText $xml = Get-UiTextRetry
$cur = $null $cur = $null
foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) { foreach ($n in [regex]::Matches($xml,'<node[^>]*class="android\.widget\.Button"[^>]*>')) {
$t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value) $t = [System.Net.WebUtility]::HtmlDecode([regex]::Match($n.Value,'text="([^"]*)"').Groups[1].Value)
@@ -132,14 +188,24 @@ function Set-Mode([string]$want) {
} }
} }
} }
if (-not $cur) { throw "mode button not found (no Button whose text contains '$F_MODE_TAG')" } if (-not $cur) {
# 4a: controls may be auto-hidden (landscape). Wake and fold the miss
# into the SAME try budget instead of a single re-read + instant throw.
Wake-Controls $xml
continue
}
Write-Host " mode button label now: [$($cur.label)] (try $try)" Write-Host " mode button label now: [$($cur.label)] (try $try)"
if ($cur.label.Contains($wantTag)) { return $true } if ($cur.label.Contains($wantTag)) { return $true }
[void](AdbShell @("input","tap","$($cur.cx)","$($cur.cy)")) [void](AdbShell @("input","tap","$($cur.cx)","$($cur.cy)"))
Start-Sleep -Milliseconds 700 Start-Sleep -Milliseconds 700
} }
$final = ((VisibleTexts (Get-UiText)) -join " || ") # Fail LOUD with evidence: a bare text dump cannot show what the screen
throw "mode did not switch to $want after 4 taps; screen: [$final]" # actually looked like (this is how the 2026-11-02 step-3 death became a
# guessing game; one PNG makes the next occurrence self-evidencing).
$evShot = Join-Path $ev "mode-switch-fail-$PID.png"
try { & powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out $evShot | Out-Null } catch { }
$final = ((VisibleTexts (Get-UiTextRetry)) -join " || ")
throw "mode did not switch to $want after 6 tries; screen: [$final] shot: $evShot"
} }
# The video SurfaceView bounds - touch coordinates must land inside it. # The video SurfaceView bounds - touch coordinates must land inside it.
# #
@@ -301,7 +367,7 @@ if ($ag.Count -gt 0 -and (Test-Path $AgentExe) -and ((Get-Item $AgentExe).LastWr
if ($ag.Count -eq 0) { if ($ag.Count -eq 0) {
if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe" } if (-not (Test-Path $AgentExe)) { throw "missing $AgentExe" }
Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null Start-Process $AgentExe -ArgumentList "--port","$Port","--code",$Code -WindowStyle Hidden | Out-Null
Start-Sleep -Seconds 1; Write-Host " agent started (v4)" Start-Sleep -Seconds 1; Write-Host " agent started (v5)"
} else { Write-Host " agent running and current" } } else { Write-Host " agent running and current" }
$origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim() $origIme = ((AdbShell @("settings","get","secure","default_input_method")) -join "").Trim()
@@ -320,8 +386,10 @@ try {
$hud = "" $hud = ""
if (-not (Wait-Live 14 ([ref]$hud))) { throw "never reached a live session" } if (-not (Wait-Live 14 ([ref]$hud))) { throw "never reached a live session" }
Write-Host " HUD: $hud" Write-Host " HUD: $hud"
if (-not $hud.Contains("v4")) { throw "V-B4 FAIL: HUD does not report protocol v4; got [$hud]" } # HUD version pin (3b/V-B4). PROTO bump MUST sync this string -
Write-Host " PASS: live session on protocol v4" # see MainActivity PROTO_LABEL comment (4b R-5 pair).
if (-not $hud.Contains("v5")) { throw "V-B4 FAIL: HUD does not report protocol v5; got [$hud]" }
Write-Host " PASS: live session on protocol v5"
$xml = Get-UiText $xml = Get-UiText
$sb = SurfaceBounds $xml $sb = SurfaceBounds $xml
@@ -331,6 +399,39 @@ try {
Write-Host " surface bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)" Write-Host " surface bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-b1-live.png" | Out-Null & powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out "$ev\v-b1-live.png" | Out-Null
# 4a pollution guard: the mode PERSISTS in prefs across runs, and a run that
# threw between Set-Mode "absolute" and its restore left absolute installed
# (measured 2026-11-01: step 2 ran in absolute mode, the DOWN absolute move
# "teleported" 1441px - correct product behavior, wrong precondition).
# Force relative, then settle back into the hidden steady state: while the
# controls are hidden, gestures never re-show them, so the geometry stays
# stable for steps 2/2b/2c.
$hudNow = Get-HudText
Write-Host " HUD at start: $hudNow"
# Converge to relative UNCONDITIONALLY via Set-Mode, which reads the BUTTON
# label and returns early if already relative (idempotent, authoritative).
# The old guard `if ($hudNow.Contains(absolute))` silently skipped when the
# HUD read came back empty or straddled the auto-hide (N24 observation-surface
# trap) - a mode left polluted by a previous run that threw after Set-Mode
# absolute then poisoned step 2c: absolute mode's DOWN DOES send an absolute
# move (correct product behavior), which the leak-point check read as a
# teleport (measured again 2026-11-02: prefs relative_mode=false at start,
# V-B1a "passed" on direction alone because absolute swipes also have
# direction - the whole step 2 block was running in the wrong mode).
[void](Set-Mode "relative")
Start-Sleep -Milliseconds 3800 # let the 3s auto-hide re-engage
$xh = Get-UiText
if ($xh -match 'class="android\.widget\.Button"') { throw "precondition FAIL: controls did not re-hide after 3.8s" }
$hsb = SurfaceBounds $xh
# NB: PowerShell variable names are CASE-INSENSITIVE - a local `$sh` here
# silently clobbers `$SH` (screen height) and every later [PC]::Set($SW/2,$SH/2)
# dies with "Object[] has no op_Division" (measured 2026-11-01). Prefix locals
# distinctly ($hsb = hidden-state bounds).
if (-not $hsb) { throw "precondition FAIL: no video bounds in hidden state" }
$vx1=$hsb[0]; $vy1=$hsb[1]; $vx2=$hsb[2]; $vy2=$hsb[3]
$vcx=[int](($vx1+$vx2)/2); $vcy=[int](($vy1+$vy2)/2)
Write-Host " hidden steady-state bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
# ================================================================= # =================================================================
Write-Host "=== 2. V-B1a: relative mode moves PC pointer (swipe right) ===" Write-Host "=== 2. V-B1a: relative mode moves PC pointer (swipe right) ==="
# Park the PC cursor at a known point so the delta is unambiguous. # Park the PC cursor at a known point so the delta is unambiguous.
@@ -373,55 +474,112 @@ try {
# absolute move. Tap at the far LEFT of the view; the pointer must not jump # absolute move. Tap at the far LEFT of the view; the pointer must not jump
# to the left edge of the desktop. # to the left edge of the desktop.
# #
# HARNESS DISCIPLINE (measured 2026-10-31): a swipe's final relative events # DISCRIMINATION (2026-11-01, landscape): a real absolute-leak has an exact
# can still be in flight when we re-park the cursor. The first attempt showed # signature - the pointer lands at the tap's desktop mapping
# phantom "jumps" of 15px and 298px that were really the PREVIOUS swipe's # (0.05*1920=96, 540) and STAYS. Residual swipe events instead keep the
# queued deltas landing after our SetCursorPos. So: wait for the link to go # cursor DRIFTING for seconds (observed: stable-looking 4-read windows
# quiet, then require the baseline to be STABLE across two reads before # followed by -890/-1418px surges - bursts with >1.2s gaps between them).
# judging. Otherwise this test would report a product defect that isn't one. # So: settle (6 equal reads), tap, then sample 4x over 2s.
Start-Sleep -Milliseconds 1200 # all samples == before -> PASS
[PC]::Set([int]($SW*0.8), [int]($SH/2)) # samples drift -> harness residue: retry (max 3)
Start-Sleep -Milliseconds 500 # settled AT leak point -> product FAIL (no retry will change it)
$stable = $false # Absolute-leak landing point for this tap: view fraction 0.05 maps to
for ($i=0; $i -lt 6; $i++) { # desktop x = 0.05*(SW-1); y = view center row -> desktop mid row.
$r1 = [PC]::Get(); Start-Sleep -Milliseconds 350; $r2 = [PC]::Get() $leakX = [int][Math]::Round(0.05 * ($SW-1))
if ($r1[0] -eq $r2[0] -and $r1[1] -eq $r2[1]) { $stable = $true; break } $leakY = [int][Math]::Round((($vcy-$vy1) / [double]($vy2-$vy1)) * ($SH-1))
Write-Host " cursor still settling: ($($r1[0]),$($r1[1])) -> ($($r2[0]),$($r2[1]))" $verdict = "retry"
for ($att=1; $att -le 3 -and $verdict -eq "retry"; $att++) {
Start-Sleep -Milliseconds 2000
[PC]::Set([int]($SW*0.8), [int]($SH/2))
Start-Sleep -Milliseconds 1500
$stable = $false
for ($i=0; $i -lt 8; $i++) {
$runs = @()
for ($k=0; $k -lt 6; $k++) { $runs += ,([PC]::Get()); Start-Sleep -Milliseconds 250 }
$eq = $true
foreach ($r in $runs) { if ($r[0] -ne $runs[0][0] -or $r[1] -ne $runs[0][1]) { $eq = $false } }
if ($eq) { $stable = $true; break }
Write-Host (" att{0}: cursor still settling (last {1},{2}); re-parking" -f $att,$runs[-1][0],$runs[-1][1])
[PC]::Set([int]($SW*0.8), [int]($SH/2))
Start-Sleep -Milliseconds 800
}
if (-not $stable) { throw "cannot obtain a stable cursor baseline (residual events never drained)" }
$beforeTap = [PC]::Get()
$tapX = [int]($vx1 + ($vx2-$vx1)*0.05)
[void](AdbShell @("input","tap","$tapX","$vcy"))
$samples = @()
for ($k=0; $k -lt 4; $k++) { Start-Sleep -Milliseconds 500; $samples += ,([PC]::Get()) }
$dbgTap = ((VisibleTexts (Get-UiText)) -join " || ")
$maxJump = 0
foreach ($s in $samples) {
$j = [Math]::Max([Math]::Abs($s[0]-$beforeTap[0]), [Math]::Abs($s[1]-$beforeTap[1]))
if ($j -gt $maxJump) { $maxJump = $j }
}
$last = $samples[-1]
$drift = $false
for ($k=1; $k -lt 4; $k++) {
if ($samples[$k][0] -ne $samples[0][0] -or $samples[$k][1] -ne $samples[0][1]) { $drift = $true }
}
$atLeak = ([Math]::Abs($last[0]-$leakX) -le 8) -and ([Math]::Abs($last[1]-$leakY) -le 8)
# NB: "$_[0]" inside a double-quoted string expands only `$_` and prints
# "[0]" literally (PS 5.1) - use explicit sub-expressions.
$sampleStr = ($samples | ForEach-Object { "(" + $_[0] + "," + $_[1] + ")" }) -join " "
Write-Host (" att{0}: tap ({1},{2}) -> samples {3} maxJump={4} drift={5} atLeakPoint={6} (leak=({7},{8}))" -f `
$att,$beforeTap[0],$beforeTap[1],$sampleStr,$maxJump,$drift,$atLeak,$leakX,$leakY)
if ($dbgTap.Contains($F_LONGPRESS)) { throw "tap was interpreted as a long press; harness timing is wrong" }
if ($atLeak) { throw ("V-B1a FAIL: pointer settled at the absolute-leak point ({0},{1}) - DOWN sent an absolute move" -f $last[0],$last[1]) }
if ($maxJump -le 8 -and -not $drift) { $verdict = "pass"; break }
if (-not $drift) {
# settled far away but NOT at the leak point: some other single event
# moved it - that IS a product-relevant teleport, retry once more
# before condemning (a queued burst can also settle far away).
Write-Host " att${att}: settled far (jump=$maxJump) but not at leak point - treating as residue, retrying"
}
} }
if (-not $stable) { throw "cannot obtain a stable cursor baseline (residual relative events never drained)" } if ($verdict -ne "pass") { throw "V-B1a FAIL: pointer moved on touch-down in relative mode across 3 attempts (residual or teleport - see samples above)" }
$beforeTap = [PC]::Get()
$tapX = [int]($vx1 + ($vx2-$vx1)*0.05)
[void](AdbShell @("input","tap","$tapX","$vcy"))
Start-Sleep -Milliseconds 700
$afterTap = [PC]::Get()
$jump = [Math]::Max([Math]::Abs($afterTap[0] - $beforeTap[0]), [Math]::Abs($afterTap[1] - $beforeTap[1]))
$dbgTap = ((VisibleTexts (Get-UiText)) -join " || ")
Write-Host " tap at view-left x=$tapX ; pointer ($($beforeTap[0]),$($beforeTap[1])) -> ($($afterTap[0]),$($afterTap[1])) jump=$jump px"
if ($dbgTap.Contains($F_LONGPRESS)) { throw "tap was interpreted as a long press; harness timing is wrong" }
if ($jump -gt 8) { throw "V-B1a FAIL: touch-down teleported the pointer by $jump px (absolute move leaked into relative mode)" }
Write-Host " PASS: relative mode leaves the pointer where it was (no teleport)" Write-Host " PASS: relative mode leaves the pointer where it was (no teleport)"
# ================================================================= # =================================================================
Write-Host "=== 3. V-B2: switch to ABSOLUTE mode, re-run R7 touch mapping ===" Write-Host "=== 3. V-B2: switch to ABSOLUTE mode, re-run R7 touch mapping ==="
[void](Set-Mode "absolute") [void](Set-Mode "absolute")
$hud2 = "" # 4a geometry race (measured 2026-11-01): Set-Mode WOK the controls, so the
$m2 = [regex]::Match((Get-UiText),'text="([^"]*RTT=[^"]*)"') # video shrank to 1216x684 - but each dump costs ~2s and the auto-hide timer
if ($m2.Success) { $hud2 = [System.Net.WebUtility]::HtmlDecode($m2.Groups[1].Value) } # is 3s, so by tap time the controls had hidden again and the video was back
# to full height. Tapping with the stale visible-state geometry produced
# "errors" of 281-357px that were pure harness error (y landed at the old
# center row 341, not 540). Wait for the hidden steady state and sample
# THOSE bounds - in hidden state a video tap only re-arms the timer, it never
# re-shows the controls, so the geometry is stable for the whole R7 loop.
Start-Sleep -Milliseconds 3800
$hud2 = Get-HudText
$sb = SurfaceBounds (Get-UiText)
if (-not $sb) { throw "V-B2 FAIL: no video bounds in hidden steady state" }
$vx1=$sb[0];$vy1=$sb[1];$vx2=$sb[2];$vy2=$sb[3]
$vcx=[int](($vx1+$vx2)/2); $vcy=[int](($vy1+$vy2)/2)
Write-Host " hidden steady-state bounds = [$vx1,$vy1][$vx2,$vy2] center=($vcx,$vcy)"
Write-Host " HUD: $hud2" Write-Host " HUD: $hud2"
if (-not $hud2.Contains($F_ABSOLUTE)) { throw "V-B2 FAIL: HUD still does not say absolute; got [$hud2]" } if (-not $hud2.Contains($F_ABSOLUTE)) { throw "V-B2 FAIL: HUD still does not say absolute; got [$hud2]" }
Write-Host " PASS: mode switched to absolute (button label and HUD agree)" Write-Host " PASS: mode switched to absolute (button label and HUD agree)"
# R7 expectations for 1920x1080 (physical pixels, 125% scaling does NOT apply # R7 expectations derived from the LIVE desktop size (physical pixels; 125%
# to the injected desktop space - the agent injects into the physical desktop) # scaling does NOT apply - the agent injects into the physical desktop, and
# normX=frac*65535 lands at frac*(SW-1)).
# Fixed 1920-based literals broke the moment the capture display became
# 2560x1440 (2026-11-02: reported 575px error = 0.9*(2560-1920), a harness
# staleness, not a mapping regression). AGENTS.md: coordinate expectations
# MUST be recomputed for the current resolution. Same script already uses
# dynamic $SW/$SH for V-B1a and the leak point - this block was the outlier.
$r7 = @( $r7 = @(
@{ frac=0.10; ex=[int](192); ey=[int](108) }, @{ frac=0.10 },
@{ frac=0.90; ex=[int](1728); ey=[int](972) }, @{ frac=0.90 },
@{ frac=0.50; ex=[int](960); ey=[int](540) } @{ frac=0.50 }
) )
$worst = 0 $worst = 0
foreach ($c in $r7) { foreach ($c in $r7) {
$tx = [int]($vx1 + ($vx2-$vx1)*$c.frac) $tx = [int]($vx1 + ($vx2-$vx1)*$c.frac)
$ty = [int]($vy1 + ($vy2-$vy1)*$c.frac) $ty = [int]($vy1 + ($vy2-$vy1)*$c.frac)
$c.ex = [int][Math]::Round($c.frac * ($SW-1))
$c.ey = [int][Math]::Round($c.frac * ($SH-1))
[void](AdbShell @("input","tap","$tx","$ty")) [void](AdbShell @("input","tap","$tx","$ty"))
Start-Sleep -Milliseconds 400 Start-Sleep -Milliseconds 400
$p = [PC]::Get() $p = [PC]::Get()
@@ -443,21 +601,48 @@ try {
# Use a desktop of our own: the proof's WinForms target window (R-11: never # Use a desktop of our own: the proof's WinForms target window (R-11: never
# inject into a window we do not own - it would modify user data). # inject into a window we do not own - it would modify user data).
# NB: the 3b version launched this WITHOUT its Mandatory params, so the
# child died instantly and the right-click landed at raw screen center
# (possible user window - near-miss on R-11). Launch properly now and park
# the pointer INSIDE our own window: topmost form at (80,80) 420x140.
$target = Join-Path $repo "tools\text_target.ps1" $target = Join-Path $repo "tools\text_target.ps1"
if (-not (Test-Path $target)) { throw "missing $target" } if (-not (Test-Path $target)) { throw "missing $target" }
$tp = Start-Process powershell -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass","-File",$target ` # UNIQUE per-run dir (same orphan-stop-flag trap as phone_chain_proof,
-PassThru -WindowStyle Minimized # fixed 2026-11-01): startup Remove-Item must not delete the retirement
Start-Sleep -Seconds 3 # channel of targets left by a previous failed run.
Write-Host " target window started (pid $($tp.Id))" $ttmp = Join-Path $env:TEMP "vbx4a-$PID"
New-Item -ItemType Directory -Force -Path $ttmp | Out-Null
$pidFile = "$ttmp\target.pid"; $stopFile = "$ttmp\stop.flag"; $resultFile = "$ttmp\received.txt"
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
$tp = Start-Process powershell -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass","-File",$target, `
"-PidFile",$pidFile,"-ResultFile",$resultFile,"-StopFile",$stopFile -PassThru -WindowStyle Hidden
$deadline = (Get-Date).AddSeconds(8)
while (-not (Test-Path $pidFile) -and (Get-Date) -lt $deadline) { Start-Sleep -Milliseconds 200 }
if (-not (Test-Path $pidFile)) { throw "target window did not start (no pid file)" }
Write-Host " target window started (pid $(Get-Content $pidFile))"
# Park the pointer over the middle of the target window's edit area by # Long-press and drag run in the HIDDEN steady state: geometry is stable
# absolute injection first (so we know where the long press lands). # there (video taps only re-arm the hide timer, they never re-show the
[PC]::Set([int]($SW/2), [int]($SH/2)) # controls), and the counter read wakes the HUD afterwards.
function Enter-HiddenSteady {
Start-Sleep -Milliseconds 3800
$x = Get-UiText
if ($x -match 'class="android\.widget\.Button"') { throw "precondition FAIL: controls did not hide" }
$s = SurfaceBounds $x
if (-not $s) { throw "precondition FAIL: no hidden-state video bounds" }
return $s
}
$rcBefore = RightClickCount # may wake the controls; re-enter hidden after
$hsb4 = Enter-HiddenSteady
$hx1=$hsb4[0];$hy1=$hsb4[1];$hx2=$hsb4[2];$hy2=$hsb4[3]
$hcx=[int](($hx1+$hx2)/2); $hcy=[int](($hy1+$hy2)/2)
Write-Host " right-click counter before: $rcBefore ; hidden center=($hcx,$hcy)"
# The long press right-clicks AT THE REMOTE POINTER, so park the pointer on
# our own target window first - the menu then opens over a window we own.
[PC]::Set(290, 150)
Start-Sleep -Milliseconds 300 Start-Sleep -Milliseconds 300
$preLong = [PC]::Get() [void](AdbShell @("input","swipe","$hcx","$hcy","$hcx","$hcy","900")) # press & hold ~900ms, no movement
$rcBefore = RightClickCount
Write-Host " right-click counter before: $rcBefore"
[void](AdbShell @("input","swipe","$vcx","$vcy","$vcx","$vcy","900")) # press & hold ~900ms, no movement
Start-Sleep -Milliseconds 700 Start-Sleep -Milliseconds 700
$rcAfter = RightClickCount $rcAfter = RightClickCount
$debug = ((VisibleTexts (Get-UiText)) -join " || ") $debug = ((VisibleTexts (Get-UiText)) -join " || ")
@@ -472,7 +657,8 @@ try {
Write-Host "=== 4b. V-B3 negative: a DRAG must not fire long press ===" Write-Host "=== 4b. V-B3 negative: a DRAG must not fire long press ==="
$rcBeforeDrag = RightClickCount $rcBeforeDrag = RightClickCount
[void](AdbShell @("input","swipe","$vcx","$vcy",$([int]($vcx+120)),$([int]($vcy+80)),900)) Enter-HiddenSteady | Out-Null
[void](AdbShell @("input","swipe","$hcx","$hcy",$([int]($hcx+120)),$([int]($hcy+80)),900))
Start-Sleep -Milliseconds 700 Start-Sleep -Milliseconds 700
$rcAfterDrag = RightClickCount $rcAfterDrag = RightClickCount
$debug2 = ((VisibleTexts (Get-UiText)) -join " || ") $debug2 = ((VisibleTexts (Get-UiText)) -join " || ")
@@ -487,10 +673,18 @@ try {
$rcBeforeTwo = RightClickCount $rcBeforeTwo = RightClickCount
# adb cannot synthesize a two-finger tap (lessons R-6): this is a MANUAL item. # adb cannot synthesize a two-finger tap (lessons R-6): this is a MANUAL item.
Write-Host " (two-finger tap is not drivable via adb - recorded as manual, not asserted here)" Write-Host " (two-finger tap is not drivable via adb - recorded as manual, not asserted here)"
Stop-Process -Id $tp.Id -Force -ErrorAction SilentlyContinue
} }
finally { finally {
# 4a pollution discipline: end with no live session - the agent's 10s idle
# reclaim then happens BETWEEN runs, not in the middle of the next run's
# connect (measured 2026-11-01: a stranded zombie left the next proof unable
# to reach a live session at all).
[void](AdbShell @("am","force-stop","com.verify.remote"))
# target window exits via its stop flag (text_target contract); kill is the
# fallback. In finally so a throw never leaves a topmost window on the user's desktop.
if ($stopFile) { [void](New-Item -ItemType File -Force -Path $stopFile -ErrorAction SilentlyContinue) }
Start-Sleep -Milliseconds 400
if ($tp -and -not $tp.HasExited) { Stop-Process -Id $tp.Id -Force -ErrorAction SilentlyContinue }
[void][PC]::Set($origCursor[0], $origCursor[1]) [void][PC]::Set($origCursor[0], $origCursor[1])
[void](AdbShell @("svc","power","stayon","false")) [void](AdbShell @("svc","power","stayon","false"))
if ($origIme -and $origIme -ne "null") { if ($origIme -and $origIme -ne "null") {
+5 -1
View File
@@ -24,7 +24,11 @@ $ErrorActionPreference = "Stop"
# Repo root derived from this script's location (tools/ is one level under the # Repo root derived from this script's location (tools/ is one level under the
# repo root) so the proof runs on any clone, not just the author's D: drive. # repo root) so the proof runs on any clone, not just the author's D: drive.
$repo = (Resolve-Path "$PSScriptRoot\..").Path $repo = (Resolve-Path "$PSScriptRoot\..").Path
$tmp = "$env:TEMP\remote-text-proof" # UNIQUE per run: a fixed path lets the startup Remove-Item delete the stop.flag
# that leftover targets from a previous FAILED run are watching, so they never
# retire and keep writing into the shared received.txt
# (same trap fixed in phone_chain_proof, measured 2026-11-01).
$tmp = "$env:TEMP\remote-text-proof-$PID"
$base = "D:\working\environment" $base = "D:\working\environment"
$env:RUSTUP_HOME = "$base\rust" $env:RUSTUP_HOME = "$base\rust"
+8
View File
@@ -198,6 +198,10 @@ pub struct Frame {
pub ts_ms: u64, pub ts_ms: u64,
pub seq: u64, pub seq: u64,
pub keyframe: bool, pub keyframe: bool,
/// v5:帧发送时刻的远端光标位置(采集画面内物理像素)。
/// `None` = 隐藏/离屏/后端未实现(语义单一来源在被控端,见 `VideoFrameHeader::cursor`)。
/// 桌面 viewer 不消费该字段(看着自己的鼠标),安卓据它画标记。
pub cursor: Option<(i16, i16)>,
pub payload: Vec<u8>, pub payload: Vec<u8>,
} }
@@ -209,6 +213,7 @@ impl From<(VideoFrameHeader, Vec<u8>)> for Frame {
ts_ms: h.ts_ms, ts_ms: h.ts_ms,
seq: h.seq, seq: h.seq,
keyframe: h.keyframe, keyframe: h.keyframe,
cursor: h.cursor,
payload, payload,
} }
} }
@@ -678,12 +683,15 @@ mod tests {
ts_ms: 42, ts_ms: 42,
seq: 7, seq: 7,
keyframe: true, keyframe: true,
cursor: Some((640, 480)),
}; };
let payload = vec![0u8; 16]; let payload = vec![0u8; 16];
let f: Frame = (h.clone(), payload.clone()).into(); let f: Frame = (h.clone(), payload.clone()).into();
assert_eq!((f.width, f.height), (1920, 1080)); assert_eq!((f.width, f.height), (1920, 1080));
assert_eq!(f.seq, 7); assert_eq!(f.seq, 7);
assert!(f.keyframe); assert!(f.keyframe);
// v5:光标字段必须透传,丢它=标记永远不显示,且丢法是静默的
assert_eq!(f.cursor, Some((640, 480)));
assert_eq!(f.payload, payload); assert_eq!(f.payload, payload);
} }