Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 yu
|
0f3f1cf7f7 |
No files matched your search
@@ -1,117 +0,0 @@
|
||||
# B 侧设计文档（主控端 + 协议 + 网络）
|
||||
|
||||
> 负责人：成员 B ｜ 主责 crate：`viewer/`、`common/` ｜ 配合：A（agent/）
|
||||
> 按里程碑推进，每个里程碑独立 feature 分支，合入即打 tag。
|
||||
|
||||
---
|
||||
|
||||
## B1 · 联调基线（1-2 天，和 A 合作完成）
|
||||
|
||||
目标：拿到 H.264 版双机实测数据，作为一切优化的对照基线。
|
||||
|
||||
1. **HUD 扩展**：新增 丢包率(估算)/抖动(jitter)/帧间隔 p95 三项读数
|
||||
- 抖动 = 相邻帧 E2E 差值的绝对值 EMA；帧间隔 p95 用环形缓冲取 95 分位；
|
||||
- 码流异常计数（解码失败次数）也上 HUD，验证关键帧自愈是否生效。
|
||||
2. **测试脚本** `tools/bench.md`：固定三个场景各测 60s —— 静止桌面 / 播放视频 / 拖动窗口，记录 HUD 截图。
|
||||
3. **产出**：基线数据表填入本文件末尾。
|
||||
|
||||
## B2 · 码率自适应（3-4 天）
|
||||
|
||||
目标：弱网自动降码率保流畅，好网缓慢升码率提画质。
|
||||
|
||||
### 协议扩展（common/）
|
||||
|
||||
```rust
|
||||
/// B2 新增控制消息（流 2）
|
||||
RateReport {
|
||||
loss_permille: u16, // 接收端估算丢包率 ‰
|
||||
rtt_ms: u32, // 最近 RTT
|
||||
e2e_ms: u32, // 平滑后端到端延迟
|
||||
}
|
||||
```
|
||||
|
||||
viewer 每 2s 汇总一次发送（流 2 复用，无新增握手）。
|
||||
|
||||
### 控制算法（先简后精，AIMD 风格）
|
||||
|
||||
| 网络状态（连续 3 个周期） | 动作 |
|
||||
|---|---|
|
||||
| 丢包 > 8% 或 E2E > 400ms | 码率 ×0.6（下限 1Mbps），同时通知编码器降帧率到 20fps |
|
||||
| 丢包 3-8% | 码率 ×0.9 |
|
||||
| 丢包 < 1% 且 E2E < 150ms 持续 10s | 码率 +500Kbps 步进上探（上限 15Mbps） |
|
||||
|
||||
### agent 侧配合点（给 A 的接口需求）
|
||||
|
||||
- 需要运行时改码率的入口：优先查 openh264 crate 是否暴露 `SetOption(RC_PARAM)`；
|
||||
- 若 crate 不支持 → 方案 B：保留 encoder 实例但按新码率重建（重建成本 ~50ms，降码率不频繁，可接受）；
|
||||
- 降级到极限（1Mbps 仍丢包 >15%）→ 主动降分辨率 1080p→720p（A 实现，协议传 `ResChange` 通知，viewer 重配解码器与窗口尺寸）。
|
||||
|
||||
## B3 · P2P 打洞 + 信令 + 中继（1.5-2 周，核心里程碑）
|
||||
|
||||
目标：跨公网可用。三层结构：
|
||||
|
||||
```
|
||||
┌─────────┐ ①注册/交换地址(WebSocket) ┌──────────┐
|
||||
│ viewer │ ───────────────────────────→ │ 信令服务器 │
|
||||
└────┬────┘ └──────────┘
|
||||
│ ②拿到对方候选地址(IP:port×N) ↑
|
||||
│ ③a 直连(打洞) ──────────────────── agent 注册
|
||||
│ ③b 失败 → 走 TURN 中继
|
||||
```
|
||||
|
||||
### 信令服务器（B 主建，A 参与评审）
|
||||
|
||||
- 最小实现：单二进制 Rust + WebSocket（tokio-tungstenite），无数据库；
|
||||
- 消息：`Register{agent_id}` / `Connect{agent_id}` / `Answer{candidates, auth_code_hint}`；
|
||||
- agent 开机注册自身 ID（设备码），viewer 输对方 ID 发起；
|
||||
- 中立服务器，不碰媒体数据，带宽成本 ≈ 0。
|
||||
|
||||
### 候选地址与打洞
|
||||
|
||||
1. agent 侧收集候选：本机局域网地址、路由器公网映射（STUN 探测，自建 coturn 的 stun 或公共 stun）；
|
||||
2. 双方经信令交换候选后**同时**向对方发探测包（UDP 打洞，锥形 NAT 成功率高；对称 NAT 基本打不通）；
|
||||
3. 打通 → 直连（QUIC 直接到该地址）；打不通 → TURN 中继兜底。
|
||||
|
||||
### 中继方案（务实路线）
|
||||
|
||||
QUIC 本身没有标准 TURN 集成，两条路：
|
||||
- **路线一（推荐先做）**：自建中继服务 `relay` —— viewer 把 QUIC 包封装转发给 relay，relay 转给 agent（本质是应用层 TURN，复用信令服务器的部署）；
|
||||
- **路线二（二期）**：coturn + 手写 ICE-lite，仅在路线一性能不足时投入。
|
||||
|
||||
### 安全（随 B3 一起做，替代现在的跳过校验）
|
||||
|
||||
- 预共享验证码升级为 **PAKE（SPAKE2）**：验证码不落服务器、不落网络，双方推导会话密钥；
|
||||
- 信令只传地址候选，媒体端到端加密保持 QUIC TLS 不变。
|
||||
|
||||
## B4 · 多屏 + 文件传输（各 2-3 天，穿插做）
|
||||
|
||||
**多屏**：
|
||||
- 协议：`ScreenList{screens: Vec<{id, w, h}>}`（AuthOk 里带）、`SelectScreen{id}`；
|
||||
- agent 枚举 DXGI outputs（A 实现），viewer 加屏幕切换下拉；
|
||||
- 第二期再做"多屏同显"。
|
||||
|
||||
**文件传输**：
|
||||
- 首选走已有 QUIC 连接开独立双向流（不引入 aria2 依赖，部署简单）；
|
||||
- 协议：`FileOffer{name, size}` → `FileAccept` → 原始分块流（1MB 块 + 校验）；
|
||||
- viewer 做拖拽发送 + 保存对话框。
|
||||
|
||||
---
|
||||
|
||||
## 里程碑与分支规划
|
||||
|
||||
| 里程碑 | 分支 | 预估 | 验收标准 |
|
||||
|---|---|---|---|
|
||||
| B1 联调基线 | feature/hud-baseline | 1-2 天 | 三场景基线数据入库 |
|
||||
| B2 码率自适应 | feature/rate-adapt | 3-4 天 | 模拟弱网（clumsy/丢包工具）下自动降码率，画面不断流 |
|
||||
| B3 打洞+中继 | feature/p2p-relay | 1.5-2 周 | 两台跨公网机器通过 ID 互联；打洞失败自动走中继 |
|
||||
| B4 多屏/文件 | feature/screens-files | 3-5 天 | 多屏切换、拖拽传文件可用 |
|
||||
|
||||
顺序建议：B1 → B2 → B3 → B4（B3 是价值最大项，B4 可穿插）。
|
||||
|
||||
## 基线数据表（B1 完成后填写）
|
||||
|
||||
| 场景 | FPS | RTT | E2E | 码率 | 丢包 | 抖动 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 静止桌面 | | | | | | |
|
||||
| 视频播放 | | | | | | |
|
||||
| 拖动窗口 | | | | | | |
|
||||
Generated
+32
-247
@@ -6,14 +6,9 @@ version = 4
|
||||
name = "agent"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"agent-core",
|
||||
"anyhow",
|
||||
"common",
|
||||
"lz4_flex",
|
||||
"nvenc",
|
||||
"openh264",
|
||||
"quinn",
|
||||
"rcgen",
|
||||
"rustls",
|
||||
"tokio",
|
||||
"windows",
|
||||
]
|
||||
@@ -31,17 +26,6 @@ dependencies = [
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "android-ffi"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"common",
|
||||
"jni 0.21.1",
|
||||
"tokio",
|
||||
"viewer-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "anyhow"
|
||||
version = "1.0.104"
|
||||
@@ -69,27 +53,6 @@ dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bitfields"
|
||||
version = "1.0.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef6e59298da389bc0649c7463856b34c6e17fe542f88939426ede4436c6b1195"
|
||||
dependencies = [
|
||||
"bitfields-impl",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bitfields-impl"
|
||||
version = "1.0.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f2c044f98f86f15414668d6c8187c7e4fadab1ad2b31680f648703e0fe07c555"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
"thiserror 2.0.21",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bitflags"
|
||||
version = "1.3.2"
|
||||
@@ -132,12 +95,6 @@ dependencies = [
|
||||
"shlex",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cesu8"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6d43a04d8753f35258c91f8ec639f792891f748a1edbd759cf1dcea3382ad83c"
|
||||
|
||||
[[package]]
|
||||
name = "cfg-if"
|
||||
version = "1.0.5"
|
||||
@@ -404,22 +361,6 @@ dependencies = [
|
||||
"web-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jni"
|
||||
version = "0.21.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1a87aa2bb7d2af34197c04845522473242e1aa17c12f4935d5856491a7fb8c97"
|
||||
dependencies = [
|
||||
"cesu8",
|
||||
"cfg-if",
|
||||
"combine",
|
||||
"jni-sys 0.3.1",
|
||||
"log",
|
||||
"thiserror 1.0.69",
|
||||
"walkdir",
|
||||
"windows-sys 0.45.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jni"
|
||||
version = "0.22.4"
|
||||
@@ -429,10 +370,10 @@ dependencies = [
|
||||
"cfg-if",
|
||||
"combine",
|
||||
"jni-macros",
|
||||
"jni-sys 0.4.1",
|
||||
"jni-sys",
|
||||
"log",
|
||||
"simd_cesu8",
|
||||
"thiserror 2.0.21",
|
||||
"thiserror",
|
||||
"walkdir",
|
||||
"windows-link",
|
||||
]
|
||||
@@ -450,15 +391,6 @@ dependencies = [
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jni-sys"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "41a652e1f9b6e0275df1f15b32661cf0d4b78d4d87ddec5e0c3c20f097433258"
|
||||
dependencies = [
|
||||
"jni-sys 0.4.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jni-sys"
|
||||
version = "0.4.1"
|
||||
@@ -667,15 +599,6 @@ version = "0.2.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
|
||||
|
||||
[[package]]
|
||||
name = "nvenc"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"bitfields",
|
||||
"libloading",
|
||||
"windows",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -806,7 +729,7 @@ dependencies = [
|
||||
"rustc-hash",
|
||||
"rustls",
|
||||
"socket2",
|
||||
"thiserror 2.0.21",
|
||||
"thiserror",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"web-time",
|
||||
@@ -830,7 +753,7 @@ dependencies = [
|
||||
"rustls-pki-types",
|
||||
"rustls-platform-verifier",
|
||||
"slab",
|
||||
"thiserror 2.0.21",
|
||||
"thiserror",
|
||||
"tinyvec",
|
||||
"tracing",
|
||||
"web-time",
|
||||
@@ -1015,7 +938,7 @@ checksum = "1167586491e2b18b8bfbb293e8180ec17c201c4f076d7cb3070ca964e7598f98"
|
||||
dependencies = [
|
||||
"core-foundation",
|
||||
"core-foundation-sys",
|
||||
"jni 0.22.4",
|
||||
"jni",
|
||||
"log",
|
||||
"once_cell",
|
||||
"rustls",
|
||||
@@ -1273,33 +1196,13 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "thiserror"
|
||||
version = "1.0.69"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52"
|
||||
dependencies = [
|
||||
"thiserror-impl 1.0.69",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "thiserror"
|
||||
version = "2.0.21"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
|
||||
dependencies = [
|
||||
"thiserror-impl 2.0.21",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "thiserror-impl"
|
||||
version = "1.0.69"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
"thiserror-impl",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1424,18 +1327,6 @@ dependencies = [
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "viewer-core"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"common",
|
||||
"openh264",
|
||||
"quinn",
|
||||
"rustls",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "walkdir"
|
||||
version = "2.5.0"
|
||||
@@ -1653,54 +1544,32 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windows"
|
||||
version = "0.62.2"
|
||||
version = "0.58.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
|
||||
dependencies = [
|
||||
"windows-collections",
|
||||
"windows-core",
|
||||
"windows-future",
|
||||
"windows-numerics",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-collections"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
|
||||
checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-targets",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-core"
|
||||
version = "0.62.2"
|
||||
version = "0.58.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
|
||||
checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
|
||||
dependencies = [
|
||||
"windows-implement",
|
||||
"windows-interface",
|
||||
"windows-link",
|
||||
"windows-result",
|
||||
"windows-strings",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-future"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
"windows-threading",
|
||||
"windows-targets",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-implement"
|
||||
version = "0.60.2"
|
||||
version = "0.58.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
|
||||
checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -1709,9 +1578,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windows-interface"
|
||||
version = "0.59.3"
|
||||
version = "0.58.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
|
||||
checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -1724,41 +1593,23 @@ version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "windows-numerics"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-result"
|
||||
version = "0.4.1"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
|
||||
checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
"windows-targets",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-strings"
|
||||
version = "0.5.1"
|
||||
version = "0.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
|
||||
checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.45.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "75283be5efb2831d37ea142365f009c02ec203cd29a3ebecbc093d52315b66d0"
|
||||
dependencies = [
|
||||
"windows-targets 0.42.2",
|
||||
"windows-result",
|
||||
"windows-targets",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1767,7 +1618,7 @@ version = "0.52.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
|
||||
dependencies = [
|
||||
"windows-targets 0.52.6",
|
||||
"windows-targets",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1779,76 +1630,34 @@ dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-targets"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8e5180c00cd44c9b1c88adb3693291f1cd93605ded80c250a75d472756b4d071"
|
||||
dependencies = [
|
||||
"windows_aarch64_gnullvm 0.42.2",
|
||||
"windows_aarch64_msvc 0.42.2",
|
||||
"windows_i686_gnu 0.42.2",
|
||||
"windows_i686_msvc 0.42.2",
|
||||
"windows_x86_64_gnu 0.42.2",
|
||||
"windows_x86_64_gnullvm 0.42.2",
|
||||
"windows_x86_64_msvc 0.42.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-targets"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
|
||||
dependencies = [
|
||||
"windows_aarch64_gnullvm 0.52.6",
|
||||
"windows_aarch64_msvc 0.52.6",
|
||||
"windows_i686_gnu 0.52.6",
|
||||
"windows_aarch64_gnullvm",
|
||||
"windows_aarch64_msvc",
|
||||
"windows_i686_gnu",
|
||||
"windows_i686_gnullvm",
|
||||
"windows_i686_msvc 0.52.6",
|
||||
"windows_x86_64_gnu 0.52.6",
|
||||
"windows_x86_64_gnullvm 0.52.6",
|
||||
"windows_x86_64_msvc 0.52.6",
|
||||
"windows_i686_msvc",
|
||||
"windows_x86_64_gnu",
|
||||
"windows_x86_64_gnullvm",
|
||||
"windows_x86_64_msvc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-threading"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_gnullvm"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_gnullvm"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_msvc"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_msvc"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnu"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnu"
|
||||
version = "0.52.6"
|
||||
@@ -1861,48 +1670,24 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_msvc"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_msvc"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnu"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnu"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnullvm"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnullvm"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_msvc"
|
||||
version = "0.42.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_msvc"
|
||||
version = "0.52.6"
|
||||
|
||||
+4
-7
@@ -1,11 +1,8 @@
|
||||
[workspace]
|
||||
resolver = "2"
|
||||
# agent-core：平台无关会话层 + 软件编码视频管线（当前仅 linux-agent 使用）
|
||||
# linux-agent：Linux 被控端（X11 + XTest）
|
||||
# viewer-core：安卓主控的连接/协议/收帧层（桌面 headless 可验证；Android 经 JNI 复用）
|
||||
# android-ffi：viewer-core 的 JNI 薄层（cdylib，交叉编译成 libandroidffi.so）
|
||||
members = ["common", "agent-core", "agent", "linux-agent", "viewer", "viewer-core", "android-ffi"]
|
||||
# 默认成员保持 Windows 三件套，`cargo build --release` 行为与改动前完全一致；
|
||||
# agent-core：平台无关的被控端会话层（Windows/Linux 共用）；linux-agent：Linux 被控端
|
||||
members = ["common", "agent-core", "agent", "linux-agent", "viewer"]
|
||||
# 默认成员保持 Windows 三件套，`cargo build --release` 依旧只出 agent.exe / viewer.exe；
|
||||
# Linux 被控端显式构建：`cargo build --release -p linux-agent`
|
||||
default-members = ["common", "agent", "viewer"]
|
||||
|
||||
@@ -18,5 +15,5 @@ quinn = "0.11"
|
||||
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "logging"] }
|
||||
rcgen = { version = "0.13", default-features = false, features = ["ring", "pem"] }
|
||||
lz4_flex = "0.11"
|
||||
windows = "0.62"
|
||||
windows = "0.58"
|
||||
minifb = "0.27"
|
||||
@@ -27,4 +27,4 @@ agent.exe --port 47800 --code 123456
|
||||
viewer.exe --addr <被控机IP> --port 47800 --code 123456
|
||||
```
|
||||
|
||||
详见《构建运行手册.md》、《Git协作规范.md》与《打洞与信令设计.md》。
|
||||
详见《构建运行手册.md》与《Git协作规范.md》。
|
||||
+26
-267
@@ -11,8 +11,8 @@
|
||||
|
||||
use anyhow::Result;
|
||||
use common::{
|
||||
encode_video_frame, h264_contains_idr, read_control, write_control, ControlMsg, InputEvent,
|
||||
MouseButton, VideoFrameHeader,
|
||||
encode_video_frame, read_control, write_control, ControlMsg, InputEvent, MouseButton,
|
||||
VideoFrameHeader,
|
||||
};
|
||||
use openh264::encoder::{
|
||||
BitRate, Encoder, EncoderConfig, FrameRate, IntraFramePeriod, RateControlMode, UsageType,
|
||||
@@ -48,21 +48,6 @@ pub trait InputInjector: Send + Sync + 'static {
|
||||
fn mouse_button(&self, b: MouseButton, down: bool);
|
||||
fn wheel(&self, dy: i32);
|
||||
fn key(&self, scancode: u16, extended: bool, down: bool);
|
||||
|
||||
/// 协议 v3 的文本注入（UTF-16 code unit）。
|
||||
///
|
||||
/// **默认实现刻意不注入任何东西，但必须留下可见痕迹**：Linux 后端本轮未实现
|
||||
/// 文本注入（X11 需要 keysym/XKB 映射，与 Windows 的 `KEYEVENTF_UNICODE`
|
||||
/// 不同构，另立循环）。全局规则要求异常路径有可见后果——用户对着 Linux 被控端
|
||||
/// 打字时「什么都没发生」不能是静默的，日志必须说明是没实现。
|
||||
///
|
||||
/// 日志纪律：只打印长度，**不打印用户输入内容**。
|
||||
fn text(&self, units: &[u16]) {
|
||||
eprintln!(
|
||||
"[agent-core] 文本注入未实现（忽略 {} 个 UTF-16 code unit）",
|
||||
units.len()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// 把一条协议输入事件分发到注入后端。
|
||||
@@ -74,17 +59,6 @@ pub fn dispatch_input(inj: &dyn InputInjector, ev: InputEvent) {
|
||||
InputEvent::Wheel { dy } => inj.wheel(dy),
|
||||
InputEvent::KeyDown { scancode, extended } => inj.key(scancode, extended, true),
|
||||
InputEvent::KeyUp { scancode, extended } => inj.key(scancode, extended, false),
|
||||
InputEvent::Text { units } => {
|
||||
// 上限在前端截断，截断必须可见（否则用户以为长文本已送达）。
|
||||
let (units, truncated) = common::clamp_text_units(units);
|
||||
if truncated {
|
||||
eprintln!(
|
||||
"[agent-core] 文本超过上限，已截断到 {} 个 code unit",
|
||||
common::MAX_TEXT_UNITS
|
||||
);
|
||||
}
|
||||
inj.text(&units);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -105,14 +79,11 @@ pub struct EncoderSettings {
|
||||
|
||||
impl Default for EncoderSettings {
|
||||
fn default() -> Self {
|
||||
// 关键帧周期：以**时间**为准而非固定帧数，避免帧率变化后恢复窗口漂移。
|
||||
// 1 秒 @30fps = 30 帧。主恢复路径是 KeyFrameReq（协议 v2 后已确认生效），
|
||||
// 自动 IDR 只是兜底；此前 300 帧 @30fps 意味着兜底窗口长达 10 秒。
|
||||
let max_fps = 30.0f32;
|
||||
// 对标 UU：屏幕内容实时档；关键帧周期 300 帧 ≈ 10s
|
||||
Self {
|
||||
bitrate_bps: 5_000_000,
|
||||
max_fps,
|
||||
intra_period: (max_fps as u32).max(1),
|
||||
max_fps: 30.0,
|
||||
intra_period: 300,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -154,30 +125,23 @@ enum LoopEnd {
|
||||
}
|
||||
|
||||
/// 编码一帧并推入下行通道。返回 false 表示通道已关闭（会话结束）。
|
||||
///
|
||||
/// `seq` 由调用方持有并自增：帧序号必须覆盖**每一次实际发送**（含静止桌面重发），
|
||||
/// 不能由编码器计数（编码器只知道编码了多少帧，不知道发了多少帧）。
|
||||
fn encode_send(
|
||||
encoder: &mut Encoder,
|
||||
raw: &[u8],
|
||||
(w, h): (u32, u32),
|
||||
ts_ms: u64,
|
||||
seq: u64,
|
||||
tx: &watch::Sender<Option<Vec<u8>>>,
|
||||
last_frame: &mut Option<Vec<u8>>,
|
||||
) -> bool {
|
||||
let yuv = YUVBuffer::from_bgra8_source(BgraSliceU8::new(raw, (w as usize, h as usize)));
|
||||
match encoder.encode(&yuv) {
|
||||
Ok(ebs) => {
|
||||
let payload = ebs.to_vec();
|
||||
let header = VideoFrameHeader {
|
||||
width: w,
|
||||
height: h,
|
||||
ts_ms,
|
||||
seq,
|
||||
keyframe: h264_contains_idr(&payload),
|
||||
};
|
||||
let frame = encode_video_frame(&header, &payload);
|
||||
let yuv = YUVBuffer::from_bgra8_source(BgraSliceU8::new(raw, (w as usize, h as usize)));
|
||||
match encoder.encode(&yuv) {
|
||||
Ok(ebs) => {
|
||||
let frame = encode_video_frame(&header, &ebs.to_vec());
|
||||
if tx.send(Some(frame.clone())).is_err() {
|
||||
return false;
|
||||
}
|
||||
@@ -214,17 +178,8 @@ fn capture_session(
|
||||
// 保留最近一帧裸像素：静止桌面收到关键帧请求时，仍能重编一个真关键帧
|
||||
let mut last_raw: Option<Vec<u8>> = None;
|
||||
let mut last_sent = Instant::now() - Duration::from_secs(10);
|
||||
// 帧序号：覆盖每一次实际发送（含静止桌面重发），供主控端做丢帧检测
|
||||
let mut seq: u64 = 0;
|
||||
|
||||
loop {
|
||||
// 会话结束守卫：发送端已 drop 接收器时立即退出。
|
||||
// 与 agent/src/capture.rs 同源——静止桌面分支不发送，只靠 send 失败退出
|
||||
// 会让线程在画面不动时永不退出（X11 无 DXGI 式独占，危害较小但同样泄漏线程）。
|
||||
if tx.receiver_count() == 0 {
|
||||
eprintln!("[capture] no receiver, stop");
|
||||
return LoopEnd::ChannelClosed;
|
||||
}
|
||||
let iter_start = Instant::now();
|
||||
match cap.acquire(80) {
|
||||
Ok(true) => match cap.copy_frame() {
|
||||
@@ -234,10 +189,9 @@ fn capture_session(
|
||||
encoder.force_intra_frame();
|
||||
}
|
||||
let ts = start.elapsed().as_millis() as u64;
|
||||
if !encode_send(&mut encoder, &raw, (w, h), ts, seq, tx, &mut last_frame) {
|
||||
if !encode_send(&mut encoder, &raw, (w, h), ts, tx, &mut last_frame) {
|
||||
return LoopEnd::ChannelClosed; // 会话结束
|
||||
}
|
||||
seq = seq.wrapping_add(1);
|
||||
last_raw = Some(raw);
|
||||
last_sent = Instant::now();
|
||||
// 帧间隔节流到 max_fps
|
||||
@@ -252,56 +206,24 @@ fn capture_session(
|
||||
}
|
||||
},
|
||||
Ok(false) => {
|
||||
// 静止桌面：先看关键帧请求，再决定是否走 500ms 保活节流。
|
||||
//
|
||||
// ⚠️ 这里必须保证「消费了标志就一定发送」：先前写法是
|
||||
// `if force && last_raw.is_some() { ... }`，当 force 为真但
|
||||
// last_raw 还是 None（刚建会话、尚未抓到任何一帧）时，标志被
|
||||
// swap 消费掉却没有发送，解码端的这次请求就永久丢失。
|
||||
let force = keyreq.swap(false, Ordering::Relaxed);
|
||||
let can_key = last_raw.is_some();
|
||||
if force && !can_key {
|
||||
// 还没有裸帧可重编：把请求还回去，让下一轮（拿到首帧后）继续生效
|
||||
keyreq.store(true, Ordering::Relaxed);
|
||||
eprintln!("[capture] KeyFrameReq 暂存：尚无裸帧可重编");
|
||||
std::thread::sleep(Duration::from_millis(8));
|
||||
} else if force {
|
||||
if force && last_raw.is_some() {
|
||||
// 静止桌面下的关键帧恢复：画面没变不代表可以不给关键帧——
|
||||
// 解码端一旦出错，若只会收到旧 P 帧就永远恢复不了。
|
||||
// 关键帧请求绕过 500ms 节流：它是错误恢复路径，延迟一次 IDR
|
||||
// 的代价是持续黑屏，远高于多发一帧的带宽成本。
|
||||
// 解码端一旦出错，若只会收到旧 P 帧就永远恢复不了
|
||||
encoder.force_intra_frame();
|
||||
let raw = last_raw.as_ref().expect("can_key 已保证");
|
||||
let raw = last_raw.as_ref().expect("上面已判断 is_some");
|
||||
let ts = start.elapsed().as_millis() as u64;
|
||||
if !encode_send(&mut encoder, raw, (w, h), ts, seq, tx, &mut last_frame) {
|
||||
if !encode_send(&mut encoder, raw, (w, h), ts, tx, &mut last_frame) {
|
||||
return LoopEnd::ChannelClosed;
|
||||
}
|
||||
seq = seq.wrapping_add(1);
|
||||
last_sent = Instant::now();
|
||||
} else if last_sent.elapsed() > Duration::from_millis(500) && last_frame.is_some() {
|
||||
// 桌面静止：低频重发最后一帧（重复 P 帧，解码端画面不变）。
|
||||
// 帧头必须重建：直接 clone 旧帧会让 seq/ts_ms 停在上次发送时刻，
|
||||
// 破坏 seq 单调递增契约。
|
||||
// 桌面静止：低频重发最后一帧（重复 P 帧，解码端画面不变）
|
||||
let f = last_frame.as_ref().expect("上面已判断 is_some");
|
||||
match common::split_video_frame(f) {
|
||||
Ok((mut hdr, payload)) => {
|
||||
hdr.seq = seq;
|
||||
hdr.ts_ms = start.elapsed().as_millis() as u64;
|
||||
let frame = encode_video_frame(&hdr, payload);
|
||||
if tx.send(Some(frame)).is_err() {
|
||||
if tx.send(Some(f.clone())).is_err() {
|
||||
return LoopEnd::ChannelClosed;
|
||||
}
|
||||
seq = seq.wrapping_add(1);
|
||||
last_sent = Instant::now();
|
||||
}
|
||||
Err(e) => {
|
||||
// 自己产出的帧解析失败 = 编码/封装路径有 bug。
|
||||
// 丢弃缓存而不是每轮重复报错：静止桌面下这里会以 ~125Hz 触发。
|
||||
eprintln!("[capture] 静止重发时帧格式异常，丢弃缓存: {e}");
|
||||
last_frame = None;
|
||||
std::thread::sleep(Duration::from_millis(8));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
std::thread::sleep(Duration::from_millis(8));
|
||||
}
|
||||
@@ -369,13 +291,12 @@ where
|
||||
|
||||
// 传输参数调优：
|
||||
// - 收发窗口放大，避免高 RTT 下吞吐被 窗口/RTT 死锁（默认 512KB 流窗口）
|
||||
// - 发送缓冲上限 512KB：QUIC 内部排队 ≈ 3 帧 @5Mbps，E2E 延迟天花板 ~150ms；
|
||||
// 配合上游 latest-wins 丢帧，超量部分在进 QUIC 前就被丢弃
|
||||
// （与 Windows 端 agent/src/main.rs 保持一致，沿用同一次调优结论）
|
||||
// - 发送缓冲上限 2MB：QUIC 内部排队有界（约 1 帧多），配合上游 latest-wins 丢帧
|
||||
// 把端到端排队延迟压在 ~0.4s 内
|
||||
let mut transport = quinn::TransportConfig::default();
|
||||
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||
transport.send_window(512 * 1024);
|
||||
transport.send_window(2 * 1024 * 1024);
|
||||
let mut server_config = quinn::ServerConfig::with_single_cert(vec![cert], key.into())?;
|
||||
server_config.transport_config(Arc::new(transport));
|
||||
|
||||
@@ -396,46 +317,20 @@ where
|
||||
let (mut send, mut recv) = conn.accept_bi().await?;
|
||||
|
||||
let first = read_control(&mut recv).await?;
|
||||
// 先校验版本，再校验验证码：版本不一致时字段布局可能不同，验证码的解析结果不可信
|
||||
let auth_err: Option<String> = match &first {
|
||||
ControlMsg::Auth {
|
||||
code: c,
|
||||
proto_version,
|
||||
} => {
|
||||
if *proto_version != common::PROTO_VERSION {
|
||||
Some(format!(
|
||||
"protocol version mismatch: client={proto_version}, server={}",
|
||||
common::PROTO_VERSION
|
||||
))
|
||||
} else if *c != cfg.code {
|
||||
Some("bad code".into())
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
other => Some(format!("expected Auth, got {other:?}")),
|
||||
};
|
||||
if let Some(msg) = auth_err {
|
||||
write_control(&mut send, &ControlMsg::AuthFail(msg.clone())).await?;
|
||||
let code_ok = matches!(&first, ControlMsg::Auth { code: c } if *c == cfg.code);
|
||||
if !code_ok {
|
||||
write_control(&mut send, &ControlMsg::AuthFail("bad code".into())).await?;
|
||||
send.finish()?;
|
||||
// 必须等对端读到 AuthFail 并挂断再退出：直接 return 会让进程带走 QUIC
|
||||
// 缓冲区里尚未发出的数据，客户端只会看到 "connection lost" 而不是 AuthFail
|
||||
let _ = tokio::time::timeout(Duration::from_secs(5), conn.closed()).await;
|
||||
anyhow::bail!(msg);
|
||||
anyhow::bail!("auth failed");
|
||||
}
|
||||
|
||||
// 采集后端先建一次：既拿桌面尺寸回给主控端，也避免白建一个 duplication 只为读尺寸
|
||||
let first_cap = capture_factory()?;
|
||||
let (w, h) = first_cap.size();
|
||||
write_control(
|
||||
&mut send,
|
||||
&ControlMsg::AuthOk {
|
||||
width: w,
|
||||
height: h,
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
write_control(&mut send, &ControlMsg::AuthOk { width: w, height: h }).await?;
|
||||
println!("[agent] auth ok, desktop {w}x{h}");
|
||||
|
||||
// 流 2：键鼠输入 + 控制上行（与视频分离，避免被视频积压阻塞）
|
||||
@@ -454,10 +349,8 @@ where
|
||||
let (ctl_tx, mut ctl_rx) = mpsc::unbounded_channel::<ControlMsg>();
|
||||
let t0 = Instant::now();
|
||||
|
||||
// 读循环（流 2）：输入事件 + Ping；Pong 附带**视频下行方向**的 QUIC 丢包统计
|
||||
// （conn.stats().path 统计的是本端发送方向，被控端的发送方向正是视频下行）
|
||||
// 读循环（流 2）：输入事件 + Ping
|
||||
let keyreq_reader = keyreq.clone();
|
||||
let conn_stats = conn.clone();
|
||||
let reader = tokio::spawn(async move {
|
||||
loop {
|
||||
match read_control(&mut irecv).await {
|
||||
@@ -467,13 +360,7 @@ where
|
||||
}
|
||||
Ok(ControlMsg::Ping(t)) => {
|
||||
let agent_ts = t0.elapsed().as_millis() as u64;
|
||||
let p = conn_stats.stats().path;
|
||||
let _ = ctl_tx.send(ControlMsg::Pong {
|
||||
t,
|
||||
agent_ts,
|
||||
lost: p.lost_packets,
|
||||
sent: p.sent_packets,
|
||||
});
|
||||
let _ = ctl_tx.send(ControlMsg::Pong { t, agent_ts });
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(_) => break,
|
||||
@@ -509,131 +396,3 @@ where
|
||||
println!("[agent] session closed");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::sync::Mutex;
|
||||
|
||||
/// 记录调用轨迹的注入后端：验证分发与截断，不碰任何真实输入。
|
||||
#[derive(Default)]
|
||||
struct MockInjector {
|
||||
calls: Mutex<Vec<String>>,
|
||||
}
|
||||
|
||||
impl MockInjector {
|
||||
fn take(&self) -> Vec<String> {
|
||||
let mut g = self.calls.lock().unwrap();
|
||||
std::mem::take(&mut g)
|
||||
}
|
||||
}
|
||||
|
||||
impl InputInjector for MockInjector {
|
||||
fn mouse_move(&self, x: u16, y: u16) {
|
||||
self.calls.lock().unwrap().push(format!("move {x},{y}"));
|
||||
}
|
||||
fn mouse_button(&self, b: MouseButton, down: bool) {
|
||||
self.calls.lock().unwrap().push(format!("btn {b:?} {down}"));
|
||||
}
|
||||
fn wheel(&self, dy: i32) {
|
||||
self.calls.lock().unwrap().push(format!("wheel {dy}"));
|
||||
}
|
||||
fn key(&self, scancode: u16, extended: bool, down: bool) {
|
||||
self.calls
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push(format!("key {scancode:#x} ext={extended} {down}"));
|
||||
}
|
||||
// 刻意**不**实现 text()：让默认实现进入测试面，验证「后端没做文本注入
|
||||
// 时也必须被 dispatch_input 正常调用、且不 panic」——Linux 端正是这个形态。
|
||||
}
|
||||
|
||||
/// 实现了 text() 的后端，用来验证截断后的长度真的传给了注入侧。
|
||||
struct TextCapableInjector {
|
||||
received: Mutex<Vec<usize>>,
|
||||
}
|
||||
|
||||
impl InputInjector for TextCapableInjector {
|
||||
fn mouse_move(&self, _x: u16, _y: u16) {}
|
||||
fn mouse_button(&self, _b: MouseButton, _down: bool) {}
|
||||
fn wheel(&self, _dy: i32) {}
|
||||
fn key(&self, _scancode: u16, _extended: bool, _down: bool) {}
|
||||
fn text(&self, units: &[u16]) {
|
||||
self.received.lock().unwrap().push(units.len());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dispatch_routes_each_event_kind() {
|
||||
let inj = MockInjector::default();
|
||||
dispatch_input(&inj, InputEvent::MouseMove { x: 1, y: 2 });
|
||||
dispatch_input(&inj, InputEvent::MouseDown(MouseButton::Right));
|
||||
dispatch_input(&inj, InputEvent::Wheel { dy: -120 });
|
||||
dispatch_input(
|
||||
&inj,
|
||||
InputEvent::KeyDown {
|
||||
scancode: 0x48,
|
||||
extended: true,
|
||||
},
|
||||
);
|
||||
let calls = inj.take();
|
||||
assert_eq!(
|
||||
calls,
|
||||
vec![
|
||||
"move 1,2",
|
||||
"btn Right true",
|
||||
"wheel -120",
|
||||
"key 0x48 ext=true true",
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
/// v3 关键回归：`Text` 必须被分发，且**后端没有实现 text() 时不能 panic**。
|
||||
/// 这是 Linux/X11 后端的真实形态——用户对着 Linux 被控端打字会走到这里。
|
||||
#[test]
|
||||
fn text_event_reaches_default_backend_without_panic() {
|
||||
let inj = MockInjector::default();
|
||||
dispatch_input(
|
||||
&inj,
|
||||
InputEvent::Text {
|
||||
units: vec![0x4F60, 0x597D],
|
||||
},
|
||||
);
|
||||
// 默认实现只记日志，不产生调用记录；此处断言"没炸"即为通过。
|
||||
assert!(inj.take().is_empty());
|
||||
}
|
||||
|
||||
/// 截断契约：超过 `MAX_TEXT_UNITS` 的输入必须以截断后的长度到达注入侧。
|
||||
/// 不截断会让长粘贴把远端注入队列刷爆；截断不告知则是静默丢数据。
|
||||
#[test]
|
||||
fn text_is_clamped_to_protocol_cap() {
|
||||
let inj = TextCapableInjector {
|
||||
received: Mutex::new(vec![]),
|
||||
};
|
||||
dispatch_input(
|
||||
&inj,
|
||||
InputEvent::Text {
|
||||
units: vec![0x61; common::MAX_TEXT_UNITS + 500],
|
||||
},
|
||||
);
|
||||
dispatch_input(
|
||||
&inj,
|
||||
InputEvent::Text {
|
||||
units: vec![0x62; 10],
|
||||
},
|
||||
);
|
||||
let got = inj.received.lock().unwrap().clone();
|
||||
assert_eq!(got, vec![common::MAX_TEXT_UNITS, 10], "超限截断、未超限原样");
|
||||
}
|
||||
|
||||
/// 空 Text 仍会调用后端（是否忽略由后端决定）：主控侧 `send_text` 已拦掉空输入，
|
||||
/// 但被控端不能假设对端守规矩——线格式输入一律按不可信处理。
|
||||
#[test]
|
||||
fn empty_text_still_dispatches() {
|
||||
let inj = TextCapableInjector {
|
||||
received: Mutex::new(vec![]),
|
||||
};
|
||||
dispatch_input(&inj, InputEvent::Text { units: vec![] });
|
||||
assert_eq!(inj.received.lock().unwrap().clone(), vec![0]);
|
||||
}
|
||||
}
|
||||
+2
-9
@@ -4,18 +4,11 @@ version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[dependencies]
|
||||
# 会话层（QUIC/TLS/认证/编码/丢帧）在 agent-core，与 linux-agent 共用
|
||||
agent-core = { path = "../agent-core" }
|
||||
common = { path = "../common" }
|
||||
anyhow.workspace = true
|
||||
tokio.workspace = true
|
||||
quinn.workspace = true
|
||||
rustls.workspace = true
|
||||
rcgen.workspace = true
|
||||
lz4_flex.workspace = true
|
||||
openh264 = { version = "0.9.9", features = ["source"] }
|
||||
# 上游 nvenc 0.1.0 的 encode_picture 只设 picture_type=IDR、不置 FORCEIDR 标志，
|
||||
# 导致运行期 KeyFrameReq 被编码器静默忽略（实测输出 NAL type=1 而非 5）。
|
||||
# vendor 版本补上了该标志，见 vendor/nvenc/src/safe/encoder.rs 的「本项目补丁」注释。
|
||||
nvenc = { path = "../vendor/nvenc" }
|
||||
windows = { workspace = true, features = [
|
||||
"Win32_Foundation",
|
||||
"Win32_Graphics_Direct3D",
|
||||
|
||||
@@ -1,316 +0,0 @@
|
||||
//! 临时诊断工具：抓 agent 下发的码流，逐帧打印 NAL 结构并试解码。
|
||||
//!
|
||||
//! 目的：定位「首帧能解、之后连续 decode failed」的根因，不靠猜。
|
||||
//!
|
||||
//! 用法：
|
||||
//! cargo run --release -p agent --bin dump_stream -- --port 47805 --code t
|
||||
//! cargo run --release -p agent --bin dump_stream -- --port 47805 --code t --skip-at 2
|
||||
//!
|
||||
//! `--skip-at N`：模拟主控端的「积压跳帧」行为——第 N 帧读出但不喂给解码器，
|
||||
//! 用来验证「跳帧是否直接导致后续所有 P 帧解不出来」。
|
||||
|
||||
use anyhow::{bail, Result};
|
||||
use common::{read_control, read_video_frame, write_control, ControlMsg};
|
||||
use openh264::decoder::Decoder;
|
||||
use rustls::client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier};
|
||||
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
|
||||
use rustls::{DigitallySignedStruct, SignatureScheme};
|
||||
use std::net::ToSocketAddrs;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
#[derive(Debug)]
|
||||
struct SkipVerify;
|
||||
impl ServerCertVerifier for SkipVerify {
|
||||
fn verify_server_cert(
|
||||
&self,
|
||||
_e: &CertificateDer<'_>,
|
||||
_i: &[CertificateDer<'_>],
|
||||
_s: &ServerName<'_>,
|
||||
_o: &[u8],
|
||||
_n: UnixTime,
|
||||
) -> Result<ServerCertVerified, rustls::Error> {
|
||||
Ok(ServerCertVerified::assertion())
|
||||
}
|
||||
fn verify_tls12_signature(
|
||||
&self,
|
||||
_m: &[u8],
|
||||
_c: &CertificateDer<'_>,
|
||||
_d: &DigitallySignedStruct,
|
||||
) -> Result<HandshakeSignatureValid, rustls::Error> {
|
||||
Ok(HandshakeSignatureValid::assertion())
|
||||
}
|
||||
fn verify_tls13_signature(
|
||||
&self,
|
||||
_m: &[u8],
|
||||
_c: &CertificateDer<'_>,
|
||||
_d: &DigitallySignedStruct,
|
||||
) -> Result<HandshakeSignatureValid, rustls::Error> {
|
||||
Ok(HandshakeSignatureValid::assertion())
|
||||
}
|
||||
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
|
||||
vec![
|
||||
SignatureScheme::RSA_PKCS1_SHA256,
|
||||
SignatureScheme::ECDSA_NISTP256_SHA256,
|
||||
SignatureScheme::ED25519,
|
||||
SignatureScheme::RSA_PSS_SHA256,
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
/// 扫 Annex-B 起始码，返回 NAL 类型列表（如 [7, 8, 5]）
|
||||
fn nal_types(payload: &[u8]) -> Vec<u8> {
|
||||
let mut out = Vec::new();
|
||||
let mut i = 0usize;
|
||||
while i + 3 < payload.len() {
|
||||
let sc = if payload[i..].starts_with(&[0, 0, 0, 1]) {
|
||||
4
|
||||
} else if payload[i..].starts_with(&[0, 0, 1]) {
|
||||
3
|
||||
} else {
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
match payload.get(i + sc) {
|
||||
Some(&b) => {
|
||||
out.push(b & 0x1F);
|
||||
i += sc + 1;
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn describe(types: &[u8]) -> String {
|
||||
let has = |t: u8| types.contains(&t);
|
||||
let mut s = String::new();
|
||||
if has(7) {
|
||||
s.push_str("SPS ");
|
||||
}
|
||||
if has(8) {
|
||||
s.push_str("PPS ");
|
||||
}
|
||||
if has(5) {
|
||||
s.push_str("IDR ");
|
||||
}
|
||||
if has(1) {
|
||||
s.push_str("P ");
|
||||
}
|
||||
if has(6) {
|
||||
s.push_str("SEI ");
|
||||
}
|
||||
if s.is_empty() {
|
||||
s.push_str("(未知)");
|
||||
}
|
||||
format!("{}[{}]", s.trim(), types.len())
|
||||
}
|
||||
|
||||
fn arg(name: &str) -> Option<String> {
|
||||
let a: Vec<String> = std::env::args().collect();
|
||||
a.iter()
|
||||
.position(|x| x == name)
|
||||
.and_then(|i| a.get(i + 1))
|
||||
.cloned()
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
rustls::crypto::ring::default_provider()
|
||||
.install_default()
|
||||
.ok();
|
||||
|
||||
let port: u16 = arg("--port")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(47800);
|
||||
let code = arg("--code").unwrap_or_else(|| "123456".into());
|
||||
let skip_at: Option<usize> = arg("--skip-at").map(|s| s.parse()).transpose()?;
|
||||
let total: usize = arg("--frames")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(30);
|
||||
// --save <路径>：把**裸 H.264 码流**（各帧载荷按到达顺序直接拼接）
|
||||
// 写成 Annex-B 文件，供安卓 MediaCodec 等外部解码器验证兼容性。
|
||||
// 注意存的是载荷而非完整帧（不含自定义帧头），因为外部解码器只认标准 Annex-B。
|
||||
let save_path = arg("--save");
|
||||
|
||||
let tls = rustls::ClientConfig::builder()
|
||||
.dangerous()
|
||||
.with_custom_certificate_verifier(Arc::new(SkipVerify))
|
||||
.with_no_client_auth();
|
||||
let qc: quinn::crypto::rustls::QuicClientConfig = Arc::new(tls).try_into()?;
|
||||
let mut transport = quinn::TransportConfig::default();
|
||||
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||
let mut client_config = quinn::ClientConfig::new(Arc::new(qc));
|
||||
client_config.transport_config(Arc::new(transport));
|
||||
let mut endpoint = quinn::Endpoint::client("0.0.0.0:0".parse()?)?;
|
||||
endpoint.set_default_client_config(client_config);
|
||||
|
||||
let addr = ("127.0.0.1", port)
|
||||
.to_socket_addrs()?
|
||||
.next()
|
||||
.ok_or_else(|| anyhow::anyhow!("解析失败"))?;
|
||||
let conn = endpoint.connect(addr, "localhost")?.await?;
|
||||
println!("[dump] connected");
|
||||
|
||||
let (mut send, mut recv) = conn.open_bi().await?;
|
||||
write_control(
|
||||
&mut send,
|
||||
&ControlMsg::Auth {
|
||||
code,
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
match read_control(&mut recv).await? {
|
||||
ControlMsg::AuthOk {
|
||||
width,
|
||||
height,
|
||||
proto_version,
|
||||
} => println!("[dump] auth ok {width}x{height} proto={proto_version}"),
|
||||
other => bail!("认证失败: {other:?}"),
|
||||
}
|
||||
|
||||
let (mut isend, _irecv) = conn.open_bi().await?;
|
||||
|
||||
// 入会先请求一次 IDR，和 viewer 行为一致。
|
||||
// 注意：KeyFrameReq 必须走**流 2**（isend），agent 的读循环只监听流 2。
|
||||
write_control(&mut isend, &ControlMsg::KeyFrameReq).await?;
|
||||
|
||||
let mut decoder = Decoder::new()?;
|
||||
let mut sent_kfreq = false;
|
||||
// 累积裸码流，用于 --save
|
||||
let mut saved: Vec<u8> = Vec::new();
|
||||
|
||||
println!("\n序号 seq kf 载荷字节 NAL 结构 解码结果");
|
||||
println!("{}", "-".repeat(72));
|
||||
|
||||
for i in 0..total {
|
||||
let (hdr, payload) = match tokio::time::timeout(
|
||||
Duration::from_secs(10),
|
||||
read_video_frame(&mut recv),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(r) => r?,
|
||||
Err(_) => {
|
||||
println!("[{i}] 读帧超时（桌面可能完全静止）");
|
||||
break;
|
||||
}
|
||||
};
|
||||
|
||||
let types = nal_types(&payload);
|
||||
let is_idr = types.contains(&5);
|
||||
|
||||
// 需要在 save 里也保留这一帧时，先记录到缓冲
|
||||
if save_path.is_some() {
|
||||
saved.extend_from_slice(&payload);
|
||||
}
|
||||
|
||||
// 交叉校验：帧头 keyframe 标志必须与码流实际 NAL 类型一致。
|
||||
// 这条不一致意味着时间戳/序号契约或编码器标志有 bug，比画面花屏更难排查。
|
||||
if hdr.keyframe != is_idr {
|
||||
println!(
|
||||
" ⚠️ 帧头 keyframe={} 但码流实际 IDR={}（seq={}）",
|
||||
hdr.keyframe, is_idr, hdr.seq
|
||||
);
|
||||
}
|
||||
|
||||
// 模拟主控端积压跳帧：读出但不解码
|
||||
if Some(i) == skip_at {
|
||||
println!(
|
||||
"[{i:2}] {:3} {:<5} {:8} {:<14} ⏭ 跳过解码（模拟积压跳帧）",
|
||||
hdr.seq,
|
||||
hdr.keyframe,
|
||||
payload.len(),
|
||||
describe(&types)
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
let res = match decoder.decode(&payload) {
|
||||
Ok(Some(_)) => "✅ 解出画面".to_string(),
|
||||
Ok(None) => "○ 无输出（缓冲中）".to_string(),
|
||||
Err(e) => format!("❌ {e}"),
|
||||
};
|
||||
|
||||
println!(
|
||||
"[{i:2}] {:3} {:<5} {:8} {:<14} {}",
|
||||
hdr.seq,
|
||||
hdr.keyframe,
|
||||
payload.len(),
|
||||
describe(&types),
|
||||
res
|
||||
);
|
||||
|
||||
// 和 viewer 一样：解码失败就请求关键帧
|
||||
if res.starts_with('❌') && !sent_kfreq {
|
||||
sent_kfreq = true;
|
||||
write_control(&mut isend, &ControlMsg::KeyFrameReq).await?;
|
||||
println!(" ↑ 已发 KeyFrameReq");
|
||||
}
|
||||
}
|
||||
|
||||
// 诊断：静止桌面下再请求一次关键帧，看是否真能拿到 IDR
|
||||
println!("\n[dump] 再发一次 KeyFrameReq，观察 10 帧…");
|
||||
write_control(&mut isend, &ControlMsg::KeyFrameReq).await?;
|
||||
for i in 0..10 {
|
||||
match tokio::time::timeout(Duration::from_secs(5), read_video_frame(&mut recv)).await {
|
||||
Ok(Ok((hdr, payload))) => {
|
||||
let types = nal_types(&payload);
|
||||
println!(
|
||||
"[k{i:2}] seq={:3} kf={:<5} {:6}B {}",
|
||||
hdr.seq,
|
||||
hdr.keyframe,
|
||||
payload.len(),
|
||||
describe(&types)
|
||||
);
|
||||
if types.contains(&5) {
|
||||
println!(" ↑ 收到 IDR");
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
println!("[k{i:2}] 超时/流关闭");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 导出裸码流文件（Annex-B 拼接），供外部解码器验证
|
||||
if let Some(p) = save_path.as_ref() {
|
||||
std::fs::write(p, &saved)?;
|
||||
let idr_cnt = {
|
||||
// 粗略统计 IDR 数量：扫起始码后的 NAL 类型
|
||||
let mut n = 0;
|
||||
let mut i = 0usize;
|
||||
while i + 3 < saved.len() {
|
||||
let sc = if saved[i..].starts_with(&[0, 0, 0, 1]) {
|
||||
4
|
||||
} else if saved[i..].starts_with(&[0, 0, 1]) {
|
||||
3
|
||||
} else {
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
match saved.get(i + sc) {
|
||||
Some(&b) if b & 0x1F == 5 => {
|
||||
n += 1;
|
||||
i += sc + 1;
|
||||
}
|
||||
Some(_) => i += sc + 1,
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
n
|
||||
};
|
||||
println!(
|
||||
"\n[dump] 已保存裸码流: {p}（{} 字节，含 {idr_cnt} 个 IDR）",
|
||||
saved.len()
|
||||
);
|
||||
}
|
||||
|
||||
let _ = conn.close(0u32.into(), b"done");
|
||||
Ok(())
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
//! 编码基准：1080p 噪声帧（最恶劣压缩工况）对比单线程与 4 线程 openh264。
|
||||
//! 运行：cargo run --release -p agent --bin encbench
|
||||
|
||||
use openh264::encoder::{
|
||||
BitRate, Encoder, EncoderConfig, FrameRate, IntraFramePeriod, RateControlMode, UsageType,
|
||||
};
|
||||
use openh264::formats::{BgraSliceU8, YUVBuffer};
|
||||
use openh264::OpenH264API;
|
||||
use std::time::Instant;
|
||||
|
||||
fn bench(threads: u16, frames: usize) -> f64 {
|
||||
let mut enc = Encoder::with_api_config(
|
||||
OpenH264API::from_source(),
|
||||
EncoderConfig::new()
|
||||
.bitrate(BitRate::from_bps(5_000_000))
|
||||
.max_frame_rate(FrameRate::from_hz(30.0))
|
||||
.rate_control_mode(RateControlMode::Bitrate)
|
||||
.usage_type(UsageType::ScreenContentRealTime)
|
||||
.intra_frame_period(IntraFramePeriod::from_num_frames(300))
|
||||
.num_threads(threads),
|
||||
)
|
||||
.expect("encoder");
|
||||
|
||||
// 预生成 10 张不同噪声帧轮换（保证每帧 P 帧都真实编码，避免运动估计空转）
|
||||
let mut frames_in = Vec::with_capacity(10);
|
||||
for f in 0..10u32 {
|
||||
let mut frame = vec![0u8; 1920 * 1080 * 4];
|
||||
let mut seed = 12345u32.wrapping_add(f * 7919);
|
||||
for px in frame.chunks_exact_mut(4) {
|
||||
seed = seed.wrapping_mul(1103515245).wrapping_add(12345);
|
||||
px.copy_from_slice(&seed.to_le_bytes());
|
||||
}
|
||||
frames_in.push(YUVBuffer::from_bgra8_source(BgraSliceU8::new(&frame, (1920, 1080))));
|
||||
}
|
||||
|
||||
// 预热 5 帧（首帧 IDR、线程池初始化）
|
||||
for (i, y) in frames_in.iter().enumerate().take(5) {
|
||||
let _ = enc.encode(y).unwrap_or_else(|_| panic!("frame {i}"));
|
||||
}
|
||||
let t = Instant::now();
|
||||
for (i, y) in frames_in.iter().cycle().take(frames).enumerate() {
|
||||
let _ = enc.encode(y).unwrap_or_else(|_| panic!("frame {i}"));
|
||||
}
|
||||
t.elapsed().as_millis() as f64 / frames as f64
|
||||
}
|
||||
|
||||
fn main() {
|
||||
println!("1080p 噪声帧编码基准（含 BGRA→YUV 转换）");
|
||||
let t = Instant::now();
|
||||
let _ = YUVBuffer::from_bgra8_source(BgraSliceU8::new(
|
||||
&vec![7u8; 1920 * 1080 * 4],
|
||||
(1920, 1080),
|
||||
));
|
||||
println!("BGRA→YUV 转换: {}ms/帧", t.elapsed().as_millis());
|
||||
let s = bench(1, 20);
|
||||
println!("单线程: {s:.1}ms/帧 → {}fps", (1000.0 / s) as u32);
|
||||
let m = bench(4, 20);
|
||||
println!("4 线程: {m:.1}ms/帧 → {}fps", (1000.0 / m) as u32);
|
||||
}
|
||||
+33
-471
@@ -1,35 +1,20 @@
|
||||
//! DXGI Desktop Duplication 采集（Windows 8+）。
|
||||
//! 编码器自动选择：NVENC 硬编（N 卡：GPU 内 BGRA→H.264 零转换零拷贝直通）
|
||||
//! → openh264 软编兜底（无 N 卡/驱动异常时自动启用）。
|
||||
//! 支持主控端 KeyFrameReq 强制关键帧（解码异常恢复）。
|
||||
//! Windows 屏幕采集：DXGI Desktop Duplication（Windows 8+）。
|
||||
//!
|
||||
//! 采集循环、H.264 编码、丢帧与下行都在 `agent-core`（两端共用），本模块只负责
|
||||
//! DXGI 的「取新帧 → 拷到 staging 纹理 → 映射内存」这三步。
|
||||
//!
|
||||
//! `ScreenCapturer` 的语义与 Linux 端一致：`acquire()` 返回 false 即代表超时且画面未变，
|
||||
//! 由 agent-core 决定低频重发上一帧。
|
||||
|
||||
use agent_core::ScreenCapturer;
|
||||
use anyhow::{anyhow, Result};
|
||||
use common::{encode_video_frame, h264_contains_idr, VideoFrameHeader};
|
||||
use nvenc::bitstream::BitStream;
|
||||
use nvenc::encoder::RegisteredResource;
|
||||
use nvenc::session::{InitParams, Session};
|
||||
use nvenc::sys::enums::{
|
||||
NVencBufferFormat, NVencPicStruct, NVencPicType, NVencParamsRcMode, NVencTuningInfo,
|
||||
};
|
||||
use nvenc::sys::guids::{NV_ENC_CODEC_H264_GUID, NV_ENC_PRESET_P3_GUID};
|
||||
use openh264::encoder::{
|
||||
BitRate, Encoder, EncoderConfig, FrameRate, IntraFramePeriod, RateControlMode, UsageType,
|
||||
};
|
||||
use openh264::formats::{BgraSliceU8, YUVBuffer};
|
||||
use openh264::OpenH264API;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::watch::Sender;
|
||||
use windows::core::Interface;
|
||||
use windows::Win32::Foundation::HMODULE;
|
||||
use windows::Win32::Graphics::Direct3D::{
|
||||
D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0,
|
||||
};
|
||||
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0};
|
||||
use windows::Win32::Graphics::Direct3D11::{
|
||||
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Resource, ID3D11Texture2D,
|
||||
D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE, D3D11_CREATE_DEVICE_FLAG,
|
||||
D3D11_CPU_ACCESS_READ, D3D11_MAP_READ, D3D11_MAPPED_SUBRESOURCE,
|
||||
D3D11_SDK_VERSION, D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT, D3D11_USAGE_STAGING,
|
||||
D3D11_CREATE_DEVICE_FLAG, D3D11_CPU_ACCESS_READ, D3D11_MAP_READ,
|
||||
D3D11_MAPPED_SUBRESOURCE, D3D11_SDK_VERSION, D3D11_TEXTURE2D_DESC, D3D11_USAGE_STAGING,
|
||||
};
|
||||
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_SAMPLE_DESC};
|
||||
use windows::Win32::Graphics::Dxgi::{
|
||||
@@ -37,26 +22,10 @@ use windows::Win32::Graphics::Dxgi::{
|
||||
DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
|
||||
};
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
/// 周期 IDR 的帧间隔（兜底恢复点）。
|
||||
///
|
||||
/// 两个编码器共用同一常量，避免 NVENC 与软编的恢复窗口不一致——此前分别是
|
||||
/// 240 帧@60fps=4s 与 300 帧@60fps=5s，而 Linux 端又是 300 帧@30fps=10s。
|
||||
///
|
||||
/// 取值理由：**主恢复路径是 KeyFrameReq，不是自动 IDR**（协议 v2 后已确认请求
|
||||
/// 真正生效，见 vendor/nvenc 的 FORCEIDR 补丁）。自动 IDR 只用于兜底
|
||||
/// 「请求连续丢失」这种极端情况。60 帧 @60fps = 1 秒，既足够兜底，
|
||||
/// 又不会因过密的整帧内编码抬高码率。
|
||||
pub const INTRA_PERIOD_FRAMES: u32 = 60;
|
||||
|
||||
pub struct Duplication {
|
||||
pub device: ID3D11Device,
|
||||
_device: ID3D11Device,
|
||||
context: ID3D11DeviceContext,
|
||||
staging_res: ID3D11Resource,
|
||||
/// GPU 内中间纹理（NVENC 注册目标；DEFAULT usage，BGRA 与采集一致）
|
||||
gpu_res: ID3D11Resource,
|
||||
gpu_tex: ID3D11Texture2D,
|
||||
dup: IDXGIOutputDuplication,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
@@ -70,13 +39,13 @@ impl Duplication {
|
||||
D3D11CreateDevice(
|
||||
None,
|
||||
D3D_DRIVER_TYPE_HARDWARE,
|
||||
HMODULE(std::ptr::null_mut()),
|
||||
HMODULE::default(),
|
||||
D3D11_CREATE_DEVICE_FLAG(0),
|
||||
Some(&[D3D_FEATURE_LEVEL_11_0]),
|
||||
D3D11_SDK_VERSION,
|
||||
Some(&raw mut device),
|
||||
Some(&mut device),
|
||||
None,
|
||||
Some(&raw mut context),
|
||||
Some(&mut context),
|
||||
)
|
||||
.map_err(|e| anyhow!("D3D11CreateDevice failed: {e}"))?;
|
||||
let device = device.ok_or_else(|| anyhow!("no d3d device"))?;
|
||||
@@ -96,7 +65,6 @@ impl Duplication {
|
||||
let (width, height) = (desc.ModeDesc.Width, desc.ModeDesc.Height);
|
||||
anyhow::ensure!(width > 0 && height > 0, "bad desktop size");
|
||||
|
||||
// staging：软编路径 Map 读内存
|
||||
let sdesc = D3D11_TEXTURE2D_DESC {
|
||||
Width: width,
|
||||
Height: height,
|
||||
@@ -119,36 +87,25 @@ impl Duplication {
|
||||
let staging = staging_opt.ok_or_else(|| anyhow!("no staging texture"))?;
|
||||
let staging_res: ID3D11Resource = staging.cast()?;
|
||||
|
||||
// GPU 中间纹理：NVENC 路径的注册目标（DEFAULT usage，GPU 可读）
|
||||
let gdesc = D3D11_TEXTURE2D_DESC {
|
||||
Usage: D3D11_USAGE_DEFAULT,
|
||||
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
|
||||
CPUAccessFlags: 0,
|
||||
..sdesc
|
||||
};
|
||||
let mut gpu_opt: Option<ID3D11Texture2D> = None;
|
||||
device
|
||||
.CreateTexture2D(&gdesc, None, Some(&mut gpu_opt))
|
||||
.map_err(|e| anyhow!("CreateTexture2D(gpu) failed: {e}"))?;
|
||||
let gpu_tex = gpu_opt.ok_or_else(|| anyhow!("no gpu texture"))?;
|
||||
let gpu_res: ID3D11Resource = gpu_tex.cast()?;
|
||||
|
||||
Ok(Self {
|
||||
device,
|
||||
_device: device,
|
||||
context,
|
||||
staging_res,
|
||||
gpu_res,
|
||||
gpu_tex,
|
||||
dup,
|
||||
width,
|
||||
height,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// true=拿到新帧并已复制；false=超时无更新。
|
||||
/// 同时复制到 staging（软编读内存）与 GPU 中间纹理（NVENC 注册输入）。
|
||||
pub fn acquire(&mut self, timeout_ms: u32) -> Result<bool> {
|
||||
impl ScreenCapturer for Duplication {
|
||||
fn size(&self) -> (u32, u32) {
|
||||
(self.width, self.height)
|
||||
}
|
||||
|
||||
/// true=拿到新帧并已复制到 staging；false=超时无更新
|
||||
fn acquire(&mut self, timeout_ms: u32) -> Result<bool> {
|
||||
unsafe {
|
||||
let mut info = DXGI_OUTDUPL_FRAME_INFO::default();
|
||||
let mut resource: Option<IDXGIResource> = None;
|
||||
@@ -157,7 +114,6 @@ impl Duplication {
|
||||
let res = resource.ok_or_else(|| anyhow!("no desktop resource"))?;
|
||||
let src: ID3D11Resource = res.cast()?;
|
||||
self.context.CopyResource(&self.staging_res, &src);
|
||||
self.context.CopyResource(&self.gpu_res, &src);
|
||||
let _ = self.dup.ReleaseFrame();
|
||||
Ok(true)
|
||||
}
|
||||
@@ -167,8 +123,8 @@ impl Duplication {
|
||||
}
|
||||
}
|
||||
|
||||
/// 在 staging 映射窗口内执行回调（软编路径）。回调参数 = (像素数据, 行距)。
|
||||
pub fn with_mapped<R>(&mut self, f: impl FnOnce(&[u8], usize) -> R) -> Result<R> {
|
||||
/// 把 staging 纹理映射到内存并去掉行间距
|
||||
fn copy_frame(&mut self) -> Result<Vec<u8>> {
|
||||
unsafe {
|
||||
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
|
||||
self.context
|
||||
@@ -177,411 +133,17 @@ impl Duplication {
|
||||
let (w, h) = (self.width as usize, self.height as usize);
|
||||
let pitch = mapped.RowPitch as usize;
|
||||
let src = std::slice::from_raw_parts(mapped.pData as *const u8, pitch * h);
|
||||
let r = f(src, pitch);
|
||||
let mut out = Vec::with_capacity(w * h * 4);
|
||||
for row in 0..h {
|
||||
out.extend_from_slice(&src[row * pitch..row * pitch + w * 4]);
|
||||
}
|
||||
self.context.Unmap(&self.staging_res, 0);
|
||||
Ok(r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// NVENC 硬编封装：H.264 High、CBR 码率控制、UltraLowLatency 调优、无 B 帧。
|
||||
/// 输入 = 注册的 GPU 纹理（BGRA 与 DXGI 采集字节序一致，零转换）。
|
||||
struct NvencEncoder {
|
||||
encoder: nvenc::encoder::Encoder,
|
||||
registered: RegisteredResource,
|
||||
bitstream: BitStream,
|
||||
frame_count: usize,
|
||||
}
|
||||
|
||||
impl NvencEncoder {
|
||||
fn new(dup: &mut Duplication, bitrate_bps: u32) -> Result<Self> {
|
||||
// 打开会话（无 N 卡 / 驱动不含编码库时在此失败）
|
||||
let session: Session<_> = Session::open_dx(&dup.device)
|
||||
.map_err(|e| anyhow!("open_dx: {e:?}"))?;
|
||||
let codecs = session
|
||||
.get_encode_codecs()
|
||||
.map_err(|e| anyhow!("codecs: {e:?}"))?;
|
||||
anyhow::ensure!(codecs.contains(&NV_ENC_CODEC_H264_GUID), "NVENC no H264 codec");
|
||||
let (session, mut config) = session
|
||||
.get_encode_preset_config_ex(
|
||||
NV_ENC_CODEC_H264_GUID,
|
||||
NV_ENC_PRESET_P3_GUID,
|
||||
NVencTuningInfo::UltraLowLatency,
|
||||
)
|
||||
.map_err(|e| anyhow!("preset_config: {e:?}"))?;
|
||||
// CBR 桌面实时档：码率恒定，延迟可预测（对标 UU 3.2Mbps@60fps）
|
||||
config.preset_cfg.rc_params.rate_control_mode = NVencParamsRcMode::CBR;
|
||||
config.preset_cfg.rc_params.average_bit_rate = bitrate_bps;
|
||||
// 周期 IDR 是**兜底恢复点**，不是主恢复路径——主路径是 KeyFrameReq
|
||||
// （协议 v2 后已确认真正生效，见 vendor/nvenc 的 FORCEIDR 补丁）。
|
||||
//
|
||||
// 取值权衡：GOP 越短，丢帧后自动恢复越快，但每次 IDR 都是整帧内编码，
|
||||
// 码率开销显著（1080p 桌面 IDR 通常数倍于 P 帧）。
|
||||
// 60 帧 @60fps = 1s：既覆盖「KeyFrameReq 连续丢失」这种极端情况，
|
||||
// 又把兜底窗口压到 1 秒以内。原先 240 帧 = 4s，在请求已可靠的前提下
|
||||
// 只会让受害时长无谓地长 4 倍。
|
||||
config.preset_cfg.gop_len = INTRA_PERIOD_FRAMES;
|
||||
config.preset_cfg.frame_interval_p = 1; // 无 B 帧
|
||||
|
||||
let init = InitParams {
|
||||
encode_guid: NV_ENC_CODEC_H264_GUID,
|
||||
preset_guid: NV_ENC_PRESET_P3_GUID,
|
||||
resolution: [dup.width, dup.height],
|
||||
aspect_ratio: [dup.width, dup.height],
|
||||
frame_rate: [60, 1],
|
||||
tuning_info: NVencTuningInfo::UltraLowLatency,
|
||||
buffer_format: NVencBufferFormat::ARGB,
|
||||
encode_config: &mut config.preset_cfg,
|
||||
enable_ptd: true,
|
||||
max_encoder_resolution: [dup.width, dup.height],
|
||||
};
|
||||
let encoder = session
|
||||
.init_encoder(init)
|
||||
.map_err(|e| anyhow!("init_encoder: {e:?}"))?;
|
||||
let registered = encoder
|
||||
.register_resource_dx11(&dup.gpu_tex, NVencBufferFormat::ARGB, 0)
|
||||
.map_err(|e| anyhow!("register_resource: {e:?}"))?;
|
||||
let bitstream = encoder
|
||||
.create_bitstream_buffer()
|
||||
.map_err(|e| anyhow!("bitstream: {e:?}"))?;
|
||||
Ok(Self {
|
||||
encoder,
|
||||
registered,
|
||||
bitstream,
|
||||
frame_count: 0,
|
||||
})
|
||||
}
|
||||
|
||||
/// 编码当前 GPU 纹理内容，返回 Annex B H.264 码流。
|
||||
fn encode(&mut self, force_idr: bool, ts_ms: u64) -> Result<Vec<u8>> {
|
||||
// 首帧强制 IDR：解码端从任意时刻接入，没有 IDR 无法解出后续 P 帧
|
||||
let ty = if force_idr || self.frame_count == 0 {
|
||||
NVencPicType::IDR
|
||||
} else {
|
||||
NVencPicType::P
|
||||
};
|
||||
self.encoder.encode_picture(
|
||||
&self.registered,
|
||||
&self.bitstream,
|
||||
self.frame_count,
|
||||
ts_ms,
|
||||
NVencBufferFormat::ARGB,
|
||||
NVencPicStruct::Frame,
|
||||
ty,
|
||||
None,
|
||||
)
|
||||
.map_err(|e| anyhow!("encode_picture: {e:?}"))?;
|
||||
self.frame_count += 1;
|
||||
// 同步读出码流（NVENC 单帧编码 1-3ms，阻塞等待可接受）
|
||||
let lock = self
|
||||
.bitstream
|
||||
.try_lock(true)
|
||||
.map_err(|e| anyhow!("bitstream lock: {e:?}"))?;
|
||||
let out = lock.as_slice().to_vec();
|
||||
drop(lock);
|
||||
if out.is_empty() {
|
||||
anyhow::bail!("nvenc empty bitstream");
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn desktop_size() -> Result<(u32, u32)> {
|
||||
let d = Duplication::new()?;
|
||||
Ok((d.width, d.height))
|
||||
}
|
||||
|
||||
/// 采集主循环：独立线程运行；连接断开（通道关闭）时退出。
|
||||
/// 通道为 watch 单槽（latest-wins）：发送即覆盖，网络慢时不产生积压。
|
||||
/// congestion：发送侧 write 超时置位 → 跳过当前帧（不编码不发送），注入率自适应降速。
|
||||
pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>, congestion: Arc<AtomicBool>) {
|
||||
let start = Instant::now();
|
||||
loop {
|
||||
let mut dup = match Duplication::new() {
|
||||
Ok(d) => d,
|
||||
Err(e) => {
|
||||
eprintln!("[capture] init failed: {e}, retry in 2s");
|
||||
std::thread::sleep(Duration::from_secs(2));
|
||||
continue;
|
||||
}
|
||||
};
|
||||
eprintln!(
|
||||
"[capture] duplication started {}x{}",
|
||||
dup.width, dup.height
|
||||
);
|
||||
|
||||
// ---- 编码器自动选择：NVENC 硬编优先，失败落回 openh264 软编 ----
|
||||
enum EncoderKind {
|
||||
Nvenc(Box<NvencEncoder>),
|
||||
Soft(Box<openh264::encoder::Encoder>),
|
||||
}
|
||||
let mut encoder: EncoderKind = match NvencEncoder::new(&mut dup, 5_000_000) {
|
||||
Ok(nv) => {
|
||||
eprintln!("[capture] encoder = NVENC hardware (CBR 5Mbps@60fps, ARGB direct)");
|
||||
EncoderKind::Nvenc(Box::new(nv))
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] NVENC unavailable ({e}); fallback to openh264 software");
|
||||
match Encoder::with_api_config(
|
||||
OpenH264API::from_source(),
|
||||
EncoderConfig::new()
|
||||
.bitrate(BitRate::from_bps(5_000_000))
|
||||
.max_frame_rate(FrameRate::from_hz(60.0))
|
||||
.rate_control_mode(RateControlMode::Bitrate)
|
||||
.usage_type(UsageType::ScreenContentRealTime)
|
||||
.intra_frame_period(IntraFramePeriod::from_num_frames(
|
||||
INTRA_PERIOD_FRAMES,
|
||||
))
|
||||
.num_threads(4),
|
||||
) {
|
||||
Ok(e) => {
|
||||
eprintln!("[capture] encoder = openh264 software (5Mbps@60fps, 4T)");
|
||||
EncoderKind::Soft(Box::new(e))
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] encoder init failed: {e}");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// 最近一帧的**裸载荷**与关键帧标志：静止重发时要用它们重建帧头，
|
||||
// 不能直接重发整帧（那样 seq/ts_ms 会停在上次发送时刻）
|
||||
let mut last_payload: Option<Vec<u8>> = None;
|
||||
let mut last_kf = false;
|
||||
let mut last_sent = Instant::now() - Duration::from_secs(10);
|
||||
let min_frame = Duration::from_millis(16); // 60fps 上限
|
||||
let mut scratch: Vec<u8> = Vec::new(); // 软编：行距不连续时的复用缓冲
|
||||
// 帧序号：每发一帧 +1（含静止桌面的重复帧），供主控端做丢帧检测
|
||||
let mut seq: u64 = 0;
|
||||
// 编码耗时统计（每 120 帧输出一次均值，ms）
|
||||
let mut stat_frames = 0u64;
|
||||
let mut stat_convert_ms = 0u64;
|
||||
let mut stat_encode_ms = 0u64;
|
||||
let mut stat_skipped = 0u64; // 拥塞跳帧计数
|
||||
|
||||
// `$kf` 由调用点显式给出：NVENC 路径知道 `force_key`，软编路径同理。
|
||||
// 不要在这里猜——首帧和 KeyFrameReq 都会产生 IDR，猜错会让解码端错过重同步点。
|
||||
macro_rules! emit {
|
||||
($payload:expr, $kf:expr) => {{
|
||||
let header = VideoFrameHeader {
|
||||
width: dup.width,
|
||||
height: dup.height,
|
||||
ts_ms: start.elapsed().as_millis() as u64,
|
||||
seq,
|
||||
keyframe: $kf,
|
||||
};
|
||||
let payload_vec: Vec<u8> = $payload;
|
||||
let frame = encode_video_frame(&header, &payload_vec);
|
||||
if tx.send(Some(frame.clone())).is_err() {
|
||||
return; // 会话结束
|
||||
}
|
||||
last_payload = Some(payload_vec);
|
||||
last_kf = $kf;
|
||||
last_sent = Instant::now();
|
||||
seq = seq.wrapping_add(1);
|
||||
stat_frames += 1;
|
||||
}};
|
||||
}
|
||||
|
||||
loop {
|
||||
// 会话结束守卫：发送端已 drop 接收器时立即退出。
|
||||
// 只靠 `tx.send` 失败退出是不够的——桌面完全静止时这里根本不发送，
|
||||
// 线程会永远持有 duplication，导致下一个会话的 DuplicateOutput 失败
|
||||
// （DXGI 每个输出仅允许一个活动实例）。
|
||||
if tx.receiver_count() == 0 {
|
||||
eprintln!("[capture] no receiver, stop");
|
||||
return;
|
||||
}
|
||||
let iter_start = Instant::now();
|
||||
match dup.acquire(80) {
|
||||
Ok(true) => {
|
||||
// 拥塞反馈：发送侧排队深 → 跳过本帧（NVENC 少输入一帧，参考链连续；
|
||||
// 解码端收到的帧序 = 编码端实际输出序，无花屏风险）
|
||||
//
|
||||
// ⚠️ 必须放在消费 keyreq **之前**：先前顺序是「先 swap 再判断拥塞」，
|
||||
// 一旦本帧因拥塞被跳过，已消费的 KeyFrameReq 就随 continue 一起丢失，
|
||||
// 解码端等不到 IDR（与下方静止分支同源的缺陷）。
|
||||
if congestion.swap(false, Ordering::Relaxed) {
|
||||
stat_skipped += 1;
|
||||
let el = iter_start.elapsed();
|
||||
if el < min_frame {
|
||||
std::thread::sleep(min_frame - el);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// 主控端请求关键帧（解码异常恢复）。走到这里一定会编码，
|
||||
// 因此可以安全地消费标志。
|
||||
let force_key = keyreq.swap(false, Ordering::Relaxed);
|
||||
match &mut encoder {
|
||||
EncoderKind::Nvenc(enc) => {
|
||||
let enc_start = Instant::now();
|
||||
let r = enc.encode(force_key, start.elapsed().as_millis() as u64);
|
||||
stat_encode_ms += enc_start.elapsed().as_millis() as u64;
|
||||
match r {
|
||||
Ok(payload) => {
|
||||
let kf = h264_contains_idr(&payload);
|
||||
emit!(payload, kf)
|
||||
}
|
||||
Err(e) => eprintln!("[capture] nvenc encode failed: {e}"),
|
||||
}
|
||||
}
|
||||
EncoderKind::Soft(enc) => {
|
||||
if force_key {
|
||||
enc.force_intra_frame();
|
||||
}
|
||||
// 映射窗口内直接做 BGRA→YUV（行距连续时零拷贝，否则复用 scratch）
|
||||
let conv_start = Instant::now();
|
||||
let dims = (dup.width as usize, dup.height as usize);
|
||||
let conv = dup.with_mapped(|src, pitch| {
|
||||
if pitch == dims.0 * 4 {
|
||||
YUVBuffer::from_bgra8_source(BgraSliceU8::new(src, dims))
|
||||
} else {
|
||||
scratch.resize(dims.0 * dims.1 * 4, 0);
|
||||
for (row, dst) in
|
||||
scratch.chunks_exact_mut(dims.0 * 4).enumerate()
|
||||
{
|
||||
dst.copy_from_slice(
|
||||
&src[row * pitch..row * pitch + dims.0 * 4],
|
||||
);
|
||||
}
|
||||
YUVBuffer::from_bgra8_source(BgraSliceU8::new(&scratch, dims))
|
||||
}
|
||||
});
|
||||
match conv {
|
||||
Ok(yuv) => {
|
||||
stat_convert_ms +=
|
||||
conv_start.elapsed().as_millis() as u64;
|
||||
let enc_start = Instant::now();
|
||||
let r = enc.encode(&yuv);
|
||||
stat_encode_ms += enc_start.elapsed().as_millis() as u64;
|
||||
match r {
|
||||
Ok(ebs) => {
|
||||
let payload = ebs.to_vec();
|
||||
let kf = h264_contains_idr(&payload);
|
||||
emit!(payload, kf)
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] encode failed: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] frame convert failed: {e}");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if stat_frames > 0 && stat_frames % 120 == 0 {
|
||||
eprintln!(
|
||||
"[capture] avg convert {}ms, encode {}ms/frame, skipped {} (last 120)",
|
||||
stat_convert_ms / 120,
|
||||
stat_encode_ms / 120,
|
||||
stat_skipped
|
||||
);
|
||||
stat_convert_ms = 0;
|
||||
stat_encode_ms = 0;
|
||||
stat_skipped = 0;
|
||||
}
|
||||
// 帧间隔节流到 ~60fps
|
||||
let el = iter_start.elapsed();
|
||||
if el < min_frame {
|
||||
std::thread::sleep(min_frame - el);
|
||||
}
|
||||
}
|
||||
Ok(false) => {
|
||||
// 桌面静止：低频重发（丢包后恢复 + 会话保活）。
|
||||
//
|
||||
// ⚠️ 这里曾有三个缺陷，全部围绕「KeyFrameReq 被静默丢弃」：
|
||||
// 1. `swap` 写在时间闸**外面**：请求若落在距上次发送 500ms 之内，
|
||||
// 标志已被消费但分支不进入，该次请求直接丢失（手册 §6.1 第 3 条）。
|
||||
// 2. 软编分支只重发旧码流、不看 force_key：永远吐不出 IDR（手册 §6.1 第 2 条）。
|
||||
// 3. NVENC 分支虽然把 force_key 传给了编码器，但受第 1 条限制。
|
||||
//
|
||||
// 现在的语义：**关键帧请求绕过 500ms 节流**——它是错误恢复路径，
|
||||
// 延迟一次 IDR 的代价是解码端持续黑屏，远高于多发一帧的带宽成本。
|
||||
let force_key = keyreq.swap(false, Ordering::Relaxed);
|
||||
let due = force_key || last_sent.elapsed() > Duration::from_millis(500);
|
||||
if due {
|
||||
match &mut encoder {
|
||||
EncoderKind::Nvenc(enc) => {
|
||||
// 重编当前 GPU 帧（内容未变，输出极小；force_key 时为 IDR）
|
||||
let r = enc.encode(force_key, start.elapsed().as_millis() as u64);
|
||||
match r {
|
||||
Ok(payload) => {
|
||||
let kf = h264_contains_idr(&payload);
|
||||
emit!(payload, kf)
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] nvenc idle encode failed: {e}")
|
||||
}
|
||||
}
|
||||
}
|
||||
EncoderKind::Soft(enc) => {
|
||||
// 软编：静止桌面没有新像素可编，只能重发最近一帧。
|
||||
// 但 force_key 时必须真的重编出 IDR —— 重发旧 P 帧对
|
||||
// 已经解不动的解码端毫无帮助。
|
||||
//
|
||||
// 裸像素来源：staging 纹理里仍是上一帧内容（DXGI 只在
|
||||
// 有新帧时覆盖），重新 Map 一次即可，不必每帧额外拷 14MB。
|
||||
if force_key {
|
||||
enc.force_intra_frame();
|
||||
let dims = (dup.width as usize, dup.height as usize);
|
||||
let conv = dup.with_mapped(|src, pitch| {
|
||||
if pitch == dims.0 * 4 {
|
||||
YUVBuffer::from_bgra8_source(BgraSliceU8::new(
|
||||
src, dims,
|
||||
))
|
||||
} else {
|
||||
scratch.resize(dims.0 * dims.1 * 4, 0);
|
||||
for (row, dst) in
|
||||
scratch.chunks_exact_mut(dims.0 * 4).enumerate()
|
||||
{
|
||||
dst.copy_from_slice(
|
||||
&src[row * pitch..row * pitch + dims.0 * 4],
|
||||
);
|
||||
}
|
||||
YUVBuffer::from_bgra8_source(BgraSliceU8::new(
|
||||
&scratch, dims,
|
||||
))
|
||||
}
|
||||
});
|
||||
match conv {
|
||||
Ok(yuv) => match enc.encode(&yuv) {
|
||||
Ok(ebs) => {
|
||||
let payload = ebs.to_vec();
|
||||
let kf = h264_contains_idr(&payload);
|
||||
emit!(payload, kf)
|
||||
}
|
||||
Err(e) => eprintln!(
|
||||
"[capture] idle keyframe encode failed: {e}"
|
||||
),
|
||||
},
|
||||
Err(e) => {
|
||||
eprintln!("[capture] idle keyframe convert failed: {e}")
|
||||
}
|
||||
}
|
||||
} else if let Some(p) = last_payload.as_ref() {
|
||||
// 非关键帧的保活重发：复用载荷但重建帧头（seq 必须递增）
|
||||
let kf = last_kf;
|
||||
emit!(p.clone(), kf)
|
||||
}
|
||||
}
|
||||
}
|
||||
last_sent = Instant::now();
|
||||
} else {
|
||||
std::thread::sleep(Duration::from_millis(8));
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("[capture] acquire failed: {e}");
|
||||
break; // ACCESS_LOST 等，重建
|
||||
}
|
||||
}
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(300));
|
||||
}
|
||||
/// 采集器工厂：agent-core 在采集后端失效时会重新调用
|
||||
pub fn new_capturer() -> Result<Box<dyn ScreenCapturer>> {
|
||||
Ok(Box::new(Duplication::new()?))
|
||||
}
|
||||
+43
-101
@@ -1,12 +1,16 @@
|
||||
//! 输入注入：SendInput 实现（鼠标归一化绝对坐标 + 扫描码键盘 + v3 文本注入）。
|
||||
//! Windows 输入注入：SendInput（鼠标虚拟桌面绝对坐标 + 扫描码键盘）。
|
||||
//!
|
||||
//! 协议里的坐标/扫描码语义两端一致，只是落地 API 不同：
|
||||
//! - 鼠标：`MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK`，直接吃 0..65535 归一化值；
|
||||
//! - 键盘：`KEYEVENTF_SCANCODE` 直接送 Set-1 扫描码（Linux 端则需换算成 X11 keycode）。
|
||||
|
||||
use common::{InputEvent, MouseButton};
|
||||
use agent_core::InputInjector;
|
||||
use common::MouseButton;
|
||||
use windows::Win32::UI::Input::KeyboardAndMouse::{
|
||||
SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY,
|
||||
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, KEYEVENTF_UNICODE, MOUSEEVENTF_ABSOLUTE,
|
||||
MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN,
|
||||
MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEINPUT, VIRTUAL_KEY,
|
||||
MOUSE_EVENT_FLAGS,
|
||||
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_MIDDLEDOWN,
|
||||
MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP,
|
||||
MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEINPUT, VIRTUAL_KEY, MOUSE_EVENT_FLAGS,
|
||||
};
|
||||
use windows::Win32::UI::WindowsAndMessaging::{
|
||||
GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN,
|
||||
@@ -74,111 +78,49 @@ pub fn virtual_desktop() -> (i32, i32, i32, i32) {
|
||||
}
|
||||
}
|
||||
|
||||
/// v3 文本注入：把一段 UTF-16 code unit 作为键盘事件送入前台应用。
|
||||
///
|
||||
/// 语义来源（实读，不凭记忆）：
|
||||
/// [KEYBDINPUT (winuser.h)](https://learn.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-keybdinput)
|
||||
/// - `KEYEVENTF_UNICODE`（0x0004）：`wVk` **必须为 0**，`wScan` 即要送出的 Unicode 字符；
|
||||
/// 系统合成 `VK_PACKET` 按键事件，经 `TranslateMessage` 转为带该字符的 `WM_CHAR`。
|
||||
/// - 文档原文是 "This flag can only be combined with the KEYEVENTF_KEYUP flag"。
|
||||
/// 本项目**仍先发不带 KEYUP 的 down**（与现有 `send_key` 的 down/up 成对形态一致），
|
||||
/// 把那句读作「不能与 SCANCODE/EXTENDEDKEY 混用」是**我的推断，文档没有明说**。
|
||||
/// 若真机验收发现 down 事件被系统丢弃（表现为一个字符进两次或不进），
|
||||
/// 退路是只发 `KEYEVENTF_UNICODE | KEYEVENTF_KEYUP` 单事件——改一处 `mk` 即可。
|
||||
///
|
||||
/// 为什么一次性批量提交（而非逐字符一对 down/up）：
|
||||
/// 代理对（U+1F600 = D83D DE00）的两个 code unit 必须在输入队列里**相邻**，
|
||||
/// 远端才会把它们合成成一个字符。逐字符成对（down,up,down,up）会在两个
|
||||
/// down 之间插入 up 事件；Windows 对 VK_PACKET 孤立代理项的缓冲规则文档
|
||||
/// 没有描述，所以这里选择让两个 down 严格相邻的形态，由真机 emoji 验收（V7）定论。
|
||||
///
|
||||
/// 为什么是**一次** `SendInput` 调用（而不是 down 批 + up 批两次）：
|
||||
/// [Keyboard Input Overview](https://learn.microsoft.com/en-us/windows/win32/inputdev/about-keyboard-input)
|
||||
/// 的 Simulating Input 一节明确——单调用注入的一批事件"系统确保不会被其他输入事件
|
||||
/// 打散"（the system ensures that no other input events intermingle with the
|
||||
/// simulated events），两次调用之间则可能插入真实键盘/鼠标事件，代理对就有被打散风险。
|
||||
/// 于是把 N 个 down + N 个 up 拼成一个数组一次提交，顺序仍是全部 down 先于全部 up。
|
||||
///
|
||||
/// 上限由调用方（`common::clamp_text_units`，4096）保证；这里对空输入直接返回，
|
||||
/// 因为 `SendInput(0)` 会返回 0（失败）且没有任何注入语义。
|
||||
fn send_text(units: &[u16]) {
|
||||
if units.is_empty() {
|
||||
return;
|
||||
}
|
||||
let mk = |scan: u16, up: bool| INPUT {
|
||||
r#type: INPUT_KEYBOARD,
|
||||
Anonymous: INPUT_0 {
|
||||
ki: KEYBDINPUT {
|
||||
wVk: VIRTUAL_KEY(0), // KEYEVENTF_UNICODE 要求 wVk=0（官方文档）
|
||||
wScan: scan,
|
||||
dwFlags: if up {
|
||||
KEYEVENTF_UNICODE | KEYEVENTF_KEYUP
|
||||
} else {
|
||||
KEYEVENTF_UNICODE
|
||||
},
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
},
|
||||
},
|
||||
};
|
||||
// 一次提交：全部 down，紧接全部 up。
|
||||
let inputs: Vec<INPUT> = units
|
||||
.iter()
|
||||
.map(|&u| mk(u, false))
|
||||
.chain(units.iter().map(|&u| mk(u, true)))
|
||||
.collect();
|
||||
let n = unsafe { SendInput(&inputs, std::mem::size_of::<INPUT>() as i32) };
|
||||
// 可见失败后果：SendInput 返回「实际插入的事件数」，不等于请求数即失败
|
||||
// （常见原因：桌面被 Secure UI 锁定、被 BlockInput 拦截）。
|
||||
if n as usize != inputs.len() {
|
||||
eprintln!("[input] SendInput 文本注入不完整: {n}/{}", inputs.len());
|
||||
/// SendInput 注入后端（无状态，直接调 Win32）
|
||||
pub struct WinInput;
|
||||
|
||||
impl WinInput {
|
||||
pub fn new() -> Self {
|
||||
Self
|
||||
}
|
||||
}
|
||||
|
||||
pub fn handle(ev: InputEvent) {
|
||||
match ev {
|
||||
InputEvent::MouseMove { x, y } => {
|
||||
// 归一化坐标系 = **被采集的那个输出**（capture 用 EnumOutputs(0)，
|
||||
// 即主屏；AuthOk 的 width/height 也是它）。
|
||||
// MOUSEEVENTF_ABSOLUTE 不带 VIRTUALDESK 时，0..=65535 恰好映射到
|
||||
// 主显示器——与采集空间一致。
|
||||
// 曾经带 VIRTUALDESK：多显示器机器上注入空间变成整个虚拟桌面
|
||||
// （比主屏宽），手机画面与指针位置系统性错位（2026-10-30 实测）。
|
||||
// 单显示器时两者重合，所以桌面端从未暴露此问题。
|
||||
impl Default for WinInput {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl InputInjector for WinInput {
|
||||
fn mouse_move(&self, x: u16, y: u16) {
|
||||
// 主控端已按虚拟桌面归一化，这里直接透传 0..65535
|
||||
send_mouse(
|
||||
MOUSEEVENTF_ABSOLUTE.0 | MOUSEEVENTF_MOVE.0,
|
||||
MOUSEEVENTF_ABSOLUTE.0 | MOUSEEVENTF_VIRTUALDESK.0 | MOUSEEVENTF_MOVE.0,
|
||||
x as i32,
|
||||
y as i32,
|
||||
0,
|
||||
);
|
||||
}
|
||||
InputEvent::MouseDown(b) => match b {
|
||||
MouseButton::Left => send_mouse(MOUSEEVENTF_LEFTDOWN, 0, 0, 0),
|
||||
MouseButton::Right => send_mouse(MOUSEEVENTF_RIGHTDOWN.0, 0, 0, 0),
|
||||
MouseButton::Middle => send_mouse(MOUSEEVENTF_MIDDLEDOWN.0, 0, 0, 0),
|
||||
},
|
||||
InputEvent::MouseUp(b) => match b {
|
||||
MouseButton::Left => send_mouse(MOUSEEVENTF_LEFTUP, 0, 0, 0),
|
||||
MouseButton::Right => send_mouse(MOUSEEVENTF_RIGHTUP.0, 0, 0, 0),
|
||||
MouseButton::Middle => send_mouse(MOUSEEVENTF_MIDDLEUP.0, 0, 0, 0),
|
||||
},
|
||||
InputEvent::Wheel { dy } => {
|
||||
|
||||
fn mouse_button(&self, b: MouseButton, down: bool) {
|
||||
let flags = match (b, down) {
|
||||
(MouseButton::Left, true) => MOUSEEVENTF_LEFTDOWN,
|
||||
(MouseButton::Left, false) => MOUSEEVENTF_LEFTUP,
|
||||
(MouseButton::Right, true) => MOUSEEVENTF_RIGHTDOWN.0,
|
||||
(MouseButton::Right, false) => MOUSEEVENTF_RIGHTUP.0,
|
||||
(MouseButton::Middle, true) => MOUSEEVENTF_MIDDLEDOWN.0,
|
||||
(MouseButton::Middle, false) => MOUSEEVENTF_MIDDLEUP.0,
|
||||
};
|
||||
send_mouse(flags, 0, 0, 0);
|
||||
}
|
||||
|
||||
fn wheel(&self, dy: i32) {
|
||||
send_mouse(MOUSEEVENTF_WHEEL.0, 0, 0, dy as u32);
|
||||
}
|
||||
InputEvent::KeyDown { scancode, extended } => send_key(scancode, extended, false),
|
||||
InputEvent::KeyUp { scancode, extended } => send_key(scancode, extended, true),
|
||||
InputEvent::Text { units } => {
|
||||
// 上限截断必须可见（与 agent-core::dispatch_input 同口径；Windows 端
|
||||
// 走自己的 handle()，不经过 agent-core，所以这里同源补齐——R-3 教训）。
|
||||
let (units, truncated) = common::clamp_text_units(units);
|
||||
if truncated {
|
||||
eprintln!(
|
||||
"[input] 文本超过上限，已截断到 {} 个 code unit",
|
||||
common::MAX_TEXT_UNITS
|
||||
);
|
||||
}
|
||||
// 只打印长度：注入日志不得含用户输入内容（安全红线）
|
||||
send_text(&units);
|
||||
}
|
||||
|
||||
fn key(&self, scancode: u16, extended: bool, down: bool) {
|
||||
send_key(scancode, extended, !down);
|
||||
}
|
||||
}
|
||||
+14
-218
@@ -1,232 +1,28 @@
|
||||
//! 被控端 Agent：DXGI 桌面复制采集 → H.264 编码 → QUIC 下行视频；上行接收输入事件注入。
|
||||
//! P0 优化：视频帧经 watch 单槽通道（latest-wins，网络慢时只发最新帧，不积压）；
|
||||
//! 键鼠/控制走独立 QUIC 双向流，不与视频互相阻塞。
|
||||
//! 会话模型：**串行多会话**——accept 循环一次只服务一个主控，会话结束后回到 accept
|
||||
//! 等下一个连接（进程不退出）。改成并发多会话需要先解决 DXGI 单实例独占，暂不做。
|
||||
//! 被控端 Agent（Windows）：DXGI 桌面复制采集 + SendInput 键鼠注入。
|
||||
//!
|
||||
//! 会话层（QUIC/TLS 握手、验证码认证、H.264 编码下行、latest-wins 丢帧、输入分发）在
|
||||
//! `agent-core`，与 `linux-agent` 共用同一份实现；本 crate 只提供平台相关的采集与注入，
|
||||
//! 避免两端各写一份握手/认证造成协议漂移。
|
||||
|
||||
mod capture;
|
||||
mod input;
|
||||
|
||||
use agent_core::{parse_args, serve, EncoderSettings, InputInjector, SessionConfig};
|
||||
use anyhow::Result;
|
||||
use common::{read_control, write_control, ControlMsg};
|
||||
use rustls::pki_types::PrivatePkcs8KeyDer;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::io::AsyncWriteExt;
|
||||
use tokio::sync::{mpsc, watch};
|
||||
|
||||
fn parse_args() -> Result<(u16, String)> {
|
||||
let mut port = 47800u16;
|
||||
let mut code = "123456".to_string();
|
||||
let a: Vec<String> = std::env::args().collect();
|
||||
let mut i = 1;
|
||||
while i < a.len() {
|
||||
match a[i].as_str() {
|
||||
"--port" => port = a[i + 1].parse()?,
|
||||
"--code" => code = a[i + 1].clone(),
|
||||
_ => {}
|
||||
}
|
||||
i += 2;
|
||||
}
|
||||
Ok((port, code))
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let (port, code) = parse_args()?;
|
||||
rustls::crypto::ring::default_provider()
|
||||
.install_default()
|
||||
.ok();
|
||||
|
||||
// 自签名证书（MVP：客户端跳过校验；正式版改为固定公钥/账号体系）
|
||||
let ck = rcgen::generate_simple_self_signed(vec!["localhost".into()])?;
|
||||
let cert = ck.cert.der().clone();
|
||||
let key = PrivatePkcs8KeyDer::from(ck.key_pair.serialize_der());
|
||||
let injector: Arc<dyn InputInjector> = Arc::new(input::WinInput::new());
|
||||
|
||||
// 传输参数调优：
|
||||
// - 收发窗口放大，避免高 RTT 下吞吐被 窗口/RTT 死锁（默认 512KB 流窗口）
|
||||
// - 发送缓冲上限 64KB：QUIC 内部排队硬上限（实测 WiFi ~3Mbps 吞吐 → E2E ≤ ~170ms；
|
||||
// 突发吞吐 = 64KB/7ms ≈ 72Mbps，不限制带宽）。注意此窗口为连接级，流 2 输入/Ping
|
||||
// 数据量极小不受影响。配合下方拥塞反馈跳帧，把排队压在数帧以内
|
||||
let mut transport = quinn::TransportConfig::default();
|
||||
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||
transport.send_window(64 * 1024);
|
||||
// 空闲超时：主控端被强杀（非正常 close）时收不到它的 Ping，连接会一直占住
|
||||
// 串行 accept，下一个连接要等几十秒才能建立。10s = 20 个 Ping 周期，
|
||||
// 正常会话（Ping 每 500ms）永不触发，只用来回收死连接。
|
||||
transport.max_idle_timeout(Some(
|
||||
Duration::from_secs(10)
|
||||
.try_into()
|
||||
.expect("10s fits in VarInt"),
|
||||
));
|
||||
let mut server_config = quinn::ServerConfig::with_single_cert(vec![cert], key.into())?;
|
||||
server_config.transport_config(Arc::new(transport));
|
||||
|
||||
let endpoint = quinn::Endpoint::server(
|
||||
server_config,
|
||||
format!("0.0.0.0:{port}").parse()?,
|
||||
)?;
|
||||
println!("[agent] listening 0.0.0.0:{port}, verify code = {code}");
|
||||
|
||||
// 串行多会话：一次只服务一个主控，结束后回到 accept。
|
||||
// 不做并发多会话——DXGI Desktop Duplication 每个输出仅允许一个活动实例，
|
||||
// 第二个 DuplicateOutput 会让第一个收到 ACCESS_LOST。
|
||||
// 会话失败（含认证失败）只记日志，不能让进程退出——否则每次连接都要重启被控端。
|
||||
loop {
|
||||
let Some(incoming) = endpoint.accept().await else {
|
||||
println!("[agent] endpoint closed, exit");
|
||||
return Ok(());
|
||||
};
|
||||
match incoming.await {
|
||||
Ok(conn) => {
|
||||
println!("[agent] peer connected: {}", conn.remote_address());
|
||||
if let Err(e) = serve_session(conn, &code).await {
|
||||
println!("[agent] session ended: {e:#}");
|
||||
}
|
||||
println!("[agent] waiting for next connection");
|
||||
}
|
||||
Err(e) => println!("[agent] handshake failed: {e}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 单次会话：认证 → 采集/发送/输入 → 会话结束。
|
||||
/// 返回 Err 表示会话异常结束（含认证失败），调用方继续 accept。
|
||||
async fn serve_session(conn: quinn::Connection, code: &str) -> Result<()> {
|
||||
// 流 1：认证 + 视频下行
|
||||
let (mut send, mut recv) = conn.accept_bi().await?;
|
||||
|
||||
// 认证：先校验版本，再校验验证码。
|
||||
// 顺序刻意如此——版本不一致时字段布局可能不同，验证码的解析结果不可信。
|
||||
let first = read_control(&mut recv).await?;
|
||||
let auth_err: Option<String> = match &first {
|
||||
ControlMsg::Auth {
|
||||
code: c,
|
||||
proto_version,
|
||||
} => {
|
||||
if *proto_version != common::PROTO_VERSION {
|
||||
Some(format!(
|
||||
"protocol version mismatch: client={proto_version}, server={}",
|
||||
common::PROTO_VERSION
|
||||
))
|
||||
} else if *c != code {
|
||||
Some("bad code".into())
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
other => Some(format!("expected Auth, got {other:?}")),
|
||||
};
|
||||
if let Some(msg) = auth_err {
|
||||
write_control(&mut send, &ControlMsg::AuthFail(msg.clone())).await?;
|
||||
send.finish()?;
|
||||
// 必须等对端读到 AuthFail 再关连接：直接返回会让进程带走 QUIC 缓冲区里
|
||||
// 尚未发出的数据，客户端只会看到 "connection lost" 而不是 AuthFail。
|
||||
// （agent-core 已有同款修复，此处同源补齐——手册 §6.1 第 1 条）
|
||||
let _ = tokio::time::timeout(Duration::from_secs(5), conn.closed()).await;
|
||||
anyhow::bail!(msg);
|
||||
}
|
||||
let (w, h) = capture::desktop_size()?;
|
||||
write_control(
|
||||
&mut send,
|
||||
&ControlMsg::AuthOk {
|
||||
width: w,
|
||||
height: h,
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
serve(
|
||||
SessionConfig { port, code },
|
||||
EncoderSettings::default(),
|
||||
// DXGI 后端失效（ACCESS_LOST 等）时 agent-core 会重新调用本工厂
|
||||
capture::new_capturer,
|
||||
injector,
|
||||
)
|
||||
.await?;
|
||||
println!("[agent] auth ok, desktop {w}x{h}");
|
||||
|
||||
// 流 2：键鼠输入 + 控制上行（与视频分离，避免被视频积压阻塞）
|
||||
let (mut isend, mut irecv) = conn.accept_bi().await?;
|
||||
|
||||
// 采集线程 → 视频帧（watch 单槽：新帧覆盖旧帧，latest-wins）
|
||||
let (tx, mut frames_rx) = watch::channel::<Option<Vec<u8>>>(None);
|
||||
let keyreq = Arc::new(AtomicBool::new(false));
|
||||
let keyreq_capture = keyreq.clone();
|
||||
// 拥塞反馈标志：发送侧 write 超过单帧预算(20ms) → 置位，采集线程跳帧降注入率
|
||||
let congestion = Arc::new(AtomicBool::new(false));
|
||||
let congestion_capture = congestion.clone();
|
||||
// 采集线程退出信号：会话结束必须等它真正退出才能 accept 下一个连接——
|
||||
// DXGI Desktop Duplication 每个输出只允许一个活动实例，旧线程还持有 duplication
|
||||
// 时新线程的 DuplicateOutput 会让旧实例收到 ACCESS_LOST（表现为下一个会话黑屏）。
|
||||
let (done_tx, done_rx) = std::sync::mpsc::channel::<()>();
|
||||
std::thread::spawn(move || {
|
||||
capture::run(tx, keyreq_capture, congestion_capture);
|
||||
let _ = done_tx.send(());
|
||||
});
|
||||
|
||||
// 上行控制消息（Pong）经通道交给主循环写流 2，避免多任务争抢
|
||||
let (ctl_tx, mut ctl_rx) = mpsc::unbounded_channel::<ControlMsg>();
|
||||
let t0 = std::time::Instant::now();
|
||||
|
||||
// 读循环（流 2）：输入事件 + Ping；Pong 附带视频下行方向的 QUIC 丢包统计
|
||||
// （conn.stats().path 统计的是本端发送方向，agent 的发送方向即视频下行）
|
||||
let keyreq_reader = keyreq.clone();
|
||||
let conn_stats = conn.clone();
|
||||
let reader = tokio::spawn(async move {
|
||||
loop {
|
||||
match read_control(&mut irecv).await {
|
||||
Ok(ControlMsg::Input(ev)) => input::handle(ev),
|
||||
Ok(ControlMsg::KeyFrameReq) => {
|
||||
keyreq_reader.store(true, Ordering::Relaxed);
|
||||
}
|
||||
Ok(ControlMsg::Ping(t)) => {
|
||||
let agent_ts = t0.elapsed().as_millis() as u64;
|
||||
let p = conn_stats.stats().path;
|
||||
let _ = ctl_tx.send(ControlMsg::Pong {
|
||||
t,
|
||||
agent_ts,
|
||||
lost: p.lost_packets,
|
||||
sent: p.sent_packets,
|
||||
});
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(_) => break,
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
changed = frames_rx.changed() => {
|
||||
if changed.is_err() {
|
||||
break; // 采集线程退出
|
||||
}
|
||||
if let Some(data) = frames_rx.borrow().clone() {
|
||||
let wt = std::time::Instant::now();
|
||||
let ok = send.write_all(&data).await.is_ok();
|
||||
// write 耗时 > 单帧预算(16.7ms 的 1.2 倍) → QUIC 窗口排队深，
|
||||
// 通知采集线程跳帧，把注入率降到网络实际吞吐水平
|
||||
if wt.elapsed() > std::time::Duration::from_millis(20) {
|
||||
congestion.store(true, Ordering::Relaxed);
|
||||
}
|
||||
if !ok {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
ctl = ctl_rx.recv() => {
|
||||
match ctl {
|
||||
Some(m) => {
|
||||
if write_control(&mut isend, &m).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
reader.abort();
|
||||
// 关掉流并丢弃接收端，让采集线程下一次 send 失败而退出
|
||||
let _ = send.finish();
|
||||
drop(frames_rx);
|
||||
// 等采集线程真正退出后再返回，调用方才会 accept 下一个连接（DXGI 独占，见上）
|
||||
let _ = tokio::task::spawn_blocking(move || done_rx.recv_timeout(Duration::from_secs(2))).await;
|
||||
println!("[agent] session closed");
|
||||
Ok(())
|
||||
.await
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
[package]
|
||||
name = "android-ffi"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# 安卓主控的 JNI 薄层：把 viewer-core 的异步 Session 包成 Kotlin 可同步调用的句柄。
|
||||
# 2026-10-29 裁决变更：原方案 A2 选 UniFFI，但其实训运行时不在 Maven Central /
|
||||
# Mozilla maven（查证 404），需自编译发布，成本超过收益。改为裸 JNI——阶段 0
|
||||
# （D:\working\android\phase0）已验证该链路真机可跑。接口刻意保持窄（连接/取帧/
|
||||
# 发输入/销毁），将来若引入 UniFFI 也只是替换本层。
|
||||
[lib]
|
||||
name = "androidffi"
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
viewer-core = { path = "../viewer-core" }
|
||||
common = { path = "../common" }
|
||||
anyhow.workspace = true
|
||||
tokio.workspace = true
|
||||
jni = "0.21"
|
||||
@@ -1,355 +0,0 @@
|
||||
//! android-ffi：viewer-core 的 JNI 薄层（阶段 1）。
|
||||
//!
|
||||
//! 线程模型（决定接口形态的原因）：
|
||||
//! - Kotlin 解码线程调用 `nativeNextFrame` 同步取帧（内部 block_on，一次一帧）；
|
||||
//! - Kotlin UI 线程调用 `nativeSend*` 发输入——tokio unbounded channel 入队不阻塞；
|
||||
//! - 共用一个 current-thread runtime：`block_on(next_frame)` 期间 runtime 仍在驱动
|
||||
//! QUIC 收发与 Ping；输入入队只是 channel send，不需要 runtime 参与。
|
||||
//!
|
||||
//! **锁纪律**：`nativeNextFrame` 在等帧期间**不得持有**会话锁——桌面静止时帧间隔
|
||||
//! 可达 500ms，持锁会让 UI 线程的输入调用同步卡住。做法：短暂持锁克隆
|
||||
//! `Arc<Session>` 与 `&'static Runtime` 引用后立即释放，再 block_on。
|
||||
//! 代价：block_on 期间 destroy 会 use-after-free——阶段 1 由 Kotlin 侧
|
||||
//! 「先停解码线程、后 destroy」的顺序纪律保证（登记在方案风险清单）。
|
||||
//!
|
||||
//! 句柄约定：`nativeConnect` 返回 Box 裸指针（永不为 0）作为 jlong；
|
||||
//! `nativeDestroy` 释放。句柄释放后不得再使用，Kotlin 侧用置 0 保护。
|
||||
//!
|
||||
//! 帧返回约定：byte[]，布局 = [1 标志][8 seq LE][8 ts LE][4 w LE][4 h LE][payload]。
|
||||
//! 定长头 25 字节；Kotlin 读 buf[0] 判断 keyframe（标志位复用为：0=P 帧, 1=IDR 帧，
|
||||
//! 会话结束则整个返回 null）。
|
||||
|
||||
use common::InputEvent;
|
||||
use jni::objects::{JCharArray, JClass, JObject, JString};
|
||||
use jni::sys::{jbyteArray, jint, jlong, jstring};
|
||||
use jni::JNIEnv;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use viewer_core::{ConnectConfig, Session};
|
||||
|
||||
/// 帧头长度：keyframe 标志(1) + seq(8) + ts(8) + w(4) + h(4)
|
||||
const FRAME_HEADER_LEN: usize = 25;
|
||||
|
||||
struct SessionBox {
|
||||
session: Arc<Session>,
|
||||
runtime: tokio::runtime::Runtime,
|
||||
}
|
||||
|
||||
/// 连接被控端。成功返回句柄（>0），失败返回 0；错误消息经 `nativeLastError()` 取。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeConnect<'local>(
|
||||
mut env: JNIEnv<'local>,
|
||||
_class: JClass<'local>,
|
||||
host: JString<'local>,
|
||||
port: jint,
|
||||
code: JString<'local>,
|
||||
) -> jlong {
|
||||
let host_s = match env.get_string(&host) {
|
||||
Ok(s) => s.to_string_lossy().into_owned(),
|
||||
Err(_) => {
|
||||
set_last_error("host 参数不是合法字符串");
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
let code_s = match env.get_string(&code) {
|
||||
Ok(s) => s.to_string_lossy().into_owned(),
|
||||
Err(_) => {
|
||||
set_last_error("code 参数不是合法字符串");
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
let built = std::panic::catch_unwind(|| {
|
||||
let runtime = match tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(e) => return Err(format!("runtime init: {e}")),
|
||||
};
|
||||
let cfg = ConnectConfig::direct(host_s, port as u16, code_s);
|
||||
match runtime.block_on(Session::connect(cfg)) {
|
||||
Ok(session) => Ok(SessionBox {
|
||||
session: Arc::new(session),
|
||||
runtime,
|
||||
}),
|
||||
Err(e) => Err(format!("{e:#}")),
|
||||
}
|
||||
});
|
||||
|
||||
match built {
|
||||
Ok(Ok(boxed)) => {
|
||||
set_last_error("");
|
||||
Box::into_raw(Box::new(Mutex::new(boxed))) as jlong
|
||||
}
|
||||
Ok(Err(msg)) => {
|
||||
set_last_error(&msg);
|
||||
0
|
||||
}
|
||||
Err(_) => {
|
||||
set_last_error("panic in nativeConnect");
|
||||
0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn set_last_error(msg: &str) {
|
||||
let mut slot = LAST_ERROR.lock().unwrap();
|
||||
if msg.is_empty() {
|
||||
*slot = None;
|
||||
} else {
|
||||
*slot = Some(msg.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
static LAST_ERROR: Mutex<Option<String>> = Mutex::new(None);
|
||||
|
||||
/// 取最近一次 nativeConnect 的错误消息（无错误返回 null）。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeLastError<'local>(
|
||||
env: JNIEnv<'local>,
|
||||
_class: JClass<'local>,
|
||||
) -> jstring {
|
||||
let msg = LAST_ERROR.lock().unwrap().take();
|
||||
match msg {
|
||||
Some(m) => env
|
||||
.new_string(m)
|
||||
.map(|s| s.into_raw())
|
||||
.unwrap_or(std::ptr::null_mut()),
|
||||
None => std::ptr::null_mut(),
|
||||
}
|
||||
}
|
||||
|
||||
/// 取下一帧（阻塞直到有帧或会话结束）。返回 null = 会话结束。
|
||||
/// 仅允许解码线程调用；destroy 前必须先停止本线程（见文件头锁纪律）。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeNextFrame<'local>(
|
||||
env: JNIEnv<'local>,
|
||||
_class: JClass<'local>,
|
||||
handle: jlong,
|
||||
) -> jbyteArray {
|
||||
// 短暂持锁：克隆 Arc 与拿 runtime 引用，随后立即释放——等帧不能挡输入。
|
||||
let (session, runtime) = {
|
||||
let Some(boxed) = handle_to_box(handle) else {
|
||||
return std::ptr::null_mut();
|
||||
};
|
||||
let guard = lock_box(boxed);
|
||||
(guard.session.clone(), &guard.runtime as *const tokio::runtime::Runtime)
|
||||
};
|
||||
// SAFETY: 见文件头「锁纪律」。Kotlin 保证解码线程存活期间不 destroy。
|
||||
let rt = unsafe { &*runtime };
|
||||
|
||||
match rt.block_on(session.next_frame()) {
|
||||
None => std::ptr::null_mut(),
|
||||
Some(f) => {
|
||||
let mut buf = vec![0u8; FRAME_HEADER_LEN + f.payload.len()];
|
||||
buf[0] = f.keyframe as u8;
|
||||
buf[1..9].copy_from_slice(&f.seq.to_le_bytes());
|
||||
buf[9..17].copy_from_slice(&f.ts_ms.to_le_bytes());
|
||||
buf[17..21].copy_from_slice(&f.width.to_le_bytes());
|
||||
buf[21..25].copy_from_slice(&f.height.to_le_bytes());
|
||||
buf[FRAME_HEADER_LEN..].copy_from_slice(&f.payload);
|
||||
match env.byte_array_from_slice(&buf) {
|
||||
Ok(a) => a.into_raw(),
|
||||
Err(_) => std::ptr::null_mut(),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 会话统计打包：高 32 位 rtt_ms，低 32 位 frames_in 低 32 位。-1 = 句柄无效。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeStats(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
) -> jlong {
|
||||
let Some(boxed) = handle_to_box(handle) else {
|
||||
return -1;
|
||||
};
|
||||
let guard = lock_box(boxed);
|
||||
let st = guard.session.stats();
|
||||
// 打包进 i64：高 32 位 rtt_ms、低 32 位 frames_in，Kotlin 侧移位还原
|
||||
(((st.rtt_ms & 0xFFFF_FFFF) << 32) | (st.frames_in & 0xFFFF_FFFF)) as jlong
|
||||
}
|
||||
|
||||
/// 被控端桌面尺寸，打包成 i64：高 32 位宽、低 32 位高。0 = 句柄无效。
|
||||
/// Kotlin 侧必须在建解码器之前调用，用返回值定 SurfaceView 宽高比。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeDesktopSize(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
) -> jlong {
|
||||
let Some(boxed) = handle_to_box(handle) else {
|
||||
return 0;
|
||||
};
|
||||
let guard = lock_box(boxed);
|
||||
let (w, h) = guard.session.desktop_size();
|
||||
(((w as u64 & 0xFFFF_FFFF) << 32) | (h as u64 & 0xFFFF_FFFF)) as jlong
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseMove(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
x: jint,
|
||||
y: jint,
|
||||
) {
|
||||
with_session(handle, |s| {
|
||||
s.send_input(InputEvent::MouseMove {
|
||||
x: x.clamp(0, 65535) as u16,
|
||||
y: y.clamp(0, 65535) as u16,
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
/// button: 0=左 1=右 其他=中；down: 1=按下 0=抬起
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendMouseButton(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
button: jint,
|
||||
down: jint,
|
||||
) {
|
||||
use common::MouseButton;
|
||||
let b = match button {
|
||||
0 => MouseButton::Left,
|
||||
1 => MouseButton::Right,
|
||||
_ => MouseButton::Middle,
|
||||
};
|
||||
with_session(handle, |s| {
|
||||
s.send_input(if down != 0 {
|
||||
InputEvent::MouseDown(b)
|
||||
} else {
|
||||
InputEvent::MouseUp(b)
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendWheel(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
dy: jint,
|
||||
) {
|
||||
with_session(handle, |s| s.send_input(InputEvent::Wheel { dy }));
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendKey(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
scancode: jint,
|
||||
extended: jint,
|
||||
down: jint,
|
||||
) {
|
||||
with_session(handle, |s| {
|
||||
s.send_input(if down != 0 {
|
||||
InputEvent::KeyDown {
|
||||
scancode: scancode as u16,
|
||||
extended: extended != 0,
|
||||
}
|
||||
} else {
|
||||
InputEvent::KeyUp {
|
||||
scancode: scancode as u16,
|
||||
extended: extended != 0,
|
||||
}
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
/// 协议 v3：文本上行。`units` 是 Kotlin 侧的 `CharArray`（UTF-16 code unit）。
|
||||
///
|
||||
/// 为什么不用 `JString`：jni 0.21 的 `get_string` 走 Modified-UTF-8 → Rust `String`，
|
||||
/// 落单代理项（IME 提交边界常见）会在这次往返里丢失。`char[]` 经
|
||||
/// `GetCharArrayRegion` 按 u16 直接拷贝，零转换（`jchar = u16`，见 jni-sys-0.3.1/lib.rs:15）。
|
||||
///
|
||||
/// 截断与空输入的判断在 `viewer_core::Session::send_text` 里做，与被控端同一口径。
|
||||
/// 失败必须有可见后果：JNI 取数失败走 eprintln（Android 上落 stderr→logcat），
|
||||
/// 不静默返回成功——用户对着远端打字却没字，最坏的就是「看起来发送了」。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeSendText<'local>(
|
||||
// 不需要 `mut`：jni 0.21.1 的 `get_array_length` / `get_char_array_region`
|
||||
// 都是 `&self`（jnienv.rs:1652 / :1887），与 `new_string` 之类的 `&mut self` 不同。
|
||||
env: JNIEnv<'local>,
|
||||
_class: JClass<'local>,
|
||||
handle: jlong,
|
||||
units: JObject<'local>,
|
||||
) {
|
||||
if units.as_raw().is_null() {
|
||||
// Kotlin 侧不会传 null，但 JNI 入口的对端数据一律不信任（全局安全规则）
|
||||
return;
|
||||
}
|
||||
let arr = JCharArray::from(units);
|
||||
let len = match env.get_array_length(&arr) {
|
||||
Ok(n) if n > 0 => n as usize,
|
||||
Ok(_) => return,
|
||||
Err(e) => {
|
||||
eprintln!("[android-ffi] get_array_length failed: {e}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mut buf = vec![0u16; len];
|
||||
if let Err(e) = env.get_char_array_region(&arr, 0, &mut buf) {
|
||||
eprintln!("[android-ffi] get_char_array_region failed: {e}");
|
||||
return;
|
||||
}
|
||||
with_session(handle, |s| s.send_text(&buf));
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeRequestKeyframe(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
) {
|
||||
with_session(handle, |s| s.request_keyframe());
|
||||
}
|
||||
|
||||
/// 释放会话。Kotlin 侧调用后必须把句柄置 0，且先停解码线程。
|
||||
#[no_mangle]
|
||||
pub extern "system" fn Java_com_verify_remote_RustBridge_nativeDestroy(
|
||||
_env: JNIEnv,
|
||||
_class: JClass,
|
||||
handle: jlong,
|
||||
) {
|
||||
if handle == 0 {
|
||||
return;
|
||||
}
|
||||
let real = handle as *mut Mutex<SessionBox>;
|
||||
unsafe {
|
||||
let _ = Box::from_raw(real);
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_to_box(handle: jlong) -> Option<&'static Mutex<SessionBox>> {
|
||||
if handle == 0 {
|
||||
return None;
|
||||
}
|
||||
let real = handle as *mut Mutex<SessionBox>;
|
||||
// 悬垂指针无法在 Rust 侧检测：阶段 1 由 Kotlin「destroy 后置 0 + 先停线程」
|
||||
// 的顺序纪律保证。违反即 UB，登记在方案风险清单（阶段 2 可加代际表消除）。
|
||||
// NonNull::new 是安全函数；解引用才是 unsafe 点
|
||||
std::ptr::NonNull::new(real).map(|p| unsafe { p.as_ref() })
|
||||
}
|
||||
|
||||
/// 锁中毒容忍：SessionBox 内部无跨线程共享可变状态（Session 各方法内部自带
|
||||
/// 并发原语），中毒后继续用是安全的。
|
||||
fn lock_box(b: &Mutex<SessionBox>) -> std::sync::MutexGuard<'_, SessionBox> {
|
||||
match b.lock() {
|
||||
Ok(g) => g,
|
||||
Err(p) => p.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
fn with_session(handle: jlong, f: impl FnOnce(&Session)) {
|
||||
let Some(boxed) = handle_to_box(handle) else {
|
||||
return;
|
||||
};
|
||||
let guard = lock_box(boxed);
|
||||
f(&guard.session);
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
# Gradle / Android 构建产物
|
||||
.gradle/
|
||||
build/
|
||||
app/build/
|
||||
local.properties
|
||||
|
||||
# Rust 交叉编译产物（由 build-android.ps1 生成到 app/libs）
|
||||
app/libs/
|
||||
|
||||
# 验证截图
|
||||
evidence/
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
*.iml
|
||||
@@ -1,56 +0,0 @@
|
||||
plugins {
|
||||
id("com.android.application")
|
||||
id("org.jetbrains.kotlin.android")
|
||||
}
|
||||
|
||||
android {
|
||||
namespace = "com.verify.remote"
|
||||
compileSdk = 34
|
||||
|
||||
defaultConfig {
|
||||
applicationId = "com.verify.remote"
|
||||
minSdk = 26
|
||||
targetSdk = 34
|
||||
versionCode = 1
|
||||
versionName = "0.1-phase1"
|
||||
// 阶段 1 只测真机 arm64；x86_64 留给模拟器回归
|
||||
ndk {
|
||||
abiFilters += listOf("arm64-v8a")
|
||||
}
|
||||
}
|
||||
|
||||
buildTypes {
|
||||
release {
|
||||
isMinifyEnabled = false
|
||||
}
|
||||
}
|
||||
|
||||
compileOptions {
|
||||
sourceCompatibility = JavaVersion.VERSION_11
|
||||
targetCompatibility = JavaVersion.VERSION_11
|
||||
}
|
||||
kotlinOptions {
|
||||
jvmTarget = "11"
|
||||
}
|
||||
|
||||
// .so 由外部脚本（build-android.ps1）预先拷到 app/libs/arm64-v8a/，
|
||||
// 不让 Gradle 调 cargo——构建职责分离：Rust 侧失败与 Gradle 侧失败可立刻区分。
|
||||
sourceSets {
|
||||
getByName("main") {
|
||||
jniLibs.srcDirs("libs")
|
||||
}
|
||||
}
|
||||
|
||||
packaging {
|
||||
jniLibs {
|
||||
// AGP 默认已按 useLegacyPackaging 处理；保持默认压缩+extractNativeLibs=false
|
||||
// 由 AGP 保证 STORED+对齐，手动链的坑（lessons 记录）在 Gradle 下不存在
|
||||
useLegacyPackaging = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
// 阶段 1 刻意零第三方 UI 依赖：纯 View 代码布局，减少变量。
|
||||
// AndroidX 在阶段 2 做正式 UI 时再引入。
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
|
||||
|
||||
<!-- 阶段 1：局域网连 agent.exe 需要网络 -->
|
||||
<uses-permission android:name="android.permission.INTERNET" />
|
||||
|
||||
<application
|
||||
android:label="Remote Viewer"
|
||||
android:hasCode="true">
|
||||
<activity
|
||||
android:name=".MainActivity"
|
||||
android:exported="true"
|
||||
android:keepScreenOn="true"
|
||||
android:windowSoftInputMode="adjustPan"
|
||||
android:configChanges="orientation|screenSize|keyboardHidden">
|
||||
<intent-filter>
|
||||
<action android:name="android.intent.action.MAIN" />
|
||||
<category android:name="android.intent.category.LAUNCHER" />
|
||||
</intent-filter>
|
||||
</activity>
|
||||
</application>
|
||||
</manifest>
|
||||
@@ -1,159 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
import android.media.MediaCodec
|
||||
import android.media.MediaFormat
|
||||
import android.util.Log
|
||||
import android.view.Surface
|
||||
import java.nio.ByteBuffer
|
||||
|
||||
/**
|
||||
* MediaCodec H.264 解码器封装（阶段 1：同步模式 + Surface 直出）。
|
||||
*
|
||||
* 两条实测教训直接写进实现（来源见个人层 lessons / baseline）：
|
||||
* 1. Exynos 等设备带 requires-allocate-on-input/output-ports Quirk，
|
||||
* 必须用 getInputBuffer(i)/getOutputBuffer(i)，不用已废弃的数组版 API。
|
||||
* 2. 参考帧链断裂后 P 帧必然解不出：解码线程维护 awaitingIdr 状态，
|
||||
* 只喂 IDR 之后的帧；错误后 flush + requestKeyframe 并等 IDR（R-3）。
|
||||
*
|
||||
* MediaFormat 不设 csd：码流每个 IDR 自带 SPS/PPS（dump_stream 实测
|
||||
* "SPS PPS IDR" 结构），解码器从码流内联参数集自举。
|
||||
*/
|
||||
class H264Decoder(private val surface: Surface) {
|
||||
|
||||
@Volatile
|
||||
var stopped = false
|
||||
private set
|
||||
|
||||
private var codec: MediaCodec? = null
|
||||
private val bufferInfo = MediaCodec.BufferInfo()
|
||||
|
||||
/** 等待 IDR 标志：true 时 P 帧一律不喂解码器 */
|
||||
private var awaitingIdr = true
|
||||
|
||||
private var configuredW = 0
|
||||
private var configuredH = 0
|
||||
|
||||
/** 解码一帧。返回 false = 该帧被丢弃（等 IDR 中或缓冲满），不视为错误。 */
|
||||
fun queue(frame: RustBridge.Frame): Boolean {
|
||||
// 等待 IDR：参考链未建立/已断裂时，P 帧喂进去只会产生错误
|
||||
if (awaitingIdr && !frame.keyframe) return false
|
||||
if (frame.keyframe) awaitingIdr = false
|
||||
|
||||
val c = ensureCodec(frame) ?: return false
|
||||
return try {
|
||||
val idx = c.dequeueInputBuffer(INPUT_TIMEOUT_US)
|
||||
if (idx < 0) return false // 输入满：阶段 1 直接丢，由消费方决定
|
||||
val inBuf: ByteBuffer = c.getInputBuffer(idx) ?: return false
|
||||
inBuf.clear()
|
||||
inBuf.put(frame.payload)
|
||||
c.queueInputBuffer(idx, 0, frame.payload.size, frame.tsMs * 1000, 0)
|
||||
drainOnce(c)
|
||||
true
|
||||
} catch (e: MediaCodec.CodecException) {
|
||||
Log.e(TAG, "codec error isRecoverable=${e.isRecoverable}", e)
|
||||
invalidate()
|
||||
false
|
||||
} catch (e: IllegalStateException) {
|
||||
Log.e(TAG, "codec illegal state", e)
|
||||
invalidate()
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/** 解码错误/跳帧后调用：flush 复位解码器（三级恢复第一级），
|
||||
* 置等待标志；调用方负责 requestKeyframe。 */
|
||||
fun invalidate() {
|
||||
awaitingIdr = true
|
||||
try {
|
||||
codec?.flush()
|
||||
} catch (e: Exception) {
|
||||
// flush 失败说明解码器不可恢复：销毁，等下一个 IDR 重建
|
||||
Log.w(TAG, "flush failed, releasing codec", e)
|
||||
releaseCodec()
|
||||
}
|
||||
}
|
||||
|
||||
/** 确保解码器就绪。只在 keyframe 到来时才配置/重建（首配与重配同路径）。 */
|
||||
private fun ensureCodec(frame: RustBridge.Frame): MediaCodec? {
|
||||
val c = codec
|
||||
if (c != null && configuredW == frame.width && configuredH == frame.height) {
|
||||
return c
|
||||
}
|
||||
if (c != null) {
|
||||
Log.i(TAG, "size changed ${configuredW}x$configuredH -> ${frame.width}x${frame.height}, reconfigure")
|
||||
releaseCodec()
|
||||
}
|
||||
if (codec == null && frame.keyframe) {
|
||||
val name = pickDecoderName()
|
||||
val nc = if (name.isEmpty()) MediaCodec.createDecoderByType(MIME)
|
||||
else MediaCodec.createByCodecName(name)
|
||||
val fmt = MediaFormat.createVideoFormat(MIME, frame.width, frame.height)
|
||||
nc.configure(fmt, surface, null, 0)
|
||||
nc.start()
|
||||
codec = nc
|
||||
configuredW = frame.width
|
||||
configuredH = frame.height
|
||||
Log.i(TAG, "decoder configured ${frame.width}x${frame.height} name=${nc.name}")
|
||||
}
|
||||
return codec
|
||||
}
|
||||
|
||||
/** 优先硬解；查不到就返回空串让系统自选（避免选中软件解码器）。 */
|
||||
private fun pickDecoderName(): String {
|
||||
return try {
|
||||
android.media.MediaCodecList(android.media.MediaCodecList.REGULAR_CODECS)
|
||||
.codecInfos
|
||||
.firstOrNull { info ->
|
||||
!info.isEncoder &&
|
||||
info.supportedTypes.any { t -> t.equals(MIME, true) } &&
|
||||
!info.name.startsWith("OMX.google.") &&
|
||||
!info.name.startsWith("c2.android.")
|
||||
}?.name ?: ""
|
||||
} catch (e: Exception) {
|
||||
""
|
||||
}
|
||||
}
|
||||
|
||||
private fun drainOnce(c: MediaCodec) {
|
||||
while (!stopped) {
|
||||
val idx = c.dequeueOutputBuffer(bufferInfo, 0)
|
||||
when {
|
||||
idx == MediaCodec.INFO_TRY_AGAIN_LATER -> return
|
||||
idx == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED ->
|
||||
Log.i(TAG, "output format: ${c.outputFormat}")
|
||||
idx >= 0 -> {
|
||||
// Surface 直出：渲染由解码器完成，不取图像数据
|
||||
c.releaseOutputBuffer(idx, true)
|
||||
}
|
||||
else -> return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun release() {
|
||||
stopped = true
|
||||
releaseCodec()
|
||||
}
|
||||
|
||||
private fun releaseCodec() {
|
||||
try {
|
||||
codec?.stop()
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "stop ignored", e)
|
||||
}
|
||||
try {
|
||||
codec?.release()
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "release ignored", e)
|
||||
}
|
||||
codec = null
|
||||
configuredW = 0
|
||||
configuredH = 0
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "H264Decoder"
|
||||
private const val MIME = "video/avc"
|
||||
private const val INPUT_TIMEOUT_US = 4_000L
|
||||
}
|
||||
}
|
||||
@@ -1,499 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
import android.app.Activity
|
||||
import android.content.Context
|
||||
import android.os.Bundle
|
||||
import android.text.InputType
|
||||
import android.util.Log
|
||||
import android.view.Gravity
|
||||
import android.view.KeyEvent
|
||||
import android.view.SurfaceHolder
|
||||
import android.view.View
|
||||
import android.view.inputmethod.InputMethodManager
|
||||
import android.widget.Button
|
||||
import android.widget.EditText
|
||||
import android.widget.FrameLayout
|
||||
import android.widget.LinearLayout
|
||||
import android.widget.LinearLayout.LayoutParams
|
||||
import android.widget.TextView
|
||||
import kotlin.concurrent.thread
|
||||
|
||||
/**
|
||||
* 安卓主控端（阶段 2b）：正式连接界面 + 远控画面 + 触摸操作 + 软键盘文本输入。
|
||||
*
|
||||
* 两态：
|
||||
* - 连接态：host / port / code 表单 + 状态文本（阶段 1/2a 只能靠 `am start --es host`
|
||||
* 传参，本界面对它补位；intent extras 仍保留且优先级更高，构建脚本继续可用）。
|
||||
* - 远控态：letterbox 画面 + 底部状态/操作条（帧数、RTT、诊断、键盘、断开）。
|
||||
*
|
||||
* 线程模型（与 android-ffi 的锁纪律对应）：
|
||||
* - UI 线程：触摸/按键/IME 提交 → nativeSend*（入队不阻塞）；连接/断开
|
||||
* - 解码线程：nativeNextFrame 阻塞取帧 → H264Decoder.queue
|
||||
* - 断开顺序：**先置 stopDecoding 并 join 解码线程，再 destroy**
|
||||
* （Rust 侧明确：解码线程存活期间 destroy 是 UB）
|
||||
*
|
||||
* 持久化：只存 host/port（连接成功后落盘）；**验证码不落盘**（2026-10-30 用户裁决）。
|
||||
*/
|
||||
class MainActivity : Activity(), RemoteSurfaceView.Listener {
|
||||
|
||||
private lateinit var surfaceView: RemoteSurfaceView
|
||||
private lateinit var statusView: TextView
|
||||
private lateinit var connectPanel: LinearLayout
|
||||
private lateinit var remotePanel: LinearLayout
|
||||
private lateinit var hostInput: EditText
|
||||
private lateinit var portInput: EditText
|
||||
private lateinit var codeInput: EditText
|
||||
private lateinit var connectButton: Button
|
||||
private lateinit var connectStatus: TextView
|
||||
|
||||
@Volatile
|
||||
private var handle = 0L
|
||||
|
||||
@Volatile
|
||||
private var stopDecoding = false
|
||||
private var decodeThread: Thread? = null
|
||||
|
||||
/** 连接中标志：防重复点击起出第二个会话（句柄未回填前 handle 仍是 0） */
|
||||
@Volatile
|
||||
private var connecting = false
|
||||
|
||||
@Volatile
|
||||
private var remoteW = 0
|
||||
@Volatile
|
||||
private var remoteH = 0
|
||||
|
||||
/** 最近一次输入事件（触摸/IME）诊断：logcat 被限流，状态栏是唯一可靠通道 */
|
||||
@Volatile
|
||||
private var lastTouchDbg = "-"
|
||||
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
Log.i(TAG, "onCreate start")
|
||||
|
||||
surfaceView = RemoteSurfaceView(this).apply {
|
||||
listener = this@MainActivity
|
||||
onDebug = { msg -> lastTouchDbg = msg; refreshStatus() }
|
||||
holder.addCallback(object : SurfaceHolder.Callback {
|
||||
override fun surfaceCreated(holder: SurfaceHolder) {}
|
||||
override fun surfaceChanged(h: SurfaceHolder, f: Int, w: Int, ht: Int) {}
|
||||
override fun surfaceDestroyed(holder: SurfaceHolder) {
|
||||
// Surface 没了解码器输出会失效：断开，回到连接界面重连
|
||||
disconnect()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
val root = LinearLayout(this).apply {
|
||||
orientation = LinearLayout.VERTICAL
|
||||
setBackgroundColor(0xFF000000.toInt())
|
||||
}
|
||||
connectPanel = buildConnectPanel()
|
||||
remotePanel = buildRemotePanel()
|
||||
val lp = LinearLayout.LayoutParams(LinearLayout.LayoutParams.MATCH_PARENT, 0, 1f)
|
||||
root.addView(connectPanel, lp)
|
||||
root.addView(remotePanel, LinearLayout.LayoutParams(LinearLayout.LayoutParams.MATCH_PARENT, 0, 1f))
|
||||
setContentView(root)
|
||||
|
||||
showConnectState("未连接")
|
||||
|
||||
// intent 入口优先级高于表单默认值：build-android.ps1 与调试脚本依赖它
|
||||
val hostExtra = intent?.getStringExtra("host")
|
||||
if (!hostExtra.isNullOrEmpty()) {
|
||||
hostInput.setText(hostExtra)
|
||||
portInput.setText(intent.getIntExtra("port", 47800).toString())
|
||||
intent.getStringExtra("code")?.let { codeInput.setText(it) }
|
||||
startConnect()
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 连接态 UI ----
|
||||
|
||||
private fun buildConnectPanel(): LinearLayout {
|
||||
val prefs = getSharedPreferences(PREFS, Context.MODE_PRIVATE)
|
||||
// 只回填 host/port（上次**成功连接**过的值）；验证码每次手输
|
||||
hostInput = editText(
|
||||
"被控端 IP",
|
||||
InputType.TYPE_CLASS_TEXT,
|
||||
prefs.getString("host", "").orEmpty(),
|
||||
)
|
||||
portInput = editText(
|
||||
"端口",
|
||||
InputType.TYPE_CLASS_NUMBER,
|
||||
prefs.getInt("port", 47800).toString(),
|
||||
)
|
||||
// 验证码刻意不从 prefs 读（不落盘）
|
||||
codeInput = editText("验证码", InputType.TYPE_CLASS_TEXT, "")
|
||||
connectButton = Button(this).apply {
|
||||
text = "连接"
|
||||
setOnClickListener { startConnect() }
|
||||
}
|
||||
connectStatus = TextView(this).apply {
|
||||
textSize = 13f
|
||||
setTextColor(0xFFBBBBBB.toInt())
|
||||
setPadding(0, dp(8), 0, 0)
|
||||
}
|
||||
val title = TextView(this).apply {
|
||||
text = "连接被控端"
|
||||
textSize = 18f
|
||||
setTextColor(0xFFFFFFFF.toInt())
|
||||
setPadding(0, dp(16), 0, dp(8))
|
||||
}
|
||||
return LinearLayout(this).apply {
|
||||
orientation = LinearLayout.VERTICAL
|
||||
setPadding(dp(20), dp(24), dp(20), dp(24))
|
||||
addView(title)
|
||||
addView(hostInput)
|
||||
addView(portInput)
|
||||
addView(codeInput)
|
||||
addView(connectButton)
|
||||
addView(connectStatus)
|
||||
}
|
||||
}
|
||||
|
||||
private fun editText(hint: String, type: Int, initial: String) = EditText(this).apply {
|
||||
inputType = type
|
||||
this.hint = hint
|
||||
setText(initial)
|
||||
textSize = 16f
|
||||
setTextColor(0xFFFFFFFF.toInt())
|
||||
setHintTextColor(0xFF777777.toInt())
|
||||
// 表单文字不能被画面抢占焦点
|
||||
isSingleLine = true
|
||||
}
|
||||
|
||||
// ---- 远控态 UI ----
|
||||
|
||||
private fun buildRemotePanel(): LinearLayout {
|
||||
statusView = TextView(this).apply {
|
||||
textSize = 12f
|
||||
setPadding(dp(12), dp(6), dp(12), dp(6))
|
||||
setBackgroundColor(0xCC000000.toInt())
|
||||
setTextColor(0xFFFFFFFF.toInt())
|
||||
maxLines = 3
|
||||
}
|
||||
// 画面居中、黑边填充。
|
||||
// 用 FrameLayout + MATCH_PARENT（不是 WRAP_CONTENT）：FrameLayout 会把
|
||||
// 子视图的 WRAP_CONTENT 当 MATCH_PARENT 处理，onMeasure 里拿到的
|
||||
// MeasureSpec 就成了 EXACTLY 全尺寸，letterbox 比例收缩被直接忽略
|
||||
// （2026-10-30 实测：视图被撑成 1080x2001，触摸按 2001 归一化 → 坐标全错）。
|
||||
// MATCH_PARENT + gravity=center 时父容器传 AT_MOST，onMeasure 的
|
||||
// setMeasuredDimension 才真正决定尺寸，再被 gravity 居中摆放。
|
||||
val stage = FrameLayout(this).apply {
|
||||
setBackgroundColor(0xFF000000.toInt())
|
||||
addView(
|
||||
surfaceView,
|
||||
FrameLayout.LayoutParams(
|
||||
FrameLayout.LayoutParams.MATCH_PARENT,
|
||||
FrameLayout.LayoutParams.MATCH_PARENT,
|
||||
Gravity.CENTER,
|
||||
),
|
||||
)
|
||||
}
|
||||
val keyboardButton = Button(this).apply {
|
||||
text = "键盘"
|
||||
setOnClickListener { showKeyboard() }
|
||||
}
|
||||
val disconnectButton = Button(this).apply {
|
||||
text = "断开"
|
||||
setOnClickListener { disconnect() }
|
||||
}
|
||||
val bar = LinearLayout(this).apply {
|
||||
orientation = LinearLayout.HORIZONTAL
|
||||
addView(keyboardButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
|
||||
addView(disconnectButton, LayoutParams(0, LayoutParams.WRAP_CONTENT, 1f))
|
||||
}
|
||||
return LinearLayout(this).apply {
|
||||
orientation = LinearLayout.VERTICAL
|
||||
visibility = View.GONE
|
||||
addView(stage, LayoutParams(LayoutParams.MATCH_PARENT, 0, 1f))
|
||||
// 固定高度而非 WRAP_CONTENT：与 weight=1 的兄弟共存时，
|
||||
// WRAP_CONTENT 可能被压成 0（实测 bounds=[0,0][0,0]，状态栏不可见）
|
||||
addView(statusView, LayoutParams(LayoutParams.MATCH_PARENT, dp(64)))
|
||||
addView(bar, LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.WRAP_CONTENT))
|
||||
}
|
||||
}
|
||||
|
||||
private fun showConnectState(msg: String) {
|
||||
connecting = false
|
||||
remotePanelVisible(false)
|
||||
connectStatus.text = msg
|
||||
}
|
||||
|
||||
private fun showRemoteState() {
|
||||
remotePanelVisible(true)
|
||||
connectPanel.visibility = View.GONE
|
||||
}
|
||||
|
||||
private fun remotePanelVisible(show: Boolean) {
|
||||
connectPanel.visibility = if (show) View.GONE else View.VISIBLE
|
||||
remotePanel.visibility = if (show) View.VISIBLE else View.GONE
|
||||
}
|
||||
|
||||
/**
|
||||
* 显式唤起软键盘。不做自动弹：自动弹会挡画面，且 vivo ROM 行为不一致。
|
||||
* SurfaceView 已在触摸模式下可聚焦（见 RemoteSurfaceView.init），
|
||||
* 否则系统不会建立 InputConnection。
|
||||
*/
|
||||
private fun showKeyboard() {
|
||||
if (handle == 0L) return
|
||||
surfaceView.requestFocus()
|
||||
val imm = getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager
|
||||
imm.showSoftInput(surfaceView, InputMethodManager.SHOW_IMPLICIT)
|
||||
lastTouchDbg = "showSoftInput 已请求（焦点=${surfaceView.hasFocus()}）"
|
||||
refreshStatus()
|
||||
}
|
||||
|
||||
// ---- RemoteSurfaceView.Listener ----
|
||||
|
||||
override fun sendMouseMove(x: Int, y: Int) {
|
||||
val h = handle
|
||||
if (h != 0L) RustBridge.nativeSendMouseMove(h, x, y)
|
||||
}
|
||||
|
||||
override fun sendMouseButton(button: Int, down: Boolean) {
|
||||
val h = handle
|
||||
if (h != 0L) RustBridge.nativeSendMouseButton(h, button, if (down) 1 else 0)
|
||||
}
|
||||
|
||||
override fun sendWheel(dy: Int) {
|
||||
val h = handle
|
||||
if (h != 0L) RustBridge.nativeSendWheel(h, dy)
|
||||
}
|
||||
|
||||
/** 协议 v3：IME 提交的整段文本（UTF-16 code unit，含落单代理项原样透传） */
|
||||
override fun sendText(units: CharArray) {
|
||||
val h = handle
|
||||
if (h != 0L) RustBridge.nativeSendText(h, units)
|
||||
}
|
||||
|
||||
override fun sendKey(key: WinKey, down: Boolean) {
|
||||
val h = handle
|
||||
if (h != 0L) {
|
||||
RustBridge.nativeSendKey(h, key.scancode, if (key.extended) 1 else 0, if (down) 1 else 0)
|
||||
}
|
||||
}
|
||||
|
||||
override fun isReadyForInput(): Boolean = handle != 0L
|
||||
|
||||
/**
|
||||
* 按键转发（物理键盘 / adb 注入 / 无 IME 时的软键盘）：
|
||||
* - 返回键：先收 IME，否则断开回表单（阶段 2a 语义延续）；
|
||||
* - **可打印**的 `unicodeChar`（0x20 起、排除 0x7F）→ v3 **Text** 通道：
|
||||
* 字符注入与远端键盘布局无关。注意 `getUnicodeChar()` 给的是单个
|
||||
* UTF-16 code unit 而非码点，所以这条路只覆盖 BMP；增补平面字符由
|
||||
* IME `commitText` 成对送达，不经这里；
|
||||
* - 控制值（Enter/Tab/方向键…）与其余控制键 → Set-1 扫描码通道（down/up 配对）：
|
||||
* 远端应用期待的是按键事件，不是 WM_CHAR；
|
||||
* - 自动重复（`repeatCount > 0`，长按）只发一次 down：重复由远端键盘驱动，
|
||||
* 本地重发会打爆消息队列。
|
||||
*
|
||||
* 与 IME 路径互斥：IME 显示时按键先被输入法消费（走 commitText），不会到这里；
|
||||
* IME 隐藏时才走本方法。两条路同时生效会重复注入。
|
||||
*
|
||||
* 诊断：vivo logcat 被限流（lessons R-6），路由结果写状态栏，只报 keyCode
|
||||
* 与去向，不报字符内容。
|
||||
*/
|
||||
override fun onKeyDown(keyCode: Int, event: KeyEvent?): Boolean {
|
||||
if (keyCode == KeyEvent.KEYCODE_BACK) {
|
||||
val imm = getSystemService(Context.INPUT_METHOD_SERVICE) as InputMethodManager
|
||||
// isAcceptingText = 当前有 IME 连接且正在接收输入（API 17+）
|
||||
if (imm.isAcceptingText) {
|
||||
imm.hideSoftInputFromWindow(surfaceView.windowToken, 0)
|
||||
return true
|
||||
}
|
||||
if (handle != 0L) {
|
||||
disconnect()
|
||||
return true
|
||||
}
|
||||
return super.onKeyDown(keyCode, event)
|
||||
}
|
||||
val h = handle
|
||||
if (h == 0L || event == null) return super.onKeyDown(keyCode, event)
|
||||
// 只报 keyCode 与路由结果，不报字符内容（诊断通道会显示在屏幕上）
|
||||
val cp = event.unicodeChar
|
||||
if (cp >= 0x20 && cp != 0x7F) {
|
||||
if (event.repeatCount == 0) RustBridge.nativeSendText(h, charArrayOf(cp.toChar()))
|
||||
lastTouchDbg = "key code=$keyCode -> text 1u"
|
||||
runOnUiThread { refreshStatus() }
|
||||
return true
|
||||
}
|
||||
val key = WinScancode.of(keyCode) ?: run {
|
||||
lastTouchDbg = "key code=$keyCode 未映射"
|
||||
runOnUiThread { refreshStatus() }
|
||||
return super.onKeyDown(keyCode, event)
|
||||
}
|
||||
if (event.repeatCount == 0) {
|
||||
RustBridge.nativeSendKey(h, key.scancode, if (key.extended) 1 else 0, 1)
|
||||
}
|
||||
lastTouchDbg = "key code=$keyCode -> scan ${key.scancode} ext=${key.extended}"
|
||||
runOnUiThread { refreshStatus() }
|
||||
return true
|
||||
}
|
||||
|
||||
override fun onKeyUp(keyCode: Int, event: KeyEvent?): Boolean {
|
||||
val h = handle
|
||||
if (keyCode == KeyEvent.KEYCODE_BACK || h == 0L || event == null) {
|
||||
return super.onKeyUp(keyCode, event)
|
||||
}
|
||||
// 字符事件没有「抬起」语义（Text 通道是整段提交），只回扫控制键。
|
||||
// 门槛与 onKeyDown 一致：不可打印的 unicodeChar（Enter/Tab）仍需发 key up，
|
||||
// 否则 onKeyDown 发了 down、这里不发 up，远端会留下按住未释放的键。
|
||||
val cp = event.unicodeChar
|
||||
if (cp >= 0x20 && cp != 0x7F) return true
|
||||
val key = WinScancode.of(keyCode) ?: return super.onKeyUp(keyCode, event)
|
||||
RustBridge.nativeSendKey(h, key.scancode, if (key.extended) 1 else 0, 0)
|
||||
return true
|
||||
}
|
||||
|
||||
// ---- 连接 / 解码 ----
|
||||
|
||||
private fun startConnect() {
|
||||
if (connecting || handle != 0L) return
|
||||
val host = hostInput.text.toString().trim()
|
||||
val code = codeInput.text.toString()
|
||||
val port = portInput.text.toString().trim().toIntOrNull()
|
||||
when {
|
||||
host.isEmpty() -> {
|
||||
connectStatus.text = "请填写被控端 IP"
|
||||
return
|
||||
}
|
||||
port == null || port !in 1..65535 -> {
|
||||
connectStatus.text = "端口必须是 1..65535"
|
||||
return
|
||||
}
|
||||
code.isEmpty() -> {
|
||||
connectStatus.text = "请填写验证码"
|
||||
return
|
||||
}
|
||||
}
|
||||
connecting = true
|
||||
connectButton.isEnabled = false
|
||||
showRemoteState()
|
||||
statusView.text = "连接 $host:$port …"
|
||||
thread(name = "connect") {
|
||||
try {
|
||||
val h = RustBridge.connect(host, port, code)
|
||||
// 尺寸必须在解码前拿到：用它定 SurfaceView 宽高比。
|
||||
// 若等首帧再改，视图会重新测量，把刚建好的解码器 surface 打掉。
|
||||
val (w, ht) = RustBridge.desktopSize(h)
|
||||
handle = h
|
||||
remoteW = w
|
||||
remoteH = ht
|
||||
connecting = false
|
||||
runOnUiThread {
|
||||
connectButton.isEnabled = true
|
||||
// 只有认证成功的会话才值得记住；验证码不落盘（用户裁决）
|
||||
getSharedPreferences(PREFS, Context.MODE_PRIVATE)
|
||||
.edit()
|
||||
.putString("host", host)
|
||||
.putInt("port", port)
|
||||
.apply()
|
||||
if (w > 0 && ht > 0) {
|
||||
surfaceView.videoAspectRatio = w.toFloat() / ht
|
||||
}
|
||||
refreshStatus()
|
||||
startDecodeLoop(h)
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.e(TAG, "connect failed", e)
|
||||
val msg = e.message ?: "连接失败"
|
||||
runOnUiThread {
|
||||
connectButton.isEnabled = true
|
||||
handle = 0L
|
||||
showConnectState("连接失败：$msg")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun startDecodeLoop(h: Long) {
|
||||
stopDecoding = false
|
||||
decodeThread = thread(name = "decode") {
|
||||
val surface = surfaceView.holder.surface
|
||||
var tries = 0
|
||||
while (!surface.isValid && tries++ < 100) Thread.sleep(50)
|
||||
if (!surface.isValid) {
|
||||
Log.e(TAG, "surface never became valid")
|
||||
return@thread
|
||||
}
|
||||
val dec = H264Decoder(surface)
|
||||
var frames = 0L
|
||||
var lastKeyReq = 0L
|
||||
while (!stopDecoding) {
|
||||
val f = try {
|
||||
RustBridge.nextFrame(h)
|
||||
} catch (e: Exception) {
|
||||
Log.e(TAG, "nextFrame threw", e)
|
||||
null
|
||||
} ?: break
|
||||
val fed = dec.queue(f)
|
||||
if (!fed && !f.keyframe && System.currentTimeMillis() - lastKeyReq > 200) {
|
||||
RustBridge.nativeRequestKeyframe(h) // 等 IDR 中：限频请求
|
||||
lastKeyReq = System.currentTimeMillis()
|
||||
}
|
||||
frames++
|
||||
if (frames % 120 == 0L) {
|
||||
lastFrames = frames
|
||||
lastRtt = RustBridge.rttMs(h)
|
||||
runOnUiThread { refreshStatus() }
|
||||
}
|
||||
}
|
||||
dec.release()
|
||||
Log.i(TAG, "decode loop ended, frames=$frames")
|
||||
runOnUiThread { refreshStatus() }
|
||||
}
|
||||
}
|
||||
|
||||
private fun disconnect() {
|
||||
if (handle == 0L) return
|
||||
stopDecoding = true
|
||||
// 顺序纪律：等解码线程退出后才能 destroy（见 android-ffi 文件头）。
|
||||
// 卡帧风险：agent 静止时 500ms 一帧、QUIC idle 超时后流必关，6s 足够覆盖。
|
||||
decodeThread?.join(6000)
|
||||
decodeThread = null
|
||||
val h = handle
|
||||
handle = 0L
|
||||
remoteW = 0
|
||||
remoteH = 0
|
||||
lastFrames = 0
|
||||
lastRtt = 0
|
||||
RustBridge.destroy(h)
|
||||
showConnectState("已断开")
|
||||
}
|
||||
|
||||
override fun onDestroy() {
|
||||
super.onDestroy()
|
||||
disconnect()
|
||||
}
|
||||
|
||||
/** 状态栏统一刷新：连接态 + 帧数/RTT + 最近输入诊断（logcat 被限流，这是唯一通道） */
|
||||
@Volatile
|
||||
private var lastFrames = 0L
|
||||
@Volatile
|
||||
private var lastRtt = 0L
|
||||
|
||||
private fun refreshStatus() {
|
||||
if (handle == 0L) {
|
||||
statusView.text = if (connecting) statusView.text else "未连接"
|
||||
return
|
||||
}
|
||||
statusView.text = buildString {
|
||||
append("已连接 v$PROTO_LABEL ${remoteW}x$remoteH 帧=$lastFrames RTT=${lastRtt}ms\n")
|
||||
append("输入: $lastTouchDbg")
|
||||
}
|
||||
}
|
||||
|
||||
private fun dp(v: Int) = (v * resources.displayMetrics.density).toInt()
|
||||
|
||||
companion object {
|
||||
private const val TAG = "MainActivity"
|
||||
private const val PREFS = "connect"
|
||||
|
||||
/**
|
||||
* 与协议版本保持一致：远控画面状态栏显示当前握手版本（方案 R5 要求）。
|
||||
* 值随 `common::PROTO_VERSION` 人工同步——Android 侧不做线格式解析，
|
||||
* 版本由 Rust 握手校验，这里只作显示与排障用。
|
||||
*/
|
||||
private const val PROTO_LABEL = "3"
|
||||
}
|
||||
}
|
||||
@@ -1,121 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
import android.view.KeyEvent
|
||||
import android.view.View
|
||||
import android.view.inputmethod.BaseInputConnection
|
||||
import android.view.inputmethod.EditorInfo
|
||||
|
||||
/**
|
||||
* P1 IME 通道（loop-2b-plan §5，用户已裁决）：**本地没有编辑缓冲，远端才是权威**。
|
||||
*
|
||||
* 三条约定（与方案一致）：
|
||||
* 1. `commitText` = 用户确认的整段文本 → 直接走协议 v3 `Text` 通道（布局无关、支持中文）；
|
||||
* 2. `sendKeyEvent` = 硬键盘/控制键 → 既有扫描码通道（Enter/Backspace/方向键…）；
|
||||
* 3. `deleteSurroundingText`：本地不知道远端内容，`before>0 && after==0`
|
||||
* 是 IME 退格最常见形态 → 发对应次数的 Backspace；其余形态记录后放弃（U3）。
|
||||
*
|
||||
* composing（拼音未确认串）不显示到远端（U4）：`setComposingText` 吞掉但返回 true，
|
||||
* 让 IME 维持本地组合状态，候选确认后仍会走 `commitText`。
|
||||
*
|
||||
* 日志纪律：debug 通道只报**长度与事件类型**，不报用户输入内容（安全红线）。
|
||||
*/
|
||||
class RemoteInputConnection(
|
||||
target: View,
|
||||
private val sendText: (CharArray) -> Unit,
|
||||
private val sendKey: (WinKey, Boolean) -> Unit,
|
||||
private val debug: (String) -> Unit,
|
||||
) : BaseInputConnection(target, false) {
|
||||
|
||||
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
|
||||
val units = text.toString().toCharArray()
|
||||
if (units.isEmpty()) return true
|
||||
sendText(units)
|
||||
debug("commit ${units.size}u")
|
||||
return true
|
||||
}
|
||||
|
||||
override fun sendKeyEvent(event: KeyEvent): Boolean {
|
||||
if (event.action == KeyEvent.ACTION_MULTIPLE) {
|
||||
val chars: String? = event.characters
|
||||
if (!chars.isNullOrEmpty()) {
|
||||
sendText(chars.toCharArray())
|
||||
debug("multiple ${chars.length}u")
|
||||
}
|
||||
return true
|
||||
}
|
||||
// 字符事件优先于扫描码：多数软键盘发的是 keyCode=KEYCODE_UNKNOWN +
|
||||
// unicodeChar（adb `input text` 同款形态）。这一分支必须排在 WinScancode
|
||||
// 查表**之前**——否则 UNKNOWN(0) 查不到表会被当「未映射」吞掉，字就丢了。
|
||||
// 注意 `getUnicodeChar()` 返回的是单个 UTF-16 code unit（不是码点），
|
||||
// 所以这条路只覆盖 BMP；增补平面字符由 `commitText` 成对送达。
|
||||
// 只收**可打印**字符（空格起、排除 0x7F）：控制值（Enter/Tab）必须走扫描码，
|
||||
// 远端应用期待的是按键事件而非 WM_CHAR。见 MainActivity.onKeyDown 同款门槛。
|
||||
val cp = event.unicodeChar
|
||||
if (event.action == KeyEvent.ACTION_DOWN && cp >= 0x20 && cp != 0x7F) {
|
||||
sendText(charArrayOf(cp.toChar()))
|
||||
debug("keychar 1u")
|
||||
return true
|
||||
}
|
||||
val key = WinScancode.of(event.keyCode)
|
||||
if (key == null) {
|
||||
debug("IME 未映射键 code=${event.keyCode}")
|
||||
return false
|
||||
}
|
||||
sendKey(key, event.action == KeyEvent.ACTION_DOWN)
|
||||
return true
|
||||
}
|
||||
|
||||
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean =
|
||||
deleteBackspaces(beforeLength, afterLength)
|
||||
|
||||
override fun deleteSurroundingTextInCodePoints(beforeLength: Int, afterLength: Int): Boolean =
|
||||
deleteBackspaces(beforeLength, afterLength)
|
||||
|
||||
private fun deleteBackspaces(before: Int, after: Int): Boolean {
|
||||
if (before == 0 && after == 0) return true
|
||||
if (after != 0) {
|
||||
// U3：forward-delete 与「先删 after 再删 before」的形态本地无远端真值可依
|
||||
debug("delete($before,$after)：after>0 形态未支持")
|
||||
return false
|
||||
}
|
||||
val bk = WinScancode.of(KeyEvent.KEYCODE_DEL) ?: return false
|
||||
repeat(before.coerceAtMost(MAX_BACKSPACE)) {
|
||||
sendKey(bk, true)
|
||||
sendKey(bk, false)
|
||||
}
|
||||
debug("backspace x$before")
|
||||
return true
|
||||
}
|
||||
|
||||
/** 吞掉 composing 但返回 true：IME 保持本地组合流程，候选确认仍会 commitText。 */
|
||||
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean = true
|
||||
|
||||
override fun setComposingRegion(start: Int, end: Int): Boolean = true
|
||||
|
||||
override fun finishComposingText(): Boolean = true
|
||||
|
||||
// 本地无可信缓冲区（远端才是权威），一律回空串——比 null 温和，多数 IME 兼容。
|
||||
override fun getTextBeforeCursor(n: Int, flags: Int): CharSequence = ""
|
||||
override fun getTextAfterCursor(n: Int, flags: Int): CharSequence = ""
|
||||
override fun getSelectedText(flags: Int): CharSequence = ""
|
||||
override fun getCursorCapsMode(reqModes: Int): Int = 0
|
||||
|
||||
override fun performEditorAction(actionCode: Int): Boolean {
|
||||
if (actionCode == EditorInfo.IME_ACTION_DONE) {
|
||||
// 软键盘「完成/回车」在远控语义里就是 Enter
|
||||
WinScancode.of(KeyEvent.KEYCODE_ENTER)?.let {
|
||||
sendKey(it, true)
|
||||
sendKey(it, false)
|
||||
}
|
||||
debug("IME_ACTION_DONE→Enter")
|
||||
return true
|
||||
}
|
||||
debug("editorAction $actionCode 忽略")
|
||||
return true
|
||||
}
|
||||
|
||||
companion object {
|
||||
/** 单次退格请求上限：防 IME 异常大值刷爆消息队列（与 agent 侧 WHEEL_MAX_CLICKS 同思路） */
|
||||
private const val MAX_BACKSPACE = 64
|
||||
}
|
||||
}
|
||||
@@ -1,278 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
import android.content.Context
|
||||
import android.text.InputType
|
||||
import android.view.MotionEvent
|
||||
import android.view.SurfaceView
|
||||
import android.view.inputmethod.EditorInfo
|
||||
import android.view.inputmethod.InputConnection
|
||||
import kotlin.math.abs
|
||||
import kotlin.math.hypot
|
||||
|
||||
/**
|
||||
* 保持视频宽高比的 SurfaceView（letterbox），把触摸手势映射为远端鼠标事件，
|
||||
* 并作为软键盘的宿主（协议 v3 文本通道 + 扫描码控制键）。
|
||||
*
|
||||
* 为什么视图尺寸必须等于画面尺寸：触摸坐标按视图宽高归一化到远端 0..=65535。
|
||||
* 若视图被拉伸（阶段 1 用 MATCH_PARENT），画面与视图不再一一对应，手指位置与
|
||||
* 远端指针位置就会错位；letterbox 后黑边不属于本视图，触摸天然只在画面内产生事件。
|
||||
*
|
||||
* 手势映射（桌面远控的通行方案，参考 Moonlight / Scrcpy 的交互形态，
|
||||
* 实现为本项目独立编写）：
|
||||
* - 单指轻点 = 左键单击
|
||||
* - 单指按下并移动 = 左键拖拽（窗口拖动、文字选中）
|
||||
* - 双指轻点 = 右键
|
||||
* - 双指上下滑动 = 滚轮
|
||||
*
|
||||
* 刻意不做「手指即光标」的绝对定位：手指会完全挡住目标，桌面精细操作不可用。
|
||||
*
|
||||
* **IME 接线（阶段 2b）**：本视图是文本编辑器（`onCheckIsTextEditor`），
|
||||
* 软键盘的提交经 `RemoteInputConnection` 上行。`isFocusableInTouchMode` 必须为
|
||||
* true，否则触摸态拿不到焦点、系统不建立 InputConnection（阶段 2a 只有
|
||||
* `isFocusable`，这就是当时软键盘弹不出来的原因之一）。
|
||||
*/
|
||||
class RemoteSurfaceView(context: Context) : SurfaceView(context) {
|
||||
|
||||
/** 由 Activity 实现，转发到 RustBridge */
|
||||
interface Listener {
|
||||
fun sendMouseMove(x: Int, y: Int)
|
||||
fun sendMouseButton(button: Int, down: Boolean)
|
||||
fun sendWheel(dy: Int)
|
||||
/** 协议 v3：一段 UTF-16 code unit（IME 提交结果） */
|
||||
fun sendText(units: CharArray)
|
||||
/** 控制键（Enter/Backspace/方向键等）走扫描码通道 */
|
||||
fun sendKey(key: WinKey, down: Boolean)
|
||||
/** 连接态：只有已连接才声明自己是文本编辑器，避免未连接时误弹键盘 */
|
||||
fun isReadyForInput(): Boolean
|
||||
}
|
||||
|
||||
var listener: Listener? = null
|
||||
|
||||
/** 视频宽高比（w/h）。>0 时视图按此比例测量。 */
|
||||
var videoAspectRatio = 0f
|
||||
set(value) {
|
||||
if (value > 0f && field != value) {
|
||||
field = value
|
||||
requestLayout()
|
||||
}
|
||||
}
|
||||
|
||||
private enum class Mode { IDLE, TAP, DRAG, TWO_FINGER }
|
||||
|
||||
private var mode = Mode.IDLE
|
||||
private var downX = 0f
|
||||
private var downY = 0f
|
||||
private var lastX = 0f
|
||||
private var lastY = 0f
|
||||
private var downTime = 0L
|
||||
private var twoFingerStartY = 0f
|
||||
private var twoFingerDownTime = 0L
|
||||
private var twoFingerMoved = false
|
||||
private var wheelAccum = 0f
|
||||
|
||||
/** 判定「拖拽 vs 点击」的位移阈值：12dp */
|
||||
private val dragSlopPx = resources.displayMetrics.density * 12f
|
||||
|
||||
init {
|
||||
isFocusable = true
|
||||
// IME 需要触摸态焦点；没有这行，点「键盘」按钮系统不会建立输入连接
|
||||
isFocusableInTouchMode = true
|
||||
}
|
||||
|
||||
// ---- IME 通道 ----
|
||||
|
||||
override fun onCheckIsTextEditor(): Boolean = listener?.isReadyForInput() == true
|
||||
|
||||
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
|
||||
val l = listener ?: return null
|
||||
if (!l.isReadyForInput()) return null
|
||||
// TYPE_CLASS_TEXT：让输入法给出正常键盘（含中文候选）。
|
||||
// 常量从 InputType 引用而非 EditorInfo：Kotlin 不保证经实现类名访问接口常量。
|
||||
// 刻意不加 TYPE_TEXT_FLAG_NO_SUGGESTIONS——中文输入依赖候选词。
|
||||
outAttrs.inputType = InputType.TYPE_CLASS_TEXT
|
||||
// 抑制全屏/extract 模式抢走画面（API 29 上两个标志都还在，具体效果真机验证，见 U4/U5）
|
||||
outAttrs.imeOptions =
|
||||
EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
|
||||
// 右下动作键文案。字段名是 actionLabel（javap android-34 核实，不是 imeActionLabel）
|
||||
outAttrs.actionLabel = "发送"
|
||||
return RemoteInputConnection(
|
||||
target = this,
|
||||
sendText = { l.sendText(it) },
|
||||
sendKey = { k, down -> l.sendKey(k, down) },
|
||||
debug = { msg -> onDebug?.invoke(msg) },
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* letterbox 测量：在父容器给出的可用空间内按视频宽高比取最大内接矩形。
|
||||
*
|
||||
* 关键：父容器给 WRAP_CONTENT 时 MeasureSpec 是 AT_MOST，getSize() 是
|
||||
* **上限**而非实际可用尺寸。据此算出的尺寸必须自己 setMeasuredDimension，
|
||||
* 否则父容器会把视图撑满（2026-10-30 实测：视图变成 1080x2001 而非
|
||||
* 1080x607，触摸按 2001 归一化 → 坐标全错，点中心落到画面外）。
|
||||
*/
|
||||
override fun onMeasure(widthSpec: Int, heightSpec: Int) {
|
||||
val ar = videoAspectRatio
|
||||
if (ar <= 0f) {
|
||||
super.onMeasure(widthSpec, heightSpec)
|
||||
return
|
||||
}
|
||||
val availW = MeasureSpec.getSize(widthSpec)
|
||||
val availH = MeasureSpec.getSize(heightSpec)
|
||||
if (availW <= 0 || availH <= 0) {
|
||||
super.onMeasure(widthSpec, heightSpec)
|
||||
return
|
||||
}
|
||||
var w = availW
|
||||
var h = (availW / ar).toInt()
|
||||
if (h > availH) {
|
||||
h = availH
|
||||
w = (availH * ar).toInt()
|
||||
}
|
||||
setMeasuredDimension(w.coerceAtLeast(1), h.coerceAtLeast(1))
|
||||
}
|
||||
|
||||
private fun normX(x: Float) = (x / width * 65535).toInt().coerceIn(0, 65535)
|
||||
private fun normY(y: Float) = (y / height * 65535).toInt().coerceIn(0, 65535)
|
||||
|
||||
private fun moved(fromX: Float, fromY: Float, toX: Float, toY: Float) =
|
||||
hypot((toX - fromX).toDouble(), (toY - fromY).toDouble()) > dragSlopPx
|
||||
|
||||
private fun midX(e: MotionEvent) =
|
||||
if (e.pointerCount >= 2) (e.getX(0) + e.getX(1)) / 2f else e.x
|
||||
|
||||
private fun midY(e: MotionEvent) =
|
||||
if (e.pointerCount >= 2) (e.getY(0) + e.getY(1)) / 2f else e.y
|
||||
|
||||
/** 诊断回调：把触摸情况显示到状态栏（本机 logcat 被系统限流，日志不可靠） */
|
||||
var onDebug: ((String) -> Unit)? = null
|
||||
|
||||
override fun onTouchEvent(event: MotionEvent): Boolean {
|
||||
val l = listener ?: run {
|
||||
onDebug?.invoke("touch: listener=null")
|
||||
return false
|
||||
}
|
||||
if (width == 0 || height == 0) return false
|
||||
val act = when (event.actionMasked) {
|
||||
MotionEvent.ACTION_DOWN -> "DOWN"
|
||||
MotionEvent.ACTION_MOVE -> "MOVE"
|
||||
MotionEvent.ACTION_UP -> "UP"
|
||||
MotionEvent.ACTION_POINTER_DOWN -> "P2DOWN"
|
||||
MotionEvent.ACTION_POINTER_UP -> "P2UP"
|
||||
MotionEvent.ACTION_CANCEL -> "CANCEL"
|
||||
else -> event.actionMasked.toString()
|
||||
}
|
||||
onDebug?.invoke(
|
||||
"$act view=(${event.x.toInt()},${event.y.toInt()}) " +
|
||||
"norm=(${normX(event.x)},${normY(event.y)}) ${width}x$height"
|
||||
)
|
||||
|
||||
when (event.actionMasked) {
|
||||
MotionEvent.ACTION_DOWN -> {
|
||||
downX = event.x; downY = event.y
|
||||
lastX = event.x; lastY = event.y
|
||||
downTime = System.currentTimeMillis()
|
||||
mode = Mode.TAP
|
||||
// 先把远端指针移到位，点击/拖拽都以此为基准
|
||||
l.sendMouseMove(normX(event.x), normY(event.y))
|
||||
}
|
||||
|
||||
MotionEvent.ACTION_POINTER_DOWN -> {
|
||||
// 第二指落下 = 双指手势开始。若已进入拖拽（远端鼠标按着），
|
||||
// 必须先抬起，否则远端会留下一个被按住未释放的左键。
|
||||
if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false)
|
||||
mode = Mode.TWO_FINGER
|
||||
twoFingerMoved = false
|
||||
wheelAccum = 0f
|
||||
twoFingerStartY = midY(event)
|
||||
lastX = midX(event); lastY = midY(event)
|
||||
twoFingerDownTime = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
MotionEvent.ACTION_MOVE -> when (mode) {
|
||||
Mode.TAP -> if (moved(downX, downY, event.x, event.y)) {
|
||||
mode = Mode.DRAG
|
||||
l.sendMouseButton(BTN_LEFT, true)
|
||||
lastX = event.x; lastY = event.y
|
||||
l.sendMouseMove(normX(event.x), normY(event.y))
|
||||
}
|
||||
|
||||
Mode.DRAG -> {
|
||||
lastX = event.x; lastY = event.y
|
||||
l.sendMouseMove(normX(event.x), normY(event.y))
|
||||
}
|
||||
|
||||
Mode.TWO_FINGER -> {
|
||||
val cx = midX(event); val cy = midY(event)
|
||||
if (abs(cy - twoFingerStartY) > dragSlopPx || moved(lastX, lastY, cx, cy)) {
|
||||
twoFingerMoved = true
|
||||
}
|
||||
// 手指下移（cy 增大）= 内容向上翻 = Windows 滚轮向下 → dy 取负
|
||||
wheelAccum += (cy - lastY)
|
||||
val notches = (wheelAccum / WHEEL_PX_PER_NOTCH).toInt()
|
||||
if (notches != 0) {
|
||||
l.sendWheel(-notches * WHEEL_DELTA)
|
||||
wheelAccum -= notches * WHEEL_PX_PER_NOTCH
|
||||
}
|
||||
lastX = cx; lastY = cy
|
||||
}
|
||||
|
||||
Mode.IDLE -> {}
|
||||
}
|
||||
|
||||
MotionEvent.ACTION_POINTER_UP -> {
|
||||
// 双指之一抬起：先判定右键，再把剩余手指的移动视为已结束手势
|
||||
if (mode == Mode.TWO_FINGER) {
|
||||
maybeRightClick(l)
|
||||
mode = Mode.IDLE
|
||||
}
|
||||
}
|
||||
|
||||
MotionEvent.ACTION_UP -> {
|
||||
when (mode) {
|
||||
Mode.TAP -> {
|
||||
// 轻点 = 左键单击（位置已在 DOWN 时发送）
|
||||
l.sendMouseButton(BTN_LEFT, true)
|
||||
l.sendMouseButton(BTN_LEFT, false)
|
||||
performClick()
|
||||
}
|
||||
Mode.DRAG -> l.sendMouseButton(BTN_LEFT, false)
|
||||
Mode.TWO_FINGER -> maybeRightClick(l)
|
||||
Mode.IDLE -> {}
|
||||
}
|
||||
mode = Mode.IDLE
|
||||
}
|
||||
|
||||
MotionEvent.ACTION_CANCEL -> {
|
||||
if (mode == Mode.DRAG) l.sendMouseButton(BTN_LEFT, false)
|
||||
mode = Mode.IDLE
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/** 双指未移动且按住时间短 → 右键 */
|
||||
private fun maybeRightClick(l: Listener) {
|
||||
if (twoFingerMoved) return
|
||||
if (System.currentTimeMillis() - twoFingerDownTime > TWO_FINGER_TAP_MAX_MS) return
|
||||
l.sendMouseMove(normX(lastX), normY(lastY))
|
||||
l.sendMouseButton(BTN_RIGHT, true)
|
||||
l.sendMouseButton(BTN_RIGHT, false)
|
||||
performClick()
|
||||
}
|
||||
|
||||
@Suppress("ClickableViewAccessibility")
|
||||
override fun performClick(): Boolean {
|
||||
super.performClick()
|
||||
return true
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val BTN_LEFT = 0
|
||||
const val BTN_RIGHT = 1
|
||||
const val BTN_MIDDLE = 2
|
||||
private const val WHEEL_DELTA = 120
|
||||
private const val WHEEL_PX_PER_NOTCH = 60f
|
||||
private const val TWO_FINGER_TAP_MAX_MS = 500L
|
||||
}
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
/**
|
||||
* viewer-core（Rust）的 JNI 声明。符号约定：Java_com_verify_remote_RustBridge_*，
|
||||
* 与 android-ffi/src/lib.rs 一一对应。
|
||||
*
|
||||
* 句柄纪律（Rust 侧文件头有完整说明）：
|
||||
* - nativeConnect 返回 >0 句柄；0 表示失败，错误消息取 [lastError]
|
||||
* - destroy 后必须置 0；**destroy 前必须先停掉解码线程**（否则 use-after-free）
|
||||
*/
|
||||
object RustBridge {
|
||||
|
||||
private const val FRAME_HEADER_LEN = 25
|
||||
|
||||
init {
|
||||
System.loadLibrary("androidffi")
|
||||
}
|
||||
|
||||
private external fun nativeConnect(host: String, port: Int, code: String): Long
|
||||
external fun nativeLastError(): String?
|
||||
/** 返回 null = 会话结束；否则 [FRAME_HEADER_LEN] 字节头 + Annex-B 载荷 */
|
||||
external fun nativeNextFrame(handle: Long): ByteArray?
|
||||
external fun nativeStats(handle: Long): Long
|
||||
external fun nativeDesktopSize(handle: Long): Long
|
||||
external fun nativeSendMouseMove(handle: Long, x: Int, y: Int)
|
||||
external fun nativeSendMouseButton(handle: Long, button: Int, down: Int)
|
||||
external fun nativeSendWheel(handle: Long, dy: Int)
|
||||
external fun nativeSendKey(handle: Long, scancode: Int, extended: Int, down: Int)
|
||||
/**
|
||||
* 协议 v3 文本上行。参数刻意用 `CharArray`（UTF-16 code unit）而不是 `String`：
|
||||
* jni 侧 `get_string` 走 Modified-UTF-8 往返，IME 提交边界的落单代理项会在
|
||||
* 往返中丢失；`char[]` 经 `GetCharArrayRegion` 按 u16 原样拷贝。
|
||||
*/
|
||||
external fun nativeSendText(handle: Long, units: CharArray)
|
||||
external fun nativeRequestKeyframe(handle: Long)
|
||||
external fun nativeDestroy(handle: Long)
|
||||
|
||||
/** 连接。失败抛 IllegalStateException（带 Rust 侧错误消息）。 */
|
||||
fun connect(host: String, port: Int, code: String): Long {
|
||||
val h = nativeConnect(host, port, code)
|
||||
if (h == 0L) {
|
||||
throw IllegalStateException(nativeLastError() ?: "connect failed")
|
||||
}
|
||||
return h
|
||||
}
|
||||
|
||||
fun destroy(handle: Long) {
|
||||
if (handle != 0L) nativeDestroy(handle)
|
||||
}
|
||||
|
||||
/** 解出的帧（头字段已拆，payload 可直接喂 MediaCodec） */
|
||||
data class Frame(
|
||||
val keyframe: Boolean,
|
||||
val seq: Long,
|
||||
val tsMs: Long,
|
||||
val width: Int,
|
||||
val height: Int,
|
||||
val payload: ByteArray,
|
||||
)
|
||||
|
||||
/** 取帧 + 拆头。null = 会话结束。 */
|
||||
fun nextFrame(handle: Long): Frame? {
|
||||
val raw = nativeNextFrame(handle) ?: return null
|
||||
if (raw.size < FRAME_HEADER_LEN) return null
|
||||
return Frame(
|
||||
keyframe = raw[0] != 0.toByte(),
|
||||
seq = readLong(raw, 1),
|
||||
tsMs = readLong(raw, 9),
|
||||
width = readInt(raw, 17),
|
||||
height = readInt(raw, 21),
|
||||
payload = raw.copyOfRange(FRAME_HEADER_LEN, raw.size),
|
||||
)
|
||||
}
|
||||
|
||||
fun rttMs(handle: Long): Long {
|
||||
val packed = nativeStats(handle)
|
||||
return if (packed < 0) 0 else (packed ushr 32)
|
||||
}
|
||||
|
||||
fun framesIn(handle: Long): Long {
|
||||
val packed = nativeStats(handle)
|
||||
return if (packed < 0) 0 else packed and 0xFFFF_FFFF
|
||||
}
|
||||
|
||||
/**
|
||||
* 被控端桌面尺寸 (w, h)。连接成功后、建解码器之前调用。
|
||||
* 返回 (0,0) 表示句柄无效。
|
||||
*/
|
||||
fun desktopSize(handle: Long): Pair<Int, Int> {
|
||||
val packed = nativeDesktopSize(handle)
|
||||
if (packed == 0L) return 0 to 0
|
||||
val w = ((packed ushr 32) and 0xFFFF_FFFF).toInt()
|
||||
val h = (packed and 0xFFFF_FFFF).toInt()
|
||||
return w to h
|
||||
}
|
||||
|
||||
private fun readLong(b: ByteArray, off: Int): Long {
|
||||
var v = 0L
|
||||
for (i in 0 until 8) v = v or ((b[off + i].toLong() and 0xFF) shl (8 * i))
|
||||
return v
|
||||
}
|
||||
|
||||
private fun readInt(b: ByteArray, off: Int): Int {
|
||||
return (b[off].toInt() and 0xFF) or
|
||||
((b[off + 1].toInt() and 0xFF) shl 8) or
|
||||
((b[off + 2].toInt() and 0xFF) shl 16) or
|
||||
((b[off + 3].toInt() and 0xFF) shl 24)
|
||||
}
|
||||
}
|
||||
@@ -1,126 +0,0 @@
|
||||
package com.verify.remote
|
||||
|
||||
import android.view.KeyEvent
|
||||
|
||||
/**
|
||||
* Android KeyEvent.KEYCODE_* → Windows PS/2 Set-1 扫描码 + extended 标志。
|
||||
*
|
||||
* **扫描码值直接取自桌面 viewer 自己的映射（`viewer/src/keymap.rs`，项目内部事实）**，
|
||||
* 不是凭记忆写的；Android 侧一律用命名常量 `KeyEvent.KEYCODE_*` 引用，
|
||||
* 数值由编译器解析（已用 `javap -constants android.jar` 验证这些常量在 API 34 存在）。
|
||||
*
|
||||
* 分工（协议 v3 之后）：
|
||||
* - 可打印字符优先走 **Text 通道**（`KeyEvent.unicodeChar` / IME `commitText`），
|
||||
* 布局无关、中文可用；
|
||||
* - 本表服务控制键（Enter/Tab/Backspace/方向键/功能键）与物理键盘，
|
||||
* 走 `KeyDown/KeyUp` 扫描码通道。
|
||||
*
|
||||
* 不做 Shift/Ctrl 组合状态跟踪：软键盘的大写字母由 IME 直接 commit 字符，
|
||||
* 不需要本地修饰键语义。物理键盘组合键（Ctrl+C 等）登记为后续事项。
|
||||
*/
|
||||
data class WinKey(val scancode: Int, val extended: Boolean)
|
||||
|
||||
object WinScancode {
|
||||
|
||||
fun of(keyCode: Int): WinKey? = when (keyCode) {
|
||||
// 字母区（Set-1 make code，同 keymap.rs）
|
||||
KeyEvent.KEYCODE_A -> WinKey(0x1E, false)
|
||||
KeyEvent.KEYCODE_B -> WinKey(0x30, false)
|
||||
KeyEvent.KEYCODE_C -> WinKey(0x2E, false)
|
||||
KeyEvent.KEYCODE_D -> WinKey(0x20, false)
|
||||
KeyEvent.KEYCODE_E -> WinKey(0x12, false)
|
||||
KeyEvent.KEYCODE_F -> WinKey(0x21, false)
|
||||
KeyEvent.KEYCODE_G -> WinKey(0x22, false)
|
||||
KeyEvent.KEYCODE_H -> WinKey(0x23, false)
|
||||
KeyEvent.KEYCODE_I -> WinKey(0x17, false)
|
||||
KeyEvent.KEYCODE_J -> WinKey(0x24, false)
|
||||
KeyEvent.KEYCODE_K -> WinKey(0x25, false)
|
||||
KeyEvent.KEYCODE_L -> WinKey(0x26, false)
|
||||
KeyEvent.KEYCODE_M -> WinKey(0x32, false)
|
||||
KeyEvent.KEYCODE_N -> WinKey(0x31, false)
|
||||
KeyEvent.KEYCODE_O -> WinKey(0x18, false)
|
||||
KeyEvent.KEYCODE_P -> WinKey(0x19, false)
|
||||
KeyEvent.KEYCODE_Q -> WinKey(0x10, false)
|
||||
KeyEvent.KEYCODE_R -> WinKey(0x13, false)
|
||||
KeyEvent.KEYCODE_S -> WinKey(0x1F, false)
|
||||
KeyEvent.KEYCODE_T -> WinKey(0x14, false)
|
||||
KeyEvent.KEYCODE_U -> WinKey(0x16, false)
|
||||
KeyEvent.KEYCODE_V -> WinKey(0x2F, false)
|
||||
KeyEvent.KEYCODE_W -> WinKey(0x11, false)
|
||||
KeyEvent.KEYCODE_X -> WinKey(0x2D, false)
|
||||
KeyEvent.KEYCODE_Y -> WinKey(0x15, false)
|
||||
KeyEvent.KEYCODE_Z -> WinKey(0x2C, false)
|
||||
|
||||
// 数字主键区（1..9, 0；Android keycode 不连续，逐个写）
|
||||
KeyEvent.KEYCODE_1 -> WinKey(0x02, false)
|
||||
KeyEvent.KEYCODE_2 -> WinKey(0x03, false)
|
||||
KeyEvent.KEYCODE_3 -> WinKey(0x04, false)
|
||||
KeyEvent.KEYCODE_4 -> WinKey(0x05, false)
|
||||
KeyEvent.KEYCODE_5 -> WinKey(0x06, false)
|
||||
KeyEvent.KEYCODE_6 -> WinKey(0x07, false)
|
||||
KeyEvent.KEYCODE_7 -> WinKey(0x08, false)
|
||||
KeyEvent.KEYCODE_8 -> WinKey(0x09, false)
|
||||
KeyEvent.KEYCODE_9 -> WinKey(0x0A, false)
|
||||
KeyEvent.KEYCODE_0 -> WinKey(0x0B, false)
|
||||
|
||||
// 符号区（US 布局物理位置，同 keymap.rs）
|
||||
KeyEvent.KEYCODE_SPACE -> WinKey(0x39, false)
|
||||
KeyEvent.KEYCODE_ENTER -> WinKey(0x1C, false)
|
||||
KeyEvent.KEYCODE_DEL -> WinKey(0x0E, false) // Backspace
|
||||
KeyEvent.KEYCODE_FORWARD_DEL -> WinKey(0x53, true) // Delete（扩展）
|
||||
KeyEvent.KEYCODE_TAB -> WinKey(0x0F, false)
|
||||
KeyEvent.KEYCODE_ESCAPE -> WinKey(0x01, false)
|
||||
KeyEvent.KEYCODE_CAPS_LOCK -> WinKey(0x3A, false)
|
||||
KeyEvent.KEYCODE_GRAVE -> WinKey(0x29, false)
|
||||
KeyEvent.KEYCODE_MINUS -> WinKey(0x0C, false)
|
||||
KeyEvent.KEYCODE_EQUALS -> WinKey(0x0D, false)
|
||||
KeyEvent.KEYCODE_LEFT_BRACKET -> WinKey(0x1A, false)
|
||||
KeyEvent.KEYCODE_RIGHT_BRACKET -> WinKey(0x1B, false)
|
||||
KeyEvent.KEYCODE_BACKSLASH -> WinKey(0x2B, false)
|
||||
KeyEvent.KEYCODE_SEMICOLON -> WinKey(0x27, false)
|
||||
KeyEvent.KEYCODE_APOSTROPHE -> WinKey(0x28, false)
|
||||
KeyEvent.KEYCODE_COMMA -> WinKey(0x33, false)
|
||||
KeyEvent.KEYCODE_PERIOD -> WinKey(0x34, false)
|
||||
KeyEvent.KEYCODE_SLASH -> WinKey(0x35, false)
|
||||
|
||||
// 修饰键
|
||||
KeyEvent.KEYCODE_SHIFT_LEFT -> WinKey(0x2A, false)
|
||||
// 右 Shift **不是**扩展键：微软 Keyboard Input Overview 明确
|
||||
// "The right-hand Shift key is not considered an extended key either;
|
||||
// it has a separate scan code (0x36) instead."（本会话实读该页核实）
|
||||
KeyEvent.KEYCODE_SHIFT_RIGHT -> WinKey(0x36, false)
|
||||
KeyEvent.KEYCODE_CTRL_LEFT -> WinKey(0x1D, false)
|
||||
KeyEvent.KEYCODE_CTRL_RIGHT -> WinKey(0x1D, true)
|
||||
KeyEvent.KEYCODE_ALT_LEFT -> WinKey(0x38, false)
|
||||
KeyEvent.KEYCODE_ALT_RIGHT -> WinKey(0x38, true)
|
||||
KeyEvent.KEYCODE_META_LEFT -> WinKey(0x5B, true)
|
||||
KeyEvent.KEYCODE_META_RIGHT -> WinKey(0x5C, true)
|
||||
|
||||
// 导航区（全部扩展键）
|
||||
KeyEvent.KEYCODE_DPAD_UP -> WinKey(0x48, true)
|
||||
KeyEvent.KEYCODE_DPAD_DOWN -> WinKey(0x50, true)
|
||||
KeyEvent.KEYCODE_DPAD_LEFT -> WinKey(0x4B, true)
|
||||
KeyEvent.KEYCODE_DPAD_RIGHT -> WinKey(0x4D, true)
|
||||
KeyEvent.KEYCODE_MOVE_HOME -> WinKey(0x47, true)
|
||||
KeyEvent.KEYCODE_MOVE_END -> WinKey(0x4F, true)
|
||||
KeyEvent.KEYCODE_PAGE_UP -> WinKey(0x49, true)
|
||||
KeyEvent.KEYCODE_PAGE_DOWN -> WinKey(0x51, true)
|
||||
KeyEvent.KEYCODE_INSERT -> WinKey(0x52, true)
|
||||
|
||||
// 功能键（F11/F12 的 0x57/0x58 与 linux-agent keycode_of 注释同源）
|
||||
KeyEvent.KEYCODE_F1 -> WinKey(0x3B, false)
|
||||
KeyEvent.KEYCODE_F2 -> WinKey(0x3C, false)
|
||||
KeyEvent.KEYCODE_F3 -> WinKey(0x3D, false)
|
||||
KeyEvent.KEYCODE_F4 -> WinKey(0x3E, false)
|
||||
KeyEvent.KEYCODE_F5 -> WinKey(0x3F, false)
|
||||
KeyEvent.KEYCODE_F6 -> WinKey(0x40, false)
|
||||
KeyEvent.KEYCODE_F7 -> WinKey(0x41, false)
|
||||
KeyEvent.KEYCODE_F8 -> WinKey(0x42, false)
|
||||
KeyEvent.KEYCODE_F9 -> WinKey(0x43, false)
|
||||
KeyEvent.KEYCODE_F10 -> WinKey(0x44, false)
|
||||
KeyEvent.KEYCODE_F11 -> WinKey(0x57, false)
|
||||
KeyEvent.KEYCODE_F12 -> WinKey(0x58, false)
|
||||
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
@@ -1,85 +0,0 @@
|
||||
# Phase 1 build chain: Rust .so -> jniLibs -> Gradle debug APK -> install -> launch
|
||||
#
|
||||
# Keep this file ASCII-only (PS 5.1 reads BOM-less .ps1 as GBK; see lessons).
|
||||
#
|
||||
# Usage (from apps/android-viewer/):
|
||||
# powershell -ExecutionPolicy Bypass -File build-android.ps1 [-Host_ 192.168.1.5] [-Port 47800] [-Code t] [-SkipRust]
|
||||
# (param is Host_ because $Host is a PowerShell automatic variable)
|
||||
#
|
||||
# Prereqs (Windows dev box):
|
||||
# - Rust + Android targets + cargo-ndk + NDK (see personal docs/baseline.md toolchain table)
|
||||
# - JDK 17 + Gradle 8.14 (paths in $vars below, adjust to your machine)
|
||||
# - local.properties with sdk.dir (gitignored; template: sdk.dir=<android sdk path>)
|
||||
|
||||
param(
|
||||
[string]$Host_ = "192.168.1.5",
|
||||
[int]$Port = 47800,
|
||||
[string]$Code = "t",
|
||||
[switch]$SkipRust
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# repo root = two levels up from this script (apps/android-viewer -> repo)
|
||||
$proj = $PSScriptRoot
|
||||
$repo = (Resolve-Path "$proj\..\..").Path
|
||||
$env_ = "D:\working\environment"
|
||||
$gradle = "D:\working\android\gradle\gradle-8.14.3\bin\gradle.bat"
|
||||
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
|
||||
|
||||
$env:RUSTUP_HOME = "$env_\rust"
|
||||
$env:CARGO_HOME = "$env_\cargo"
|
||||
$env:PATH = "$env_\mingw64\bin;$env_\cargo\bin;$env:PATH"
|
||||
$env:ANDROID_HOME = "D:\working\android\sdk"
|
||||
$env:ANDROID_NDK_HOME = "D:\working\android\sdk\ndk\30.0.16248370"
|
||||
$env:GRADLE_USER_HOME = "D:\working\android\gradle-home"
|
||||
$env:JAVA_HOME = "D:\working\java\jdk17"
|
||||
|
||||
if (-not (Test-Path "$proj\local.properties")) {
|
||||
Set-Content "$proj\local.properties" 'sdk.dir=D\:\\working\\android\\sdk' -Encoding ascii
|
||||
Write-Host "generated local.properties (sdk.dir)"
|
||||
}
|
||||
|
||||
if (-not $SkipRust) {
|
||||
Write-Host "=== 1/4 cargo ndk: android-ffi (arm64 release) ===" -ForegroundColor Cyan
|
||||
Push-Location $repo
|
||||
# cargo writes progress to stderr; PS 5.1 + ErrorActionPreference=Stop turns
|
||||
# every stderr line into a terminating error. Relax for native calls and
|
||||
# judge by exit code (same for gradle below).
|
||||
$ErrorActionPreference = "Continue"
|
||||
cargo ndk -t arm64-v8a build -p android-ffi --release 2>&1 |
|
||||
Select-String "error|Finished|Building"
|
||||
if ($LASTEXITCODE -ne 0) { throw "cargo ndk build failed (exit $LASTEXITCODE)" }
|
||||
$ErrorActionPreference = "Stop"
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
Write-Host "=== 2/4 copy .so into app/libs ===" -ForegroundColor Cyan
|
||||
$so = "$repo\target\aarch64-linux-android\release\libandroidffi.so"
|
||||
if (-not (Test-Path $so)) { throw "missing $so (run without -SkipRust)" }
|
||||
New-Item -ItemType Directory -Force -Path "$proj\app\libs\arm64-v8a" | Out-Null
|
||||
Copy-Item $so "$proj\app\libs\arm64-v8a\libandroidffi.so" -Force
|
||||
Write-Host (" libandroidffi.so: {0:N0} KB" -f ((Get-Item $so).Length/1KB))
|
||||
|
||||
Write-Host "=== 3/4 gradle assembleDebug ===" -ForegroundColor Cyan
|
||||
Push-Location $proj
|
||||
$ErrorActionPreference = "Continue"
|
||||
& $gradle :app:assembleDebug --console=plain 2>&1 | Select-Object -Last 25
|
||||
$code = $LASTEXITCODE
|
||||
$ErrorActionPreference = "Stop"
|
||||
Pop-Location
|
||||
if ($code -ne 0) { throw "gradle build failed (exit $code)" }
|
||||
|
||||
$apk = "$proj\app\build\outputs\apk\debug\app-debug.apk"
|
||||
if (-not (Test-Path $apk)) { throw "apk not found: $apk" }
|
||||
Write-Host (" APK: {0:N0} KB" -f ((Get-Item $apk).Length/1KB))
|
||||
|
||||
Write-Host "=== 4/4 install + launch ===" -ForegroundColor Cyan
|
||||
$ErrorActionPreference = "Continue"
|
||||
& $adb install -r $apk
|
||||
if ($LASTEXITCODE -ne 0) { throw "install failed (phone must allow USB install)" }
|
||||
& $adb logcat -c | Out-Null
|
||||
& $adb shell am start -n com.verify.remote/.MainActivity `
|
||||
--es host $Host_ --ei port $Port --es code $Code | Out-Null
|
||||
$ErrorActionPreference = "Stop"
|
||||
Write-Host "launched. watch: adb logcat -s MainActivity:H264Decoder RustBridge:* viewer-core:* AndroidRuntime:E"
|
||||
Write-Host "after 10s capture screen: adb exec-out screencap -p > screen.png"
|
||||
@@ -1,4 +0,0 @@
|
||||
plugins {
|
||||
id("com.android.application") version "8.9.2" apply false
|
||||
id("org.jetbrains.kotlin.android") version "2.1.20" apply false
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
# 固定 JDK 17（不动全局 JAVA_HOME，遵守 2026-10-29 环境约束）
|
||||
org.gradle.java.home=D\:\\working\\java\\jdk17
|
||||
android.useAndroidX=false
|
||||
org.gradle.jvmargs=-Xmx2g
|
||||
@@ -1,17 +0,0 @@
|
||||
pluginManagement {
|
||||
repositories {
|
||||
google()
|
||||
mavenCentral()
|
||||
gradlePluginPortal()
|
||||
}
|
||||
}
|
||||
dependencyResolutionManagement {
|
||||
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
|
||||
repositories {
|
||||
google()
|
||||
mavenCentral()
|
||||
}
|
||||
}
|
||||
|
||||
rootProject.name = "remote-viewer"
|
||||
include(":app")
|
||||
+10
-445
@@ -1,45 +1,21 @@
|
||||
//! 通用协议与帧封装：控制消息（bincode + 长度前缀）与视频帧格式。
|
||||
|
||||
/// P2P 打洞信令协议（信令通道专用，不含中继；三端共享契约）
|
||||
pub mod signaling;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
|
||||
/// 媒体/控制协议版本。
|
||||
///
|
||||
/// **改动规则**：任何对 `ControlMsg` 或 `VideoFrameHeader` 字段的增删改都必须递增此值。
|
||||
/// 理由：线格式是 bincode（按位置编码，无字段名），新增字段会让旧端解析失败或错位，
|
||||
/// 且失败方式可能是静默的错误数据而非明确报错。递增后由 `Auth` 握手显式拒绝旧端，
|
||||
/// 把"连上了但行为诡异"变成"连不上且提示升级"。
|
||||
///
|
||||
/// - v1：初版（无版本字段，隐式）
|
||||
/// - v2：`Auth`/`AuthOk` 携带版本；帧头增加 `seq`/`keyframe`
|
||||
/// - v3：新增 `InputEvent::Text`（文本注入通道，UTF-16 code unit）。
|
||||
/// `InputEvent` 因携带 `Vec<u16>` 不再是 `Copy`；三端必须同步升级，
|
||||
/// 旧端由 `Auth` 握手显式拒绝。
|
||||
pub const PROTO_VERSION: u16 = 3;
|
||||
|
||||
/// 控制通道消息（双向）
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub enum ControlMsg {
|
||||
/// 主控端认证（临时验证码，MVP 明文走 QUIC-TLS 加密通道）。
|
||||
/// `proto_version` 必须等于 [`PROTO_VERSION`]，否则被控端回 `AuthFail` 并拒绝。
|
||||
Auth { code: String, proto_version: u16 },
|
||||
/// 认证通过。`proto_version` 回显被控端的版本，供主控端确认双方一致。
|
||||
AuthOk {
|
||||
width: u32,
|
||||
height: u32,
|
||||
proto_version: u16,
|
||||
},
|
||||
/// 主控端认证（临时验证码，MVP 明文走 QUIC-TLS 加密通道）
|
||||
Auth { code: String },
|
||||
AuthOk { width: u32, height: u32 },
|
||||
AuthFail(String),
|
||||
KeyFrameReq,
|
||||
Ping(u64),
|
||||
/// t=主控端发送时刻（主控时钟），agent_ts=被控端收到时刻（被控时钟）
|
||||
/// 两者配合可估计时钟偏移，进而用帧头 ts_ms 计算端到端延迟
|
||||
/// lost/sent = agent 发送方向（视频下行）的 QUIC 丢包/发包累计计数（conn.stats().path）
|
||||
Pong { t: u64, agent_ts: u64, lost: u64, sent: u64 },
|
||||
Pong { t: u64, agent_ts: u64 },
|
||||
Input(InputEvent),
|
||||
}
|
||||
|
||||
@@ -50,12 +26,9 @@ pub enum MouseButton {
|
||||
Middle,
|
||||
}
|
||||
|
||||
/// 输入事件。鼠标坐标为**被采集显示器**（主屏）归一化 0..=65535
|
||||
/// （协议 v2 语义，v3 未改；`agent/src/input.rs` 的注入空间与此一致，见个人层 lessons R-4）；
|
||||
/// 输入事件。鼠标坐标为虚拟桌面归一化 0..=65535；
|
||||
/// 按键为 Set-1 扫描码，extended 标记扩展键（方向键/Home 等，需要 EXTENDEDKEY 标志）。
|
||||
///
|
||||
/// v3 起本类型不再 `Copy`：`Text` 携带堆上 `Vec<u16>`。需要复制的地方用 `.clone()`。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
|
||||
pub enum InputEvent {
|
||||
MouseMove { x: u16, y: u16 },
|
||||
MouseDown(MouseButton),
|
||||
@@ -63,60 +36,14 @@ pub enum InputEvent {
|
||||
Wheel { dy: i32 },
|
||||
KeyDown { scancode: u16, extended: bool },
|
||||
KeyUp { scancode: u16, extended: bool },
|
||||
/// v3 新增：**文本注入通道**。`units` 为 UTF-16 code unit 序列。
|
||||
///
|
||||
/// 为什么不是 `String`：Windows 注入侧（`KEYEVENTF_UNICODE`）与 Java 字符串的
|
||||
/// 原生表示都是 UTF-16 code unit；用 `String` 要走 UTF-8 往返，遇到 IME 提交边界
|
||||
/// 产生的落单代理项会**有损**。`Vec<u16>` 忠实透传，由接收端按 Win32 语义处理。
|
||||
///
|
||||
/// 为什么整段携带、而不是逐 code unit 一条消息：一次 IME 候选确认常是多个汉字；
|
||||
/// 代理对（emoji 等增补平面字符）**必须在同一次注入批次里成对提交**才能被目标
|
||||
/// 应用合成成一个字符，拆开注入必现乱码。
|
||||
///
|
||||
/// 上限 [`MAX_TEXT_UNITS`]：超长输入截断且两侧打警告日志（不静默丢弃）。
|
||||
/// 另一道防线是 [`read_control`] 的 1 MiB 消息长度闸。
|
||||
///
|
||||
/// 日志纪律：中间层与注入端只允许打印 `units.len()`，**不得**打印用户输入内容。
|
||||
Text { units: Vec<u16> },
|
||||
}
|
||||
|
||||
/// [`InputEvent::Text`] 单次携带的 code unit 上限。
|
||||
///
|
||||
/// 与 `read_control` 的 1 MiB 消息闸是两道独立防线：前者防恶意对端把接收端分配打爆，
|
||||
/// 后者防合法但异常长的 IME 提交把注入队列刷爆。4096 unit ≈ 8 KiB，
|
||||
/// 远超正常 IME 一次提交（数十 unit），又足够小使单次 `SendInput` 批量仍可接受。
|
||||
pub const MAX_TEXT_UNITS: usize = 4096;
|
||||
|
||||
/// 把文本输入截断到 [`MAX_TEXT_UNITS`]，返回 `(可注入的 units, 是否发生截断)`。
|
||||
///
|
||||
/// 调用方**必须**在截断为真时打印日志（可见后果）；本函数只负责机械截断，
|
||||
/// 不自己打印——因为 `common` 同时被主控端与被控端复用，日志口径由调用方决定。
|
||||
pub fn clamp_text_units(units: Vec<u16>) -> (Vec<u16>, bool) {
|
||||
if units.len() > MAX_TEXT_UNITS {
|
||||
let mut trimmed = units;
|
||||
trimmed.truncate(MAX_TEXT_UNITS);
|
||||
(trimmed, true)
|
||||
} else {
|
||||
(units, false)
|
||||
}
|
||||
}
|
||||
|
||||
/// 视频帧头。载荷为 H.264 Annex-B 码流（含 00 00 00 01 起始码）。
|
||||
///
|
||||
/// 字段语义：
|
||||
/// - `width` / `height`：被控端桌面尺寸（逻辑尺寸，非编码对齐后的尺寸）
|
||||
/// - `ts_ms`：被控端采集时刻（被控时钟），配合 Ping/Pong 估计的时钟偏移算端到端延迟
|
||||
/// - `seq`：帧序号，从 0 单调递增，每发一帧 +1（含静止桌面的重复帧）。
|
||||
/// 用于丢帧检测与弱网统计；注意它统计的是"发送序"，不是"显示序"
|
||||
/// - `keyframe`：本帧是否 IDR。解码端（尤其 MediaCodec）需要它决定是否重置解码器，
|
||||
/// 不必再自己扫 NAL 起始码判断
|
||||
/// 视频帧头（载荷为 LZ4 压缩的 BGRA 像素，按行紧凑排列）
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct VideoFrameHeader {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub ts_ms: u64,
|
||||
pub seq: u64,
|
||||
pub keyframe: bool,
|
||||
}
|
||||
|
||||
async fn read_u32le<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<u32> {
|
||||
@@ -150,9 +77,7 @@ pub async fn read_control<R: AsyncReadExt + Unpin>(r: &mut R) -> Result<ControlM
|
||||
Ok(bincode::deserialize(&buf)?)
|
||||
}
|
||||
|
||||
/// 编码视频帧：[总长 u32][头长 u32][bincode 头][H.264 载荷]
|
||||
///
|
||||
/// `总长` 不含自身这 4 字节，等于 `4 + 头长 + 载荷长`。
|
||||
/// 编码视频帧：[总长 u32][头长 u32][bincode 头][LZ4 载荷]
|
||||
pub fn encode_video_frame(header: &VideoFrameHeader, payload: &[u8]) -> Vec<u8> {
|
||||
let hb = bincode::serialize(header).expect("serialize header");
|
||||
let total = 4 + hb.len() + payload.len();
|
||||
@@ -164,374 +89,14 @@ pub fn encode_video_frame(header: &VideoFrameHeader, payload: &[u8]) -> Vec<u8>
|
||||
out
|
||||
}
|
||||
|
||||
/// 单帧总长上限（256 MiB）。超过即视为协议错误或恶意输入。
|
||||
const MAX_VIDEO_FRAME: usize = 256 << 20;
|
||||
|
||||
/// 判断一段 Annex-B H.264 码流里是否含 IDR 片（NAL 类型 5）。
|
||||
///
|
||||
/// 用途：填充 [`VideoFrameHeader::keyframe`]。
|
||||
/// 为什么不问编码器要这个标志——NVENC 与 openh264 的 API 都能间接推出，
|
||||
/// 但两者规则不同（首帧强制、`force_intra_frame`、GOP 自动 IDR 交织），
|
||||
/// 而**实际输出**是唯一可信来源。直接看码流，两种编码器一套逻辑。
|
||||
///
|
||||
/// 只识别 3/4 字节起始码；`start-code emulation prevention` 由编码器保证，
|
||||
/// 因此不会在 SPS/PPS 载荷里误判出起始码。
|
||||
pub fn h264_contains_idr(payload: &[u8]) -> bool {
|
||||
let mut i = 0usize;
|
||||
while i + 3 < payload.len() {
|
||||
let sc = if payload[i..].starts_with(&[0, 0, 0, 1]) {
|
||||
4
|
||||
} else if payload[i..].starts_with(&[0, 0, 1]) {
|
||||
3
|
||||
} else {
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
match payload.get(i + sc) {
|
||||
// nal_unit_type 占首字节低 5 位
|
||||
Some(&b) if b & 0x1F == 5 => return true,
|
||||
Some(_) => i += sc + 1,
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
/// 读一个视频帧。
|
||||
///
|
||||
/// 对端数据一律视为不可信输入：`total` 与 `hlen` 都必须做**上下界**校验，
|
||||
/// 否则构造一个 `total < 4` 的帧就能让接收端在切片处 panic（进程级拒绝服务）。
|
||||
pub async fn read_video_frame<R: AsyncReadExt + Unpin>(
|
||||
r: &mut R,
|
||||
) -> Result<(VideoFrameHeader, Vec<u8>)> {
|
||||
let total = read_u32le(r).await? as usize;
|
||||
// 下界：至少要容得下「头长字段(4) + 一个非空的 bincode 头」。
|
||||
// 上界：防止对端声明超大长度导致巨量分配。
|
||||
anyhow::ensure!(
|
||||
(4..MAX_VIDEO_FRAME).contains(&total),
|
||||
"video frame length out of range: {total}"
|
||||
);
|
||||
|
||||
anyhow::ensure!(total < 256 << 20, "video frame too large: {total}");
|
||||
let mut buf = vec![0u8; total];
|
||||
r.read_exact(&mut buf).await?;
|
||||
|
||||
let hlen = u32::from_le_bytes(buf[0..4].try_into().expect("已校验 total >= 4")) as usize;
|
||||
// hlen 必须落在 [1, total-4] 内：0 无法反序列化，超出则切片越界
|
||||
anyhow::ensure!(
|
||||
hlen > 0 && hlen <= total - 4,
|
||||
"video header length invalid: hlen={hlen}, total={total}"
|
||||
);
|
||||
|
||||
let hlen = u32::from_le_bytes(buf[0..4].try_into().unwrap()) as usize;
|
||||
let header: VideoFrameHeader = bincode::deserialize(&buf[4..4 + hlen])?;
|
||||
Ok((header, buf[4 + hlen..].to_vec()))
|
||||
}
|
||||
|
||||
/// 把一个已编码的完整帧拆回 `(帧头, 载荷)`。
|
||||
///
|
||||
/// 用途：静止桌面重发时，被控端需要复用上一帧的**载荷**但重建**帧头**
|
||||
/// （`seq` 必须递增、`ts_ms` 必须是本次发送时刻）。直接 `clone` 整个帧会让
|
||||
/// 这两个字段停在上一次发送的时刻，破坏 `seq` 单调递增契约。
|
||||
///
|
||||
/// 只接受本模块 [`encode_video_frame`] 产出的字节；格式不符返回 `Err`。
|
||||
pub fn split_video_frame(frame: &[u8]) -> Result<(VideoFrameHeader, &[u8])> {
|
||||
anyhow::ensure!(frame.len() >= 8, "frame too short: {}", frame.len());
|
||||
let hlen = u32::from_le_bytes(frame[4..8].try_into().expect("已校验 len >= 8")) as usize;
|
||||
anyhow::ensure!(
|
||||
hlen > 0 && hlen <= frame.len() - 8,
|
||||
"header length invalid: hlen={hlen}, len={}",
|
||||
frame.len()
|
||||
);
|
||||
let header: VideoFrameHeader = bincode::deserialize(&frame[8..8 + hlen])?;
|
||||
Ok((header, &frame[8 + hlen..]))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tokio::io::duplex;
|
||||
|
||||
fn hdr(seq: u64, keyframe: bool) -> VideoFrameHeader {
|
||||
VideoFrameHeader {
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
ts_ms: 12345,
|
||||
seq,
|
||||
keyframe,
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 控制消息 ----
|
||||
|
||||
#[test]
|
||||
fn control_roundtrip_preserves_version() {
|
||||
let m = ControlMsg::Auth {
|
||||
code: "123456".into(),
|
||||
proto_version: PROTO_VERSION,
|
||||
};
|
||||
let bytes = encode_control(&m);
|
||||
// 长度前缀 + 载荷
|
||||
let len = u32::from_le_bytes(bytes[0..4].try_into().unwrap()) as usize;
|
||||
assert_eq!(len, bytes.len() - 4);
|
||||
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
|
||||
match back {
|
||||
ControlMsg::Auth { code, proto_version } => {
|
||||
assert_eq!(code, "123456");
|
||||
assert_eq!(proto_version, PROTO_VERSION);
|
||||
}
|
||||
other => panic!("期望 Auth，实际 {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn control_write_read_roundtrip() {
|
||||
let (mut a, mut b) = duplex(64 * 1024);
|
||||
let msgs = [
|
||||
ControlMsg::Auth {
|
||||
code: "x".into(),
|
||||
proto_version: PROTO_VERSION,
|
||||
},
|
||||
ControlMsg::AuthOk {
|
||||
width: 800,
|
||||
height: 600,
|
||||
proto_version: PROTO_VERSION,
|
||||
},
|
||||
ControlMsg::KeyFrameReq,
|
||||
ControlMsg::Ping(42),
|
||||
ControlMsg::Pong {
|
||||
t: 1,
|
||||
agent_ts: 2,
|
||||
lost: 3,
|
||||
sent: 4,
|
||||
},
|
||||
];
|
||||
for m in &msgs {
|
||||
write_control(&mut a, m).await.unwrap();
|
||||
let got = read_control(&mut b).await.unwrap();
|
||||
assert_eq!(format!("{got:?}"), format!("{m:?}"));
|
||||
}
|
||||
}
|
||||
|
||||
// ---- v3 文本注入通道 ----
|
||||
|
||||
/// 版本守卫：v3 是本轮契约变更的登记点，改动必须显式经过这里。
|
||||
#[test]
|
||||
fn proto_version_is_three_after_text_event() {
|
||||
assert_eq!(PROTO_VERSION, 3, "新增 InputEvent::Text 后必须递增到 3");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_event_roundtrips_utf16_units() {
|
||||
// 「你好」+ 一个 emoji 的代理对（U+1F600 = D83D DE00）
|
||||
let units: Vec<u16> = vec![0x4F60, 0x597D, 0xD83D, 0xDE00];
|
||||
let m = ControlMsg::Input(InputEvent::Text {
|
||||
units: units.clone(),
|
||||
});
|
||||
let bytes = encode_control(&m);
|
||||
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
|
||||
match back {
|
||||
ControlMsg::Input(InputEvent::Text { units: got }) => assert_eq!(got, units),
|
||||
other => panic!("期望 Input::Text，实际 {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// 落单代理项必须原样往返：IME 提交边界可能给出孤立高/低代理项，
|
||||
/// 用 `String` 承载会在这里有损（这正是选 `Vec<u16>` 的理由）。
|
||||
#[test]
|
||||
fn text_event_keeps_lone_surrogate() {
|
||||
let units = vec![0xD83Du16];
|
||||
let m = ControlMsg::Input(InputEvent::Text { units: units.clone() });
|
||||
let bytes = encode_control(&m);
|
||||
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
|
||||
match back {
|
||||
ControlMsg::Input(InputEvent::Text { units: got }) => assert_eq!(got, units),
|
||||
other => panic!("期望 Input::Text，实际 {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_units_clamped_at_boundary() {
|
||||
// 边界：恰好上限不截断；超一个才截断
|
||||
let exact = vec![0x61u16; MAX_TEXT_UNITS];
|
||||
let (kept, truncated) = clamp_text_units(exact);
|
||||
assert_eq!(kept.len(), MAX_TEXT_UNITS);
|
||||
assert!(!truncated);
|
||||
|
||||
let over = vec![0x61u16; MAX_TEXT_UNITS + 1];
|
||||
let (kept, truncated) = clamp_text_units(over);
|
||||
assert_eq!(kept.len(), MAX_TEXT_UNITS);
|
||||
assert!(truncated, "超限必须返回截断标志，让调用方打印警告");
|
||||
|
||||
let (empty, truncated) = clamp_text_units(vec![]);
|
||||
assert!(empty.is_empty());
|
||||
assert!(!truncated);
|
||||
}
|
||||
|
||||
/// 空 `Text` 也必须能往返（IME 可能提交空串；调用方负责决定忽略还是报错）。
|
||||
#[test]
|
||||
fn text_event_empty_roundtrips() {
|
||||
let m = ControlMsg::Input(InputEvent::Text { units: vec![] });
|
||||
let bytes = encode_control(&m);
|
||||
let back: ControlMsg = bincode::deserialize(&bytes[4..]).unwrap();
|
||||
match back {
|
||||
ControlMsg::Input(InputEvent::Text { units }) => assert!(units.is_empty()),
|
||||
other => panic!("期望 Input::Text，实际 {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// 上限内的 `Text` 必须能通过 `read_control` 的长度闸；
|
||||
/// 反向验证：伪造一个超限的声明长度必须在分配前被拒。
|
||||
#[tokio::test]
|
||||
async fn text_within_cap_passes_control_length_gate() {
|
||||
let (mut a, mut b) = duplex(1 << 20);
|
||||
let msg = ControlMsg::Input(InputEvent::Text {
|
||||
units: vec![0x4F60; MAX_TEXT_UNITS],
|
||||
});
|
||||
write_control(&mut a, &msg).await.unwrap();
|
||||
match read_control(&mut b).await.unwrap() {
|
||||
ControlMsg::Input(InputEvent::Text { units }) => assert_eq!(units.len(), MAX_TEXT_UNITS),
|
||||
other => panic!("期望 Input::Text，实际 {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 视频帧头 ----
|
||||
|
||||
#[test]
|
||||
fn video_header_roundtrip_keeps_seq_and_keyframe() {
|
||||
for (seq, kf) in [(0u64, true), (1, false), (u64::MAX, false)] {
|
||||
let h = hdr(seq, kf);
|
||||
let payload = b"\x00\x00\x00\x01\x65fake-nal";
|
||||
let bytes = encode_video_frame(&h, payload);
|
||||
|
||||
let total = u32::from_le_bytes(bytes[0..4].try_into().unwrap()) as usize;
|
||||
assert_eq!(total, bytes.len() - 4, "总长字段应不含自身");
|
||||
|
||||
let hlen = u32::from_le_bytes(bytes[4..8].try_into().unwrap()) as usize;
|
||||
let back: VideoFrameHeader = bincode::deserialize(&bytes[8..8 + hlen]).unwrap();
|
||||
assert_eq!(back.seq, seq);
|
||||
assert_eq!(back.keyframe, kf);
|
||||
assert_eq!(&bytes[8 + hlen..], payload);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn video_frame_write_read_roundtrip() {
|
||||
let (mut a, mut b) = duplex(256 * 1024);
|
||||
let h = hdr(7, true);
|
||||
let payload = vec![0xABu8; 4096];
|
||||
let bytes = encode_video_frame(&h, &payload);
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &bytes).await.unwrap();
|
||||
|
||||
let (h2, p2) = read_video_frame(&mut b).await.unwrap();
|
||||
assert_eq!(h2.seq, 7);
|
||||
assert!(h2.keyframe);
|
||||
assert_eq!((h2.width, h2.height), (1920, 1080));
|
||||
assert_eq!(p2, payload);
|
||||
}
|
||||
|
||||
// ---- 不可信输入（P3 的核心回归） ----
|
||||
|
||||
/// `total < 4` 必须报错而不是 panic：旧实现在这里 `buf[0..4]` 会越界 panic。
|
||||
#[tokio::test]
|
||||
async fn rejects_total_below_minimum() {
|
||||
for total in [0u32, 1, 3] {
|
||||
let (mut a, mut b) = duplex(64);
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &total.to_le_bytes())
|
||||
.await
|
||||
.unwrap();
|
||||
let err = read_video_frame(&mut b).await.unwrap_err().to_string();
|
||||
assert!(
|
||||
err.contains("out of range"),
|
||||
"total={total} 应被拒绝，实际错误: {err}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// `total == 4` 但没有头数据：hlen 读出来是 0，必须拒绝。
|
||||
#[tokio::test]
|
||||
async fn rejects_zero_header_length() {
|
||||
let (mut a, mut b) = duplex(64);
|
||||
let mut bytes = 4u32.to_le_bytes().to_vec();
|
||||
bytes.extend_from_slice(&0u32.to_le_bytes());
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &bytes)
|
||||
.await
|
||||
.unwrap();
|
||||
let err = read_video_frame(&mut b).await.unwrap_err().to_string();
|
||||
assert!(err.contains("header length invalid"), "实际错误: {err}");
|
||||
}
|
||||
|
||||
/// hlen 声明超过 total-4：旧实现会切片越界 panic。
|
||||
#[tokio::test]
|
||||
async fn rejects_header_length_beyond_frame() {
|
||||
let (mut a, mut b) = duplex(64);
|
||||
// total = 8 → body 必须正好 8 字节（hlen 字段 4 + 4 字节填充）
|
||||
let mut bytes = 8u32.to_le_bytes().to_vec();
|
||||
bytes.extend_from_slice(&999u32.to_le_bytes()); // hlen = 999 > total-4 = 4
|
||||
bytes.extend_from_slice(&[0u8; 4]);
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &bytes)
|
||||
.await
|
||||
.unwrap();
|
||||
let err = read_video_frame(&mut b).await.unwrap_err().to_string();
|
||||
assert!(err.contains("header length invalid"), "实际错误: {err}");
|
||||
}
|
||||
|
||||
/// 超过 256 MiB 上限必须在**分配前**拒绝，不能先 vec! 再检查。
|
||||
#[tokio::test]
|
||||
async fn rejects_oversized_frame_without_allocating() {
|
||||
let (mut a, mut b) = duplex(64);
|
||||
let total = (MAX_VIDEO_FRAME as u32) + 1;
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &total.to_le_bytes())
|
||||
.await
|
||||
.unwrap();
|
||||
let err = read_video_frame(&mut b).await.unwrap_err().to_string();
|
||||
assert!(err.contains("out of range"), "实际错误: {err}");
|
||||
}
|
||||
|
||||
/// 头部字节损坏（非合法 bincode）应返回 Err 而非 panic。
|
||||
#[tokio::test]
|
||||
async fn rejects_corrupt_header_bytes() {
|
||||
let (mut a, mut b) = duplex(64);
|
||||
let garbage = [0xFFu8; 8];
|
||||
let total = (4 + garbage.len()) as u32;
|
||||
let mut bytes = total.to_le_bytes().to_vec();
|
||||
bytes.extend_from_slice(&(garbage.len() as u32).to_le_bytes());
|
||||
bytes.extend_from_slice(&garbage);
|
||||
tokio::io::AsyncWriteExt::write_all(&mut a, &bytes)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(read_video_frame(&mut b).await.is_err());
|
||||
}
|
||||
|
||||
// ---- IDR 识别（P2 的 keyframe 标志来源） ----
|
||||
|
||||
#[test]
|
||||
fn detects_idr_in_annex_b() {
|
||||
// SPS(7) + PPS(8) + IDR 片(5)，4 字节起始码
|
||||
let idr = [
|
||||
0, 0, 0, 1, 0x67, 0xAA, // SPS
|
||||
0, 0, 0, 1, 0x68, 0xBB, // PPS
|
||||
0, 0, 0, 1, 0x65, 0xCC, // IDR
|
||||
];
|
||||
assert!(h264_contains_idr(&idr));
|
||||
|
||||
// 3 字节起始码也应识别
|
||||
let idr3 = [0, 0, 1, 0x65, 0xCC];
|
||||
assert!(h264_contains_idr(&idr3));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_non_idr_payloads() {
|
||||
// 纯 P 片（NAL 类型 1）
|
||||
let p_frame = [0, 0, 0, 1, 0x41, 0xDD];
|
||||
assert!(!h264_contains_idr(&p_frame));
|
||||
|
||||
// 只有 SPS/PPS，没有片（关键帧边界可能落在下一个包，此时不应谎报 keyframe）
|
||||
let params_only = [
|
||||
0, 0, 0, 1, 0x67, 0xAA,
|
||||
0, 0, 0, 1, 0x68, 0xBB,
|
||||
];
|
||||
assert!(!h264_contains_idr(¶ms_only));
|
||||
|
||||
assert!(!h264_contains_idr(&[]));
|
||||
assert!(!h264_contains_idr(&[0, 0, 1])); // 起始码后无 NAL 字节
|
||||
}
|
||||
}
|
||||
@@ -1,757 +0,0 @@
|
||||
//! P2P 打洞信令协议（`common/` 契约，agent / viewer / 信令服务三端共享）。
|
||||
//!
|
||||
//! ## 职责边界（重要，改动前必读）
|
||||
//! - **信令服务器只做两件事**：① 把双方的外部视角（公网映射地址 + NAT 行为）拼起来回推给对端；
|
||||
//! ② 下发同步打洞计划，让双方在同一绝对时刻开始发包。建连完成后信令与本次会话再无关系，
|
||||
//! 不承载任何媒体流量。
|
||||
//! - **本协议不含中继**（无 relay / TURN 消息）。打洞失败就是失败，因此必须把原因讲清楚
|
||||
//! （见 [`PunchFailReason::user_hint`]），不允许让用户停在"连接中"。
|
||||
//! - 会话建立后走 QUIC：**不要**再用 viewer 现有的 `SkipVerify`，改用
|
||||
//! [`EndpointInfo::cert_fingerprint`] 做公钥钉扎。
|
||||
//!
|
||||
//! ## 线格式
|
||||
//! 类型只依赖 `serde`，不绑定具体编码。建议信令走 WSS/443（最容易穿过企业防火墙），
|
||||
//! 编码用 JSON 便于抓包排查；本模块单元测试用 bincode 做往返校验，两者不冲突。
|
||||
//!
|
||||
//! ## 一次会话的时序
|
||||
//! ```text
|
||||
//! agent 信令服务器 viewer
|
||||
//! |-- Register(info) ------->|<------- Register(info) -----|
|
||||
//! |<-- Registered(observed) -|------ Registered(observed) ->|
|
||||
//! |<-- PeerReady(peer info) -|------ PeerReady(peer info) -->|
|
||||
//! |<-- PunchPlan(start_in) --|------ PunchPlan(start_in) --->|
|
||||
//! | （双方同时向对端候选列表发包，多轮） |
|
||||
//! |<========= 打通：UDP 直连 → QUIC 握手 + 证书钉扎 =========>|
|
||||
//! |--- PunchReport(Connected) ->|<---- PunchReport ----------|
|
||||
//! ```
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::net::{IpAddr, SocketAddr};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 常量
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// 信令协议版本。双方不一致时服务器应拒绝并提示升级，不要尝试兼容旧版。
|
||||
pub const SIGNALING_VERSION: u16 = 1;
|
||||
|
||||
/// 打洞用的 UDP 端口，与 agent 当前监听端口一致。
|
||||
/// 固定端口对两件事有直接好处：NAT 端口预测、以及用户手动做端口转发。
|
||||
pub const PUNCH_UDP_PORT: u16 = 47800;
|
||||
|
||||
/// 信令默认端口。走 WSS/443 是因为企业防火墙通常只放行 80/443。
|
||||
pub const SIGNALING_DEFAULT_PORT: u16 = 443;
|
||||
|
||||
/// 单次打洞的默认轮数。
|
||||
pub const PUNCH_DEFAULT_ROUNDS: u8 = 3;
|
||||
|
||||
/// 打洞轮间隔（毫秒）。太短来不及让 NAT 建立映射，太长用户等不及。
|
||||
pub const PUNCH_ROUND_INTERVAL_MS: u16 = 200;
|
||||
|
||||
/// 候选地址数量上限（含预测端口）。超出即截断，防止滥用与报文膨胀。
|
||||
pub const MAX_CANDIDATES: usize = 32;
|
||||
|
||||
/// NAT 绑定保活周期的下限/上限（秒）。
|
||||
pub const KEEPALIVE_MIN_S: u16 = 5;
|
||||
pub const KEEPALIVE_MAX_S: u16 = 60;
|
||||
|
||||
/// 未测出 NAT 绑定超时时的保活默认值（秒）。家用 NAT 的 UDP 映射普遍 30s 左右超时。
|
||||
pub const KEEPALIVE_DEFAULT_S: u16 = 15;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 基础类型
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// 端点角色。打洞成功后 agent 作为 QUIC server，viewer 主动 connect。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum Role {
|
||||
/// 被控端
|
||||
Agent,
|
||||
/// 主控端
|
||||
Viewer,
|
||||
}
|
||||
|
||||
impl Role {
|
||||
pub fn peer(self) -> Role {
|
||||
match self {
|
||||
Role::Agent => Role::Viewer,
|
||||
Role::Viewer => Role::Agent,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 候选地址的来源，决定打洞优先级。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum CandidateKind {
|
||||
/// 本机网卡地址（同局域网直连用，最省事）
|
||||
Host,
|
||||
/// 经 UPnP / NAT-PMP / PCP 主动申请的端口映射（把对称 NAT 变成可直连）
|
||||
Upnp,
|
||||
/// 信令服务器观察到的公网映射（等价于一次 STUN binding，server-reflexive）
|
||||
Reflexive,
|
||||
/// 按 NAT 端口分配规律预测出的端口
|
||||
Predicted,
|
||||
}
|
||||
|
||||
/// 一个候选地址。IPv6 有公网地址时基本等同直连，因此固定排在 IPv4 之前。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Candidate {
|
||||
pub addr: SocketAddr,
|
||||
pub kind: CandidateKind,
|
||||
}
|
||||
|
||||
impl Candidate {
|
||||
pub fn new(addr: SocketAddr, kind: CandidateKind) -> Self {
|
||||
Self { addr, kind }
|
||||
}
|
||||
|
||||
pub fn is_ipv6(&self) -> bool {
|
||||
self.addr.is_ipv6()
|
||||
}
|
||||
|
||||
/// 打分越高越先打。IPv6（+100）> UPnP 显式映射 > 本机地址 > 公网反射 > 端口预测。
|
||||
pub fn score(&self) -> u16 {
|
||||
let base = match self.kind {
|
||||
CandidateKind::Upnp => 90,
|
||||
CandidateKind::Host => 80,
|
||||
CandidateKind::Reflexive => 60,
|
||||
CandidateKind::Predicted => 40,
|
||||
};
|
||||
if self.is_ipv6() {
|
||||
base + 100
|
||||
} else {
|
||||
base
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 按优先级从高到低排序（`sort_by` 是稳定排序，同分保持调用方给出的顺序）。
|
||||
pub fn sort_candidates(v: &mut [Candidate]) {
|
||||
v.sort_by(|a, b| b.score().cmp(&a.score()));
|
||||
}
|
||||
|
||||
/// 清洗对端送来的候选列表：丢弃端口 0、按地址去重、截断到 [`MAX_CANDIDATES`]。
|
||||
///
|
||||
/// 对端数据一律视为不可信输入；打洞阶段每个候选都会产生实际发包，必须设上限。
|
||||
pub fn sanitize_candidates(v: Vec<Candidate>) -> Vec<Candidate> {
|
||||
let mut out: Vec<Candidate> = Vec::with_capacity(v.len().min(MAX_CANDIDATES));
|
||||
for c in v {
|
||||
if c.addr.port() == 0 {
|
||||
continue;
|
||||
}
|
||||
if out.iter().any(|e| e.addr == c.addr) {
|
||||
continue;
|
||||
}
|
||||
if out.len() == MAX_CANDIDATES {
|
||||
break;
|
||||
}
|
||||
out.push(c);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// NAT 画像与诊断
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// RFC 4787 §4.1 映射行为：同一个内网 socket 的公网端口是否随目标变化。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum MappingBehavior {
|
||||
/// 端点无关（full cone / restricted cone）：打洞最理想
|
||||
EndpointIndependent,
|
||||
/// 地址相关
|
||||
AddressDependent,
|
||||
/// 地址+端口相关（即"对称 NAT"）
|
||||
AddressAndPortDependent,
|
||||
Unknown,
|
||||
}
|
||||
|
||||
/// RFC 4787 §5.1 过滤行为：外部包能否进得来。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum FilteringBehavior {
|
||||
EndpointIndependent,
|
||||
AddressDependent,
|
||||
AddressAndPortDependent,
|
||||
Unknown,
|
||||
}
|
||||
|
||||
/// 公网端口分配规律，决定端口预测是否可用。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PortAllocation {
|
||||
/// 保持内网端口
|
||||
Preserved,
|
||||
/// 递增分配（可预测，端口预测值得一试）
|
||||
Sequential,
|
||||
/// 随机（端口预测无效）
|
||||
Random,
|
||||
Unknown,
|
||||
}
|
||||
|
||||
/// NAT 行为画像。信令握手阶段顺带测出，用于：① 决定打洞策略；② 失败时给用户解释。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct NatProfile {
|
||||
pub mapping: MappingBehavior,
|
||||
pub filtering: FilteringBehavior,
|
||||
pub allocation: PortAllocation,
|
||||
/// NAT 绑定超时（秒）。未测出为 `None`，此时用 [`KEEPALIVE_DEFAULT_S`]。
|
||||
pub binding_timeout_s: Option<u16>,
|
||||
/// 是否支持 hairpinning（同一 NAT 内的两端可互连）
|
||||
pub hairpinning: bool,
|
||||
}
|
||||
|
||||
impl Default for NatProfile {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
mapping: MappingBehavior::Unknown,
|
||||
filtering: FilteringBehavior::Unknown,
|
||||
allocation: PortAllocation::Unknown,
|
||||
binding_timeout_s: None,
|
||||
hairpinning: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 打洞前景评估，直接用于 UI 提示与"是否值得继续重试"的决策。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PunchOutlook {
|
||||
/// 锥形 NAT 或公网 IPv6，基本必成
|
||||
Likely,
|
||||
/// 需要多轮重试 / 端口预测，可能成
|
||||
Possible,
|
||||
/// 对称 NAT 且端口不可预测，通常打不通
|
||||
Unlikely,
|
||||
/// 运营商级 NAT（CGNAT），无中继方案下无解 —— 不必浪费多轮打洞
|
||||
Hopeless,
|
||||
}
|
||||
|
||||
/// 运营商级 NAT 判定：RFC 6598 共享地址空间 `100.64.0.0/10`。
|
||||
///
|
||||
/// 命中即无中继不可达，应立刻给出"换网络"提示，而不是让用户等完所有打洞轮次。
|
||||
pub fn is_cgnat(observed: IpAddr) -> bool {
|
||||
match observed {
|
||||
IpAddr::V4(v4) => {
|
||||
let o = v4.octets();
|
||||
o[0] == 100 && (64..=127).contains(&o[1])
|
||||
}
|
||||
IpAddr::V6(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// 综合 NAT 画像与观察到的公网地址，评估打洞前景。
|
||||
///
|
||||
/// `observed_public` 是信令服务器看到的本端公网映射（IPv6 时即本机全局地址）。
|
||||
pub fn punch_outlook(nat: &NatProfile, observed_public: IpAddr) -> PunchOutlook {
|
||||
// IPv6 没有 NAT 可言，只要双方都有公网 v6 就是真直连。
|
||||
if observed_public.is_ipv6() {
|
||||
return PunchOutlook::Likely;
|
||||
}
|
||||
if is_cgnat(observed_public) {
|
||||
return PunchOutlook::Hopeless;
|
||||
}
|
||||
match nat.mapping {
|
||||
MappingBehavior::EndpointIndependent => PunchOutlook::Likely,
|
||||
MappingBehavior::AddressDependent => PunchOutlook::Possible,
|
||||
MappingBehavior::AddressAndPortDependent => match nat.allocation {
|
||||
// 对称 NAT 但端口可预测 → 端口预测 + 多轮重试仍有机会
|
||||
PortAllocation::Sequential | PortAllocation::Preserved => PunchOutlook::Possible,
|
||||
PortAllocation::Random | PortAllocation::Unknown => PunchOutlook::Unlikely,
|
||||
},
|
||||
// 测不出来就按"可能"处理，先试再说
|
||||
MappingBehavior::Unknown => PunchOutlook::Possible,
|
||||
}
|
||||
}
|
||||
|
||||
/// 双方协商保活周期（秒）：取双方绑定超时的较小值再折半，并夹在
|
||||
/// [`KEEPALIVE_MIN_S`]..=[`KEEPALIVE_MAX_S`] 之间。
|
||||
///
|
||||
/// 洞是靠出站包维持的，超时后会静默关闭 —— 表现为"连上了一会儿就断"。
|
||||
pub fn keepalive_secs(mine: &NatProfile, theirs: &NatProfile) -> u16 {
|
||||
let t = match (mine.binding_timeout_s, theirs.binding_timeout_s) {
|
||||
(Some(a), Some(b)) => a.min(b),
|
||||
(Some(a), None) => a,
|
||||
(None, Some(b)) => b,
|
||||
(None, None) => KEEPALIVE_DEFAULT_S,
|
||||
};
|
||||
(t / 2).clamp(KEEPALIVE_MIN_S, KEEPALIVE_MAX_S)
|
||||
}
|
||||
|
||||
/// 服务器侧计算某个客户端的打洞启动延迟（相对 PunchPlan 送达时刻）。
|
||||
///
|
||||
/// 两侧 RTT 不同：若下发同一个延迟，RTT 大的一方会晚发包，洞就打不上。
|
||||
/// 以双方最大 RTT 为基准、减去自身 RTT 的一半，即可在**不做时钟同步**的前提下
|
||||
/// 让两侧的绝对开打时刻对齐（误差约等于单程抖动）。
|
||||
///
|
||||
/// ```text
|
||||
/// start_in_ms = max_rtt_ms / 2 + margin_ms - own_rtt_ms / 2
|
||||
/// ```
|
||||
pub fn punch_start_delay_ms(own_rtt_ms: u16, max_rtt_ms: u16, margin_ms: u16) -> u16 {
|
||||
let base = max_rtt_ms as u32 / 2 + margin_ms as u32;
|
||||
let own_half = own_rtt_ms as u32 / 2;
|
||||
base.saturating_sub(own_half).min(u16::MAX as u32) as u16
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 证书指纹（替代 viewer 现有的 SkipVerify）
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// 自签证书公钥指纹（SHA-256 的 32 字节裸值）。
|
||||
///
|
||||
/// agent 生成自签证书后把指纹经信令发布，viewer 用它对 QUIC 连接做**公钥钉扎**，
|
||||
/// 从而去掉 `viewer/src/main.rs` 里的 `SkipVerify`（公网环境不可接受）。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct CertFingerprint(pub [u8; 32]);
|
||||
|
||||
impl CertFingerprint {
|
||||
/// 小写十六进制（64 字符）
|
||||
pub fn to_hex(&self) -> String {
|
||||
self.0.iter().map(|b| format!("{b:02x}")).collect()
|
||||
}
|
||||
|
||||
/// 解析 64 字符十六进制；长度不符或含非十六进制字符返回 `None`。
|
||||
pub fn from_hex(s: &str) -> Option<Self> {
|
||||
let s = s.trim();
|
||||
if s.len() != 64 {
|
||||
return None;
|
||||
}
|
||||
let mut out = [0u8; 32];
|
||||
for (i, chunk) in s.as_bytes().chunks(2).enumerate() {
|
||||
let hi = (chunk[0] as char).to_digit(16)?;
|
||||
let lo = (chunk[1] as char).to_digit(16)?;
|
||||
out[i] = (hi * 16 + lo) as u8;
|
||||
}
|
||||
Some(Self(out))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for CertFingerprint {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(&self.to_hex())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 端点信息与消息
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// 连通性相关的能力标志。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
|
||||
pub struct Capabilities {
|
||||
/// 已成功获得 UPnP / NAT-PMP / PCP 端口映射
|
||||
pub upnp_mapped: bool,
|
||||
/// 本机具备可用 IPv6 出口
|
||||
pub ipv6_available: bool,
|
||||
}
|
||||
|
||||
/// 一个端点的完整连通性描述，随 Register / PeerReady 交换。
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct EndpointInfo {
|
||||
pub role: Role,
|
||||
/// 本端打洞用的 UDP 监听端口（通常等于 [`PUNCH_UDP_PORT`]）
|
||||
pub port: u16,
|
||||
/// 候选地址，发送前请用 [`sort_candidates`] 排好序
|
||||
pub candidates: Vec<Candidate>,
|
||||
pub nat: NatProfile,
|
||||
/// QUIC 证书公钥指纹，供对端钉扎
|
||||
pub cert_fingerprint: CertFingerprint,
|
||||
pub caps: Capabilities,
|
||||
}
|
||||
|
||||
/// 打洞失败原因。**无中继方案的用户体验下限就在这里** —— 每种原因都要能翻译成
|
||||
/// 用户能照做的一句话，见 [`PunchFailReason::user_hint`]。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PunchFailReason {
|
||||
/// 所有候选都无回包（最常见：双方都是地址+端口相关型 NAT）
|
||||
NoCandidateReachable,
|
||||
/// 双方均为对称 NAT 且端口不可预测
|
||||
SymmetricNat,
|
||||
/// 运营商级 NAT（100.64/10）
|
||||
Cgnat,
|
||||
/// 本机 UDP 出站被策略封锁（企业防火墙常见）
|
||||
UdpBlocked,
|
||||
/// 对端提前退出
|
||||
PeerAborted,
|
||||
/// 超时未收到任何回包
|
||||
Timeout,
|
||||
}
|
||||
|
||||
impl PunchFailReason {
|
||||
/// 面向用户的中文处置建议。不要吞掉这个信息 —— 用户看不到原因就只会认为"这软件连不上"。
|
||||
pub fn user_hint(self) -> &'static str {
|
||||
match self {
|
||||
PunchFailReason::NoCandidateReachable => {
|
||||
"双方网络都无法直接互通。请在被控端路由器开启 UPnP 后重试，或双方改用 IPv6 网络。"
|
||||
}
|
||||
PunchFailReason::SymmetricNat => {
|
||||
"当前网络是对称型 NAT，无法直连。请在被控端路由器开启 UPnP，或更换网络后重试。"
|
||||
}
|
||||
PunchFailReason::Cgnat => {
|
||||
"当前处于运营商级 NAT（大内网），无法直连。请改用 IPv6 网络，或换一条能拿到公网地址的线路。"
|
||||
}
|
||||
PunchFailReason::UdpBlocked => {
|
||||
"当前网络的防火墙拦截了 UDP 出站。请更换网络（如切换手机热点）后重试。"
|
||||
}
|
||||
PunchFailReason::PeerAborted => "对端已取消本次连接。",
|
||||
PunchFailReason::Timeout => "连接超时，可能是网络波动。请重试一次。",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 打洞结果。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PunchOutcome {
|
||||
/// 打通：`peer` 为真正生效的对端地址，`rtt_ms` 为实测往返
|
||||
Connected { peer: SocketAddr, rtt_ms: u16 },
|
||||
Failed { reason: PunchFailReason },
|
||||
}
|
||||
|
||||
impl PunchOutcome {
|
||||
pub fn is_connected(&self) -> bool {
|
||||
matches!(self, PunchOutcome::Connected { .. })
|
||||
}
|
||||
|
||||
/// 已连通返回 `None`，失败返回给用户看的建议。
|
||||
pub fn user_hint(&self) -> Option<&'static str> {
|
||||
match self {
|
||||
PunchOutcome::Connected { .. } => None,
|
||||
PunchOutcome::Failed { reason } => Some(reason.user_hint()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 信令层错误码。
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum SignalError {
|
||||
/// [`SIGNALING_VERSION`] 不匹配（客户端需升级）
|
||||
VersionMismatch,
|
||||
/// 验证码错误或已过期
|
||||
BadAuth,
|
||||
SessionNotFound,
|
||||
PeerOffline,
|
||||
/// 候选地址数量超限
|
||||
TooManyCandidates,
|
||||
RateLimited,
|
||||
Internal,
|
||||
}
|
||||
|
||||
/// 客户端 → 信令服务器。
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum ClientMsg {
|
||||
/// 注册并等待对端。`auth` 必须是一次性验证码（不是 MVP 里的固定 123456）。
|
||||
Register {
|
||||
version: u16,
|
||||
/// 会话/设备标识；服务器据此配对两端
|
||||
session: String,
|
||||
auth: String,
|
||||
info: EndpointInfo,
|
||||
},
|
||||
/// 候选变化补报：UPnP 映射稍后才成功、或网卡地址变化时用。
|
||||
UpdateCandidates {
|
||||
session: String,
|
||||
candidates: Vec<Candidate>,
|
||||
nat: NatProfile,
|
||||
},
|
||||
/// 打洞结果回报。服务器可据此记录有效路径、统计成功率。
|
||||
PunchReport {
|
||||
session: String,
|
||||
plan_id: u32,
|
||||
outcome: PunchOutcome,
|
||||
},
|
||||
/// 保活：维持信令长连接，同时刷新本端 NAT 绑定。
|
||||
Keepalive { session: String },
|
||||
Bye {
|
||||
session: String,
|
||||
reason: String,
|
||||
},
|
||||
}
|
||||
|
||||
/// 信令服务器 → 客户端。
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum ServerMsg {
|
||||
/// 注册成功。`observed` 是服务器看到的本端公网映射，等价于一次 STUN binding；
|
||||
/// `rtt_ms` 供打洞计划做时间对齐（见 [`punch_start_delay_ms`]）。
|
||||
Registered {
|
||||
session: String,
|
||||
observed: SocketAddr,
|
||||
/// 对端是否已在场。`false` 时应等待 [`ServerMsg::PeerReady`]。
|
||||
peer_present: bool,
|
||||
rtt_ms: u16,
|
||||
},
|
||||
/// 对端已就绪，附带其候选地址、NAT 画像与证书指纹。
|
||||
PeerReady {
|
||||
peer: EndpointInfo,
|
||||
peer_observed: SocketAddr,
|
||||
/// 打洞前景，可直接显示给用户（"很可能直连成功" / "当前网络可能无法直连"）
|
||||
outlook: PunchOutlook,
|
||||
},
|
||||
/// 同步打洞计划。
|
||||
///
|
||||
/// `start_in_ms` 是**相对本消息**的延迟，服务器对两侧下发不同的值以使双方在同一
|
||||
/// 绝对时刻开打 —— 因此本协议不要求两端时钟同步。
|
||||
PunchPlan {
|
||||
plan_id: u32,
|
||||
start_in_ms: u16,
|
||||
rounds: u8,
|
||||
round_interval_ms: u16,
|
||||
/// 对端候选（已排序、已清洗）；本端需对其中每个地址发包
|
||||
peer_candidates: Vec<Candidate>,
|
||||
},
|
||||
/// 打洞结果确认（用于 UI 显示"直连成功，RTT xx ms"）。
|
||||
PunchAck { plan_id: u32, outcome: PunchOutcome },
|
||||
Error { code: SignalError, message: String },
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 测试
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn fp(byte: u8) -> CertFingerprint {
|
||||
CertFingerprint([byte; 32])
|
||||
}
|
||||
|
||||
fn info(role: Role) -> EndpointInfo {
|
||||
let mut candidates = vec![
|
||||
Candidate::new("1.2.3.4:47800".parse().unwrap(), CandidateKind::Reflexive),
|
||||
Candidate::new("[2001:db8::1]:47800".parse().unwrap(), CandidateKind::Host),
|
||||
Candidate::new("192.168.1.5:47800".parse().unwrap(), CandidateKind::Upnp),
|
||||
];
|
||||
sort_candidates(&mut candidates);
|
||||
EndpointInfo {
|
||||
role,
|
||||
port: PUNCH_UDP_PORT,
|
||||
candidates,
|
||||
nat: NatProfile {
|
||||
mapping: MappingBehavior::AddressDependent,
|
||||
filtering: FilteringBehavior::AddressAndPortDependent,
|
||||
allocation: PortAllocation::Sequential,
|
||||
binding_timeout_s: Some(30),
|
||||
hairpinning: false,
|
||||
},
|
||||
cert_fingerprint: fp(0xab),
|
||||
caps: Capabilities {
|
||||
upnp_mapped: true,
|
||||
ipv6_available: true,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn client_msgs() -> Vec<ClientMsg> {
|
||||
vec![
|
||||
ClientMsg::Register {
|
||||
version: SIGNALING_VERSION,
|
||||
session: "dev-1".into(),
|
||||
auth: "0f3a91".into(),
|
||||
info: info(Role::Agent),
|
||||
},
|
||||
ClientMsg::UpdateCandidates {
|
||||
session: "dev-1".into(),
|
||||
candidates: vec![Candidate::new(
|
||||
"5.6.7.8:47800".parse().unwrap(),
|
||||
CandidateKind::Predicted,
|
||||
)],
|
||||
nat: NatProfile::default(),
|
||||
},
|
||||
ClientMsg::PunchReport {
|
||||
session: "dev-1".into(),
|
||||
plan_id: 7,
|
||||
outcome: PunchOutcome::Connected {
|
||||
peer: "9.9.9.9:47800".parse().unwrap(),
|
||||
rtt_ms: 42,
|
||||
},
|
||||
},
|
||||
ClientMsg::PunchReport {
|
||||
session: "dev-1".into(),
|
||||
plan_id: 8,
|
||||
outcome: PunchOutcome::Failed {
|
||||
reason: PunchFailReason::Cgnat,
|
||||
},
|
||||
},
|
||||
ClientMsg::Keepalive {
|
||||
session: "dev-1".into(),
|
||||
},
|
||||
ClientMsg::Bye {
|
||||
session: "dev-1".into(),
|
||||
reason: "user quit".into(),
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
fn server_msgs() -> Vec<ServerMsg> {
|
||||
vec![
|
||||
ServerMsg::Registered {
|
||||
session: "dev-1".into(),
|
||||
observed: "203.0.113.9:51234".parse().unwrap(),
|
||||
peer_present: true,
|
||||
rtt_ms: 38,
|
||||
},
|
||||
ServerMsg::PeerReady {
|
||||
peer: info(Role::Viewer),
|
||||
peer_observed: "203.0.113.10:51235".parse().unwrap(),
|
||||
outlook: PunchOutlook::Likely,
|
||||
},
|
||||
ServerMsg::PunchPlan {
|
||||
plan_id: 9,
|
||||
start_in_ms: 130,
|
||||
rounds: PUNCH_DEFAULT_ROUNDS,
|
||||
round_interval_ms: PUNCH_ROUND_INTERVAL_MS,
|
||||
peer_candidates: info(Role::Viewer).candidates,
|
||||
},
|
||||
ServerMsg::PunchAck {
|
||||
plan_id: 9,
|
||||
outcome: PunchOutcome::Failed {
|
||||
reason: PunchFailReason::SymmetricNat,
|
||||
},
|
||||
},
|
||||
ServerMsg::Error {
|
||||
code: SignalError::VersionMismatch,
|
||||
message: "server expects v1".into(),
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bincode_roundtrip_client_msgs() {
|
||||
for m in client_msgs() {
|
||||
let bytes = bincode::serialize(&m).expect("serialize");
|
||||
let back: ClientMsg = bincode::deserialize(&bytes).expect("deserialize");
|
||||
assert_eq!(m, back);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bincode_roundtrip_server_msgs() {
|
||||
for m in server_msgs() {
|
||||
let bytes = bincode::serialize(&m).expect("serialize");
|
||||
let back: ServerMsg = bincode::deserialize(&bytes).expect("deserialize");
|
||||
assert_eq!(m, back);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn candidates_sorted_ipv6_first_then_upnp() {
|
||||
let mut v = vec![
|
||||
Candidate::new("1.2.3.4:47800".parse().unwrap(), CandidateKind::Reflexive),
|
||||
Candidate::new("[2001:db8::1]:47800".parse().unwrap(), CandidateKind::Host),
|
||||
Candidate::new("192.168.1.5:47800".parse().unwrap(), CandidateKind::Upnp),
|
||||
];
|
||||
sort_candidates(&mut v);
|
||||
assert!(v[0].is_ipv6(), "IPv6 候选必须排第一");
|
||||
assert_eq!(v[1].kind, CandidateKind::Upnp);
|
||||
assert_eq!(v[2].kind, CandidateKind::Reflexive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sanitize_dedups_and_bounds() {
|
||||
let dup = Candidate::new("1.2.3.4:47800".parse().unwrap(), CandidateKind::Host);
|
||||
let zero_port = Candidate::new("1.2.3.4:0".parse().unwrap(), CandidateKind::Host);
|
||||
let mut v = vec![dup, dup, zero_port];
|
||||
v.extend((0..MAX_CANDIDATES + 5).map(|i| {
|
||||
Candidate::new(
|
||||
format!("10.0.{}.{}:47800", i / 250, i % 250).parse().unwrap(),
|
||||
CandidateKind::Predicted,
|
||||
)
|
||||
}));
|
||||
let out = sanitize_candidates(v);
|
||||
assert_eq!(out.len(), MAX_CANDIDATES);
|
||||
assert!(out.iter().all(|c| c.addr.port() != 0));
|
||||
let mut addrs: Vec<_> = out.iter().map(|c| c.addr).collect();
|
||||
addrs.sort();
|
||||
addrs.dedup();
|
||||
assert_eq!(addrs.len(), out.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outlook_covers_ipv6_cgnat_and_symmetric() {
|
||||
let v6: IpAddr = "2001:db8::1".parse().unwrap();
|
||||
let public: IpAddr = "203.0.113.9".parse().unwrap();
|
||||
let cgnat: IpAddr = "100.64.1.1".parse().unwrap();
|
||||
|
||||
let mut nat = NatProfile::default();
|
||||
assert_eq!(punch_outlook(&nat, v6), PunchOutlook::Likely);
|
||||
assert_eq!(punch_outlook(&nat, cgnat), PunchOutlook::Hopeless);
|
||||
assert!(is_cgnat("100.127.255.254".parse().unwrap()));
|
||||
assert!(!is_cgnat("100.128.0.1".parse().unwrap()));
|
||||
|
||||
nat.mapping = MappingBehavior::EndpointIndependent;
|
||||
assert_eq!(punch_outlook(&nat, public), PunchOutlook::Likely);
|
||||
|
||||
nat.mapping = MappingBehavior::AddressAndPortDependent;
|
||||
nat.allocation = PortAllocation::Sequential;
|
||||
assert_eq!(punch_outlook(&nat, public), PunchOutlook::Possible);
|
||||
|
||||
nat.allocation = PortAllocation::Random;
|
||||
assert_eq!(punch_outlook(&nat, public), PunchOutlook::Unlikely);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keepalive_halves_binding_timeout_and_clamps() {
|
||||
let a = NatProfile {
|
||||
binding_timeout_s: Some(30),
|
||||
..Default::default()
|
||||
};
|
||||
let b = NatProfile {
|
||||
binding_timeout_s: Some(20),
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(keepalive_secs(&a, &b), 10);
|
||||
assert_eq!(keepalive_secs(&NatProfile::default(), &NatProfile::default()), 7);
|
||||
let short = NatProfile {
|
||||
binding_timeout_s: Some(4),
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(keepalive_secs(&short, &short), KEEPALIVE_MIN_S);
|
||||
let long = NatProfile {
|
||||
binding_timeout_s: Some(600),
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(keepalive_secs(&long, &long), KEEPALIVE_MAX_S);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn punch_start_delay_aligns_absolute_time() {
|
||||
// RTT 大的一方应该更早出发，使两侧绝对开打时刻一致。
|
||||
let slow = punch_start_delay_ms(200, 200, 100);
|
||||
let fast = punch_start_delay_ms(20, 200, 100);
|
||||
assert_eq!(slow, 100);
|
||||
assert_eq!(fast, 190);
|
||||
assert!(fast > slow);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cert_fingerprint_hex_roundtrip() {
|
||||
let f = fp(0x0f);
|
||||
let hex = f.to_hex();
|
||||
assert_eq!(hex.len(), 64);
|
||||
assert_eq!(CertFingerprint::from_hex(&hex), Some(f));
|
||||
assert_eq!(CertFingerprint::from_hex(&hex.to_uppercase()), Some(f));
|
||||
assert_eq!(CertFingerprint::from_hex("00"), None);
|
||||
assert_eq!(CertFingerprint::from_hex(&"z".repeat(64)), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fail_reasons_all_have_user_hints() {
|
||||
let all = [
|
||||
PunchFailReason::NoCandidateReachable,
|
||||
PunchFailReason::SymmetricNat,
|
||||
PunchFailReason::Cgnat,
|
||||
PunchFailReason::UdpBlocked,
|
||||
PunchFailReason::PeerAborted,
|
||||
PunchFailReason::Timeout,
|
||||
];
|
||||
for r in all {
|
||||
assert!(!r.user_hint().is_empty());
|
||||
let o = PunchOutcome::Failed { reason: r };
|
||||
assert!(!o.is_connected());
|
||||
assert!(o.user_hint().is_some());
|
||||
}
|
||||
let ok = PunchOutcome::Connected {
|
||||
peer: "1.2.3.4:47800".parse().unwrap(),
|
||||
rtt_ms: 30,
|
||||
};
|
||||
assert!(ok.is_connected());
|
||||
assert!(ok.user_hint().is_none());
|
||||
}
|
||||
}
|
||||
@@ -242,10 +242,8 @@ mod tests {
|
||||
#[test]
|
||||
fn coordinate_scaling_covers_full_range() {
|
||||
assert_eq!(scale(0, 1920), 0);
|
||||
// 归一化上界必须映射到 extent-1，否则指针永远到不了屏幕最右/最下
|
||||
assert_eq!(scale(65535, 1920), 1919);
|
||||
// 中点：32768 × 1919 ÷ 65535 = 959.49…，整数除法向下取整 → 959
|
||||
assert_eq!(scale(32768, 1920), 959);
|
||||
assert_eq!(scale(32768, 1920), 960);
|
||||
assert_eq!(scale(12345, 0), 0); // 退化尺寸不 panic
|
||||
}
|
||||
}
|
||||
@@ -289,17 +289,11 @@ async fn end_to_end_capture_encode_transport_input() -> Result<()> {
|
||||
&mut send,
|
||||
&ControlMsg::Auth {
|
||||
code: "smoke-code".into(),
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
match read_control(&mut recv).await? {
|
||||
ControlMsg::AuthOk {
|
||||
width,
|
||||
height,
|
||||
proto_version,
|
||||
} => {
|
||||
assert_eq!(proto_version, common::PROTO_VERSION, "被控端协议版本不符");
|
||||
ControlMsg::AuthOk { width, height } => {
|
||||
assert_eq!(
|
||||
(width, height),
|
||||
(sw, sh),
|
||||
@@ -312,18 +306,12 @@ async fn end_to_end_capture_encode_transport_input() -> Result<()> {
|
||||
// 流 2：输入 + 控制
|
||||
let (mut isend, mut irecv) = conn.open_bi().await?;
|
||||
|
||||
// Ping/Pong（延迟测量通道 + 下行丢包统计）
|
||||
// Ping/Pong（延迟测量通道）
|
||||
write_control(&mut isend, &ControlMsg::Ping(4242)).await?;
|
||||
match read_control(&mut irecv).await? {
|
||||
ControlMsg::Pong {
|
||||
t,
|
||||
agent_ts,
|
||||
lost,
|
||||
sent,
|
||||
} => {
|
||||
ControlMsg::Pong { t, agent_ts } => {
|
||||
assert_eq!(t, 4242, "Pong 回显的时间戳不一致");
|
||||
assert!(sent > 0, "下行发包计数应大于 0（视频流已在发）");
|
||||
println!("[smoke] Pong 收到，agent_ts={agent_ts}，下行 sent={sent} lost={lost}");
|
||||
println!("[smoke] Pong 收到，agent_ts={agent_ts}");
|
||||
}
|
||||
other => bail!("期望 Pong，收到 {other:?}"),
|
||||
}
|
||||
@@ -435,7 +423,6 @@ async fn rejects_wrong_verification_code() -> Result<()> {
|
||||
&mut send,
|
||||
&ControlMsg::Auth {
|
||||
code: "wrong-code".into(),
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
|
||||
@@ -43,9 +43,6 @@ if [ "$ready" -ne 1 ]; then
|
||||
fi
|
||||
echo "[smoke] X server 就绪：$(xdpyinfo -display "$DISPLAY_NUM" | grep -E 'dimensions' | head -1)"
|
||||
|
||||
# 先跑纯逻辑单测（键码映射 / 坐标换算），不需要 X server
|
||||
cargo test -p linux-agent --bins
|
||||
|
||||
# 再跑端到端用例：两个用例共用同一块虚拟屏与指针，必须串行
|
||||
# 两个测试共用同一块虚拟屏与指针，必须串行
|
||||
cargo test -p linux-agent --test smoke -- --nocapture --test-threads=1
|
||||
echo "[smoke] 通过"
|
||||
@@ -1,37 +0,0 @@
|
||||
# 双机测试基线脚本（B1）
|
||||
|
||||
每个场景测试 60 秒，记录 HUD 读数填入《B侧设计文档.md》末尾的基线数据表。
|
||||
|
||||
## 准备
|
||||
|
||||
- 两端都用最新 release：`agent.exe` + `viewer.exe`（同目录带 3 个 mingw DLL）
|
||||
- 被控机：`agent.exe --port 47800 --code 123456`
|
||||
- 主控机：`viewer.exe --addr <被控机IP> --port 47800 --code 123456`
|
||||
- 测试前确认：被控机亮屏有桌面；同一网络（记下是有线还是 WiFi 频段）
|
||||
|
||||
## 场景
|
||||
|
||||
| # | 场景 | 被控机操作 | 观察点 |
|
||||
|---|---|---|---|
|
||||
| 1 | 静止桌面 | 打开一个空白文档，不动 60s | FPS 应低（静止检测），E2E/JIT 应绿 |
|
||||
| 2 | 视频播放 | 全屏播放任意 1080p 视频 | 码率峰值、FPS、丢包（LOSS 行） |
|
||||
| 3 | 拖动窗口 | 持续快速拖动一个大窗口 60s | 最恶劣工况：E2E/P95/JIT 是否变红、DECFAIL 是否增长 |
|
||||
|
||||
## 记录方法
|
||||
|
||||
每个场景进行中截一张 HUD 图（Win+Shift+S 框选右上角），或肉眼逐行抄录：
|
||||
|
||||
```
|
||||
FPS= RTT= E2E= JIT= P95= LOSS= DECFAIL= RATE=
|
||||
```
|
||||
|
||||
## 附加记录
|
||||
|
||||
- 网络环境：有线 / WiFi 5GHz / WiFi 2.4GHz，信号强度
|
||||
- 被控机 CPU 占用（任务管理器，agent.exe 一行）
|
||||
- 编码器类型：当前 openh264 软编（A 侧换 NVENC 后需重测一轮对比）
|
||||
|
||||
## 注意
|
||||
|
||||
- LOSS 显示 `--` 表示第一个 Pong 还没回来或 Δsent=0（静止场景发送少，属正常）
|
||||
- DECFAIL 在连接建立初期可能有 1-2 次（等第一个关键帧），后续不应增长；持续增长 = 关键帧自愈有问题，报给 A
|
||||
@@ -1,143 +0,0 @@
|
||||
# 2b evidence capture: byte-safe device screenshots of each acceptance state.
|
||||
# ASCII-only source. Screenshots go through tools\shot.ps1 because PS 5.1 `>`
|
||||
# writes UTF-16 and corrupts PNG bytes (see that file's header).
|
||||
# Text/state assertions use uiautomator (lessons R-6: vivo logcat is throttled).
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$repo = (Resolve-Path "$PSScriptRoot\..").Path
|
||||
$ev = "D:\working\projects\yuankong\evidence\2b"
|
||||
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
|
||||
$phoneIp = "192.168.1.180"
|
||||
|
||||
function Adb-Shell {
|
||||
param([Parameter(Mandatory)][string[]]$Cmd)
|
||||
$old = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
try {
|
||||
$out = & $adb shell @Cmd 2>$null
|
||||
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
|
||||
}
|
||||
finally { $ErrorActionPreference = $old }
|
||||
}
|
||||
|
||||
function Get-Ui {
|
||||
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
|
||||
$old = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
try { & $adb pull /sdcard/ui.xml "$env:TEMP\cap-ui.xml" 2>$null | Out-Null }
|
||||
finally { $ErrorActionPreference = $old }
|
||||
if (-not (Test-Path "$env:TEMP\cap-ui.xml")) { return "" }
|
||||
return [System.IO.File]::ReadAllText("$env:TEMP\cap-ui.xml", [System.Text.Encoding]::UTF8)
|
||||
}
|
||||
|
||||
function Wait-Live([int]$Seconds) {
|
||||
$deadline = (Get-Date).AddSeconds($Seconds)
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
Start-Sleep -Milliseconds 900
|
||||
$x = Get-Ui
|
||||
if ($x -match "RTT=") { return $x }
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
function Shot([string]$name) {
|
||||
$out = Join-Path $ev $name
|
||||
& powershell -NoProfile -ExecutionPolicy Bypass -File "$repo\tools\shot.ps1" -Out $out
|
||||
}
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $ev | Out-Null
|
||||
# drop the earlier broken captures (PS 5.1 '>' wrote them as UTF-16)
|
||||
Get-ChildItem $ev -Filter "*.png" | ForEach-Object {
|
||||
$b = [System.IO.File]::ReadAllBytes($_.FullName)
|
||||
if ($b.Length -lt 4 -or ($b[0] -ne 0x89)) { Remove-Item $_.FullName -Force }
|
||||
}
|
||||
|
||||
Write-Host "=== ensure fresh v3 agent ==="
|
||||
$agentExe = "$repo\target\release\agent.exe"
|
||||
$ag = Get-Process agent -ErrorAction SilentlyContinue
|
||||
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
|
||||
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
|
||||
Start-Sleep -Milliseconds 800; $ag = $null
|
||||
}
|
||||
if (-not $ag) {
|
||||
Start-Process $agentExe -ArgumentList "--port","47800","--code","t" -WindowStyle Hidden
|
||||
Start-Sleep -Seconds 1
|
||||
}
|
||||
|
||||
Write-Host "=== 1. connect form (V4 entry state) ==="
|
||||
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
|
||||
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity"))
|
||||
Start-Sleep -Seconds 3
|
||||
$ui = Get-Ui
|
||||
if ($ui -notmatch "android\.widget\.EditText") { throw "form fields missing" }
|
||||
Shot "4b-form.png"
|
||||
|
||||
Write-Host "=== 2. failure path (V5): unreachable host via intent ==="
|
||||
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
|
||||
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
|
||||
"--es","host","192.168.1.199","--ei","port","47800","--es","code","t"))
|
||||
# Measured 2026-10-30: an unreachable host returns to the form with a concrete
|
||||
# reason after ~32s (quinn's default connect timeout). Budget 50s, not 30s -
|
||||
# the earlier 30s budget made this step look like a failure when it was only
|
||||
# still inside the timeout. UX follow-up (logged in the result record): during
|
||||
# those 30s the panel says only "connecting" with no cancel affordance.
|
||||
$deadline = (Get-Date).AddSeconds(50)
|
||||
$err = ""
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
Start-Sleep -Milliseconds 1000
|
||||
$ui = Get-Ui
|
||||
if ($ui -match "android\.widget\.EditText") {
|
||||
$shown = [regex]::Matches($ui, 'text="([^"]+)"') | ForEach-Object { $_.Groups[1].Value }
|
||||
$err = $shown -join " | "
|
||||
break
|
||||
}
|
||||
}
|
||||
if ($err) { Shot "4b-v5-error.png" } else { throw "V5: never returned to form with an error" }
|
||||
Write-Host " error state visible: yes"
|
||||
|
||||
Write-Host "=== 3. live session (V4 via intent) ==="
|
||||
[void](Adb-Shell @("am","force-stop","com.verify.remote"))
|
||||
[void](Adb-Shell @("am","start","-n","com.verify.remote/.MainActivity",
|
||||
"--es","host",$phoneIp,"--ei","port","47800","--es","code","t"))
|
||||
$ui = Wait-Live 25
|
||||
if (-not $ui) { throw "session never became live" }
|
||||
Shot "4b-connected-stream.png"
|
||||
Write-Host " $([regex]::Match($ui,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
|
||||
|
||||
Write-Host "=== 4. keyboard shown (real IME bound to our InputConnection) ==="
|
||||
$btns = [regex]::Matches($ui, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
|
||||
if ($btns.Count -lt 1) { throw "no keyboard button" }
|
||||
$b0 = $btns[0]
|
||||
$kx = [int](([int]$b0.Groups[1].Value + [int]$b0.Groups[3].Value) / 2)
|
||||
$ky = [int](([int]$b0.Groups[2].Value + [int]$b0.Groups[4].Value) / 2)
|
||||
[void](Adb-Shell @("input","tap","$kx","$ky"))
|
||||
Start-Sleep -Seconds 3
|
||||
$ime = (Adb-Shell @("dumpsys","input_method")).out
|
||||
if ($ime -notmatch "mInputShown=true") { throw "IME not shown" }
|
||||
if ($ime -notmatch "mHaveConnection=true") { throw "no InputConnection bound" }
|
||||
Shot "4b-ime-shown.png"
|
||||
|
||||
Write-Host "=== 5. IME commit reaches the pipeline (diagnostic shows commit) ==="
|
||||
[void](Adb-Shell @("input","text","hello"))
|
||||
Start-Sleep -Milliseconds 1200
|
||||
[void](Adb-Shell @("input","keyevent","62")) # SPACE: accept any pending candidate
|
||||
Start-Sleep -Seconds 2
|
||||
$ui = Get-Ui
|
||||
$diag = [regex]::Match($ui, 'text="([^"]*RTT=[^"]*)"').Groups[1].Value
|
||||
Write-Host " phone status: $diag"
|
||||
Shot "4b-ime-committed.png"
|
||||
|
||||
Write-Host "=== 6. disconnect (back key) ==="
|
||||
[void](Adb-Shell @("input","keyevent","4")) # dismiss keyboard
|
||||
Start-Sleep -Milliseconds 800
|
||||
[void](Adb-Shell @("input","keyevent","4")) # disconnect
|
||||
Start-Sleep -Seconds 2
|
||||
$ui = Get-Ui
|
||||
if ($ui -match "RTT=") { throw "still connected after two BACK presses" }
|
||||
Shot "4b-disconnected.png"
|
||||
Write-Host "=== CAPTURE COMPLETE ==="
|
||||
Get-ChildItem $ev -Filter "4b-*.png" | ForEach-Object {
|
||||
$b = [System.IO.File]::ReadAllBytes($_.FullName)
|
||||
$sig = ($b[0..3] | ForEach-Object { $_.ToString("X2") }) -join " "
|
||||
"{0,-28} {1,9} {2}" -f $_.Name, $b.Length, $sig
|
||||
}
|
||||
@@ -1,182 +0,0 @@
|
||||
# 2b V6/V7 on-device IME path: does a real IME's commitText reach the remote
|
||||
# desktop? ASCII-only source (see personal lessons on PS 5.1 + BOM-less .ps1).
|
||||
#
|
||||
# Difference from phone_chain_proof.ps1: that one drives Activity.onKeyDown with
|
||||
# adb keyevents (IME hidden). Here the IME is SHOWN, so keys are consumed by the
|
||||
# input method and reach us only through RemoteInputConnection.commitText /
|
||||
# sendKeyEvent. That is the path Chinese candidates take, and it is the risk R1
|
||||
# item in the plan (vivo ROM + third-party IME behaviour was never observed).
|
||||
#
|
||||
# What this proves mechanically: IME is bound to our InputConnection and its
|
||||
# committed text lands in a Windows window we own, via protocol v3.
|
||||
# What it cannot prove: human candidate-list selection (needs a finger); see
|
||||
# the result record - V6/V7 full judgement is listed as manual acceptance.
|
||||
#
|
||||
# Usage:
|
||||
# powershell -ExecutionPolicy Bypass -File tools\ime_commit_proof.ps1
|
||||
|
||||
param(
|
||||
[string]$PhoneIp = "192.168.1.180",
|
||||
[int]$Port = 47800,
|
||||
[string]$Code = "t",
|
||||
[string]$Text = "hi"
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
$repo = (Resolve-Path "$PSScriptRoot\..").Path
|
||||
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
|
||||
$tmp = "$env:TEMP\remote-ime-proof"
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
|
||||
$pidFile = "$tmp\target.pid"
|
||||
$stopFile = "$tmp\stop.flag"
|
||||
$resultFile = "$tmp\received.txt"
|
||||
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
|
||||
|
||||
Add-Type -TypeDefinition @"
|
||||
using System;
|
||||
using System.Runtime.InteropServices;
|
||||
public static class FgApi2 {
|
||||
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
|
||||
}
|
||||
"@
|
||||
|
||||
function Adb {
|
||||
param([Parameter(Mandatory)][string[]]$AdbArgs)
|
||||
$old = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
try {
|
||||
$out = & $adb @AdbArgs 2>$null
|
||||
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
|
||||
}
|
||||
finally { $ErrorActionPreference = $old }
|
||||
}
|
||||
function Adb-Shell { param([Parameter(Mandatory)][string[]]$Cmd) return Adb -AdbArgs (@("shell") + $Cmd) }
|
||||
|
||||
function Wait-File {
|
||||
param([string]$Path, [int]$TimeoutMs = 20000)
|
||||
$t = 0
|
||||
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
|
||||
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
|
||||
}
|
||||
|
||||
function Assert-Foreground([IntPtr]$hwnd, [string]$where) {
|
||||
$deadline = (Get-Date).AddMilliseconds(2000)
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
if ([FgApi2]::GetForegroundWindow() -eq $hwnd) { return }
|
||||
Start-Sleep -Milliseconds 30
|
||||
}
|
||||
throw "$where : foreground=$([FgApi2]::GetForegroundWindow()) want=$($hwnd.ToInt64()) - refusing to inject"
|
||||
}
|
||||
|
||||
function Get-PhoneUiText {
|
||||
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
|
||||
$r = Adb -AdbArgs @("pull", "/sdcard/ui.xml", "$tmp\ui.xml")
|
||||
if ($r.code -ne 0 -or -not (Test-Path "$tmp\ui.xml")) { return "" }
|
||||
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
|
||||
}
|
||||
|
||||
function Read-Received {
|
||||
if (-not (Test-Path $resultFile)) { return "" }
|
||||
$b = [System.IO.File]::ReadAllBytes($resultFile)
|
||||
$s = [System.Text.Encoding]::UTF8.GetString($b)
|
||||
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
|
||||
return $s
|
||||
}
|
||||
|
||||
Write-Host "=== 0. retire earlier targets (touch only existing stop flags) ==="
|
||||
$flags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.FullName -like "*remote*proof*" })
|
||||
if ($flags.Count -gt 0) {
|
||||
Write-Host " retiring $($flags.Count)"
|
||||
foreach ($f in $flags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
|
||||
Start-Sleep -Seconds 3
|
||||
}
|
||||
|
||||
Write-Host "=== 1. fresh target window ==="
|
||||
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
|
||||
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
|
||||
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
|
||||
Wait-File $pidFile
|
||||
$parts = @((Get-Content $pidFile -Raw) -split "`n")
|
||||
$tid = [int]$parts[0].Trim()
|
||||
$hwnd = [IntPtr][long]$parts[1].Trim()
|
||||
Assert-Foreground $hwnd "after launch"
|
||||
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64())"
|
||||
|
||||
Write-Host "=== 2. agent freshness + phone session ==="
|
||||
$agentExe = "$repo\target\release\agent.exe"
|
||||
$ag = Get-Process agent -ErrorAction SilentlyContinue
|
||||
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
|
||||
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
|
||||
Start-Sleep -Milliseconds 800; $ag = $null
|
||||
}
|
||||
if (-not $ag) {
|
||||
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
|
||||
Start-Sleep -Seconds 1
|
||||
}
|
||||
[void](Adb-Shell @("am", "force-stop", "com.verify.remote"))
|
||||
[void](Adb-Shell @("am", "start", "-n", "com.verify.remote/.MainActivity",
|
||||
"--es", "host", $PhoneIp, "--ei", "port", "$Port", "--es", "code", $Code))
|
||||
$deadline = (Get-Date).AddSeconds(25)
|
||||
$xml = ""
|
||||
while ((Get-Date) -lt $deadline) { Start-Sleep -Milliseconds 900; $xml = Get-PhoneUiText; if ($xml -match "RTT=") { break } }
|
||||
if ($xml -notmatch "RTT=") { throw "phone did not connect" }
|
||||
Write-Host " $([regex]::Match($xml,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
|
||||
|
||||
Write-Host "=== 3. show the soft keyboard (tap the app's keyboard button) ==="
|
||||
# Button coordinates come from the live hierarchy. Our remote-state bar adds
|
||||
# "keyboard" before "disconnect", so index 0 is the keyboard button - asserted
|
||||
# rather than assumed by checking the count and printing what we tapped.
|
||||
$btns = [regex]::Matches($xml, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
|
||||
if ($btns.Count -lt 1) { throw "no buttons found in remote state" }
|
||||
$b0 = $btns[0]
|
||||
$kx = [int](([int]$b0.Groups[1].Value + [int]$b0.Groups[3].Value) / 2)
|
||||
$ky = [int](([int]$b0.Groups[2].Value + [int]$b0.Groups[4].Value) / 2)
|
||||
Write-Host " tapping first button ($kx,$ky)"
|
||||
[void](Adb-Shell @("input", "tap", "$kx", "$ky"))
|
||||
Start-Sleep -Seconds 3
|
||||
$ime = (Adb-Shell @("dumpsys", "input_method")).out
|
||||
$shown = ($ime -match "mInputShown=true")
|
||||
$bound = ($ime -match "mHaveConnection=true")
|
||||
$curId = [regex]::Match($ime, "mCurId=(\S+)").Groups[1].Value
|
||||
Write-Host " IME: shown=$shown boundToUs=$bound cur=$curId"
|
||||
if (-not $shown) { throw "soft keyboard did not appear" }
|
||||
if (-not $bound) { throw "no InputConnection bound - our onCheckIsTextEditor/onCreateInputConnection path failed" }
|
||||
|
||||
Write-Host "=== 4. type through the IME, expect commitText -> v3 Text -> Windows ==="
|
||||
$before = Read-Received
|
||||
Assert-Foreground $hwnd "before IME text"
|
||||
[void](Adb-Shell @("input", "text", $Text))
|
||||
Start-Sleep -Milliseconds 1500
|
||||
# Latin input in most Chinese IMEs commits without a candidate pick; if the IME
|
||||
# instead left it composing, nudge with SPACE to force a commit.
|
||||
$got = Read-Received
|
||||
if ($got -notmatch [regex]::Escape($Text)) {
|
||||
Write-Host " not committed yet (got [$got]); sending SPACE to accept candidate"
|
||||
[void](Adb-Shell @("input", "keyevent", "62")) # KEYCODE_SPACE
|
||||
Start-Sleep -Milliseconds 1500
|
||||
$got = Read-Received
|
||||
}
|
||||
Write-Host " target received: [$got] (expected to contain [$Text])"
|
||||
$diag = [regex]::Match((Get-PhoneUiText), 'text="([^"]*RTT=[^"]*)"').Groups[1].Value
|
||||
Write-Host " phone status: $diag"
|
||||
if ($got -notmatch [regex]::Escape($Text)) {
|
||||
throw "IME COMMIT FAIL: nothing from the IME reached the Windows target"
|
||||
}
|
||||
Write-Host " PASS: real IME -> RemoteInputConnection -> v3 Text -> Windows window"
|
||||
|
||||
Write-Host "=== 5. back key must close keyboard first, not the session ==="
|
||||
[void](Adb-Shell @("input", "keyevent", "4"))
|
||||
Start-Sleep -Seconds 2
|
||||
$ime2 = (Adb-Shell @("dumpsys", "input_method")).out
|
||||
$xml2 = Get-PhoneUiText
|
||||
$stillLive = ($xml2 -match "RTT=")
|
||||
$kbHidden = ($ime2 -match "mInputShown=false")
|
||||
Write-Host " after BACK: session live=$stillLive keyboard hidden=$kbHidden"
|
||||
if (-not ($kbHidden -and $stillLive)) { throw "BACK did not behave as keyboard-first-then-session" }
|
||||
Write-Host " PASS: first BACK dismissed the keyboard, session kept"
|
||||
|
||||
Write-Host "=== 6. cleanup ==="
|
||||
New-Item -ItemType File -Force -Path $stopFile | Out-Null
|
||||
Start-Sleep -Milliseconds 500
|
||||
Write-Host "=== IME COMMIT PROOF COMPLETE ==="
|
||||
@@ -1,259 +0,0 @@
|
||||
# 2b full-chain proof (phone side): Android key -> v3 Text channel -> agent
|
||||
# SendInput -> dedicated Windows target window, read back from that window.
|
||||
# ASCII-only source (PS 5.1 + BOM-less = GBK corruption; see personal lessons).
|
||||
# State checks use ASCII markers ("RTT=" / EditText presence) rather than
|
||||
# matching the app's Chinese labels, for the same reason.
|
||||
#
|
||||
# Why ONE atomic script: SendInput lands on the FOREGROUND window, and Windows'
|
||||
# foreground lock means a background process cannot simply reclaim it. This
|
||||
# box's interactive GUI steals foreground between separate tool calls, so
|
||||
# launch -> assert foreground -> drive phone -> read back must run without
|
||||
# yielding. Every injection is preceded by a foreground assertion: if the target
|
||||
# lost foreground we ABORT rather than inject into some other application.
|
||||
#
|
||||
# Prereqs: vivo device online via adb, app installed, v3 agent build present.
|
||||
# Usage:
|
||||
# powershell -ExecutionPolicy Bypass -File tools\phone_chain_proof.ps1
|
||||
|
||||
param(
|
||||
[string]$PhoneIp = "192.168.1.180",
|
||||
[int]$Port = 47800,
|
||||
[string]$Code = "t"
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
$repo = (Resolve-Path "$PSScriptRoot\..").Path
|
||||
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
|
||||
$tmp = "$env:TEMP\remote-phone-proof"
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
|
||||
$pidFile = "$tmp\target.pid"
|
||||
$stopFile = "$tmp\stop.flag"
|
||||
$resultFile = "$tmp\received.txt"
|
||||
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
|
||||
|
||||
Add-Type -TypeDefinition @"
|
||||
using System;
|
||||
using System.Runtime.InteropServices;
|
||||
public static class FgApi {
|
||||
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
|
||||
}
|
||||
"@
|
||||
|
||||
function Adb {
|
||||
param([Parameter(Mandatory)][string[]]$AdbArgs)
|
||||
# adb writes progress ("1 file pulled...") to stderr; PS 5.1 with
|
||||
# $ErrorActionPreference=Stop turns every stderr line into a terminating
|
||||
# error. Same workaround build-android.ps1 documents: relax for native
|
||||
# calls, judge by exit code and captured stdout.
|
||||
$old = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
try {
|
||||
$out = & $adb @AdbArgs 2>$null
|
||||
return @{ out = ($out -join "`n"); code = $LASTEXITCODE }
|
||||
}
|
||||
finally { $ErrorActionPreference = $old }
|
||||
}
|
||||
|
||||
function Adb-Shell {
|
||||
param([Parameter(Mandatory)][string[]]$Cmd)
|
||||
return Adb -AdbArgs (@("shell") + $Cmd)
|
||||
}
|
||||
|
||||
function Wait-File {
|
||||
param([string]$Path, [int]$TimeoutMs = 20000)
|
||||
$t = 0
|
||||
while (-not (Test-Path $Path) -and $t -lt ($TimeoutMs / 150)) { Start-Sleep -Milliseconds 150; $t++ }
|
||||
if (-not (Test-Path $Path)) { throw "timeout waiting for $Path" }
|
||||
}
|
||||
|
||||
function Assert-Foreground([IntPtr]$hwnd, [string]$where) {
|
||||
# Bounded wait, not a single instant check: the target re-asserts itself
|
||||
# every 60ms in proof mode, so a momentary steal by the interactive desktop
|
||||
# is recoverable. Still refuse to inject if it never comes back - a stray
|
||||
# injection lands in whatever window DOES hold foreground.
|
||||
$deadline = (Get-Date).AddMilliseconds(1500)
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
if ([FgApi]::GetForegroundWindow() -eq $hwnd) { return }
|
||||
Start-Sleep -Milliseconds 30
|
||||
}
|
||||
throw "$where : foreground=$([FgApi]::GetForegroundWindow()) want=$hwnd after 1.5s - refusing to inject"
|
||||
}
|
||||
|
||||
function Read-Received {
|
||||
Wait-File $resultFile 8000
|
||||
$b = [System.IO.File]::ReadAllBytes($resultFile)
|
||||
$s = [System.Text.Encoding]::UTF8.GetString($b)
|
||||
if ($s.Length -gt 0 -and $s[0] -eq [char]0xFEFF) { $s = $s.Substring(1) }
|
||||
return $s
|
||||
}
|
||||
|
||||
function Get-PhoneUiText {
|
||||
[void](Adb-Shell @("uiautomator", "dump", "/sdcard/ui.xml"))
|
||||
$r = Adb -AdbArgs @("pull", "/sdcard/ui.xml", "$tmp\ui.xml")
|
||||
if ($r.code -ne 0 -or -not (Test-Path "$tmp\ui.xml")) { return "" }
|
||||
return [System.IO.File]::ReadAllText("$tmp\ui.xml", [System.Text.Encoding]::UTF8)
|
||||
}
|
||||
|
||||
function Wait-For-LiveSession([int]$Seconds, [ref]$outXml) {
|
||||
$deadline = (Get-Date).AddSeconds($Seconds)
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
Start-Sleep -Milliseconds 900
|
||||
$x = Get-PhoneUiText
|
||||
if ($x -match "RTT=") { $outXml.Value = $x; return $true }
|
||||
}
|
||||
$outXml.Value = (Get-PhoneUiText)
|
||||
return $false
|
||||
}
|
||||
|
||||
Write-Host "=== 0. stale-agent guard (same rule as the desktop proof) ==="
|
||||
$agentExe = "$repo\target\release\agent.exe"
|
||||
$ag = Get-Process agent -ErrorAction SilentlyContinue
|
||||
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
|
||||
Write-Host " running agent predates current build; restarting it"
|
||||
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
|
||||
Start-Sleep -Milliseconds 800
|
||||
$ag = $null
|
||||
}
|
||||
if (-not $ag) {
|
||||
Start-Process $agentExe -ArgumentList "--port",$Port,"--code",$Code -WindowStyle Hidden
|
||||
Start-Sleep -Seconds 1
|
||||
Write-Host " agent started (v3)"
|
||||
}
|
||||
|
||||
Write-Host "=== 1. launch target window (own process, empty field) ==="
|
||||
# Retire leftovers from earlier runs WITHOUT killing them. Killing a foreground
|
||||
# window hands foreground to the next app (this box's interactive GUI), and the
|
||||
# foreground lock then refuses a brand-new background process any claim - the
|
||||
# new target could never take foreground back.
|
||||
# Signal only stop files that ALREADY EXIST: creating a fresh one here would
|
||||
# hit the stop path of the run we are about to start, and that target would
|
||||
# close itself the moment it launched (seen 2026-10-30: pid reported then gone).
|
||||
$existingFlags = @(Get-ChildItem -Path $env:TEMP -Recurse -Filter "stop.flag" -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.FullName -like "*remote*proof*" })
|
||||
if ($existingFlags.Count -gt 0) {
|
||||
Write-Host " retiring $($existingFlags.Count) earlier target(s) via their own stop file"
|
||||
foreach ($f in $existingFlags) { (Get-Item $f.FullName).LastWriteTime = Get-Date }
|
||||
Start-Sleep -Seconds 3
|
||||
}
|
||||
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
|
||||
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
|
||||
"-ResultFile",$resultFile,"-StopFile",$stopFile,"-GrabForeground"
|
||||
Wait-File $pidFile
|
||||
# pid file carries "pid<n><hwnd>": the target writes its own handle once the
|
||||
# form is realized, which removes the Get-Process MainWindowHandle race.
|
||||
$parts = @((Get-Content $pidFile -Raw) -split "`n")
|
||||
$tid = [int]$parts[0].Trim()
|
||||
$hwnd = [IntPtr][long]$parts[1].Trim()
|
||||
if ($hwnd -eq 0) { throw "target reported a null window handle" }
|
||||
Assert-Foreground $hwnd "after launch"
|
||||
Write-Host " target pid=$tid hwnd=$($hwnd.ToInt64()) foreground=yes"
|
||||
|
||||
Write-Host "=== 2. start app, connect via intent extras ==="
|
||||
[void](Adb-Shell @("am", "force-stop", "com.verify.remote"))
|
||||
[void](Adb-Shell @("am", "start", "-n", "com.verify.remote/.MainActivity",
|
||||
"--es", "host", $PhoneIp, "--ei", "port", "$Port", "--es", "code", $Code))
|
||||
$xmlRef = ""
|
||||
if (-not (Wait-For-LiveSession 25 ([ref]$xmlRef))) { throw "phone never reported a live session (no RTT marker)" }
|
||||
$script:xml = $xmlRef
|
||||
Write-Host " phone: $([regex]::Match($script:xml,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
|
||||
Assert-Foreground $hwnd "after connect"
|
||||
|
||||
Write-Host "=== 3. phone keys A B C -> Activity.onKeyDown -> v3 Text channel ==="
|
||||
# 29/30/31 = KEYCODE_A/B/C. adb-injected keys arrive with a printable
|
||||
# unicodeChar, so the app routes them to nativeSendText (IME hidden). That is
|
||||
# the product path introduced in 2b, not a stand-in for it.
|
||||
foreach ($k in 29,30,31) {
|
||||
Assert-Foreground $hwnd "before keyevent $k"
|
||||
[void](Adb-Shell @("input", "keyevent", "$k"))
|
||||
Start-Sleep -Milliseconds 250
|
||||
}
|
||||
$deadline = (Get-Date).AddSeconds(10)
|
||||
$got = ""
|
||||
while ((Get-Date) -lt $deadline) {
|
||||
$got = Read-Received
|
||||
if ($got -eq "abc") { break }
|
||||
Start-Sleep -Milliseconds 300
|
||||
}
|
||||
Write-Host " target received: [$got]"
|
||||
if ($got -ne "abc") {
|
||||
Write-Host " foreground now: $([FgApi]::GetForegroundWindow()) target: $hwnd"
|
||||
throw "FULL-CHAIN FAIL: expected [abc] got [$got]"
|
||||
}
|
||||
Write-Host " PASS: Android key -> v3 Text -> agent KEYEVENTF_UNICODE -> Windows window"
|
||||
|
||||
Write-Host "=== 4. phone ENTER (66): control key uses the scancode channel ==="
|
||||
# Enter carries no printable unicodeChar, so it must go down the WinScancode
|
||||
# path. Observable acceptance: session survives and keeps streaming (a
|
||||
# single-line TextBox swallows Enter, so glyph-level proof is not possible).
|
||||
Assert-Foreground $hwnd "before ENTER"
|
||||
[void](Adb-Shell @("input", "keyevent", "66"))
|
||||
Start-Sleep -Milliseconds 900
|
||||
$xmlRef = ""
|
||||
if (-not (Wait-For-LiveSession 6 ([ref]$xmlRef))) { throw "session died after ENTER" }
|
||||
Write-Host " session alive: $([regex]::Match($xmlRef,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
|
||||
|
||||
Write-Host "=== 5. phone BACK (4): disconnect and return to form ==="
|
||||
[void](Adb-Shell @("input", "keyevent", "4"))
|
||||
Start-Sleep -Seconds 2
|
||||
$xml2 = Get-PhoneUiText
|
||||
if ($xml2 -match "RTT=") { throw "BACK did not disconnect (still live session)" }
|
||||
if ($xml2 -notmatch "android\.widget\.EditText") { throw "BACK did not return to the form" }
|
||||
Write-Host " PASS: disconnected, form visible again"
|
||||
|
||||
Write-Host "=== 6. connect again FROM THE FORM (V4 through real UI, not intent) ==="
|
||||
# Form typing uses keyevent, not 'adb input text': with the Chinese IME active,
|
||||
# text injection gets composed (ASCII '.' becomes U+3002). We disable the IME,
|
||||
# restore it in finally, and never assume a Chinese IME is absent.
|
||||
$origIme = (Adb-Shell @("settings", "get", "secure", "default_input_method")).out.Trim()
|
||||
Write-Host " original IME: [$origIme]"
|
||||
if ($origIme -and $origIme -ne "null") { [void](Adb-Shell @("ime", "disable", $origIme)) }
|
||||
try {
|
||||
$xml3 = Get-PhoneUiText
|
||||
$fields = [regex]::Matches($xml3, '<node[^>]*class="android\.widget\.EditText"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
|
||||
if ($fields.Count -lt 3) { throw "expected 3 form fields, found $($fields.Count)" }
|
||||
function Center($m) {
|
||||
$a = [int]$m.Groups[1].Value; $b = [int]$m.Groups[2].Value
|
||||
$c = [int]$m.Groups[3].Value; $d = [int]$m.Groups[4].Value
|
||||
return @(([int](($a + $c) / 2)), ([int](($b + $d) / 2)))
|
||||
}
|
||||
$ipC = Center $fields[0]; $codeC = Center $fields[2]
|
||||
# KEYCODE_0=7..KEYCODE_9=16, PERIOD=56, DEL=67, T=48 (A=29 -> T=29+19;
|
||||
# all from javap against android-34, not memory)
|
||||
[void](Adb-Shell @("input", "tap", "$($ipC[0])", "$($ipC[1])"))
|
||||
Start-Sleep -Milliseconds 400
|
||||
for ($i = 0; $i -lt 24; $i++) { [void](Adb-Shell @("input", "keyevent", "67")) }
|
||||
foreach ($k in @(8,16,9,56,8,13,15,56,8,56,8,15,7)) { # 192.168.1.180
|
||||
[void](Adb-Shell @("input", "keyevent", "$k"))
|
||||
}
|
||||
[void](Adb-Shell @("input", "tap", "$($codeC[0])", "$($codeC[1])"))
|
||||
Start-Sleep -Milliseconds 400
|
||||
for ($i = 0; $i -lt 6; $i++) { [void](Adb-Shell @("input", "keyevent", "67")) }
|
||||
[void](Adb-Shell @("input", "keyevent", "48"))
|
||||
$xml3 = Get-PhoneUiText
|
||||
$vals = [regex]::Matches($xml3, '<node[^>]*class="android\.widget\.EditText"[^>]*text="([^"]*)"') |
|
||||
ForEach-Object { $_.Groups[1].Value }
|
||||
Write-Host " form now: $($vals -join ' | ')"
|
||||
$btn = [regex]::Match($xml3, '<node[^>]*class="android\.widget\.Button"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"')
|
||||
if (-not $btn.Success) { throw "connect button not found" }
|
||||
$bc = Center $btn
|
||||
Assert-Foreground $hwnd "before form connect"
|
||||
[void](Adb-Shell @("input", "tap", "$($bc[0])", "$($bc[1])"))
|
||||
$xmlRef = ""
|
||||
if (-not (Wait-For-LiveSession 25 ([ref]$xmlRef))) {
|
||||
$shown = [regex]::Matches($xmlRef, 'text="([^"]+)"') | ForEach-Object { $_.Groups[1].Value }
|
||||
throw "V4 FAIL: form connect did not reach a live session. screen=[$($shown -join ' | ')]"
|
||||
}
|
||||
Write-Host " V4 PASS via form UI: $([regex]::Match($xmlRef,'text="([^"]*RTT=[^"]*)"').Groups[1].Value)"
|
||||
}
|
||||
finally {
|
||||
if ($origIme -and $origIme -ne "null") {
|
||||
[void](Adb-Shell @("ime", "enable", $origIme))
|
||||
[void](Adb-Shell @("ime", "set", $origIme))
|
||||
Write-Host " IME restored: $((Adb-Shell @("settings", "get", "secure", "default_input_method")).out.Trim())"
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "=== 7. stop target ==="
|
||||
New-Item -ItemType File -Force -Path $stopFile | Out-Null
|
||||
Start-Sleep -Milliseconds 600
|
||||
Write-Host "=== PHONE CHAIN PROOF COMPLETE ==="
|
||||
@@ -1,365 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""UDP 打洞探针 —— 两台不同网络的机器各跑一个，用来摸清你们线路的底牌。
|
||||
|
||||
回答四个问题：
|
||||
1. 这条线路的 NAT 能不能打洞？
|
||||
2. 要几轮才能打通？（轮数越少说明 NAT 越宽松）
|
||||
3. 对方的源端口有没有被 NAT 改写？（判断是不是对称 NAT）
|
||||
4. 有没有公网 IPv6？IPv6 直连通不通？
|
||||
|
||||
用法
|
||||
----
|
||||
A 机（假设公网 1.2.3.4）:
|
||||
py punch_probe.py --peer 2.3.4.5:47800
|
||||
|
||||
B 机（假设公网 2.3.4.5）:
|
||||
py punch_probe.py --peer 1.2.3.4:47800
|
||||
|
||||
IPv6（双方都有公网 v6 时）:
|
||||
py punch_probe.py --peer [2408:8207:1234::1]:47800
|
||||
|
||||
先查自己的公网映射（走 STUN，不需要对方）:
|
||||
py punch_probe.py --stun
|
||||
|
||||
被动监听（不发包，只看能不能收到对方打进来的包）:
|
||||
py punch_probe.py
|
||||
|
||||
说明
|
||||
----
|
||||
- 两台机器必须**同时**启动，脚本靠「双方同时向对方发包」来穿透 NAT。
|
||||
- 先各自查公网 IP（浏览器搜 "my ip" 或 --stun），再互相填。
|
||||
- 只依赖 Python 标准库。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
MAGIC = b"PUNCHv1|"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 地址解析
|
||||
|
||||
def parse_addr(s):
|
||||
"""解析 'ip:port' 或 '[v6]:port' -> (host, port, family)"""
|
||||
s = s.strip()
|
||||
if s.startswith("["):
|
||||
end = s.index("]")
|
||||
host = s[1:end]
|
||||
port = int(s[end + 2:])
|
||||
return host, port, socket.AF_INET6
|
||||
host, _, p = s.rpartition(":")
|
||||
return host, int(p), socket.AF_INET
|
||||
|
||||
|
||||
def fmt_addr(sockaddr):
|
||||
host, port = sockaddr[0], sockaddr[1]
|
||||
if ":" in host:
|
||||
return "[%s]:%d" % (host, port)
|
||||
return "%s:%d" % (host, port)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- STUN
|
||||
|
||||
def stun_mapped(server="stun.l.google.com", port=19302, timeout=3.0, sock=None):
|
||||
"""最小 STUN Binding 请求，返回 (ip, port)；失败返回 None。
|
||||
|
||||
sock 不为 None 时复用它（用来测「同一个本地端口对另一个目标」的映射端口，
|
||||
这是判断 NAT 是不是对称型的关键）。
|
||||
"""
|
||||
txid = b"\x5a" * 12
|
||||
req = struct.pack(">HHI12s", 0x0001, 0, 0x2112A442, txid)
|
||||
try:
|
||||
infos = socket.getaddrinfo(server, port, 0, socket.SOCK_DGRAM)
|
||||
except OSError:
|
||||
return None
|
||||
fam, sockaddr = infos[0][0], infos[0][4]
|
||||
own = sock is None
|
||||
s = socket.socket(fam, socket.SOCK_DGRAM) if own else sock
|
||||
s.settimeout(timeout)
|
||||
try:
|
||||
s.sendto(req, sockaddr)
|
||||
data, _ = s.recvfrom(2048)
|
||||
except OSError:
|
||||
return None
|
||||
finally:
|
||||
if own:
|
||||
s.close()
|
||||
|
||||
if len(data) < 20:
|
||||
return None
|
||||
off = 20
|
||||
while off + 4 <= len(data):
|
||||
atype, alen = struct.unpack(">HH", data[off:off + 4])
|
||||
val = data[off + 4:off + 4 + alen]
|
||||
if atype == 0x0020 and len(val) >= 8: # XOR-MAPPED-ADDRESS
|
||||
famv = val[1]
|
||||
xport = struct.unpack(">H", val[2:4])[0] ^ 0x2112
|
||||
if famv == 0x01 and len(val) >= 8:
|
||||
ip = bytes(a ^ b for a, b in zip(val[4:8], b"\x21\x12\xA4\x42"))
|
||||
return socket.inet_ntoa(ip), xport
|
||||
if famv == 0x02 and len(val) >= 20:
|
||||
mask = b"\x21\x12\xA4\x42" + txid
|
||||
ip = bytes(a ^ b for a, b in zip(val[4:20], mask))
|
||||
return socket.inet_ntop(socket.AF_INET6, ip), xport
|
||||
off += 4 + alen + ((4 - alen % 4) % 4)
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 信令
|
||||
|
||||
def discover_peer(sock, server, room, timeout=60.0):
|
||||
"""注册到房间，等服务器回推对方的公网地址。
|
||||
|
||||
注意：这里**复用同一个 socket**——服务器看到的源地址，就是 NAT 为
|
||||
「本机 → 服务器」这条流分配的映射。对锥形 NAT 而言，它与「本机 → 对端」
|
||||
的映射相同，所以拿它去打洞是有效的；对对称 NAT 则不同，打不通属预期。
|
||||
"""
|
||||
host, port = server[0], server[1]
|
||||
deadline = time.time() + timeout
|
||||
last_send = 0.0
|
||||
while time.time() < deadline:
|
||||
now = time.time()
|
||||
if now - last_send > 1.0:
|
||||
try:
|
||||
sock.sendto(b"REGISTER|" + room.encode("utf-8"), (host, port))
|
||||
except OSError as e:
|
||||
print(" 注册发送失败：%s" % e)
|
||||
return None
|
||||
last_send = now
|
||||
try:
|
||||
data, _src = sock.recvfrom(2048)
|
||||
except socket.timeout:
|
||||
continue
|
||||
except ConnectionResetError:
|
||||
continue
|
||||
except OSError as e:
|
||||
if getattr(e, "winerror", None) == 10054:
|
||||
continue
|
||||
return None
|
||||
txt = data.decode("utf-8", "replace").strip()
|
||||
if txt.startswith("PEER|"):
|
||||
h, p, _f = parse_addr(txt[5:])
|
||||
return (h, p)
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 主逻辑
|
||||
|
||||
class Probe:
|
||||
def __init__(self, bind_host, port, family):
|
||||
self.sock = socket.socket(family, socket.SOCK_DGRAM)
|
||||
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
if family == socket.AF_INET6:
|
||||
try:
|
||||
self.sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
|
||||
except OSError:
|
||||
pass
|
||||
self.sock.bind((bind_host, port))
|
||||
self.sock.settimeout(0.2)
|
||||
# Windows 特有坑：曾经发过包给对方，但对方端口没开 → 会收到 ICMP 端口不可达，
|
||||
# 之后下一次 recv 直接抛 WSAECONNRESET(10054)。不打这个 ioctl，
|
||||
# 接收循环会被"自己之前的失败发送"打断，之后再也收不到包。
|
||||
# （Rust 实现同样要处理：tokio recv 返回 ConnectionReset 时不能退出循环）
|
||||
if sys.platform == "win32":
|
||||
try:
|
||||
self.sock.ioctl(socket.SIO_UDP_CONNRESET, False)
|
||||
except (AttributeError, OSError):
|
||||
pass
|
||||
self.port = self.sock.getsockname()[1]
|
||||
self.stop = threading.Event()
|
||||
self.first_src = None
|
||||
self.packets = 0
|
||||
self.junk = 0
|
||||
|
||||
def receiver(self, ignore=()):
|
||||
"""接收线程。
|
||||
|
||||
只认「以 MAGIC 开头」且「来自对端而非信令服务器」的包。
|
||||
信令服务器的 PEER/WAIT 回复绝不能算作打通——否则客户端反复注册引来的
|
||||
回复会造成假阳性（已踩过：把服务器的 PEER 回复误判成打洞成功）。
|
||||
"""
|
||||
while not self.stop.is_set():
|
||||
try:
|
||||
data, src = self.sock.recvfrom(2048)
|
||||
except socket.timeout:
|
||||
continue
|
||||
except ConnectionResetError:
|
||||
# ICMP 端口不可达留下的错误，忽略并继续收（绝不能 break）
|
||||
continue
|
||||
except OSError as e:
|
||||
if getattr(e, "winerror", None) == 10054:
|
||||
continue
|
||||
break
|
||||
|
||||
# ---- 过滤：非打洞包一律丢弃
|
||||
if not data.startswith(MAGIC) or (ignore and src[:2] in ignore):
|
||||
self.junk += 1
|
||||
continue
|
||||
|
||||
if self.first_src is None:
|
||||
self.first_src = src
|
||||
print("\n ✅ 收到对方的包！")
|
||||
print(" 对方源地址（NAT 之后）= %s" % fmt_addr(src))
|
||||
if src[1] != self.port:
|
||||
print(" 对方源端口 = %d（本机端口 %d）；若双方约定端口相同，"
|
||||
"说明 NAT 改写了端口" % (src[1], self.port))
|
||||
else:
|
||||
print(" 对方源端口与约定端口一致 → NAT 保留了端口")
|
||||
self.packets += 1
|
||||
if self.packets <= 3:
|
||||
print(" [%s] 收到 %d 字节" % (fmt_addr(src), len(data)))
|
||||
|
||||
def send_rounds(self, peer_sockaddr, rounds, interval):
|
||||
for i in range(1, rounds + 1):
|
||||
if self.stop.is_set():
|
||||
return
|
||||
payload = MAGIC + struct.pack(">d", time.time())
|
||||
try:
|
||||
self.sock.sendto(payload, peer_sockaddr)
|
||||
except OSError as e:
|
||||
print(" 发送失败：%s" % e)
|
||||
return
|
||||
if i == 1 or i % 5 == 0 or self.first_src is not None:
|
||||
mark = "✅" if self.first_src is not None else " "
|
||||
print(" %s 第 %2d/%d 轮已发出" % (mark, i, rounds))
|
||||
if self.first_src is not None:
|
||||
return
|
||||
time.sleep(interval)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(add_help=True, description="UDP 打洞探针")
|
||||
ap.add_argument("--peer", help="对方地址 ip:port 或 [v6]:port；不给则只被动监听")
|
||||
ap.add_argument("--port", type=int, default=47800, help="本地端口（默认 47800）")
|
||||
ap.add_argument("--rounds", type=int, default=25, help="发包轮数（默认 25）")
|
||||
ap.add_argument("--interval", type=float, default=0.2, help="轮间隔秒（默认 0.2）")
|
||||
ap.add_argument("--grace", type=float, default=2.5,
|
||||
help="发完后继续监听多少秒等回包（默认 2.5）")
|
||||
ap.add_argument("--server", help="信令服务器 ip:port（与 --room 配合，自动配对）")
|
||||
ap.add_argument("--room", help="房间名；同一房间的两个客户端会被配对")
|
||||
ap.add_argument("--wait", type=float, default=60.0,
|
||||
help="等对方加入房间的秒数（默认 60）")
|
||||
ap.add_argument("--stun", action="store_true", help="只查询自己的公网映射后退出")
|
||||
ap.add_argument("--stun-port", type=int, default=0,
|
||||
help="配合 --stun：用指定本地端口发 STUN（与打洞端口一致时才能对比映射）")
|
||||
args = ap.parse_args()
|
||||
|
||||
# ---- 只查 STUN
|
||||
if args.stun:
|
||||
s = None
|
||||
if args.stun_port:
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
s.bind(("0.0.0.0", args.stun_port))
|
||||
print("[STUN] 用本机端口 %d 查询（与打洞同端口）..." % args.stun_port)
|
||||
else:
|
||||
print("[STUN] 查询公网映射（临时端口）...")
|
||||
r = stun_mapped(sock=s)
|
||||
if s:
|
||||
s.close()
|
||||
if r:
|
||||
print(" 对 Google STUN 的映射 = %s:%d" % r)
|
||||
if args.stun_port:
|
||||
print(" 本机端口 = %d" % args.stun_port)
|
||||
print(" 下一步：对比信令服务器日志里记录的源端口")
|
||||
print(" 两者相同 → 端点无关映射（锥形 NAT）→ 打洞大概率成功")
|
||||
print(" 两者不同 → 对称 NAT → 打洞困难，需要端口预测或中继")
|
||||
else:
|
||||
print(" 查询失败（STUN 19302 可能被封；可稍后重试）")
|
||||
return 0
|
||||
|
||||
# ---- 决定地址族与模式
|
||||
peer = None
|
||||
if args.peer:
|
||||
host, pport, family = parse_addr(args.peer)
|
||||
peer = (host, pport)
|
||||
bind_host = "::" if family == socket.AF_INET6 else "0.0.0.0"
|
||||
mode = "双向打洞（手工填地址）"
|
||||
elif args.server and args.room:
|
||||
_h, _p, family = parse_addr(args.server)
|
||||
bind_host = "::" if family == socket.AF_INET6 else "0.0.0.0"
|
||||
mode = "信令自动配对 + 打洞"
|
||||
else:
|
||||
family = socket.AF_INET
|
||||
bind_host = "0.0.0.0"
|
||||
mode = "被动监听"
|
||||
|
||||
probe = Probe(bind_host, args.port, family)
|
||||
print("=" * 60)
|
||||
print(" UDP 打洞探针 模式=%s" % mode)
|
||||
print(" 本机监听 %s:%d" % (bind_host, probe.port))
|
||||
if peer:
|
||||
print(" 发往 %s （%s）" % (fmt_addr(peer + (0, 0)),
|
||||
"IPv6" if family == socket.AF_INET6 else "IPv4"))
|
||||
print("=" * 60)
|
||||
|
||||
# ---- 信令配对：不用自己查公网 IP，服务器看到的就是你的公网映射
|
||||
if args.server and args.room and peer is None:
|
||||
srv = parse_addr(args.server)[:2]
|
||||
print("\n向信令服务器 %s 注册 room=%s ..." % (fmt_addr(srv), args.room))
|
||||
found = discover_peer(probe.sock, srv, args.room, timeout=args.wait)
|
||||
if found is None:
|
||||
print(" ❌ 超时：没等到对方加入同一个房间")
|
||||
probe.sock.close()
|
||||
return 1
|
||||
peer = found
|
||||
print(" ✅ 配对成功，对方公网地址 = %s" % fmt_addr(found))
|
||||
|
||||
# 信令服务器的回复（PEER/WAIT）绝不能算作打通，接收线程必须把它排除掉
|
||||
ignore = set()
|
||||
if args.server:
|
||||
_sh, _sp, _sf = parse_addr(args.server)
|
||||
ignore.add((_sh, _sp))
|
||||
|
||||
t = threading.Thread(target=probe.receiver, args=(ignore,), daemon=True)
|
||||
t.start()
|
||||
|
||||
try:
|
||||
if peer:
|
||||
print("\n开始同时发包（若对方也在发包，第 1~3 轮通常就通）...")
|
||||
probe.send_rounds(peer, args.rounds, args.interval)
|
||||
# 对方可能比我晚启动：发完后继续监听，否则会把"晚到的成功"误判成失败
|
||||
if probe.first_src is None:
|
||||
print("\n发包结束，继续监听 %.1f 秒等对方回包 ..." % args.grace)
|
||||
waited = 0.0
|
||||
while waited < args.grace and probe.first_src is None:
|
||||
time.sleep(0.1)
|
||||
waited += 0.1
|
||||
else:
|
||||
print("\n被动监听中，Ctrl+C 退出 ...")
|
||||
while not probe.stop.is_set():
|
||||
time.sleep(0.3)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
probe.stop.set()
|
||||
time.sleep(0.3)
|
||||
probe.sock.close()
|
||||
|
||||
print("\n" + "-" * 60)
|
||||
if probe.first_src is not None:
|
||||
print("结论：✅ 打洞成功")
|
||||
print(" 对方实际源地址 = %s" % fmt_addr(probe.first_src))
|
||||
print(" 共收到 %d 个包" % probe.packets)
|
||||
else:
|
||||
print("结论：❌ 本轮没有收到对方的打洞包")
|
||||
if probe.junk:
|
||||
print(" （另外收到 %d 个非打洞包并已忽略：那是信令服务器的回复，不算打通）"
|
||||
% probe.junk)
|
||||
print(" 可能原因（按概率排序）：")
|
||||
print(" 1. 一端或两端是对称型 NAT / 运营商级 NAT(CGNAT) → 需要端口预测或中继")
|
||||
print(" 2. 一方根本没发包（对方没启动、或地址填错/端口填错）")
|
||||
print(" 3. UDP 被防火墙拦截（企业网络常见）")
|
||||
print(" 4. 双方没有真正同时发包（时间差超过 NAT 表项寿命）")
|
||||
print(" 下一步：两端各跑一次 --stun，看看是不是落在 100.64~100.127 网段（CGNAT）")
|
||||
print("-" * 60)
|
||||
return 0 if probe.first_src is not None else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,90 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""极简 rendezvous（信令）服务器 —— POC 用
|
||||
|
||||
作用只有一个：**告诉双方「对方看到的你」是什么地址**。
|
||||
双方注册到同一个房间后，服务器把各自观察到的公网地址回推给对方，
|
||||
之后双方直接打洞，服务器不再参与（也不承载任何媒体流量）。
|
||||
|
||||
用法（跑在有公网 IP 的机器上，比如 43.140.215.244）:
|
||||
py rendezvous.py --port 47999
|
||||
|
||||
客户端：
|
||||
py punch_probe.py --server <服务器IP>:47999 --room test01 --port 47800
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
|
||||
# 房间成员多久没动静就清掉
|
||||
MEMBER_TTL = 30.0
|
||||
|
||||
|
||||
def fmt(sockaddr):
|
||||
host, port = sockaddr[0], sockaddr[1]
|
||||
return "[%s]:%d" % (host, port) if ":" in host else "%s:%d" % (host, port)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="极简 rendezvous 服务器")
|
||||
ap.add_argument("--bind", default="0.0.0.0")
|
||||
ap.add_argument("--port", type=int, default=47999)
|
||||
args = ap.parse_args()
|
||||
|
||||
# 重定向到文件时 stdout 会全缓冲，日志要等进程退出才出现；强制行缓冲
|
||||
try:
|
||||
sys.stdout.reconfigure(line_buffering=True)
|
||||
except (AttributeError, OSError):
|
||||
pass
|
||||
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
sock.bind((args.bind, args.port))
|
||||
|
||||
print("[rendezvous] 监听 udp://%s:%d" % (args.bind, args.port))
|
||||
print("[rendezvous] 房间成员 TTL = %.0fs，Ctrl+C 退出\n" % MEMBER_TTL)
|
||||
|
||||
rooms = {} # room -> [(addr, last_seen)]
|
||||
|
||||
try:
|
||||
while True:
|
||||
data, src = sock.recvfrom(2048)
|
||||
txt = data.decode("utf-8", "replace").strip()
|
||||
now = time.time()
|
||||
|
||||
if not txt.startswith("REGISTER|"):
|
||||
continue
|
||||
room = txt.split("|", 1)[1].strip()
|
||||
if not room:
|
||||
continue
|
||||
|
||||
# 清掉过期成员 + 去掉本机的旧记录（支持重复注册）
|
||||
members = [(a, t) for (a, t) in rooms.get(room, [])
|
||||
if now - t < MEMBER_TTL and a[1] != src[1]]
|
||||
members.append((src, now))
|
||||
rooms[room] = members
|
||||
|
||||
print("[rendezvous] %s 注册 room=%s (现有 %d 人) 源地址=%s"
|
||||
% (fmt(src), room, len(members), fmt(src)))
|
||||
|
||||
if len(members) >= 2:
|
||||
a1, _ = members[0]
|
||||
a2, _ = members[1]
|
||||
sock.sendto(b"PEER|" + fmt(a2).encode(), a1)
|
||||
sock.sendto(b"PEER|" + fmt(a1).encode(), a2)
|
||||
print("[rendezvous] -> 配对成功：%s <==> %s"
|
||||
% (fmt(a1), fmt(a2)))
|
||||
else:
|
||||
sock.sendto(b"WAIT", src)
|
||||
print("[rendezvous] -> 等待第二个人加入 ...")
|
||||
except KeyboardInterrupt:
|
||||
print("\n[rendezvous] 退出")
|
||||
finally:
|
||||
sock.close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,23 +0,0 @@
|
||||
# Byte-safe device screenshot helper.
|
||||
# ASCII-only. PS 5.1 `>` redirection writes UTF-16, which corrupts PNG bytes
|
||||
# (measured 2026-10-30: `adb exec-out screencap -p > x.png` produced a file
|
||||
# starting with FF FE). Capture on-device and pull instead.
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][string]$Out,
|
||||
[string]$Device = "/sdcard/shot.png"
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
$adb = "D:\working\android\sdk\platform-tools\adb.exe"
|
||||
$old = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
try {
|
||||
& $adb shell screencap -p $Device | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) { throw "screencap failed ($LASTEXITCODE)" }
|
||||
& $adb pull $Device $Out | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) { throw "pull failed ($LASTEXITCODE)" }
|
||||
}
|
||||
finally { $ErrorActionPreference = $old }
|
||||
$b = [System.IO.File]::ReadAllBytes($Out)
|
||||
$sig = ($b[0..3] | ForEach-Object { $_.ToString("X2") }) -join " "
|
||||
if ($sig -ne "89 50 4E 47") { throw "not a PNG (sig=$sig): $Out" }
|
||||
Write-Host ("OK {0} ({1:N0} bytes)" -f (Split-Path $Out -Leaf), $b.Length)
|
||||
@@ -1,171 +0,0 @@
|
||||
# 2b V6/V7 proof: v3 text injection into a dedicated WinForms target window,
|
||||
# read back from that process itself.
|
||||
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
|
||||
# Non-ASCII payloads are built from explicit code points, never literals.
|
||||
#
|
||||
# Usage:
|
||||
# powershell -ExecutionPolicy Bypass -File tools\text_injection_proof.ps1
|
||||
#
|
||||
# Chain: viewer-core text_check --units <utf16> -> v3 agent SendInput
|
||||
# KEYEVENTF_UNICODE -> foreground WinForms TextBox -> self-reported text.
|
||||
#
|
||||
# Why NOT Notepad (2026-10-30): Windows 11 Notepad hosts all tabs in ONE
|
||||
# process and this dev box had a user document open in it - injecting "into
|
||||
# Notepad" would mean injecting into the user's real file. The WinForms target
|
||||
# is a separate process we own, opened on an empty field.
|
||||
#
|
||||
# Foreground handling (measured, not assumed): a form with TopMost=true that
|
||||
# is activated at startup KEEPS foreground for seconds; AppActivate /
|
||||
# SetForegroundWindow called from here actually STEALS it away. So we assert
|
||||
# the target is foreground before injecting, and refuse to inject otherwise -
|
||||
# a stray injection would land in whatever window was foreground.
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
# Repo root derived from this script's location (tools/ is one level under the
|
||||
# repo root) so the proof runs on any clone, not just the author's D: drive.
|
||||
$repo = (Resolve-Path "$PSScriptRoot\..").Path
|
||||
$tmp = "$env:TEMP\remote-text-proof"
|
||||
$base = "D:\working\environment"
|
||||
|
||||
$env:RUSTUP_HOME = "$base\rust"
|
||||
$env:CARGO_HOME = "$base\cargo"
|
||||
$env:PATH = "$base\mingw64\bin;$base\cargo\bin;" + $env:PATH
|
||||
|
||||
Add-Type -TypeDefinition @"
|
||||
using System;
|
||||
using System.Runtime.InteropServices;
|
||||
public static class WinFg {
|
||||
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
|
||||
}
|
||||
"@
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
|
||||
$pidFile = "$tmp\target.pid"
|
||||
$stopFile = "$tmp\stop.flag"
|
||||
$resultFile = "$tmp\received.txt"
|
||||
Remove-Item $pidFile,$stopFile,$resultFile -ErrorAction SilentlyContinue
|
||||
|
||||
function Wait-File {
|
||||
param([string]$Path, [int]$TimeoutMs = 15000)
|
||||
$tries = 0
|
||||
while (-not (Test-Path $Path) -and $tries -lt ($TimeoutMs / 150)) {
|
||||
Start-Sleep -Milliseconds 150; $tries++
|
||||
}
|
||||
if (-not (Test-Path $Path)) { throw "file did not appear: $Path" }
|
||||
}
|
||||
|
||||
function Read-Received {
|
||||
Wait-File $resultFile
|
||||
$bytes = [System.IO.File]::ReadAllBytes($resultFile)
|
||||
$text = [System.Text.Encoding]::UTF8.GetString($bytes)
|
||||
if ($text.Length -gt 0 -and $text[0] -eq [char]0xFEFF) { $text = $text.Substring(1) }
|
||||
return $text
|
||||
}
|
||||
|
||||
function CodePointsToString([int[]]$cps) {
|
||||
-join ($cps | ForEach-Object { [char]$_ })
|
||||
}
|
||||
|
||||
function Show-Text([string]$s) {
|
||||
# ASCII-safe console preview: non-ASCII rendered as U+XXXX
|
||||
($s.ToCharArray() | ForEach-Object {
|
||||
if ([int]$_ -ge 32 -and [int]$_ -lt 127) { $_ } else { "U+{0:X4}" -f [int]$_ }
|
||||
}) -join ""
|
||||
}
|
||||
|
||||
function Assert-TargetForeground([IntPtr]$hwnd) {
|
||||
$fg = [WinFg]::GetForegroundWindow()
|
||||
if ($fg -ne $hwnd) {
|
||||
throw "foreground is $fg, not target $hwnd - refusing to inject (would hit another app)"
|
||||
}
|
||||
}
|
||||
|
||||
function Send-Units([int[]]$cps, [IntPtr]$hwnd, [int]$delayMs = 1500) {
|
||||
$units = $cps -join ","
|
||||
$exe = "$repo\target\debug\examples\text_check.exe"
|
||||
if (-not (Test-Path $exe)) { throw "missing $exe (cargo build -p viewer-core --example text_check)" }
|
||||
Assert-TargetForeground $hwnd
|
||||
& $exe --port 47800 --code t --units $units --delay $delayMs --hold 800
|
||||
$code = $LASTEXITCODE
|
||||
Assert-TargetForeground $hwnd # 注入窗口在 --delay 期间必须仍是靶窗
|
||||
if ($code -ne 0) { throw "text_check exited $code" }
|
||||
Start-Sleep -Milliseconds 500
|
||||
}
|
||||
|
||||
Write-Host "=== 0. ensure CURRENT v3 agent is the one running ==="
|
||||
$ag = Get-Process agent -ErrorAction SilentlyContinue
|
||||
$agentExe = "$repo\target\release\agent.exe"
|
||||
if ($ag -and (Get-Item $agentExe).LastWriteTime -gt $ag.StartTime) {
|
||||
Write-Host " running agent predates current build; restarting"
|
||||
Stop-Process -Name agent -Force -ErrorAction SilentlyContinue
|
||||
Start-Sleep -Milliseconds 800
|
||||
$ag = $null
|
||||
}
|
||||
if (-not $ag) {
|
||||
Start-Process -FilePath $agentExe -ArgumentList "--port","47800","--code","t" -WindowStyle Hidden
|
||||
Start-Sleep -Seconds 1
|
||||
Write-Host " agent started (v3)"
|
||||
}
|
||||
|
||||
Write-Host "=== 1. launch WinForms target (own process, empty field) ==="
|
||||
# NO -WindowStyle Hidden: the hidden host makes the form invisible, and an
|
||||
# invisible window can never be foreground (measured 2026-10-30).
|
||||
Start-Process powershell.exe -ArgumentList "-NoProfile","-ExecutionPolicy","Bypass",`
|
||||
"-File","$repo\tools\text_target.ps1","-PidFile",$pidFile,`
|
||||
"-ResultFile",$resultFile,"-StopFile",$stopFile
|
||||
Wait-File $pidFile
|
||||
# target.pid holds "pid<n><hwnd>" (see text_target.ps1 Shown handler); take line 1.
|
||||
$parts = @((Get-Content $pidFile -Raw) -split "`n")
|
||||
$tid = [int]$parts[0].Trim()
|
||||
$tp = Get-Process -Id $tid
|
||||
Start-Sleep -Milliseconds 1200
|
||||
$hwnd = [IntPtr][long]$parts[1].Trim()
|
||||
if ($hwnd -eq 0) { throw "target reported a null window handle" }
|
||||
Write-Host " target pid=$tid hwnd=$hwnd title=[$($tp.MainWindowTitle)]"
|
||||
Assert-TargetForeground $hwnd
|
||||
Write-Host " target is foreground: yes"
|
||||
|
||||
Write-Host "=== 2. V6: four CJK BMP characters (Ni Hao Shi Jie) ==="
|
||||
$cjk = @(0x4F60, 0x597D, 0x4E16, 0x754C)
|
||||
Send-Units $cjk $hwnd
|
||||
$got6 = Read-Received
|
||||
$exp6 = CodePointsToString $cjk
|
||||
Write-Host (" received [{0}] expected [{1}]" -f (Show-Text $got6), (Show-Text $exp6))
|
||||
if ($got6 -ne $exp6) { throw "V6 FAIL" }
|
||||
Write-Host " V6 PASS - BMP CJK arrived intact through v3 Text + KEYEVENTF_UNICODE"
|
||||
|
||||
Write-Host "=== 3. V7: exclaim + emoji U+1F600 (UTF-16 surrogate pair) appended ==="
|
||||
$mix = @(0x0021, 0xD83D, 0xDE00)
|
||||
Send-Units $mix $hwnd
|
||||
$got7 = Read-Received
|
||||
$exp7 = $exp6 + (CodePointsToString $mix)
|
||||
Write-Host (" received len={0} [{1}]" -f $got7.Length, (Show-Text $got7))
|
||||
Write-Host (" expected len={0} [{1}]" -f $exp7.Length, (Show-Text $exp7))
|
||||
if ($got7 -ne $exp7) { throw "V7 FAIL - mismatch" }
|
||||
if ($got7.Contains([char]0xFFFD)) { throw "V7 FAIL - U+FFFD present: pair got split" }
|
||||
Write-Host " V7 PASS - surrogate pair survived as one character, no replacement char"
|
||||
|
||||
Write-Host "=== 4. negative guard: missing --units must NOT inject anything ==="
|
||||
$before = Read-Received
|
||||
$exe = "$repo\target\debug\examples\text_check.exe"
|
||||
# Omit --units entirely (passing an empty string would be eaten by PowerShell
|
||||
# argument handling and parsed as the next flag). viewer-core's parse_units
|
||||
# rejects the degenerate input before any session is even opened.
|
||||
# No 2>&1 here: viewer-core logs go to stderr by design; merging them turns
|
||||
# every normal log line into a PowerShell error record.
|
||||
& $exe --port 47800 --code t --delay 100 --hold 100 | Out-Null
|
||||
$emptyCode = $LASTEXITCODE
|
||||
Write-Host " exit=$emptyCode (expected non-zero: tool rejects empty --units)"
|
||||
if ($emptyCode -eq 0) { throw "FAIL - empty units should be rejected by the tool" }
|
||||
Start-Sleep -Milliseconds 400
|
||||
$after = Read-Received
|
||||
if ($after -ne $before) { throw "FAIL - target text changed unexpectedly: [$after]" }
|
||||
Write-Host " guard PASS - no injection happened, target text unchanged"
|
||||
|
||||
Write-Host "=== 5. stop target ==="
|
||||
New-Item -ItemType File -Force -Path $stopFile | Out-Null
|
||||
$tries = 0
|
||||
while ((Get-Process -Id $tid -ErrorAction SilentlyContinue) -and $tries -lt 40) {
|
||||
Start-Sleep -Milliseconds 150; $tries++
|
||||
}
|
||||
Write-Host "=== PROOF COMPLETE: final target text = [$(Show-Text (Read-Received))] ==="
|
||||
@@ -1,110 +0,0 @@
|
||||
# 2b V6/V7 proof target: a dedicated WinForms window we own.
|
||||
# ASCII-only source (PS 5.1 decodes BOM-less .ps1 as GBK; see personal lessons).
|
||||
#
|
||||
# Launched by text_injection_proof.ps1. Shows a topmost empty TextBox, writes
|
||||
# its current text to ResultFile (UTF-8 with BOM) on every change, exits when
|
||||
# StopFile appears. The proof injects through the remote-control pipeline and
|
||||
# polls ResultFile - so "what arrived on the controlled desktop" is reported
|
||||
# by the receiving application itself, not by a screenshot or OCR.
|
||||
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][string]$PidFile,
|
||||
[Parameter(Mandatory = $true)][string]$ResultFile,
|
||||
[Parameter(Mandatory = $true)][string]$StopFile,
|
||||
# SendInput lands on whatever is FOREGROUND. When the injector is driven
|
||||
# from another machine (phone -> agent -> this desktop), nothing in this
|
||||
# process keeps our window on top, so an unrelated window could eat the
|
||||
# injection. With this switch the form re-asserts foreground every tick and
|
||||
# the caller still asserts foreground before injecting.
|
||||
[switch]$GrabForeground
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
|
||||
# Event handler scriptblocks resolve script-scoped vars; make that explicit
|
||||
# instead of relying on the param locals staying on a visible scope chain.
|
||||
$script:PidFile = $PidFile
|
||||
$script:ResultFile = $ResultFile
|
||||
$script:StopFile = $StopFile
|
||||
$script:lastWritten = $null
|
||||
|
||||
$form = New-Object System.Windows.Forms.Form
|
||||
$form.Text = "RemoteTextTarget"
|
||||
$form.ClientSize = New-Object System.Drawing.Size(420, 140)
|
||||
$form.TopMost = $true
|
||||
$form.StartPosition = "Manual"
|
||||
$form.Location = New-Object System.Drawing.Point(80, 80)
|
||||
|
||||
$tb = New-Object System.Windows.Forms.TextBox
|
||||
$tb.Multiline = $true
|
||||
$tb.Font = New-Object System.Drawing.Font("Segoe UI", 16.0)
|
||||
$tb.Dock = "Fill"
|
||||
$form.Controls.Add($tb)
|
||||
|
||||
$script:GrabForeground = $GrabForeground
|
||||
if ($script:GrabForeground) {
|
||||
Add-Type -TypeDefinition @"
|
||||
using System;
|
||||
using System.Runtime.InteropServices;
|
||||
public static class OwnForeground {
|
||||
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
|
||||
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h);
|
||||
[DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr h, IntPtr pid);
|
||||
[DllImport("kernel32.dll")] public static extern uint GetCurrentThreadId();
|
||||
[DllImport("user32.dll")] public static extern bool AttachThreadInput(uint a, uint b, bool f);
|
||||
|
||||
// Deliberately NO keybd_event(VK_MENU). The usual 'tap ALT to beat the
|
||||
// foreground lock' trick corrupts exactly what we measure: after ALT, Windows
|
||||
// treats the next character as a menu MNEMONIC, so the first injected
|
||||
// character is eaten. Measured 2026-10-30: ALT re-asserting every 60ms turned
|
||||
// three injected characters into only 'b' in the target.
|
||||
// Attaching input queues is a lock workaround that synthesises no key input.
|
||||
public static bool Claim(IntPtr h) {
|
||||
uint fgThread = GetWindowThreadProcessId(GetForegroundWindow(), IntPtr.Zero);
|
||||
uint myThread = GetCurrentThreadId();
|
||||
AttachThreadInput(myThread, fgThread, true);
|
||||
bool ok = SetForegroundWindow(h);
|
||||
AttachThreadInput(myThread, fgThread, false);
|
||||
return ok && GetForegroundWindow() == h;
|
||||
}
|
||||
public static IntPtr Now() { return GetForegroundWindow(); }
|
||||
}
|
||||
"@
|
||||
}
|
||||
|
||||
$timer = New-Object System.Windows.Forms.Timer
|
||||
$timer.Interval = 150
|
||||
$timer.Add_Tick({
|
||||
if (Test-Path $script:StopFile) {
|
||||
$form.Close()
|
||||
return
|
||||
}
|
||||
if ($script:GrabForeground) {
|
||||
# Re-assert only when actually lost (unconditional claiming every tick
|
||||
# would fight the user for the desktop outside a proof run).
|
||||
if ([OwnForeground]::Now() -ne $form.Handle) {
|
||||
[void]$form.Activate()
|
||||
[void][OwnForeground]::Claim($form.Handle)
|
||||
}
|
||||
}
|
||||
if ($tb.Text -ne $script:lastWritten) {
|
||||
$script:lastWritten = $tb.Text
|
||||
$enc = New-Object System.Text.UTF8Encoding($true)
|
||||
[System.IO.File]::WriteAllText($script:ResultFile, $tb.Text, $enc)
|
||||
}
|
||||
if (-not $tb.Focused) { [void]$tb.Focus() }
|
||||
})
|
||||
|
||||
$form.Add_Shown({
|
||||
$form.Activate()
|
||||
[void]$tb.Focus()
|
||||
# Report BOTH pid and window handle: a proof caller resolving the handle via
|
||||
# Get-Process races with window creation and can read 0. Handle is stable
|
||||
# here because the form is realized by the time Shown fires.
|
||||
[System.IO.File]::WriteAllText($script:PidFile, "$PID`n" + $form.Handle.ToInt64())
|
||||
})
|
||||
|
||||
$timer.Start()
|
||||
[void]$form.ShowDialog()
|
||||
$timer.Stop()
|
||||
@@ -1,2 +0,0 @@
|
||||
/target
|
||||
/.vscode
|
||||
Vendored
-63
@@ -1,63 +0,0 @@
|
||||
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
|
||||
#
|
||||
# When uploading crates to the registry Cargo will automatically
|
||||
# "normalize" Cargo.toml files for maximal compatibility
|
||||
# with all versions of Cargo and also rewrite `path` dependencies
|
||||
# to registry (e.g., crates.io) dependencies.
|
||||
#
|
||||
# If you are reading this file be aware that the original Cargo.toml
|
||||
# will likely look very different (and much more reasonable).
|
||||
# See Cargo.toml.orig for the original contents.
|
||||
|
||||
[package]
|
||||
edition = "2024"
|
||||
rust-version = "1.85.1"
|
||||
name = "nvenc"
|
||||
version = "0.1.0"
|
||||
authors = ["AlsoSylv"]
|
||||
build = false
|
||||
autolib = false
|
||||
autobins = false
|
||||
autoexamples = false
|
||||
autotests = false
|
||||
autobenches = false
|
||||
description = "Wrapper for the nvidia NVENC API"
|
||||
readme = "README.md"
|
||||
keywords = [
|
||||
"nvidia",
|
||||
"graphics",
|
||||
"media",
|
||||
"video",
|
||||
"ffi",
|
||||
]
|
||||
categories = [
|
||||
"multimedia",
|
||||
"external-ffi-bindings",
|
||||
]
|
||||
license = "MIT"
|
||||
repository = "https://github.com/AlsoSylv/nvenc"
|
||||
|
||||
[lib]
|
||||
name = "nvenc"
|
||||
path = "src/lib.rs"
|
||||
# 上游 README 的示例是已过时的 Linux GLX 用法（Session<NeedsConfig> 等类型已不存在），
|
||||
# 直接跑 doctest 会失败。本项目只做 vendor 补丁、不维护文档示例，故关闭。
|
||||
doctest = false
|
||||
|
||||
[[example]]
|
||||
name = "simple_encode"
|
||||
path = "examples/simple_encode.rs"
|
||||
|
||||
[dependencies.bitfields]
|
||||
version = "1.0.2"
|
||||
|
||||
[dependencies.libloading]
|
||||
version = "0.8.9"
|
||||
|
||||
[target."cfg(windows)".dependencies.windows]
|
||||
version = "0.62"
|
||||
features = [
|
||||
"Win32_Graphics_Dxgi_Common",
|
||||
"Win32_Graphics_Direct3D",
|
||||
"Win32_Graphics_Direct3D11",
|
||||
]
|
||||
-24
@@ -1,24 +0,0 @@
|
||||
[package]
|
||||
name = "nvenc"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
authors = ["AlsoSylv"]
|
||||
description = "Wrapper for the nvidia NVENC API"
|
||||
rust-version = "1.85.1"
|
||||
readme = "README.md"
|
||||
repository = "https://github.com/AlsoSylv/nvenc"
|
||||
license = "MIT"
|
||||
keywords = ["nvidia", "graphics", "media", "video", "ffi"]
|
||||
categories = ["multimedia", "external-ffi-bindings"]
|
||||
|
||||
[dependencies]
|
||||
libloading = "0.8.9"
|
||||
bitfields = "1.0.2"
|
||||
|
||||
[target.'cfg(windows)'.dependencies.windows]
|
||||
version = "0.62"
|
||||
features = [
|
||||
"Win32_Graphics_Dxgi_Common",
|
||||
"Win32_Graphics_Direct3D",
|
||||
"Win32_Graphics_Direct3D11",
|
||||
]
|
||||
Vendored
-27
@@ -1,27 +0,0 @@
|
||||
Notice Regarding Standards. I do not provide a license or sublicense to
|
||||
any Intellectual Property Rights relating to any standards, including but not
|
||||
limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4;
|
||||
AVC/H.264; HEVC/H.265; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3
|
||||
(collectively, the "Media Technologies"). For clarity, you will pay any
|
||||
royalties due for such third party technologies, which may include the Media
|
||||
Technologies that are owed as a result of me providing the Software to you.
|
||||
|
||||
Copyright 2025 AlsoSylv
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
Vendored
-34
@@ -1,34 +0,0 @@
|
||||
### If you like my work and want to support what I do, support me on Ko-Fi 💜!
|
||||
[](https://ko-fi.com/cyberite)
|
||||
---
|
||||
|
||||
  
|
||||
|
||||
The following example is for Linux GLX
|
||||
|
||||
```rust
|
||||
use nvenc::{session::Session, encoder::Encoder};
|
||||
|
||||
fn main() {
|
||||
// Setup GLX context
|
||||
let session: Session<NeedsConfig> = Session::open_gl();
|
||||
let (session, config): (Session<NeedsInit>, NVencPresetConfig)
|
||||
= session.get_preset_config(
|
||||
NV_ENC_CODEC_H264_GUID,
|
||||
NV_ENC_PRESET_P3_GUID,
|
||||
NVencTuningInfo::LowLatency
|
||||
);
|
||||
let init_params = nvenc::session::InitParams {
|
||||
encode_guid: NV_ENC_H264_GUID,
|
||||
preset_guid: NV_ENC_PRESET_P3_GUID,
|
||||
resolution: [1920, 1080],
|
||||
aspect_ratio: [16, 9],
|
||||
frame_rate: [30, 1],
|
||||
tuning_info: NVencTuningInfo::LowLatency,
|
||||
buffer_format: NVencBufferFormat::ARGB,
|
||||
encode_config: &mut config.preset_cfg,
|
||||
enable_ptd: true,
|
||||
};
|
||||
let encoder: Encoder = session.init(init_params);
|
||||
}
|
||||
```
|
||||
-203
@@ -1,203 +0,0 @@
|
||||
use std::fs::File;
|
||||
use std::io::Write;
|
||||
use std::time::Instant;
|
||||
|
||||
use nvenc::bitstream::BitStream;
|
||||
use nvenc::session::InitParams;
|
||||
use nvenc::{
|
||||
session::Session,
|
||||
sys::guids::{NV_ENC_CODEC_H264_GUID, NV_ENC_PRESET_P3_GUID},
|
||||
};
|
||||
#[cfg(windows)]
|
||||
use windows::Win32::Graphics::Direct3D11::{ID3D11Device, ID3D11Texture2D};
|
||||
|
||||
#[cfg(windows)]
|
||||
pub fn setup() -> ID3D11Device {
|
||||
let dxgi_factory: windows::Win32::Graphics::Dxgi::IDXGIFactory =
|
||||
unsafe { windows::Win32::Graphics::Dxgi::CreateDXGIFactory() }.unwrap();
|
||||
let dxgi_adapter = unsafe { dxgi_factory.EnumAdapters(0) }.unwrap();
|
||||
|
||||
let desc = unsafe { dxgi_adapter.GetDesc() }.unwrap();
|
||||
println!("{:?}", windows::core::HSTRING::from_wide(&desc.Description));
|
||||
|
||||
let mut device = None;
|
||||
let mut device_context = None;
|
||||
unsafe {
|
||||
use windows::Win32::{
|
||||
Foundation::HMODULE,
|
||||
Graphics::{
|
||||
Direct3D::{D3D_DRIVER_TYPE_UNKNOWN, D3D_FEATURE_LEVEL_11_0},
|
||||
Direct3D11::{D3D11_CREATE_DEVICE_FLAG, D3D11_SDK_VERSION, D3D11CreateDevice},
|
||||
},
|
||||
};
|
||||
|
||||
D3D11CreateDevice(
|
||||
&dxgi_adapter,
|
||||
D3D_DRIVER_TYPE_UNKNOWN,
|
||||
HMODULE(std::ptr::null_mut()),
|
||||
D3D11_CREATE_DEVICE_FLAG(0),
|
||||
Some(&[D3D_FEATURE_LEVEL_11_0]),
|
||||
D3D11_SDK_VERSION,
|
||||
Some(&raw mut device),
|
||||
None,
|
||||
Some(&raw mut device_context),
|
||||
)
|
||||
}
|
||||
.unwrap();
|
||||
let device = device.unwrap();
|
||||
let _device_context = device_context.unwrap();
|
||||
|
||||
device
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
fn image(device: &ID3D11Device) -> ID3D11Texture2D {
|
||||
let data: [u8; 1920 * 4] = std::array::from_fn(|idx| match idx % 4 {
|
||||
0 => 255,
|
||||
1 => 0,
|
||||
2 => 0,
|
||||
3 => 255,
|
||||
_ => unreachable!(),
|
||||
});
|
||||
|
||||
let mut texture = None;
|
||||
let desc = windows::Win32::Graphics::Direct3D11::D3D11_TEXTURE2D_DESC {
|
||||
Width: 1920,
|
||||
Height: 1080,
|
||||
MipLevels: 1,
|
||||
ArraySize: 1,
|
||||
Format: windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_R8G8B8A8_UNORM_SRGB,
|
||||
SampleDesc: windows::Win32::Graphics::Dxgi::Common::DXGI_SAMPLE_DESC {
|
||||
Count: 1,
|
||||
Quality: 0,
|
||||
},
|
||||
Usage: windows::Win32::Graphics::Direct3D11::D3D11_USAGE_DEFAULT,
|
||||
BindFlags: windows::Win32::Graphics::Direct3D11::D3D11_BIND_SHADER_RESOURCE.0 as u32,
|
||||
CPUAccessFlags: windows::Win32::Graphics::Dxgi::Common::DXGI_CPU_ACCESS_NONE,
|
||||
MiscFlags: 0,
|
||||
};
|
||||
let data = windows::Win32::Graphics::Direct3D11::D3D11_SUBRESOURCE_DATA {
|
||||
pSysMem: data.as_ptr() as _,
|
||||
SysMemPitch: 1920 * 4,
|
||||
SysMemSlicePitch: 0,
|
||||
};
|
||||
unsafe {
|
||||
device.CreateTexture2D(
|
||||
&raw const desc,
|
||||
Some(&raw const data),
|
||||
Some(&raw mut texture),
|
||||
)
|
||||
}
|
||||
.unwrap();
|
||||
texture.unwrap()
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let device = setup();
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
compile_error!("Linux is currently unsupported in this example");
|
||||
#[cfg(windows)]
|
||||
let session: Session<nvenc::session::NeedsConfig> = Session::open_dx(&device).unwrap();
|
||||
assert!(
|
||||
session
|
||||
.get_encode_codecs()
|
||||
.unwrap()
|
||||
.contains(&NV_ENC_CODEC_H264_GUID)
|
||||
);
|
||||
assert!(
|
||||
session
|
||||
.get_encode_presets(NV_ENC_CODEC_H264_GUID)
|
||||
.unwrap()
|
||||
.contains(&NV_ENC_PRESET_P3_GUID)
|
||||
);
|
||||
let (session, mut config) = session
|
||||
.get_encode_preset_config_ex(
|
||||
NV_ENC_CODEC_H264_GUID,
|
||||
NV_ENC_PRESET_P3_GUID,
|
||||
nvenc::sys::enums::NVencTuningInfo::LowLatency,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let texture = image(&device);
|
||||
|
||||
config.preset_cfg.rc_params.rate_control_mode = nvenc::sys::enums::NVencParamsRcMode::VBR;
|
||||
config.preset_cfg.rc_params.average_bit_rate = 10_000_000;
|
||||
config.preset_cfg.gop_len = 0xffffffff;
|
||||
config.preset_cfg.frame_interval_p = 1;
|
||||
println!("P-Frames: {}", config.preset_cfg.frame_interval_p);
|
||||
println!("Bitrate: {}", config.preset_cfg.rc_params.average_bit_rate);
|
||||
|
||||
let init_params = InitParams {
|
||||
encode_guid: NV_ENC_CODEC_H264_GUID,
|
||||
preset_guid: NV_ENC_PRESET_P3_GUID,
|
||||
aspect_ratio: [16, 9],
|
||||
encode_config: &mut config.preset_cfg,
|
||||
tuning_info: nvenc::sys::enums::NVencTuningInfo::LowLatency,
|
||||
buffer_format: nvenc::sys::enums::NVencBufferFormat::ARGB,
|
||||
frame_rate: [30, 1],
|
||||
resolution: [1920, 1080],
|
||||
enable_ptd: true,
|
||||
max_encoder_resolution: [0, 0],
|
||||
};
|
||||
|
||||
println!("FPS: 30");
|
||||
println!("Len: 600 frames");
|
||||
println!("Expected time: {}", 600 / 30);
|
||||
println!("Codec: H264");
|
||||
println!("Tuning Info {:?}:", init_params.tuning_info);
|
||||
|
||||
let encoder = session.init_encoder(init_params).unwrap();
|
||||
let registered = encoder
|
||||
.register_resource_dx11(&texture, nvenc::sys::enums::NVencBufferFormat::ARGB, 0)
|
||||
.unwrap();
|
||||
|
||||
let (processed, to_use) = std::sync::mpsc::sync_channel::<BitStream>(2);
|
||||
let (re_use, to_process) = std::sync::mpsc::sync_channel::<BitStream>(2);
|
||||
processed
|
||||
.send(encoder.create_bitstream_buffer().unwrap())
|
||||
.unwrap();
|
||||
processed
|
||||
.send(encoder.create_bitstream_buffer().unwrap())
|
||||
.unwrap();
|
||||
|
||||
let handle = std::thread::spawn({
|
||||
move || {
|
||||
let mut out = File::create("output.h264").unwrap();
|
||||
|
||||
while let Ok(output) = to_process.recv() {
|
||||
let lock = output.try_lock(true).unwrap();
|
||||
out.write_all(lock.as_slice()).unwrap();
|
||||
drop(lock);
|
||||
if let Err(_) = processed.send(output) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
let instant = Instant::now();
|
||||
let mut i = 0;
|
||||
while i < 600 {
|
||||
if let Ok(output) = to_use.recv() {
|
||||
encoder
|
||||
.encode_picture(
|
||||
®istered,
|
||||
&output,
|
||||
i,
|
||||
instant.duration_since(Instant::now()).as_millis() as u64,
|
||||
nvenc::sys::enums::NVencBufferFormat::ARGB,
|
||||
nvenc::sys::enums::NVencPicStruct::Frame,
|
||||
nvenc::sys::enums::NVencPicType::P,
|
||||
None,
|
||||
)
|
||||
.unwrap();
|
||||
re_use.send(output).unwrap();
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
|
||||
drop(to_use);
|
||||
drop(re_use);
|
||||
handle.join().unwrap();
|
||||
}
|
||||
Vendored
-86
@@ -1,86 +0,0 @@
|
||||
#![doc = include_str!("../README.md")]
|
||||
|
||||
use std::{mem::MaybeUninit, sync::OnceLock};
|
||||
|
||||
use libloading::Library;
|
||||
|
||||
use crate::sys::{
|
||||
function_table::NVencFunctionList,
|
||||
result::{NVencError, NVencResult},
|
||||
version::NVENC_API_LIST_VERSION,
|
||||
};
|
||||
|
||||
mod safe;
|
||||
pub use safe::*;
|
||||
/// Contains the original C Types, functions, enums, and constants
|
||||
pub mod sys;
|
||||
|
||||
#[cfg(all(target_arch = "x86_64", target_os = "windows"))]
|
||||
pub const NVENC_DLL: &str = "nvEncodeAPI64.dll";
|
||||
#[cfg(all(target_arch = "x86", target_os = "windows"))]
|
||||
pub const NVENC_DLL: &str = "nvEncodeAPI.dll";
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
/// Platform specific DLL name, `libnvidia-encode.so.1` on Linux, `nvEncodeAPI64.dll` on Windows
|
||||
pub const NVENC_DLL: &str = "libnvidia-encode.so.1";
|
||||
|
||||
#[cfg(all(target_arch = "x86", windows))]
|
||||
#[macro_export]
|
||||
macro_rules! stdcall {
|
||||
(fn $args:tt $(-> $ret:tt)?) => { unsafe extern "stdcall" fn $args $(-> $ret)? };
|
||||
}
|
||||
|
||||
#[doc(hidden)]
|
||||
#[cfg(not(all(target_arch = "x86", windows)))]
|
||||
#[macro_export]
|
||||
macro_rules! stdcall {
|
||||
(fn $args:tt $(-> $ret:ty)?) => { unsafe extern "C" fn $args $(-> $ret)? };
|
||||
}
|
||||
|
||||
pub(crate) static LIBRARY: OnceLock<NVENCLibrary> = OnceLock::new();
|
||||
|
||||
/// Struct containing the library handle, init and get_max_version number functions
|
||||
pub struct NVENCLibrary {
|
||||
#[allow(unused)]
|
||||
// Lib must stay alive for nvenc to be used
|
||||
lib: Library,
|
||||
init: stdcall!(fn(list: *mut MaybeUninit<NVencFunctionList>) -> NVencResult),
|
||||
get_max_version: stdcall!(fn(version: *mut u32) -> NVencResult),
|
||||
}
|
||||
|
||||
/// Attempt to init NVENC, returns an error if the library or loading functions cannot be found
|
||||
pub fn nvenc_init() -> Result<&'static NVENCLibrary, libloading::Error> {
|
||||
if let Some(lib) = LIBRARY.get() {
|
||||
Ok(lib)
|
||||
} else {
|
||||
let lib = unsafe { Library::new(NVENC_DLL) }?;
|
||||
let init: stdcall!(fn(function_list: *mut MaybeUninit<NVencFunctionList>) -> NVencResult) =
|
||||
*unsafe { lib.get(b"NvEncodeAPICreateInstance") }?;
|
||||
let get_max_version: stdcall!(fn(version: *mut u32) -> NVencResult) =
|
||||
*unsafe { lib.get(b"NvEncodeAPIGetMaxSupportedVersion") }?;
|
||||
|
||||
Ok(LIBRARY.get_or_init(|| NVENCLibrary {
|
||||
lib,
|
||||
init,
|
||||
get_max_version,
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
impl NVENCLibrary {
|
||||
/// Create a new instance of the NVencFunctionList,
|
||||
pub fn create_instance(&self) -> Result<NVencFunctionList, NVencError> {
|
||||
let mut list = MaybeUninit::zeroed();
|
||||
unsafe { *(list.as_mut_ptr() as *mut u32) = NVENC_API_LIST_VERSION };
|
||||
unsafe { (self.init)(&raw mut list) }.into_error().unwrap();
|
||||
Ok(unsafe { list.assume_init() })
|
||||
}
|
||||
|
||||
pub fn get_max_version(&self) -> Result<u32, NVencError> {
|
||||
let mut version = 0;
|
||||
unsafe { (self.get_max_version)(&raw mut version) }.into_error()?;
|
||||
let minor = version & 0x0F;
|
||||
let version = (version >> 4) | minor << 28;
|
||||
Ok(version)
|
||||
}
|
||||
}
|
||||
Vendored
-53
@@ -1,53 +0,0 @@
|
||||
use std::{ffi::c_void, sync::Arc};
|
||||
|
||||
use crate::{safe::encoder::EncoderInternal, sys::result::NVencError};
|
||||
|
||||
pub struct BitStream {
|
||||
pub(crate) buffer: *mut c_void,
|
||||
pub(crate) encoder: Arc<EncoderInternal>,
|
||||
}
|
||||
|
||||
impl Drop for BitStream {
|
||||
fn drop(&mut self) {
|
||||
println!("Dropping bitstream buffer");
|
||||
self.encoder.destroy_bitstream_buffer(self.buffer).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
unsafe impl Send for BitStream {}
|
||||
|
||||
impl BitStream {
|
||||
/// Attempts to lock the bit stream, if `wait` is true it will wait
|
||||
/// otherwise a `LockBusy` Error may be returned, in which case the
|
||||
/// client should retry in a few milliseconds
|
||||
pub fn try_lock(&self, wait: bool) -> Result<BitStreamLockGuard<'_>, NVencError> {
|
||||
let lock = self.encoder.lock_bit_stream_buffer(self.buffer, wait)?;
|
||||
Ok(BitStreamLockGuard {
|
||||
buffer: self,
|
||||
data_ptr: lock.bitstream_buffer,
|
||||
data_len: lock.bitstream_size_in_bytes,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Holds a reference to the `BitStream` and holds the data and associated fields
|
||||
pub struct BitStreamLockGuard<'a> {
|
||||
buffer: &'a BitStream,
|
||||
data_ptr: *mut c_void,
|
||||
data_len: u32,
|
||||
}
|
||||
|
||||
impl BitStreamLockGuard<'_> {
|
||||
pub fn as_slice(&self) -> &[u8] {
|
||||
unsafe { std::slice::from_raw_parts(self.data_ptr as _, self.data_len as _) }
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Drop for BitStreamLockGuard<'a> {
|
||||
fn drop(&mut self) {
|
||||
self.buffer
|
||||
.encoder
|
||||
.unlock_bit_stream_buffer(self.buffer.buffer)
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
Vendored
-517
@@ -1,517 +0,0 @@
|
||||
use std::ffi::{CStr, c_void};
|
||||
use std::marker::PhantomData;
|
||||
use std::mem::ManuallyDrop;
|
||||
use std::sync::Arc;
|
||||
use std::{mem::MaybeUninit, ptr::NonNull};
|
||||
|
||||
#[cfg(windows)]
|
||||
use windows::Win32::Graphics::Direct3D11::ID3D11Texture2D;
|
||||
|
||||
use crate::input_buffer::InputBuffer;
|
||||
use crate::safe::bitstream::BitStream;
|
||||
use crate::sys::enums::{
|
||||
NVencBufferFormat, NVencBufferUsage, NVencInputResourceType, NVencMemoryHeap, NVencPicStruct,
|
||||
NVencPicType,
|
||||
};
|
||||
use crate::sys::structs::{
|
||||
Guid, NV_ENC_CREATE_INPUT_BUFFER_VER, NV_ENC_LOCK_INPUT_BUFFER_VER, NVencCodecPicParams,
|
||||
NVencCreateInputBuffer, NVencLockInputBuffer, NVencPicParamsH264, NVencRegisterResource,
|
||||
};
|
||||
use crate::sys::{
|
||||
enums::{NVencPicFlags, NVencTuningInfo},
|
||||
function_table::NVencFunctionList,
|
||||
result::NVencError,
|
||||
structs::{
|
||||
NV_ENC_CONFIG_VER, NV_ENC_CREATE_BITSTREAM_BUFFER_VER, NV_ENC_LOCK_BITSTREAM_VER,
|
||||
NV_ENC_MAP_INPUT_RESOURCE_VER, NV_ENC_PIC_PARAMS_VER, NV_ENC_PRESET_CONFIG_VER,
|
||||
NV_ENC_REGISTER_RESOURCE_VER, NVencCreateBitstreamBuffer, NVencInitializeParams,
|
||||
NVencLockBitStream, NVencPicParams, NVencPresetConfig,
|
||||
},
|
||||
};
|
||||
|
||||
pub struct Encoder {
|
||||
pub(crate) encoder: Arc<EncoderInternal>,
|
||||
pub(crate) marker: PhantomData<*mut ()>,
|
||||
}
|
||||
|
||||
pub trait EncoderInput {
|
||||
#[doc(hidden)]
|
||||
fn as_ptr(&self) -> *mut c_void;
|
||||
|
||||
fn pitch(&self) -> u32;
|
||||
fn height(&self) -> u32;
|
||||
fn width(&self) -> u32;
|
||||
}
|
||||
|
||||
impl EncoderInput for RegisteredResource {
|
||||
fn as_ptr(&self) -> *mut c_void {
|
||||
self.mapped
|
||||
}
|
||||
|
||||
fn pitch(&self) -> u32 {
|
||||
self.pitch
|
||||
}
|
||||
|
||||
fn height(&self) -> u32 {
|
||||
self.height
|
||||
}
|
||||
|
||||
fn width(&self) -> u32 {
|
||||
self.width
|
||||
}
|
||||
}
|
||||
|
||||
impl EncoderInput for InputBuffer {
|
||||
fn as_ptr(&self) -> *mut c_void {
|
||||
self.buffer
|
||||
}
|
||||
|
||||
fn pitch(&self) -> u32 {
|
||||
self.width
|
||||
}
|
||||
|
||||
fn height(&self) -> u32 {
|
||||
self.height
|
||||
}
|
||||
|
||||
fn width(&self) -> u32 {
|
||||
self.width
|
||||
}
|
||||
}
|
||||
|
||||
impl Encoder {
|
||||
#[cfg(windows)]
|
||||
/// Registers a DX11 texture as an nvenc input resource
|
||||
pub fn register_resource_dx11(
|
||||
&self,
|
||||
resource: &ID3D11Texture2D,
|
||||
format: NVencBufferFormat,
|
||||
pitch: u32,
|
||||
) -> Result<RegisteredResource, NVencError> {
|
||||
self.encoder.register_resource_dx(resource, format, pitch)
|
||||
}
|
||||
|
||||
/// # Safety
|
||||
/// `resource` must not be null
|
||||
/// `ty` and `format` should match the original buffer
|
||||
pub unsafe fn register_resource_raw(
|
||||
&self,
|
||||
resource: *mut c_void,
|
||||
format: NVencBufferFormat,
|
||||
usage: NVencBufferUsage,
|
||||
ty: NVencInputResourceType,
|
||||
resolution: [u32; 2],
|
||||
pitch: u32,
|
||||
) -> Result<*mut c_void, NVencError> {
|
||||
unsafe {
|
||||
self.encoder
|
||||
.register_resource_raw(resource, format, usage, ty, resolution, pitch)
|
||||
}
|
||||
}
|
||||
|
||||
/// Takes an encoder inout, and outputs a bit stream that can be read back
|
||||
pub fn encode_picture<I: EncoderInput>(
|
||||
&self,
|
||||
input: &I,
|
||||
output: &BitStream,
|
||||
frame_count: usize,
|
||||
timestamp: u64,
|
||||
format: NVencBufferFormat,
|
||||
pic_struct: NVencPicStruct,
|
||||
ty: NVencPicType,
|
||||
codec_params: Option<CodecPicParams>,
|
||||
) -> Result<(), NVencError> {
|
||||
self.encoder.encode_picture(
|
||||
input,
|
||||
output,
|
||||
frame_count,
|
||||
timestamp,
|
||||
format,
|
||||
pic_struct,
|
||||
ty,
|
||||
codec_params,
|
||||
)
|
||||
}
|
||||
|
||||
/// Ends the encoder session
|
||||
pub fn end_encode(&self) -> Result<(), NVencError> {
|
||||
self.encoder.end_encode()
|
||||
}
|
||||
|
||||
/// Creates a bitstream bfufer that can be used for output storage
|
||||
pub fn create_bitstream_buffer(&self) -> Result<BitStream, NVencError> {
|
||||
self.encoder.create_bitstream_buffer()
|
||||
}
|
||||
|
||||
pub fn create_input_buffer(
|
||||
&self,
|
||||
width: u32,
|
||||
height: u32,
|
||||
memory_heap: NVencMemoryHeap,
|
||||
format: NVencBufferFormat,
|
||||
) -> Result<InputBuffer, NVencError> {
|
||||
Ok(InputBuffer {
|
||||
encoder: self.encoder.clone(),
|
||||
buffer: self
|
||||
.encoder
|
||||
.create_input_buffer(width, height, memory_heap, format)?,
|
||||
width,
|
||||
height,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct EncoderInternal {
|
||||
pub(super) encoder: NonNull<c_void>,
|
||||
pub(super) function_list: NVencFunctionList,
|
||||
}
|
||||
|
||||
/// The Nvidia encoder is technically thread-safe
|
||||
unsafe impl Send for EncoderInternal {}
|
||||
|
||||
/// The Nvidia encoder handles syncing via `lock_bit_stream` which is all that is allowed to be Send
|
||||
unsafe impl Sync for EncoderInternal {}
|
||||
|
||||
impl Drop for EncoderInternal {
|
||||
fn drop(&mut self) {
|
||||
println!("Dropping encoder, this should happen last");
|
||||
unsafe { (self.function_list.nvenc_destroy_encoder)(self.encoder.as_ptr()) };
|
||||
}
|
||||
}
|
||||
|
||||
impl EncoderInternal {
|
||||
pub(crate) fn get_encode_preset_config_ex(
|
||||
&self,
|
||||
codec: Guid,
|
||||
preset: Guid,
|
||||
tuning_info: NVencTuningInfo,
|
||||
) -> Result<NVencPresetConfig, NVencError> {
|
||||
let mut config: std::mem::MaybeUninit<NVencPresetConfig> = std::mem::MaybeUninit::zeroed();
|
||||
unsafe { &mut *config.as_mut_ptr() }.version = NV_ENC_PRESET_CONFIG_VER;
|
||||
unsafe { &mut *config.as_mut_ptr() }.preset_cfg.version = NV_ENC_CONFIG_VER;
|
||||
unsafe {
|
||||
(self.function_list.nvenc_get_encode_preset_config_ex)(
|
||||
self.encoder.as_ptr(),
|
||||
codec,
|
||||
preset,
|
||||
tuning_info,
|
||||
&raw mut config,
|
||||
)
|
||||
.into_error()?
|
||||
};
|
||||
|
||||
Ok(unsafe { config.assume_init() })
|
||||
}
|
||||
|
||||
pub(crate) fn init_encoder(
|
||||
&self,
|
||||
mut init_params: NVencInitializeParams,
|
||||
) -> Result<(), NVencError> {
|
||||
unsafe {
|
||||
(self.function_list.nvenc_initialize_encoder)(
|
||||
self.encoder.as_ptr(),
|
||||
&raw mut init_params,
|
||||
)
|
||||
}
|
||||
.into_error()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
pub fn register_resource_dx(
|
||||
self: &Arc<EncoderInternal>,
|
||||
resource: &ID3D11Texture2D,
|
||||
format: NVencBufferFormat,
|
||||
pitch: u32,
|
||||
) -> Result<RegisteredResource, NVencError> {
|
||||
use windows::Win32::Graphics::Direct3D11::D3D11_TEXTURE2D_DESC;
|
||||
use windows::core::Interface;
|
||||
|
||||
let mut desc = D3D11_TEXTURE2D_DESC::default();
|
||||
unsafe { resource.GetDesc(&raw mut desc) };
|
||||
|
||||
let registered = unsafe {
|
||||
self.register_resource_raw(
|
||||
resource.as_raw(),
|
||||
NVencBufferFormat::Undefined,
|
||||
NVencBufferUsage::Image,
|
||||
NVencInputResourceType::DirectX,
|
||||
[desc.Width, desc.Height],
|
||||
pitch,
|
||||
)?
|
||||
};
|
||||
|
||||
let mapped = unsafe { self.map_input_resource_raw(registered, format)? };
|
||||
|
||||
Ok(RegisteredResource {
|
||||
encoder: self.clone(),
|
||||
pitch,
|
||||
width: desc.Width,
|
||||
height: desc.Height,
|
||||
registered,
|
||||
mapped,
|
||||
})
|
||||
}
|
||||
|
||||
pub unsafe fn map_input_resource_raw(
|
||||
&self,
|
||||
resource: *mut c_void,
|
||||
format: NVencBufferFormat,
|
||||
) -> Result<*mut c_void, NVencError> {
|
||||
let mut map: crate::sys::structs::NVencMapInputResource = unsafe { std::mem::zeroed() };
|
||||
map.version = NV_ENC_MAP_INPUT_RESOURCE_VER;
|
||||
map.registered_resource = resource;
|
||||
map.mapped_buffer_format = format;
|
||||
|
||||
unsafe {
|
||||
(self.function_list.nvenc_map_input_resource)(self.encoder.as_ptr(), &raw mut map)
|
||||
}
|
||||
.into_error()?;
|
||||
|
||||
Ok(map.mapped_resource)
|
||||
}
|
||||
|
||||
pub unsafe fn register_resource_raw(
|
||||
&self,
|
||||
resource: *mut c_void,
|
||||
format: NVencBufferFormat,
|
||||
usage: NVencBufferUsage,
|
||||
ty: NVencInputResourceType,
|
||||
resolution: [u32; 2],
|
||||
pitch: u32,
|
||||
) -> Result<*mut c_void, NVencError> {
|
||||
let mut register_resource: NVencRegisterResource = unsafe { std::mem::zeroed() };
|
||||
register_resource.buffer_format = format;
|
||||
register_resource.buffer_usage = usage;
|
||||
register_resource.resource_type = ty;
|
||||
register_resource.version = NV_ENC_REGISTER_RESOURCE_VER;
|
||||
register_resource.width = resolution[0];
|
||||
register_resource.height = resolution[1];
|
||||
register_resource.pitch = pitch;
|
||||
register_resource.sub_resource_index = 0;
|
||||
register_resource.resource_to_register = resource;
|
||||
unsafe {
|
||||
(self.function_list.nvenc_register_resource)(
|
||||
self.encoder.as_ptr(),
|
||||
&raw mut register_resource,
|
||||
)
|
||||
.into_error()
|
||||
}?;
|
||||
Ok(register_resource.registered_resource)
|
||||
}
|
||||
|
||||
pub unsafe fn unmap_input_resource(
|
||||
&self,
|
||||
mapped_resource: *mut c_void,
|
||||
) -> Result<(), NVencError> {
|
||||
unsafe {
|
||||
(self.function_list.nvenc_unmap_input_resource)(self.encoder.as_ptr(), mapped_resource)
|
||||
}
|
||||
.into_error()
|
||||
}
|
||||
|
||||
pub unsafe fn unregister_resource(
|
||||
&self,
|
||||
registered_resource: *mut c_void,
|
||||
) -> Result<(), NVencError> {
|
||||
unsafe {
|
||||
(self.function_list.nvenc_unregister_resource)(
|
||||
self.encoder.as_ptr(),
|
||||
registered_resource,
|
||||
)
|
||||
}
|
||||
.into_error()
|
||||
}
|
||||
|
||||
pub fn create_input_buffer(
|
||||
&self,
|
||||
width: u32,
|
||||
height: u32,
|
||||
memory_heap: NVencMemoryHeap,
|
||||
format: NVencBufferFormat,
|
||||
) -> Result<*mut c_void, NVencError> {
|
||||
let mut input = NVencCreateInputBuffer {
|
||||
version: NV_ENC_CREATE_INPUT_BUFFER_VER,
|
||||
width: width,
|
||||
height: height,
|
||||
memory_heap,
|
||||
buffer_fmt: format,
|
||||
rsvd: 0,
|
||||
input_buffer: std::ptr::null_mut(),
|
||||
p_sys_mem_buffer: std::ptr::null_mut(),
|
||||
rsvd1: unsafe { std::mem::zeroed() },
|
||||
rsvd2: unsafe { std::mem::zeroed() },
|
||||
};
|
||||
unsafe {
|
||||
(self.function_list.nvenc_create_input_buffer)(self.encoder.as_ptr(), &raw mut input)
|
||||
.into_error()?
|
||||
};
|
||||
|
||||
Ok(input.input_buffer)
|
||||
}
|
||||
|
||||
pub fn destroy_input_buffer(&self, buffer: *mut c_void) -> Result<(), NVencError> {
|
||||
unsafe { (self.function_list.nvenc_destroy_input_buffer)(self.encoder.as_ptr(), buffer) }
|
||||
.into_error()
|
||||
}
|
||||
|
||||
pub fn lock_input_buffer(&self, buffer: *mut c_void) -> Result<(*mut c_void, u32), NVencError> {
|
||||
let mut lock = NVencLockInputBuffer {
|
||||
version: NV_ENC_LOCK_INPUT_BUFFER_VER,
|
||||
bitflags: 0,
|
||||
input_buffer: buffer,
|
||||
buffer_data_ptr: std::ptr::null_mut(),
|
||||
pitch: 0,
|
||||
rsvd1: unsafe { std::mem::zeroed() },
|
||||
rsvd2: unsafe { std::mem::zeroed() },
|
||||
};
|
||||
|
||||
unsafe {
|
||||
(self.function_list.nvenc_lock_input_buffer)(self.encoder.as_ptr(), &raw mut lock)
|
||||
.into_error()?;
|
||||
}
|
||||
|
||||
Ok((lock.buffer_data_ptr, lock.pitch))
|
||||
}
|
||||
|
||||
pub fn unlock_input_buffer(&self, locked_buffer: *mut c_void) -> Result<(), NVencError> {
|
||||
unsafe {
|
||||
(self.function_list.nvenc_unlock_input_buffer)(self.encoder.as_ptr(), locked_buffer)
|
||||
.into_error()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn encode_picture<I: EncoderInput>(
|
||||
&self,
|
||||
input: &I,
|
||||
output: &BitStream,
|
||||
frame_count: usize,
|
||||
timestamp: u64,
|
||||
format: NVencBufferFormat,
|
||||
pic_struct: NVencPicStruct,
|
||||
ty: NVencPicType,
|
||||
codec_params: Option<CodecPicParams>,
|
||||
) -> Result<(), NVencError> {
|
||||
let mut params: MaybeUninit<NVencPicParams> = MaybeUninit::zeroed();
|
||||
let sys_codec = match codec_params {
|
||||
None => NVencCodecPicParams { rsvd: [0; 256] },
|
||||
Some(CodecPicParams::H264(codec)) => NVencCodecPicParams {
|
||||
h264_pic_params: ManuallyDrop::new(codec),
|
||||
},
|
||||
};
|
||||
unsafe { &mut *params.as_mut_ptr() }.version = NV_ENC_PIC_PARAMS_VER;
|
||||
unsafe { &mut *params.as_mut_ptr() }.input_width = input.width();
|
||||
unsafe { &mut *params.as_mut_ptr() }.input_height = input.height();
|
||||
unsafe { &mut *params.as_mut_ptr() }.input_pitch = input.pitch();
|
||||
unsafe { &mut *params.as_mut_ptr() }.output_bitstream = output.buffer;
|
||||
unsafe { &mut *params.as_mut_ptr() }.input_buffer = input.as_ptr();
|
||||
unsafe { &mut *params.as_mut_ptr() }.buffer_format = format;
|
||||
unsafe { &mut *params.as_mut_ptr() }.picture_struct = pic_struct;
|
||||
// [本项目补丁] 只设 picture_type=IDR 不足以让 NVENC 真正输出 IDR 切片，
|
||||
// 必须同时置 encode_pic_flags |= NV_ENC_PIC_FLAG_FORCEIDR。
|
||||
//
|
||||
// 上游 0.1.0 未设置该标志，导致运行期请求关键帧时编码器静默忽略、
|
||||
// 继续输出 P 帧（实测：picture_type=IDR 但输出 NAL type=1）。
|
||||
// 首帧之所以"看起来正常"，是因为 frame_idx==0 本来就强制 IDR，与本字段无关。
|
||||
//
|
||||
// 语义区分：IntraRefresh 走 ForceIntra，其余 IDR 类走 ForceIDR。
|
||||
// 必须在 `picture_type = ty` 之前计算，否则 ty 已被移动。
|
||||
let pic_flags = match &ty {
|
||||
NVencPicType::IDR => NVencPicFlags::ForceIDR as u32,
|
||||
NVencPicType::IntraRefresh => NVencPicFlags::ForceIntra as u32,
|
||||
_ => 0,
|
||||
};
|
||||
unsafe { &mut *params.as_mut_ptr() }.picture_type = ty;
|
||||
unsafe { &mut *params.as_mut_ptr() }.encode_pic_flags = pic_flags;
|
||||
unsafe { &mut *params.as_mut_ptr() }.frame_idx = frame_count as u32;
|
||||
unsafe { &mut *params.as_mut_ptr() }.input_time_stamp = timestamp;
|
||||
unsafe { &mut *params.as_mut_ptr() }.codec_pic_params = sys_codec;
|
||||
match unsafe {
|
||||
(self.function_list.nvenc_encode_picture)(self.encoder.as_ptr(), &raw mut params)
|
||||
.into_error()
|
||||
} {
|
||||
Err(err) => {
|
||||
let ptr =
|
||||
unsafe { (self.function_list.nvenc_get_last_error)(self.encoder.as_ptr()) };
|
||||
println!("{:?}", unsafe { CStr::from_ptr(ptr) });
|
||||
Err(err)
|
||||
}
|
||||
Ok(()) => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn end_encode(&self) -> Result<(), NVencError> {
|
||||
let mut params: MaybeUninit<NVencPicParams> = MaybeUninit::zeroed();
|
||||
unsafe { &mut *params.as_mut_ptr() }.version = NV_ENC_PIC_PARAMS_VER;
|
||||
unsafe { &mut *params.as_mut_ptr() }.encode_pic_flags = NVencPicFlags::Eos as u32;
|
||||
unsafe {
|
||||
(self.function_list.nvenc_encode_picture)(self.encoder.as_ptr(), &raw mut params)
|
||||
.into_error()
|
||||
}?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn lock_bit_stream_buffer(
|
||||
&self,
|
||||
buffer: *mut c_void,
|
||||
wait: bool,
|
||||
) -> Result<NVencLockBitStream, NVencError> {
|
||||
let mut lock: MaybeUninit<NVencLockBitStream> = MaybeUninit::zeroed();
|
||||
unsafe { &mut *lock.as_mut_ptr() }.version = NV_ENC_LOCK_BITSTREAM_VER;
|
||||
unsafe { &mut *lock.as_mut_ptr() }.output_bit_stream = buffer;
|
||||
unsafe { &mut *lock.as_mut_ptr() }
|
||||
.bit_fields
|
||||
.set_do_not_wait(!wait);
|
||||
unsafe { (self.function_list.nvenc_lock_bit_stream)(self.encoder.as_ptr(), &raw mut lock) }
|
||||
.into_error()?;
|
||||
Ok(unsafe { lock.assume_init() })
|
||||
}
|
||||
|
||||
pub(crate) fn unlock_bit_stream_buffer(&self, buffer: *mut c_void) -> Result<(), NVencError> {
|
||||
unsafe { (self.function_list.nvenc_unlock_bit_stream)(self.encoder.as_ptr(), buffer) }
|
||||
.into_error()
|
||||
}
|
||||
|
||||
pub fn create_bitstream_buffer(self: &Arc<Self>) -> Result<BitStream, NVencError> {
|
||||
let mut params: NVencCreateBitstreamBuffer = unsafe { std::mem::zeroed() };
|
||||
params.version = NV_ENC_CREATE_BITSTREAM_BUFFER_VER;
|
||||
unsafe {
|
||||
(self.function_list.nvenc_create_bit_stream_buffer)(
|
||||
self.encoder.as_ptr(),
|
||||
&raw mut params,
|
||||
)
|
||||
}
|
||||
.into_error()?;
|
||||
Ok(BitStream {
|
||||
buffer: params.bitstream_buffer,
|
||||
encoder: self.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn destroy_bitstream_buffer(&self, buffer: *mut c_void) -> Result<(), NVencError> {
|
||||
unsafe {
|
||||
(self.function_list.nvenc_destory_bit_stream_buffer)(self.encoder.as_ptr(), buffer)
|
||||
}
|
||||
.into_error()
|
||||
}
|
||||
}
|
||||
|
||||
pub enum CodecPicParams {
|
||||
H264(NVencPicParamsH264),
|
||||
}
|
||||
|
||||
pub struct RegisteredResource {
|
||||
registered: *mut c_void,
|
||||
mapped: *mut c_void,
|
||||
pitch: u32,
|
||||
width: u32,
|
||||
height: u32,
|
||||
encoder: Arc<EncoderInternal>,
|
||||
}
|
||||
|
||||
impl Drop for RegisteredResource {
|
||||
fn drop(&mut self) {
|
||||
unsafe { self.encoder.unmap_input_resource(self.mapped).unwrap() };
|
||||
unsafe { self.encoder.unregister_resource(self.registered).unwrap() };
|
||||
}
|
||||
}
|
||||
-60
@@ -1,60 +0,0 @@
|
||||
use std::{ffi::c_void, sync::Arc};
|
||||
|
||||
use crate::{encoder::EncoderInternal, sys::result::NVencError};
|
||||
|
||||
pub struct InputBuffer {
|
||||
pub(crate) encoder: Arc<EncoderInternal>,
|
||||
pub(crate) buffer: *mut c_void,
|
||||
pub(crate) width: u32,
|
||||
pub(crate) height: u32,
|
||||
}
|
||||
|
||||
impl InputBuffer {
|
||||
pub fn lock(&self) -> Result<InputBufferLock<'_>, NVencError> {
|
||||
let (lock_ptr, pitch) = self.encoder.lock_input_buffer(self.buffer)?;
|
||||
Ok(InputBufferLock {
|
||||
input_buffer: self,
|
||||
lock_ptr,
|
||||
pitch,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for InputBuffer {
|
||||
fn drop(&mut self) {
|
||||
self.encoder.destroy_input_buffer(self.buffer).unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
pub struct InputBufferLock<'a> {
|
||||
input_buffer: &'a InputBuffer,
|
||||
lock_ptr: *mut c_void,
|
||||
pitch: u32,
|
||||
}
|
||||
|
||||
impl InputBufferLock<'_> {
|
||||
pub unsafe fn data_ptr(&self) -> *mut u8 {
|
||||
self.lock_ptr as _
|
||||
}
|
||||
|
||||
pub fn width(&self) -> u32 {
|
||||
self.input_buffer.width
|
||||
}
|
||||
|
||||
pub fn height(&self) -> u32 {
|
||||
self.input_buffer.height
|
||||
}
|
||||
|
||||
pub fn pitch(&self) -> u32 {
|
||||
self.pitch
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for InputBufferLock<'_> {
|
||||
fn drop(&mut self) {
|
||||
self.input_buffer
|
||||
.encoder
|
||||
.unlock_input_buffer(self.lock_ptr)
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
Vendored
-8
@@ -1,8 +0,0 @@
|
||||
/// Contains the type for the safe BitStream buffer wrapper
|
||||
pub mod bitstream;
|
||||
/// Wrapper for the NVEncoder after creating an active session
|
||||
pub mod encoder;
|
||||
/// Wrapper around an NVenc input buffer
|
||||
pub mod input_buffer;
|
||||
/// Type-state wrapper around session initialization
|
||||
pub mod session;
|
||||
Vendored
-198
@@ -1,198 +0,0 @@
|
||||
use std::{marker::PhantomData, sync::Arc};
|
||||
|
||||
use crate::{
|
||||
encoder::{Encoder, EncoderInternal},
|
||||
sys::{
|
||||
enums::{NVencBufferFormat, NVencDeviceType, NVencTuningInfo},
|
||||
result::NVencError,
|
||||
structs::{
|
||||
Guid, NV_ENC_INITIALIZE_PARAMS_VER, NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
|
||||
NVencConfig, NVencInitializeParams, NVencOpenEncodeSessionExParams, NVencPresetConfig,
|
||||
},
|
||||
version::NVENC_API_VERSION,
|
||||
},
|
||||
};
|
||||
|
||||
pub struct Session<T> {
|
||||
encoder: Arc<EncoderInternal>,
|
||||
p: PhantomData<T>,
|
||||
}
|
||||
|
||||
pub struct NeedsConfig;
|
||||
|
||||
pub struct NeedsInit;
|
||||
|
||||
#[cfg(windows)]
|
||||
use windows::core::Interface;
|
||||
|
||||
impl Session<NeedsConfig> {
|
||||
#[cfg(windows)]
|
||||
pub fn open_dx(device: &impl Interface) -> Result<Self, NVencError> {
|
||||
let lib = crate::nvenc_init().unwrap();
|
||||
let function_list = lib.create_instance()?;
|
||||
let mut session_params: NVencOpenEncodeSessionExParams = unsafe { std::mem::zeroed() };
|
||||
session_params.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
|
||||
session_params.device_type = NVencDeviceType::DirectX;
|
||||
session_params.api_version = NVENC_API_VERSION;
|
||||
session_params.device = device.as_raw();
|
||||
let mut encoder = std::ptr::null_mut();
|
||||
unsafe {
|
||||
(function_list.nvenc_open_encode_session_ex)(&raw mut session_params, &raw mut encoder)
|
||||
.into_error()
|
||||
}?;
|
||||
assert!(!encoder.is_null());
|
||||
let encoder = EncoderInternal {
|
||||
encoder: unsafe { std::ptr::NonNull::new_unchecked(encoder) },
|
||||
function_list,
|
||||
};
|
||||
Ok(Self {
|
||||
encoder: Arc::new(encoder),
|
||||
p: PhantomData,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn open_gl() -> Result<Self, NVencError> {
|
||||
let lib = crate::nvenc_init().unwrap();
|
||||
let function_list = lib.create_instance()?;
|
||||
let mut session_params: NVencOpenEncodeSessionExParams = unsafe { std::mem::zeroed() };
|
||||
session_params.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
|
||||
session_params.device_type = NVencDeviceType::OpenGL;
|
||||
session_params.api_version = NVENC_API_VERSION;
|
||||
session_params.device = std::ptr::null_mut();
|
||||
|
||||
let mut encoder = std::ptr::null_mut();
|
||||
unsafe {
|
||||
(function_list.nvenc_open_encode_session_ex)(&raw mut session_params, &raw mut encoder)
|
||||
.into_error()
|
||||
}?;
|
||||
assert!(!encoder.is_null());
|
||||
let encoder = EncoderInternal {
|
||||
encoder: unsafe { std::ptr::NonNull::new_unchecked(encoder) },
|
||||
function_list,
|
||||
};
|
||||
Ok(Self {
|
||||
encoder: Arc::new(encoder),
|
||||
p: PhantomData,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn get_encode_codecs(&self) -> Result<Box<[Guid]>, NVencError> {
|
||||
let mut count = 0;
|
||||
unsafe {
|
||||
(self.encoder.function_list.nvenc_get_encoder_guid_count)(
|
||||
self.encoder.encoder.as_ptr(),
|
||||
&raw mut count,
|
||||
)
|
||||
.into_error()
|
||||
}?;
|
||||
let mut array = vec![Guid::default(); count as usize];
|
||||
unsafe {
|
||||
(self.encoder.function_list.nvenc_get_encoder_guids)(
|
||||
self.encoder.encoder.as_ptr(),
|
||||
array.as_mut_ptr(),
|
||||
count,
|
||||
&raw mut count,
|
||||
)
|
||||
}
|
||||
.into_error()?;
|
||||
array.truncate(count as usize);
|
||||
Ok(array.into())
|
||||
}
|
||||
|
||||
pub fn get_encode_presets(&self, codec: Guid) -> Result<Box<[Guid]>, NVencError> {
|
||||
let mut count = 0;
|
||||
unsafe {
|
||||
(self.encoder.function_list.nvenc_get_encode_preset_count)(
|
||||
self.encoder.encoder.as_ptr(),
|
||||
codec.clone(),
|
||||
&raw mut count,
|
||||
)
|
||||
.into_error()
|
||||
}?;
|
||||
let mut array = vec![Guid::default(); count as usize];
|
||||
unsafe {
|
||||
(self.encoder.function_list.nvenc_get_encode_preset_guids)(
|
||||
self.encoder.encoder.as_ptr(),
|
||||
codec,
|
||||
array.as_mut_ptr(),
|
||||
count,
|
||||
&raw mut count,
|
||||
)
|
||||
}
|
||||
.into_error()?;
|
||||
array.truncate(count as usize);
|
||||
Ok(array.into())
|
||||
}
|
||||
|
||||
pub fn get_encode_preset_config_ex(
|
||||
self,
|
||||
codec: Guid,
|
||||
preset: Guid,
|
||||
tuning_info: NVencTuningInfo,
|
||||
) -> Result<(Session<NeedsInit>, NVencPresetConfig), NVencError> {
|
||||
self.encoder
|
||||
.get_encode_preset_config_ex(codec, preset, tuning_info)
|
||||
.map(|config| {
|
||||
(
|
||||
Session {
|
||||
encoder: self.encoder,
|
||||
p: PhantomData,
|
||||
},
|
||||
config,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
pub struct InitParams<'a> {
|
||||
pub encode_guid: Guid,
|
||||
pub preset_guid: Guid,
|
||||
pub resolution: [u32; 2],
|
||||
pub aspect_ratio: [u32; 2],
|
||||
pub frame_rate: [u32; 2],
|
||||
pub tuning_info: NVencTuningInfo,
|
||||
pub buffer_format: NVencBufferFormat,
|
||||
pub encode_config: &'a mut NVencConfig,
|
||||
pub enable_ptd: bool,
|
||||
pub max_encoder_resolution: [u32; 2],
|
||||
// TODO: Support for async encoding and bit fields
|
||||
}
|
||||
|
||||
impl Session<NeedsInit> {
|
||||
pub fn init_encoder(self, init_params: InitParams) -> Result<Encoder, NVencError> {
|
||||
// This sucks, lol
|
||||
let raw_init_params = NVencInitializeParams {
|
||||
encode_guid: init_params.encode_guid,
|
||||
preset_guid: init_params.preset_guid,
|
||||
encode_width: init_params.resolution[0],
|
||||
encode_height: init_params.resolution[1],
|
||||
dar_width: init_params.aspect_ratio[0],
|
||||
dar_height: init_params.aspect_ratio[1],
|
||||
frame_rate_num: init_params.frame_rate[0],
|
||||
frame_rate_den: init_params.frame_rate[1],
|
||||
tuning_info: init_params.tuning_info,
|
||||
buffer_format: init_params.buffer_format,
|
||||
encode_config: init_params.encode_config as _,
|
||||
enable_ptd: init_params.enable_ptd as u32,
|
||||
version: NV_ENC_INITIALIZE_PARAMS_VER,
|
||||
enable_encode_async: 0,
|
||||
bit_flags: crate::sys::structs::NVencInitializeParamsBitfields::new(),
|
||||
priv_data_size: 0,
|
||||
rsvd: 0,
|
||||
priv_data: std::ptr::null_mut(),
|
||||
max_encode_width: init_params.max_encoder_resolution[0],
|
||||
max_encode_height: init_params.max_encoder_resolution[1],
|
||||
max_me_hint_counts_per_block: [Default::default(), Default::default()],
|
||||
num_state_buffers: 0,
|
||||
output_stats_level: crate::sys::enums::NVencOutputStatsLevel::None,
|
||||
rsvd1: unsafe { std::mem::zeroed() },
|
||||
rsvd2: unsafe { std::mem::zeroed() },
|
||||
};
|
||||
|
||||
self.encoder.init_encoder(raw_init_params)?;
|
||||
Ok(Encoder {
|
||||
encoder: self.encoder,
|
||||
marker: PhantomData,
|
||||
})
|
||||
}
|
||||
}
|
||||
Vendored
-481
@@ -1,481 +0,0 @@
|
||||
#[repr(C)]
|
||||
pub enum NVencParamsFrameFieldMode {
|
||||
Frame = 0x01,
|
||||
Field = 0x02,
|
||||
MBAFF = 0x03,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencParamsRcMode {
|
||||
ConstQP = 0x00,
|
||||
VBR = 0x01,
|
||||
CBR = 0x02,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencMultiPass {
|
||||
Disabled = 0x00,
|
||||
TwoPassQuarterResolution = 0x01,
|
||||
TwoPassFullResolution = 0x02,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencStateRestoreType {
|
||||
Full = 0x01,
|
||||
RateControl = 0x02,
|
||||
Encode = 0x03,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencOutputStatsLevel {
|
||||
None = 0,
|
||||
BlockLevel = 1,
|
||||
RowLevel = 2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencEmphasisMapLevel {
|
||||
Level0 = 0x0,
|
||||
Level1 = 0x1,
|
||||
Level2 = 0x2,
|
||||
Level3 = 0x3,
|
||||
Level4 = 0x4,
|
||||
Level5 = 0x5,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencQPMapMode {
|
||||
Disabled = 0x0,
|
||||
Emphasis = 0x1,
|
||||
Delta = 0x2,
|
||||
/// Currently unsupported as of NVENC 13.0
|
||||
Map = 0x3,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencPicStruct {
|
||||
Frame = 0x01,
|
||||
FieldTopBottom = 0x02,
|
||||
FieldBottomTop = 0x03,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencDisplayPicStruct {
|
||||
Frame = 0x0,
|
||||
FieldTopBottom = 0x1,
|
||||
FieldBottomTop = 0x2,
|
||||
Doubling = 0x3,
|
||||
Tripling = 0x4,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencPicType {
|
||||
P = 0x00,
|
||||
B = 0x01,
|
||||
IDR = 0x03,
|
||||
BI = 0x04,
|
||||
Skipped = 0x05,
|
||||
IntraRefresh = 0x06,
|
||||
NonRefP = 0x07,
|
||||
Switch = 0x08,
|
||||
UNKNOWN = 0xFF,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencMVPrecision {
|
||||
Default = 0x00,
|
||||
FullPel = 0x01,
|
||||
HalfPel = 0x02,
|
||||
QuarterPel = 0x03,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencBufferFormat {
|
||||
Undefined = 0x00000000,
|
||||
NV12 = 0x00000001,
|
||||
YV12 = 0x00000010,
|
||||
IYUV = 0x00000100,
|
||||
YUV444 = 0x00001000,
|
||||
YUV420_10Bit = 0x00010000,
|
||||
YUV444_10Bit = 0x00100000,
|
||||
ARGB = 0x01000000,
|
||||
ARGB10 = 0x02000000,
|
||||
AYUV = 0x04000000,
|
||||
ABGR = 0x10000000,
|
||||
ABGR10 = 0x20000000,
|
||||
U8 = 0x40000000,
|
||||
NV16 = 0x40000001,
|
||||
P210 = 0x40000002,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NvencLevel {
|
||||
AutoSelect = 0,
|
||||
}
|
||||
|
||||
// TODO: Translate level macro
|
||||
#[repr(C)]
|
||||
pub enum H264Level {
|
||||
H264_1 = 10,
|
||||
H264_1b = 9,
|
||||
H264_11 = 11,
|
||||
H264_12 = 12,
|
||||
H264_13 = 13,
|
||||
H264_2 = 20,
|
||||
H264_21 = 21,
|
||||
H264_22 = 22,
|
||||
H264_3 = 30,
|
||||
H264_31 = 31,
|
||||
H264_32 = 32,
|
||||
H264_4 = 40,
|
||||
H264_41 = 41,
|
||||
H264_42 = 42,
|
||||
H264_5 = 50,
|
||||
H264_51 = 51,
|
||||
H264_52 = 52,
|
||||
H264_60 = 60,
|
||||
H264_61 = 61,
|
||||
H264_62 = 62,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum HEVCLevel {
|
||||
HEVC1 = 30,
|
||||
HEVC2 = 60,
|
||||
HEVC21 = 63,
|
||||
HEVC3 = 90,
|
||||
HEVC31 = 93,
|
||||
HEVC4 = 120,
|
||||
HEVC41 = 123,
|
||||
HEVC5 = 150,
|
||||
HEVC51 = 153,
|
||||
HEVC52 = 156,
|
||||
HEVC6 = 180,
|
||||
HEVC61 = 183,
|
||||
HEVC62 = 186,
|
||||
HEVCMain = 0,
|
||||
HEVCHigh = 1,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum AV1Level {
|
||||
// Also AV1_0
|
||||
AV1_2 = 0,
|
||||
// Also AV1_1
|
||||
AV1_21 = 1,
|
||||
AV1_22 = 2,
|
||||
AV1_23 = 3,
|
||||
AV1_3 = 4,
|
||||
AV1_31 = 5,
|
||||
AV1_32 = 6,
|
||||
AV1_33 = 7,
|
||||
AV1_4 = 8,
|
||||
AV1_41 = 9,
|
||||
AV1_42 = 10,
|
||||
AV1_43 = 11,
|
||||
AV1_5 = 12,
|
||||
AV1_51 = 13,
|
||||
AV1_52 = 14,
|
||||
AV1_53 = 15,
|
||||
AV1_6 = 16,
|
||||
AV1_61 = 17,
|
||||
AV1_62 = 18,
|
||||
AV1_63 = 19,
|
||||
AV1_7 = 20,
|
||||
AV1_71 = 21,
|
||||
AV1_72 = 22,
|
||||
AV1_73 = 23,
|
||||
AV1AutoSelect = 24,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencPicFlags {
|
||||
ForceIntra = 0x1,
|
||||
ForceIDR = 0x2,
|
||||
OutputSpspps = 0x4,
|
||||
Eos = 0x8,
|
||||
DisableEncStateAdvance = 0x10,
|
||||
OuputReconFrame = 0x20,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencMemoryHeap {
|
||||
AutoSelect = 0,
|
||||
VID = 1,
|
||||
SystemCached = 2,
|
||||
SystemUncached = 3,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencBFrameRefMode {
|
||||
Disabled = 0x0,
|
||||
Each = 0x1,
|
||||
Middle = 0x2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencH264EntropyCodingMode {
|
||||
AutoSelect = 0x0,
|
||||
Cabac = 0x1,
|
||||
Cavlc = 0x2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencH264BDirectMode {
|
||||
Autoselect = 0x0,
|
||||
Disable = 0x1,
|
||||
Temporal = 0x2,
|
||||
Spatial = 0x3,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencH264FMOMode {
|
||||
AutoSelect = 0x0,
|
||||
Enable = 0x1,
|
||||
Disable = 0x2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencH264AdaptiveTransformMode {
|
||||
AutoSelect = 0x0,
|
||||
Disable = 0x1,
|
||||
Enable = 0x2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencStereoPackingMode {
|
||||
None = 0x0,
|
||||
Checkerboard = 0x1,
|
||||
ColInterleave = 0x2,
|
||||
RowInterleave = 0x3,
|
||||
SideBySide = 0x4,
|
||||
TopBottm = 0x5,
|
||||
FrameSeq = 0x6,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencInputResourceType {
|
||||
DirectX = 0x0,
|
||||
CudaDeivcePtr = 0x1,
|
||||
CudaArray = 0x2,
|
||||
OpenGLTex = 0x3,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencBufferUsage {
|
||||
Image = 0x0,
|
||||
MotionVector = 0x1,
|
||||
BitStream = 0x2,
|
||||
OutputRecon = 0x4,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
#[derive(Clone)]
|
||||
pub enum NVencDeviceType {
|
||||
DirectX = 0x0,
|
||||
Cuda = 0x1,
|
||||
OpenGL = 0x2,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencNumRefFrames {
|
||||
AutoSelect = 0x0,
|
||||
Frames1 = 0x1,
|
||||
Frames2 = 0x2,
|
||||
Frames3 = 0x3,
|
||||
Frames4 = 0x4,
|
||||
Frames5 = 0x5,
|
||||
Frames6 = 0x6,
|
||||
Frames7 = 0x7,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencTemporalFilterLevel {
|
||||
Level0 = 0,
|
||||
Level4 = 4,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencCaps {
|
||||
NumMaxBFrames,
|
||||
SupportedRateControlModes,
|
||||
FieldEncoding,
|
||||
MonoChrome,
|
||||
FMO,
|
||||
QPELMV,
|
||||
BDirectMode,
|
||||
CABAC,
|
||||
AdaptiveTransform,
|
||||
StereoMVC,
|
||||
MaxTemporalLayers,
|
||||
HierarchicalBFrames,
|
||||
LevelMax,
|
||||
LevelMin,
|
||||
SeperateColorPlane,
|
||||
WidthMax,
|
||||
HeightMax,
|
||||
TemporalSVC,
|
||||
DynResChnage,
|
||||
DynBitrateChange,
|
||||
DynForceConstQP,
|
||||
DynRcModeChange,
|
||||
SubframeReadBack,
|
||||
ConstrainedEncoding,
|
||||
IntraRefresh,
|
||||
CustomVBVBufSize,
|
||||
DynamicSliceMode,
|
||||
Invalidation,
|
||||
PreProcSupport,
|
||||
AsyncEncodeSupport,
|
||||
MbNumMax,
|
||||
MbPerSecMax,
|
||||
YUV444Encode,
|
||||
LoselessEncode,
|
||||
SupportSao,
|
||||
MeOnlyMode,
|
||||
LookAhead,
|
||||
TemporalAQ,
|
||||
MaxLtrFrames,
|
||||
SupportWeightedPrediction,
|
||||
DynamicQueryEncoderCapacity,
|
||||
BFrameRefMode,
|
||||
EmphasisLevelMap,
|
||||
WidthMin,
|
||||
HeightMin,
|
||||
MultipleRefFrames,
|
||||
AlphaLayerEncoding,
|
||||
NumEncoderEngines,
|
||||
SingleSliceIntraRefresh,
|
||||
DisableEncStateAdvance,
|
||||
OutputReconSurface,
|
||||
OutputBlockStats,
|
||||
SupportTemporalFilter,
|
||||
LookAheadLevel,
|
||||
UndirectionalB,
|
||||
MVHEVCEncode,
|
||||
SupportYUV422Encode,
|
||||
ExposedCount,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencHEVCCuSize {
|
||||
AutoSelect = 0,
|
||||
Size8x8 = 1,
|
||||
Size16x16 = 2,
|
||||
Size32x32 = 3,
|
||||
Size64x64 = 4,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencAV1PartSize {
|
||||
AutoSelect = 0,
|
||||
Size4x4 = 1,
|
||||
Size8x8 = 2,
|
||||
Size16x16 = 3,
|
||||
Size32x32 = 4,
|
||||
Size64x64 = 5,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencVUIVideoFormat {
|
||||
Component = 0,
|
||||
Pal = 1,
|
||||
NTSC = 2,
|
||||
SeCam = 3,
|
||||
Mac = 4,
|
||||
Unspecified = 5,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencVUIColorPrimaries {
|
||||
Undefined = 0,
|
||||
BT709 = 1,
|
||||
Unspecified = 2,
|
||||
Reserved = 3,
|
||||
BT470M = 4,
|
||||
BT470BG = 5,
|
||||
SMPTE170M = 6,
|
||||
SMPTE240M = 7,
|
||||
FIlm = 8,
|
||||
BT2020 = 9,
|
||||
SMPTE428 = 10,
|
||||
SMPTE431 = 11,
|
||||
SMPTE432 = 12,
|
||||
JEDECP22 = 22,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencVUITransCharacteristics {
|
||||
Undefined = 0,
|
||||
BT709 = 1,
|
||||
Unspecified = 2,
|
||||
Reserved = 3,
|
||||
BT470M = 4,
|
||||
BT470BG = 5,
|
||||
SMPTE170M = 6,
|
||||
SMPTE240M = 7,
|
||||
Linear = 8,
|
||||
Log = 9,
|
||||
LogSqrt = 10,
|
||||
IEC61966_2_4 = 11,
|
||||
SRGB = 13,
|
||||
BT2020_10 = 14,
|
||||
BT2020_12 = 15,
|
||||
SMPTE2084 = 16,
|
||||
SMPTE428 = 17,
|
||||
AribStdB67 = 18,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencVUIMatrixCoeffs {
|
||||
RGB = 0,
|
||||
BT709 = 1,
|
||||
Unspecified = 2,
|
||||
Reserved = 3,
|
||||
FCC = 4,
|
||||
BT470BG = 5,
|
||||
SMPTE170M,
|
||||
SMPTE240M,
|
||||
YCGCO = 8,
|
||||
BT2020NCL = 9,
|
||||
BT2020CL = 10,
|
||||
SMPTE2085 = 11,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencLookAheadLevel {
|
||||
Level0 = 0,
|
||||
Level1 = 1,
|
||||
Level2 = 2,
|
||||
Level3 = 3,
|
||||
AutoSelect = 15,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencBitDepth {
|
||||
Invalid = 0,
|
||||
Depth8 = 8,
|
||||
Depth10 = 10,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
#[repr(C)]
|
||||
pub enum NVencTuningInfo {
|
||||
Undefined = 0,
|
||||
HighQuality = 1,
|
||||
LowLatency = 2,
|
||||
UltraLowLatency = 3,
|
||||
Lossless = 4,
|
||||
UltraHighQuality = 5,
|
||||
Count,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub enum NVencSplitEncodeMode {
|
||||
Audo = 0,
|
||||
AutoForced = 1,
|
||||
TwoForced = 2,
|
||||
ThreeForced = 3,
|
||||
FourForced = 4,
|
||||
Disable = 15,
|
||||
}
|
||||
-214
@@ -1,214 +0,0 @@
|
||||
use std::{ffi::c_char, mem::MaybeUninit, os::raw::c_void};
|
||||
|
||||
use crate::{
|
||||
stdcall,
|
||||
sys::{
|
||||
enums::{NVencBufferFormat, NVencTuningInfo},
|
||||
result::NVencResult,
|
||||
structs::{
|
||||
Guid, NVencCapsParam, NVencCreateBitstreamBuffer, NVencCreateInputBuffer,
|
||||
NVencInitializeParams, NVencLockBitStream, NVencLockInputBuffer, NVencMapInputResource,
|
||||
NVencOpenEncodeSessionExParams, NVencPicParams, NVencPresetConfig,
|
||||
NVencRegisterResource,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
#[repr(C)]
|
||||
#[derive(Clone)]
|
||||
pub struct NVencFunctionList {
|
||||
pub version: u32,
|
||||
rsvd: u32,
|
||||
#[deprecated]
|
||||
nvenc_open_encode_session: NvencOpenEncodeSession,
|
||||
pub nvenc_get_encoder_guid_count: NvencGetEncodeGuidCount,
|
||||
nvenc_get_encoder_profile_guid_count: NvencGetEncodeProfileGuidCount,
|
||||
nvenc_get_encoder_profile_guids: NvencGetEncodeProfileGuids,
|
||||
pub nvenc_get_encoder_guids: NvencGetEncodeGuids,
|
||||
nvenc_get_input_format_count: NvencGetInputFormatCount,
|
||||
nvenc_get_input_formats: NvencGetInputFormats,
|
||||
nvenc_get_encode_caps: NvencGetEncodeCaps,
|
||||
pub nvenc_get_encode_preset_count: NvencGetEncodePresetCount,
|
||||
pub nvenc_get_encode_preset_guids: NvencGetEncodePresetGuids,
|
||||
nvenc_get_encode_preset_config: NvencGetEncodePresetConfig,
|
||||
pub nvenc_initialize_encoder: NvencInitializeEncoder,
|
||||
pub nvenc_create_input_buffer: NvencCreateInputBuffer,
|
||||
pub nvenc_destroy_input_buffer: NvencDestroyInputBuffer,
|
||||
pub nvenc_create_bit_stream_buffer: NvencCreateBitStreamBuffer,
|
||||
pub nvenc_destory_bit_stream_buffer: NvencDestroyBitStreamBuffer,
|
||||
pub nvenc_encode_picture: NvencEncodePicture,
|
||||
pub nvenc_lock_bit_stream: NvencLockBitStream,
|
||||
pub nvenc_unlock_bit_stream: NvencUnlockBitStream,
|
||||
pub nvenc_lock_input_buffer: NvencLockInputBuffer,
|
||||
pub nvenc_unlock_input_buffer: NvencUnlockInputBuffer,
|
||||
nvenc_get_encode_stats: NvencGetEncodeStats,
|
||||
nvenc_get_sequence_params: NvencGetSequenceParams,
|
||||
nvenc_register_async_event: NvencRegisterAsyncEvent,
|
||||
nvenc_unregister_async_event: NvencUnregisterAsyncEvent,
|
||||
pub nvenc_map_input_resource: NvencMapInputResource,
|
||||
pub nvenc_unmap_input_resource: NvencUnmapInputResource,
|
||||
pub nvenc_destroy_encoder: NvencDestroyEncoder,
|
||||
nvenc_invalidate_ref_frames: NvencInvalidateRefFrames,
|
||||
pub nvenc_open_encode_session_ex: NvencOpenEncodeSessionEx,
|
||||
pub nvenc_register_resource: NvencRegisterResource,
|
||||
pub nvenc_unregister_resource: NvencUnRegisterResource,
|
||||
nvenc_reconfigure_encoder: NvencReconfigureEncoder,
|
||||
rsvd1: *mut c_void,
|
||||
nvenc_create_mv_buffer: NvencCreateMVBuffer,
|
||||
nvenc_destory_mv_buffer: NvencDestoryMVBuffer,
|
||||
nvenc_run_motion_estimation_only: NvencRunMotionEstimationOnly,
|
||||
pub nvenc_get_last_error: NvencGetLastError,
|
||||
nvenc_set_io_cuda_streams: NvencSetIOCudaStreams,
|
||||
pub nvenc_get_encode_preset_config_ex: NvencGetEncodePresetConfigEx,
|
||||
nvenc_get_sequence_params_ex: NvencGetSequenceParamsEx,
|
||||
nvenc_store_encoder_state: NvencStoreEncoderState,
|
||||
nvenc_look_ahead_picture: NvencLookAheadPicture,
|
||||
rsvd2: [*mut c_void; 275],
|
||||
}
|
||||
|
||||
type NvencOpenEncodeSession =
|
||||
stdcall!(fn(device: *mut c_void, device_type: i32, encoder: *mut *mut c_void) -> NVencResult);
|
||||
type NvencGetEncodeGuidCount =
|
||||
stdcall!(fn(encoder: *mut c_void, encoder_guid_count: *mut u32) -> NVencResult);
|
||||
type NvencGetEncodeGuids = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
guids: *mut Guid,
|
||||
guid_array_size: u32,
|
||||
guid_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodeProfileGuidCount = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encode_guid: Guid,
|
||||
encoder_profile_guid_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodeProfileGuids = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
profile_guids: *mut Guid,
|
||||
guid_array_size: u32,
|
||||
guid_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetInputFormatCount = stdcall!(
|
||||
fn(encoder: *mut c_void, encoder_guid: Guid, input_fmt_count: *mut u32) -> NVencResult
|
||||
);
|
||||
type NvencGetInputFormats = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
input_formats: *mut NVencBufferFormat,
|
||||
input_format_array_size: u32,
|
||||
input_format_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodeCaps = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encoder_guid: Guid,
|
||||
caps_param: *mut NVencCapsParam,
|
||||
caps_val: *mut i32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodePresetCount = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encoder_guid: Guid,
|
||||
encoder_preset_guid_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodePresetGuids = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encode_guid: Guid,
|
||||
preset_guids: *mut Guid,
|
||||
guid_array_size: u32,
|
||||
encoder_preset_guid_count: *mut u32,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodePresetConfig = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encode_guid: Guid,
|
||||
preset_guid: Guid,
|
||||
preset_config: *mut NVencPresetConfig,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencGetEncodePresetConfigEx = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
encode_guid: Guid,
|
||||
preset_guid: Guid,
|
||||
tuning_info: NVencTuningInfo,
|
||||
preset_config: *mut MaybeUninit<NVencPresetConfig>,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencInitializeEncoder = stdcall!(
|
||||
fn(encoder: *mut c_void, create_encode_params: *mut NVencInitializeParams) -> NVencResult
|
||||
);
|
||||
type NvencCreateInputBuffer = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
create_input_buffer_params: *mut NVencCreateInputBuffer,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencDestroyInputBuffer =
|
||||
stdcall!(fn(encoder: *mut c_void, input_buffer: *mut c_void) -> NVencResult);
|
||||
type NvencCreateBitStreamBuffer = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
create_bit_stream_buffer_params: *mut NVencCreateBitstreamBuffer,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencDestroyBitStreamBuffer =
|
||||
stdcall!(fn(encoder: *mut c_void, bitstream_buffer: *mut c_void) -> NVencResult);
|
||||
type NvencEncodePicture = stdcall!(
|
||||
fn(encoder: *mut c_void, encode_pic_params: *mut MaybeUninit<NVencPicParams>) -> NVencResult
|
||||
);
|
||||
type NvencLockBitStream = stdcall!(
|
||||
fn(
|
||||
encoder: *mut c_void,
|
||||
lock_bitstream_buffer_params: *mut MaybeUninit<NVencLockBitStream>,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencUnlockBitStream =
|
||||
stdcall!(fn(encoder: *mut c_void, bitstream_buffer: *mut c_void) -> NVencResult);
|
||||
type NvencLockInputBuffer = stdcall!(
|
||||
fn(encoder: *mut c_void, lock_input_buffer_params: *mut NVencLockInputBuffer) -> NVencResult
|
||||
);
|
||||
type NvencUnlockInputBuffer =
|
||||
stdcall!(fn(encoder: *mut c_void, input_buffer: *mut c_void) -> NVencResult);
|
||||
type NvencGetEncodeStats = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencGetSequenceParams = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencRegisterAsyncEvent = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencUnregisterAsyncEvent = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencMapInputResource = stdcall!(
|
||||
fn(encoder: *mut c_void, map_input_resource_params: *mut NVencMapInputResource) -> NVencResult
|
||||
);
|
||||
type NvencUnmapInputResource =
|
||||
stdcall!(fn(encoder: *mut c_void, mapped_input_buffer: *mut c_void) -> NVencResult);
|
||||
type NvencDestroyEncoder = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencInvalidateRefFrames = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencOpenEncodeSessionEx = stdcall!(
|
||||
fn(
|
||||
open_session_ex_params: *mut NVencOpenEncodeSessionExParams,
|
||||
encoder: *mut *mut c_void,
|
||||
) -> NVencResult
|
||||
);
|
||||
type NvencRegisterResource = stdcall!(
|
||||
fn(encoder: *mut c_void, register_res_params: *mut NVencRegisterResource) -> NVencResult
|
||||
);
|
||||
type NvencUnRegisterResource =
|
||||
stdcall!(fn(encoder: *mut c_void, registered_res: *mut c_void) -> NVencResult);
|
||||
type NvencReconfigureEncoder = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
|
||||
type NvencCreateMVBuffer = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencDestoryMVBuffer = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencRunMotionEstimationOnly = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencGetLastError = stdcall!(fn(encoder: *mut c_void) -> *const c_char);
|
||||
type NvencSetIOCudaStreams = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencGetSequenceParamsEx = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencStoreEncoderState = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
type NvencLookAheadPicture = stdcall!(fn(encoder: *mut c_void) -> NVencResult);
|
||||
Vendored
-146
@@ -1,146 +0,0 @@
|
||||
use crate::sys::structs::Guid;
|
||||
|
||||
pub const NV_ENC_CODEC_H264_GUID: Guid = Guid::from_values(
|
||||
0x6bc82762,
|
||||
0x4e63,
|
||||
0x4ca4,
|
||||
[0xaa, 0x85, 0x1e, 0x50, 0xf3, 0x21, 0xf6, 0xbf],
|
||||
);
|
||||
pub const NV_ENC_CODEC_HEVC_GUID: Guid = Guid::from_values(
|
||||
0x790cdc88,
|
||||
0x4522,
|
||||
0x4d7b,
|
||||
[0x94, 0x25, 0xbd, 0xa9, 0x97, 0x5f, 0x76, 0x3],
|
||||
);
|
||||
pub const NV_ENC_CODEC_AV1_GUID: Guid = Guid::from_values(
|
||||
0x0a352289,
|
||||
0x0aa7,
|
||||
0x4759,
|
||||
[0x86, 0x2d, 0x5d, 0x15, 0xcd, 0x16, 0xd2, 0x54],
|
||||
);
|
||||
pub const NV_ENC_CODEC_PROFILE_AUTOSELECT_GUID: Guid = Guid::from_values(
|
||||
0xbfd6f8e7,
|
||||
0x233c,
|
||||
0x4341,
|
||||
[0x8b, 0x3e, 0x48, 0x18, 0x52, 0x38, 0x3, 0xf4],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_BASELINE_GUID: Guid = Guid::from_values(
|
||||
0x727bcaa,
|
||||
0x78c4,
|
||||
0x4c83,
|
||||
[0x8c, 0x2f, 0xef, 0x3d, 0xff, 0x26, 0x7c, 0x6a],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_MAIN_GUID: Guid = Guid::from_values(
|
||||
0x60b5c1d4,
|
||||
0x67fe,
|
||||
0x4790,
|
||||
[0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_HIGH_GUID: Guid = Guid::from_values(
|
||||
0xe7cbc309,
|
||||
0x4f7a,
|
||||
0x4b89,
|
||||
[0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_HIGH_10_GUID: Guid = Guid::from_values(
|
||||
0x8f0c337e,
|
||||
0x186c,
|
||||
0x48e9,
|
||||
[0xa6, 0x9d, 0x7a, 0x83, 0x34, 0x08, 0x97, 0x5],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_HIGH_422_GUID: Guid = Guid::from_values(
|
||||
0xff3242e9,
|
||||
0x613c,
|
||||
0x4295,
|
||||
[0xa1, 0xe8, 0x2a, 0x7f, 0xe9, 0x4d, 0x81, 0x33],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_HIGH_444_GUID: Guid = Guid::from_values(
|
||||
0x7ac663cb,
|
||||
0xa598,
|
||||
0x4960,
|
||||
[0xb8, 0x44, 0x33, 0x9b, 0x26, 0x1a, 0x7d, 0x52],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_STEREO_GUID: Guid = Guid::from_values(
|
||||
0x40847bf5,
|
||||
0x33f7,
|
||||
0x4601,
|
||||
[0x90, 0x84, 0xe8, 0xfe, 0x3c, 0x1d, 0xb8, 0xb7],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_PROGRESSIVE_HIGH_GUID: Guid = Guid::from_values(
|
||||
0xb405afac,
|
||||
0xf32b,
|
||||
0x417b,
|
||||
[0x89, 0xc4, 0x9a, 0xbe, 0xed, 0x3e, 0x59, 0x78],
|
||||
);
|
||||
pub const NV_ENC_H264_PROFILE_CONSTRAINED_HIGH_GUID: Guid = Guid::from_values(
|
||||
0xaec1bd87,
|
||||
0xe85b,
|
||||
0x48f2,
|
||||
[0x84, 0xc3, 0x98, 0xbc, 0xa6, 0x28, 0x50, 0x72],
|
||||
);
|
||||
pub const NV_ENC_HEVC_PROFILE_MAIN_GUID: Guid = Guid::from_values(
|
||||
0xb514c39a,
|
||||
0xb55b,
|
||||
0x40fa,
|
||||
[0x87, 0x8f, 0xf1, 0x25, 0x3b, 0x4d, 0xfd, 0xec],
|
||||
);
|
||||
pub const NV_ENC_HEVC_PROFILE_MAIN10_GUID: Guid = Guid::from_values(
|
||||
0xfa4d2b6c,
|
||||
0x3a5b,
|
||||
0x411a,
|
||||
[0x80, 0x18, 0x0a, 0x3f, 0x5e, 0x3c, 0x9b, 0xe5],
|
||||
);
|
||||
pub const NV_ENC_HEVC_PROFILE_FREXT_GUID: Guid = Guid::from_values(
|
||||
0x51ec32b5,
|
||||
0x1b4c,
|
||||
0x453c,
|
||||
[0x9c, 0xbd, 0xb6, 0x16, 0xbd, 0x62, 0x13, 0x41],
|
||||
);
|
||||
pub const NV_ENC_AV1_PROFILE_MAIN_GUID: Guid = Guid::from_values(
|
||||
0x5f2a39f5,
|
||||
0xf14e,
|
||||
0x4f95,
|
||||
[0x9a, 0x9e, 0xb7, 0x6d, 0x56, 0x8f, 0xcf, 0x97],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P1_GUID: Guid = Guid::from_values(
|
||||
0xfc0a8d3e,
|
||||
0x45f8,
|
||||
0x4cf8,
|
||||
[0x80, 0xc7, 0x29, 0x88, 0x71, 0x59, 0xe, 0xbf],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P2_GUID: Guid = Guid::from_values(
|
||||
0xf581cfb8,
|
||||
0x88d6,
|
||||
0x4381,
|
||||
[0x93, 0xf0, 0xdf, 0x13, 0xf9, 0xc2, 0x7d, 0xab],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P3_GUID: Guid = Guid::from_values(
|
||||
0x36850110,
|
||||
0x3a07,
|
||||
0x441f,
|
||||
[0x94, 0xd5, 0x36, 0x70, 0x63, 0x1f, 0x91, 0xf6],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P4_GUID: Guid = Guid::from_values(
|
||||
0x90a7b826,
|
||||
0xdf06,
|
||||
0x4862,
|
||||
[0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P5_GUID: Guid = Guid::from_values(
|
||||
0x21c6e6b4,
|
||||
0x297a,
|
||||
0x4cba,
|
||||
[0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P6_GUID: Guid = Guid::from_values(
|
||||
0x8e75c279,
|
||||
0x6299,
|
||||
0x4ab6,
|
||||
[0x83, 0x2, 0xb, 0x21, 0x5a, 0x33, 0x5c, 0xf5],
|
||||
);
|
||||
pub const NV_ENC_PRESET_P7_GUID: Guid = Guid::from_values(
|
||||
0x84848c12,
|
||||
0x6f71,
|
||||
0x4c13,
|
||||
[0x93, 0x1b, 0x53, 0xe2, 0x83, 0xf5, 0x79, 0x74],
|
||||
);
|
||||
Vendored
-6
@@ -1,6 +0,0 @@
|
||||
pub mod enums;
|
||||
pub mod function_table;
|
||||
pub mod guids;
|
||||
pub mod result;
|
||||
pub mod structs;
|
||||
pub mod version;
|
||||
Vendored
-79
@@ -1,79 +0,0 @@
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
#[repr(C)]
|
||||
pub enum NVencResult {
|
||||
Success,
|
||||
NoDevice,
|
||||
UnsupportedDevice,
|
||||
InvalidEncoderDevice,
|
||||
InvalidDevice,
|
||||
DeviceNotExist,
|
||||
InvalidPtr,
|
||||
InvalidParam,
|
||||
InvalidEvent,
|
||||
InvalidCall,
|
||||
OutOfMemory,
|
||||
NotInitialized,
|
||||
UnsupportedParam,
|
||||
LockBusy,
|
||||
NotEnoughBuffer,
|
||||
InvalidVersion,
|
||||
MapFailed,
|
||||
NeedMoreInput,
|
||||
EncoderBusy,
|
||||
EventNotRegisterd,
|
||||
Generic,
|
||||
IncompatibleClientKey,
|
||||
Unimplemented,
|
||||
ResourceRegisterFailed,
|
||||
ResourceNotRegistered,
|
||||
ResourceNotMapped,
|
||||
NeedMoreOutput,
|
||||
}
|
||||
|
||||
impl NVencResult {
|
||||
pub fn into_error(self) -> Result<(), NVencError> {
|
||||
NVencError::from_result(self)
|
||||
}
|
||||
}
|
||||
|
||||
impl NVencError {
|
||||
pub fn from_result(value: NVencResult) -> Result<(), Self> {
|
||||
if value == NVencResult::Success {
|
||||
Ok(())
|
||||
} else {
|
||||
// Safety: They are one case different, so they always align
|
||||
unsafe { Err(std::mem::transmute::<NVencResult, NVencError>(value)) }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
#[repr(C)]
|
||||
pub enum NVencError {
|
||||
NoDevice = NVencResult::NoDevice as isize,
|
||||
UnsupportedDevice,
|
||||
InvalidEncoderDevice,
|
||||
InvalidDevice,
|
||||
DeviceNotExist,
|
||||
InvalidPtr,
|
||||
InvalidEvent,
|
||||
InvalidParam,
|
||||
InvalidCall,
|
||||
OutOfMemory,
|
||||
NotInitialized,
|
||||
UnsupportedParam,
|
||||
LockBusy,
|
||||
NotEnoughBuffer,
|
||||
InvalidVersion,
|
||||
MapFailed,
|
||||
NeedMoreInput,
|
||||
EncoderBusy,
|
||||
EventNotRegisterd,
|
||||
Generic,
|
||||
IncompatibleClientKey,
|
||||
Unimplemented,
|
||||
ResourceRegisterFailed,
|
||||
ResourceNotRegistered,
|
||||
ResourceNotMapped,
|
||||
NeedMoreOutput,
|
||||
}
|
||||
Vendored
-944
@@ -1,944 +0,0 @@
|
||||
use std::{ffi::c_void, mem::ManuallyDrop};
|
||||
|
||||
use crate::sys::{
|
||||
enums::{
|
||||
NVencAV1PartSize, NVencBFrameRefMode, NVencBitDepth, NVencBufferFormat, NVencBufferUsage,
|
||||
NVencCaps, NVencDeviceType, NVencDisplayPicStruct, NVencH264AdaptiveTransformMode,
|
||||
NVencH264BDirectMode, NVencH264EntropyCodingMode, NVencH264FMOMode, NVencHEVCCuSize,
|
||||
NVencInputResourceType, NVencLookAheadLevel, NVencMVPrecision, NVencMemoryHeap,
|
||||
NVencMultiPass, NVencNumRefFrames, NVencOutputStatsLevel, NVencParamsFrameFieldMode,
|
||||
NVencParamsRcMode, NVencPicStruct, NVencPicType, NVencQPMapMode, NVencStateRestoreType,
|
||||
NVencStereoPackingMode, NVencTemporalFilterLevel, NVencTuningInfo, NVencVUIColorPrimaries,
|
||||
NVencVUIMatrixCoeffs, NVencVUITransCharacteristics, NVencVUIVideoFormat,
|
||||
},
|
||||
version::struct_version,
|
||||
};
|
||||
|
||||
#[derive(Default, Clone, PartialEq, Eq)]
|
||||
#[repr(C)]
|
||||
pub struct Guid {
|
||||
pub data1: u32,
|
||||
pub data2: u16,
|
||||
pub data3: u16,
|
||||
pub data4: [u8; 8],
|
||||
}
|
||||
|
||||
impl Guid {
|
||||
pub const fn from_values(data1: u32, data2: u16, data3: u16, data4: [u8; 8]) -> Self {
|
||||
Self {
|
||||
data1,
|
||||
data2,
|
||||
data3,
|
||||
data4,
|
||||
}
|
||||
}
|
||||
|
||||
pub const fn from_u128(uuid: u128) -> Self {
|
||||
Self {
|
||||
data1: (uuid >> 96) as u32,
|
||||
data2: ((uuid >> 80) & 0xffff) as u16,
|
||||
data3: ((uuid >> 64) & 0xffff) as u16,
|
||||
data4: (uuid as u64).to_be_bytes(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencCapsParam {
|
||||
version: u32,
|
||||
caps_to_query: NVencCaps,
|
||||
rsvd: [u32; 62],
|
||||
}
|
||||
|
||||
pub const NV_ENC_CAPS_PARAM_VERSION: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencRestoreEncoderStateParams {
|
||||
version: u32,
|
||||
buffer: u32,
|
||||
state: NVencStateRestoreType,
|
||||
rsvd: u32,
|
||||
output_bit_stream: *mut c_void,
|
||||
completion_event: *mut c_void,
|
||||
rsvd1: [u32; 64],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_RESTORE_STATE_PARAMS_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencOutputStatsBlock {
|
||||
version: u32,
|
||||
qp: u8,
|
||||
rsvd: [u8; 3],
|
||||
bit_count: u32,
|
||||
satd_cost: u32,
|
||||
rsvd1: [u32; 12],
|
||||
}
|
||||
|
||||
pub const NV_ENC_OUTPUT_STATS_BLOCK_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencOutputStatsRow {
|
||||
version: u32,
|
||||
qp: u8,
|
||||
rsvd: [u8; 3],
|
||||
bit_count: u32,
|
||||
satd_cost: u32,
|
||||
rsvd1: [u32; 12],
|
||||
}
|
||||
|
||||
pub const NV_ENC_OUTPUT_STATS_ROW_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencEncodeOutParams {
|
||||
version: u32,
|
||||
bitstream_size_in_bytes: u32,
|
||||
rsvd: [u32; 62],
|
||||
}
|
||||
|
||||
pub const NV_ENC_ENCODE_OUT_PARAMS_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencLookAheadPicParams {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
input_buffer: *mut c_void,
|
||||
picture_type: NVencPicType,
|
||||
rsvd1: [u32; 63],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_LOOKAHEAD_PIC_PARAMS_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencCreateInputBuffer {
|
||||
pub version: u32,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub memory_heap: NVencMemoryHeap,
|
||||
pub buffer_fmt: NVencBufferFormat,
|
||||
pub rsvd: u32,
|
||||
pub input_buffer: *mut c_void,
|
||||
pub p_sys_mem_buffer: *mut c_void,
|
||||
pub rsvd1: [u32; 58],
|
||||
pub rsvd2: [*mut c_void; 63],
|
||||
}
|
||||
|
||||
pub const NV_ENC_CREATE_INPUT_BUFFER_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencCreateBitstreamBuffer {
|
||||
pub version: u32,
|
||||
#[deprecated]
|
||||
size: u32,
|
||||
#[deprecated]
|
||||
memory_heap: NVencMemoryHeap,
|
||||
rsvd: u32,
|
||||
pub bitstream_buffer: *mut c_void,
|
||||
bitstream_buffer_ptr: *mut c_void,
|
||||
rsvd1: [u32; 58],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_CREATE_BITSTREAM_BUFFER_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencMVector {
|
||||
mvx: i16,
|
||||
mvy: i16,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencH264MVData {
|
||||
mv: [NVencMVector; 4],
|
||||
mb_type: u8,
|
||||
partition_type: u8,
|
||||
rsvd: u16,
|
||||
mb_cost: u32,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencHEVCMVData {
|
||||
mv: [NVencMVector; 4],
|
||||
cu_type: u8,
|
||||
cu_size: u8,
|
||||
partition_mode: u8,
|
||||
last_cu_int_ctb: u8,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencCreateMVBuffer {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
mv_buffer: *mut c_void,
|
||||
rsvd1: [u32; 254],
|
||||
rsvd2: [*mut c_void; 63],
|
||||
}
|
||||
|
||||
pub const NV_ENC_CREATE_MV_BUFFER_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencQP {
|
||||
qp_inter_p: u32,
|
||||
qp_inter_b: u32,
|
||||
qp_intra: u32,
|
||||
}
|
||||
|
||||
pub const MAX_NUM_VIEWS_MINUS_1: usize = 7;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencRcParams {
|
||||
version: u32,
|
||||
pub rate_control_mode: NVencParamsRcMode,
|
||||
const_qp: NVencQP,
|
||||
pub average_bit_rate: u32,
|
||||
max_bit_rate: u32,
|
||||
vbv_buffer_size: u32,
|
||||
vbv_initial_delay: u32,
|
||||
// TODO: bit fields
|
||||
bit_fields: NVencRcParamsBitFlags,
|
||||
min_qp: NVencQP,
|
||||
max_qp: NVencQP,
|
||||
initial_rcqp: NVencQP,
|
||||
temporal_layer_idx_mask: u32,
|
||||
temporal_layer_qp: [u8; 8],
|
||||
target_quality: u8,
|
||||
target_quality_lsb: u8,
|
||||
pub look_ahead_depth: u16,
|
||||
low_delay_key_frame_scale: u8,
|
||||
y_dc_qp_index_offset: i8,
|
||||
u_dc_qp_index_offset: i8,
|
||||
v_dc_qp_index_offset: i8,
|
||||
qp_map_mode: NVencQPMapMode,
|
||||
multi_pass: NVencMultiPass,
|
||||
alpha_layer_bitrate_ratio: u32,
|
||||
cb_qp_index_offset: i8,
|
||||
cr_qp_index_offset: i8,
|
||||
rsvd2: u16,
|
||||
lookahead_level: NVencLookAheadLevel,
|
||||
view_bitrate_ratios: [u8; MAX_NUM_VIEWS_MINUS_1],
|
||||
rsvd3: u8,
|
||||
rsvd1: u32,
|
||||
}
|
||||
|
||||
#[repr(transparent)]
|
||||
pub struct NVencRcParamsBitFlags(u32);
|
||||
|
||||
impl NVencRcParamsBitFlags {}
|
||||
|
||||
pub const NV_ENC_RC_PARAMS_VER: u32 = struct_version(1);
|
||||
|
||||
pub const MAX_NUM_CLOCK_TS: usize = 3;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencClockTimeStampSet {
|
||||
pub bitflags: u32,
|
||||
pub time_offset: u32,
|
||||
}
|
||||
|
||||
impl NVencClockTimeStampSet {}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencTimeCode {
|
||||
pub display_pic_struct: NVencDisplayPicStruct,
|
||||
pub clock_timestamp: [NVencClockTimeStampSet; MAX_NUM_CLOCK_TS],
|
||||
pub skip_clock_timestamp_insertion: u32,
|
||||
}
|
||||
|
||||
pub const MULTIVIEW_MAX_NUM_REF_DISPLAY: usize = 32;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct HEVC3DReferenceDisplayInfo {
|
||||
// TODO
|
||||
pub bitflags: u32,
|
||||
pub prec_ref_display_width: i32,
|
||||
pub prec_ref_viewing_dist: i32,
|
||||
pub num_ref_displays_minus1: i32,
|
||||
pub left_view_id: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub right_view_id: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub exponent_ref_display_width: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub mantissa_ref_display_width: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub exponent_ref_viewing_distance: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub mantissa_ref_viewing_distance: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub num_sample_shift_plus512: [i32; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub additional_shift_present_flag: [u8; MULTIVIEW_MAX_NUM_REF_DISPLAY],
|
||||
pub rsvd2: [u32; 4],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct ChromaPoints {
|
||||
x: u16,
|
||||
y: u16,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct MasteringDisplayInfo {
|
||||
pub g: ChromaPoints,
|
||||
pub b: ChromaPoints,
|
||||
pub r: ChromaPoints,
|
||||
pub white_point: ChromaPoints,
|
||||
pub max_luma: u32,
|
||||
pub min_luma: u32,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct ContentLightLevel {
|
||||
pub max_content_light_level: u16,
|
||||
pub max_pic_average_light_level: u16,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigH264VUIParameters {
|
||||
overscan_info_present_flag: u32,
|
||||
overscan_info: u32,
|
||||
video_signal_type_present_flag: u32,
|
||||
video_format: NVencVUIVideoFormat,
|
||||
video_full_range_flag: u32,
|
||||
color_description_present_flag: u32,
|
||||
color_primaries: NVencVUIColorPrimaries,
|
||||
transfer_characteristics: NVencVUITransCharacteristics,
|
||||
color_matrix: NVencVUIMatrixCoeffs,
|
||||
chroma_sample_location_flag: u32,
|
||||
chroma_sample_location_top: u32,
|
||||
chroma_sample_location_bot: u32,
|
||||
bitstream_restriction_flag: u32,
|
||||
timing_info_present_flag: u32,
|
||||
num_unit_in_ticks: u32,
|
||||
time_scale: u32,
|
||||
rsvd: [u32; 12],
|
||||
}
|
||||
|
||||
pub type NVencConfigHEVCVUIParamaters = NVencConfigH264VUIParameters;
|
||||
|
||||
#[repr(C)]
|
||||
#[derive(Default)]
|
||||
pub struct NVencExternalMeHintCountsPerBlockType {
|
||||
pub bitflags: u32,
|
||||
pub rsvd1: [u32; 3],
|
||||
}
|
||||
|
||||
#[repr(transparent)]
|
||||
pub struct NVencExternalMeHint(i32);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencExternalMeSbHint {
|
||||
flags1: i16,
|
||||
flags2: i16,
|
||||
flags3: i16,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigH264 {
|
||||
bitflags: u32,
|
||||
level: u32,
|
||||
idr_period: u32,
|
||||
seperate_color_plane_flag: u32,
|
||||
disable_deblocking_filter_idc: u32,
|
||||
num_termpoal_layers: u32,
|
||||
sps_id: u32,
|
||||
pps_id: u32,
|
||||
adaptive_transform_mode: NVencH264AdaptiveTransformMode,
|
||||
fmo_mode: NVencH264FMOMode,
|
||||
bdirect_mode: NVencH264BDirectMode,
|
||||
entropy_coding_mode: NVencH264EntropyCodingMode,
|
||||
stereo_mode: NVencStereoPackingMode,
|
||||
intra_refresh_period: u32,
|
||||
intra_refresh_count: u32,
|
||||
max_num_ref_frames: u32,
|
||||
slice_mode: u32,
|
||||
slice_mode_data: u32,
|
||||
h264_vui_parameters: NVencConfigH264VUIParameters,
|
||||
ltr_num_frames: u32,
|
||||
ltr_trust_mode: u32,
|
||||
chroma_format_idc: u32,
|
||||
max_termpoal_layers: u32,
|
||||
use_b_frames_as_ref: NVencBFrameRefMode,
|
||||
num_ref_l0: NVencNumRefFrames,
|
||||
num_ref_l1: NVencNumRefFrames,
|
||||
output_bit_depth: NVencBitDepth,
|
||||
input_bit_depth: NVencBitDepth,
|
||||
tf_level: NVencTemporalFilterLevel,
|
||||
rsvd1: [u32; 264],
|
||||
rsvd: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigHEVC {
|
||||
level: u32,
|
||||
tier: u32,
|
||||
min_cu_size: NVencHEVCCuSize,
|
||||
max_cu_size: NVencHEVCCuSize,
|
||||
bitflags: u32,
|
||||
idr_period: u32,
|
||||
intra_refresh_cnt: u32,
|
||||
max_num_ref_frames_in_dpb: u32,
|
||||
ltr_num_frames: u32,
|
||||
vps_id: u32,
|
||||
sps_id: u32,
|
||||
pps_id: u32,
|
||||
slice_mode: u32,
|
||||
slice_mode_data: u32,
|
||||
max_temporal_layers_minus_1: u32,
|
||||
hevc_vui_paramters: NVencConfigHEVCVUIParamaters,
|
||||
ltr_trust_mode: u32,
|
||||
use_b_frames_as_ref: NVencBFrameRefMode,
|
||||
tf_level: NVencTemporalFilterLevel,
|
||||
disable_deblocking_filter_idc: u32,
|
||||
output_bit_depth: NVencBitDepth,
|
||||
input_bit_depth: NVencBitDepth,
|
||||
num_temporal_layers: u32,
|
||||
num_views: u32,
|
||||
rsvd1: [u32; 208],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_MAX_TILE_COLS_AV1: usize = 64;
|
||||
pub const NV_MAX_TILE_ROWS_AV1: usize = 64;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencFilmGrainParamsAV1 {
|
||||
bitfields: u32,
|
||||
point_y_value: [u8; 14],
|
||||
point_y_scaling: [u8; 14],
|
||||
point_cb_value: [u8; 10],
|
||||
point_cb_scaling: [u8; 10],
|
||||
point_cr_value: [u8; 10],
|
||||
point_cr_scaling: [u8; 10],
|
||||
ar_coeffs_y_plus128: [u8; 24],
|
||||
ar_coeffs_cb_plus128: [u8; 25],
|
||||
ar_coeffs_cr_plus128: [u8; 25],
|
||||
rsvd2: [u8; 2],
|
||||
cb_mult: u8,
|
||||
cb_luma_mult: u8,
|
||||
cb_offset: u16,
|
||||
cr_mult: u8,
|
||||
cr_luma_mult: u8,
|
||||
cr_offset: u16,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigAV1 {
|
||||
level: u32,
|
||||
tier: u32,
|
||||
min_part_size: NVencAV1PartSize,
|
||||
max_part_size: NVencAV1PartSize,
|
||||
bitflags: u32,
|
||||
idr_period: u32,
|
||||
intra_refresh_period: u32,
|
||||
intra_refresh_count: u32,
|
||||
max_num_ref_frames_in_dpb: u32,
|
||||
num_tiles_columns: u32,
|
||||
num_tile_rows: u32,
|
||||
rsvd2: u32,
|
||||
tile_widths: *mut u32,
|
||||
tile_heights: *mut u32,
|
||||
max_temporal_layers_minus_1: u32,
|
||||
color_primaries: NVencVUIColorPrimaries,
|
||||
transfer_characteristics: NVencVUITransCharacteristics,
|
||||
matrix_coefficients: NVencVUIMatrixCoeffs,
|
||||
color_range: u32,
|
||||
chroma_sample_position: u32,
|
||||
use_b_frames_as_ref: NVencBFrameRefMode,
|
||||
film_grain_params: NVencFilmGrainParamsAV1,
|
||||
num_fwd_refs: NVencNumRefFrames,
|
||||
num_bwd_refs: NVencNumRefFrames,
|
||||
input_bit_depth: NVencBitDepth,
|
||||
ltr_num_frames: u32,
|
||||
num_temporal_layers: u32,
|
||||
tf_level: NVencTemporalFilterLevel,
|
||||
rsvd1: [u32; 230],
|
||||
rsvd3: [*mut c_void; 62],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigH264MeOnly {
|
||||
bitflags: u32,
|
||||
rsvd1: [u32; 255],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfigHEVCMeOnly {
|
||||
rsvd: [u32; 256],
|
||||
rsvd1: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub union NVencCodecConfig {
|
||||
h264: ManuallyDrop<NVencConfigH264>,
|
||||
hevc: ManuallyDrop<NVencConfigHEVC>,
|
||||
av1: ManuallyDrop<NVencConfigAV1>,
|
||||
h264_meonly: ManuallyDrop<NVencConfigH264MeOnly>,
|
||||
hevc_meonly: ManuallyDrop<NVencConfigHEVCMeOnly>,
|
||||
rsvd: [u32; 320],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencConfig {
|
||||
pub version: u32,
|
||||
pub profile_guid: Guid,
|
||||
pub gop_len: u32,
|
||||
pub frame_interval_p: i32,
|
||||
mono_chrome_encoding: u32,
|
||||
frame_field_mode: NVencParamsFrameFieldMode,
|
||||
mv_precision: NVencMVPrecision,
|
||||
pub rc_params: NVencRcParams,
|
||||
encode_codec_config: NVencCodecConfig,
|
||||
rsvd: [u32; 278],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_CONFIG_VER: u32 = struct_version(9) | (1 << 31);
|
||||
|
||||
#[bitfields::bitfield(u32)]
|
||||
pub struct NVencInitializeParamsBitfields {
|
||||
#[bits(1)]
|
||||
pub report_slice_offsets: bool,
|
||||
#[bits(1)]
|
||||
pub enable_sub_frame_write: bool,
|
||||
#[bits(1)]
|
||||
pub enable_external_me_hints: bool,
|
||||
#[bits(1)]
|
||||
pub enable_me_only_mode: bool,
|
||||
#[bits(1)]
|
||||
pub enable_weighted_prediction: bool,
|
||||
#[bits(4)]
|
||||
pub split_encode_mode: u8,
|
||||
#[bits(1)]
|
||||
pub enable_output_in_vid_mem: bool,
|
||||
#[bits(1)]
|
||||
pub enable_recon_frame_output: bool,
|
||||
#[bits(1)]
|
||||
pub enable_output_stats: bool,
|
||||
#[bits(1)]
|
||||
pub enable_uni_directional_b: bool,
|
||||
#[bits(19)]
|
||||
pub rsvd: u32,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencInitializeParams {
|
||||
pub version: u32,
|
||||
pub encode_guid: Guid,
|
||||
pub preset_guid: Guid,
|
||||
pub encode_width: u32,
|
||||
pub encode_height: u32,
|
||||
pub dar_width: u32,
|
||||
pub dar_height: u32,
|
||||
pub frame_rate_num: u32,
|
||||
pub frame_rate_den: u32,
|
||||
pub enable_encode_async: u32,
|
||||
pub enable_ptd: u32,
|
||||
pub bit_flags: NVencInitializeParamsBitfields,
|
||||
pub(crate) priv_data_size: u32,
|
||||
pub(crate) rsvd: u32,
|
||||
pub(crate) priv_data: *mut c_void,
|
||||
pub encode_config: *mut NVencConfig,
|
||||
pub max_encode_width: u32,
|
||||
pub max_encode_height: u32,
|
||||
pub max_me_hint_counts_per_block: [NVencExternalMeHintCountsPerBlockType; 2],
|
||||
pub tuning_info: NVencTuningInfo,
|
||||
pub buffer_format: NVencBufferFormat,
|
||||
pub num_state_buffers: u32,
|
||||
pub output_stats_level: NVencOutputStatsLevel,
|
||||
pub(crate) rsvd1: [u32; 284],
|
||||
pub(crate) rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_INITIALIZE_PARAMS_VER: u32 = struct_version(7) | (1 << 31);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencReconfigureParams {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
re_init_encode_params: NVencInitializeParams,
|
||||
bitflags: u32,
|
||||
rsvd2: u32,
|
||||
}
|
||||
|
||||
pub const NV_ENC_RECONFIGURE_PARAMS_VER: u32 = struct_version(2) | (1 << 31);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPresetConfig {
|
||||
pub version: u32,
|
||||
rsvd: u32,
|
||||
pub preset_cfg: NVencConfig,
|
||||
rsvd1: [u32; 256],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_PRESET_CONFIG_VER: u32 = struct_version(5) | (1 << 31);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPicParamsMVC {
|
||||
version: u32,
|
||||
view_id: u32,
|
||||
temporal_id: u32,
|
||||
priority_id: u32,
|
||||
rsvd1: [u32; 12],
|
||||
rsvd2: [*mut c_void; 8],
|
||||
}
|
||||
|
||||
pub const NV_ENC_PIC_PARAMS_MVC_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub union NVencPicParamsH264EXT {
|
||||
pub mvc_pic_params: ManuallyDrop<NVencPicParamsMVC>,
|
||||
pub rsvd1: [u32; 32],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencSeiPayload {
|
||||
pub payload_size: u32,
|
||||
pub payload_type: u32,
|
||||
pub payload: *mut u8,
|
||||
}
|
||||
|
||||
pub type NVencH264SeiPayload = NVencSeiPayload;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPicParamsH264 {
|
||||
pub display_poc_syntax: u32,
|
||||
pub rsvd3: u32,
|
||||
pub ref_pic_flag: u32,
|
||||
pub color_plane_id: u32,
|
||||
pub force_intra_refresh_with_frame_count: u32,
|
||||
pub bitflags: u32,
|
||||
pub slice_type_data: *mut u8,
|
||||
pub slice_type_array_count: u32,
|
||||
pub sei_payload_array_count: u32,
|
||||
pub sei_payload_array: *mut NVencSeiPayload,
|
||||
pub slice_mode: u32,
|
||||
pub slice_mode_data: u32,
|
||||
pub ltr_mark_framce_idx: u32,
|
||||
pub ltr_use_frame_bitmap: u32,
|
||||
pub ltr_usage_mode: u32,
|
||||
pub force_intra_slice_count: u32,
|
||||
pub force_intra_slice_idx: *mut u32,
|
||||
pub h264_ext_pic_params: NVencPicParamsH264EXT,
|
||||
pub time_code: NVencTimeCode,
|
||||
pub rsvd: [u32; 202],
|
||||
pub rsvd2: [*mut c_void; 61],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPicParamsHEVC {
|
||||
pub display_poc_syntax: u32,
|
||||
pub ref_pic_flag: u32,
|
||||
pub temporal_id: u32,
|
||||
pub force_intra_refresh_with_frame_count: u32,
|
||||
pub bitflags: u32,
|
||||
pub rsvd1: u32,
|
||||
pub slice_type_data: *mut u8,
|
||||
pub slice_type_array_count: u32,
|
||||
pub slice_mode: u32,
|
||||
pub slice_mode_data: u32,
|
||||
pub ltr_mark_frame_idx: u32,
|
||||
pub ltr_usage_frame_bitmap: u32,
|
||||
pub ltr_usage_mode: u32,
|
||||
pub sei_payload_array_count: u32,
|
||||
pub rsvd: u32,
|
||||
pub sei_payload_array: *mut NVencSeiPayload,
|
||||
pub time_code: NVencTimeCode,
|
||||
pub num_temporal_layers: u32,
|
||||
pub view_id: u32,
|
||||
pub _3d_reference_display_info: *mut HEVC3DReferenceDisplayInfo,
|
||||
pub max_cll: *mut ContentLightLevel,
|
||||
pub mastering_display: *mut MasteringDisplayInfo,
|
||||
pub rsvd2: [u32; 234],
|
||||
pub rsvd3: [*mut c_void; 58],
|
||||
}
|
||||
|
||||
pub type NVencAV1OBUPayload = NVencSeiPayload;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPicParamsAV1 {
|
||||
display_poc_syntax: u32,
|
||||
ref_pic_flag: u32,
|
||||
temporal_id: u32,
|
||||
force_intra_refresh_with_frame_count: u32,
|
||||
bitflags: u32,
|
||||
num_tile_columns: u32,
|
||||
num_tile_rows: u32,
|
||||
rsvd: u32,
|
||||
tile_widths: *mut u32,
|
||||
tile_heights: *mut u32,
|
||||
obu_payload_array_count: u32,
|
||||
rsvd1: u32,
|
||||
obu_payload_array: *mut NVencAV1OBUPayload,
|
||||
file_grain_params: *mut NVencFilmGrainParamsAV1,
|
||||
ltr_mark_frame_idx: u32,
|
||||
ltr_use_frame_bitmap: u32,
|
||||
num_temporal_layers: u32,
|
||||
rsvd4: u32,
|
||||
max_cll: *mut ContentLightLevel,
|
||||
mastering_display: *mut MasteringDisplayInfo,
|
||||
rsvd2: [u32; 242],
|
||||
rsvd3: [*mut c_void; 59],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub union NVencCodecPicParams {
|
||||
pub h264_pic_params: ManuallyDrop<NVencPicParamsH264>,
|
||||
pub hevc_pic_params: ManuallyDrop<NVencPicParamsHEVC>,
|
||||
av1_pic_params: ManuallyDrop<NVencPicParamsAV1>,
|
||||
pub rsvd: [u32; 256],
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencPicParams {
|
||||
pub version: u32,
|
||||
pub input_width: u32,
|
||||
pub input_height: u32,
|
||||
pub input_pitch: u32,
|
||||
pub encode_pic_flags: u32,
|
||||
pub frame_idx: u32,
|
||||
pub input_time_stamp: u64,
|
||||
input_duration: u64,
|
||||
pub input_buffer: *mut c_void,
|
||||
pub output_bitstream: *mut c_void,
|
||||
// TODO: Windows only, needed for async
|
||||
completion_event: *mut c_void,
|
||||
pub buffer_format: NVencBufferFormat,
|
||||
pub picture_struct: NVencPicStruct,
|
||||
pub picture_type: NVencPicType,
|
||||
pub codec_pic_params: NVencCodecPicParams,
|
||||
me_hint_counts_per_block: [NVencExternalMeHintCountsPerBlockType; 2],
|
||||
me_external_hints: *mut NVencExternalMeHint,
|
||||
rsvd2: [u32; 7],
|
||||
rsvd5: [*mut c_void; 2],
|
||||
qp_delta_map: *mut i8,
|
||||
qp_deltra_map_size: u32,
|
||||
rsvd_bit_fields: u32,
|
||||
me_hint_ref_pic_dist: [u16; 2],
|
||||
rsvd4: u32,
|
||||
alpha_buffer: *mut c_void,
|
||||
me_external_sb_hints: *mut NVencExternalMeSbHint,
|
||||
me_sb_hints_count: u32,
|
||||
state_buffer_idx: u32,
|
||||
output_recon_buffer: *mut c_void,
|
||||
rsvd3: [u32; 284],
|
||||
rsvd6: [*mut c_void; 57],
|
||||
}
|
||||
|
||||
pub const NV_ENC_PIC_PARAMS_VER: u32 = struct_version(7) | (1 << 31);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencMeOnlyParams {
|
||||
version: u32,
|
||||
input_width: u32,
|
||||
input_height: u32,
|
||||
rsvd: u32,
|
||||
input_buffer: *mut c_void,
|
||||
reference_frame: *mut c_void,
|
||||
mv_buffer: *mut c_void,
|
||||
rsvd2: u32,
|
||||
buffer_formtat: NVencBufferFormat,
|
||||
view_id: u32,
|
||||
me_hint_counts_per_block: [NVencExternalMeHintCountsPerBlockType; 2],
|
||||
me_external_hints: NVencExternalMeHint,
|
||||
rsvd1: [u32; 241],
|
||||
rsvd3: [*mut c_void; 59],
|
||||
}
|
||||
|
||||
pub const NV_ENC_MEONLY_PARAMS_VER: u32 = struct_version(4);
|
||||
|
||||
#[bitfields::bitfield(u32)]
|
||||
pub struct NVencLockBitStreamBitFields {
|
||||
#[bits(1)]
|
||||
pub do_not_wait: bool,
|
||||
#[bits(1)]
|
||||
pub ltr_frame: bool,
|
||||
#[bits(1)]
|
||||
pub get_rc_stats: bool,
|
||||
#[bits(29)]
|
||||
rsvd: u32,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencLockBitStream {
|
||||
pub version: u32,
|
||||
pub bit_fields: NVencLockBitStreamBitFields,
|
||||
pub output_bit_stream: *mut c_void,
|
||||
slice_offsets: *mut u32,
|
||||
frame_idx: u32,
|
||||
hw_encode_status: u32,
|
||||
num_slices: u32,
|
||||
pub bitstream_size_in_bytes: u32,
|
||||
output_time_stamp: u64,
|
||||
pub output_duration: u64,
|
||||
pub bitstream_buffer: *mut c_void,
|
||||
picture_type: NVencPicType,
|
||||
picture_struct: NVencPicStruct,
|
||||
framge_avg_qp: u32,
|
||||
frame_satd: u32,
|
||||
ltr_frame_idx: u32,
|
||||
ltr_frame_bitmap: u32,
|
||||
temporal_id: u32,
|
||||
inter_mb_count: u32,
|
||||
average_mvx: i32,
|
||||
average_mvy: i32,
|
||||
alpha_layer_size_in_bytes: u32,
|
||||
output_stats_ptr_size: u32,
|
||||
rsvd: u32,
|
||||
output_stats_ptr: *mut c_void,
|
||||
frame_idx_display: u32,
|
||||
rsvd1: [u32; 219],
|
||||
rsvd2: [*mut c_void; 63],
|
||||
rsvd_interanl: [u32; 8],
|
||||
}
|
||||
|
||||
pub const NV_ENC_LOCK_BITSTREAM_VER: u32 = struct_version(2) | (1 << 31);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencLockInputBuffer {
|
||||
pub version: u32,
|
||||
pub bitflags: u32,
|
||||
pub input_buffer: *mut c_void,
|
||||
pub buffer_data_ptr: *mut c_void,
|
||||
pub pitch: u32,
|
||||
pub rsvd1: [u32; 251],
|
||||
pub rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_LOCK_INPUT_BUFFER_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencMapInputResource {
|
||||
pub version: u32,
|
||||
sub_resource_idx: u32,
|
||||
input_resource: *mut c_void,
|
||||
pub registered_resource: *mut c_void,
|
||||
pub mapped_resource: *mut c_void,
|
||||
pub mapped_buffer_format: NVencBufferFormat,
|
||||
rsvd1: [u32; 251],
|
||||
rsvd: [*mut c_void; 63],
|
||||
}
|
||||
|
||||
pub const NV_ENC_MAP_INPUT_RESOURCE_VER: u32 = struct_version(4);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencInputResourceOpenGLTex {
|
||||
texture: u32,
|
||||
target: u32,
|
||||
}
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencFencePointD3D12 {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
p_fence: *mut c_void,
|
||||
wait_value: u64,
|
||||
signal_value: u64,
|
||||
bit_flags: u32,
|
||||
rsvd1: [u32; 7],
|
||||
}
|
||||
|
||||
pub const NV_ENC_FENCE_POINT_D3D12_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencInputResourceD3D12 {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
input_buffer: *mut c_void,
|
||||
input_fence_point: NVencFencePointD3D12,
|
||||
rsvd1: [u32; 16],
|
||||
rsvd2: [*mut c_void; 16],
|
||||
}
|
||||
|
||||
pub const NV_ENC_INPUT_RESOURCE_D3D12_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencOutputResourceD3D12 {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
output_buffer: *mut c_void,
|
||||
output_fence_point: NVencFencePointD3D12,
|
||||
rsvd1: [u32; 16],
|
||||
rsvd2: [*mut c_void; 16],
|
||||
}
|
||||
|
||||
pub const NV_ENC_OUTPUT_RESOURCE_D3D12_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencRegisterResource {
|
||||
pub version: u32,
|
||||
pub resource_type: NVencInputResourceType,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub pitch: u32,
|
||||
pub sub_resource_index: u32,
|
||||
pub resource_to_register: *mut c_void,
|
||||
pub registered_resource: *mut c_void,
|
||||
pub buffer_format: NVencBufferFormat,
|
||||
pub buffer_usage: NVencBufferUsage,
|
||||
pub input_fence_point: *mut NVencFencePointD3D12,
|
||||
pub chroma_offset: [u32; 2],
|
||||
pub chroma_offset_in: [u32; 2],
|
||||
pub(crate) rsvd1: [u32; 244],
|
||||
pub(crate) rsvd2: [*mut c_void; 61],
|
||||
}
|
||||
|
||||
pub const NV_ENC_REGISTER_RESOURCE_VER: u32 = struct_version(5);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencStat {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
output_bit_stream: *mut c_void,
|
||||
bit_stream_size: u32,
|
||||
pic_type: u32,
|
||||
last_valid_byte_offset: u32,
|
||||
slice_offsets: [u32; 16],
|
||||
pic_idx: u32,
|
||||
frame_avg_qp: u32,
|
||||
bit_flags: u32,
|
||||
ltr_frame_idx: u32,
|
||||
intra_mb_count: u32,
|
||||
inter_mb_count: u32,
|
||||
average_mv_x: i32,
|
||||
average_mv_y: i32,
|
||||
rsvd1: [u32; 227],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_STAT_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencSequenceParamPayload {
|
||||
version: u32,
|
||||
in_buffer_size: u32,
|
||||
sps_id: u32,
|
||||
pps_id: u32,
|
||||
spspps_buffer: *mut c_void,
|
||||
output_spspps_payload_size: *mut u32,
|
||||
rsvd: [u32; 250],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_SEQUENCE_PARAM_PAYLOAD_VER: u32 = struct_version(1);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencEventParams {
|
||||
version: u32,
|
||||
rsvd: u32,
|
||||
completion_event: *mut c_void,
|
||||
rsvd1: [u32; 254],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_EVENT_PARAMS_VER: u32 = struct_version(2);
|
||||
|
||||
#[repr(C)]
|
||||
pub struct NVencOpenEncodeSessionExParams {
|
||||
pub version: u32,
|
||||
pub device_type: NVencDeviceType,
|
||||
pub device: *mut c_void,
|
||||
rsvd: *mut c_void,
|
||||
pub api_version: u32,
|
||||
rsvd1: [u32; 253],
|
||||
rsvd2: [*mut c_void; 64],
|
||||
}
|
||||
|
||||
pub const NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER: u32 = struct_version(1);
|
||||
Vendored
-12
@@ -1,12 +0,0 @@
|
||||
//! Contains the nvenc version numbers, encoded as they are in C, for the version this crate was built against
|
||||
|
||||
pub const NVENC_MAJOR_VERSION: u16 = 13;
|
||||
pub const NVENC_MINOR_VERSION: u8 = 0;
|
||||
|
||||
pub const NVENC_API_VERSION: u32 = NVENC_MAJOR_VERSION as u32 | (NVENC_MINOR_VERSION as u32) << 24;
|
||||
|
||||
pub const NVENC_API_LIST_VERSION: u32 = struct_version(2);
|
||||
|
||||
pub const fn struct_version(nvenc_api_version: u8) -> u32 {
|
||||
NVENC_API_VERSION | (nvenc_api_version as u32) << 16 | 0x7 << 28
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
[package]
|
||||
name = "viewer-core"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
# 阶段 1 先做纯 Rust 库并在桌面 headless 验证；
|
||||
# UniFFI / cdylib 在下一步（Android 接入）再加，隔离未知项。
|
||||
[lib]
|
||||
name = "viewer_core"
|
||||
crate-type = ["lib"]
|
||||
|
||||
[dependencies]
|
||||
common = { path = "../common" }
|
||||
anyhow.workspace = true
|
||||
tokio.workspace = true
|
||||
quinn.workspace = true
|
||||
rustls.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
# headless_check 用软解交叉验证收到的码流合法（库本身刻意不依赖 openh264，
|
||||
# 解码在 Android 侧由 MediaCodec 承担）
|
||||
openh264 = { version = "0.9.9", features = ["source"] }
|
||||
@@ -1,149 +0,0 @@
|
||||
//! viewer-core 的桌面 headless 验证工具（阶段 1 验收用，不依赖 Android）。
|
||||
//!
|
||||
//! 证明三件事，逐条打印：
|
||||
//! 1. viewer-core 能完成 QUIC 连接 + 协议 v2 认证握手；
|
||||
//! 2. 帧流按契约交付（seq 单调递增、keyframe 标志与码流实际 NAL 一致）；
|
||||
//! 3. 收到的码流能被 openh264 解出画面（即：Android 侧 MediaCodec 将拿到同样
|
||||
//! 合法的 Annex-B——解码器不同，但码流合法性在这里就证明了）。
|
||||
//!
|
||||
//! 用法：
|
||||
//! 终端1: target\release\agent.exe --port 47900 --code t
|
||||
//! 终端2: cargo run -p viewer-core --bin headless_check -- --port 47900 --code t --frames 60
|
||||
|
||||
use anyhow::Result;
|
||||
use std::time::Duration;
|
||||
use viewer_core::{ConnectConfig, Session};
|
||||
|
||||
fn arg(name: &str) -> Option<String> {
|
||||
let a: Vec<String> = std::env::args().collect();
|
||||
a.iter()
|
||||
.position(|x| x == name)
|
||||
.and_then(|i| a.get(i + 1))
|
||||
.cloned()
|
||||
}
|
||||
|
||||
/// 与 dump_stream 同款：扫 Annex-B 起始码拿 NAL 类型，交叉校验 keyframe 标志
|
||||
fn has_idr(payload: &[u8]) -> bool {
|
||||
let mut i = 0usize;
|
||||
while i + 3 < payload.len() {
|
||||
let sc = if payload[i..].starts_with(&[0, 0, 0, 1]) {
|
||||
4
|
||||
} else if payload[i..].starts_with(&[0, 0, 1]) {
|
||||
3
|
||||
} else {
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
match payload.get(i + sc) {
|
||||
Some(&b) if b & 0x1F == 5 => return true,
|
||||
Some(_) => i += sc + 1,
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let port: u16 = arg("--port")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(47800);
|
||||
let code = arg("--code").unwrap_or_else(|| "123456".into());
|
||||
// --host：默认 127.0.0.1；填局域网 IP 可顺带验证防火墙入站路径
|
||||
let host = arg("--host").unwrap_or_else(|| "127.0.0.1".into());
|
||||
let frames_wanted: u64 = arg("--frames")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(60);
|
||||
|
||||
let mut session = Session::connect(ConnectConfig::direct(&host, port, code)).await?;
|
||||
println!("[check] 连接成功 host={host} peer={}", session.peer_addr());
|
||||
|
||||
let mut decoder = openh264::decoder::Decoder::new()?;
|
||||
let mut last_seq: Option<u64> = None;
|
||||
let mut got = 0u64;
|
||||
let mut seq_violations = 0u64;
|
||||
let mut kf_mismatches = 0u64;
|
||||
let mut decoded = 0u64;
|
||||
let mut decode_failed_before_idr = 0u64;
|
||||
let mut awaiting_idr = true; // 与 Android 侧相同的状态机
|
||||
let mut rtt_shown = false;
|
||||
|
||||
let deadline = std::time::Instant::now() + Duration::from_secs(30);
|
||||
while got < frames_wanted && std::time::Instant::now() < deadline {
|
||||
let Some(f) = tokio::time::timeout(Duration::from_secs(10), session.next_frame())
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
else {
|
||||
break;
|
||||
};
|
||||
got += 1;
|
||||
|
||||
// 契约 2a：seq 单调递增
|
||||
if let Some(prev) = last_seq {
|
||||
if f.seq <= prev {
|
||||
seq_violations += 1;
|
||||
println!("[check] ⚠️ seq 回退: {prev} -> {}", f.seq);
|
||||
}
|
||||
}
|
||||
last_seq = Some(f.seq);
|
||||
|
||||
// 契约 2b：keyframe 标志与码流一致
|
||||
let real_kf = has_idr(&f.payload);
|
||||
if f.keyframe != real_kf {
|
||||
kf_mismatches += 1;
|
||||
println!(
|
||||
"[check] ⚠️ 帧头 keyframe={} 但码流实际 IDR={}（seq={}）",
|
||||
f.keyframe, real_kf, f.seq
|
||||
);
|
||||
}
|
||||
|
||||
// 契约 3：与 Android 相同的状态机下解码必须成功
|
||||
if awaiting_idr && !f.keyframe {
|
||||
continue; // 等 IDR，P 帧不喂（这是 R-3 教训的实现）
|
||||
}
|
||||
if f.keyframe {
|
||||
awaiting_idr = false;
|
||||
}
|
||||
match decoder.decode(&f.payload) {
|
||||
Ok(_) => decoded += 1,
|
||||
Err(e) => {
|
||||
decode_failed_before_idr += 1;
|
||||
println!("[check] ❌ seq={} 解码失败: {e}", f.seq);
|
||||
awaiting_idr = true;
|
||||
session.request_keyframe();
|
||||
}
|
||||
}
|
||||
|
||||
// RTT 出现即记录一次
|
||||
if !rtt_shown {
|
||||
let s = session.stats();
|
||||
if s.rtt_ms > 0 {
|
||||
println!("[check] RTT={}ms frames_in={} bytes_in={}", s.rtt_ms, s.frames_in, s.bytes_in);
|
||||
rtt_shown = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let s = session.stats();
|
||||
println!("\n========== 阶段 1 headless 验收 ==========");
|
||||
println!("收到帧: {got}（目标 {frames_wanted}）");
|
||||
println!("seq 违例: {seq_violations}");
|
||||
println!("keyframe 标志不符: {kf_mismatches}");
|
||||
println!("解码成功: {decoded} / 解码失败: {decode_failed_before_idr}");
|
||||
println!("统计: rtt={}ms frames_in={} bytes_in={}", s.rtt_ms, s.frames_in, s.bytes_in);
|
||||
|
||||
let ok = got >= frames_wanted.min(30)
|
||||
&& seq_violations == 0
|
||||
&& kf_mismatches == 0
|
||||
&& decoded > 0
|
||||
&& decode_failed_before_idr == 0;
|
||||
println!(
|
||||
"结论: {}",
|
||||
if ok { "✅ viewer-core 收帧链路通过" } else { "❌ 存在未满足项" }
|
||||
);
|
||||
session.close().await;
|
||||
Ok(())
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
//! 协议 v3 文本注入链路的桌面真边界验证工具（loop-2b，不依赖 Android）。
|
||||
//!
|
||||
//! 证明的事：`Session::send_text` 经线格式到被控端后，`agent` 的
|
||||
//! `KEYEVENTF_UNICODE` 注入把**同一段 UTF-16 code unit** 送进了前台应用。
|
||||
//! 判据由外部读取目标窗口的实际文本内容（调用方负责），本工具只负责发送。
|
||||
//!
|
||||
//! 用法：
|
||||
//! 终端1: target\release\agent.exe --port 47800 --code t
|
||||
//! 终端2: 把目标应用（记事本）切到前台，然后
|
||||
//! cargo run -p viewer-core --example text_check -- \
|
||||
//! --port 47800 --code t --units 20320,22909,55357,56832
|
||||
//!
|
||||
//! `--units` 是十进制的 UTF-16 code unit 列表（逗号分隔）：
|
||||
//! 「你好😀」= 20320,22909,55357,56832（😀 是代理对）。
|
||||
//! 刻意不收自然语言字符串参数：命令行到 Rust 的编码往返会引入与本轮无关的变量，
|
||||
//! 由调用方（PowerShell）从 .NET 字符串取 char[] 更忠实。
|
||||
//!
|
||||
//! `--delay`：连接后等待毫秒再注入。给调用方把目标窗口切到前台的时间——
|
||||
//! SendInput 注入的是**前台**应用，窗口没在前台等于测了个寂寞。
|
||||
//! `--hold`：注入后保持会话的毫秒数，等消息在远端消化完再关连接。
|
||||
|
||||
use anyhow::{bail, Result};
|
||||
use std::time::Duration;
|
||||
use viewer_core::{ConnectConfig, Session};
|
||||
|
||||
fn arg(name: &str) -> Option<String> {
|
||||
let a: Vec<String> = std::env::args().collect();
|
||||
a.iter()
|
||||
.position(|x| x == name)
|
||||
.and_then(|i| a.get(i + 1))
|
||||
.cloned()
|
||||
}
|
||||
|
||||
fn parse_units(spec: &str) -> Result<Vec<u16>> {
|
||||
if spec.trim().is_empty() {
|
||||
bail!(
|
||||
"--units 缺失或为空：本工具要验证的是「有文本送出并被注入」，\
|
||||
空输入没有验证价值（空 Text 在产品路径上由 Session::send_text 拦掉）"
|
||||
);
|
||||
}
|
||||
spec.split(',')
|
||||
.map(|s| s.trim())
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|s| {
|
||||
s.parse::<u32>()
|
||||
.ok()
|
||||
.and_then(|v| u16::try_from(v).ok())
|
||||
.ok_or_else(|| anyhow::anyhow!("--units 必须是 0..=65535 的十进制数，got {s:?}"))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let port: u16 = arg("--port")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(47800);
|
||||
let code = arg("--code").unwrap_or_else(|| "123456".into());
|
||||
let host = arg("--host").unwrap_or_else(|| "127.0.0.1".into());
|
||||
let delay = arg("--delay")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(1500u64);
|
||||
let hold = arg("--hold")
|
||||
.map(|s| s.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(800u64);
|
||||
let units = parse_units(&arg("--units").unwrap_or_default())?;
|
||||
|
||||
let session = Session::connect(ConnectConfig::direct(&host, port, &code)).await?;
|
||||
println!(
|
||||
"[text_check] connected peer={} units={}",
|
||||
session.peer_addr(),
|
||||
units.len()
|
||||
);
|
||||
|
||||
tokio::time::sleep(Duration::from_millis(delay)).await;
|
||||
session.send_text(&units);
|
||||
println!("[text_check] sent (只报长度，不打印内容)");
|
||||
|
||||
tokio::time::sleep(Duration::from_millis(hold)).await;
|
||||
let st = session.stats();
|
||||
println!(
|
||||
"[text_check] frames={} rtt={}ms — 会话活着，注入已发给前台应用",
|
||||
st.frames_in, st.rtt_ms
|
||||
);
|
||||
session.close().await;
|
||||
Ok(())
|
||||
}
|
||||
@@ -1,454 +0,0 @@
|
||||
//! viewer-core：安卓主控的连接 / 协议 / 收帧层（纯 Rust，平台无关）。
|
||||
//!
|
||||
//! 设计边界（对齐《多平台架构设计》与 2026-10-29 方案裁决）：
|
||||
//! - **只做「怎么连 + 收发什么」**：QUIC 会话、认证握手、视频帧流、输入上行、
|
||||
//! Ping/Pong、KeyFrameReq。解码（MediaCodec）、渲染（SurfaceView）、触摸映射
|
||||
//! 全在 Kotlin 侧，本 crate 不碰像素，也不引入 openh264。
|
||||
//! - **连接层是唯一的可替换点**：`PeerTarget::Direct` 现在只支持 `IP:端口` 局域网
|
||||
//! 直连；将来设备码 + 信令 + 打洞只换 `connect()` 内部。**不定义 Relay 变体**
|
||||
//! （中继已被团队裁决排除，见《多平台架构设计》§6.1）。
|
||||
//! - 帧经 mpsc 通道交付，**不丢帧**：消费方（Kotlin）按 `keyframe` 标志自行决定
|
||||
//! 跳帧策略，且跳帧后必须 `request_keyframe()` 并等 IDR（协议 v2 提供该字段）。
|
||||
//! 丢帧决策放在这里会让 Rust 层承担它看不到的解码器状态。
|
||||
//!
|
||||
//! 安全现状：`insecure_skip_verify` 默认 true（与桌面 viewer 的 SkipVerify 一致，
|
||||
//! 仅限局域网测试）。产品化前必须换成信令下发的证书指纹钉扎（`signaling.rs`
|
||||
//! 已有 `CertFingerprint`），该项登记在方案阶段 4。
|
||||
|
||||
use anyhow::{bail, Context, Result};
|
||||
use common::{
|
||||
read_control, read_video_frame, write_control, ControlMsg, InputEvent, VideoFrameHeader,
|
||||
PROTO_VERSION,
|
||||
};
|
||||
use rustls::client::danger::{
|
||||
HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier,
|
||||
};
|
||||
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
|
||||
use rustls::{DigitallySignedStruct, SignatureScheme};
|
||||
use std::net::ToSocketAddrs;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
/// 连接目标。当前只有局域网直连；设备码 + 打洞在后续阶段替换 `connect()` 内部实现。
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum PeerTarget {
|
||||
/// 局域网 / 可直连地址
|
||||
Direct { host: String, port: u16 },
|
||||
}
|
||||
|
||||
/// 一次会话的参数
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ConnectConfig {
|
||||
pub target: PeerTarget,
|
||||
/// 被控端验证码
|
||||
pub code: String,
|
||||
/// 仅测试用：跳过服务端证书校验（与桌面 viewer 行为一致）。
|
||||
/// 公网/产品形态必须为 false 并启用证书钉扎。
|
||||
pub insecure_skip_verify: bool,
|
||||
}
|
||||
|
||||
impl ConnectConfig {
|
||||
/// 局域网测试的便捷构造
|
||||
pub fn direct(host: impl Into<String>, port: u16, code: impl Into<String>) -> Self {
|
||||
Self {
|
||||
target: PeerTarget::Direct {
|
||||
host: host.into(),
|
||||
port,
|
||||
},
|
||||
code: code.into(),
|
||||
insecure_skip_verify: true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 交付给消费方的一帧视频。字段语义与 `common::VideoFrameHeader` 一致，
|
||||
/// payload 为标准 Annex-B H.264（可直接喂 MediaCodec）。
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Frame {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub ts_ms: u64,
|
||||
pub seq: u64,
|
||||
pub keyframe: bool,
|
||||
pub payload: Vec<u8>,
|
||||
}
|
||||
|
||||
impl From<(VideoFrameHeader, Vec<u8>)> for Frame {
|
||||
fn from((h, payload): (VideoFrameHeader, Vec<u8>)) -> Self {
|
||||
Self {
|
||||
width: h.width,
|
||||
height: h.height,
|
||||
ts_ms: h.ts_ms,
|
||||
seq: h.seq,
|
||||
keyframe: h.keyframe,
|
||||
payload,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 会话状态快照（供 UI 显示，不做 EMA——那属于 Kotlin 的 HUD 层）
|
||||
#[derive(Debug, Clone, Copy, Default)]
|
||||
pub struct SessionStats {
|
||||
/// 最近一次 Ping 往返（ms），0 = 还没测出
|
||||
pub rtt_ms: u64,
|
||||
/// 收到的帧数
|
||||
pub frames_in: u64,
|
||||
/// 收到的字节数（视频载荷累计）
|
||||
pub bytes_in: u64,
|
||||
}
|
||||
|
||||
/// MVP：跳过服务端证书校验（自签证书，仅局域网）。
|
||||
#[derive(Debug)]
|
||||
struct SkipVerify;
|
||||
impl ServerCertVerifier for SkipVerify {
|
||||
fn verify_server_cert(
|
||||
&self,
|
||||
_end_entity: &CertificateDer<'_>,
|
||||
_intermediates: &[CertificateDer<'_>],
|
||||
_server_name: &ServerName<'_>,
|
||||
_ocsp_response: &[u8],
|
||||
_now: UnixTime,
|
||||
) -> std::result::Result<ServerCertVerified, rustls::Error> {
|
||||
Ok(ServerCertVerified::assertion())
|
||||
}
|
||||
fn verify_tls12_signature(
|
||||
&self,
|
||||
_message: &[u8],
|
||||
_certificate: &CertificateDer<'_>,
|
||||
_dss: &DigitallySignedStruct,
|
||||
) -> std::result::Result<HandshakeSignatureValid, rustls::Error> {
|
||||
Ok(HandshakeSignatureValid::assertion())
|
||||
}
|
||||
fn verify_tls13_signature(
|
||||
&self,
|
||||
_message: &[u8],
|
||||
_certificate: &CertificateDer<'_>,
|
||||
_dss: &DigitallySignedStruct,
|
||||
) -> std::result::Result<HandshakeSignatureValid, rustls::Error> {
|
||||
Ok(HandshakeSignatureValid::assertion())
|
||||
}
|
||||
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
|
||||
vec![
|
||||
SignatureScheme::RSA_PKCS1_SHA256,
|
||||
SignatureScheme::ECDSA_NISTP256_SHA256,
|
||||
SignatureScheme::ED25519,
|
||||
SignatureScheme::RSA_PSS_SHA256,
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
/// 输入/控制命令（写入流 2）
|
||||
enum Command {
|
||||
Input(InputEvent),
|
||||
KeyFrameReq,
|
||||
Ping(u64),
|
||||
}
|
||||
|
||||
/// 已建立的会话。Drop 时后台任务随连接关闭自然退出；显式 `close()` 更干净。
|
||||
pub struct Session {
|
||||
tx: mpsc::UnboundedSender<Command>,
|
||||
/// 帧接收端。通道关闭 = 会话结束（被控端断开或流错误）。
|
||||
/// Mutex 包一层是为了让 `next_frame` 取 `&self`：JNI 层里解码线程独占 recv，
|
||||
/// 输入线程并发调 `send_input`（走无锁的 unbounded tx），两者互不阻塞。
|
||||
frames: Arc<tokio::sync::Mutex<mpsc::Receiver<Frame>>>,
|
||||
stats: Arc<std::sync::Mutex<SessionStats>>,
|
||||
/// 被控端桌面尺寸（AuthOk 携带）。主控端据此在解码前定好视图宽高比——
|
||||
/// 若等首帧再改尺寸会触发 SurfaceView 重新测量，把刚创建的解码器 surface 打掉。
|
||||
desktop: (u32, u32),
|
||||
/// 保持 Endpoint 存活（drop 会销毁连接）
|
||||
_endpoint: quinn::Endpoint,
|
||||
conn: quinn::Connection,
|
||||
}
|
||||
|
||||
impl Session {
|
||||
/// 建立连接并完成认证握手 + 首帧关键帧请求。
|
||||
pub async fn connect(cfg: ConnectConfig) -> Result<Self> {
|
||||
rustls::crypto::ring::default_provider()
|
||||
.install_default()
|
||||
.ok();
|
||||
|
||||
let tls = if cfg.insecure_skip_verify {
|
||||
rustls::ClientConfig::builder()
|
||||
.dangerous()
|
||||
.with_custom_certificate_verifier(Arc::new(SkipVerify))
|
||||
.with_no_client_auth()
|
||||
} else {
|
||||
// 证书钉扎尚未接入（方案阶段 4）。这里明确报错而不是静默放行。
|
||||
bail!("insecure_skip_verify=false 需要证书钉扎，尚未实现（见方案阶段 4）");
|
||||
};
|
||||
let qc: quinn::crypto::rustls::QuicClientConfig = Arc::new(tls)
|
||||
.try_into()
|
||||
.context("build QUIC client config")?;
|
||||
|
||||
// 接收窗口放大：视频下行大流量，quinn 默认 512KB 流窗口会卡吞吐
|
||||
// （与桌面 viewer / agent 的实测调优一致）
|
||||
let mut transport = quinn::TransportConfig::default();
|
||||
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||
let mut client_config = quinn::ClientConfig::new(Arc::new(qc));
|
||||
client_config.transport_config(Arc::new(transport));
|
||||
|
||||
let mut endpoint = quinn::Endpoint::client("0.0.0.0:0".parse()?)?;
|
||||
endpoint.set_default_client_config(client_config);
|
||||
|
||||
let PeerTarget::Direct { host, port } = &cfg.target;
|
||||
let addr = (host.as_str(), *port)
|
||||
.to_socket_addrs()?
|
||||
.next()
|
||||
.with_context(|| format!("resolve {host}:{port}"))?;
|
||||
let conn = endpoint
|
||||
.connect(addr, "localhost")?
|
||||
.await
|
||||
.with_context(|| format!("QUIC connect {addr}"))?;
|
||||
|
||||
// ---- 流 1：认证 + 视频下行 ----
|
||||
let (mut v_send, mut v_recv) = conn.open_bi().await?;
|
||||
write_control(
|
||||
&mut v_send,
|
||||
&ControlMsg::Auth {
|
||||
code: cfg.code.clone(),
|
||||
proto_version: PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.context("send Auth")?;
|
||||
let desktop: (u32, u32);
|
||||
match read_control(&mut v_recv).await.context("read AuthOk")? {
|
||||
ControlMsg::AuthOk {
|
||||
width,
|
||||
height,
|
||||
proto_version,
|
||||
} => {
|
||||
if proto_version != PROTO_VERSION {
|
||||
bail!(
|
||||
"protocol mismatch: agent={proto_version}, viewer-core={PROTO_VERSION}"
|
||||
);
|
||||
}
|
||||
desktop = (width, height);
|
||||
log(&format!("auth ok, remote desktop {width}x{height}"));
|
||||
}
|
||||
ControlMsg::AuthFail(e) => bail!("auth failed: {e}"),
|
||||
m => bail!("unexpected first message: {m:?}"),
|
||||
}
|
||||
|
||||
// ---- 流 2：输入 + 控制 + Ping/Pong ----
|
||||
let (mut ctl_send, mut ctl_recv) = conn.open_bi().await?;
|
||||
// 入会即请求 IDR：从 GOP 中间接入时裸 P 帧解不出来（桌面端同款教训）
|
||||
write_control(&mut ctl_send, &ControlMsg::KeyFrameReq).await?;
|
||||
|
||||
let stats = Arc::new(std::sync::Mutex::new(SessionStats::default()));
|
||||
|
||||
// 帧通道：容量 2 足够——消费方（Kotlin）每帧都要喂解码器，
|
||||
// 积压超过 2 帧说明该跳帧的是消费方的决策，不是这里的。
|
||||
let (frame_tx, frames) = mpsc::channel::<Frame>(2);
|
||||
{
|
||||
let stats = stats.clone();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
match read_video_frame(&mut v_recv).await {
|
||||
Ok((h, payload)) => {
|
||||
{
|
||||
let mut s = stats.lock().unwrap();
|
||||
s.frames_in += 1;
|
||||
s.bytes_in += payload.len() as u64;
|
||||
}
|
||||
let frame: Frame = (h, payload).into();
|
||||
if frame_tx.send(frame).await.is_err() {
|
||||
break; // 消费方已退出
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
log(&format!("video stream ended: {e}"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 命令通道 → 流 2 写任务
|
||||
let (cmd_tx, mut cmd_rx) = mpsc::unbounded_channel::<Command>();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Some(cmd) = cmd_rx.recv().await else {
|
||||
break;
|
||||
};
|
||||
let msg = match cmd {
|
||||
Command::Input(ev) => ControlMsg::Input(ev),
|
||||
Command::KeyFrameReq => ControlMsg::KeyFrameReq,
|
||||
Command::Ping(t) => ControlMsg::Ping(t),
|
||||
};
|
||||
if write_control(&mut ctl_send, &msg).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// 流 2 读任务：Pong → RTT
|
||||
{
|
||||
let stats = stats.clone();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
match read_control(&mut ctl_recv).await {
|
||||
Ok(ControlMsg::Pong { t, .. }) => {
|
||||
let rtt = monotonic_ms().saturating_sub(t);
|
||||
stats.lock().unwrap().rtt_ms = rtt;
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(e) => {
|
||||
log(&format!("control stream ended: {e}"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Ping 任务：500ms 一次（与桌面 viewer 一致）
|
||||
{
|
||||
let cmd_tx_ping = cmd_tx.clone();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
if cmd_tx_ping.send(Command::Ping(monotonic_ms())).is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
Ok(Self {
|
||||
tx: cmd_tx,
|
||||
frames: Arc::new(tokio::sync::Mutex::new(frames)),
|
||||
stats,
|
||||
desktop,
|
||||
_endpoint: endpoint,
|
||||
conn,
|
||||
})
|
||||
}
|
||||
|
||||
/// 被控端桌面尺寸 `(width, height)`，来自 AuthOk，解码前即可用。
|
||||
pub fn desktop_size(&self) -> (u32, u32) {
|
||||
self.desktop
|
||||
}
|
||||
|
||||
/// 取下一帧。`None` = 会话结束。
|
||||
/// `&self`：JNI 层里解码线程独占 recv，输入线程可并发 `send_input`。
|
||||
pub async fn next_frame(&self) -> Option<Frame> {
|
||||
self.frames.lock().await.recv().await
|
||||
}
|
||||
|
||||
/// 发送一个输入事件（非阻塞入队，由写任务串行发出）。
|
||||
pub fn send_input(&self, ev: InputEvent) {
|
||||
let _ = self.tx.send(Command::Input(ev));
|
||||
}
|
||||
|
||||
/// 协议 v3：发送一段文本（UTF-16 code unit）到被控端注入。
|
||||
///
|
||||
/// 为什么参数是 `&[u16]` 而不是 `&str`：被控端的注入通道（Windows
|
||||
/// `KEYEVENTF_UNICODE`）与主控端来源（Java 字符串）都是 UTF-16 code unit，
|
||||
/// 中间任何一次 UTF-8 往返都会把 IME 提交边界的落单代理项变成替换字符。
|
||||
/// 见 `common::InputEvent::Text` 的取舍说明。
|
||||
///
|
||||
/// 超限处理：截断到 [`common::MAX_TEXT_UNITS`] 并打日志——静默丢弃会让用户
|
||||
/// 以为长文本已送达。日志只报长度，不报内容。
|
||||
pub fn send_text(&self, units: &[u16]) {
|
||||
if units.is_empty() {
|
||||
return; // 空提交没有注入语义，也不该占用一条消息
|
||||
}
|
||||
let (units, truncated) = common::clamp_text_units(units.to_vec());
|
||||
if truncated {
|
||||
log(&format!(
|
||||
"文本超过上限，已截断到 {} 个 code unit",
|
||||
common::MAX_TEXT_UNITS
|
||||
));
|
||||
}
|
||||
self.send_input(InputEvent::Text { units });
|
||||
}
|
||||
|
||||
/// 请求被控端强制关键帧（解码器错误/跳帧后的恢复路径）。
|
||||
pub fn request_keyframe(&self) {
|
||||
let _ = self.tx.send(Command::KeyFrameReq);
|
||||
}
|
||||
|
||||
/// 当前会话统计快照。
|
||||
pub fn stats(&self) -> SessionStats {
|
||||
*self.stats.lock().unwrap()
|
||||
}
|
||||
|
||||
/// 被控端地址（日志/UI 用）
|
||||
pub fn peer_addr(&self) -> String {
|
||||
self.conn.remote_address().to_string()
|
||||
}
|
||||
|
||||
/// 主动结束会话。
|
||||
pub async fn close(self) {
|
||||
let _ = self.conn.close(0u32.into(), b"viewer closing");
|
||||
}
|
||||
}
|
||||
|
||||
/// 单调毫秒时钟（进程启动起算）。Ping/Pong 两端共用同一口径即可测 RTT。
|
||||
fn monotonic_ms() -> u64 {
|
||||
use std::sync::OnceLock;
|
||||
static T0: OnceLock<Instant> = OnceLock::new();
|
||||
T0.get_or_init(Instant::now).elapsed().as_millis() as u64
|
||||
}
|
||||
|
||||
/// 轻量日志出口。阶段 2 接 UniFFI 时替换为 Kotlin 侧回调。
|
||||
fn log(msg: &str) {
|
||||
eprintln!("[viewer-core] {msg}");
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn frame_from_header_preserves_contract_fields() {
|
||||
let h = VideoFrameHeader {
|
||||
width: 1920,
|
||||
height: 1080,
|
||||
ts_ms: 42,
|
||||
seq: 7,
|
||||
keyframe: true,
|
||||
};
|
||||
let payload = vec![0u8; 16];
|
||||
let f: Frame = (h.clone(), payload.clone()).into();
|
||||
assert_eq!((f.width, f.height), (1920, 1080));
|
||||
assert_eq!(f.seq, 7);
|
||||
assert!(f.keyframe);
|
||||
assert_eq!(f.payload, payload);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn direct_config_defaults_to_insecure_lan_mode() {
|
||||
let c = ConnectConfig::direct("192.168.1.5", 47800, "abc");
|
||||
assert!(c.insecure_skip_verify);
|
||||
match c.target {
|
||||
PeerTarget::Direct { host, port } => {
|
||||
assert_eq!(host, "192.168.1.5");
|
||||
assert_eq!(port, 47800);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn strict_verify_mode_is_explicit_unimplemented_error() {
|
||||
// 安全红线不能静默：显式要求校验时必须是明确错误，而不是悄悄放行。
|
||||
// 不用 unwrap_err()：它要求 Ok 类型（Session）实现 Debug，而连接对象
|
||||
// 的 Debug 输出没有验证价值。
|
||||
let cfg = ConnectConfig {
|
||||
target: PeerTarget::Direct {
|
||||
host: "127.0.0.1".into(),
|
||||
port: 1,
|
||||
},
|
||||
code: "x".into(),
|
||||
insecure_skip_verify: false,
|
||||
};
|
||||
match Session::connect(cfg).await {
|
||||
Err(e) => assert!(e.to_string().contains("证书钉扎"), "实际错误: {e}"),
|
||||
Ok(_) => panic!("insecure_skip_verify=false 必须明确报错，不得静默放行"),
|
||||
}
|
||||
}
|
||||
}
|
||||
+31
-290
@@ -81,15 +81,8 @@ fn now_ms() -> u64 {
|
||||
T0.get().unwrap().elapsed().as_millis() as u64
|
||||
}
|
||||
|
||||
/// 帧到达间隔环形缓冲（取 p95 用）
|
||||
struct GapRing {
|
||||
last: Option<Instant>,
|
||||
ring: [u32; 64],
|
||||
idx: usize,
|
||||
filled: usize,
|
||||
}
|
||||
|
||||
/// 会话统计（HUD 数据源，全部原子量，跨任务共享）
|
||||
#[derive(Default)]
|
||||
struct Stats {
|
||||
/// 最近 RTT ms（0=未知）
|
||||
rtt: AtomicU64,
|
||||
@@ -102,83 +95,6 @@ struct Stats {
|
||||
/// 累计收帧数 / 字节数
|
||||
frames: AtomicU64,
|
||||
bytes: AtomicU64,
|
||||
/// 抖动：相邻帧 |ΔE2E| 的 EMA（ms×16 定点存储避免浮点原子）
|
||||
jitter_x16: AtomicU64,
|
||||
prev_e2e: AtomicI64,
|
||||
/// agent 发送侧（视频下行）QUIC 丢包/发包累计（最近一次 Pong）
|
||||
agent_lost: AtomicU64,
|
||||
agent_sent: AtomicU64,
|
||||
/// H.264 解码失败计数（关键帧自愈验证指标）
|
||||
dec_fail: AtomicU64,
|
||||
/// 当前是否处于「参考链断裂、等待 IDR」状态（HUD 显示 RECOVER）
|
||||
awaiting_idr: AtomicBool,
|
||||
/// 帧到达间隔环形缓冲
|
||||
gaps: std::sync::Mutex<GapRing>,
|
||||
}
|
||||
|
||||
impl Default for Stats {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
rtt: AtomicU64::new(0),
|
||||
offset: AtomicI64::new(0),
|
||||
offset_valid: AtomicBool::new(false),
|
||||
e2e: AtomicU64::new(0),
|
||||
e2e_valid: AtomicBool::new(false),
|
||||
frames: AtomicU64::new(0),
|
||||
bytes: AtomicU64::new(0),
|
||||
jitter_x16: AtomicU64::new(0),
|
||||
prev_e2e: AtomicI64::new(-1),
|
||||
agent_lost: AtomicU64::new(0),
|
||||
agent_sent: AtomicU64::new(0),
|
||||
dec_fail: AtomicU64::new(0),
|
||||
awaiting_idr: AtomicBool::new(true),
|
||||
gaps: std::sync::Mutex::new(GapRing {
|
||||
last: None,
|
||||
ring: [0; 64],
|
||||
idx: 0,
|
||||
filled: 0,
|
||||
}),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Stats {
|
||||
/// 记录一次帧到达：更新间隔环形缓冲与抖动 EMA
|
||||
fn on_frame_arrival(&self, est: Option<i64>) {
|
||||
let now = Instant::now();
|
||||
if let Ok(mut g) = self.gaps.lock() {
|
||||
if let Some(t) = g.last {
|
||||
let gap = (now - t).as_millis() as u32;
|
||||
let i = g.idx;
|
||||
g.ring[i] = gap;
|
||||
g.idx = (i + 1) % g.ring.len();
|
||||
g.filled = (g.filled + 1).min(g.ring.len());
|
||||
}
|
||||
g.last = Some(now);
|
||||
}
|
||||
// 抖动 = |本帧E2E - 上帧E2E| 的 EMA（RFC3550 风格，÷16 衰减）
|
||||
if let Some(e) = est {
|
||||
let prev = self.prev_e2e.load(Ordering::Relaxed);
|
||||
self.prev_e2e.store(e, Ordering::Relaxed);
|
||||
if prev >= 0 {
|
||||
let d = (e - prev).unsigned_abs() as u64;
|
||||
let old = self.jitter_x16.load(Ordering::Relaxed);
|
||||
let sm = if old == 0 { d * 16 } else { old + d * 16 / 16 - old / 16 };
|
||||
self.jitter_x16.store(sm, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 帧间隔 p95（ms），样本不足返回 None
|
||||
fn gap_p95(&self) -> Option<u32> {
|
||||
let g = self.gaps.lock().ok()?;
|
||||
if g.filled < 8 {
|
||||
return None;
|
||||
}
|
||||
let mut v: Vec<u32> = g.ring[..g.filled].to_vec();
|
||||
v.sort_unstable();
|
||||
v.get(v.len() * 95 / 100).copied()
|
||||
}
|
||||
}
|
||||
|
||||
fn main() -> Result<()> {
|
||||
@@ -221,27 +137,9 @@ async fn run(addr: String, port: u16, code: String) -> Result<()> {
|
||||
|
||||
let (mut send, mut recv) = conn.open_bi().await?;
|
||||
|
||||
write_control(
|
||||
&mut send,
|
||||
&ControlMsg::Auth {
|
||||
code,
|
||||
proto_version: common::PROTO_VERSION,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
write_control(&mut send, &ControlMsg::Auth { code }).await?;
|
||||
match read_control(&mut recv).await? {
|
||||
ControlMsg::AuthOk {
|
||||
width,
|
||||
height,
|
||||
proto_version,
|
||||
} => {
|
||||
// 版本已在 Auth 里声明，这里再核对一次：防止被控端是旧版却"接受"了认证
|
||||
if proto_version != common::PROTO_VERSION {
|
||||
bail!(
|
||||
"protocol version mismatch: server={proto_version}, client={}",
|
||||
common::PROTO_VERSION
|
||||
);
|
||||
}
|
||||
ControlMsg::AuthOk { width, height } => {
|
||||
println!("[viewer] auth ok, remote desktop {width}x{height}");
|
||||
// 流 2：键鼠输入 + Ping/Pong 独立通道，不与视频流竞争
|
||||
let (isend, irecv) = conn.open_bi().await?;
|
||||
@@ -262,134 +160,48 @@ fn session(
|
||||
let (w, h) = (width as usize, height as usize);
|
||||
let stats = Arc::new(Stats::default());
|
||||
|
||||
// 读任务（流 1）：H.264 解码 → RGB → 单槽 latest-wins；同时统计端到端延迟与流量
|
||||
// 帧格式 (src_w, src_h, rgb)；(0,0,_) = 解码失败占位，触发 KeyFrameReq
|
||||
// 单槽而非无界队列：渲染跟不上 60fps 到达率时旧帧直接被覆盖，绝不积压
|
||||
//（无界 mpsc 在 NVENC 60fps 下会无限积压 → 延迟线性增长 + 渲染 drain 时
|
||||
// 长时间阻塞窗口消息泵 → 窗口假死、HUD 停刷新）
|
||||
let latest: Arc<std::sync::Mutex<Option<(u32, u32, Vec<u8>)>>> =
|
||||
Arc::new(std::sync::Mutex::new(None));
|
||||
// 读任务（流 1）：H.264 解码 → RGB → mpsc；同时统计端到端延迟与流量
|
||||
let (frame_tx, frame_rx) = std::sync::mpsc::channel::<Vec<u8>>();
|
||||
{
|
||||
let stats = stats.clone();
|
||||
let latest = latest.clone();
|
||||
let mut decoder = openh264::decoder::Decoder::new()
|
||||
.map_err(|e| anyhow::anyhow!("decoder init failed: {e}"))?;
|
||||
let mut rgb_buf: Vec<u8> = Vec::new();
|
||||
let mut last_kfreq = Instant::now();
|
||||
let mut skip_stale = 0u64;
|
||||
let mut last_arr: Option<Instant> = None;
|
||||
// 「等待 IDR」状态：参考链一旦断裂（跳帧或解码失败），后续 P 帧**必然**
|
||||
// 解不出来。此时必须停止喂解码器，只等 IDR 重新同步。
|
||||
//
|
||||
// 之前没有这个状态，后果是：跳掉一帧后，每个 P 帧都被送进 decoder →
|
||||
// 每个都失败 → 每个都打日志 → 每个都触发一次 KeyFrameReq。
|
||||
// 表现为「decode failed」刷屏，且解码器反复在错误状态里挣扎。
|
||||
//
|
||||
// 放在 Stats 里（而非局部变量）以便 HUD 显示 RECOVER 状态。
|
||||
// 初始 true：入会时已发 KeyFrameReq，拿到第一个 IDR 前不该硬解。
|
||||
tokio::spawn(async move {
|
||||
let mut size_logged = false;
|
||||
loop {
|
||||
match common::read_video_frame(&mut recv).await {
|
||||
Ok((hdr, payload)) => {
|
||||
stats.bytes.fetch_add(payload.len() as u64, Ordering::Relaxed);
|
||||
stats.frames.fetch_add(1, Ordering::Relaxed);
|
||||
// 与上一帧的到达间隔：≤100ms 才算"突发到达"。
|
||||
// 桌面静止时 agent 以 2Hz 空闲重发，帧龄天然 ≥500ms，
|
||||
// 不能按陈旧帧丢弃——否则静止桌面所有帧被跳过 →
|
||||
// 永久黑屏 + KeyFrameReq/IDR 死循环（v0.8 实测教训）
|
||||
let burst = matches!(last_arr, Some(t) if t.elapsed() <= Duration::from_millis(100));
|
||||
last_arr = Some(Instant::now());
|
||||
// 端到端延迟 = 接收时刻 - 时钟偏移 - 采集时刻（EMA 平滑）
|
||||
let mut est = None;
|
||||
let mut stale = false;
|
||||
if stats.offset_valid.load(Ordering::Relaxed) {
|
||||
let e =
|
||||
let est =
|
||||
now_ms() as i64 - stats.offset.load(Ordering::Relaxed)
|
||||
- hdr.ts_ms as i64;
|
||||
if (0..5000).contains(&e) {
|
||||
// 积压判定必须同时满足：突发到达（背靠背）+ 帧龄>250ms。
|
||||
// QUIC 可靠有序，积压须逐帧读出但可不解码——跳过解码
|
||||
// 让队列以网络速度排空
|
||||
stale = burst && e > 250;
|
||||
// 空闲重发帧的帧龄不代表显示延迟，不计入 EMA
|
||||
if burst || e < 250 {
|
||||
if (0..5000).contains(&est) {
|
||||
let old = stats.e2e.load(Ordering::Relaxed) as i64;
|
||||
let sm = if stats.e2e_valid.load(Ordering::Relaxed) {
|
||||
(old * 4 + e) / 5
|
||||
(old * 4 + est) / 5
|
||||
} else {
|
||||
e
|
||||
est
|
||||
};
|
||||
stats.e2e.store(sm as u64, Ordering::Relaxed);
|
||||
stats.e2e_valid.store(true, Ordering::Relaxed);
|
||||
est = Some(sm);
|
||||
}
|
||||
}
|
||||
}
|
||||
stats.on_frame_arrival(est);
|
||||
if stale {
|
||||
skip_stale += 1;
|
||||
if skip_stale % 60 == 1 {
|
||||
eprintln!(
|
||||
"[viewer] backlog: stale burst frame skipped (total {skip_stale})"
|
||||
);
|
||||
}
|
||||
// 跳帧直接切断 H.264 参考链：之后所有 P 帧都解不出来。
|
||||
// 立刻转入「等待 IDR」并请求关键帧，绝不能让后续 P 帧
|
||||
// 继续喂给解码器（那是必然失败的徒劳尝试）。
|
||||
stats.awaiting_idr.store(true, Ordering::Relaxed);
|
||||
if last_kfreq.elapsed() >= Duration::from_millis(200) {
|
||||
last_kfreq = Instant::now();
|
||||
if let Ok(mut slot) = latest.lock() {
|
||||
*slot = Some((0, 0, Vec::new()));
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// 参考链未恢复时，P 帧一律跳过——此为正确性要求，不是优化。
|
||||
if stats.awaiting_idr.load(Ordering::Relaxed) && !hdr.keyframe {
|
||||
skip_stale += 1;
|
||||
// 补发请求，防止首次 KeyFrameReq 丢失（时间闸限频）
|
||||
if last_kfreq.elapsed() >= Duration::from_millis(200) {
|
||||
last_kfreq = Instant::now();
|
||||
if let Ok(mut slot) = latest.lock() {
|
||||
*slot = Some((0, 0, Vec::new()));
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// 拿到 IDR 才能重新开始解码
|
||||
if hdr.keyframe {
|
||||
stats.awaiting_idr.store(false, Ordering::Relaxed);
|
||||
}
|
||||
match decoder.decode(&payload) {
|
||||
Ok(Some(yuv)) => {
|
||||
let (dw, dh) = yuv.dimensions();
|
||||
let (dw, dh) = (dw as u32, dh as u32);
|
||||
if !size_logged {
|
||||
size_logged = true;
|
||||
eprintln!(
|
||||
"[viewer] decoded frame {dw}x{dh} (window {}x{})",
|
||||
hdr.width, hdr.height
|
||||
);
|
||||
}
|
||||
rgb_buf.clear();
|
||||
rgb_buf.resize(dw as usize * dh as usize * 3, 0);
|
||||
yuv.write_rgb8(&mut rgb_buf);
|
||||
// latest-wins：直接覆盖旧帧，无积压
|
||||
if let Ok(mut slot) = latest.lock() {
|
||||
*slot = Some((dw, dh, rgb_buf.clone()));
|
||||
let mut rgb = vec![0u8; dw * dh * 3];
|
||||
yuv.write_rgb8(&mut rgb);
|
||||
if frame_tx.send(rgb).is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => {
|
||||
// 解码异常：参考链已不可信，转入等待 IDR
|
||||
stats.awaiting_idr.store(true, Ordering::Relaxed);
|
||||
stats.dec_fail.fetch_add(1, Ordering::Relaxed);
|
||||
// 解码异常：请求被控端强制关键帧
|
||||
eprintln!("[viewer] decode failed: {e}, request keyframe");
|
||||
if let Ok(mut slot) = latest.lock() {
|
||||
*slot = Some((0, 0, Vec::new()));
|
||||
}
|
||||
let _ = frame_tx.send(Vec::new()); // 空帧占位唤醒渲染循环
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -408,11 +220,9 @@ fn session(
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
match read_control(&mut irecv).await {
|
||||
Ok(ControlMsg::Pong { t, agent_ts, lost, sent }) => {
|
||||
Ok(ControlMsg::Pong { t, agent_ts }) => {
|
||||
let r = now_ms();
|
||||
stats.rtt.store(r.saturating_sub(t), Ordering::Relaxed);
|
||||
stats.agent_lost.store(lost, Ordering::Relaxed);
|
||||
stats.agent_sent.store(sent, Ordering::Relaxed);
|
||||
// agent_ts ≈ (t+r)/2 时刻的 agent 时钟读数 → 偏移 EMA
|
||||
let est = agent_ts as i64 - ((t + r) / 2) as i64;
|
||||
let old = stats.offset.load(Ordering::Relaxed);
|
||||
@@ -425,10 +235,7 @@ fn session(
|
||||
stats.offset_valid.store(true, Ordering::Relaxed);
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(e) => {
|
||||
eprintln!("[viewer] control stream closed: {e} (RTT/LOSS 将停止更新)");
|
||||
break;
|
||||
}
|
||||
Err(_) => break,
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -448,11 +255,9 @@ fn session(
|
||||
let push = |tx: &tokio::sync::mpsc::UnboundedSender<Vec<u8>>, m: ControlMsg| {
|
||||
let _ = tx.send(common::encode_control(&m));
|
||||
};
|
||||
// 入会立即请求 IDR：避免从 GOP 中间接入收到裸 P 帧解码失败导致启动黑屏
|
||||
push(&msg_tx, ControlMsg::KeyFrameReq);
|
||||
|
||||
let mut window = Window::new(
|
||||
"mvp-remote v0.8.1-CatchUp (Esc 退出)",
|
||||
"mvp-remote viewer (Esc 退出)",
|
||||
w,
|
||||
h,
|
||||
WindowOptions {
|
||||
@@ -473,8 +278,6 @@ fn session(
|
||||
let mut hud_bytes = 0u64;
|
||||
let mut hud_fps = 0u64;
|
||||
let mut hud_mbps = 0f64;
|
||||
let mut hud_lost = 0u64;
|
||||
let mut hud_sent = 0u64;
|
||||
|
||||
loop {
|
||||
if !window.is_open() || window.is_key_down(Key::Escape) {
|
||||
@@ -487,16 +290,19 @@ fn session(
|
||||
push(&msg_tx, ControlMsg::Ping(now_ms()));
|
||||
}
|
||||
|
||||
// 1) 渲染最新帧（单槽 latest-wins：每迭代只取最新一帧，天然防积压）
|
||||
if let Ok(mut slot) = latest.lock() {
|
||||
if let Some((sw, sh, raw)) = slot.take() {
|
||||
// 1) 渲染最新帧（只保留最新，丢弃积压）；空帧=解码异常，请求关键帧
|
||||
loop {
|
||||
match frame_rx.try_recv() {
|
||||
Ok(raw) => {
|
||||
if raw.is_empty() {
|
||||
push(&msg_tx, ControlMsg::KeyFrameReq);
|
||||
} else if sw > 0 && sh > 0 {
|
||||
blit(&mut buffer, w, h, &raw, sw as usize, sh as usize);
|
||||
} else if raw.len() == w * h * 3 {
|
||||
blit(&mut buffer, &raw);
|
||||
dirty = true;
|
||||
}
|
||||
}
|
||||
Err(_) => break,
|
||||
}
|
||||
}
|
||||
|
||||
// 2) 键盘
|
||||
@@ -561,9 +367,7 @@ fn session(
|
||||
}
|
||||
}
|
||||
|
||||
// 6) HUD：指标每秒刷新一次；每秒至少强制呈现一次（视频冻结/黑屏时 HUD 依然可见）
|
||||
let mut need_present = dirty;
|
||||
let (hud_loss, hud_jit, hud_p95, hud_dec);
|
||||
// 6) HUD：每秒刷新一次指标，画在右上角
|
||||
if hud_last.elapsed() >= Duration::from_secs(1) {
|
||||
let f = stats.frames.load(Ordering::Relaxed);
|
||||
let b = stats.bytes.load(Ordering::Relaxed);
|
||||
@@ -571,47 +375,12 @@ fn session(
|
||||
hud_mbps = (b.saturating_sub(hud_bytes)) as f64 * 8.0 / 1e6;
|
||||
hud_frames = f;
|
||||
hud_bytes = b;
|
||||
// 丢包率 = Δlost / Δsent（agent 发送方向，随 Pong 同步）
|
||||
let l = stats.agent_lost.load(Ordering::Relaxed);
|
||||
let s = stats.agent_sent.load(Ordering::Relaxed);
|
||||
hud_loss = if s > hud_sent {
|
||||
((l - hud_lost) as f64 / (s - hud_sent) as f64 * 100.0).min(99.9)
|
||||
} else {
|
||||
-1.0
|
||||
};
|
||||
hud_lost = l;
|
||||
hud_sent = s;
|
||||
hud_jit = stats.jitter_x16.load(Ordering::Relaxed) / 16;
|
||||
hud_p95 = stats.gap_p95();
|
||||
hud_dec = stats.dec_fail.load(Ordering::Relaxed);
|
||||
hud_last = Instant::now();
|
||||
need_present = true; // HUD 面板每秒至少重绘一次
|
||||
} else {
|
||||
hud_loss = f64::NAN;
|
||||
hud_jit = 0;
|
||||
hud_p95 = None;
|
||||
hud_dec = 0;
|
||||
}
|
||||
if need_present {
|
||||
if dirty {
|
||||
let rtt = stats.rtt.load(Ordering::Relaxed);
|
||||
let e2e = stats.e2e.load(Ordering::Relaxed);
|
||||
let e2e_ok = stats.e2e_valid.load(Ordering::Relaxed);
|
||||
let loss_str = if hud_loss.is_nan() {
|
||||
"--".to_string()
|
||||
} else if hud_loss < 0.0 {
|
||||
"--".to_string()
|
||||
} else {
|
||||
format!("{hud_loss:.1}%")
|
||||
};
|
||||
let loss_c = if hud_loss.is_nan() || hud_loss < 0.0 {
|
||||
C_GRAY
|
||||
} else {
|
||||
lat_color(hud_loss < 1.0, hud_loss < 5.0)
|
||||
};
|
||||
let jit_c = lat_color(hud_jit < 10, hud_jit < 30);
|
||||
let p95_c = hud_p95
|
||||
.map(|p| lat_color(p < 60, p < 150))
|
||||
.unwrap_or(C_GRAY);
|
||||
draw_hud(
|
||||
&mut buffer,
|
||||
w,
|
||||
@@ -623,25 +392,6 @@ fn session(
|
||||
&if e2e_ok { format!("{e2e}ms") } else { "--".into() },
|
||||
lat_color(e2e < 100, e2e < 200),
|
||||
),
|
||||
("JIT", &format!("{hud_jit}ms"), jit_c),
|
||||
("P95", &hud_p95.map(|p| format!("{p}ms")).unwrap_or("--".into()), p95_c),
|
||||
("LOSS", &loss_str, loss_c),
|
||||
("DECFAIL", &format!("{hud_dec}"), if hud_dec == 0 { C_GRAY } else { C_RED }),
|
||||
(
|
||||
// 参考链断裂、正在等 IDR 的显式指示：此时画面冻结是**预期行为**，
|
||||
// 不是故障。没有这个指示，用户会以为软件卡死。
|
||||
"STATE",
|
||||
&if stats.awaiting_idr.load(Ordering::Relaxed) {
|
||||
"RECOVER".to_string()
|
||||
} else {
|
||||
"OK".to_string()
|
||||
},
|
||||
if stats.awaiting_idr.load(Ordering::Relaxed) {
|
||||
C_AMBER
|
||||
} else {
|
||||
C_GREEN
|
||||
},
|
||||
),
|
||||
("RATE", &format!("{:.0}MBP", hud_mbps), C_GRAY),
|
||||
("RES", &format!("{w}x{h}"), C_GRAY),
|
||||
],
|
||||
@@ -723,7 +473,6 @@ fn glyph(c: u8) -> [u8; 5] {
|
||||
b':' => [0x00, 0x36, 0x36, 0x00, 0x00],
|
||||
b'.' => [0x00, 0x60, 0x60, 0x00, 0x00],
|
||||
b'-' => [0x08, 0x08, 0x08, 0x08, 0x08],
|
||||
b'%' => [0x13, 0x18, 0x04, 0x0A, 0x11],
|
||||
b' ' => [0x00; 5],
|
||||
_ => [0x00; 5],
|
||||
}
|
||||
@@ -797,17 +546,9 @@ fn draw_hud(buffer: &mut [u32], w: usize, lines: &[(&str, &str, u32)]) {
|
||||
}
|
||||
}
|
||||
|
||||
/// RGB888 字节（openh264 解码输出）→ minifb 0RGB u32 缓冲。
|
||||
/// 支持源尺寸与窗口不同（解码器可能对齐到 16 倍数，如 1080→1088）：
|
||||
/// 逐行拷贝重叠区域，源小于窗口时右侧/下方留黑。
|
||||
fn blit(buffer: &mut [u32], w: usize, h: usize, raw: &[u8], dw: usize, dh: usize) {
|
||||
let cw = w.min(dw);
|
||||
let ch = h.min(dh);
|
||||
for row in 0..ch {
|
||||
let src = &raw[row * dw * 3..row * dw * 3 + cw * 3];
|
||||
let dst = &mut buffer[row * w..row * w + cw];
|
||||
for (px, chunk) in dst.iter_mut().zip(src.chunks_exact(3)) {
|
||||
/// RGB888 字节（openh264 解码输出）→ minifb 0RGB u32 缓冲
|
||||
fn blit(buffer: &mut [u32], raw: &[u8]) {
|
||||
for (px, chunk) in buffer.iter_mut().zip(raw.chunks_exact(3)) {
|
||||
*px = ((chunk[0] as u32) << 16) | ((chunk[1] as u32) << 8) | chunk[2] as u32;
|
||||
}
|
||||
}
|
||||
}
|
||||
-301
@@ -1,301 +0,0 @@
|
||||
# 借鉴映射与落地规范
|
||||
|
||||
> 配套《多平台架构设计.md》。本文回答三个问题：**每个模块怎么设计、看谁、抄什么、不抄什么。**
|
||||
> 状态：架构部分已定稿；第 2 章的「逆向需求清单」等同事的逆向资料回填。
|
||||
|
||||
---
|
||||
|
||||
## 0. 使用方式
|
||||
|
||||
| 角色 | 看哪几章 |
|
||||
|---|---|
|
||||
| A（被控侧） | §3.5 platform、§3.6 agent-core、§4 表中 agent 相关行 |
|
||||
| B（主控+协议） | §3.1–3.4、§3.7 viewer-core、§5 许可 |
|
||||
| 移动端（待补人） | §3.4 media（解码后端）、§3.8 apps |
|
||||
| 全员 | §2 逆向红线（踩了会出大事） |
|
||||
|
||||
---
|
||||
|
||||
## 1. 两条铁律
|
||||
|
||||
1. **架构由需求决定，不由参考项目决定。** 参考项目用来解决"这一块业界怎么做的"，不是用来决定"我们做成什么样"。
|
||||
2. **能抄的是算法和协议思路，抄不到的是平台集成和 UI。** 后者是体力活，任何参考项目都救不了（它们自己也是硬写的）。
|
||||
|
||||
---
|
||||
|
||||
## 2. 逆向资料的用法与红线
|
||||
|
||||
### 2.1 红线（务必转告同事）
|
||||
|
||||
| 可以做 | 不可以做 |
|
||||
|---|---|
|
||||
| 记录**功能清单、交互流程、界面布局** | 把反编译产物、反汇编清单提交进仓库 |
|
||||
| 记录**用户可见的错误提示、状态显示** | 逐字复制对方的 UI 文案、图标、素材 |
|
||||
| 推断**产品模型**（账号/设备/授权怎么设计） | 抄对方私有协议的具体编码（抄了也不能互通） |
|
||||
| 记录**网络行为的宏观特征**（是否中转、连接耗时） | 提取并移植对方的算法实现 |
|
||||
|
||||
**仓库里只提交"行为结论"，不提交"逆向产物"。** 建议逆向资料单独放一个**不纳入 git** 的目录（或内部网盘），文档里只写结论。这条不是洁癖——商用产品里出现竞品反编译代码是实打实的法律风险。
|
||||
|
||||
### 2.2 逆向需求清单（直接发给同事）
|
||||
|
||||
**A. 产品功能面（最重要，优先做）**
|
||||
|
||||
1. 主控端完整交互流程截图流：登录 → 设备列表 → 连接（输密码/免密）→ 会话中 → 断开
|
||||
2. 每个平台的功能矩阵：Win/Linux/Android/iOS 各有哪些功能、哪些缺失
|
||||
3. 会话中工具栏全清单：分辨率切换、多屏、文件传输、剪贴板、音频开关、画质/帧率档位、游戏模式
|
||||
4. 设置项全清单（分"被控设置"和"主控设置"两类）
|
||||
5. 错误提示文案全集 + 断线重连时的用户可见行为
|
||||
|
||||
**B. 身份与授权模型**
|
||||
|
||||
6. 设备 ID 的形式（长度/字符集）、在哪里查看、能否自定义
|
||||
7. 授权方式有几种：临时密码 / 固定密码 / 免密白名单 / 账号直接授权？默认是哪种
|
||||
8. 账号体系：是否必须登录、登录后设备列表怎么呈现、一台设备能否被多账号访问
|
||||
9. 被控端的"允许谁连"控制（黑名单/白名单/单次确认弹窗）
|
||||
|
||||
**C. 被控端形态**
|
||||
|
||||
10. 是不是系统服务？开机自启吗？锁屏/注销/UAC 提权界面能不能连？
|
||||
11. 安装包形态、是否需要管理员权限、卸载残留
|
||||
|
||||
**D. 网络行为（只看宏观，不要深入协议）**
|
||||
|
||||
12. 连接时用户是否能看到"直连 / 中转"标识
|
||||
13. 典型连接耗时、转中继时的画质变化
|
||||
14. 端口/协议宏观特征：UDP 还是 TCP 为主、监听端口是否固定
|
||||
|
||||
**E. 商业化（可选但有用）**
|
||||
|
||||
15. 免费额度/收费点/增值功能边界
|
||||
|
||||
### 2.3 逆向资料回填位置
|
||||
|
||||
| 资料 | 回填到 |
|
||||
|---|---|
|
||||
| A 组 | 本文 §6「产品范围基线」+ 里程碑取舍 |
|
||||
| B/C 组 | §3.3 session（身份模型影响状态机） |
|
||||
| D 组 | §3.2 conn（验证我们的打洞/中继策略） |
|
||||
| E 组 | 不在本文，属于商业决策 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 模块级设计规范
|
||||
|
||||
### 3.1 `proto` — 协议
|
||||
|
||||
**职责**：信令消息、控制消息、媒体容器格式。三端（agent/viewer/信令服务）共享契约。
|
||||
|
||||
**设计点**
|
||||
- **双协议分离**：信令协议（`signaling.rs`，文本友好、低频、可 JSON）与媒体控制协议（`lib.rs`，二进制、高频）**不要混**。前者要便于抓包排查，后者要极致紧凑。
|
||||
- **版本协商**：`SIGNALING_VERSION` 不一致直接拒绝，不做兼容。媒体协议用"能力协商"而非版本号（见 §3.3）。
|
||||
- **消息演进**：现用 bincode。二期若换 protobuf，**必须一次性切换**，不要两套并存。
|
||||
- **单帧长度上限**：现有 `ensure!(len < 1<<20)` 这类边界检查要保留，且信令侧对端候选列表必须经 `sanitize_candidates`。
|
||||
|
||||
**看谁 / 抄什么**
|
||||
|
||||
| 项目 | 抄什么 | 不抄 |
|
||||
|---|---|---|
|
||||
| RustDesk `libs/hbb_common` | 消息分类方式（信令/中继/控制分层） | 具体协议编码（AGPL + 抄了不互通） |
|
||||
| Moonlight `moonlight-common-c` | **控制消息与媒体分离**的成熟做法 | 它的私有控制协议 |
|
||||
|
||||
**产出物**：`crates/proto/src/{lib.rs, signaling.rs}`
|
||||
|
||||
---
|
||||
|
||||
### 3.2 `conn` — 连接层（最值得借鉴的一块）
|
||||
|
||||
**职责**：信令客户端、打洞编排、路径选择、QUIC 会话、保活。
|
||||
|
||||
**设计点**
|
||||
- **`PathProvider` 抽象**（已在《打洞与信令设计.md》定义）：现在只实现 Direct，类型上预留 Relay。
|
||||
- **打洞编排**：候选收集（v6/Host/UPnP/Reflexive/Predicted）→ 排序 → 信令交换 → 同步打洞 → 落选清理。
|
||||
- **同一 socket 双角色**：打洞用的 UDP socket 必须与后续 QUIC 是同一个，否则洞不在同一映射上。
|
||||
- **保活**：按双方 `binding_timeout_s` 折半，5–60s。**这是"连一会儿就断"的根治点。**
|
||||
- **不用 QUIC 握手当探测**：先用轻量 UDP 探测包确认通路（秒级失败），再 `connect`。
|
||||
|
||||
**看谁 / 抄什么**
|
||||
|
||||
| 项目 | 抄什么 | 许可 |
|
||||
|---|---|---|
|
||||
| **iroh** | **整套**：打洞编排、中继回退、地址发现。技术栈与你们重合（Rust + quinn） | Apache-2.0 ✅ **可直接依赖** |
|
||||
| webrtc-rs / pion/ice | ICE 状态机、候选配对（candidate pairing）策略 | MIT/Apache ✅ |
|
||||
| Tailscale / headscale | DERP 中继的**回退判定条件**与降级策略 | BSD-3 ✅ |
|
||||
| RustDesk `rendezvous_mediator` 相关 | 信令 ↔ 打洞的**编排顺序**（读思路） | AGPL ⚠️ 只读 |
|
||||
| libjuice / libnice | ICE 精简实现（作为对照，不必依赖） | MPL / LGPL |
|
||||
|
||||
> **建议**：`conn` 优先评估**直接依赖 iroh**。它解决的正是你们纠结的"打洞要不要中继"，而且给了"直连优先、失败回退"的完整实现。自研仅当 iroh 有硬伤（体积、可控性、特定 NAT 场景）时才考虑。
|
||||
|
||||
**产出物**：`crates/conn/`
|
||||
|
||||
---
|
||||
|
||||
### 3.3 `session` — 会话编排（"一体程序"的核心）
|
||||
|
||||
**职责**：会话状态机、角色仲裁、能力协商、生命周期管理、被控端准入控制。
|
||||
|
||||
**设计点**
|
||||
- **状态机双线**：主控线 / 被控线各自独立，同一进程可并发（被控中也能主控别人）。
|
||||
- **角色仲裁**：打洞阶段双方对称，**谁先打通谁定角色**（被控方做 QUIC server），不预设。
|
||||
- **能力协商**：被控端上报硬件能力（可用编码器、最大分辨率、帧率上限、是否支持音频），主控端选择。**用能力协商代替版本号**——旧客户端缺字段就是"不支持"，天然向后兼容。
|
||||
- **准入控制**：单会话/多会话、白名单、单次确认弹窗。**这一条直接来自 §2 逆向需求 B 组第 7/9 项**，等资料回填后定稿。
|
||||
- **失败出口**：`PunchFailReason::user_hint()` 必须一路透传到 UI，不允许出现无解释的"连接中"。
|
||||
|
||||
**看谁 / 抄什么**
|
||||
|
||||
| 项目 | 抄什么 |
|
||||
|---|---|
|
||||
| RustDesk `src/server` / `src/client` | 会话生命周期与连接管理的分层方式（读思路） |
|
||||
| Sunshine + Moonlight | host/client 两侧的**能力协商**与配置下发流程 |
|
||||
| — | 状态机本身建议自己画，不要套用别人的（各家产品逻辑差异大） |
|
||||
|
||||
**产出物**：`crates/session/`
|
||||
|
||||
---
|
||||
|
||||
### 3.4 `media` — 编解码抽象
|
||||
|
||||
**职责**：编码/解码 trait、各平台后端、零拷贝通路、码率自适应闭环。
|
||||
|
||||
**设计点**
|
||||
- **不对称策略**：**编码用 FFmpeg**（只在 Win/Linux，桌面端不心疼包体），**解码用平台原生**（四个主控平台都要，移动端必须硬解且要零拷贝）。
|
||||
- **trait 收口**（见《多平台架构设计.md》§4 决策 4），上层不允许出现平台判断。
|
||||
- **零拷贝**：解码器直接输出到渲染表面（D3D11 纹理 / `SurfaceView` / `CAMetalLayer`），中间不做 CPU 拷贝。
|
||||
- **软硬兜底**：硬编失败回落 openh264；硬解失败回落软解（桌面可，移动端应直接提示而非硬撑）。
|
||||
- **码率自适应闭环**：接收端统计（丢包/抖动/帧间隔 p95）→ 反馈编码端 → AIMD 调码率。协议已留 `RateReport`（见 B侧设计文档 B2）。
|
||||
|
||||
**看谁 / 抄什么**
|
||||
|
||||
| 项目 | 抄什么 | 许可 |
|
||||
|---|---|---|
|
||||
| **libwebrtc** `modules/congestion_controller/goog_cc` | **GCC 拥塞控制**的完整思路——这是码率自适应的教科书 | BSD-3 ✅ 可参考可移植 |
|
||||
| **libwebrtc** `modules/video_coding/jitter_buffer` | 抖动缓冲与丢包重排 | BSD-3 ✅ |
|
||||
| **Sunshine** `src/video` + `src/platform` | **抓屏→编码→打包**的完整管线组织，尤其是多编码器抽象与回落 | GPL-3.0 ⚠️ 只读 |
|
||||
| **Moonlight** `moonlight-common-c` | 接收端解码→渲染→HUD 的管线，以及**丢包恢复策略** | GPL-3.0 ⚠️ 只读 |
|
||||
| FFmpeg | NVENC/QSV/AMF/VAAPI 的调用姿势与参数 | LGPL/GPL ⚠️ 注意构建选项 |
|
||||
| openh264 | 软编兜底（已在用） | BSD ✅ |
|
||||
|
||||
**产出物**：`crates/media/`
|
||||
|
||||
---
|
||||
|
||||
### 3.5 `platform` — L0 平台层（抄不到，只能自己写）
|
||||
|
||||
**职责**：抓屏、注入、渲染表面、音频 IO、权限、常驻。
|
||||
|
||||
**为什么抄不到**：这层是各平台 API 的体力活。参考项目能给的是"用哪个 API、踩过什么坑"，不是可复用代码。
|
||||
|
||||
**设计点**
|
||||
|
||||
| 平台 | 关键坑 |
|
||||
|---|---|
|
||||
| Win 抓屏 | DXGI DD 在**会话切换/RDP 断开**时失效，必须处理 `DXGI_ERROR_ACCESS_LOST` 并重建 |
|
||||
| Win 注入 | 服务进程注入到用户会话需要 `CreateProcessAsUser` + 会话隔离处理；UAC 提权界面默认收不到输入（需 `uiAccess` 签名） |
|
||||
| Win 常驻 | 服务 + 用户会话代理双进程，锁屏可连靠的是**服务侧保持会话** |
|
||||
| Linux 抓屏 | X11 用 XShm/XComposite；**Wayland 必须走 PipeWire + xdg-desktop-portal**（有授权弹窗，无法静默） |
|
||||
| Linux 注入 | X11 用 XTest；**Wayland 用 libei/uinput**（uinput 需 root 或 udev 规则） |
|
||||
| Linux 常驻 | systemd 服务 + 会话切换处理（Wayland 的 portal 授权是最大障碍） |
|
||||
| 渲染表面 | 四平台各一套；**必须支持"表面尺寸≠视频尺寸"的缩放与拉伸模式** |
|
||||
|
||||
**看谁**
|
||||
|
||||
| 项目 | 看什么 |
|
||||
|---|---|
|
||||
| Sunshine `src/platform/{windows,linux}` | 抓屏与注入的分平台组织方式、Wayland 处理 |
|
||||
| RustDesk `libs/scrap` | 同样的抓屏抽象（多平台统一 trait）——**这是它最有价值的部分** |
|
||||
| libei / xdg-desktop-portal 官方文档 | Wayland 的正规姿势（别用野路子） |
|
||||
|
||||
**产出物**：`crates/platform/`
|
||||
|
||||
---
|
||||
|
||||
### 3.6 `agent-core` / `viewer-core` — 两条链路
|
||||
|
||||
**职责**：把现有 `agent`/`viewer` 的 main 逻辑抽成库，供一体程序按角色调用。
|
||||
|
||||
**设计点**
|
||||
- **`agent-core`**：capture → encode → 打包 → 发送；输入接收 → 注入。A 已在进行中（`agent-core/src/lib.rs` 398 行）。
|
||||
- **`viewer-core`**：接收 → 解码 → 渲染表面；输入采集 → 发送。
|
||||
- **生命周期**：两者都不允许自己决定退出，由 `session` 层驱动。
|
||||
- **单槽 latest-wins**：现有做法保留（网络慢时只发最新帧），这是正确设计。
|
||||
|
||||
**看谁**：RustDesk（抽库方式）、Sunshine/Moonlight（管线组织）。
|
||||
|
||||
**产出物**：`crates/agent-core/`、`crates/viewer-core/`
|
||||
|
||||
---
|
||||
|
||||
### 3.7 `apps/*` — UI 与常驻（第二个抄不到的地方）
|
||||
|
||||
**职责**：UI 壳、常驻服务、安装包。
|
||||
|
||||
**设计点**
|
||||
- **硬约束**：所有业务逻辑在 Rust，UI 只调一层稳定 API（决定了将来能否换 UI 框架）。
|
||||
- **视频走原生子表面**，绝不在 WebView 里逐帧渲染。
|
||||
- **一体 = 一个安装包 + 两个进程**：前台 UI + 后台被控服务，本地 IPC。
|
||||
- **移动端只有前台进程**（纯主控）。
|
||||
|
||||
**看谁**
|
||||
|
||||
| 项目 | 抄什么 |
|
||||
|---|---|
|
||||
| Tauri v2 官方 mobile 示例 | WKWebView/Android WebView 与原生 surface 的叠加方式 |
|
||||
| RustDesk `flutter/` | 同一套 UI 适配多端的组织方式（它用 Flutter，可对照取舍） |
|
||||
| Moonlight `moonlight-android` / `moonlight-ios` | **移动端主控的完整交互**：虚拟鼠标、手势映射、软键盘、性能档位 |
|
||||
|
||||
**产出物**：`apps/desktop/`、`apps/android/`、`apps/ios/`、`apps/daemon-win/`、`apps/daemon-linux/`
|
||||
|
||||
---
|
||||
|
||||
## 4. 借鉴映射总表
|
||||
|
||||
| 模块 | 主参考 | 许可 | 用法 |
|
||||
|---|---|---|---|
|
||||
| 打洞 / 中继 / 地址发现 | **iroh** | Apache-2.0 | **直接依赖** |
|
||||
| ICE / STUN / TURN | webrtc-rs、pion、coturn | MIT/BSD | 依赖或参考 |
|
||||
| 中继回退策略 | Tailscale / headscale | BSD-3 | 读思路 |
|
||||
| 拥塞控制 / 抖动缓冲 | **libwebrtc** | BSD-3 | 读思路，重写 |
|
||||
| 抓屏→编码管线 | **Sunshine** | GPL-3.0 | 只读 |
|
||||
| 接收端解码→渲染 | **Moonlight** | GPL-3.0 | 只读 |
|
||||
| 整机架构 / 抽库方式 | **RustDesk** | **AGPL-3.0** | **只读，绝不抄代码** |
|
||||
| 移动端主控交互 | Moonlight mobile | GPL-3.0 | 只读 |
|
||||
| Rust↔移动端绑定 | UniFFI | MPL-2.0 | 依赖 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 许可结论与依赖策略
|
||||
|
||||
| 许可 | 能否进产品 | 项目 |
|
||||
|---|---|---|
|
||||
| MIT / Apache-2.0 / BSD | ✅ 可依赖、可移植 | iroh、webrtc-rs、pion、coturn、libwebrtc、UniFFI(MPL 弱) |
|
||||
| LGPL | ⚠️ 可动态链接，改库要开源 | FFmpeg、GStreamer、libnice |
|
||||
| **GPL / AGPL** | ❌ **不可进代码** | **RustDesk(AGPL)、Sunshine/Moonlight(GPL)** |
|
||||
|
||||
**AGPL 特别提示**：网络服务也算分发。引入 RustDesk 代码 = 你们整个产品连服务端都要开源。
|
||||
|
||||
**依赖准入流程**（补充进《Git协作规范.md》的合并规则）：新增依赖必须在 PR 描述里写明**许可证 + 是否修改其源码**，由另一人确认。
|
||||
|
||||
---
|
||||
|
||||
## 6. 产品范围基线（待逆向资料回填）
|
||||
|
||||
> 拿到 §2.2 的 A 组资料后填这里，用于定"做哪些功能、砍哪些平台"。
|
||||
|
||||
| 功能 | UU 有 | 我们一期 | 备注 |
|
||||
|---|---|---|---|
|
||||
| 桌面画面 | | ✅ | 已有 |
|
||||
| 键鼠控制 | | ✅ | 已有 |
|
||||
| 音频 | | ? | |
|
||||
| 文件传输 | | ? | |
|
||||
| 剪贴板同步 | | ? | |
|
||||
| 多屏 | | ? | |
|
||||
| 移动端主控 | | ? | 人力问题见《多平台架构设计.md》§8 |
|
||||
| 无人值守 | | ? | 安全含义，需固定密码+白名单 |
|
||||
|
||||
---
|
||||
|
||||
## 7. 待拍板
|
||||
|
||||
1. `conn` 是否**直接依赖 iroh**（而不是自研打洞）——建议先花 2 天做技术验证
|
||||
2. 逆向资料回来后，"产品范围基线"的取舍（尤其移动端）
|
||||
3. `media` 编码侧是否引入 FFmpeg（包体/构建成本 vs 省下的适配工作）
|
||||
4. 依赖准入流程是否写进《Git协作规范.md》
|
||||
-315
@@ -1,315 +0,0 @@
|
||||
# 多平台架构设计
|
||||
|
||||
> 需求：**主控端** Windows / Linux / Android / iOS；**被控端** Windows / Linux。
|
||||
> 桌面端（Win/Linux）主控+被控合为一体；移动端仅主控。
|
||||
> 前置文档：《打洞与信令设计.md》《B侧设计文档.md》《构建运行手册.md》
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话架构
|
||||
|
||||
**一个 Rust 核心 + 一层薄平台适配 + 可替换的 UI 壳。**
|
||||
业务逻辑（连接/打洞/会话/协议）100% 在 Rust 里跨平台共享；只有**抓屏、编解码、注入、渲染表面**四件事分平台写；UI 层只管"壳"，**永远不碰视频流**。
|
||||
|
||||
---
|
||||
|
||||
## 1. 纵向分层
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ L4 UI 壳 Win/Linux/Android/iOS 各自实现 │
|
||||
│ 设备列表 / 设置 / 连接管理 / 悬浮控件 │
|
||||
│ ★ 绝不放视频渲染 │
|
||||
├──────────────────────────────────────────────────────────┤
|
||||
│ L3 会话层 session 纯 Rust，全平台共享 │
|
||||
│ 会话状态机、角色仲裁、能力协商、keepalive 调度 │
|
||||
├──────────────────────────────────────────────────────────┤
|
||||
│ L2 连接层 conn 纯 Rust，全平台共享 │
|
||||
│ 信令客户端、打洞编排、中继接口(预留)、QUIC(quinn)多流 │
|
||||
├──────────────────────────────────────────────────────────┤
|
||||
│ L1 媒体层 media Rust + 平台后端 │
|
||||
│ 编码/解码/色彩转换/缩放/音频(Opus) trait 化 │
|
||||
├──────────────────────────────────────────────────────────┤
|
||||
│ L0 平台层 platform 唯一必须分平台的地方 │
|
||||
│ 抓屏 / 注入 / 音频IO / 渲染表面 / 权限 / 常驻 │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**分界线原则**：L1 以上任何代码不允许出现 `#[cfg(windows)]`。平台差异只能通过 L0 的 trait 暴露。
|
||||
|
||||
---
|
||||
|
||||
## 2. 仓库结构
|
||||
|
||||
```
|
||||
remote/
|
||||
├── crates/
|
||||
│ ├── proto/ # 协议：信令 + 控制 + 媒体容器（现 common/）
|
||||
│ ├── conn/ # 信令客户端 / 打洞编排 / QUIC 会话 / 中继接口
|
||||
│ ├── session/ # 会话状态机 + 角色仲裁（"一体"的核心）
|
||||
│ ├── agent-core/ # 被控链路：capture → encode → send [A 已在做]
|
||||
│ ├── viewer-core/ # 主控链路：recv → decode → surface
|
||||
│ ├── media/ # 编解码 trait + 各平台后端 + openh264 软兜底
|
||||
│ └── platform/ # L0 实现，按 cfg 编译各平台
|
||||
├── apps/
|
||||
│ ├── desktop/ # Tauri v2 壳：Win/Linux 一体程序
|
||||
│ ├── android/ # 主控（Tauri mobile 或原生 Kotlin）
|
||||
│ ├── ios/ # 主控（Tauri mobile 或原生 Swift）
|
||||
│ ├── daemon-win/ # Windows 服务：被控常驻
|
||||
│ └── daemon-linux/ # systemd：被控常驻
|
||||
├── bindings/ # UniFFI 生成的 Kotlin/Swift 绑定
|
||||
└── tools/ # bench / 信令服务器
|
||||
```
|
||||
|
||||
`agent-core` 与 `viewer-core` 的拆分是**一体化的前提**：现在的 `agent`/`viewer` 是两个 main，必须先变成库，才能被同一个进程按角色调用。A 正在做的 `agent-core` 抽取就是在铺这条路。
|
||||
|
||||
---
|
||||
|
||||
## 3. 平台能力矩阵
|
||||
|
||||
| 能力 | Windows | Linux | Android | iOS |
|
||||
|---|---|---|---|---|
|
||||
| 抓屏 | DXGI Desktop Duplication | X11 (XShm) / Wayland (PipeWire portal) | — | — |
|
||||
| 硬编 | NVENC / QSV / AMF | VAAPI / NVENC | — | — |
|
||||
| 硬解 | D3D11VA / NVDEC | VAAPI | MediaCodec | VideoToolbox |
|
||||
| 渲染表面 | D3D11 子窗口 | X11/GL 子窗口 | SurfaceView | UIView + CAMetalLayer |
|
||||
| 键鼠注入 | SendInput | XTest / uinput / libei | — | — |
|
||||
| 键鼠采集 | Raw Input | evdev | 触摸→鼠标映射 | 触摸→鼠标映射 |
|
||||
| 音频出 | WASAPI loopback | PulseAudio monitor | — | — |
|
||||
| 音频入/放 | WASAPI | PulseAudio | AudioTrack | AVAudioEngine |
|
||||
| 常驻服务 | Windows Service | systemd | — | — |
|
||||
| **角色** | **主 + 被** | **主 + 被** | **仅主** | **仅主** |
|
||||
|
||||
移动端没有"被控"，所以 Android/iOS **不需要**抓屏、编码、注入、服务化——少了一半工作量，这是需求里最省事的一条。
|
||||
|
||||
---
|
||||
|
||||
## 4. 六个关键决策
|
||||
|
||||
### 决策 1：视频渲染绝不走 UI 框架
|
||||
|
||||
从 WebView / Compose / Flutter 里逐帧渲染 60fps 视频必然掉帧、增延迟。
|
||||
**做法**：UI 框架只画"壳"，视频走**原生子表面**叠加在 UI 之上——
|
||||
Win: 子 HWND + D3D11；Linux: 子窗口 + GL；Android: `SurfaceView`；iOS: `UIView` + `CAMetalLayer`。
|
||||
MediaCodec / VideoToolbox 都能**直接解码到该 Surface，零拷贝**。这是 Parsec/ToDesk 这类产品的通行做法。
|
||||
|
||||
### 决策 2：UI 框架 —— 推荐 Tauri v2，但必须可替换
|
||||
|
||||
| 方案 | 覆盖 | 评价 |
|
||||
|---|---|---|
|
||||
| **Tauri v2** | Win/Linux/Android/iOS | ✅ 推荐：Rust 原生、一套 UI 四端、IPC 天然 |
|
||||
| Flutter | 四端 | UI 强，但 Rust 桥（flutter_rust_bridge）+ 平台视图叠加更绕 |
|
||||
| 各平台原生 ×4 | 四端 | 性能最好，**2-3 人团队不可能维护** |
|
||||
|
||||
选 Tauri v2 的理由：你们本来就写 Rust，视频走原生表面后 WebView 只做表单和列表，性能足够。
|
||||
**代价**：Tauri 移动端还不够成熟（WKWebView 与原生 surface 叠加、生命周期）。所以——
|
||||
|
||||
> **硬性约束：所有业务逻辑必须在 Rust 里，UI 只通过一层稳定 API 调用。**
|
||||
> 这样移动端若体验不达标，可以单点退化成原生 Kotlin/Swift + UniFFI，桌面仍用 Tauri，核心零改动。
|
||||
|
||||
### 决策 3：Rust ↔ 移动端绑定用 UniFFI
|
||||
|
||||
6 个平台角色组合下手写 JNI/FFI 会失控。UniFFI 从 Rust 生成 Kotlin/Swift 绑定，含 async、错误类型、对象生命周期，是当前标准解法。
|
||||
|
||||
### 决策 4：编码用 FFmpeg，解码用平台原生
|
||||
|
||||
不对称，但都有理由：
|
||||
|
||||
- **编码（只在 Win/Linux 被控端）**：用 FFmpeg 统一接管 NVENC/QSV/AMF/VAAPI，openh264 软编兜底。桌面端不心疼包体，省下大量适配。
|
||||
- **解码（四个主控平台都要）**：直接用平台原生 API。
|
||||
- 移动端**必须**硬解（软解烧电池、发热、跑不动 1080p60）
|
||||
- 平台原生能零拷贝直出渲染表面，FFmpeg 反而多一次拷贝
|
||||
- 绕开 FFmpeg 在 Android/iOS 上的交叉编译地狱和许可问题
|
||||
|
||||
`media` crate 用 trait 收口：
|
||||
|
||||
```rust
|
||||
pub trait VideoDecoder {
|
||||
fn configure(&mut self, cfg: &VideoConfig, target: SurfaceHandle) -> Result<()>;
|
||||
fn decode(&mut self, packet: &[u8]) -> Result<Option<FrameHandle>>;
|
||||
}
|
||||
|
||||
pub trait VideoEncoder {
|
||||
fn configure(&mut self, cfg: &EncodeConfig) -> Result<()>;
|
||||
fn encode(&mut self, frame: &FrameHandle, out: &mut Vec<u8>) -> Result<()>;
|
||||
fn request_keyframe(&mut self);
|
||||
}
|
||||
```
|
||||
|
||||
### 决策 5："一体程序" = 一个安装包 + 两个进程
|
||||
|
||||
用户视角是一个 app，实现上职责分离：
|
||||
|
||||
```
|
||||
┌────────────────────────┐ ┌──────────────────────────┐
|
||||
│ 前台进程（UI） │ IPC │ 后台常驻（被控服务） │
|
||||
│ 主控画面 / 设备列表 │<──────>│ 抓屏·编码·注入·保活 │
|
||||
│ 退出即结束 │ 本地管道│ 开机自启 / 锁屏可连 │
|
||||
└────────────────────────┘ └──────────────────────────┘
|
||||
▲ ▲
|
||||
│ 移动端只有这一个 │ 仅 Win/Linux
|
||||
```
|
||||
|
||||
理由：**两种角色的生命周期天然冲突**——被控要"开机就在、没人操作也在"，主控要"跟窗口走、关了就走"。硬塞进一个进程会导致主控一退出就掉线。
|
||||
IPC 用命名管道（Win）/ Unix socket（Linux），或复用 QUIC 走本地回环。
|
||||
|
||||
### 决策 6：中继接口必须预留 —— 哪怕现在不实现
|
||||
|
||||
`conn` 层定义：
|
||||
|
||||
```rust
|
||||
pub trait PathProvider {
|
||||
/// 返回一条可用数据通道：直连优先，失败时可回退
|
||||
async fn establish(&self, peer: &PeerInfo) -> Result<Path>;
|
||||
}
|
||||
|
||||
pub enum Path { Direct { addr: SocketAddr }, Relay { endpoint: EndpointId } }
|
||||
```
|
||||
|
||||
现在只实现 `Direct`。**但类型上留出 `Relay`**，将来补中继不必动会话层。
|
||||
|
||||
> ⚠️ **这条与"不做中继"的决定直接相关，见 §6。**
|
||||
|
||||
---
|
||||
|
||||
## 5. 会话状态机（L3 核心）
|
||||
|
||||
一体程序两条线：
|
||||
|
||||
```
|
||||
主控线: Idle → Signalling → Punching → Handshaking → Streaming → Closing
|
||||
被控线: Idle → Registered → Incoming → Punching → Handshaking → Serving → Closing
|
||||
↑ 同一进程两线可并发（被控中也能主控别人）
|
||||
```
|
||||
|
||||
要点：
|
||||
- 打洞阶段**双方对称**：谁都不预设客户端/服务端，各自向对端候选发包，谁先打通谁定角色（被控方做 QUIC server）
|
||||
- 会话能力协商：分辨率、编码器、码率上限、是否带音频——由**被控端**根据硬件能力给候选，主控端选
|
||||
- 单进程内用 tokio 一runtime 跑多会话；被控同时只服务一个主控（可配置）
|
||||
|
||||
---
|
||||
|
||||
## 6. ✅ 已裁决：不做中继（2026-10-28）
|
||||
|
||||
> **裁决结论**：公司不提供中继服务器，中继不做。以下保留原始分析以免后人重蹈讨论。
|
||||
|
||||
**手机在蜂窝网络下几乎 100% 处于运营商级 NAT（CGNAT），打洞打不通。**
|
||||
这不是实现问题，是网络事实——`100.64.0.0/10` 那段地址根本没给你可映射的公网端口。
|
||||
|
||||
后果：**"主控端支持安卓/iOS" + "不做中继" = 手机只能在 WiFi 下用。**
|
||||
地铁上、外面、用流量，一律连不上。
|
||||
|
||||
三条出路：
|
||||
|
||||
| 方案 | 代价 | 说明 |
|
||||
|---|---|---|
|
||||
| **A. 只靠 IPv6** | 部分用户不可用 | 国内三大运营商 v6 覆盖率高，手机 v6 可用性远好于 v4 公网 |
|
||||
| B. 补一个只为移动端服务的中继 | 视频带宽成本 | **已排除：公司不提供额外服务器** |
|
||||
| C. 接受"手机只能 WiFi" | 产品力受限 | 零成本，但要写进产品说明 |
|
||||
|
||||
### 6.1 裁决内容与边界（2026-10-28）
|
||||
|
||||
| 项 | 裁决 |
|
||||
|---|---|
|
||||
| 中继 / TURN / relay 服务器 | **不做**。公司不提供额外服务器，且无预算计划 |
|
||||
| IPv6 | **一等公民**。候选地址排序 v6 优先，`signaling.rs` 已实现 |
|
||||
| 中继接口预留 | **不预留**。不写 `PathProvider` trait、不写 `Path::Relay` 变体 |
|
||||
| 连接层边界 | **仍要做**。理由见 §6.2，与中继无关 |
|
||||
|
||||
**为什么不留中继接口**：留接口的收益是「将来补中继不用动会话层」，但既然中继已被排除，
|
||||
`Path::Relay` 这类变体永远不会被构造，只会产生 `dead_code` 警告，要么 `#[allow]` 要么日后删除。
|
||||
**零收益的预留就是纯负担。**
|
||||
|
||||
### 6.2 连接层边界与中继接口是两件事
|
||||
|
||||
这两件事常被混为一谈，必须分开：
|
||||
|
||||
| | 内容 | 现在做不做 |
|
||||
|---|---|---|
|
||||
| **连接层边界** | 把"怎么连"（信令/打洞/直连）与"连上干什么"（收流/解码/渲染）分开 | ✅ **要做** |
|
||||
| **中继接口** | `enum Path { Direct, Relay }` 这类类型预留 | ❌ **不做** |
|
||||
|
||||
连接层边界本身就有独立价值，与中继无关：
|
||||
|
||||
1. **连接方式确定会变**：局域网 `IP:端口` → 将来设备码 + 打洞，这是已定的路线；
|
||||
2. **安卓主控要复用**：`viewer-core` 收流解码渲染不应知道对面是怎么连上的；
|
||||
3. **可测性**：解码逻辑能脱离真连接单独测试。
|
||||
|
||||
做法是把它抽成一个函数边界（输入目标描述、返回 `quinn::Connection`），
|
||||
**不是**定义 trait 或枚举。
|
||||
|
||||
### 6.3 对产品的影响（必须写进对外说明）
|
||||
|
||||
| 网络环境 | 能否连接 |
|
||||
|---|---|
|
||||
| 双方任一侧无公网 IPv6，且处于 CGNAT 后 | ❌ **无法连接，无解** |
|
||||
| 双方都有公网 IPv6 | ✅ 可直连 |
|
||||
| 局域网 / 双方都有可打洞的公网 IPv4 | ✅ 可直连 |
|
||||
| 对称 NAT（双方） | ❌ 通常不通 |
|
||||
|
||||
**这不是 bug，是已知边界。** 用户在外网连不上时，UI 必须给出可执行提示
|
||||
（`PunchFailReason::user_hint()` 已提供文案），不能停留在"连接中"。
|
||||
|
||||
---
|
||||
|
||||
## 7. 分期路线（不要一次做 6 个组合）
|
||||
|
||||
| 里程碑 | 内容 | 预估 | 依赖 |
|
||||
|---|---|---|---|
|
||||
| **M1** | Win↔Win 打通：信令 + 打洞 + 证书钉扎 | 2-3 周 | 协议已就绪 |
|
||||
| **M2** | Win 一体化：agent-core 抽库、会话状态机、前台/后台双进程 | 2-3 周 | M1 |
|
||||
| **M3** | Win 被控服务化（开机即在线、锁屏可连、UAC 处理） | 2 周 | M2 |
|
||||
| **M4** | Linux 被控（X11 先行，Wayland/PipeWire 预留） | 2-3 周 | M2，A 在做 |
|
||||
| **M5** | 桌面 UI（Tauri v2 壳 + 原生视频表面） | 3 周 | M2 |
|
||||
| **M6** | **Android 主控**（MediaCodec + UniFFI + 触摸映射） | 3-4 周 | M5 |
|
||||
| **M7** | **iOS 主控**（VideoToolbox + SwiftUI/UniFFI） | 3-4 周 | M6 |
|
||||
| **M8** | Linux 主控（复用 M5 的桌面 UI） | 1 周 | M5 |
|
||||
| 穿插 | 音频(Opus) / 文件传输 / 多屏 | 各 1-2 周 | — |
|
||||
|
||||
**先做 Android 再做 iOS**：不用苹果开发者账号、真机调试方便、热重载快、审核风险低。iOS 放最后。
|
||||
|
||||
---
|
||||
|
||||
## 8. ⚠️ 必须说清楚的人力问题
|
||||
|
||||
现有团队 **2-3 人**，目前连"Win↔Win 打通"都还没做完。而这份需求是：
|
||||
|
||||
- 2 个被控平台（抓屏 + 编码 + 注入 + 服务化）× 各自两套显示服务器（Linux X11/Wayland）
|
||||
- 4 个主控平台（解码 + 渲染 + 输入 + UI）× 各自一套原生集成
|
||||
- 外加信令、打洞、音频、文件、多屏、账号体系
|
||||
|
||||
**这是 6-10 人年的工作量，2-3 人做不完。** 不是唱衰，是排期事实——Android + iOS 两个平台的 UI 和平台集成，单独拿出来就够一个人全职做半年。
|
||||
|
||||
可行的收缩方式（选一个）：
|
||||
|
||||
1. **砍掉 iOS**（苹果审核 + 真机调试 + 开发者账号成本最高，收益最小）
|
||||
2. **移动端只做 Android**，iOS 用 Web 端替代（但 Web 端无法硬解，体验差）
|
||||
3. **桌面优先，做深做透**，移动端放到"产品验证之后"，先招人
|
||||
4. 移动端主控**外包**，核心协议和桌面端自己掌握
|
||||
|
||||
我的建议是 **3 或 1**：先把 Win/Linux 两端做成一款真的好用的产品（含打洞、服务化、音频、文件），移动端等有用户了再投入。现在同时铺 6 个平台，最大概率是每个都半成品。
|
||||
|
||||
---
|
||||
|
||||
## 9. 给 A / B 的分工调整建议
|
||||
|
||||
| 人 | 主责 | 说明 |
|
||||
|---|---|---|
|
||||
| A | `platform/` 被控侧：Win 抓屏/编码/注入/服务化 + Linux 被控 | 与现状一致 |
|
||||
| B | `proto/` + `conn/` + `session/` + `viewer-core/` + 桌面 UI | 与现状一致 |
|
||||
| — | **移动端（Android/iOS）** | **无人** ← 这是缺口 |
|
||||
|
||||
现有分工在"桌面 + 协议"上是自洽的，**移动端是纯增量，必须补人或砍需求**。
|
||||
|
||||
---
|
||||
|
||||
## 10. 开放问题（需要拍板）
|
||||
|
||||
> §10 第 1 条已于 2026-10-28 裁决，见 §6。
|
||||
|
||||
1. ~~中继：确认走 §6 的 A 方案（v6 优先、留接口不实现）？~~
|
||||
**已裁决：不做中继、不留接口、v6 优先、但保留连接层边界。详见 §6.1 / §6.2。**
|
||||
2. 移动端：先 Android 还是直接砍 iOS？
|
||||
3. UI 框架：确认 Tauri v2，还是先原生桌面 + 后面再说移动？
|
||||
4. 账号体系：现在是"输验证码"，要支持设备列表就得有账号/设备 ID 体系——什么时候做？
|
||||
5. 被控服务化后，主控端要不要支持"无人值守开机连接"（有安全含义，需要固定密码 + 白名单）？
|
||||
-124
@@ -1,124 +0,0 @@
|
||||
# 打洞与信令设计（无中继方案）
|
||||
|
||||
> 状态：协议契约已落在 `common/src/signaling.rs`，已合入 main（`703eb64`）。
|
||||
> 决策：**不做中继（无 TURN / coturn）**，服务器只保留信令 + STUN 职能。
|
||||
> **裁决确认（2026-10-28）**：公司不提供中继服务器，本决策为最终结论，
|
||||
> 同时**不预留中继接口**（理由见《多平台架构设计.md》§6.1 / §6.2）。
|
||||
> 关联：`构建运行手册.md` §7 第 4 项、`Git协作规范.md` 二·B 第 ③ 项。
|
||||
|
||||
## 1. 结论先行：信令不是中继
|
||||
|
||||
| | 信令服务器 | 中继服务器（已砍掉） |
|
||||
|---|---|---|
|
||||
| 承载媒体流量 | ❌ 不承载 | ✅ 全部承载 |
|
||||
| 生命周期 | 只在建连那几秒 | 整个会话期间 |
|
||||
| 成本 | 一台最低配 VPS 足够 | 视频流带宽，主要成本项 |
|
||||
| 必要性 | **必须**（无它则无法交换地址） | 可选（可用率兜底） |
|
||||
|
||||
砍掉中继 ≠ 不要服务器。信令必须保留，因为它解决的是死循环：双方都在 NAT 后，
|
||||
各自只知道自己内网地址，**"自己的公网映射"这个信息只有第三方发包过来时才能看见**。
|
||||
|
||||
## 2. 无中继的代价：可用率，不是功能
|
||||
|
||||
打洞成功率由 NAT 类型决定，与代码质量基本无关：
|
||||
|
||||
| NAT 类型 | 典型场景 | 无中继 |
|
||||
|---|---|---|
|
||||
| Full cone / Restricted cone | 家用路由器默认 | ✅ 很高 |
|
||||
| Port-restricted cone | 家用宽带常见 | ✅ 需同时发包 |
|
||||
| Symmetric NAT | 部分企业网关 | ⚠️ 仅端口可预测时可行 |
|
||||
| CGNAT（`100.64.0.0/10`） | 移动/社区宽带 | ❌ 无解（IPv6 除外） |
|
||||
|
||||
预期 P2P 成功率 **80–90%**，其余连不上。
|
||||
|
||||
**产品边界（需写进对外说明）**：目标用户框定为「家用宽带 + IPv6 可用 + 可开 UPnP」；
|
||||
企业网络 / CGNAT 标注为不支持。这样剩余失败就不是 bug，是已知边界。
|
||||
|
||||
## 3. 打洞流程（协议见 `common/src/signaling.rs`）
|
||||
|
||||
```
|
||||
agent 信令服务器 viewer
|
||||
|-- Register(info) ------->|<------- Register(info) -----|
|
||||
|<-- Registered(observed) -|------ Registered(observed) ->| observed = 公网映射（等价一次 STUN binding）
|
||||
|<-- PeerReady(peer info) -|------ PeerReady(peer info) -->| + 打洞前景 outlook
|
||||
|<-- PunchPlan(start_in) --|------ PunchPlan(start_in) --->| 两侧 start_in 不同，绝对时刻对齐
|
||||
| （双方同时向对端候选列表发包，多轮） |
|
||||
|<========= 打通：UDP 直连 → QUIC 握手 + 证书钉扎 =========>|
|
||||
|--- PunchReport(Connected) ->|<---- PunchReport ----------|
|
||||
```
|
||||
|
||||
### 3.1 候选地址与优先级
|
||||
|
||||
`Candidate::score()`：IPv6（+100）> UPnP 显式映射(90) > 本机地址(80) > 公网反射(60) > 端口预测(40)。
|
||||
|
||||
**IPv6 是无中继策略的最强后盾** —— 双方都有公网 v6 就是真直连，成功率≈100%，
|
||||
因此候选列表按 v6 → v4 排序，`EndpointInfo` 两类都带。
|
||||
|
||||
### 3.2 同时发包的时间对齐（不需要时钟同步）
|
||||
|
||||
两侧 RTT 不同，若下发同一个延迟，RTT 大的一方晚发包就打不上。服务器按
|
||||
`punch_start_delay_ms(own_rtt, max_rtt, margin)` 给两侧算不同的 `start_in_ms`：
|
||||
|
||||
```
|
||||
start_in_ms = max_rtt_ms / 2 + margin_ms - own_rtt_ms / 2
|
||||
```
|
||||
|
||||
`start_in_ms` 是**相对消息送达**的延迟，所以两端时钟不需要同步，误差≈单程抖动。
|
||||
|
||||
### 3.3 保活
|
||||
|
||||
NAT 的 UDP 映射默认 30s 左右超时，洞是出站包维持的，超时后静默关闭
|
||||
（表现为"连上了一会儿就断"）。`keepalive_secs()` 取双方 `binding_timeout_s`
|
||||
较小值折半，夹在 5–60s。
|
||||
|
||||
## 4. 提高成功率的实现清单（按性价比）
|
||||
|
||||
1. **IPv6 优先**：候选排序已实现；agent 侧需枚举全局 v6 地址填 `CandidateKind::Host`。
|
||||
2. **UPnP / NAT-PMP / PCP**：agent 启动时就尝试为 `PUNCH_UDP_PORT(47800)` 申请映射，
|
||||
成功后以 `CandidateKind::Upnp` 补报（走 `ClientMsg::UpdateCandidates`）。
|
||||
一次成功就把对称 NAT 变成可直连，收益最大。
|
||||
3. **双方同时发包**：见 §3.2，这是打洞能成的关键。
|
||||
4. **端口预测**：`PortAllocation::Sequential/Preserved` 时才值得做，`Predicted` 候选由
|
||||
`sanitize_candidates` 统一清洗（去重、丢端口 0、上限 32 个）。
|
||||
5. **多轮重试**：`rounds=3 / 200ms`，一次不通不代表不通。
|
||||
|
||||
**不要走 TCP 打洞**：QUIC 只跑 UDP，TCP simultaneous open 成功率低，性价比不值得。
|
||||
|
||||
## 5. 实现约束（两端各自注意）
|
||||
|
||||
- **别拿 QUIC 握手当探测**：`quinn::Endpoint::connect` 打不通会持续重试，失败判定被
|
||||
idle timeout 拖很久。应先用同一 UDP socket 发轻量探测包确认通路（秒级），
|
||||
确认后再 `connect`；端口扫描也放这一层，便宜且可并行。
|
||||
- **UDP socket 复用**：打洞用的 socket 要和后续 QUIC 用同一个，否则洞不在同一个映射上。
|
||||
`quinn::Endpoint` 支持同时 `accept()` 与 `connect()`，可以在一个 Endpoint 上做完整流程。
|
||||
- **固定端口**：`47800` 对端口预测和用户手动端口转发都友好，不要随机端口。
|
||||
|
||||
## 6. 安全（本次必须同步改，否则公网不可用）
|
||||
|
||||
1. **证书钉扎**：`viewer/src/main.rs` 的 `SkipVerify` 必须替换为对
|
||||
`EndpointInfo.cert_fingerprint` 的校验（`CertFingerprint` 已定义 hex 编解码）。
|
||||
2. **认证**：`ClientMsg::Register.auth` 必须是一次性验证码（替代固定 `123456`/`888888`），
|
||||
带过期与失败限速，否则公网等于没有认证。
|
||||
3. **候选列表视为不可信输入**：一律经 `sanitize_candidates` 后再发包。
|
||||
|
||||
## 7. 失败必须有明确出口（无中继的用户体验下限）
|
||||
|
||||
没有中继兜底，就必须把"为什么连不上"讲清楚，不允许用户停在"连接中"。
|
||||
`PunchFailReason::user_hint()` 已给出每种原因对应的可执行建议，例如：
|
||||
|
||||
- `Cgnat` → "当前处于运营商级 NAT（大内网），无法直连。请改用 IPv6 网络……"
|
||||
- `SymmetricNat` → "请在被控端路由器开启 UPnP 后重试。"
|
||||
- `UdpBlocked` → "当前网络的防火墙拦截了 UDP 出站，请切换手机热点后重试。"
|
||||
|
||||
`punch_outlook()` 还能在**开打之前**就给出前景（`Likely/Possible/Unlikely/Hopeless`），
|
||||
`Hopeless`（CGNAT）时可直接跳过打洞轮次并立刻提示，不让用户白等。
|
||||
|
||||
## 8. 后续动作
|
||||
|
||||
| 顺序 | 事项 | 负责 |
|
||||
|---|---|---|
|
||||
| 1 | `common/src/signaling.rs` 过审合入 main | B 提 PR / A review |
|
||||
| 2 | 信令服务最小实现（WSS/443，配对 + 回推 observed + 下发 PunchPlan） | B |
|
||||
| 3 | agent：候选收集（v6/Host/UPnP/Reflexive）、同步打洞、保活 | A |
|
||||
| 4 | viewer：候选收集、同步打洞、改为证书钉扎 | B |
|
||||
| 5 | 双机联调：家庭宽带 / 手机热点 / CGNAT 三种网络各跑一轮 | A + B |
|
||||
@@ -1,135 +0,0 @@
|
||||
# 打洞连通性验证记录
|
||||
|
||||
> 目的：用最小成本确认「自研 P2P 打洞」在本项目目标网络下是否可行，再决定要不要投入实现。
|
||||
> 工具：`tools/punch_probe.py`（探针）、`tools/rendezvous.py`（极简信令服务器）
|
||||
> 结论：**本机线路为锥形 NAT + 端口保留，打洞可行性高。**
|
||||
|
||||
---
|
||||
|
||||
## 1. 测试环境
|
||||
|
||||
| 角色 | 地址 | 说明 |
|
||||
|---|---|---|
|
||||
| 信令服务器 | `47.98.245.66:47999/udp` | 阿里云 ECS，Ubuntu 24.04.2，python 3.12.3 |
|
||||
| 客户端 | 出口 `116.3.201.115` | 联通线路，位于 NAT 之后 |
|
||||
| 工具 | `tools/rendezvous.py` + `tools/punch_probe.py` | 纯 Python 标准库 |
|
||||
|
||||
部署位置：`/root/uu-punch-poc/`（与服务器上其他服务完全隔离）
|
||||
|
||||
---
|
||||
|
||||
## 2. 验证过程与结果
|
||||
|
||||
| # | 验证项 | 结果 |
|
||||
|---|---|---|
|
||||
| 1 | 信令服务器回环注册 | ✅ 能注册、能记录 |
|
||||
| 2 | 服务器进程/监听 | ✅ `0.0.0.0:47999`，长期运行稳定 |
|
||||
| 3 | **跨公网注册** | ✅ 服务器日志出现 `116.3.201.115:47800` |
|
||||
| 4 | **双方配对与地址回推** | ✅ `配对成功：116.3.201.115:47800 <==> :47801` |
|
||||
| 5 | 同一 NAT 内两实例互打 | ❌ 失败（见 §4.1，属 hairpinning 限制） |
|
||||
| 6 | **NAT 映射行为判定** | ✅ **端点无关映射 + 端口保留**（见 §3） |
|
||||
|
||||
---
|
||||
|
||||
## 3. 关键结论：NAT 类型 = 锥形 + 端口保留
|
||||
|
||||
用**同一个本地端口 47800** 分别打两个完全不同的目标，观察公网映射：
|
||||
|
||||
| 目标 | 观察到的公网映射 |
|
||||
|---|---|
|
||||
| Google STUN（境外） | `116.3.201.115:47800` |
|
||||
| 自建信令服务器 `47.98.245.66:47999` | `116.3.201.115:47800` |
|
||||
|
||||
**不同目标 → 同一个公网端口 → 端点无关映射（Endpoint-Independent Mapping，即锥形 NAT），且端口原样保留。**
|
||||
|
||||
按打洞成功率排序：
|
||||
|
||||
```
|
||||
锥形 + 端口保留 ← 本机（最优）
|
||||
锥形 + 端口改写 成功率高
|
||||
对称 NAT 困难，需端口预测
|
||||
CGNAT 基本无解
|
||||
```
|
||||
|
||||
**推论**：信令服务器观察到的地址，对该客户端是**可复用的真实映射**，对端直接向它发包即可打洞，无需端口预测。这是最理想的情形。
|
||||
|
||||
复现命令：
|
||||
|
||||
```powershell
|
||||
# 一定要带 --stun-port，用与打洞相同的端口，否则映射不可比
|
||||
py tools\punch_probe.py --stun --stun-port 47800
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 踩到的坑（Rust 实现必须处理）
|
||||
|
||||
### 4.1 hairpinning：同一公网 IP 下两实例互打会失败
|
||||
|
||||
两个客户端都在 `116.3.201.115` 后面，互相打对方的公网地址，需要路由器支持 **NAT 环回（hairpinning）**。本线路不支持。
|
||||
|
||||
**这不代表真实打洞会失败**——真实场景是两台机器位于**不同**公网 IP 之后。此测试只能验证"信令配对"链路，不能验证穿透能力。
|
||||
|
||||
### 4.2 Windows：ICMP 端口不可达会打断接收循环
|
||||
|
||||
对方尚未启动时发包 → 收到 ICMP port unreachable → 下一次 `recv` 抛 `WSAECONNRESET(10054)`。若把它当致命错误 `break`，接收循环就此死掉，之后对方真打进来也收不到。
|
||||
|
||||
处理方式（两层）：
|
||||
|
||||
```python
|
||||
sock.ioctl(socket.SIO_UDP_CONNRESET, False) # 1) 关闭该行为
|
||||
# 2) 收到该错误时 continue，绝不能 break
|
||||
```
|
||||
|
||||
**Rust 对应**：`tokio` 的 `recv_from` 会返回 `ConnectionReset`，同样不能 `?` 或 `break`。
|
||||
|
||||
### 4.3 发完就退出会漏掉晚到的回包
|
||||
|
||||
对方可能比自己晚启动。发完最后一包立即关闭 socket，会把"晚到的成功"误判为失败。
|
||||
→ 发完后保留 grace 期（默认 2.5s）继续监听。
|
||||
|
||||
### 4.4 部署层的两道防火墙
|
||||
|
||||
| 层 | 现象 | 处理 |
|
||||
|---|---|---|
|
||||
| 主机 ufw | `INPUT DROP`，无规则则静默丢弃 | `ufw allow 47999/udp` |
|
||||
| 云安全组 | 包到不了网卡（tcpdump 零捕获） | 控制台加 UDP 入方向规则 |
|
||||
|
||||
**排查手法**：`tcpdump -n -i any udp port <port>` 零捕获 = 包在到达主机前就被丢，问题在云安全组而不在主机防火墙。
|
||||
|
||||
> 注意：首次配置时误把安全组规则设成了 **TCP**，UDP 包仍被丢弃。协议类型必须选 UDP。
|
||||
|
||||
### 4.5 假阳性：信令回复被误判为"打通"
|
||||
|
||||
客户端在拿到对端地址前会反复注册，服务器的 `PEER` / `WAIT` 回复可能在 `discover_peer` 返回之后才到达。若接收线程不做区分，会把**服务器发来的回复**当成"对方打进来了"，输出假的"✅ 打洞成功"。
|
||||
|
||||
实测踩到的形态：报告的"对方源地址"是**信令服务器的 IP:端口**（`47.98.245.66:47999`），包长 22 字节——而真正的打洞包固定是 **16 字节**（`PUNCHv1|` + 8 字节时间戳）。
|
||||
|
||||
**判据（Rust 实现必须照做）**：
|
||||
|
||||
1. 只认以协议魔数开头的包，长度必须是约定值
|
||||
2. 显式排除信令服务器的 `IP:端口`
|
||||
3. 结论里如果出现"非打洞包 N 个"，说明收到了信令回复，**不能**据此判定成功
|
||||
|
||||
---
|
||||
|
||||
## 5. 尚未验证
|
||||
|
||||
| 项 | 为什么重要 | 怎么验 |
|
||||
|---|---|---|
|
||||
| **两台不同网络的机器互打** | 唯一能证明真实穿透能力的测试 | 一台机器 + 一台连手机热点的笔记本 |
|
||||
| 手机热点（CGNAT）场景 | 决定"无中继"方案对移动端是否成立 | 同上，热点那端即为 CGNAT |
|
||||
| IPv6 直连 | 若两端都有公网 v6，可绕开 NAT | `--peer [v6地址]:端口` |
|
||||
| 对端 NAT 是否同样友好 | 打洞需要两端配合 | 在对方网络重复 §3 的映射判定 |
|
||||
|
||||
**下一步优先级：手机热点组合 > IPv6 > 其他。**
|
||||
|
||||
---
|
||||
|
||||
## 6. 对实现的影响
|
||||
|
||||
1. **信令服务器的职责就此确认**：注册、配对、回推观察到的源地址。不承载媒体，建连后即退出。
|
||||
2. **信令与打洞必须复用同一个 socket**：服务器看到的映射端口，必须就是打洞时使用的端口，否则映射不可复用。
|
||||
- Rust 实现中：该 socket 需继续交给 `quinn::Endpoint` 使用（一次绑定，三处复用：信令 / 打洞 / QUIC）。
|
||||
3. **失败判定要快**：先做 NAT 行为探测，若判定为对称 NAT 直接给出结论，不必耗完多轮。
|
||||
4. **保活**：NAT 表项会超时，建连后需按 `binding_timeout_s` 折半做 keepalive（协议中已定义 `keepalive_secs()`）。
|
||||
@@ -1,17 +1,17 @@
|
||||
# MVP 远程控制 Demo · 构建运行手册
|
||||
|
||||
> 项目路径：`D:\code\remote\`
|
||||
> 形态：Rust workspace：`common`（协议）、`agent-core`（平台无关会话层，当前供 linux-agent 使用）、`agent`（Windows 被控端）、`linux-agent`（Linux 被控端）、`viewer`（主控端）
|
||||
> 形态：Rust workspace：`common`（协议）、`agent-core`（平台无关会话层）、`agent`（Windows 被控端）、`linux-agent`（Linux 被控端）、`viewer`（主控端）
|
||||
|
||||
## 1. 功能范围（量级一 Demo）
|
||||
|
||||
- ✅ DXGI Desktop Duplication 全屏采集 + NVENC 硬编（无 N 卡时自动回落 openh264 软编）
|
||||
- ✅ QUIC 加密传输（自签名证书，传输层已加密）
|
||||
- ✅ DXGI Desktop Duplication 全屏采集（GPU 拷贝，脏区/硬编码留待二期）
|
||||
- ✅ LZ4 压缩 + QUIC 加密传输（自签名证书，传输层已加密）
|
||||
- ✅ 验证码认证（默认 123456）
|
||||
- ✅ 键鼠远程控制（SendInput 注入，虚拟桌面绝对坐标 + 扫描码）
|
||||
- ✅ 桌面静止时低频重发（保持画面新鲜）
|
||||
- ✅ Linux 被控端（X11 采集 + XTest 注入，见第 8 节）
|
||||
- ❌ 暂无：P2P 打洞/中继（当前仅局域网/可直连地址）、音频、文件传输
|
||||
- ✅ Linux 被控端（X11 采集 + XTest 注入，与 Windows 端共用同一套协议与会话层）
|
||||
- ❌ 暂无：硬件编码、P2P 打洞/中继（当前仅局域网/可直连地址）、音频、文件传输
|
||||
|
||||
## 2. 环境要求
|
||||
|
||||
@@ -20,34 +20,31 @@
|
||||
- `rustup default stable-x86_64-pc-windows-gnu`，装到 `%USERPROFILE%\.cargo\bin`
|
||||
- MinGW-w64：`D:\mingw64\bin`（gcc 16.2.0）
|
||||
- 构建时 PATH 需同时包含上面两个目录（缺 MinGW 会在链接 openh264 的 C++ 时失败）
|
||||
- **cargo 网络**：国内直连 crates.io 会超时，且 **cargo 不读 Windows 系统代理**，必须显式告诉它。
|
||||
已在本机 `%USERPROFILE%\.cargo\config.toml` 写入（Clash 之类混合端口，按实际改）：
|
||||
- **cargo 源**：国内直连 crates.io 会超时（cargo 不读系统代理），需配置镜像。已在本机
|
||||
`%USERPROFILE%\.cargo\config.toml` 写入 USTC 源：
|
||||
|
||||
```toml
|
||||
[http]
|
||||
proxy = "http://127.0.0.1:7897"
|
||||
[source.crates-io]
|
||||
replace-with = "ustc"
|
||||
[source.ustc]
|
||||
registry = "sparse+https://mirrors.ustc.edu.cn/crates.io-index/"
|
||||
```
|
||||
|
||||
⚠️ **不要用国内镜像做源替换**：实测 USTC 镜像的索引 `cksum` 与其实际下发的 `.crate`
|
||||
字节**不一致**（`nvenc 0.1.0`：索引写 `3227c3b0…`，包体实际 `ab6fde0e…`），
|
||||
会报 `failed to verify the checksum of nvenc`；且源替换会改变 registry 身份，
|
||||
使 `Cargo.lock` 里已有的 checksum 与全部缓存失配。**走官方源 + 代理最稳。**
|
||||
|
||||
> 注：旧版本手册写的 `D:\wordkb\...\uurev\tools\rust` 路径已不存在，按上面重新准备工具链即可。
|
||||
|
||||
## 3. 构建
|
||||
|
||||
```bash
|
||||
# Git Bash 写法：PATH 需含 rustup 的 cargo 与 MinGW-w64
|
||||
export PATH="$USERPROFILE/.cargo/bin:/d/mingw64/bin:$PATH"
|
||||
cd /d/code/remote # 换成你自己的工作目录 / worktree
|
||||
export RUSTUP_HOME="D:\\wordkb\\2026-10-22-15-52-47\\uurev\\tools\\rust\\rustup"
|
||||
export CARGO_HOME="D:\\wordkb\\2026-10-22-15-52-47\\uurev\\tools\\rust\\cargo"
|
||||
export PATH="/d/wordkb/2026-10-22-15-52-47/uurev/tools/rust/cargo/bin:/d/wordkb/2026-10-22-15-52-47/uurev/tools/mingw64/bin:$PATH"
|
||||
cd /d/wordkb/2026-10-22-15-52-47/mvp-remote
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
产物：
|
||||
- 被控端 `target/release/agent.exe`
|
||||
- 主控端 `target/release/viewer.exe`
|
||||
- 编码基准（可选）`target/release/encbench.exe`
|
||||
|
||||
## 4. 运行（两台机器，或同机双开验证）
|
||||
|
||||
@@ -83,30 +80,6 @@ viewer.exe --addr <被控端IP> --port 47800 --code 888888
|
||||
4. 高 DPI/多屏：当前只采集主输出（EnumOutputs(0)）。
|
||||
5. 主控端窗口固定为远端分辨率，颜色序 BGRA→0RGB 若异常可在 `blit()` 一键翻转。
|
||||
|
||||
### 6.1 待修问题（已定位，尚未修，均在 Windows `agent/`）
|
||||
|
||||
1. **错误验证码时 `AuthFail` 送不达**：`agent/src/main.rs` 写完 `AuthFail` 后立刻
|
||||
`send.finish()` 并返回，进程退出会把 QUIC 缓冲区里尚未发出的数据一起带走，
|
||||
客户端只会看到 `connection lost` 而不是 `AuthFail`。
|
||||
修法：`finish()` 后等对端读到并挂断（`conn.closed()` + 超时）再退出。
|
||||
|
||||
2. **软编路径下静止桌面仍不响应关键帧请求**（`04c02d9` 只修了 NVENC 那一半）：
|
||||
`agent/src/capture.rs` 的 `Ok(false)`（画面未变）分支里，`force_key` 被
|
||||
`keyreq.swap` 消费后，只有 `EncoderKind::Nvenc` 分支拿它去重编 IDR；
|
||||
`EncoderKind::Soft` 分支仍然只是「重发最后一帧码流副本」，标志被直接丢弃。
|
||||
后果：**无 N 卡 / 驱动异常（走软编兜底）的机器上，解码端一旦出错，静止画面下
|
||||
永远拿不到 IDR，表现为黑屏。**
|
||||
修法：软编路径同样保留最近一帧裸像素，`force_key` 时重编一个真关键帧。
|
||||
|
||||
3. **单次 `KeyFrameReq` 可能被静默吞掉**：同一个 `keyreq.swap(false, …)` 写在了
|
||||
`if last_sent.elapsed() > 500ms` 的**外面**，因此请求若落在距上次发送 500ms 之内，
|
||||
标志已被消费但分支不进入，该次请求就丢了，必须等下一次请求才可能生效
|
||||
（主控端目前周期性重发，故表现为延迟而非永久黑屏）。
|
||||
修法：把 `swap` 移进时间闸内，或先在闸外预判、进闸后再消费。
|
||||
|
||||
> 第 1、2 条在 `feature/agent-linux` 分支的 `agent-core` 里已经实现（**Linux 侧生效**），
|
||||
> 但按"不与他线争抢 `agent/`"的约定，**本期没有改动 Windows 代码**。
|
||||
|
||||
## 7. 下一阶段任务队列（Phase 1 剩余）
|
||||
|
||||
1. H.264：接 openh264（软编兜底）→ NVENC/QSV（硬编优先）
|
||||
@@ -117,25 +90,21 @@ viewer.exe --addr <被控端IP> --port 47800 --code 888888
|
||||
|
||||
## 8. Linux 被控端（linux-agent）
|
||||
|
||||
### 8.1 现状与边界（重要）
|
||||
### 8.1 架构：会话层抽到 agent-core
|
||||
|
||||
本期**只在新增文件里做，未改动 Windows `agent/`**，原因见下：
|
||||
平台差异只留在采集/注入两个模块里，QUIC 握手、认证、编码、丢帧、输入分发全部共用，
|
||||
**主控端 viewer 不需要任何改动**：
|
||||
|
||||
| crate / 文件 | 职责 | 当前使用者 |
|
||||
| crate / 文件 | 职责 | 平台 |
|
||||
|---|---|---|
|
||||
| `agent-core/src/lib.rs` | QUIC/TLS 握手、传输调优、验证码认证、会话/select 循环、latest-wins 单槽、输入分发，以及软件编码视频管线（openh264） | **仅 linux-agent** |
|
||||
| `agent-core/src/lib.rs` | QUIC/TLS 握手、验证码认证、H.264 编码下行、latest-wins 丢帧、输入分发 | 共用 |
|
||||
| `agent/src/capture.rs` | DXGI Desktop Duplication 采集 | Windows |
|
||||
| `agent/src/input.rs` | SendInput 注入 | Windows |
|
||||
| `linux-agent/src/capture.rs` | X11 `GetImage` 采集 + 整帧变更检测 | Linux |
|
||||
| `linux-agent/src/input.rs` | XTest 注入 + 扫描码→X11 keycode 映射 | Linux |
|
||||
| `agent/`（Windows） | 保留原有自持会话循环 + NVENC/软编双路编码，**本期未动** | Windows |
|
||||
|
||||
**为什么 Windows 端没有一并接过来**：`agent/` 的 NVENC 路径是「GPU 内 BGRA→H.264 零转换零拷贝」，
|
||||
编码器必须直接吃 DXGI 的 GPU 纹理；而 `agent-core` 目前的接缝是 `ScreenCapturer`（返回 CPU 上的
|
||||
BGRA 字节），硬接会把零拷贝优势废掉。正确的接缝应是「平台视频管线产出**已编码帧**」
|
||||
（`VideoSource` 语义），待 `agent/` 由 NVENC 那条线稳定后再重切，届时应把 `openh264` 从
|
||||
`agent-core` 移到各平台 crate。
|
||||
|
||||
在此之前 Windows 与 Linux 各持一份会话循环，属于**有意的过渡态**：`common/` 协议改动需要两处同步
|
||||
（例如本期跟着补的 `Pong{lost,sent}`）。
|
||||
两端靠 `ScreenCapturer` / `InputInjector` 两个 trait 注入，编码档位（5Mbps / 30fps / 关键帧周期 300）
|
||||
与线格式完全一致；协议**未做任何改动**。
|
||||
|
||||
### 8.2 构建
|
||||
|
||||
@@ -146,7 +115,7 @@ export PATH="$USERPROFILE/.cargo/bin:/d/mingw64/bin:$PATH" # Git Bash 写法
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
Linux 侧显式指定包（跨平台交叉编译不推荐，用容器最省事）：
|
||||
Linux 侧显式指定包（交叉编译不推荐，用容器最省事）：
|
||||
|
||||
```bash
|
||||
cargo build --release -p linux-agent
|
||||
@@ -167,19 +136,9 @@ viewer.exe --addr <被控机IP> --port 47800 --code 123456
|
||||
bash scripts/linux-smoke.sh
|
||||
```
|
||||
|
||||
脚本自建 Xvfb 虚拟屏后跑**单测 + 端到端**两层：
|
||||
|
||||
- 单测（不需要 X server）：键码映射、坐标换算等纯逻辑；
|
||||
- 端到端：X11 采集 → openh264 编码 → QUIC 下行解帧、验证码认证（含**错误验证码被拒**）、
|
||||
Ping/Pong（含下行丢包统计字段）、静止桌面下的关键帧恢复，以及**输入注入实证**——
|
||||
注入鼠标移动后，从另一条 X11 连接读回指针真实坐标。
|
||||
|
||||
> 单测那层别省：坐标换算的取整边界就是靠它才发现的（集成测试有 ±2 像素容差，
|
||||
> 期望值写错也照样绿）。
|
||||
|
||||
> 静止关键帧那条用例做过**变异测试**：把修复退回会失败、恢复后通过，确认断言不是摆设。
|
||||
> 注意扰动量必须小（画 64x64 小方块）——整屏刷白会触发编码器场景切换自动 IDR，
|
||||
> 于是"重发旧帧"和"真重编关键帧"拿到同样的字节，用例就失去区分度。
|
||||
脚本自建 Xvfb 虚拟屏后跑端到端测试，覆盖：X11 采集 → openh264 编码 → QUIC 下行解帧、
|
||||
验证码认证（含**错误验证码被拒**）、Ping/Pong、静止桌面下的关键帧恢复（断言拿到 IDR），
|
||||
以及**输入注入实证**——注入鼠标移动后，从另一条 X11 连接读回指针真实坐标。
|
||||
|
||||
### 8.5 键码映射（两端最关键的差异）
|
||||
|
||||
@@ -193,7 +152,7 @@ bash scripts/linux-smoke.sh
|
||||
### 8.6 Linux 侧已知限制
|
||||
|
||||
1. 仅支持 32bpp TrueColor 视觉（绝大多数桌面满足），其他格式启动即明确报错而非静默出错；
|
||||
2. 采集走核心协议 `GetImage`，每帧经 X socket 全量传输，静止桌面也会持续轮询抓帧；
|
||||
**MIT-SHM 共享内存 + XDamage 脏区**是下一步优化（锚点已写在 `linux-agent/src/capture.rs` 顶部注释）；
|
||||
2. 采集走核心协议 `GetImage`，每帧经 X socket 全量传输；**MIT-SHM 共享内存 + XDamage 脏区**
|
||||
是下一步优化（锚点已写在 `linux-agent/src/capture.rs` 顶部注释）；
|
||||
3. 仅覆盖 X11 / XWayland；纯 Wayland（无 XWayland）需另走 PipeWire 门户；
|
||||
4. 键位表按 Xorg evdev 标准 keycode 编写，自定义键盘布局可能需要调整。
|
||||
Reference in new issue
Block a user