feat(agent): 抽出 agent-core 并新增 Linux 被控端(X11 + XTest)
- 新增 agent-core:平台无关会话层(QUIC/TLS 握手、验证码认证、H.264 编码下行、 latest-wins 丢帧、输入分发),平台差异经 ScreenCapturer / InputInjector 两个 trait 注入;Windows agent 一并改接过来,避免两端各写一份握手/认证造成协议漂移 - 新增 linux-agent:X11 GetImage 采集 + 整帧变更检测;XTest 注入 + 扫描码→X11 keycode 映射(主键区 = scancode+8,E0 扩展键必须查表,含单元测试) - 协议零改动:复用 Set-1 扫描码与虚拟桌面归一化坐标,viewer 无需任何修改 - 修复:静止桌面下关键帧请求无人响应——解码端一旦出错将永远无法恢复。 现在保留最近一帧裸像素,收到 KeyFrameReq 时重编一个真关键帧 - 修复:错误验证码时 AuthFail 送不达——进程退出会带走 QUIC 缓冲区内尚未发出的 数据,客户端只能看到 connection lost;现等对端挂断再退出 - 新增端到端冒烟测试 tests/smoke.rs 与 scripts/linux-smoke.sh(自建 Xvfb,无需 图形桌面):覆盖采集→编码→QUIC 解帧、认证(含错误码拒绝)、Ping/Pong、 静止桌面关键帧恢复,以及输入注入实证(注入后从另一条 X11 连接读回指针坐标) - 该关键帧用例已做变异测试:退回修复会红,恢复后转绿,确认断言有效 - 手册:修正已失效的工具链路径,补充 cargo 镜像说明与 Linux 章节 验证: - Windows: cargo build --release 通过,产物 agent.exe 9.32MB / viewer.exe 8.73MB - Linux(容器): cargo test -p linux-agent 通过,2 passed
This commit is contained in:
parent
e0a076d570
commit
0f3f1cf7f7
15
.gitattributes
vendored
Normal file
15
.gitattributes
vendored
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
# 统一换行符,避免跨平台踩坑
|
||||||
|
# shell 脚本必须 LF:Windows 上若被 checkout 成 CRLF,容器/Linux 里执行会报
|
||||||
|
# "$'\r': command not found",或被 sed/grep 当成带 \r 的内容
|
||||||
|
*.sh text eol=lf
|
||||||
|
|
||||||
|
# Windows 侧脚本保持 CRLF
|
||||||
|
*.ps1 text eol=crlf
|
||||||
|
*.bat text eol=crlf
|
||||||
|
|
||||||
|
# 二进制/生成物不参与换行转换
|
||||||
|
*.exe binary
|
||||||
|
*.dll binary
|
||||||
|
*.png binary
|
||||||
|
*.jpg binary
|
||||||
|
*.ico binary
|
||||||
53
Cargo.lock
generated
53
Cargo.lock
generated
@ -5,16 +5,25 @@ version = 4
|
|||||||
[[package]]
|
[[package]]
|
||||||
name = "agent"
|
name = "agent"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
|
dependencies = [
|
||||||
|
"agent-core",
|
||||||
|
"anyhow",
|
||||||
|
"common",
|
||||||
|
"tokio",
|
||||||
|
"windows",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "agent-core"
|
||||||
|
version = "0.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"common",
|
"common",
|
||||||
"lz4_flex",
|
|
||||||
"openh264",
|
"openh264",
|
||||||
"quinn",
|
"quinn",
|
||||||
"rcgen",
|
"rcgen",
|
||||||
"rustls",
|
"rustls",
|
||||||
"tokio",
|
"tokio",
|
||||||
"windows",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@ -303,6 +312,16 @@ dependencies = [
|
|||||||
"slab",
|
"slab",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "gethostname"
|
||||||
|
version = "1.1.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
|
||||||
|
dependencies = [
|
||||||
|
"rustix",
|
||||||
|
"windows-link",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "getrandom"
|
name = "getrandom"
|
||||||
version = "0.2.17"
|
version = "0.2.17"
|
||||||
@ -452,6 +471,19 @@ dependencies = [
|
|||||||
"redox_syscall 0.9.5",
|
"redox_syscall 0.9.5",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "linux-agent"
|
||||||
|
version = "0.1.0"
|
||||||
|
dependencies = [
|
||||||
|
"agent-core",
|
||||||
|
"anyhow",
|
||||||
|
"common",
|
||||||
|
"quinn",
|
||||||
|
"rustls",
|
||||||
|
"tokio",
|
||||||
|
"x11rb",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "linux-raw-sys"
|
name = "linux-raw-sys"
|
||||||
version = "0.12.1"
|
version = "0.12.1"
|
||||||
@ -1673,6 +1705,23 @@ dependencies = [
|
|||||||
"pkg-config",
|
"pkg-config",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "x11rb"
|
||||||
|
version = "0.13.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
|
||||||
|
dependencies = [
|
||||||
|
"gethostname",
|
||||||
|
"rustix",
|
||||||
|
"x11rb-protocol",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "x11rb-protocol"
|
||||||
|
version = "0.13.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "ea6fc2961e4ef194dcbfe56bb845534d0dc8098940c7e5c012a258bfec6701bd"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "xcursor"
|
name = "xcursor"
|
||||||
version = "0.3.11"
|
version = "0.3.11"
|
||||||
|
|||||||
@ -1,6 +1,10 @@
|
|||||||
[workspace]
|
[workspace]
|
||||||
resolver = "2"
|
resolver = "2"
|
||||||
members = ["common", "agent", "viewer"]
|
# agent-core:平台无关的被控端会话层(Windows/Linux 共用);linux-agent:Linux 被控端
|
||||||
|
members = ["common", "agent-core", "agent", "linux-agent", "viewer"]
|
||||||
|
# 默认成员保持 Windows 三件套,`cargo build --release` 依旧只出 agent.exe / viewer.exe;
|
||||||
|
# Linux 被控端显式构建:`cargo build --release -p linux-agent`
|
||||||
|
default-members = ["common", "agent", "viewer"]
|
||||||
|
|
||||||
[workspace.dependencies]
|
[workspace.dependencies]
|
||||||
anyhow = "1"
|
anyhow = "1"
|
||||||
|
|||||||
13
agent-core/Cargo.toml
Normal file
13
agent-core/Cargo.toml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
[package]
|
||||||
|
name = "agent-core"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
common = { path = "../common" }
|
||||||
|
anyhow.workspace = true
|
||||||
|
tokio.workspace = true
|
||||||
|
quinn.workspace = true
|
||||||
|
rustls.workspace = true
|
||||||
|
rcgen.workspace = true
|
||||||
|
openh264 = { version = "0.9.9", features = ["source"] }
|
||||||
398
agent-core/src/lib.rs
Normal file
398
agent-core/src/lib.rs
Normal file
@ -0,0 +1,398 @@
|
|||||||
|
//! 平台无关的被控端会话层:QUIC/TLS 握手、验证码认证、H.264 编码下行、输入分发。
|
||||||
|
//!
|
||||||
|
//! 从 `agent/`(Windows)抽出来,供 Windows 与 Linux 被控端共用,避免两端各写一份
|
||||||
|
//! QUIC 握手/认证/丢帧逻辑造成协议漂移。平台差异只通过两个 trait 注入:
|
||||||
|
//!
|
||||||
|
//! - [`ScreenCapturer`]:屏幕采集(Windows = DXGI Desktop Duplication,Linux = X11)
|
||||||
|
//! - [`InputInjector`]:输入注入(Windows = SendInput,Linux = XTest)
|
||||||
|
//!
|
||||||
|
//! 线格式完全复用 `common`,**不改协议**:键盘沿用 Set-1 扫描码 + extended 标志,
|
||||||
|
//! 鼠标沿用虚拟桌面归一化坐标 0..=65535。
|
||||||
|
|
||||||
|
use anyhow::Result;
|
||||||
|
use common::{
|
||||||
|
encode_video_frame, read_control, write_control, ControlMsg, InputEvent, MouseButton,
|
||||||
|
VideoFrameHeader,
|
||||||
|
};
|
||||||
|
use openh264::encoder::{
|
||||||
|
BitRate, Encoder, EncoderConfig, FrameRate, IntraFramePeriod, RateControlMode, UsageType,
|
||||||
|
};
|
||||||
|
use openh264::formats::{BgraSliceU8, YUVBuffer};
|
||||||
|
use openh264::OpenH264API;
|
||||||
|
use rustls::pki_types::PrivatePkcs8KeyDer;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
use tokio::sync::{mpsc, watch};
|
||||||
|
|
||||||
|
/// 屏幕采集后端。
|
||||||
|
///
|
||||||
|
/// 语义刻意对齐 DXGI Desktop Duplication,使 Linux 侧只需实现同一套模型:
|
||||||
|
/// `acquire()` 阻塞至多 `timeout_ms`,返回是否拿到**新**帧;`copy_frame()` 再取出像素。
|
||||||
|
pub trait ScreenCapturer: Send {
|
||||||
|
/// 桌面尺寸(物理像素)
|
||||||
|
fn size(&self) -> (u32, u32);
|
||||||
|
|
||||||
|
/// 阻塞至多 `timeout_ms`。
|
||||||
|
/// `Ok(true)` = 有新帧(已拷进内部缓冲,可由 `copy_frame` 取出);
|
||||||
|
/// `Ok(false)` = 超时且画面未变(静止桌面)。
|
||||||
|
fn acquire(&mut self, timeout_ms: u32) -> Result<bool>;
|
||||||
|
|
||||||
|
/// 取出最近一帧,BGRA 紧凑排列(已去掉 row pitch),供 openh264 直接编码。
|
||||||
|
fn copy_frame(&mut self) -> Result<Vec<u8>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 输入注入后端。实现需可在异步运行时中被多任务共享,故要求 `Send + Sync`。
|
||||||
|
pub trait InputInjector: Send + Sync + 'static {
|
||||||
|
fn mouse_move(&self, x: u16, y: u16);
|
||||||
|
fn mouse_button(&self, b: MouseButton, down: bool);
|
||||||
|
fn wheel(&self, dy: i32);
|
||||||
|
fn key(&self, scancode: u16, extended: bool, down: bool);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 把一条协议输入事件分发到注入后端。
|
||||||
|
pub fn dispatch_input(inj: &dyn InputInjector, ev: InputEvent) {
|
||||||
|
match ev {
|
||||||
|
InputEvent::MouseMove { x, y } => inj.mouse_move(x, y),
|
||||||
|
InputEvent::MouseDown(b) => inj.mouse_button(b, true),
|
||||||
|
InputEvent::MouseUp(b) => inj.mouse_button(b, false),
|
||||||
|
InputEvent::Wheel { dy } => inj.wheel(dy),
|
||||||
|
InputEvent::KeyDown { scancode, extended } => inj.key(scancode, extended, true),
|
||||||
|
InputEvent::KeyUp { scancode, extended } => inj.key(scancode, extended, false),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 会话参数
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SessionConfig {
|
||||||
|
pub port: u16,
|
||||||
|
pub code: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 编码档位。默认值即 Windows 现有档位,改动需两端同步评估。
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct EncoderSettings {
|
||||||
|
pub bitrate_bps: u32,
|
||||||
|
pub max_fps: f32,
|
||||||
|
pub intra_period: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for EncoderSettings {
|
||||||
|
fn default() -> Self {
|
||||||
|
// 对标 UU:屏幕内容实时档;关键帧周期 300 帧 ≈ 10s
|
||||||
|
Self {
|
||||||
|
bitrate_bps: 5_000_000,
|
||||||
|
max_fps: 30.0,
|
||||||
|
intra_period: 300,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 解析 `--port` / `--code`(两端共用,保持命令行一致)
|
||||||
|
pub fn parse_args() -> Result<(u16, String)> {
|
||||||
|
let mut port = 47800u16;
|
||||||
|
let mut code = "123456".to_string();
|
||||||
|
let a: Vec<String> = std::env::args().collect();
|
||||||
|
let mut i = 1;
|
||||||
|
while i + 1 < a.len() {
|
||||||
|
match a[i].as_str() {
|
||||||
|
"--port" => port = a[i + 1].parse()?,
|
||||||
|
"--code" => code = a[i + 1].clone(),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
i += 2;
|
||||||
|
}
|
||||||
|
Ok((port, code))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn make_encoder(enc: &EncoderSettings) -> Result<Encoder> {
|
||||||
|
Ok(Encoder::with_api_config(
|
||||||
|
OpenH264API::from_source(),
|
||||||
|
EncoderConfig::new()
|
||||||
|
.bitrate(BitRate::from_bps(enc.bitrate_bps))
|
||||||
|
.max_frame_rate(FrameRate::from_hz(enc.max_fps))
|
||||||
|
.rate_control_mode(RateControlMode::Bitrate)
|
||||||
|
.usage_type(UsageType::ScreenContentRealTime)
|
||||||
|
.intra_frame_period(IntraFramePeriod::from_num_frames(enc.intra_period)),
|
||||||
|
)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
enum LoopEnd {
|
||||||
|
/// 下行通道关闭 —— 会话结束,采集线程退出
|
||||||
|
ChannelClosed,
|
||||||
|
/// 采集后端失效(Windows 的 ACCESS_LOST / X11 连接断开)—— 重建后继续
|
||||||
|
Rebuild,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 编码一帧并推入下行通道。返回 false 表示通道已关闭(会话结束)。
|
||||||
|
fn encode_send(
|
||||||
|
encoder: &mut Encoder,
|
||||||
|
raw: &[u8],
|
||||||
|
(w, h): (u32, u32),
|
||||||
|
ts_ms: u64,
|
||||||
|
tx: &watch::Sender<Option<Vec<u8>>>,
|
||||||
|
last_frame: &mut Option<Vec<u8>>,
|
||||||
|
) -> bool {
|
||||||
|
let header = VideoFrameHeader {
|
||||||
|
width: w,
|
||||||
|
height: h,
|
||||||
|
ts_ms,
|
||||||
|
};
|
||||||
|
let yuv = YUVBuffer::from_bgra8_source(BgraSliceU8::new(raw, (w as usize, h as usize)));
|
||||||
|
match encoder.encode(&yuv) {
|
||||||
|
Ok(ebs) => {
|
||||||
|
let frame = encode_video_frame(&header, &ebs.to_vec());
|
||||||
|
if tx.send(Some(frame.clone())).is_err() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
*last_frame = Some(frame);
|
||||||
|
true
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("[capture] encode failed: {e}");
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 单次采集会话:抓帧 → 编码 → 推入 watch 单槽。
|
||||||
|
fn capture_session(
|
||||||
|
cap: &mut dyn ScreenCapturer,
|
||||||
|
tx: &watch::Sender<Option<Vec<u8>>>,
|
||||||
|
keyreq: &AtomicBool,
|
||||||
|
enc: &EncoderSettings,
|
||||||
|
) -> LoopEnd {
|
||||||
|
let mut encoder = match make_encoder(enc) {
|
||||||
|
Ok(e) => e,
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("[capture] encoder init failed: {e}");
|
||||||
|
return LoopEnd::ChannelClosed;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let (w, h) = cap.size();
|
||||||
|
eprintln!("[capture] started {w}x{h}");
|
||||||
|
let start = Instant::now();
|
||||||
|
let min_frame = Duration::from_secs_f32(1.0 / enc.max_fps);
|
||||||
|
let mut last_frame: Option<Vec<u8>> = None;
|
||||||
|
// 保留最近一帧裸像素:静止桌面收到关键帧请求时,仍能重编一个真关键帧
|
||||||
|
let mut last_raw: Option<Vec<u8>> = None;
|
||||||
|
let mut last_sent = Instant::now() - Duration::from_secs(10);
|
||||||
|
|
||||||
|
loop {
|
||||||
|
let iter_start = Instant::now();
|
||||||
|
match cap.acquire(80) {
|
||||||
|
Ok(true) => match cap.copy_frame() {
|
||||||
|
Ok(raw) => {
|
||||||
|
// 主控端请求关键帧(解码异常恢复)
|
||||||
|
if keyreq.swap(false, Ordering::Relaxed) {
|
||||||
|
encoder.force_intra_frame();
|
||||||
|
}
|
||||||
|
let ts = start.elapsed().as_millis() as u64;
|
||||||
|
if !encode_send(&mut encoder, &raw, (w, h), ts, tx, &mut last_frame) {
|
||||||
|
return LoopEnd::ChannelClosed; // 会话结束
|
||||||
|
}
|
||||||
|
last_raw = Some(raw);
|
||||||
|
last_sent = Instant::now();
|
||||||
|
// 帧间隔节流到 max_fps
|
||||||
|
let el = iter_start.elapsed();
|
||||||
|
if el < min_frame {
|
||||||
|
std::thread::sleep(min_frame - el);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("[capture] frame copy failed: {e}");
|
||||||
|
return LoopEnd::Rebuild;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
Ok(false) => {
|
||||||
|
let force = keyreq.swap(false, Ordering::Relaxed);
|
||||||
|
if force && last_raw.is_some() {
|
||||||
|
// 静止桌面下的关键帧恢复:画面没变不代表可以不给关键帧——
|
||||||
|
// 解码端一旦出错,若只会收到旧 P 帧就永远恢复不了
|
||||||
|
encoder.force_intra_frame();
|
||||||
|
let raw = last_raw.as_ref().expect("上面已判断 is_some");
|
||||||
|
let ts = start.elapsed().as_millis() as u64;
|
||||||
|
if !encode_send(&mut encoder, raw, (w, h), ts, tx, &mut last_frame) {
|
||||||
|
return LoopEnd::ChannelClosed;
|
||||||
|
}
|
||||||
|
last_sent = Instant::now();
|
||||||
|
} else if last_sent.elapsed() > Duration::from_millis(500) && last_frame.is_some() {
|
||||||
|
// 桌面静止:低频重发最后一帧(重复 P 帧,解码端画面不变)
|
||||||
|
let f = last_frame.as_ref().expect("上面已判断 is_some");
|
||||||
|
if tx.send(Some(f.clone())).is_err() {
|
||||||
|
return LoopEnd::ChannelClosed;
|
||||||
|
}
|
||||||
|
last_sent = Instant::now();
|
||||||
|
} else {
|
||||||
|
std::thread::sleep(Duration::from_millis(8));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("[capture] acquire failed: {e}");
|
||||||
|
return LoopEnd::Rebuild;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 采集主循环:独立线程运行;后端失效时经 `factory` 重建,通道关闭时退出。
|
||||||
|
/// 通道为 watch 单槽(latest-wins):发送即覆盖,网络慢时不产生积压。
|
||||||
|
pub fn run_capture_loop<F>(
|
||||||
|
tx: watch::Sender<Option<Vec<u8>>>,
|
||||||
|
keyreq: Arc<AtomicBool>,
|
||||||
|
mut cap: Box<dyn ScreenCapturer>,
|
||||||
|
mut factory: F,
|
||||||
|
enc: EncoderSettings,
|
||||||
|
) where
|
||||||
|
F: FnMut() -> Result<Box<dyn ScreenCapturer>>,
|
||||||
|
{
|
||||||
|
loop {
|
||||||
|
match capture_session(cap.as_mut(), &tx, keyreq.as_ref(), &enc) {
|
||||||
|
LoopEnd::ChannelClosed => return,
|
||||||
|
LoopEnd::Rebuild => {}
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
loop {
|
||||||
|
match factory() {
|
||||||
|
Ok(c) => {
|
||||||
|
cap = c;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("[capture] init failed: {e}, retry in 2s");
|
||||||
|
std::thread::sleep(Duration::from_secs(2));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 跑一次完整的被控端会话:监听 → 认证 → 视频下行 + 输入上行。
|
||||||
|
///
|
||||||
|
/// `capture_factory` 可被多次调用(采集后端失效时重建),故需为 `FnMut`。
|
||||||
|
pub async fn serve<F>(
|
||||||
|
cfg: SessionConfig,
|
||||||
|
enc: EncoderSettings,
|
||||||
|
mut capture_factory: F,
|
||||||
|
injector: Arc<dyn InputInjector>,
|
||||||
|
) -> Result<()>
|
||||||
|
where
|
||||||
|
F: FnMut() -> Result<Box<dyn ScreenCapturer>> + Send + 'static,
|
||||||
|
{
|
||||||
|
rustls::crypto::ring::default_provider()
|
||||||
|
.install_default()
|
||||||
|
.ok();
|
||||||
|
|
||||||
|
// 自签名证书(MVP:客户端跳过校验;正式版改为固定公钥/账号体系)
|
||||||
|
let ck = rcgen::generate_simple_self_signed(vec!["localhost".into()])?;
|
||||||
|
let cert = ck.cert.der().clone();
|
||||||
|
let key = PrivatePkcs8KeyDer::from(ck.key_pair.serialize_der());
|
||||||
|
|
||||||
|
// 传输参数调优:
|
||||||
|
// - 收发窗口放大,避免高 RTT 下吞吐被 窗口/RTT 死锁(默认 512KB 流窗口)
|
||||||
|
// - 发送缓冲上限 2MB:QUIC 内部排队有界(约 1 帧多),配合上游 latest-wins 丢帧
|
||||||
|
// 把端到端排队延迟压在 ~0.4s 内
|
||||||
|
let mut transport = quinn::TransportConfig::default();
|
||||||
|
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||||
|
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||||
|
transport.send_window(2 * 1024 * 1024);
|
||||||
|
let mut server_config = quinn::ServerConfig::with_single_cert(vec![cert], key.into())?;
|
||||||
|
server_config.transport_config(Arc::new(transport));
|
||||||
|
|
||||||
|
let endpoint = quinn::Endpoint::server(server_config, format!("0.0.0.0:{}", cfg.port).parse()?)?;
|
||||||
|
println!(
|
||||||
|
"[agent] listening 0.0.0.0:{}, verify code = {}",
|
||||||
|
cfg.port, cfg.code
|
||||||
|
);
|
||||||
|
|
||||||
|
let conn = endpoint
|
||||||
|
.accept()
|
||||||
|
.await
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("endpoint closed"))?
|
||||||
|
.await?;
|
||||||
|
println!("[agent] peer connected: {}", conn.remote_address());
|
||||||
|
|
||||||
|
// 流 1:认证 + 视频下行
|
||||||
|
let (mut send, mut recv) = conn.accept_bi().await?;
|
||||||
|
|
||||||
|
let first = read_control(&mut recv).await?;
|
||||||
|
let code_ok = matches!(&first, ControlMsg::Auth { code: c } if *c == cfg.code);
|
||||||
|
if !code_ok {
|
||||||
|
write_control(&mut send, &ControlMsg::AuthFail("bad code".into())).await?;
|
||||||
|
send.finish()?;
|
||||||
|
// 必须等对端读到 AuthFail 并挂断再退出:直接 return 会让进程带走 QUIC
|
||||||
|
// 缓冲区里尚未发出的数据,客户端只会看到 "connection lost" 而不是 AuthFail
|
||||||
|
let _ = tokio::time::timeout(Duration::from_secs(5), conn.closed()).await;
|
||||||
|
anyhow::bail!("auth failed");
|
||||||
|
}
|
||||||
|
|
||||||
|
// 采集后端先建一次:既拿桌面尺寸回给主控端,也避免白建一个 duplication 只为读尺寸
|
||||||
|
let first_cap = capture_factory()?;
|
||||||
|
let (w, h) = first_cap.size();
|
||||||
|
write_control(&mut send, &ControlMsg::AuthOk { width: w, height: h }).await?;
|
||||||
|
println!("[agent] auth ok, desktop {w}x{h}");
|
||||||
|
|
||||||
|
// 流 2:键鼠输入 + 控制上行(与视频分离,避免被视频积压阻塞)
|
||||||
|
let (mut isend, mut irecv) = conn.accept_bi().await?;
|
||||||
|
|
||||||
|
// 采集线程 → 视频帧(watch 单槽:新帧覆盖旧帧,latest-wins)
|
||||||
|
let (tx, mut frames_rx) = watch::channel::<Option<Vec<u8>>>(None);
|
||||||
|
let keyreq = Arc::new(AtomicBool::new(false));
|
||||||
|
let keyreq_capture = keyreq.clone();
|
||||||
|
let enc_settings = enc;
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
run_capture_loop(tx, keyreq_capture, first_cap, capture_factory, enc_settings)
|
||||||
|
});
|
||||||
|
|
||||||
|
// 上行控制消息(Pong)经通道交给主循环写流 2,避免多任务争抢
|
||||||
|
let (ctl_tx, mut ctl_rx) = mpsc::unbounded_channel::<ControlMsg>();
|
||||||
|
let t0 = Instant::now();
|
||||||
|
|
||||||
|
// 读循环(流 2):输入事件 + Ping
|
||||||
|
let keyreq_reader = keyreq.clone();
|
||||||
|
let reader = tokio::spawn(async move {
|
||||||
|
loop {
|
||||||
|
match read_control(&mut irecv).await {
|
||||||
|
Ok(ControlMsg::Input(ev)) => dispatch_input(injector.as_ref(), ev),
|
||||||
|
Ok(ControlMsg::KeyFrameReq) => {
|
||||||
|
keyreq_reader.store(true, Ordering::Relaxed);
|
||||||
|
}
|
||||||
|
Ok(ControlMsg::Ping(t)) => {
|
||||||
|
let agent_ts = t0.elapsed().as_millis() as u64;
|
||||||
|
let _ = ctl_tx.send(ControlMsg::Pong { t, agent_ts });
|
||||||
|
}
|
||||||
|
Ok(_) => {}
|
||||||
|
Err(_) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
loop {
|
||||||
|
tokio::select! {
|
||||||
|
changed = frames_rx.changed() => {
|
||||||
|
if changed.is_err() {
|
||||||
|
break; // 采集线程退出
|
||||||
|
}
|
||||||
|
if let Some(data) = frames_rx.borrow().clone() {
|
||||||
|
if send.write_all(&data).await.is_err() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ctl = ctl_rx.recv() => {
|
||||||
|
match ctl {
|
||||||
|
Some(m) => {
|
||||||
|
if write_control(&mut isend, &m).await.is_err() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
reader.abort();
|
||||||
|
println!("[agent] session closed");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
@ -4,14 +4,11 @@ version = "0.1.0"
|
|||||||
edition = "2021"
|
edition = "2021"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
# 会话层(QUIC/TLS/认证/编码/丢帧)在 agent-core,与 linux-agent 共用
|
||||||
|
agent-core = { path = "../agent-core" }
|
||||||
common = { path = "../common" }
|
common = { path = "../common" }
|
||||||
anyhow.workspace = true
|
anyhow.workspace = true
|
||||||
tokio.workspace = true
|
tokio.workspace = true
|
||||||
quinn.workspace = true
|
|
||||||
rustls.workspace = true
|
|
||||||
rcgen.workspace = true
|
|
||||||
lz4_flex.workspace = true
|
|
||||||
openh264 = { version = "0.9.9", features = ["source"] }
|
|
||||||
windows = { workspace = true, features = [
|
windows = { workspace = true, features = [
|
||||||
"Win32_Foundation",
|
"Win32_Foundation",
|
||||||
"Win32_Graphics_Direct3D",
|
"Win32_Graphics_Direct3D",
|
||||||
|
|||||||
@ -1,25 +1,19 @@
|
|||||||
//! DXGI Desktop Duplication 采集(Windows 8+)。
|
//! Windows 屏幕采集:DXGI Desktop Duplication(Windows 8+)。
|
||||||
//! 帧处理:BGRA → openh264 H.264 编码(屏幕内容实时档,5Mbps)→ 线格式。
|
//!
|
||||||
//! 支持主控端 KeyFrameReq 强制关键帧(解码异常恢复)。
|
//! 采集循环、H.264 编码、丢帧与下行都在 `agent-core`(两端共用),本模块只负责
|
||||||
|
//! DXGI 的「取新帧 → 拷到 staging 纹理 → 映射内存」这三步。
|
||||||
|
//!
|
||||||
|
//! `ScreenCapturer` 的语义与 Linux 端一致:`acquire()` 返回 false 即代表超时且画面未变,
|
||||||
|
//! 由 agent-core 决定低频重发上一帧。
|
||||||
|
|
||||||
|
use agent_core::ScreenCapturer;
|
||||||
use anyhow::{anyhow, Result};
|
use anyhow::{anyhow, Result};
|
||||||
use common::{encode_video_frame, VideoFrameHeader};
|
|
||||||
use openh264::encoder::{
|
|
||||||
BitRate, Encoder, EncoderConfig, FrameRate, IntraFramePeriod, RateControlMode, UsageType,
|
|
||||||
};
|
|
||||||
use openh264::formats::{BgraSliceU8, YUVBuffer};
|
|
||||||
use openh264::OpenH264API;
|
|
||||||
use std::sync::atomic::{AtomicBool, Ordering};
|
|
||||||
use std::sync::Arc;
|
|
||||||
use tokio::sync::watch::Sender;
|
|
||||||
use windows::core::Interface;
|
use windows::core::Interface;
|
||||||
use windows::Win32::Foundation::HMODULE;
|
use windows::Win32::Foundation::HMODULE;
|
||||||
use windows::Win32::Graphics::Direct3D::{
|
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0};
|
||||||
D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0,
|
|
||||||
};
|
|
||||||
use windows::Win32::Graphics::Direct3D11::{
|
use windows::Win32::Graphics::Direct3D11::{
|
||||||
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Resource, ID3D11Texture2D,
|
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Resource, ID3D11Texture2D,
|
||||||
D3D11_CREATE_DEVICE_FLAG, D3D11_CPU_ACCESS_READ, D3D11_MAP, D3D11_MAP_READ,
|
D3D11_CREATE_DEVICE_FLAG, D3D11_CPU_ACCESS_READ, D3D11_MAP_READ,
|
||||||
D3D11_MAPPED_SUBRESOURCE, D3D11_SDK_VERSION, D3D11_TEXTURE2D_DESC, D3D11_USAGE_STAGING,
|
D3D11_MAPPED_SUBRESOURCE, D3D11_SDK_VERSION, D3D11_TEXTURE2D_DESC, D3D11_USAGE_STAGING,
|
||||||
};
|
};
|
||||||
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_SAMPLE_DESC};
|
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_SAMPLE_DESC};
|
||||||
@ -28,8 +22,6 @@ use windows::Win32::Graphics::Dxgi::{
|
|||||||
DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
|
DXGI_OUTDUPL_DESC, DXGI_OUTDUPL_FRAME_INFO,
|
||||||
};
|
};
|
||||||
|
|
||||||
use std::time::{Duration, Instant};
|
|
||||||
|
|
||||||
pub struct Duplication {
|
pub struct Duplication {
|
||||||
_device: ID3D11Device,
|
_device: ID3D11Device,
|
||||||
context: ID3D11DeviceContext,
|
context: ID3D11DeviceContext,
|
||||||
@ -105,9 +97,15 @@ impl Duplication {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ScreenCapturer for Duplication {
|
||||||
|
fn size(&self) -> (u32, u32) {
|
||||||
|
(self.width, self.height)
|
||||||
|
}
|
||||||
|
|
||||||
/// true=拿到新帧并已复制到 staging;false=超时无更新
|
/// true=拿到新帧并已复制到 staging;false=超时无更新
|
||||||
pub fn acquire(&mut self, timeout_ms: u32) -> Result<bool> {
|
fn acquire(&mut self, timeout_ms: u32) -> Result<bool> {
|
||||||
unsafe {
|
unsafe {
|
||||||
let mut info = DXGI_OUTDUPL_FRAME_INFO::default();
|
let mut info = DXGI_OUTDUPL_FRAME_INFO::default();
|
||||||
let mut resource: Option<IDXGIResource> = None;
|
let mut resource: Option<IDXGIResource> = None;
|
||||||
@ -126,7 +124,7 @@ impl Duplication {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// 把 staging 纹理映射到内存并去掉行间距
|
/// 把 staging 纹理映射到内存并去掉行间距
|
||||||
pub fn copy_frame(&mut self) -> Result<Vec<u8>> {
|
fn copy_frame(&mut self) -> Result<Vec<u8>> {
|
||||||
unsafe {
|
unsafe {
|
||||||
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
|
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
|
||||||
self.context
|
self.context
|
||||||
@ -145,112 +143,7 @@ impl Duplication {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(dead_code)]
|
/// 采集器工厂:agent-core 在采集后端失效时会重新调用
|
||||||
pub fn desktop_size() -> Result<(u32, u32)> {
|
pub fn new_capturer() -> Result<Box<dyn ScreenCapturer>> {
|
||||||
let d = Duplication::new()?;
|
Ok(Box::new(Duplication::new()?))
|
||||||
Ok((d.width, d.height))
|
|
||||||
}
|
|
||||||
|
|
||||||
/// 采集主循环:独立线程运行;连接断开(通道关闭)时退出。
|
|
||||||
/// 通道为 watch 单槽(latest-wins):发送即覆盖,网络慢时不产生积压。
|
|
||||||
pub fn run(tx: Sender<Option<Vec<u8>>>, keyreq: Arc<AtomicBool>) {
|
|
||||||
let start = Instant::now();
|
|
||||||
loop {
|
|
||||||
let mut dup = match Duplication::new() {
|
|
||||||
Ok(d) => d,
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("[capture] init failed: {e}, retry in 2s");
|
|
||||||
std::thread::sleep(Duration::from_secs(2));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
eprintln!(
|
|
||||||
"[capture] duplication started {}x{}",
|
|
||||||
dup.width, dup.height
|
|
||||||
);
|
|
||||||
|
|
||||||
// H.264 编码器:屏幕内容实时档 + 码率控制(对标 UU:3.2Mbps@60fps)
|
|
||||||
let mut encoder = match Encoder::with_api_config(
|
|
||||||
OpenH264API::from_source(),
|
|
||||||
EncoderConfig::new()
|
|
||||||
.bitrate(BitRate::from_bps(5_000_000))
|
|
||||||
.max_frame_rate(FrameRate::from_hz(30.0))
|
|
||||||
.rate_control_mode(RateControlMode::Bitrate)
|
|
||||||
.usage_type(UsageType::ScreenContentRealTime)
|
|
||||||
.intra_frame_period(IntraFramePeriod::from_num_frames(300)),
|
|
||||||
) {
|
|
||||||
Ok(e) => e,
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("[capture] encoder init failed: {e}");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
let mut last_frame: Option<Vec<u8>> = None;
|
|
||||||
let mut last_sent = Instant::now() - Duration::from_secs(10);
|
|
||||||
let min_frame = Duration::from_millis(33); // 30fps 上限
|
|
||||||
loop {
|
|
||||||
let iter_start = Instant::now();
|
|
||||||
match dup.acquire(80) {
|
|
||||||
Ok(true) => match dup.copy_frame() {
|
|
||||||
Ok(raw) => {
|
|
||||||
// 主控端请求关键帧(解码异常恢复)
|
|
||||||
if keyreq.swap(false, Ordering::Relaxed) {
|
|
||||||
encoder.force_intra_frame();
|
|
||||||
}
|
|
||||||
let header = VideoFrameHeader {
|
|
||||||
width: dup.width,
|
|
||||||
height: dup.height,
|
|
||||||
ts_ms: start.elapsed().as_millis() as u64,
|
|
||||||
};
|
|
||||||
let yuv = YUVBuffer::from_bgra8_source(BgraSliceU8::new(
|
|
||||||
&raw,
|
|
||||||
(dup.width as usize, dup.height as usize),
|
|
||||||
));
|
|
||||||
match encoder.encode(&yuv) {
|
|
||||||
Ok(ebs) => {
|
|
||||||
let payload = ebs.to_vec();
|
|
||||||
let frame = encode_video_frame(&header, &payload);
|
|
||||||
if tx.send(Some(frame.clone())).is_err() {
|
|
||||||
return; // 会话结束
|
|
||||||
}
|
|
||||||
last_frame = Some(frame);
|
|
||||||
last_sent = Instant::now();
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("[capture] encode failed: {e}");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// 帧间隔节流到 ~30fps
|
|
||||||
let el = iter_start.elapsed();
|
|
||||||
if el < min_frame {
|
|
||||||
std::thread::sleep(min_frame - el);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("[capture] frame copy failed: {e}");
|
|
||||||
break; // 重建 duplication
|
|
||||||
}
|
|
||||||
},
|
|
||||||
Ok(false) => {
|
|
||||||
// 桌面静止:低频重发最后一帧(重复 P 帧,解码端画面不变)
|
|
||||||
if last_sent.elapsed() > Duration::from_millis(500) {
|
|
||||||
if let Some(f) = last_frame.as_ref() {
|
|
||||||
if tx.send(Some(f.clone())).is_err() {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
last_sent = Instant::now();
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
std::thread::sleep(Duration::from_millis(8));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
eprintln!("[capture] acquire failed: {e}");
|
|
||||||
break; // ACCESS_LOST 等,重建
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
std::thread::sleep(Duration::from_millis(300));
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,6 +1,11 @@
|
|||||||
//! 输入注入:SendInput 实现(鼠标归一化绝对坐标 + 扫描码键盘)。
|
//! Windows 输入注入:SendInput(鼠标虚拟桌面绝对坐标 + 扫描码键盘)。
|
||||||
|
//!
|
||||||
|
//! 协议里的坐标/扫描码语义两端一致,只是落地 API 不同:
|
||||||
|
//! - 鼠标:`MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK`,直接吃 0..65535 归一化值;
|
||||||
|
//! - 键盘:`KEYEVENTF_SCANCODE` 直接送 Set-1 扫描码(Linux 端则需换算成 X11 keycode)。
|
||||||
|
|
||||||
use common::{InputEvent, MouseButton};
|
use agent_core::InputInjector;
|
||||||
|
use common::MouseButton;
|
||||||
use windows::Win32::UI::Input::KeyboardAndMouse::{
|
use windows::Win32::UI::Input::KeyboardAndMouse::{
|
||||||
SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY,
|
SendInput, INPUT, INPUT_0, INPUT_KEYBOARD, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY,
|
||||||
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_MIDDLEDOWN,
|
KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_MIDDLEDOWN,
|
||||||
@ -73,31 +78,49 @@ pub fn virtual_desktop() -> (i32, i32, i32, i32) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn handle(ev: InputEvent) {
|
/// SendInput 注入后端(无状态,直接调 Win32)
|
||||||
match ev {
|
pub struct WinInput;
|
||||||
InputEvent::MouseMove { x, y } => {
|
|
||||||
// 主控端已按虚拟桌面归一化,这里直接透传 0..65535
|
impl WinInput {
|
||||||
send_mouse(
|
pub fn new() -> Self {
|
||||||
MOUSEEVENTF_ABSOLUTE.0 | MOUSEEVENTF_VIRTUALDESK.0 | MOUSEEVENTF_MOVE.0,
|
Self
|
||||||
x as i32,
|
}
|
||||||
y as i32,
|
}
|
||||||
0,
|
|
||||||
);
|
impl Default for WinInput {
|
||||||
}
|
fn default() -> Self {
|
||||||
InputEvent::MouseDown(b) => match b {
|
Self::new()
|
||||||
MouseButton::Left => send_mouse(MOUSEEVENTF_LEFTDOWN, 0, 0, 0),
|
}
|
||||||
MouseButton::Right => send_mouse(MOUSEEVENTF_RIGHTDOWN.0, 0, 0, 0),
|
}
|
||||||
MouseButton::Middle => send_mouse(MOUSEEVENTF_MIDDLEDOWN.0, 0, 0, 0),
|
|
||||||
},
|
impl InputInjector for WinInput {
|
||||||
InputEvent::MouseUp(b) => match b {
|
fn mouse_move(&self, x: u16, y: u16) {
|
||||||
MouseButton::Left => send_mouse(MOUSEEVENTF_LEFTUP, 0, 0, 0),
|
// 主控端已按虚拟桌面归一化,这里直接透传 0..65535
|
||||||
MouseButton::Right => send_mouse(MOUSEEVENTF_RIGHTUP.0, 0, 0, 0),
|
send_mouse(
|
||||||
MouseButton::Middle => send_mouse(MOUSEEVENTF_MIDDLEUP.0, 0, 0, 0),
|
MOUSEEVENTF_ABSOLUTE.0 | MOUSEEVENTF_VIRTUALDESK.0 | MOUSEEVENTF_MOVE.0,
|
||||||
},
|
x as i32,
|
||||||
InputEvent::Wheel { dy } => {
|
y as i32,
|
||||||
send_mouse(MOUSEEVENTF_WHEEL.0, 0, 0, dy as u32);
|
0,
|
||||||
}
|
);
|
||||||
InputEvent::KeyDown { scancode, extended } => send_key(scancode, extended, false),
|
}
|
||||||
InputEvent::KeyUp { scancode, extended } => send_key(scancode, extended, true),
|
|
||||||
|
fn mouse_button(&self, b: MouseButton, down: bool) {
|
||||||
|
let flags = match (b, down) {
|
||||||
|
(MouseButton::Left, true) => MOUSEEVENTF_LEFTDOWN,
|
||||||
|
(MouseButton::Left, false) => MOUSEEVENTF_LEFTUP,
|
||||||
|
(MouseButton::Right, true) => MOUSEEVENTF_RIGHTDOWN.0,
|
||||||
|
(MouseButton::Right, false) => MOUSEEVENTF_RIGHTUP.0,
|
||||||
|
(MouseButton::Middle, true) => MOUSEEVENTF_MIDDLEDOWN.0,
|
||||||
|
(MouseButton::Middle, false) => MOUSEEVENTF_MIDDLEUP.0,
|
||||||
|
};
|
||||||
|
send_mouse(flags, 0, 0, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn wheel(&self, dy: i32) {
|
||||||
|
send_mouse(MOUSEEVENTF_WHEEL.0, 0, 0, dy as u32);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn key(&self, scancode: u16, extended: bool, down: bool) {
|
||||||
|
send_key(scancode, extended, !down);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,142 +1,28 @@
|
|||||||
//! 被控端 Agent:DXGI 桌面复制采集 → LZ4 压缩 → QUIC 下行视频;上行接收输入事件注入。
|
//! 被控端 Agent(Windows):DXGI 桌面复制采集 + SendInput 键鼠注入。
|
||||||
//! P0 优化:视频帧经 watch 单槽通道(latest-wins,网络慢时只发最新帧,不积压);
|
//!
|
||||||
//! 键鼠/控制走独立 QUIC 双向流,不与视频互相阻塞。
|
//! 会话层(QUIC/TLS 握手、验证码认证、H.264 编码下行、latest-wins 丢帧、输入分发)在
|
||||||
|
//! `agent-core`,与 `linux-agent` 共用同一份实现;本 crate 只提供平台相关的采集与注入,
|
||||||
|
//! 避免两端各写一份握手/认证造成协议漂移。
|
||||||
|
|
||||||
mod capture;
|
mod capture;
|
||||||
mod input;
|
mod input;
|
||||||
|
|
||||||
|
use agent_core::{parse_args, serve, EncoderSettings, InputInjector, SessionConfig};
|
||||||
use anyhow::Result;
|
use anyhow::Result;
|
||||||
use common::{read_control, write_control, ControlMsg};
|
|
||||||
use rustls::pki_types::PrivatePkcs8KeyDer;
|
|
||||||
use std::sync::atomic::{AtomicBool, Ordering};
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
use tokio::io::AsyncWriteExt;
|
|
||||||
use tokio::sync::{mpsc, watch};
|
|
||||||
|
|
||||||
fn parse_args() -> Result<(u16, String)> {
|
|
||||||
let mut port = 47800u16;
|
|
||||||
let mut code = "123456".to_string();
|
|
||||||
let a: Vec<String> = std::env::args().collect();
|
|
||||||
let mut i = 1;
|
|
||||||
while i < a.len() {
|
|
||||||
match a[i].as_str() {
|
|
||||||
"--port" => port = a[i + 1].parse()?,
|
|
||||||
"--code" => code = a[i + 1].clone(),
|
|
||||||
_ => {}
|
|
||||||
}
|
|
||||||
i += 2;
|
|
||||||
}
|
|
||||||
Ok((port, code))
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::main]
|
#[tokio::main]
|
||||||
async fn main() -> Result<()> {
|
async fn main() -> Result<()> {
|
||||||
let (port, code) = parse_args()?;
|
let (port, code) = parse_args()?;
|
||||||
rustls::crypto::ring::default_provider()
|
|
||||||
.install_default()
|
|
||||||
.ok();
|
|
||||||
|
|
||||||
// 自签名证书(MVP:客户端跳过校验;正式版改为固定公钥/账号体系)
|
let injector: Arc<dyn InputInjector> = Arc::new(input::WinInput::new());
|
||||||
let ck = rcgen::generate_simple_self_signed(vec!["localhost".into()])?;
|
|
||||||
let cert = ck.cert.der().clone();
|
|
||||||
let key = PrivatePkcs8KeyDer::from(ck.key_pair.serialize_der());
|
|
||||||
|
|
||||||
// 传输参数调优:
|
serve(
|
||||||
// - 收发窗口放大,避免高 RTT 下吞吐被 窗口/RTT 死锁(默认 512KB 流窗口)
|
SessionConfig { port, code },
|
||||||
// - 发送缓冲上限 2MB:QUIC 内部排队有界(约 1 帧多),配合上游 latest-wins 丢帧
|
EncoderSettings::default(),
|
||||||
// 把端到端排队延迟压在 ~0.4s 内
|
// DXGI 后端失效(ACCESS_LOST 等)时 agent-core 会重新调用本工厂
|
||||||
let mut transport = quinn::TransportConfig::default();
|
capture::new_capturer,
|
||||||
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
injector,
|
||||||
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
)
|
||||||
transport.send_window(2 * 1024 * 1024);
|
.await
|
||||||
let mut server_config = quinn::ServerConfig::with_single_cert(vec![cert], key.into())?;
|
|
||||||
server_config.transport_config(Arc::new(transport));
|
|
||||||
|
|
||||||
let endpoint = quinn::Endpoint::server(
|
|
||||||
server_config,
|
|
||||||
format!("0.0.0.0:{port}").parse()?,
|
|
||||||
)?;
|
|
||||||
println!("[agent] listening 0.0.0.0:{port}, verify code = {code}");
|
|
||||||
|
|
||||||
let conn = endpoint
|
|
||||||
.accept()
|
|
||||||
.await
|
|
||||||
.ok_or_else(|| anyhow::anyhow!("endpoint closed"))?
|
|
||||||
.await?;
|
|
||||||
println!("[agent] peer connected: {}", conn.remote_address());
|
|
||||||
|
|
||||||
// 流 1:认证 + 视频下行
|
|
||||||
let (mut send, mut recv) = conn.accept_bi().await?;
|
|
||||||
|
|
||||||
// 认证
|
|
||||||
let first = read_control(&mut recv).await?;
|
|
||||||
let code_ok = matches!(&first, ControlMsg::Auth { code: c } if *c == code);
|
|
||||||
if !code_ok {
|
|
||||||
write_control(&mut send, &ControlMsg::AuthFail("bad code".into())).await?;
|
|
||||||
send.finish()?;
|
|
||||||
anyhow::bail!("auth failed");
|
|
||||||
}
|
|
||||||
let (w, h) = capture::desktop_size()?;
|
|
||||||
write_control(&mut send, &ControlMsg::AuthOk { width: w, height: h }).await?;
|
|
||||||
println!("[agent] auth ok, desktop {w}x{h}");
|
|
||||||
|
|
||||||
// 流 2:键鼠输入 + 控制上行(与视频分离,避免被视频积压阻塞)
|
|
||||||
let (mut isend, mut irecv) = conn.accept_bi().await?;
|
|
||||||
|
|
||||||
// 采集线程 → 视频帧(watch 单槽:新帧覆盖旧帧,latest-wins)
|
|
||||||
let (tx, mut frames_rx) = watch::channel::<Option<Vec<u8>>>(None);
|
|
||||||
let keyreq = Arc::new(AtomicBool::new(false));
|
|
||||||
let keyreq_capture = keyreq.clone();
|
|
||||||
std::thread::spawn(move || capture::run(tx, keyreq_capture));
|
|
||||||
|
|
||||||
// 上行控制消息(Pong)经通道交给主循环写流 2,避免多任务争抢
|
|
||||||
let (ctl_tx, mut ctl_rx) = mpsc::unbounded_channel::<ControlMsg>();
|
|
||||||
let t0 = std::time::Instant::now();
|
|
||||||
|
|
||||||
// 读循环(流 2):输入事件 + Ping
|
|
||||||
let keyreq_reader = keyreq.clone();
|
|
||||||
let reader = tokio::spawn(async move {
|
|
||||||
loop {
|
|
||||||
match read_control(&mut irecv).await {
|
|
||||||
Ok(ControlMsg::Input(ev)) => input::handle(ev),
|
|
||||||
Ok(ControlMsg::KeyFrameReq) => {
|
|
||||||
keyreq_reader.store(true, Ordering::Relaxed);
|
|
||||||
}
|
|
||||||
Ok(ControlMsg::Ping(t)) => {
|
|
||||||
let agent_ts = t0.elapsed().as_millis() as u64;
|
|
||||||
let _ = ctl_tx.send(ControlMsg::Pong { t, agent_ts });
|
|
||||||
}
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(_) => break,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
loop {
|
|
||||||
tokio::select! {
|
|
||||||
changed = frames_rx.changed() => {
|
|
||||||
if changed.is_err() {
|
|
||||||
break; // 采集线程退出
|
|
||||||
}
|
|
||||||
if let Some(data) = frames_rx.borrow().clone() {
|
|
||||||
if send.write_all(&data).await.is_err() {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
ctl = ctl_rx.recv() => {
|
|
||||||
match ctl {
|
|
||||||
Some(m) => {
|
|
||||||
if write_control(&mut isend, &m).await.is_err() {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
None => break,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
reader.abort();
|
|
||||||
println!("[agent] session closed");
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
|
|||||||
16
linux-agent/Cargo.toml
Normal file
16
linux-agent/Cargo.toml
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
[package]
|
||||||
|
name = "linux-agent"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
agent-core = { path = "../agent-core" }
|
||||||
|
common = { path = "../common" }
|
||||||
|
anyhow.workspace = true
|
||||||
|
tokio.workspace = true
|
||||||
|
x11rb = { version = "0.13", features = ["xtest"] }
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
# 冒烟测试里要扮演主控端,自己发起 QUIC/TLS 连接
|
||||||
|
quinn.workspace = true
|
||||||
|
rustls.workspace = true
|
||||||
147
linux-agent/src/capture.rs
Normal file
147
linux-agent/src/capture.rs
Normal file
@ -0,0 +1,147 @@
|
|||||||
|
//! X11 屏幕采集。
|
||||||
|
//!
|
||||||
|
//! MVP 走核心协议 `GetImage`:零扩展依赖、零 unsafe,先把「采集 → 编码 → QUIC」整条链路打通。
|
||||||
|
//! 变更检测用整帧字节比较,桌面静止时不上行视频(省掉编码开销)。
|
||||||
|
//!
|
||||||
|
//! 二期优化锚点(已在 `ScreenCapturer` 语义内预留,无需改 agent-core):
|
||||||
|
//! 1. **MIT-SHM**:共享内存替代 `GetImage` 的 socket 传输,省掉 X server 一次全画布拷贝;
|
||||||
|
//! 2. **XDamage**:只取变化矩形 + 只编码脏区,静止时零拷贝。
|
||||||
|
|
||||||
|
use agent_core::ScreenCapturer;
|
||||||
|
use anyhow::{anyhow, Result};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
use x11rb::connection::Connection;
|
||||||
|
use x11rb::protocol::xproto::{ConnectionExt as _, ImageFormat, VisualClass, Window};
|
||||||
|
use x11rb::rust_connection::RustConnection;
|
||||||
|
|
||||||
|
/// 帧节流上限,与 agent-core 默认编码档位(30fps)对齐
|
||||||
|
const FRAME_INTERVAL: Duration = Duration::from_millis(33);
|
||||||
|
|
||||||
|
pub struct X11Capturer {
|
||||||
|
conn: RustConnection,
|
||||||
|
root: Window,
|
||||||
|
width: u16,
|
||||||
|
height: u16,
|
||||||
|
/// 最近一帧 BGRA(既是变更检测基线,也是 copy_frame 的数据源)
|
||||||
|
last: Vec<u8>,
|
||||||
|
last_capture: Option<Instant>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl X11Capturer {
|
||||||
|
pub fn new() -> Result<Self> {
|
||||||
|
let (conn, screen_num) = x11rb::connect(None).map_err(|e| {
|
||||||
|
anyhow!(
|
||||||
|
"连接 X11 失败(DISPLAY={:?}): {e}",
|
||||||
|
std::env::var("DISPLAY").ok()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
|
||||||
|
// 先把需要的屏幕参数拷出来:随后 conn 要被移进结构体,不能再持有对它的借用
|
||||||
|
let (root, width, height, depth, visual_class, bpp) = {
|
||||||
|
let setup = conn.setup();
|
||||||
|
let screen = &setup.roots[screen_num];
|
||||||
|
|
||||||
|
// bits_per_pixel 不在 visual 上,而在 Setup 的 pixmap_formats 里(按 depth 索引)
|
||||||
|
let fmt = setup
|
||||||
|
.pixmap_formats
|
||||||
|
.iter()
|
||||||
|
.find(|f| f.depth == screen.root_depth)
|
||||||
|
.ok_or_else(|| anyhow!("找不到 depth={} 对应的像素格式", screen.root_depth))?;
|
||||||
|
|
||||||
|
let visual_class = screen
|
||||||
|
.allowed_depths
|
||||||
|
.iter()
|
||||||
|
.flat_map(|d| d.visuals.iter())
|
||||||
|
.find(|v| v.visual_id == screen.root_visual)
|
||||||
|
.map(|v| v.class)
|
||||||
|
.ok_or_else(|| anyhow!("找不到 root visual"))?;
|
||||||
|
|
||||||
|
(
|
||||||
|
screen.root,
|
||||||
|
screen.width_in_pixels,
|
||||||
|
screen.height_in_pixels,
|
||||||
|
screen.root_depth,
|
||||||
|
visual_class,
|
||||||
|
fmt.bits_per_pixel,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
|
||||||
|
anyhow::ensure!(width > 0 && height > 0, "屏幕尺寸异常 {width}x{height}");
|
||||||
|
anyhow::ensure!(
|
||||||
|
bpp == 32,
|
||||||
|
"当前仅支持 32bpp 真彩采集(depth={depth},实际 {bpp}bpp)"
|
||||||
|
);
|
||||||
|
// BGRA 字节序假设只在 TrueColor 下成立
|
||||||
|
anyhow::ensure!(
|
||||||
|
visual_class == VisualClass::TRUE_COLOR,
|
||||||
|
"当前仅支持 TrueColor 视觉(实际 {visual_class:?})"
|
||||||
|
);
|
||||||
|
|
||||||
|
println!("[capture] X11 screen {width}x{height}, depth={depth}, {bpp}bpp");
|
||||||
|
|
||||||
|
Ok(Self {
|
||||||
|
conn,
|
||||||
|
root,
|
||||||
|
width,
|
||||||
|
height,
|
||||||
|
last: Vec::new(),
|
||||||
|
last_capture: None,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 抓一帧整屏到 `last`,返回画面是否发生变化
|
||||||
|
fn grab(&mut self) -> Result<bool> {
|
||||||
|
let reply = self
|
||||||
|
.conn
|
||||||
|
.get_image(
|
||||||
|
ImageFormat::Z_PIXMAP,
|
||||||
|
self.root,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
self.width,
|
||||||
|
self.height,
|
||||||
|
!0,
|
||||||
|
)
|
||||||
|
.map_err(|e| anyhow!("get_image 请求失败: {e}"))?
|
||||||
|
.reply()
|
||||||
|
.map_err(|e| anyhow!("get_image 应答失败: {e}"))?;
|
||||||
|
|
||||||
|
let expect = self.width as usize * self.height as usize * 4;
|
||||||
|
anyhow::ensure!(
|
||||||
|
reply.data.len() == expect,
|
||||||
|
"帧数据长度异常:期望 {expect} 字节,实际 {}",
|
||||||
|
reply.data.len()
|
||||||
|
);
|
||||||
|
|
||||||
|
let changed = reply.data != self.last;
|
||||||
|
if changed {
|
||||||
|
self.last = reply.data;
|
||||||
|
}
|
||||||
|
Ok(changed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ScreenCapturer for X11Capturer {
|
||||||
|
fn size(&self) -> (u32, u32) {
|
||||||
|
(self.width as u32, self.height as u32)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// X11 核心协议没有「阻塞至新帧」语义,故这里按帧槽节流后主动抓帧:
|
||||||
|
/// 睡到下一个 1/30s 槽位 → 抓帧 → 与上一帧逐字节比较。
|
||||||
|
/// 因此 `timeout_ms` 在 Linux 实现中被忽略(agent-core 的 min_frame 节流也随之自然满足)。
|
||||||
|
fn acquire(&mut self, _timeout_ms: u32) -> Result<bool> {
|
||||||
|
if let Some(t) = self.last_capture {
|
||||||
|
let elapsed = t.elapsed();
|
||||||
|
if elapsed < FRAME_INTERVAL {
|
||||||
|
std::thread::sleep(FRAME_INTERVAL - elapsed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
self.last_capture = Some(Instant::now());
|
||||||
|
self.grab()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn copy_frame(&mut self) -> Result<Vec<u8>> {
|
||||||
|
anyhow::ensure!(!self.last.is_empty(), "尚无帧数据可拷贝");
|
||||||
|
Ok(self.last.clone())
|
||||||
|
}
|
||||||
|
}
|
||||||
249
linux-agent/src/input.rs
Normal file
249
linux-agent/src/input.rs
Normal file
@ -0,0 +1,249 @@
|
|||||||
|
//! X11 输入注入:XTest 扩展(Xorg / XWayland / Xvfb 均支持)。
|
||||||
|
//!
|
||||||
|
//! **键码映射是本模块的关键**:协议里键盘沿用 Windows 口味的 PS/2 Set-1 扫描码,
|
||||||
|
//! 而 X11 要的是 keycode,两套编号并非处处等价:
|
||||||
|
//! - 主键区:`keycode = scancode + 8`(evdev 偏移),已逐键核对;
|
||||||
|
//! - E0 扩展键:两条链路编号**完全不同**(如方向键 Set-1 是 0x48,evdev 侧 X11 keycode 是 111),必须查表。
|
||||||
|
//!
|
||||||
|
//! 鼠标沿用虚拟桌面归一化坐标 0..=65535:X11 根窗口天然横跨全部显示器,
|
||||||
|
//! 与 Windows 的 `MOUSEEVENTF_VIRTUALDESK` 语义一致,故映射到根窗口像素即可。
|
||||||
|
|
||||||
|
use agent_core::InputInjector;
|
||||||
|
use anyhow::{anyhow, Result};
|
||||||
|
use common::MouseButton;
|
||||||
|
use std::sync::Mutex;
|
||||||
|
use x11rb::connection::Connection;
|
||||||
|
use x11rb::protocol::xproto::{ConnectionExt as _, Window};
|
||||||
|
use x11rb::protocol::xtest::ConnectionExt as _;
|
||||||
|
use x11rb::rust_connection::RustConnection;
|
||||||
|
|
||||||
|
// X11 核心事件码(协议固定值)
|
||||||
|
const KEY_PRESS: u8 = 2;
|
||||||
|
const KEY_RELEASE: u8 = 3;
|
||||||
|
const BUTTON_PRESS: u8 = 4;
|
||||||
|
const BUTTON_RELEASE: u8 = 5;
|
||||||
|
const MOTION_NOTIFY: u8 = 6;
|
||||||
|
|
||||||
|
// X11 按钮编号:1=左 2=中 3=右 4=滚轮上 5=滚轮下
|
||||||
|
const BTN_LEFT: u8 = 1;
|
||||||
|
const BTN_MIDDLE: u8 = 2;
|
||||||
|
const BTN_RIGHT: u8 = 3;
|
||||||
|
const BTN_WHEEL_UP: u8 = 4;
|
||||||
|
const BTN_WHEEL_DOWN: u8 = 5;
|
||||||
|
|
||||||
|
/// Windows 滚轮一格 = 120(协议沿用该语义)
|
||||||
|
const WHEEL_DELTA: i32 = 120;
|
||||||
|
|
||||||
|
/// 单次滚轮最多合成的格数,防止异常值刷爆事件队列
|
||||||
|
const WHEEL_MAX_CLICKS: i32 = 20;
|
||||||
|
|
||||||
|
pub struct X11Injector {
|
||||||
|
/// `Mutex` 只为满足 `InputInjector: Sync`(X11 连接本身非 Sync),无竞争
|
||||||
|
conn: Mutex<RustConnection>,
|
||||||
|
root: Window,
|
||||||
|
width: i32,
|
||||||
|
height: i32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl X11Injector {
|
||||||
|
pub fn new() -> Result<Self> {
|
||||||
|
let (conn, screen_num) = x11rb::connect(None).map_err(|e| {
|
||||||
|
anyhow!(
|
||||||
|
"连接 X11 失败(DISPLAY={:?}): {e}",
|
||||||
|
std::env::var("DISPLAY").ok()
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
|
||||||
|
// XTest 缺失就没有注入能力,启动即失败好过连上后才报错
|
||||||
|
let ext = conn
|
||||||
|
.query_extension(b"XTEST")
|
||||||
|
.map_err(|e| anyhow!("查询 XTEST 扩展失败: {e}"))?
|
||||||
|
.reply()
|
||||||
|
.map_err(|e| anyhow!("查询 XTEST 扩展应答失败: {e}"))?;
|
||||||
|
anyhow::ensure!(ext.present, "X server 未提供 XTEST 扩展,无法注入输入");
|
||||||
|
|
||||||
|
// 先拷出参数,避免借用 conn 的同时又把它移进结构体
|
||||||
|
let (root, width, height) = {
|
||||||
|
let screen = &conn.setup().roots[screen_num];
|
||||||
|
(
|
||||||
|
screen.root,
|
||||||
|
screen.width_in_pixels as i32,
|
||||||
|
screen.height_in_pixels as i32,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
println!("[input] XTEST ready, root {width}x{height}");
|
||||||
|
|
||||||
|
Ok(Self {
|
||||||
|
conn: Mutex::new(conn),
|
||||||
|
root,
|
||||||
|
width,
|
||||||
|
height,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fake(&self, type_: u8, detail: u8, x: i32, y: i32) {
|
||||||
|
let conn = match self.conn.lock() {
|
||||||
|
Ok(c) => c,
|
||||||
|
Err(poisoned) => poisoned.into_inner(),
|
||||||
|
};
|
||||||
|
if let Err(e) = conn.xtest_fake_input(
|
||||||
|
type_,
|
||||||
|
detail,
|
||||||
|
0,
|
||||||
|
self.root,
|
||||||
|
x as i16,
|
||||||
|
y as i16,
|
||||||
|
0, // deviceid:0 = 核心设备
|
||||||
|
) {
|
||||||
|
eprintln!("[input] xtest_fake_input 失败: {e}");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let _ = conn.flush();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn click(&self, button: u8) {
|
||||||
|
self.fake(BUTTON_PRESS, button, 0, 0);
|
||||||
|
self.fake(BUTTON_RELEASE, button, 0, 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl InputInjector for X11Injector {
|
||||||
|
fn mouse_move(&self, x: u16, y: u16) {
|
||||||
|
// detail=0 → 绝对坐标
|
||||||
|
let px = scale(x, self.width);
|
||||||
|
let py = scale(y, self.height);
|
||||||
|
self.fake(MOTION_NOTIFY, 0, px, py);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mouse_button(&self, b: MouseButton, down: bool) {
|
||||||
|
let button = match b {
|
||||||
|
MouseButton::Left => BTN_LEFT,
|
||||||
|
MouseButton::Middle => BTN_MIDDLE,
|
||||||
|
MouseButton::Right => BTN_RIGHT,
|
||||||
|
};
|
||||||
|
self.fake(
|
||||||
|
if down { BUTTON_PRESS } else { BUTTON_RELEASE },
|
||||||
|
button,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn wheel(&self, dy: i32) {
|
||||||
|
// 协议沿用 Windows 语义:dy > 0 向上滚,一格 = 120
|
||||||
|
if dy == 0 {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let clicks = (dy.abs() / WHEEL_DELTA).clamp(1, WHEEL_MAX_CLICKS);
|
||||||
|
let button = if dy > 0 { BTN_WHEEL_UP } else { BTN_WHEEL_DOWN };
|
||||||
|
for _ in 0..clicks {
|
||||||
|
self.click(button);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn key(&self, scancode: u16, extended: bool, down: bool) {
|
||||||
|
let Some(keycode) = keycode_of(scancode, extended) else {
|
||||||
|
eprintln!("[input] 未映射的扫描码 scancode=0x{scancode:02X} extended={extended}");
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
self.fake(
|
||||||
|
if down { KEY_PRESS } else { KEY_RELEASE },
|
||||||
|
keycode,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 虚拟桌面归一化坐标(0..=65535)→ 根窗口像素坐标
|
||||||
|
fn scale(v: u16, extent: i32) -> i32 {
|
||||||
|
if extent <= 1 {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
((v as i64 * (extent as i64 - 1)) / 65535) as i32
|
||||||
|
}
|
||||||
|
|
||||||
|
/// PS/2 Set-1 扫描码 → X11 keycode。
|
||||||
|
///
|
||||||
|
/// 非扩展键恒为 `scancode + 8`(当前 viewer 的 keymap 只发该区间内的键);
|
||||||
|
/// E0 扩展键查表,编号与主键区规则无关。
|
||||||
|
fn keycode_of(scancode: u16, extended: bool) -> Option<u8> {
|
||||||
|
if extended {
|
||||||
|
// 已按 Xorg evdev 标准 keycode 分配核对
|
||||||
|
let kc = match scancode {
|
||||||
|
0x1C => 104, // 小键盘 Enter
|
||||||
|
0x1D => 105, // 右 Ctrl
|
||||||
|
0x35 => 106, // 小键盘 /
|
||||||
|
0x37 => 107, // PrintScreen
|
||||||
|
0x38 => 108, // 右 Alt
|
||||||
|
0x47 => 110, // Home
|
||||||
|
0x48 => 111, // ↑
|
||||||
|
0x49 => 112, // PageUp
|
||||||
|
0x4B => 113, // ←
|
||||||
|
0x4D => 114, // →
|
||||||
|
0x4F => 115, // End
|
||||||
|
0x50 => 116, // ↓
|
||||||
|
0x51 => 117, // PageDown
|
||||||
|
0x52 => 118, // Insert
|
||||||
|
0x53 => 119, // Delete
|
||||||
|
0x5B => 133, // 左 Super(Win)
|
||||||
|
0x5C => 134, // 右 Super(Win)
|
||||||
|
0x5D => 135, // Menu
|
||||||
|
_ => return None,
|
||||||
|
};
|
||||||
|
Some(kc)
|
||||||
|
} else {
|
||||||
|
// 0x01..=0x53 主键区,以及 0x57/0x58 的 F11/F12;其余为 Set-1 未定义位
|
||||||
|
match scancode {
|
||||||
|
0x01..=0x53 | 0x57 | 0x58 => Some((scancode + 8) as u8),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn main_block_uses_evdev_offset() {
|
||||||
|
assert_eq!(keycode_of(0x1E, false), Some(38)); // A
|
||||||
|
assert_eq!(keycode_of(0x02, false), Some(10)); // 1
|
||||||
|
assert_eq!(keycode_of(0x39, false), Some(65)); // Space
|
||||||
|
assert_eq!(keycode_of(0x1C, false), Some(36)); // Enter
|
||||||
|
assert_eq!(keycode_of(0x1D, false), Some(37)); // 左 Ctrl
|
||||||
|
assert_eq!(keycode_of(0x2A, false), Some(50)); // 左 Shift
|
||||||
|
assert_eq!(keycode_of(0x58, false), Some(96)); // F12
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extended_keys_are_not_offset() {
|
||||||
|
// 若误用 +8 规则,方向键会落到小键盘区(0x48+8=80=KP_8),这正是要防的坑
|
||||||
|
assert_eq!(keycode_of(0x48, true), Some(111)); // ↑
|
||||||
|
assert_ne!(keycode_of(0x48, true), Some(0x48 + 8));
|
||||||
|
assert_eq!(keycode_of(0x50, true), Some(116)); // ↓
|
||||||
|
assert_eq!(keycode_of(0x4B, true), Some(113)); // ←
|
||||||
|
assert_eq!(keycode_of(0x4D, true), Some(114)); // →
|
||||||
|
assert_eq!(keycode_of(0x47, true), Some(110)); // Home
|
||||||
|
assert_eq!(keycode_of(0x4F, true), Some(115)); // End
|
||||||
|
assert_eq!(keycode_of(0x49, true), Some(112)); // PageUp
|
||||||
|
assert_eq!(keycode_of(0x51, true), Some(117)); // PageDown
|
||||||
|
assert_eq!(keycode_of(0x53, true), Some(119)); // Delete
|
||||||
|
assert_eq!(keycode_of(0x5B, true), Some(133)); // 左 Super
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn undefined_scancodes_are_rejected() {
|
||||||
|
assert_eq!(keycode_of(0x54, false), None);
|
||||||
|
assert_eq!(keycode_of(0x7F, false), None);
|
||||||
|
assert_eq!(keycode_of(0x99, true), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn coordinate_scaling_covers_full_range() {
|
||||||
|
assert_eq!(scale(0, 1920), 0);
|
||||||
|
assert_eq!(scale(65535, 1920), 1919);
|
||||||
|
assert_eq!(scale(32768, 1920), 960);
|
||||||
|
assert_eq!(scale(12345, 0), 0); // 退化尺寸不 panic
|
||||||
|
}
|
||||||
|
}
|
||||||
31
linux-agent/src/main.rs
Normal file
31
linux-agent/src/main.rs
Normal file
@ -0,0 +1,31 @@
|
|||||||
|
//! Linux 被控端:X11 采集 + XTest 注入;会话层(QUIC/认证/编码/丢帧)复用 `agent-core`。
|
||||||
|
//!
|
||||||
|
//! 与 Windows 端的差异仅在这两个模块里,线格式与主控端完全一致,viewer 不需要任何改动。
|
||||||
|
//! 依赖 `DISPLAY` 环境变量(Xorg / XWayland 均可)。
|
||||||
|
|
||||||
|
mod capture;
|
||||||
|
mod input;
|
||||||
|
|
||||||
|
use agent_core::{parse_args, serve, EncoderSettings, InputInjector, ScreenCapturer, SessionConfig};
|
||||||
|
use anyhow::Result;
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() -> Result<()> {
|
||||||
|
let (port, code) = parse_args()?;
|
||||||
|
|
||||||
|
let display = std::env::var("DISPLAY").unwrap_or_else(|_| "(未设置)".to_string());
|
||||||
|
println!("[linux-agent] DISPLAY={display}");
|
||||||
|
|
||||||
|
// 注入端连接常驻(启动即校验 XTEST 扩展是否可用,缺了就直接报错退出)
|
||||||
|
let injector: Arc<dyn InputInjector> = Arc::new(input::X11Injector::new()?);
|
||||||
|
|
||||||
|
serve(
|
||||||
|
SessionConfig { port, code },
|
||||||
|
EncoderSettings::default(),
|
||||||
|
// 采集端可被重建(X11 连接断开等),故用工厂闭包
|
||||||
|
|| Ok(Box::new(capture::X11Capturer::new()?) as Box<dyn ScreenCapturer>),
|
||||||
|
injector,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
}
|
||||||
434
linux-agent/tests/smoke.rs
Normal file
434
linux-agent/tests/smoke.rs
Normal file
@ -0,0 +1,434 @@
|
|||||||
|
//! Linux 被控端端到端冒烟测试。
|
||||||
|
//!
|
||||||
|
//! 需要 X server(Xvfb 即可),通过 `DISPLAY` 指定;推荐用 `scripts/linux-smoke.sh` 一键跑。
|
||||||
|
//!
|
||||||
|
//! 覆盖链路:X11 采集 → openh264 编码 → QUIC/TLS 下行 → 主控端解帧;
|
||||||
|
//! 外加验证码认证、Ping/Pong、关键帧请求,以及**输入注入的实证断言**
|
||||||
|
//! (注入鼠标移动后,从另一条 X11 连接查询指针真实落点,而不是只看进程没崩)。
|
||||||
|
|
||||||
|
use anyhow::{bail, Context, Result};
|
||||||
|
use common::{read_control, read_video_frame, write_control, ControlMsg, InputEvent};
|
||||||
|
use rustls::client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier};
|
||||||
|
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
|
||||||
|
use rustls::{DigitallySignedStruct, SignatureScheme};
|
||||||
|
use std::io::{BufRead, BufReader};
|
||||||
|
use std::net::{TcpListener, ToSocketAddrs};
|
||||||
|
use std::process::{Child, Command, Stdio};
|
||||||
|
use std::sync::{mpsc, Arc};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
use x11rb::connection::Connection;
|
||||||
|
use x11rb::protocol::xproto::{ConnectionExt as _, Window};
|
||||||
|
use x11rb::rust_connection::RustConnection;
|
||||||
|
|
||||||
|
/// 与 viewer 一致:MVP 阶段跳过自签名证书校验
|
||||||
|
#[derive(Debug)]
|
||||||
|
struct SkipVerify;
|
||||||
|
|
||||||
|
impl ServerCertVerifier for SkipVerify {
|
||||||
|
fn verify_server_cert(
|
||||||
|
&self,
|
||||||
|
_end_entity: &CertificateDer<'_>,
|
||||||
|
_intermediates: &[CertificateDer<'_>],
|
||||||
|
_server_name: &ServerName<'_>,
|
||||||
|
_ocsp_response: &[u8],
|
||||||
|
_now: UnixTime,
|
||||||
|
) -> Result<ServerCertVerified, rustls::Error> {
|
||||||
|
Ok(ServerCertVerified::assertion())
|
||||||
|
}
|
||||||
|
fn verify_tls12_signature(
|
||||||
|
&self,
|
||||||
|
_message: &[u8],
|
||||||
|
_certificate: &CertificateDer<'_>,
|
||||||
|
_dss: &DigitallySignedStruct,
|
||||||
|
) -> Result<HandshakeSignatureValid, rustls::Error> {
|
||||||
|
Ok(HandshakeSignatureValid::assertion())
|
||||||
|
}
|
||||||
|
fn verify_tls13_signature(
|
||||||
|
&self,
|
||||||
|
_message: &[u8],
|
||||||
|
_certificate: &CertificateDer<'_>,
|
||||||
|
_dss: &DigitallySignedStruct,
|
||||||
|
) -> Result<HandshakeSignatureValid, rustls::Error> {
|
||||||
|
Ok(HandshakeSignatureValid::assertion())
|
||||||
|
}
|
||||||
|
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
|
||||||
|
vec![
|
||||||
|
SignatureScheme::RSA_PKCS1_SHA256,
|
||||||
|
SignatureScheme::ECDSA_NISTP256_SHA256,
|
||||||
|
SignatureScheme::ED25519,
|
||||||
|
SignatureScheme::RSA_PSS_SHA256,
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 被测进程的 RAII 句柄:测试无论成功失败都会收尸
|
||||||
|
struct AgentProc {
|
||||||
|
child: Child,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for AgentProc {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let _ = self.child.kill();
|
||||||
|
let _ = self.child.wait();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn expected_screen() -> (u32, u32) {
|
||||||
|
let s = std::env::var("SMOKE_SCREEN").unwrap_or_else(|_| "1280x720".to_string());
|
||||||
|
let (w, h) = s
|
||||||
|
.split_once('x')
|
||||||
|
.unwrap_or_else(|| panic!("SMOKE_SCREEN 需形如 1280x720,实际 {s:?}"));
|
||||||
|
(w.parse().expect("宽"), h.parse().expect("高"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn free_port() -> Result<u16> {
|
||||||
|
let l = TcpListener::bind("127.0.0.1:0")?;
|
||||||
|
Ok(l.local_addr()?.port())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn spawn_agent(port: u16, code: &str) -> Result<AgentProc> {
|
||||||
|
let mut child = Command::new(env!("CARGO_BIN_EXE_linux-agent"))
|
||||||
|
.args(["--port", &port.to_string(), "--code", code])
|
||||||
|
.stdout(Stdio::piped())
|
||||||
|
.stderr(Stdio::inherit())
|
||||||
|
.spawn()
|
||||||
|
.context("启动 linux-agent 失败")?;
|
||||||
|
|
||||||
|
let stdout = child.stdout.take().expect("stdout 已 piped");
|
||||||
|
let (tx, rx) = mpsc::channel();
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
for line in BufReader::new(stdout).lines().map_while(|l| l.ok()) {
|
||||||
|
println!("[agent] {line}");
|
||||||
|
let _ = tx.send(line);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// 等 QUIC 端口就绪
|
||||||
|
let deadline = Instant::now() + Duration::from_secs(30);
|
||||||
|
loop {
|
||||||
|
let left = deadline.saturating_duration_since(Instant::now());
|
||||||
|
if left.is_zero() {
|
||||||
|
bail!("等待 agent 监听超时(DISPLAY 是否已设置?)");
|
||||||
|
}
|
||||||
|
match rx.recv_timeout(left) {
|
||||||
|
Ok(line) if line.contains("listening") => return Ok(AgentProc { child }),
|
||||||
|
Ok(_) => continue,
|
||||||
|
Err(_) => bail!("等待 agent 监听超时,进程可能已退出"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn connect(port: u16) -> Result<(quinn::Endpoint, quinn::Connection)> {
|
||||||
|
let tls = rustls::ClientConfig::builder()
|
||||||
|
.dangerous()
|
||||||
|
.with_custom_certificate_verifier(Arc::new(SkipVerify))
|
||||||
|
.with_no_client_auth();
|
||||||
|
let qc: quinn::crypto::rustls::QuicClientConfig = Arc::new(tls).try_into()?;
|
||||||
|
|
||||||
|
let mut transport = quinn::TransportConfig::default();
|
||||||
|
transport.receive_window(quinn::VarInt::from_u32(16 * 1024 * 1024));
|
||||||
|
transport.stream_receive_window(quinn::VarInt::from_u32(8 * 1024 * 1024));
|
||||||
|
|
||||||
|
let mut client_config = quinn::ClientConfig::new(Arc::new(qc));
|
||||||
|
client_config.transport_config(Arc::new(transport));
|
||||||
|
|
||||||
|
let mut endpoint = quinn::Endpoint::client("0.0.0.0:0".parse()?)?;
|
||||||
|
endpoint.set_default_client_config(client_config);
|
||||||
|
|
||||||
|
let addr = ("127.0.0.1", port)
|
||||||
|
.to_socket_addrs()?
|
||||||
|
.next()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("地址解析失败"))?;
|
||||||
|
let conn = endpoint.connect(addr, "localhost")?.await?;
|
||||||
|
// endpoint 必须与 conn 同生命周期,故一并返回
|
||||||
|
Ok((endpoint, conn))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn has_start_code(payload: &[u8]) -> bool {
|
||||||
|
let head = &payload[..payload.len().min(64)];
|
||||||
|
head.windows(4).any(|w| w == [0, 0, 0, 1]) || head.windows(3).any(|w| w == [0, 0, 1])
|
||||||
|
}
|
||||||
|
|
||||||
|
/// H.264 NAL 类型:5 = IDR 片(关键帧)
|
||||||
|
const NAL_IDR: u8 = 5;
|
||||||
|
|
||||||
|
/// 按 Annex-B 起始码扫描,判断载荷里是否含指定类型的 NAL
|
||||||
|
fn contains_nal(payload: &[u8], nal_type: u8) -> bool {
|
||||||
|
let mut i = 0usize;
|
||||||
|
while i + 3 <= payload.len() {
|
||||||
|
let sc_len = if payload[i..].starts_with(&[0, 0, 0, 1]) {
|
||||||
|
4
|
||||||
|
} else if payload[i..].starts_with(&[0, 0, 1]) {
|
||||||
|
3
|
||||||
|
} else {
|
||||||
|
i += 1;
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if i + sc_len >= payload.len() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if payload[i + sc_len] & 0x1F == nal_type {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
i += sc_len + 1;
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 在根窗口左上角画一个**小**矩形,制造一次细微画面变化。
|
||||||
|
///
|
||||||
|
/// 刻意不用「整屏刷白」那类大变化:大变化会触发编码器的场景切换自动 IDR,
|
||||||
|
/// 于是「重发旧帧」和「真重编关键帧」拿到同样的字节,用例就失去区分度了。
|
||||||
|
fn perturb_screen(conn: &RustConnection, root: Window) -> Result<()> {
|
||||||
|
use x11rb::protocol::xproto::{CreateGCAux, Rectangle};
|
||||||
|
let gc = conn.generate_id()?;
|
||||||
|
conn.create_gc(
|
||||||
|
gc,
|
||||||
|
root,
|
||||||
|
&CreateGCAux::new().foreground(0x00FF_FFFF).graphics_exposures(0),
|
||||||
|
)?;
|
||||||
|
conn.poly_fill_rectangle(
|
||||||
|
root,
|
||||||
|
gc,
|
||||||
|
&[Rectangle {
|
||||||
|
x: 32,
|
||||||
|
y: 32,
|
||||||
|
width: 64,
|
||||||
|
height: 64,
|
||||||
|
}],
|
||||||
|
)?;
|
||||||
|
conn.free_gc(gc)?;
|
||||||
|
conn.flush()?;
|
||||||
|
println!("[smoke] 已扰动屏幕(根窗口画 64x64 白块)");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 读帧直到拿到一个非关键帧(跳过静止桌面的低频重发),返回该帧载荷
|
||||||
|
async fn read_until_p_frame(recv: &mut quinn::RecvStream, timeout: Duration) -> Result<Vec<u8>> {
|
||||||
|
let deadline = Instant::now() + timeout;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
let (_h, p) = tokio::time::timeout(Duration::from_secs(5), read_video_frame(recv))
|
||||||
|
.await
|
||||||
|
.context("等待扰动帧超时")??;
|
||||||
|
if !contains_nal(&p, NAL_IDR) {
|
||||||
|
return Ok(p);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bail!("扰动后未拿到非关键帧,用例前置条件不成立(扰动量是否太大?)")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 请求关键帧后,容错地等待一个 IDR:期间允许出现静止桌面的 P 帧重发。
|
||||||
|
/// 返回是否在超时内拿到 IDR。
|
||||||
|
async fn wait_for_idr(recv: &mut quinn::RecvStream, timeout: Duration) -> Result<bool> {
|
||||||
|
let deadline = Instant::now() + timeout;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
match tokio::time::timeout(Duration::from_secs(2), read_video_frame(recv)).await {
|
||||||
|
Ok(Ok((_h, p))) => {
|
||||||
|
if contains_nal(&p, NAL_IDR) {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(false)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 归一化坐标 → 根窗口像素(与 linux-agent/src/input.rs 的 scale 完全一致)
|
||||||
|
fn scale(v: u16, extent: u32) -> i16 {
|
||||||
|
if extent <= 1 {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
((v as i64 * (extent as i64 - 1)) / 65535) as i16
|
||||||
|
}
|
||||||
|
|
||||||
|
fn open_display() -> Result<(RustConnection, Window)> {
|
||||||
|
let (conn, screen_num) = x11rb::connect(None)
|
||||||
|
.map_err(|e| anyhow::anyhow!("连接 X11 失败(DISPLAY 是否已设置?): {e}"))?;
|
||||||
|
let root = conn.setup().roots[screen_num].root;
|
||||||
|
Ok((conn, root))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn pointer_position(conn: &RustConnection, root: Window) -> Result<(i16, i16)> {
|
||||||
|
let r = conn.query_pointer(root)?.reply()?;
|
||||||
|
Ok((r.root_x, r.root_y))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn wait_pointer(
|
||||||
|
conn: &RustConnection,
|
||||||
|
root: Window,
|
||||||
|
want: (i16, i16),
|
||||||
|
timeout: Duration,
|
||||||
|
) -> Result<(i16, i16)> {
|
||||||
|
let deadline = Instant::now() + timeout;
|
||||||
|
let mut last = pointer_position(conn, root)?;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
last = pointer_position(conn, root)?;
|
||||||
|
if (last.0 - want.0).abs() <= 2 && (last.1 - want.1).abs() <= 2 {
|
||||||
|
return Ok(last);
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(50));
|
||||||
|
}
|
||||||
|
Ok(last)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn end_to_end_capture_encode_transport_input() -> Result<()> {
|
||||||
|
rustls::crypto::ring::default_provider()
|
||||||
|
.install_default()
|
||||||
|
.ok();
|
||||||
|
|
||||||
|
let (sw, sh) = expected_screen();
|
||||||
|
let port = free_port()?;
|
||||||
|
let _agent = spawn_agent(port, "smoke-code")?;
|
||||||
|
let (_endpoint, conn) = connect(port).await?;
|
||||||
|
|
||||||
|
// 流 1:认证,并核对被控端上报的桌面尺寸就是 Xvfb 的尺寸
|
||||||
|
let (mut send, mut recv) = conn.open_bi().await?;
|
||||||
|
write_control(
|
||||||
|
&mut send,
|
||||||
|
&ControlMsg::Auth {
|
||||||
|
code: "smoke-code".into(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
match read_control(&mut recv).await? {
|
||||||
|
ControlMsg::AuthOk { width, height } => {
|
||||||
|
assert_eq!(
|
||||||
|
(width, height),
|
||||||
|
(sw, sh),
|
||||||
|
"agent 上报桌面尺寸与实际屏幕不符"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
other => bail!("认证应成功,却收到 {other:?}"),
|
||||||
|
}
|
||||||
|
|
||||||
|
// 流 2:输入 + 控制
|
||||||
|
let (mut isend, mut irecv) = conn.open_bi().await?;
|
||||||
|
|
||||||
|
// Ping/Pong(延迟测量通道)
|
||||||
|
write_control(&mut isend, &ControlMsg::Ping(4242)).await?;
|
||||||
|
match read_control(&mut irecv).await? {
|
||||||
|
ControlMsg::Pong { t, agent_ts } => {
|
||||||
|
assert_eq!(t, 4242, "Pong 回显的时间戳不一致");
|
||||||
|
println!("[smoke] Pong 收到,agent_ts={agent_ts}");
|
||||||
|
}
|
||||||
|
other => bail!("期望 Pong,收到 {other:?}"),
|
||||||
|
}
|
||||||
|
|
||||||
|
// 首帧:证明采集 + 编码 + 线格式整条链路通了
|
||||||
|
let (h1, p1) = tokio::time::timeout(Duration::from_secs(15), read_video_frame(&mut recv))
|
||||||
|
.await
|
||||||
|
.context("等待首个视频帧超时")??;
|
||||||
|
assert_eq!((h1.width, h1.height), (sw, sh), "视频帧头尺寸不符");
|
||||||
|
assert!(!p1.is_empty(), "视频载荷为空");
|
||||||
|
println!(
|
||||||
|
"[smoke] 首帧 {}x{} ts={} 编码后 {} 字节,头部 {:02X?}",
|
||||||
|
h1.width,
|
||||||
|
h1.height,
|
||||||
|
h1.ts_ms,
|
||||||
|
p1.len(),
|
||||||
|
&p1[..p1.len().min(12)]
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
has_start_code(&p1),
|
||||||
|
"载荷不像 H.264(前 16 字节 {:02X?})",
|
||||||
|
&p1[..p1.len().min(16)]
|
||||||
|
);
|
||||||
|
|
||||||
|
// 关键帧恢复:先在**静止**桌面上请求关键帧。
|
||||||
|
// 为了有区分度,先扰动一下屏幕,让缓存里落入一个非关键帧(P 帧),否则
|
||||||
|
// 「重发旧帧」和「真重编关键帧」两种行为拿到的字节是一样的,测不出来。
|
||||||
|
let (probe, root) = open_display()?;
|
||||||
|
perturb_screen(&probe, root)?;
|
||||||
|
let pp = read_until_p_frame(&mut recv, Duration::from_secs(10)).await?;
|
||||||
|
println!(
|
||||||
|
"[smoke] 扰动帧 {} 字节,头部 {:02X?}(应为 P 帧,无 IDR)",
|
||||||
|
pp.len(),
|
||||||
|
&pp[..pp.len().min(12)]
|
||||||
|
);
|
||||||
|
// 前置条件自检:没有这一步,用例可能因为「扰动本身产生了 IDR」而虚假通过
|
||||||
|
assert!(
|
||||||
|
!contains_nal(&pp, NAL_IDR),
|
||||||
|
"扰动帧仍是 IDR,本用例无法区分重发旧帧与真重编关键帧"
|
||||||
|
);
|
||||||
|
tokio::time::sleep(Duration::from_millis(800)).await; // 等画面重新静止
|
||||||
|
|
||||||
|
write_control(&mut isend, &ControlMsg::KeyFrameReq).await?;
|
||||||
|
assert!(
|
||||||
|
wait_for_idr(&mut recv, Duration::from_secs(3)).await?,
|
||||||
|
"静止桌面请求关键帧后 3s 内未收到 IDR:解码端一旦出错将永远无法恢复"
|
||||||
|
);
|
||||||
|
println!("[smoke] 静止桌面请求关键帧 → 收到 IDR");
|
||||||
|
|
||||||
|
// 输入注入实证:注入到屏幕中心,再从独立 X11 连接查询指针真实落点
|
||||||
|
let before = pointer_position(&probe, root)?;
|
||||||
|
write_control(
|
||||||
|
&mut isend,
|
||||||
|
&ControlMsg::Input(InputEvent::MouseMove {
|
||||||
|
x: 32768,
|
||||||
|
y: 32768,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
let want = (scale(32768, sw), scale(32768, sh));
|
||||||
|
let got = wait_pointer(&probe, root, want, Duration::from_secs(5))?;
|
||||||
|
println!("[smoke] 指针 {before:?} → {got:?}(期望 ≈{want:?})");
|
||||||
|
assert!(
|
||||||
|
(got.0 - want.0).abs() <= 2 && (got.1 - want.1).abs() <= 2,
|
||||||
|
"XTest 注入未生效:期望指针 ≈{want:?},实际 {got:?}"
|
||||||
|
);
|
||||||
|
|
||||||
|
// 键盘:链路不得因此崩溃(无窗口可断言落点)
|
||||||
|
write_control(
|
||||||
|
&mut isend,
|
||||||
|
&ControlMsg::Input(InputEvent::KeyDown {
|
||||||
|
scancode: 0x1E,
|
||||||
|
extended: false,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
write_control(
|
||||||
|
&mut isend,
|
||||||
|
&ControlMsg::Input(InputEvent::KeyUp {
|
||||||
|
scancode: 0x1E,
|
||||||
|
extended: false,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
tokio::time::sleep(Duration::from_millis(300)).await;
|
||||||
|
assert!(
|
||||||
|
conn.close_reason().is_none(),
|
||||||
|
"会话被意外关闭:{:?}",
|
||||||
|
conn.close_reason()
|
||||||
|
);
|
||||||
|
|
||||||
|
println!("[smoke] 全部断言通过");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn rejects_wrong_verification_code() -> Result<()> {
|
||||||
|
rustls::crypto::ring::default_provider()
|
||||||
|
.install_default()
|
||||||
|
.ok();
|
||||||
|
|
||||||
|
let port = free_port()?;
|
||||||
|
let _agent = spawn_agent(port, "right-code")?;
|
||||||
|
let (_endpoint, conn) = connect(port).await?;
|
||||||
|
|
||||||
|
let (mut send, mut recv) = conn.open_bi().await?;
|
||||||
|
write_control(
|
||||||
|
&mut send,
|
||||||
|
&ControlMsg::Auth {
|
||||||
|
code: "wrong-code".into(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
match read_control(&mut recv).await? {
|
||||||
|
ControlMsg::AuthFail(_) => Ok(()),
|
||||||
|
other => bail!("错误验证码竟被接受:{other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
48
scripts/linux-smoke.sh
Normal file
48
scripts/linux-smoke.sh
Normal file
@ -0,0 +1,48 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Linux 被控端冒烟测试一键脚本:自建 Xvfb 虚拟显示 → 跑端到端测试 → 收摊。
|
||||||
|
#
|
||||||
|
# 用法(容器内或任意有 Xvfb 的 Linux):
|
||||||
|
# bash scripts/linux-smoke.sh
|
||||||
|
# 可用环境变量覆盖:
|
||||||
|
# SMOKE_SCREEN=1920x1080 虚拟屏分辨率(默认 1280x720)
|
||||||
|
# SMOKE_DISPLAY=:99 X display 编号(默认 :99)
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCREEN="${SMOKE_SCREEN:-1280x720}"
|
||||||
|
DISPLAY_NUM="${SMOKE_DISPLAY:-:99}"
|
||||||
|
|
||||||
|
export DISPLAY="$DISPLAY_NUM"
|
||||||
|
export SMOKE_SCREEN="$SCREEN"
|
||||||
|
|
||||||
|
XVFB_PID=""
|
||||||
|
cleanup() {
|
||||||
|
if [ -n "$XVFB_PID" ]; then
|
||||||
|
kill "$XVFB_PID" 2>/dev/null || true
|
||||||
|
wait "$XVFB_PID" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
echo "[smoke] 启动 Xvfb $DISPLAY_NUM (${SCREEN}x24)"
|
||||||
|
Xvfb "$DISPLAY_NUM" -screen 0 "${SCREEN}x24" -nolisten tcp >/tmp/xvfb.log 2>&1 &
|
||||||
|
XVFB_PID=$!
|
||||||
|
|
||||||
|
ready=0
|
||||||
|
for _ in $(seq 1 50); do
|
||||||
|
if xdpyinfo -display "$DISPLAY_NUM" >/dev/null 2>&1; then
|
||||||
|
ready=1
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.2
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$ready" -ne 1 ]; then
|
||||||
|
echo "[smoke] Xvfb 启动失败,日志:"
|
||||||
|
cat /tmp/xvfb.log 2>/dev/null || true
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "[smoke] X server 就绪:$(xdpyinfo -display "$DISPLAY_NUM" | grep -E 'dimensions' | head -1)"
|
||||||
|
|
||||||
|
# 两个测试共用同一块虚拟屏与指针,必须串行
|
||||||
|
cargo test -p linux-agent --test smoke -- --nocapture --test-threads=1
|
||||||
|
echo "[smoke] 通过"
|
||||||
94
构建运行手册.md
94
构建运行手册.md
@ -1,7 +1,7 @@
|
|||||||
# MVP 远程控制 Demo · 构建运行手册
|
# MVP 远程控制 Demo · 构建运行手册
|
||||||
|
|
||||||
> 项目路径:`D:\wordkb\2026-10-22-15-52-47\mvp-remote\`
|
> 项目路径:`D:\code\remote\`
|
||||||
> 形态:Rust workspace,三个 crate:`common`(协议)、`agent`(被控端)、`viewer`(主控端)
|
> 形态:Rust workspace:`common`(协议)、`agent-core`(平台无关会话层)、`agent`(Windows 被控端)、`linux-agent`(Linux 被控端)、`viewer`(主控端)
|
||||||
|
|
||||||
## 1. 功能范围(量级一 Demo)
|
## 1. 功能范围(量级一 Demo)
|
||||||
|
|
||||||
@ -10,14 +10,27 @@
|
|||||||
- ✅ 验证码认证(默认 123456)
|
- ✅ 验证码认证(默认 123456)
|
||||||
- ✅ 键鼠远程控制(SendInput 注入,虚拟桌面绝对坐标 + 扫描码)
|
- ✅ 键鼠远程控制(SendInput 注入,虚拟桌面绝对坐标 + 扫描码)
|
||||||
- ✅ 桌面静止时低频重发(保持画面新鲜)
|
- ✅ 桌面静止时低频重发(保持画面新鲜)
|
||||||
- ❌ 暂无:硬件编码、P2P 打洞/中继(当前仅局域网/可直连地址)、音频、文件传输、Linux 被控
|
- ✅ Linux 被控端(X11 采集 + XTest 注入,与 Windows 端共用同一套协议与会话层)
|
||||||
|
- ❌ 暂无:硬件编码、P2P 打洞/中继(当前仅局域网/可直连地址)、音频、文件传输
|
||||||
|
|
||||||
## 2. 环境要求
|
## 2. 环境要求
|
||||||
|
|
||||||
- Windows 8+(被控端需要 DXGI Duplication;主控端 Win7 亦可)
|
- Windows 8+(被控端需要 DXGI Duplication;主控端 Win7 亦可)
|
||||||
- 工具链已全部部署在 D 盘(无需再装):
|
- Rust(GNU host:`x86_64-pc-windows-gnu`)+ MinGW-w64,已验证可用组合:
|
||||||
- Rust (GNU):`uurev/tools/rust/cargo/bin`
|
- `rustup default stable-x86_64-pc-windows-gnu`,装到 `%USERPROFILE%\.cargo\bin`
|
||||||
- MinGW-w64 14.2:`uurev/tools/mingw64/bin`
|
- MinGW-w64:`D:\mingw64\bin`(gcc 16.2.0)
|
||||||
|
- 构建时 PATH 需同时包含上面两个目录(缺 MinGW 会在链接 openh264 的 C++ 时失败)
|
||||||
|
- **cargo 源**:国内直连 crates.io 会超时(cargo 不读系统代理),需配置镜像。已在本机
|
||||||
|
`%USERPROFILE%\.cargo\config.toml` 写入 USTC 源:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[source.crates-io]
|
||||||
|
replace-with = "ustc"
|
||||||
|
[source.ustc]
|
||||||
|
registry = "sparse+https://mirrors.ustc.edu.cn/crates.io-index/"
|
||||||
|
```
|
||||||
|
|
||||||
|
> 注:旧版本手册写的 `D:\wordkb\...\uurev\tools\rust` 路径已不存在,按上面重新准备工具链即可。
|
||||||
|
|
||||||
## 3. 构建
|
## 3. 构建
|
||||||
|
|
||||||
@ -74,3 +87,72 @@ viewer.exe --addr <被控端IP> --port 47800 --code 888888
|
|||||||
3. QUIC 多流:视频/控制/键鼠分离流(键鼠零阻塞)
|
3. QUIC 多流:视频/控制/键鼠分离流(键鼠零阻塞)
|
||||||
4. P2P 打洞 + coturn 中继(自建信令服务器)
|
4. P2P 打洞 + coturn 中继(自建信令服务器)
|
||||||
5. 被控端服务化(Windows Service + 会话切换处理)
|
5. 被控端服务化(Windows Service + 会话切换处理)
|
||||||
|
|
||||||
|
## 8. Linux 被控端(linux-agent)
|
||||||
|
|
||||||
|
### 8.1 架构:会话层抽到 agent-core
|
||||||
|
|
||||||
|
平台差异只留在采集/注入两个模块里,QUIC 握手、认证、编码、丢帧、输入分发全部共用,
|
||||||
|
**主控端 viewer 不需要任何改动**:
|
||||||
|
|
||||||
|
| crate / 文件 | 职责 | 平台 |
|
||||||
|
|---|---|---|
|
||||||
|
| `agent-core/src/lib.rs` | QUIC/TLS 握手、验证码认证、H.264 编码下行、latest-wins 丢帧、输入分发 | 共用 |
|
||||||
|
| `agent/src/capture.rs` | DXGI Desktop Duplication 采集 | Windows |
|
||||||
|
| `agent/src/input.rs` | SendInput 注入 | Windows |
|
||||||
|
| `linux-agent/src/capture.rs` | X11 `GetImage` 采集 + 整帧变更检测 | Linux |
|
||||||
|
| `linux-agent/src/input.rs` | XTest 注入 + 扫描码→X11 keycode 映射 | Linux |
|
||||||
|
|
||||||
|
两端靠 `ScreenCapturer` / `InputInjector` 两个 trait 注入,编码档位(5Mbps / 30fps / 关键帧周期 300)
|
||||||
|
与线格式完全一致;协议**未做任何改动**。
|
||||||
|
|
||||||
|
### 8.2 构建
|
||||||
|
|
||||||
|
Windows 侧照旧(`default-members` 已排除 linux-agent,产物仍是 agent.exe / viewer.exe):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export PATH="$USERPROFILE/.cargo/bin:/d/mingw64/bin:$PATH" # Git Bash 写法
|
||||||
|
cargo build --release
|
||||||
|
```
|
||||||
|
|
||||||
|
Linux 侧显式指定包(交叉编译不推荐,用容器最省事):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo build --release -p linux-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
### 8.3 运行
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 被控 Linux 机器(Xorg 或 XWayland 均可,依赖 DISPLAY 环境变量)
|
||||||
|
./linux-agent --port 47800 --code 123456
|
||||||
|
# 主控端(Windows)命令行不变
|
||||||
|
viewer.exe --addr <被控机IP> --port 47800 --code 123456
|
||||||
|
```
|
||||||
|
|
||||||
|
### 8.4 冒烟测试(不需要图形桌面)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/linux-smoke.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
脚本自建 Xvfb 虚拟屏后跑端到端测试,覆盖:X11 采集 → openh264 编码 → QUIC 下行解帧、
|
||||||
|
验证码认证(含**错误验证码被拒**)、Ping/Pong、静止桌面下的关键帧恢复(断言拿到 IDR),
|
||||||
|
以及**输入注入实证**——注入鼠标移动后,从另一条 X11 连接读回指针真实坐标。
|
||||||
|
|
||||||
|
### 8.5 键码映射(两端最关键的差异)
|
||||||
|
|
||||||
|
协议里键盘沿用 Windows 口味的 PS/2 Set-1 扫描码,而 X11 要的是 keycode,两者**并非处处等价**:
|
||||||
|
|
||||||
|
- 主键区:满足 `keycode = scancode + 8`;
|
||||||
|
- **E0 扩展键编号体系完全不同**(`↑` 是 0x48,但 X11 keycode 是 111 而不是 0x50)——必须查表。
|
||||||
|
|
||||||
|
映射表在 `linux-agent/src/input.rs::keycode_of`,带单元测试防回归。
|
||||||
|
|
||||||
|
### 8.6 Linux 侧已知限制
|
||||||
|
|
||||||
|
1. 仅支持 32bpp TrueColor 视觉(绝大多数桌面满足),其他格式启动即明确报错而非静默出错;
|
||||||
|
2. 采集走核心协议 `GetImage`,每帧经 X socket 全量传输;**MIT-SHM 共享内存 + XDamage 脏区**
|
||||||
|
是下一步优化(锚点已写在 `linux-agent/src/capture.rs` 顶部注释);
|
||||||
|
3. 仅覆盖 X11 / XWayland;纯 Wayland(无 XWayland)需另走 PipeWire 门户;
|
||||||
|
4. 键位表按 Xorg evdev 标准 keycode 编写,自定义键盘布局可能需要调整。
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user