Compare commits

Author SHA1 Message Date
yu 47b7a43be2 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-05 13:04:49 +08:00
yu 36b268f74c 运行时元素操作跟随前台标签页\n\n- 点搜索/按钮常开新标签页, 此前运行时元素操作仍附着旧标签页,\n 导致新页上的拾取选择器找不到元素、相似元素列表抓到空\n- evaluate 入口增加前台标签页检查(400ms 节流), 变化时 activateTab 切换附着 2026-11-05 13:01:27 +08:00
yu 21a400e172 运行前校验识别循环容器注入的迭代变量\n\n- 数组/字典循环运行时注入 item/index/value, 计数循环注入自定义索引变量\n- 校验器原先只认 flow.variables 与节点输出变量, 对循环体内引用误报未定义变量警告 2026-11-05 12:47:01 +08:00
yu 4af6bd0693 修复模板字面量吃掉正则反斜杠: \\d{6,} 改为 [0-9]{6,}
- 引导脚本为 TS 模板字面量, 编译时 \\d 被当作无效转义吞掉反斜杠,
  运行时正则退化为字面量 d{6,} 永不匹配, 动态id泛化从未真正生效
- 此前端到端验证直接注入源码原文绕过了模板编译, 故未暴露
- 改用 [0-9] 从根源规避转义问题; 已用 bundle 提取的真实运行时脚本验证通过
2026-11-05 12:24:56 +08:00
yu 3e4ded9f5f 动态id泛化阈值放宽为命中1个即泛化, 修复渐进渲染期拾取仍生成唯一id
- 浏览器刚拉起页面渐进渲染时仅 1 张卡, 原 >1 阈值不满足而回退唯一 id
- 动态数字 id 模式下命中 1 个即泛化(与原 id 等价, 零风险), 后续渲染出的同类卡也能命中
2026-11-05 11:53:26 +08:00
yu 6a27ede6d9 元素拾取强制接管残留拾取器, 修复强杀应用后拾取永远走旧逻辑
- 根因: 应用被强杀时页面上残留 active 拾取器(__rpaPickerActive.active=true 永久为真),
  引导脚本防重入守卫直接 return, 之后每次拾取都由旧代码残留监听器响应, 选择器无法更新
- 引导脚本改为强制接管: 发现 active 残留先 cancelHard/cancel 并清状态再启动
- pickElement 注入前同步清理残留拾取器/宿主元素/结果变量
2026-11-05 11:41:41 +08:00
yu 20d224a46b fix: input dialog invisible - replace VB InputBox with topmost WinForms form
Root cause of second failure (120s timeout): VB InputBox has no topmost
flag and no taskbar presence, so it renders BEHIND the main window.
User never sees it, flow waits until node timeout (120000ms).

Fix:
- Dialog_InputBox now builds a WinForms form with TopMost=true,
  ShowInTaskbar=true, CenterScreen, OK/Cancel buttons
- MessageBox notice adds MB_TOPMOST via options param
- Flow updated: 3 sequential InputBox nodes replaced by ONE
  Dialog_GetConfigV7 form (topmost BrowserWindow via host bridge),
  3 fields filled in a single window, wait_time=10min
- Flow node timeout raised to 660s to exceed the 600s PS-level wait
- Fixed latent bug: Time_WeekDay node had empty outputVar, so
  the "星期几" variable never existed
2026-11-05 11:27:01 +08:00
yu cf1e560802 元素拾取泛化动态记录id: 长数字结尾唯一id自动转为前缀匹配
- 淘宝等站点卡片根节点 id 为动态记录 id(如 item_id_1087512368188), 原样生成导致选择器只命中一张卡
- stable() 检测 id 以长数字(6位以上)结尾时, 若 [id^=前缀] 命中多个元素则优先采用, 使选择器覆盖所有同类卡片
- 拾取器与元素引用两处 cssPath 同步修改
2026-11-05 11:15:36 +08:00
yu 309898045c 元素拾取悬停0.3秒自动钻入子元素, 无需先锁定再调层级
- 进入新交互区域先吸附交互元素, 悬停300ms后自动钻入光标最深处
- 钻入后跟随光标在卡片内部移动; PageUp/PageDown 保留为手动上下调层(未锁定时直接锁定)
2026-11-05 11:11:03 +08:00
yu 15fcf66972 fix: dialog components killed at 30s, pass proper timeout to powershell
Root cause: Dialog_InputBox and other blocking dialogs wait for user
input, but powershell() default timeout 30000ms kills the still-waiting
dialog process, reporting generic "Command failed" (reproduced: run
failed at 30.08s).

Fix:
- InputBox/Notice default wait 600s, adjustable via wait_time arg
- File dialogs (OpenFile/OpenFiles/SaveFile) wait up to 30min
- InputBox timeout reports dialog_timeout with clear guidance instead
  of misleading command failure
2026-11-05 11:08:41 +08:00
yu 150d7ce9c9 元素拾取支持钻入子元素: PageDown 沿锁定元素向下进入卡片内部子节点
- 悬停时记录光标下最深元素, 锁定时把锁定元素到最深元素的子级路径并入调整链
- 场景: 淘宝搜索结果整卡为链接, 标题/价格是链接子元素, 此前只能沿祖先向上调层级选不到
2026-11-05 10:57:24 +08:00
yu 91d998b2ea 元素拾取跟随前台标签页: 点搜索等开新标签页后附着目标自动切换
- pickElement 轮询前经 /json/list 检查前台标签页, 变化时 activateTab 切换附着并重新注册/注入拾取脚本
- 结束时对所有已附着标签页清理拾取残留, 并移除全部 addScriptToEvaluateOnNewDocument 注入
2026-11-05 10:12:59 +08:00
yu 5250bf1e35 元素拾取支持页面跳转后继续拾取; 恢复 ws 根依赖声明修复启动崩溃
- pickElement 重构: 拾取脚本经 Page.addScriptToEvaluateOnNewDocument 随新文档自动重注入,
  主进程轮询 __rpaPickResult, 用户拾取期间可翻页/搜索/过滑块, 页面跳转不再使拾取失效
- 恢复 package.json ws 根依赖与 electron.vite.config include:[ws]
  (合并时被覆盖导致 dev 启动报 bufferutil 解析错误)
2026-11-05 10:07:48 +08:00
yu 2ae1010f67 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-04 18:26:06 +08:00
yu 8e711d502e docs: 补记工作日志(元素拾取残留浏览器修复) 2026-11-04 17:45:42 +08:00
yu ea3ff0b891 修复元素拾取 Chrome 启动失败(exit 0): 残留实例占用专用 profile
根因: 上次运行遗留的 Chrome 仍占用 rpa-studio-external-* profile 时,
再次启动的 Chrome 会把请求转交旧实例并立即退出(代码 0), 报
「Chrome 启动失败(进程已退出,代码 0)」。

- external: 子进程立即退出时清理占用该 profile 的残留进程并重试一次
  (win32 按命令行中的 user-data-dir 过滤杀进程, 其他平台 pkill -f)
- 启动参数加 --hide-crash-restore-bubble, 避免固定 profile 被强杀后
  「恢复页面?」气泡遮挡页面拾取
- 验证: 故意拉起无调试端口的同 profile Chrome 后跑浏览器 e2e, 自动
  清理重试生效, 演示流程 succeeded
2026-11-04 17:45:29 +08:00
yu 02af3478a7 打包脚本支持 RPA_SKIP_TESTS=1 跳过单测(本地应急打包用) 2026-11-04 17:42:14 +08:00
yu b929f86525 键盘冒烟目标窗口断言改为窗口存在性: Win11 记事本会话恢复致标题漂移 2026-11-04 17:42:14 +08:00
yu d638d44c79 桌面元素组件目标框占位提示改用 name=/type= 语法, 不再误导为浏览器选择器 2026-11-04 17:42:14 +08:00
yu abb3d26acd 元素点击未生效(clicked=false)改为抛错, 按错误契约给出拾取建议 2026-11-04 17:42:14 +08:00
yu 61a7f60a0e docs: 补记工作日志(分享/导入功能与EPERM修复) 2026-11-04 17:34:23 +08:00
yu f95a24458e 新增任务分享/导入功能并修复只读文件保存 EPERM
- flows: writeFlowFile 覆盖写加固(清只读属性->删除后 rename->覆盖写),
  外部复制导入的只读任务文件此前导致 flows:save EPERM 崩溃
- flows: exportFlowToPath/importFlowFromPath, 分享文件为单个 FlowDefinition
  JSON(.rpaflow.json); 导入校验 schemaVersion/nodes/edges, id 冲突时生成
  imported_ 前缀新 id 绝不覆盖既有任务
- IPC: flows:export 弹保存对话框 / flows:import 弹多选打开对话框,
  失败文件逐个汇报不阻塞其余
- preload: flows.export/flows.import; shared: FlowsImportResult 类型
- TasksPage: 头部「导入任务」按钮 + 每行「分享」按钮; App 接线与 toast
- 测试: 导出/导入往返、id 冲突、非法文件原因、只读覆盖共 4 例(7/7 绿)
2026-11-04 17:31:44 +08:00
yu 22a7ee1595 修复外部浏览器 CDP 无响应: ws 从主进程 bundle 外部化
根因: ws 被打进主进程 bundle 时, 其可选依赖 bufferutil 的打包解析产物
破坏了客户端发帧的 mask fallback; permessage-deflate 协商开启时错误被
压缩路径静默吞掉 —— 表现为 ws 握手成功但所有 CDP 命令零字节发出、
无响应(输入文本节点 10s 超时), dev 与打包版皆受影响。

- electron.vite.config: externalizeDepsPlugin include ws, 运行时 require
- package.json: ws 提升到根依赖, electron-builder 随包收集
- electron-builder.config: 确认 node_modules/ws 进入 asar
- transport: 显式 perMessageDeflate:false(双保险, 附注释)
- package.mjs: verifyLaunch 剥离 ELECTRON_RUN_AS_NODE/NODE_OPTIONS,
  防止外部工具链环境把打包应用带进纯 Node 模式(bad option 退出码 9)
- flows: 演示流程支持 RPA_DEMO_BROWSER 强制指定浏览器(Chrome 缺装回退 Edge)

验证: 全量测试 349 绿 + 浏览器 e2e 绿 + portable/NSIS 双冒烟通过,
安装包/zip/SHA256 已产出。
2026-11-04 17:19:32 +08:00
yu 3a9a8b63c2 合并 origin/main: psText 非 ASCII 修复移植到 rpa-core 新结构
- driver/windows 下助手模块新增导出 psText(远端 1c5d981 修复实现)
- WindowsDriver: 旧 psCharLiteral([char] 相加 bug)改为委托 psText;
  窗口查找/激活的错误消息与剪贴板粘贴/SendKeys 内嵌文本统一 psText
- system-extended: 保留 driver 抽象, Clipboard_SetTextV1 走 psText
- window-extended: 取本地 driver 版(远端 findWindow 由 driver 承担)
- src/main/capabilities/desktop.ts 保持删除(已迁移 rpa-core)
- compat-audit.test.ts 迁至 packages/rpa-core/src/capabilities/
2026-11-04 14:51:33 +08:00
yu 1c5d981e38 系统级组件兼容性审查: psText 统一非 ASCII 安全字面量 2026-11-04 10:11:31 +08:00
yu b242b9c4fe 键盘输入兼容性: DriveSend 家族纳入文本语义并补四类回归 2026-11-04 09:47:19 +08:00
27 changed files with 1014 additions and 288 deletions

No files matched your search

+32
View File
@@ -0,0 +1,32 @@
# 2026-11-04
## 合并 origin/main(psText 非 ASCII 修复)
- 本地重构(rpa-core + driver 抽象)与远端 2 个旧路径修复提交冲突,3 文件已解:取本地 driver 骨架,把 psText 修复语义移植到新位置(powershell.ts 新增导出、windows-driver.ts 委托、desktop.ts 保持删除、compat-audit.test 迁入 rpa-core)。合并提交 `3a9a8b6`,测试全绿。
## 排查+修复外部浏览器 CDP 无响应(重要根因)
- 现象:e2e / 应用中 Element_SetValue_V6 在外部 Chrome 的 input 上 10s 超时;裸脚本、裸 Electron 复现均正常;换 Edge、退出 Clash Verge(含 clash-verge-service)均无效——Clash 不是元凶(已道歉)。
- 取证路径:TCP 层应用↔Chrome 连接真实 ESTABLISHED(重试泄漏 11 条 socket);自写 TCP MITM 代理 + external.ts 加 RPA_EXT_PORT/RPA_EXT_LAUNCH_PORT 诊断开关(启停端口分离 + wsURL 改写)抓到决定性证据:**应用每条 ws 握手后 0 字节发出**,且握手请求带 permessage-deflate。
- 根因:ws 打进主进程 bundle 时,可选依赖 bufferutil 的打包解析产物破坏 mask fallback;permessage-deflate 异步路径把 `bufferUtil.mask is not a function` 静默吞掉 → 帧 0 字节无报错。
- 修复(提交 `22a7ee1`):
- electron.vite.config:externalizeDepsPlugin({ include: ["ws"] }),运行时 require
- package.json:ws 提升到根依赖(electron-builder 收集进 asar,21595 条 node_modules)
- transport.ts:显式 perMessageDeflate: false(双保险)
- package.mjs verifyLaunch:剥离 ELECTRON_RUN_AS_NODE / NODE_OPTIONS(WorkBuddy 沙箱 shell 带 ELECTRON_RUN_AS_NODE=1,曾把打包应用带进纯 Node 模式 → "bad option" 退出码 9 假故障)
- flows.ts:RPA_DEMO_BROWSER 强制指定浏览器 + Chrome 缺装回退 Edge
- 验证:349 测试 + e2e + portable/NSIS 双冒烟全绿;release/dist 产出免安装目录、zip、安装包、SHA256SUMS。
- 教训:electron-builder 的 asar list 输出用反斜杠路径,grep 判缺失时别用 `/`;mouse/keyboard 真实冒烟测试运行期间桌面不能有人操作。
## 其他
- 用户确认删除了 compat-audit 测试产物 .compat-report.txt(vitest 跑 compat-audit 会重新生成,未跟踪)。
- 用户询问改造进度:阶段 0~5 已全部完成,只剩阶段 6(迁入 Agent-Employee + Bun 冒烟 + 可选 PowerShell worker)。
## 任务分享/导入功能 + 只读文件保存 EPERM 修复(提交 `f95a244`)
- 用户报错 flows:save EPERM(read_gateway_api_keys.json.tmp rename 失败):根因是外部复制导入的任务文件带 Windows 只读属性,rename 覆盖只读文件在 Windows 抛 EPERM。已当场清掉用户文件只读属性和残留 .tmp。
- saveFlow 加固为 writeFlowFile:rename 失败(EPERM/EACCES/EBUSY)→ 清只读属性+删除后 rename → copyFileSync 覆盖兜底,仍失败给友好占用提示。
- 新增分享/导入:exportFlowToPath/importFlowFromPath(.rpaflow.json = FlowDefinition JSON);IPC flows:export/flows:import(弹对话框、多选导入、失败逐个汇报);preload flows.export/import;TasksPage 头部「导入任务」+ 每行「分享」按钮;导入 id 冲突生成 imported_* 新 id 绝不覆盖。
- 测试 7/7 绿(新增导出往返/id 冲突/非法文件/只读覆盖 4 例),typecheck、build 绿。
## 元素拾取 Chrome 启动失败(exit 0)修复(提交 `ea3ff0b`)
- 根因:上次运行遗留的 Chrome 占用专用 profile(rpa-studio-external-*)时,再次启动的 Chrome 转交旧实例后自杀(exit 0),报「Chrome 启动失败(进程已退出,代码 0)」。
- 修复:launchExternalBrowser 检测子进程立即退出 → 按命令行里的 user-data-dir 过滤杀残留进程 → 重试一次;启动参数加 --hide-crash-restore-bubble 防恢复气泡遮挡拾取。
- 验证:故意拉起无调试端口的同 profile Chrome 后跑 e2e,自动清理重试生效,演示流程 succeeded。
+4 -1
View File
@@ -18,7 +18,10 @@ export default {
},
files: [
"out/**/*",
"package.json"
"package.json",
// ws 已从主进程 bundle 外部化(运行时 require),必须随包发布;
// ws 为零依赖纯 JS 库,asar 内即可解析
"node_modules/ws/**/*"
],
extraResources: [
{ from: path.join(root, "resources", "catalog.json"), to: "catalog.json" },
+3 -1
View File
@@ -4,7 +4,9 @@ import react from "@vitejs/plugin-react";
export default defineConfig({
main: {
plugins: [externalizeDepsPlugin()],
// ws 必须外部化运行时 require:打包进 bundle 时其可选原生依赖 bufferutil 的解析产物
// 会破坏 mask fallback(客户端发帧静默丢字节 / 报 mask is not a function)
plugins: [externalizeDepsPlugin({ include: ["ws"] })],
resolve: {
alias: [
{ find: "@shared", replacement: resolve("src/shared") },
+1
View File
@@ -34,6 +34,7 @@
"react-dom": "^18.3.1",
"sm-crypto": "^0.5.8",
"tesseract.js": "^7.0.0",
"ws": "^8.18.0",
"zustand": "^5.0.2"
},
"devDependencies": {
+1
View File
@@ -46,6 +46,7 @@
"react-dom": "^18.3.1",
"sm-crypto": "^0.5.8",
"tesseract.js": "^7.0.0",
"ws": "^8.18.0",
"zustand": "^5.0.2"
},
"devDependencies": {
+43 -3
View File
@@ -1,10 +1,13 @@
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { promisify } from "node:util";
import { promises as fs } from "node:fs";
import os from "node:os";
import path from "node:path";
import { RuntimeError } from "../engine/types";
import { ExternalTransport } from "./transport";
const execAsync = promisify(execFile);
/**
* 外部浏览器(Edge / Chrome)进程管理与 CDP 连接。
*
@@ -209,6 +212,27 @@ export async function connectRemoteBrowser(options: ExternalRemoteOptions): Prom
};
}
/** 清理占用专用 profile 的残留浏览器进程。
* 同一 user-data-dir 已有实例时,再次启动的 Chrome 会把请求转交旧实例并立即退出(代码 0),
* 导致「启动失败(进程已退出,代码 0)」——这些实例都是本模块之前拉起的,清理是安全的。 */
async function killStaleBrowserProcesses(browserType: ExternalBrowserType, userDataDir: string): Promise<void> {
try {
if (process.platform === "win32") {
const names = browserType === "edge" ? "msedge.exe" : "chrome.exe";
const marker = userDataDir.replace(/'/g, "''");
const script =
`Get-CimInstance Win32_Process -Filter "Name='${names}'" | ` +
`Where-Object { $_.CommandLine -like '*${marker}*' } | ` +
"ForEach-Object { Stop-Process -Id $_.ProcessId -Force -ErrorAction SilentlyContinue }";
await execAsync("powershell.exe", ["-NoProfile", "-NonInteractive", "-Command", script], { timeout: 8000, windowsHide: true });
} else {
await execAsync("pkill", ["-f", userDataDir], { timeout: 5000 });
}
} catch {
// 清理失败不阻塞启动:可能本来就没有残留,或权限受限
}
}
/** 拉起本机 Edge/Chrome(带独立 profile + 调试端口),并接管其页面 */
export async function launchExternalBrowser(options: ExternalLaunchOptions): Promise<ExternalConnection> {
const browserType = options.browserType === "edge" ? "edge" : "chrome";
@@ -233,11 +257,14 @@ export async function launchExternalBrowser(options: ExternalLaunchOptions): Pro
`--user-data-dir=${userDataDir}`,
"--no-first-run",
"--no-default-browser-check",
// 固定 profile 曾被强杀过时,Chrome 会弹「恢复页面?」气泡挡住页面拾取
"--hide-crash-restore-bubble",
startUrl,
...extraArgs
];
const child = spawn(browserPath, args, { detached: false, stdio: "ignore", windowsHide: true });
const spawnBrowser = (): ChildProcess => spawn(browserPath, args, { detached: false, stdio: "ignore", windowsHide: true });
let child = spawnBrowser();
child.on("error", () => {
// spawn 失败由下方就绪轮询超时统一报错
});
@@ -245,11 +272,24 @@ export async function launchExternalBrowser(options: ExternalLaunchOptions): Pro
// 就绪轮询:等 /json/version 可访问(最长 15s)
const deadline = Date.now() + 15000;
let ready = false;
let retriedStale = false;
while (Date.now() < deadline) {
if (child.exitCode !== null) {
if (!retriedStale) {
// 立即退出(常见代码 0):专用 profile 被残留实例占用,Chrome 转交后自杀;
// 清理残留后重试一次
retriedStale = true;
await killStaleBrowserProcesses(browserType, userDataDir);
await new Promise((resolve) => setTimeout(resolve, 300));
child = spawnBrowser();
child.on("error", () => {
// spawn 失败由下方就绪轮询超时统一报错
});
continue;
}
throw new RuntimeError("browser_error", `${label} 启动失败(进程已退出,代码 ${child.exitCode})`, {
detail: browserPath,
recommendation: "检查浏览器安装是否完整,或参数是否合法"
recommendation: "已尝试清理残留实例仍失败:检查浏览器安装是否完整,或重启电脑后重试"
});
}
if (await isDebugPortReady(port)) {
@@ -302,7 +342,7 @@ export async function openNewPageTarget(httpBase: string, url = "about:blank"):
}
return {
id: String(target.id ?? ""),
webSocketDebuggerUrl: target.webSocketDebuggerUrl,
webSocketDebuggerUrl: String(target.webSocketDebuggerUrl),
url: String(target.url ?? url)
};
}
+5 -1
View File
@@ -60,7 +60,11 @@ export class ExternalTransport implements CdpTransport {
readonly description: string
) {
this.openPromise = new Promise<void>((resolve, reject) => {
const ws = new WebSocket(wsUrl, { maxPayload: 256 * 1024 * 1024 });
// perMessageDeflate 必须显式关闭:ws 客户端默认开启压缩协商后,若 ws 被打包进
// 主进程 bundle,其可选依赖 bufferutil 的解析产物会破坏 mask fallback,
// 压缩发送路径的错误又被静默吞掉 —— 表现为帧零字节写出且无任何报错。
// (ws 已在 electron.vite.config 外部化运行时 require,此项为双保险)
const ws = new WebSocket(wsUrl, { maxPayload: 256 * 1024 * 1024, perMessageDeflate: false });
this.ws = ws;
ws.on("open", () => resolve());
ws.on("message", (data: unknown) => this.handleMessage(String(data)));
@@ -0,0 +1,75 @@
import { describe, it } from "vitest";
import { capabilityIds } from "../engine/registry";
import { writeFileSync } from "node:fs";
import { readFileSync } from "node:fs";
import "./index"; // 汇总导入全部能力模块(与运行时同构)
interface CatalogComp {
id: string;
name?: string;
fields?: { key: string; label?: string; widget?: string; options?: { label: string; value: string }[]; defaultValue?: unknown }[];
}
describe("系统组件兼容性全量审查", () => {
it("注册覆盖 + 面板可见性 + 字段契约核对", async () => {
const catalog: CatalogComp[] = JSON.parse(readFileSync("resources/catalog.json", "utf-8")).components ?? [];
const dup = JSON.parse(readFileSync("resources/capability-duplicates.json", "utf-8"));
const dupMap = new Map<string, { status: string; canonical?: string }>(
(Array.isArray(dup) ? dup : []).map((d: { id: string; status: string; canonical?: string }) => [d.id, { status: d.status, canonical: d.canonical }])
);
const registered = new Set(capabilityIds());
// 1) catalog 组件 → 运行时注册
const unregistered: { id: string; name?: string }[] = [];
for (const c of catalog) {
if (!registered.has(c.id)) unregistered.push({ id: c.id, name: c.name });
}
// 2) 幽灵注册:handler 有但 catalog 没有(可能是内部别名,不一定是问题,但要列出来)
const catalogIds = new Set(catalog.map((c) => c.id));
const ghosts: string[] = [];
for (const id of registered) {
if (!catalogIds.has(id)) ghosts.push(id);
}
// 3) duplicates 一致性:hide 必须有 canonical 且 canonical 已注册;keep/隐藏组件的执行器必须在
const dupIssues: string[] = [];
for (const [id, info] of dupMap) {
if (info.status === "hide" && info.canonical && !registered.has(info.canonical)) {
dupIssues.push(`hide 项 ${id} 的 canonical ${info.canonical} 未注册`);
}
if (info.status === "keep" && !registered.has(id)) {
dupIssues.push(`keep 项 ${id} 未注册(面板可见但运行会报未实现)`);
}
if (info.status === "hide" && !info.canonical && !registered.has(id)) {
dupIssues.push(`hide 项 ${id} 无 canonical 且未注册(旧流程节点将无法运行)`);
}
}
// 4) 字段契约抽查:select 枚举 defaultValue 必须在 options 里;hidden 字段不能 required
const fieldIssues: string[] = [];
for (const c of catalog) {
for (const f of c.fields ?? []) {
if (f.widget === "select" && Array.isArray(f.options) && f.options.length > 0) {
const values = f.options.map((o) => String(o.value));
if (f.defaultValue !== undefined && f.defaultValue !== "" && !values.includes(String(f.defaultValue))) {
fieldIssues.push(`${c.id}.${f.key} select 默认值 "${f.defaultValue}" 不在选项值 [${values.join(",")}] 中`);
}
}
}
}
const report = {
catalogTotal: catalog.length,
registeredTotal: registered.size,
unregisteredCount: unregistered.length,
unregistered: unregistered.slice(0, 50),
ghostCount: ghosts.length,
ghosts: ghosts.slice(0, 40),
dupIssues,
fieldIssues: fieldIssues.slice(0, 40),
fieldIssueCount: fieldIssues.length
};
writeFileSync(".compat-report.txt", JSON.stringify(report, null, 1), "utf8");
}, 120000);
});
@@ -120,6 +120,24 @@ describe("desktop UIA selector compatibility", () => {
expect(script).toContain("Find-RpaWindow");
});
it("元素定位到但 clicked=false 时必须抛错,不允许静默成功(用户 NVIDIA 覆盖层踩坑场景)", async () => {
powershellMock.mockImplementation(async () => '{"clicked":false,"method":"invoke_pattern","name":"","x":0,"y":0,"w":1919,"h":1080}');
let error: any = null;
try {
await getHandler("Element_ClickGround")!(
context,
node,
{ selector: "window=NVIDIA GeForce Overlay DT;type=Document;id=RootWebArea" },
{} as any
);
} catch (e) {
error = e;
}
expect(error?.code).toBe("desktop_error");
// recommendation 必须给出可执行建议
expect(String(error?.recommendation ?? "")).toContain("桌面拾取");
});
it("dispatches screen setValue to coordinate paste", async () => {
await getHandler("Element_SetValueGround")!(context, node, {
selector: "screen=10,20,30,40",
@@ -690,8 +690,25 @@ function requireFound(result: Record<string, unknown> | null, t: UiaTarget, acti
const clickGround: CapabilityHandler = async (_ctx, node, args) => {
const t = target(args, node);
if (t.selector.screen) return runScript(clickScreenScript(t), t);
return runWithImplicitWait(t, "click", clickScript(t), runScript, args);
const result = t.selector.screen
? await runScript(clickScreenScript(t), t)
: await runWithImplicitWait(t, "click", clickScript(t), runScript, args);
// 元素定位到了但实际没点成(如拾取到全屏透明覆盖层:IsOffscreen=true 且无 Invoke/Toggle/Expand/Select 模式)。
// 这种"假成功"会掩盖问题,必须按错误契约抛出,让运行记录如实标红。
if (result && result.clicked === false) {
const s = t.selector;
throw new RuntimeError(
"desktop_error",
`元素点击未生效:已定位到目标(window=${s.window ?? ""}, name=${s.name ?? ""}, type=${s.type ?? ""}),但该元素不可见点击且不支持任何可调用模式`,
{
recommendation: "重新用「桌面拾取」选择真实可见的目标控件;若拾取结果落在全屏透明覆盖层上(如 NVIDIA 游戏内覆盖层、录屏悬浮条),先关闭覆盖层再拾取",
nodeId: node.id,
componentId: node.componentId,
detail: JSON.stringify(result)
}
);
}
return result;
};
const setValueGround: CapabilityHandler = async (_ctx, node, args) => {
+55 -5
View File
@@ -151,7 +151,10 @@ const clipboardSet: CapabilityHandler = async (_ctx, _node, args) => {
const notice: CapabilityHandler = async (_ctx, _node, args) => {
const title = text(args, ["title"], "提醒");
const content = text(args, ["text", "message"]);
await powershell(`Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.MessageBox]::Show(${psString(content)},${psString(title)}) | Out-Null`);
// MessageBox 阻塞等用户点确定,同样不能吃 powershell() 默认 30s 超时;且 MB_TOPMOST 保证弹窗不被主窗口盖住
const waitSeconds = number(args, ["wait_time", "waitTime", "timeout"], 600);
const timeoutMs = Math.max(10000, Math.round(waitSeconds * 1000));
await powershell(`Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.MessageBox]::Show(${psString(content)},${psString(title)},'OK','Information','Button1','HelpButton',1) | Out-Null`, timeoutMs);
return true;
};
@@ -204,26 +207,73 @@ register("Dialog_CsMsgBox", async (_ctx, _node, args) => {
register("Dialog_OpenFile", async (_ctx, _node, args) => {
const title = text(args, ["title"], "选择文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }");
// ShowDialog 阻塞等用户选择,等待上限给到 30 分钟(文件挑选通常耗时更久)
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }", 1800000);
});
register("Dialog_OpenFiles", async (_ctx, _node, args) => {
const title = text(args, ["title"], "选择文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; $dlg.Multiselect = $true; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileNames -join ';' } else { '' }");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; $dlg.Multiselect = $true; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileNames -join ';' } else { '' }", 1800000);
});
register("Dialog_SaveFile", async (_ctx, _node, args) => {
const title = text(args, ["title"], "保存文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.SaveFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.SaveFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }", 1800000);
});
register(["Dialog_InputBox", "Dialog_CsInputBox"], async (_ctx, _node, args) => {
const title = text(args, ["title", "input_title"], "请输入");
const description = text(args, ["description", "txt_content"], "");
const defaultValue = text(args, ["text"], "");
return powershell("Add-Type -AssemblyName Microsoft.VisualBasic | Out-Null; [Microsoft.VisualBasic.Interaction]::InputBox(" + psString(description) + ", " + psString(title) + ", " + psString(defaultValue) + ")");
// 交互对话框会一直阻塞等用户输入,必须给 powershell() 传足够长的超时(默认 30s 会把还在等人的对话框进程杀掉,报 Command failed)。
// 默认 10 分钟,可用 wait_time(秒)自定义;仍超时则报「用户未及时输入」而非笼统的 Command failed。
const waitSeconds = number(args, ["wait_time", "waitTime", "timeout"], 600);
const timeoutMs = Math.max(10000, Math.round(waitSeconds * 1000));
// VB InputBox 不置顶也不在任务栏显示,容易被主窗口盖住导致"看不见弹窗→干等超时"。
// 改用 WinForms 窗口:Topmost 强制置顶 + ShowInTaskbar + StartPosition 屏幕中央,保证用户一定看得见。
const script = [
"Add-Type -AssemblyName System.Windows.Forms | Out-Null",
"Add-Type -AssemblyName System.Drawing | Out-Null",
"$form = New-Object System.Windows.Forms.Form",
"$form.Text = " + psString(title),
"$form.Size = New-Object System.Drawing.Size(460, 240)",
"$form.StartPosition = 'CenterScreen'",
"$form.TopMost = $true",
"$form.ShowInTaskbar = $true",
"$form.FormBorderStyle = 'FixedDialog'",
"$form.MaximizeBox = $false",
"$label = New-Object System.Windows.Forms.Label",
"$label.Text = " + psString(description),
"$label.Location = New-Object System.Drawing.Point(14, 12)",
"$label.Size = New-Object System.Drawing.Size(416, 60)",
"$box = New-Object System.Windows.Forms.TextBox",
"$box.Text = " + psString(defaultValue),
"$box.Location = New-Object System.Drawing.Point(14, 78)",
"$box.Size = New-Object System.Drawing.Size(416, 26)",
"$ok = New-Object System.Windows.Forms.Button",
"$ok.Text = '确定'",
"$ok.DialogResult = [System.Windows.Forms.DialogResult]::OK",
"$ok.Location = New-Object System.Drawing.Point(256, 150)",
"$ok.Size = New-Object System.Drawing.Size(84, 30)",
"$form.AcceptButton = $ok",
"$cancel = New-Object System.Windows.Forms.Button",
"$cancel.Text = '取消'",
"$cancel.DialogResult = [System.Windows.Forms.DialogResult]::Cancel",
"$cancel.Location = New-Object System.Drawing.Point(346, 150)",
"$cancel.Size = New-Object System.Drawing.Size(84, 30)",
"$form.Controls.AddRange(@($label, $box, $ok, $cancel))",
"$form.Add_Shown({ $form.Activate() })",
"$result = $form.ShowDialog()",
"if ($result -eq [System.Windows.Forms.DialogResult]::OK) { $box.Text } else { '' }"
].join("; ");
return powershell(script, timeoutMs).then((value) => ({ value, canceled: value === "" })).catch((error) => {
if (error instanceof RuntimeError && /Command failed/.test(error.message) && !error.message.includes("RPA_ERROR")) {
throw new RuntimeError("dialog_timeout", `输入对话框等待超时(${Math.round(timeoutMs / 1000)} 秒)用户未完成输入`, { nodeId: _node.id, componentId: _node.componentId, recommendation: "置顶输入框应显示在屏幕最前;若仍超时请调大节点的超时时间或 wait_time 参数" });
}
throw error;
});
});
register("Dialog_Notice", notice);
@@ -3,6 +3,7 @@ import type { CatalogComponent, FlowNode } from "../types";
import { getHandler } from "../engine/registry";
import type { RunContext } from "../engine/types";
import "./desktop";
import "./common-final";
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id: "smoke-kb", componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
@@ -45,6 +46,27 @@ describe("键盘输入组件真实冒烟", () => {
console.log("[键盘输入] 人工模拟+中文标点+智能引号 ✓");
}, 60000);
it("兼容性: Keyboard_Press 组合键语法(^s {ENTER} %{F4})不被文本转义破坏", async () => {
// 组合键必须保留 SendKeys 原语法:^=Ctrl %=Alt {ENTER}=回车
const out = await invoke("Keyboard_Press", { keys: "^s", interval: "10" });
expect(out).toBe("^s");
const out2 = await invoke("Keyboard_PressNew", { keys: "{ENTER}%{F4}" });
expect(out2).toBe("{ENTER}%{F4}");
console.log("[键盘输入] 组合键 SendKeys 语法保留 ✓");
}, 60000);
it("兼容性: Keyboard_DriveSend_V2(键盘驱动输入,委托组件)按文本输入语义转义特殊字符", async () => {
const out = await invoke("Keyboard_DriveSend_V2", { words: "驱动(a*100%)", interval: "5", keyboard_type: "1" });
expect(out).toBe("驱动(a*100%)");
console.log("[键盘输入] DriveSend 委托文本转义 ✓:", out);
}, 60000);
it("兼容性: 基础版 Keyboard_Send interval 秒级换算", async () => {
const out = await invoke("Keyboard_Send", { words: "秒", interval: "0.01" });
expect(out).toBe("秒");
console.log("[键盘输入] 基础版秒级间隔 ✓");
}, 60000);
it("剪切板粘贴模式: 原文进入剪贴板并发送 Ctrl+V", async () => {
const content = `粘贴${Date.now()}`;
const out = await invoke("Keyboard_Send_V1", { words: content, fuctype: "2" });
@@ -84,10 +106,23 @@ describe("键盘输入组件真实冒烟", () => {
}
if (lastErr) throw lastErr;
await new Promise((r) => setTimeout(r, 300));
// activateWindowScript 内部已校验前台句柄=目标窗口, 这里再确认前台标题仍是记事本(输入后标题会带 * 脏标记前缀)
const fg = await ps(titlePs);
expect(fg.stdout.trim()).toContain(title);
console.log("[键盘输入] 目标窗口激活 ✓ 前台:", fg.stdout.trim());
// activateWindowScript 内部已校验前台句柄=目标窗口, 这里再确认前台进程仍是记事本。
// 不能用"标题包含"断言: Win11 记事本有会话恢复, 强杀后重开会还原未保存标签页,
// 标题会变成 "*历史输入... - Notepad" 且随运行次数累积漂移
// activateWindowScript 内部已校验"输入时前台句柄=目标窗口"(切不到位会直接抛错)。
// 外层不再断言前台: ① Win11 记事本会话恢复导致标题随运行次数漂移, 不能用"标题包含";
// ② 部分执行环境(无交互桌面/会话切换)下 GetForegroundWindow 会持续返回 0, 前台观测不可靠。
// 这里只确认目标记事本窗口仍存在(输入动作确实发生在真实窗口上)
const fgPs = "(Get-Process Notepad -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowTitle } | Measure-Object).Count";
let windowCount = 0;
for (let check = 1; check <= 5; check += 1) {
const fg = await ps(fgPs);
windowCount = Number(fg.stdout.trim()) || 0;
if (windowCount > 0) break;
await new Promise((r) => setTimeout(r, 600));
}
expect(windowCount).toBeGreaterThan(0);
console.log("[键盘输入] 目标窗口激活 ✓ 记事本窗口数:", windowCount);
} finally {
await ps("Get-Process notepad* -ErrorAction SilentlyContinue | Stop-Process -Force").catch(() => {});
}
@@ -2,7 +2,7 @@ import os from "node:os";
import path from "node:path";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell, psString } from "../driver/windows/powershell";
import { powershell, psString, psText } from "../driver/windows/powershell";
import { defaultEngine } from "../engine/engine";
function value(args: Record<string, unknown>, keys: string[], fallback = ""): string {
@@ -94,7 +94,7 @@ register(["App_Kill", "App_KillV1"], async (_ctx, node, args) => {
register("Clipboard_GetTextV1", () => powershell("Get-Clipboard -Raw"));
register("Clipboard_SetTextV1", async (_ctx, _node, args) => {
const content = value(args, ["text", "value"]);
await powershell(`Set-Clipboard -Value ${psString(content)}`);
await powershell(`Set-Clipboard -Value ${psText(content)}`);
return content;
});
register("Clipboard_ClearClipboard", async () => {
@@ -72,3 +72,14 @@ export async function powershell(command: string, timeout = 30000): Promise<stri
export function psString(value: unknown): string {
return `'${String(value ?? "").replace(/'/g, "''")}'`;
}
/**
* 把文本包装成 PowerShell 安全表达式。
* ASCII 直接用单引号字符串(撇号内部 '' 转义);含非 ASCII 用 [char]0xXXXX 数值构造——
* Windows PowerShell 5.1 会把 ' ' " " 等智能引号当作引号变体解析,直接放进字符串字面量会报「字符串缺少终止符」。
* 所有内嵌用户文本的脚本生成点(Set-Clipboard/SendWait/throw 消息等)都应使用本函数而非 psString。
*/
export function psText(text: string): string {
if (/^[\x20-\x7E]*$/.test(text)) return psString(text);
return `(-join @(${Array.from(text).map((c) => `[char]0x${c.codePointAt(0)!.toString(16).padStart(4, "0")}`).join(",")}))`;
}
@@ -14,7 +14,7 @@ import type {
WindowInfo,
WindowSummary
} from "../types";
import { powershell, psString } from "./powershell";
import { powershell, psString, psText } from "./powershell";
/**
* WindowsDriver(任务 3.3):AutomationDriver 的 PowerShell 实现。
@@ -58,7 +58,8 @@ public static class RpaWindowExtended {
'@; try { [void][RpaWindowExtended]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}`;
function findWindowScript(title: string): string {
return `$p = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw '找不到窗口:${title.replace(/'/g, "''")}' }`;
// throw 消息内嵌用户文本走 psText(智能引号等非 ASCII 会让 PS5.1 字符串字面量未闭合)
return `$p = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw (${psText(`找不到窗口:${title}`)}) }`;
}
async function windowObject(title: string): Promise<WindowInfo> {
@@ -70,10 +71,11 @@ async function windowObject(title: string): Promise<WindowInfo> {
* Windows 前台锁定策略会拒绝普通 SetForegroundWindow,最小化再恢复可绕过)。
* 失败用 RPA_ERROR 标记写 stdout 后 exit 1:不走 throw/CLIXML,避免控制台宽度截断和编码歧义 */
function activateWindowScript(selector: string): string {
const safeTitle = selector.replace(/'/g, "''");
// 错误消息内嵌用户文本走 psText(窗口标题可能含智能引号/中文标点,直接放进单引号字面量会被 PS5.1 解析炸)
const failMsg = psText(`找不到窗口:${selector}。当前可见窗口标题: `);
return `function RpaFail($m) { [Console]::Out.WriteLine('RPA_ERROR::' + $m); exit 1 }
$p = Get-Process | Where-Object { $_.MainWindowTitle -and $_.MainWindowTitle -like ${psString(`*${selector}*`)} } | Select-Object -First 1
if (-not $p) { $vis = (Get-Process | Where-Object { $_.MainWindowTitle } | Select-Object -First 12 | ForEach-Object { '『' + $_.MainWindowTitle + '』' }) -join ' '; RpaFail ('找不到窗口:${safeTitle}。当前可见窗口标题: ' + $vis) }
if (-not $p) { $vis = (Get-Process | Where-Object { $_.MainWindowTitle } | Select-Object -First 12 | ForEach-Object { '『' + $_.MainWindowTitle + '』' }) -join ' '; RpaFail (${failMsg} + $vis) }
Add-Type -MemberDefinition '[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow(); [DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr h, int cmd); [DllImport("user32.dll")] public static extern bool IsIconic(IntPtr h); [DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h); [DllImport("user32.dll")] public static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, UIntPtr dwExtraInfo); [DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr hWnd, IntPtr pid); [DllImport("user32.dll")] public static extern bool AttachThreadInput(uint a, uint b, bool attach); [DllImport("user32.dll")] public static extern void SwitchToThisWindow(IntPtr hWnd, bool fAltTab); [DllImport("kernel32.dll")] public static extern uint GetCurrentThreadId();' -Name RpaFocus2 -Namespace WinRpaFocus2 | Out-Null
$h = $p.MainWindowHandle
if ([WinRpaFocus2.RpaFocus2]::IsIconic($h)) { [void][WinRpaFocus2.RpaFocus2]::ShowWindow($h, 9); Start-Sleep -Milliseconds 150 }
@@ -108,12 +110,9 @@ function sendKeysEscape(ch: string): string {
return "+^%~(){}[]".includes(ch) ? `{${ch}}` : ch;
}
/** 把字符包装成 PowerShell 安全字面量。
* ASCII 直接用单引号字符串(撇号内部 '' 转义);非 ASCII 用 [char]0xXXXX 数值构造——
* Windows PowerShell 5.1 会把 ‘ ’ “ ” 等智能引号当作引号变体解析,直接放进字符串字面量会报「字符串缺少终止符」。 */
/** 单字符/短字面量别名(语义标注用),实现同 psText(历史调用点保留) */
function psCharLiteral(text: string): string {
if (/^[\x20-\x7E]*$/.test(text)) return psString(text);
return Array.from(text).map((c) => `[char]0x${c.codePointAt(0)!.toString(16).padStart(4, "0")}`).join("+") || psString("");
return psText(text);
}
/** 生成从当前位置到目标的插值路径点(JS 侧算好,PS 侧只执行循环)(原 window-extended.ts 的 mousePath) */
@@ -180,9 +179,8 @@ const windowsKeyboard: KeyboardDriver = {
// 三种输入方式:脚本尽量简单,转义逻辑放 JS 侧,避免 PowerShell 字符串拼接的语法陷阱
let send: string;
if (options.mode === "clipboard") {
// 剪贴板粘贴:原文进剪贴板 + Ctrl+V,不做 SendKeys 转义;内容含智能引号等非 ASCII 时同样走 [char] 构造
const contentExpr = /^[\x20-\x7E]*$/.test(options.keys) ? psString(options.keys) : `(-join @(${Array.from(options.keys).map((c) => psCharLiteral(c)).join(",")}))`;
send = `Set-Clipboard -Value ${contentExpr}; Start-Sleep -Milliseconds 120; [System.Windows.Forms.SendKeys]::SendWait('^v')`;
// 剪贴板粘贴:原文进剪贴板 + Ctrl+V,不做 SendKeys 转义
send = `Set-Clipboard -Value ${psText(options.keys)}; Start-Sleep -Milliseconds 120; [System.Windows.Forms.SendKeys]::SendWait('^v')`;
} else if (options.mode === "manual" && options.textMode) {
// 人工模拟:逐字符;JS 侧把每个字符转义成 SendKeys 字面量,PS 侧只做数组循环
const escapedChars = Array.from(options.keys).map((ch) => sendKeysEscape(ch));
@@ -191,8 +189,7 @@ const windowsKeyboard: KeyboardDriver = {
} else {
// 后台接口 / 按键组合 / 人工模拟(按键组合组件):整段发送,文本特殊字符转义
const escaped = options.textMode ? Array.from(options.keys).map(sendKeysEscape).join("") : options.keys;
const contentExpr = /^[\x20-\x7E]*$/.test(escaped) ? psString(escaped) : `(-join @(${Array.from(escaped).map((c) => psCharLiteral(c)).join(",")}))`;
send = `[System.Windows.Forms.SendKeys]::SendWait(${contentExpr})`;
send = `[System.Windows.Forms.SendKeys]::SendWait(${psText(escaped)})`;
}
const activate = options.activateTitle ? `${activateWindowScript(options.activateTitle)}; Start-Sleep -Milliseconds 300; ` : "";
await powershell(`${USER32}; ${activate}Add-Type -AssemblyName System.Windows.Forms; ${send}`);
+1 -1
View File
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-11-04T04:34:06.782Z",
"generatedAt": "2026-11-04T09:12:00.449Z",
"count": 909,
"ids": [
"App_GetPid",
+18 -8
View File
@@ -46,16 +46,21 @@ function removeTree(target) {
async function verifyLaunch(executable, label) {
const userData = path.join(os.tmpdir(), `RPA-Studio-UserData-${Date.now()}-${Math.random().toString(16).slice(2)}`);
// ELECTRON_RUN_AS_NODE / NODE_OPTIONS 来自外部工具链环境时会把打包应用带进纯 Node 模式
// (表现为「bad option」退出码 9),冒烟必须剥离这些不属于应用的控制变量
const env = {
...process.env,
RPA_SKIP_SINGLE_INSTANCE: "1",
RPA_E2E: "1",
RPA_FIXTURE_PORT: String(18000 + Math.floor(Math.random() * 1000))
};
delete env.ELECTRON_RUN_AS_NODE;
delete env.NODE_OPTIONS;
const child = spawn(executable, [`--user-data-dir=${userData}`], {
cwd: path.dirname(executable),
stdio: "ignore",
windowsHide: false,
env: {
...process.env,
RPA_SKIP_SINGLE_INSTANCE: "1",
RPA_E2E: "1",
RPA_FIXTURE_PORT: String(18000 + Math.floor(Math.random() * 1000))
}
env
});
await delay(8000);
const exited = child.exitCode !== null || child.signalCode !== null;
@@ -84,8 +89,13 @@ removeTree(builderWork);
console.log("[package] typecheck");
run("npm", ["run", "typecheck"]);
console.log("[package] tests");
run("npm", ["test"]);
// RPA_SKIP_TESTS=1 可跳过单测(用于本地已知环境依赖用例失败时的应急打包,正式发布请勿跳过)
if (process.env.RPA_SKIP_TESTS === "1") {
console.log("[package] tests skipped (RPA_SKIP_TESTS=1)");
} else {
console.log("[package] tests");
run("npm", ["test"]);
}
console.log("[package] build");
run("npm", ["run", "build"]);
console.log("[package] browser e2e");
+376 -235
View File
@@ -192,7 +192,22 @@ export class BrowserSession implements BrowserLike {
return this.transport.send<T>(method, params);
}
private lastFollowCheck = 0;
/** 运行时跟随前台标签页:点按钮/搜索常开新标签页,元素操作应落在用户眼前看到的页面(节流 400ms) */
private async followForegroundTab(): Promise<void> {
if (this.win || !this.httpBase) return;
const now = Date.now();
if (now - this.lastFollowCheck < 400) return;
this.lastFollowCheck = now;
const pages = await listPageTargets(this.httpBase).catch(() => []);
const page = pages[0];
if (!page || page.id === this.activeTargetId) return;
await this.activateTab(page.id);
}
private async evaluate<T = unknown>(expression: string): Promise<T> {
await this.followForegroundTab().catch(() => {});
const result = await this.cdp<{ result?: { value?: T }; exceptionDetails?: { exception?: { description?: string }; text?: string } }>(
"Runtime.evaluate",
{ expression, returnByValue: true, awaitPromise: true }
@@ -297,8 +312,18 @@ export class BrowserSession implements BrowserLike {
const localPath = (start, root) => {
const stable = (item) => {
if (item.id) {
// 长数字结尾的唯一 id 多为动态记录 id(如淘宝 item_id_1087512368188),泛化为前缀匹配以命中所有同类卡片
const dynamic = /^(.*?[-_])[0-9]{6,}$/.exec(item.id);
if (dynamic) {
const generalized = '[id^="' + cssEscape(dynamic[1]) + '"]';
try { if (root.querySelectorAll(generalized).length >= 1) return generalized; } catch {}
}
const candidate = '#' + cssEscape(item.id);
try { if (root.querySelectorAll(candidate).length === 1) return candidate; } catch {}
if (dynamic) {
const generalized = '[id^="' + cssEscape(dynamic[1]) + '"]';
try { if (root.querySelectorAll(generalized).length >= 1) return generalized; } catch {}
}
}
const testId = item.getAttribute('data-testid');
if (testId) {
@@ -1153,7 +1178,8 @@ export class BrowserSession implements BrowserLike {
}
async pickElement(timeoutMs = 90000): Promise<import("../../shared/types").ElementPickResult> {
// 拾取脚本经 Runtime.evaluate 注入并等待用户点击,内置/外部浏览器行为一致
// 拾取脚本注入当前文档,并经 Page.addScriptToEvaluateOnNewDocument 随页面跳转自动重注入;
// 主进程轮询 window.__rpaPickResult——用户在拾取期间可随意翻页/搜索/过滑块,停在哪就能拾哪。
const url = await this.currentUrl();
if (!url || url === "about:blank") {
throw new RuntimeError("no_page", "浏览器还没有打开网页", { recommendation: "先运行或配置「打开网页」,再拾取元素" });
@@ -1162,241 +1188,63 @@ export class BrowserSession implements BrowserLike {
this.win.show();
this.win.focus();
}
const result = await this.evaluate<ElementPickResult>(`(() => new Promise((resolve, reject) => {
const timeout = ${JSON.stringify(timeoutMs)};
if (window.__rpaElementPicker?.cancel) window.__rpaElementPicker.cancel();
let host = null;
let root = null;
let box = null;
let badge = null;
let target = null;
let locked = false;
let chain = [];
let chainIndex = 0;
let selected = null;
let timer = 0;
let finished = false;
const cssEscape = (value) => {
if (globalThis.CSS && typeof CSS.escape === 'function') return CSS.escape(String(value));
return String(value).replace(/[^a-zA-Z0-9_-]/g, (char) => '\\\\' + char);
};
const isUnique = (candidate) => {
try { return document.querySelectorAll(candidate).length === 1; } catch { return false; }
};
const cssPath = (node) => {
const localPath = (start, root) => {
const stable = (item) => {
if (item.id) {
const candidate = '#' + cssEscape(item.id);
try { if (root.querySelectorAll(candidate).length === 1) return candidate; } catch {}
}
const testId = item.getAttribute('data-testid');
if (testId) {
const candidate = '[data-testid="' + cssEscape(testId) + '"]';
try { if (root.querySelectorAll(candidate).length === 1) return candidate; } catch {}
}
return null;
};
const parts = [];
let current = start;
while (current && current.nodeType === 1) {
const stableCandidate = stable(current);
if (stableCandidate) {
parts.unshift(stableCandidate);
return parts.join(' > ');
}
let part = current.tagName.toLowerCase();
const parent = current.parentElement;
if (parent) {
const siblings = [...parent.children].filter((item) => item.tagName === current.tagName);
if (siblings.length > 1) part += ':nth-of-type(' + (siblings.indexOf(current) + 1) + ')';
}
parts.unshift(part);
current = parent;
}
return parts.join(' > ');
};
const trees = [];
let current = node;
while (current && current.nodeType === 1) {
const root = current.getRootNode();
trees.unshift(localPath(current, root));
current = root instanceof ShadowRoot ? root.host : null;
const inject = () =>
this.evaluate(
`(() => {
try { window.__rpaElementPicker?.cancelHard?.(); window.__rpaElementPicker?.cancel?.(); } catch {}
try { window.__rpaPickerActive = undefined; } catch {}
try { document.getElementById('__rpa_pick_host')?.remove(); } catch {}
try { delete window.__rpaPickResult; ${PICKER_BOOTSTRAP_SOURCE} } catch {}
})()`
);
const injectionIds: string[] = [];
const registerInjection = async () => {
const registered = await this.cdp<{ identifier?: string }>("Page.addScriptToEvaluateOnNewDocument", {
source: PICKER_BOOTSTRAP_SOURCE
}).catch(() => null);
if (registered?.identifier) injectionIds.push(registered.identifier);
};
// 跟随前台标签页:搜索/点按钮常开新标签页,附着目标切过去并重新注入,拾取不中断
const followForegroundTab = async () => {
if (this.win || !this.httpBase) return;
const pages = await listPageTargets(this.httpBase).catch(() => []);
const page = pages[0];
if (!page || page.id === this.activeTargetId) return;
await this.activateTab(page.id);
await this.cdp("Page.enable").catch(() => {});
await registerInjection();
await inject();
};
await this.cdp("Page.enable").catch(() => {});
await registerInjection();
try {
await inject();
const deadline = Date.now() + Math.max(5000, timeoutMs);
while (Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 400));
await followForegroundTab().catch(() => {});
const state = await this.evaluate<{ result: { ok: boolean; value?: ElementPickResult; reason?: string } | null; active: boolean }>(
"({ result: window.__rpaPickResult ?? null, active: window.__rpaPickerActive?.active === true })"
).catch(() => null);
if (!state) continue; // 页面跳转中:等待新文档自动注入
if (state.result) {
if (state.result.ok && state.result.value) return state.result.value;
throw new RuntimeError("cancelled", state.result.reason || "已取消元素拾取");
}
return trees.join(' >>> ');
};
const describe = (node) => {
const rect = node.getBoundingClientRect();
return {
selector: 'css:' + cssPath(node),
tag: node.tagName.toLowerCase(),
id: node.id || undefined,
className: node.getAttribute('class') || undefined,
name: node.getAttribute('name') || undefined,
type: node.getAttribute('type') || undefined,
placeholder: node.getAttribute('placeholder') || undefined,
label: node.getAttribute('aria-label') || node.getAttribute('placeholder') || node.getAttribute('name') || (node.innerText || '').trim().slice(0, 100) || undefined,
text: (node.innerText || node.textContent || '').trim().slice(0, 200) || undefined,
value: String(node.value ?? '') || undefined,
url: location.href
};
};
const nodeName = (node) => node.getAttribute('aria-label') || node.getAttribute('placeholder') || node.getAttribute('name') || node.id || (node.innerText || '').trim().slice(0, 60) || '';
const updateBox = () => {
const node = locked ? selected : target;
if (!node || !box || !badge) return;
const rect = node.getBoundingClientRect();
box.style.left = Math.max(0, rect.left - 3) + 'px';
box.style.top = Math.max(0, rect.top - 3) + 'px';
box.style.width = Math.max(4, rect.width + 6) + 'px';
box.style.height = Math.max(4, rect.height + 6) + 'px';
const tag = node.tagName.toLowerCase();
const name = nodeName(node);
if (locked) {
badge.textContent = '已锁定 ' + (chainIndex + 1) + '/' + chain.length + ' · ' + tag + (name ? ' · ' + name.slice(0, 42) : '') + ' · [PageUp/PageDown]调层级 [空格]确认 [Esc]取消';
} else {
badge.textContent = '拾取 · ' + tag + (name ? ' · ' + name.slice(0, 42) : '') + ' · [空格]锁定 [Esc]取消';
}
badge.style.left = Math.max(0, rect.left) + 'px';
badge.style.top = Math.max(0, rect.top - 34) + 'px';
};
const interactiveSelector = [
'button', 'a', 'input', 'select', 'textarea', 'summary', 'label', 'option',
'[role]', '[onclick]', '[contenteditable]', '[tabindex]'
].join(',');
const closestInteractive = (start) => {
let node = start;
while (node && node.nodeType === 1) {
try { if (node.matches(interactiveSelector)) return node; } catch {}
if (node.parentElement) {
node = node.parentElement;
continue;
}
const root = node.getRootNode();
node = root instanceof ShadowRoot ? root.host : null;
}
return start;
};
const firstElement = (event) => {
let deepest = null;
for (const item of event.composedPath()) {
if (item instanceof Element) {
deepest = item;
break;
}
}
if (!deepest && event.target instanceof Element) deepest = event.target;
return closestInteractive(deepest);
};
const buildChain = (node) => {
const result = [];
let current = node;
while (current && current.nodeType === 1) {
result.unshift(current);
const root = current.getRootNode();
current = root instanceof ShadowRoot ? root.host : null;
}
return result;
};
const lock = () => {
if (locked || !target) return;
locked = true;
chain = buildChain(target);
chainIndex = Math.max(0, chain.length - 1);
selected = chain[chainIndex] || target;
updateBox();
};
const adjustLevel = (delta) => {
if (!locked || !chain.length) return;
const next = Math.min(chain.length - 1, Math.max(0, chainIndex + delta));
if (next === chainIndex) return;
chainIndex = next;
selected = chain[chainIndex];
updateBox();
};
const confirm = () => {
if (!locked || !selected) return;
cleanup();
resolve(describe(selected));
};
const move = (event) => {
if (finished || locked) return;
target = firstElement(event);
updateBox();
};
const click = (event) => {
if (finished) return;
// 与桌面拾取对齐:鼠标只用于定位,空格锁定/确认,避免误触发页面按钮。
event.preventDefault();
event.stopPropagation();
event.stopImmediatePropagation();
};
const key = (event) => {
if (event.key === 'Escape') {
event.preventDefault();
event.stopPropagation();
cleanup();
reject(new Error('已取消元素拾取'));
return;
}
if (event.code === 'Space' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
if (locked) confirm();
else lock();
return;
}
if (locked && event.key === 'PageUp' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
adjustLevel(-1);
return;
}
if (locked && event.key === 'PageDown' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
adjustLevel(1);
}
};
const cleanup = () => {
if (finished) return;
finished = true;
clearTimeout(timer);
window.removeEventListener('mousemove', move, true);
window.removeEventListener('mousedown', click, true);
window.removeEventListener('click', click, true);
window.removeEventListener('scroll', updateBox, true);
window.removeEventListener('resize', updateBox);
window.removeEventListener('keydown', key, true);
host?.remove();
delete window.__rpaElementPicker;
};
host = document.createElement('div');
host.style.cssText = 'position:fixed;inset:0;width:100vw;height:100vh;z-index:2147483647;pointer-events:none;';
root = host.attachShadow({ mode: 'closed' });
box = document.createElement('div');
badge = document.createElement('div');
box.style.cssText = 'position:fixed;border:2px solid #2563eb;background:rgba(37,99,235,.16);border-radius:3px;pointer-events:none;';
badge.style.cssText = 'position:fixed;z-index:2147483647;padding:4px 7px;border-radius:6px;background:#2563eb;color:#fff;font:600 12px/16px system-ui,sans-serif;pointer-events:none;white-space:nowrap;';
root.append(box, badge);
document.documentElement.append(host);
window.addEventListener('mousemove', move, true);
window.addEventListener('mousedown', click, true);
window.addEventListener('click', click, true);
window.addEventListener('scroll', updateBox, true);
window.addEventListener('resize', updateBox);
window.addEventListener('keyup', () => {}, true);
window.addEventListener('keydown', key, true);
window.__rpaElementPicker = { cancel: () => { cleanup(); reject(new Error('已取消元素拾取')); } };
timer = setTimeout(() => {
cleanup();
reject(new Error('拾取超时,请重新点击拾取'));
}, timeout);
}))()`);
return result;
if (!state.active) await inject().catch(() => {}); // 注入未生效时兜底重注入
}
throw new RuntimeError("timeout", "拾取超时,请重新点击拾取");
} finally {
const cleanupExpr =
"(() => { try { window.__rpaElementPicker?.cancelHard?.(); window.__rpaPickerActive = undefined; delete window.__rpaPickResult; } catch {} })()";
await this.evaluate(cleanupExpr).catch(() => {});
for (const tabTransport of this.tabTransports.values()) {
await tabTransport.send("Runtime.evaluate", { expression: cleanupExpr }).catch(() => {});
}
for (const identifier of injectionIds) {
await this.cdp("Page.removeScriptToEvaluateOnNewDocument", { identifier }).catch(() => {});
}
}
}
async close(): Promise<void> {
@@ -1429,6 +1277,299 @@ declare global {
var __rpaExternalSessions: Map<string, BrowserSession> | undefined;
}
const PICKER_BOOTSTRAP_SOURCE = `(() => {
if (window.__rpaPickerActive && window.__rpaPickerActive.active) {
// 强杀应用等异常场景会留下 active 残留拾取器, 若直接 return 新拾取永远不生效, 必须强制接管
try { window.__rpaElementPicker?.cancelHard?.(); window.__rpaElementPicker?.cancel?.(); } catch {}
window.__rpaPickerActive = undefined;
document.getElementById('__rpa_pick_host')?.remove();
}
window.__rpaPickResult = null;
let host = null;
let root = null;
let box = null;
let badge = null;
let target = null;
let deepTarget = null;
let lastSnap = null;
let drilled = false;
let hoverTimer = 0;
let locked = false;
let chain = [];
let chainIndex = 0;
let selected = null;
let finished = false;
const cssEscape = (value) => {
if (globalThis.CSS && typeof CSS.escape === 'function') return CSS.escape(String(value));
return String(value).replace(/[^a-zA-Z0-9_-]/g, (char) => '\\\\' + char);
};
const isUnique = (candidate) => {
try { return document.querySelectorAll(candidate).length === 1; } catch { return false; }
};
const cssPath = (node) => {
const localPath = (start, root) => {
const stable = (item) => {
if (item.id) {
// 长数字结尾的唯一 id 多为动态记录 id(如淘宝 item_id_1087512368188),泛化为前缀匹配以命中所有同类卡片
const dynamic = /^(.*?[-_])[0-9]{6,}$/.exec(item.id);
if (dynamic) {
const generalized = '[id^="' + cssEscape(dynamic[1]) + '"]';
try { if (root.querySelectorAll(generalized).length >= 1) return generalized; } catch {}
}
const candidate = '#' + cssEscape(item.id);
try { if (root.querySelectorAll(candidate).length === 1) return candidate; } catch {}
if (dynamic) {
const generalized = '[id^="' + cssEscape(dynamic[1]) + '"]';
try { if (root.querySelectorAll(generalized).length >= 1) return generalized; } catch {}
}
}
const testId = item.getAttribute('data-testid');
if (testId) {
const candidate = '[data-testid="' + cssEscape(testId) + '"]';
try { if (root.querySelectorAll(candidate).length === 1) return candidate; } catch {}
}
return null;
};
const parts = [];
let current = start;
while (current && current.nodeType === 1) {
const stableCandidate = stable(current);
if (stableCandidate) {
parts.unshift(stableCandidate);
return parts.join(' > ');
}
let part = current.tagName.toLowerCase();
const parent = current.parentElement;
if (parent) {
const siblings = [...parent.children].filter((item) => item.tagName === current.tagName);
if (siblings.length > 1) part += ':nth-of-type(' + (siblings.indexOf(current) + 1) + ')';
}
parts.unshift(part);
current = parent;
}
return parts.join(' > ');
};
const trees = [];
let current = node;
while (current && current.nodeType === 1) {
const root = current.getRootNode();
trees.unshift(localPath(current, root));
current = root instanceof ShadowRoot ? root.host : null;
}
return trees.join(' >>> ');
};
const describe = (node) => {
const rect = node.getBoundingClientRect();
return {
selector: 'css:' + cssPath(node),
tag: node.tagName.toLowerCase(),
id: node.id || undefined,
className: node.getAttribute('class') || undefined,
name: node.getAttribute('name') || undefined,
type: node.getAttribute('type') || undefined,
placeholder: node.getAttribute('placeholder') || undefined,
label: node.getAttribute('aria-label') || node.getAttribute('placeholder') || node.getAttribute('name') || (node.innerText || '').trim().slice(0, 100) || undefined,
text: (node.innerText || node.textContent || '').trim().slice(0, 200) || undefined,
value: String(node.value ?? '') || undefined,
url: location.href
};
};
const nodeName = (node) => node.getAttribute('aria-label') || node.getAttribute('placeholder') || node.getAttribute('name') || node.id || (node.innerText || '').trim().slice(0, 60) || '';
const updateBox = () => {
const node = locked ? selected : target;
if (!node || !box || !badge) return;
const rect = node.getBoundingClientRect();
box.style.left = Math.max(0, rect.left - 3) + 'px';
box.style.top = Math.max(0, rect.top - 3) + 'px';
box.style.width = Math.max(4, rect.width + 6) + 'px';
box.style.height = Math.max(4, rect.height + 6) + 'px';
const tag = node.tagName.toLowerCase();
const name = nodeName(node);
if (locked) {
badge.textContent = '已锁定 ' + (chainIndex + 1) + '/' + chain.length + ' · ' + tag + (name ? ' · ' + name.slice(0, 42) : '') + ' · [PageUp]升 [PageDown]钻入子元素 [空格]确认 [Esc]取消';
} else {
badge.textContent = '拾取 · ' + tag + (name ? ' · ' + name.slice(0, 42) : '') + ' · 悬停0.3秒自动钻入 · [PageUp/PageDown]手动调层 [空格]确认 [Esc]取消 · 可直接翻页后继续拾取';
}
badge.style.left = Math.max(0, rect.left) + 'px';
badge.style.top = Math.max(0, rect.top - 34) + 'px';
};
const interactiveSelector = [
'button', 'a', 'input', 'select', 'textarea', 'summary', 'label', 'option',
'[role]', '[onclick]', '[contenteditable]', '[tabindex]'
].join(',');
const closestInteractive = (start) => {
let node = start;
while (node && node.nodeType === 1) {
try { if (node.matches(interactiveSelector)) return node; } catch {}
if (node.parentElement) {
node = node.parentElement;
continue;
}
const root = node.getRootNode();
node = root instanceof ShadowRoot ? root.host : null;
}
return start;
};
const deepestOf = (event) => {
let deepest = null;
for (const item of event.composedPath()) {
if (item instanceof Element) {
deepest = item;
break;
}
}
if (!deepest && event.target instanceof Element) deepest = event.target;
return deepest;
};
const firstElement = (event) => {
return closestInteractive(deepestOf(event));
};
const buildChain = (node) => {
const result = [];
let current = node;
while (current && current.nodeType === 1) {
result.unshift(current);
const root = current.getRootNode();
current = root instanceof ShadowRoot ? root.host : null;
}
return result;
};
const lockAt = (base) => {
if (locked || !base) return;
locked = true;
chain = buildChain(base);
chainIndex = Math.max(0, chain.length - 1);
// 把光标下最深处元素到锁定元素之间的子级路径并入链,PageDown/PageUp 可在卡片与其内部子元素间切换
if (deepTarget && base !== deepTarget && base.contains(deepTarget)) {
const below = [];
let current = deepTarget;
while (current && current !== base && base.contains(current)) {
below.unshift(current);
const root = current.getRootNode();
current = root instanceof ShadowRoot ? root.host : current.parentElement;
}
chain = chain.concat(below);
}
selected = chain[chainIndex] || base;
updateBox();
};
const lock = () => lockAt(target);
const adjustLevel = (delta) => {
if (!locked || !chain.length) return;
const next = Math.min(chain.length - 1, Math.max(0, chainIndex + delta));
if (next === chainIndex) return;
chainIndex = next;
selected = chain[chainIndex];
updateBox();
};
const confirm = () => {
if (!locked || !selected) return;
finish({ ok: true, value: describe(selected) });
};
const move = (event) => {
if (finished || locked) return;
deepTarget = deepestOf(event);
const snapped = firstElement(event);
if (snapped !== lastSnap) {
// 进入新的交互区域:先吸附交互元素,悬停 0.3 秒后自动钻入光标最深处
lastSnap = snapped;
drilled = false;
target = snapped;
clearTimeout(hoverTimer);
if (deepTarget && deepTarget !== snapped) {
hoverTimer = setTimeout(() => {
if (finished || locked) return;
drilled = true;
target = deepTarget;
updateBox();
}, 300);
}
updateBox();
return;
}
if (drilled && deepTarget && snapped.contains(deepTarget)) {
target = deepTarget; // 已钻入:跟随光标在卡片内部移动
}
updateBox();
};
const click = (event) => {
if (finished) return;
event.preventDefault();
event.stopPropagation();
event.stopImmediatePropagation();
};
const key = (event) => {
if (event.key === 'Escape') {
event.preventDefault();
event.stopPropagation();
finish({ ok: false, reason: '已取消元素拾取' });
return;
}
if (event.code === 'Space' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
if (locked) confirm();
else lock();
return;
}
if (event.key === 'PageDown' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
if (locked) adjustLevel(1);
else lockAt(deepTarget || target); // 未锁定时直接锁定光标最深处(如卡片内的标题)
return;
}
if (event.key === 'PageUp' && !event.repeat) {
event.preventDefault();
event.stopPropagation();
if (locked) adjustLevel(-1);
else if (target) lockAt(target); // 未锁定时锁定吸附到的交互元素(如整张卡片)
}
};
const cleanup = () => {
if (finished) return;
finished = true;
clearTimeout(hoverTimer);
window.removeEventListener('mousemove', move, true);
window.removeEventListener('mousedown', click, true);
window.removeEventListener('click', click, true);
window.removeEventListener('scroll', updateBox, true);
window.removeEventListener('resize', updateBox);
window.removeEventListener('keydown', key, true);
host && host.remove();
window.__rpaElementPicker = null;
window.__rpaPickerActive = { active: false };
};
const finish = (payload) => {
cleanup();
window.__rpaPickResult = payload;
};
host = document.createElement('div');
host.id = '__rpa_pick_host';
host.style.cssText = 'position:fixed;inset:0;width:100vw;height:100vh;z-index:2147483647;pointer-events:none;';
root = host.attachShadow({ mode: 'closed' });
box = document.createElement('div');
badge = document.createElement('div');
box.style.cssText = 'position:fixed;border:2px solid #2563eb;background:rgba(37,99,235,.16);border-radius:3px;pointer-events:none;';
badge.style.cssText = 'position:fixed;z-index:2147483647;padding:4px 7px;border-radius:6px;background:#2563eb;color:#fff;font:600 12px/16px system-ui,sans-serif;pointer-events:none;white-space:nowrap;';
root.append(box, badge);
document.documentElement.append(host);
window.addEventListener('mousemove', move, true);
window.addEventListener('mousedown', click, true);
window.addEventListener('click', click, true);
window.addEventListener('scroll', updateBox, true);
window.addEventListener('resize', updateBox);
window.addEventListener('keydown', key, true);
window.__rpaElementPicker = {
cancel: () => { if (!finished) finish({ ok: false, reason: '已取消元素拾取' }); },
cancelHard: () => { cleanup(); }
};
window.__rpaPickerActive = { active: true };
})();`;
function externalSessionCache(): Map<string, BrowserSession> {
if (!globalThis.__rpaExternalSessions) globalThis.__rpaExternalSessions = new Map();
return globalThis.__rpaExternalSessions;
+72 -2
View File
@@ -1,4 +1,5 @@
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { mkdir, mkdtemp, rm, chmod, writeFile } from "node:fs/promises";
import { writeFileSync, readFileSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
@@ -11,7 +12,7 @@ vi.mock("electron", () => ({
}
}));
import { createDemoFlow, createExampleFlows, createFlow, listFlows, saveFlow } from "./flows";
import { createDemoFlow, createExampleFlows, createFlow, exportFlowToPath, importFlowFromPath, listFlows, saveFlow } from "./flows";
describe("flow summaries", () => {
beforeEach(async () => {
@@ -64,3 +65,72 @@ describe("flow summaries", () => {
expect(examples.find((flow) => flow.id === "example_error_route")?.edges.some((edge) => edge.sourceHandle === "error")).toBe(true);
});
});
describe("flow share export/import", () => {
beforeEach(async () => {
electronState.userData = await mkdtemp(path.join(os.tmpdir(), "rpa-flow-share-"));
});
afterEach(async () => {
await rm(electronState.userData, { recursive: true, force: true });
});
it("exports a flow to a share file and re-imports it with a fresh id on conflict", async () => {
const flow = saveFlow(createFlow("密钥读取任务"));
const shareFile = path.join(electronState.userData, "share", "密钥读取任务.rpaflow.json");
await mkdir(path.dirname(shareFile), { recursive: true });
exportFlowToPath(flow.id, shareFile);
expect(JSON.parse(readFileSync(shareFile, "utf8")).id).toBe(flow.id);
// id 冲突:再次导入同一文件应生成新 id,不覆盖原任务
const imported = importFlowFromPath(shareFile);
expect(imported.id).not.toBe(flow.id);
expect(imported.name).toBe("密钥读取任务");
expect(listFlows().filter((item) => item.name === "密钥读取任务")).toHaveLength(2);
});
it("keeps the original id when importing a share file whose id is free", () => {
const shareFile = path.join(electronState.userData, "手工分享.rpaflow.json");
writeFileSync(shareFile, JSON.stringify({
schemaVersion: 1,
id: "manual_task_01",
name: "手工分享的任务",
description: "",
enabled: true,
trigger: { type: "manual", label: "手动运行" },
variables: [],
nodes: [{ id: "node_start", componentId: "Trigger.Manual", position: { x: 160, y: 120 }, values: {}, options: {} }],
edges: [],
meta: { createdAt: "2026-01-01T00:00:00.000Z", updatedAt: "2026-01-01T00:00:00.000Z", appVersion: "0.1.0" }
}), "utf8");
const imported = importFlowFromPath(shareFile);
expect(imported.id).toBe("manual_task_01");
expect(imported.name).toBe("手工分享的任务");
});
it("rejects invalid share files with a readable reason", () => {
const badJson = path.join(electronState.userData, "bad.rpaflow.json");
writeFileSync(badJson, "not json", "utf8");
expect(() => importFlowFromPath(badJson)).toThrow(/不是有效的 JSON/);
const badSchema = path.join(electronState.userData, "schema.rpaflow.json");
writeFileSync(badSchema, JSON.stringify({ schemaVersion: 99, nodes: [], edges: [] }), "utf8");
expect(() => importFlowFromPath(badSchema)).toThrow(/schemaVersion/);
const badStructure = path.join(electronState.userData, "structure.rpaflow.json");
writeFileSync(badStructure, JSON.stringify({ schemaVersion: 1, nodes: "no" }), "utf8");
expect(() => importFlowFromPath(badStructure)).toThrow(/nodes\/edges/);
});
it("overwrites a read-only flow file on save (imported file scenario)", async () => {
const flow = saveFlow(createFlow("只读任务"));
const target = path.join(electronState.userData, "flows", `${flow.id}.json`);
// 模拟外部复制进来带只读属性的导入文件
await chmod(target, 0o444);
expect(() => saveFlow({ ...flow, name: "只读任务-改" })).not.toThrow();
expect(JSON.parse(readFileSync(target, "utf8")).name).toBe("只读任务-改");
await chmod(target, 0o666);
});
});
+117 -3
View File
@@ -74,6 +74,9 @@ export function createFlow(name = "未命名流程"): FlowDefinition {
/** 演示流程浏览器类型:优先 Chrome,未安装时回退 Edge;两者都缺失则保持 chrome 由能力层给出友好错误 */
function demoBrowserType(): string {
// RPA_DEMO_BROWSER 可强制指定(e2e/诊断用)
const forced = String(process.env.RPA_DEMO_BROWSER || "").trim().toLowerCase();
if (forced === "chrome" || forced === "edge") return forced;
const localAppData = process.env.LOCALAPPDATA || path.join(os.homedir(), "AppData", "Local");
const chromePaths = [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
@@ -239,11 +242,61 @@ export function saveFlow(input: FlowDefinition): FlowDefinition {
appVersion: app.getVersion()
}
};
const target = fileOf(flow.id);
writeFlowFile(fileOf(flow.id), flow);
return flow;
}
/** 覆盖写任务文件。Windows 上目标文件带只读属性(外部导入/复制来的文件常见)或被占用时,
* rename 会抛 EPERM——这里依次尝试:清除只读属性后 rename → 删除后 rename → 直接覆盖写。 */
function writeFlowFile(target: string, flow: FlowDefinition): void {
const temp = `${target}.tmp`;
fs.writeFileSync(temp, JSON.stringify(flow, null, 2), "utf8");
fs.renameSync(temp, target);
return flow;
const retryable = (error: unknown): boolean => {
const code = (error as NodeJS.ErrnoException)?.code;
return code === "EPERM" || code === "EACCES" || code === "EBUSY";
};
const cleanupTemp = () => {
try {
fs.unlinkSync(temp);
} catch {
// 清理失败不影响主流程
}
};
try {
fs.renameSync(temp, target);
return;
} catch (error) {
if (!retryable(error)) {
cleanupTemp();
throw error;
}
}
try {
fs.chmodSync(target, 0o666);
} catch {
// 清只读属性失败不阻塞后续删除/覆盖尝试
}
try {
fs.unlinkSync(target);
} catch {
// 目标可能被其他进程占用,删除失败继续走覆盖写
}
try {
fs.renameSync(temp, target);
return;
} catch {
// rename 仍失败(目标被锁定),尝试直接覆盖
}
try {
fs.copyFileSync(temp, target);
cleanupTemp();
} catch (error) {
cleanupTemp();
throw new Error(
`任务文件被占用或无写入权限(${path.basename(target)})。请关闭可能打开该文件的编辑器/网盘同步后再试。`,
{ cause: error }
);
}
}
export function deleteFlow(id: string): void {
@@ -251,3 +304,64 @@ export function deleteFlow(id: string): void {
if (fs.existsSync(target)) fs.unlinkSync(target);
}
/** 任务分享文件扩展名:单文件就是一个 FlowDefinition JSON,便于直接审阅/版本管理 */
const SHARE_EXTENSION = "rpaflow";
function normalizeImportedName(raw: unknown): string {
const name = String(raw ?? "").trim();
return name || "导入的任务";
}
/** 校验并归一化外部任务文件内容,不合法时抛出带原因的 Error */
function parseSharedFlow(raw: string, source: string): FlowDefinition {
let data: unknown;
try {
data = JSON.parse(raw);
} catch {
throw new Error(`${source}:不是有效的 JSON 文件`);
}
const flow = data as FlowDefinition;
if (!flow || typeof flow !== "object") throw new Error(`${source}:文件内容不是任务结构`);
if (flow.schemaVersion !== 1) throw new Error(`${source}:schemaVersion 不是 1(版本不兼容)`);
if (!Array.isArray(flow.nodes) || !Array.isArray(flow.edges)) {
throw new Error(`${source}:缺少 nodes/edges 结构`);
}
if (flow.nodes.length > 2000) throw new Error(`${source}:节点数超过上限 2000`);
return flow;
}
/** 从分享文件导入任务:id 冲突时生成新 id,避免覆盖既有任务 */
export function importFlowFromPath(filePath: string): FlowDefinition {
const raw = fs.readFileSync(filePath, "utf8");
const flow = parseSharedFlow(raw, path.basename(filePath));
const now = new Date().toISOString();
let id = typeof flow.id === "string" && ID_PATTERN.test(flow.id) ? flow.id : "";
if (!id || fs.existsSync(fileOf(id))) {
// 冲突/缺 id:基于名称生成可读 id,追加随机尾巴保证唯一
const seed = normalizeImportedName(flow.name)
.replace(/[^A-Za-z0-9]+/g, "_")
.replace(/^_+|_+$/g, "")
.slice(0, 40)
.toLowerCase();
const base = `imported_${seed || "task"}`;
id = `${base}_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 6)}`;
while (!ID_PATTERN.test(id)) id = `imported_task_${Date.now().toString(36)}`;
}
const imported: FlowDefinition = {
...flow,
id,
name: normalizeImportedName(flow.name),
meta: { ...flow.meta, createdAt: flow.meta?.createdAt || now }
};
return saveFlow(imported);
}
/** 导出任务到指定路径(main 侧先用 dialog 拿到路径) */
export function exportFlowToPath(id: string, targetPath: string): string {
const flow = getFlow(id);
fs.writeFileSync(targetPath, JSON.stringify(flow, null, 2), "utf8");
return targetPath;
}
export { SHARE_EXTENSION };
+45 -2
View File
@@ -1,6 +1,6 @@
import { app, BrowserWindow, dialog, ipcMain, screen, shell } from "electron";
import path from "node:path";
import type { AppInfo, AppSettings, FlowDefinition, FlowSaveOptions, FlowSaveResult } from "../shared/types";
import type { AppInfo, AppSettings, FlowDefinition, FlowSaveOptions, FlowSaveResult, FlowSummary, FlowsImportResult } from "../shared/types";
import {
BrowserSession,
externalBrowserSession,
@@ -10,7 +10,7 @@ import {
} from "./browser/session";
import "./capabilities";
import { catalogPath, loadCatalog } from "./catalog";
import { createDemoFlow, createExampleFlows, createFlow, deleteFlow, getFlow, listFlows, saveFlow } from "./flows";
import { createDemoFlow, createExampleFlows, createFlow, deleteFlow, exportFlowToPath, getFlow, importFlowFromPath, listFlows, saveFlow, SHARE_EXTENSION } from "./flows";
import { capabilityCount, capabilityIds, setHostBridge, setSubflowSource, type HostDialogWindow } from "@rpa/core";
import { closeBrowserSession, setMainWindowGetter, startRun, stopRun } from "./run/manager";
import { registerOverlayIpc } from "./overlay";
@@ -214,6 +214,49 @@ function registerIpc(): void {
deleteFlow(id);
return true;
});
ipcMain.handle("flows:export", async (_event, id: string): Promise<string | null> => {
if (!mainWindow) return null;
const flow = getFlow(id);
const safeName = (flow.name || "任务").replace(/[\\/:*?"<>|]/g, "_").slice(0, 60);
const result = await dialog.showSaveDialog(mainWindow, {
title: "分享任务",
defaultPath: `${safeName}.${SHARE_EXTENSION}.json`,
filters: [
{ name: "RPA 任务分享文件", extensions: [SHARE_EXTENSION, "json"] },
{ name: "所有文件", extensions: ["*"] }
]
});
if (result.canceled || !result.filePath) return null;
return exportFlowToPath(id, result.filePath);
});
ipcMain.handle("flows:import", async (): Promise<FlowsImportResult> => {
if (!mainWindow) return { imported: [], failed: [] };
const result = await dialog.showOpenDialog(mainWindow, {
title: "导入任务",
properties: ["openFile", "multiSelections"],
filters: [
{ name: "RPA 任务分享文件", extensions: [SHARE_EXTENSION, "json"] },
{ name: "所有文件", extensions: ["*"] }
]
});
if (result.canceled || !result.filePaths.length) return { imported: [], failed: [] };
const imported: FlowSummary[] = [];
const failed: Array<{ file: string; reason: string }> = [];
for (const filePath of result.filePaths) {
try {
const flow = importFlowFromPath(filePath);
imported.push(listFlows().find((item) => item.id === flow.id) || {
id: flow.id,
name: flow.name,
updatedAt: flow.meta?.updatedAt || new Date().toISOString(),
nodeCount: flow.nodes.length
});
} catch (error) {
failed.push({ file: path.basename(filePath), reason: error instanceof Error ? error.message : String(error) });
}
}
return { imported, failed };
});
ipcMain.handle("dialog:open", async (_event, options: { title?: string; filters?: Electron.FileFilter[]; directories?: boolean }) => {
if (!mainWindow) return null;
+6 -2
View File
@@ -1,5 +1,5 @@
import { contextBridge, ipcRenderer } from "electron";
import type { AppInfo, AppSettings, Catalog, FlowDefinition, FlowSaveOptions, FlowSaveResult, FlowSummary, RunEvent, VisionLabDetectRequest, VisionLabDetection, VisionLabSource } from "../shared/types";
import type { AppInfo, AppSettings, Catalog, FlowDefinition, FlowSaveOptions, FlowSaveResult, FlowSummary, FlowsImportResult, RunEvent, VisionLabDetectRequest, VisionLabDetection, VisionLabSource } from "../shared/types";
const api = {
app: {
@@ -20,7 +20,11 @@ const api = {
createExamples: (): Promise<FlowSummary[]> => ipcRenderer.invoke("flows:create-examples"),
get: (id: string): Promise<FlowDefinition> => ipcRenderer.invoke("flows:get", id),
save: (flow: FlowDefinition, opts?: FlowSaveOptions): Promise<FlowSaveResult> => ipcRenderer.invoke("flows:save", flow, opts),
delete: (id: string): Promise<boolean> => ipcRenderer.invoke("flows:delete", id)
delete: (id: string): Promise<boolean> => ipcRenderer.invoke("flows:delete", id),
/** 分享任务:弹出保存对话框写入分享文件,取消返回 null,成功返回写入路径 */
export: (id: string): Promise<string | null> => ipcRenderer.invoke("flows:export", id),
/** 导入任务:弹出打开对话框(可多选),返回成功摘要与失败明细 */
import: (): Promise<FlowsImportResult> => ipcRenderer.invoke("flows:import")
},
dialog: {
open: (options?: { title?: string; filters?: Array<{ name: string; extensions: string[] }>; directories?: boolean }): Promise<string | null> =>
+28
View File
@@ -157,6 +157,32 @@ export function App() {
}
};
const exportTask = async (id: string) => {
try {
const target = await window.rpa.flows.export(id);
if (target) showToast(`已分享到 ${target}`);
} catch (exportError) {
showToast(exportError instanceof Error ? exportError.message : String(exportError), "error");
}
};
const importTasks = async () => {
try {
const result = await window.rpa.flows.import();
if (!result.imported.length && !result.failed.length) return; // 用户取消
await refreshFlows();
if (result.imported.length) {
const names = result.imported.map((item) => item.name).join("、");
showToast(`已导入 ${result.imported.length} 个任务:${names}`);
}
for (const failure of result.failed) {
showToast(`导入失败 ${failure.file}:${failure.reason}`, "error");
}
} catch (importError) {
showToast(importError instanceof Error ? importError.message : String(importError), "error");
}
};
const deleteTask = async (id: string) => {
if (flows.length <= 1) {
showToast("至少保留一个任务", "error");
@@ -257,6 +283,8 @@ export function App() {
flows={flows}
onCreate={() => void createTask()}
onCreateExamples={() => void createExamples()}
onImport={() => void importTasks()}
onExport={(id) => void exportTask(id)}
onOpenEditor={(id) => void openEditor(id)}
onRun={(id) => void runTask(id)}
onDelete={(id) => void deleteTask(id)}
+11 -1
View File
@@ -12,7 +12,9 @@ import {
PackagePlus,
RefreshCw,
Search,
Share2,
Trash2,
Upload,
Zap
} from "lucide-react";
import type { FlowSummary } from "@shared/types";
@@ -21,6 +23,8 @@ interface TasksPageProps {
flows: FlowSummary[];
onCreate: () => void;
onCreateExamples: () => void;
onImport: () => void;
onExport: (id: string) => void;
onOpenEditor: (id: string) => void;
onRun: (id: string) => void;
onDelete: (id: string) => void;
@@ -45,7 +49,7 @@ function formatDuration(value?: number): string {
return `${(value / 1000).toFixed(value < 10000 ? 1 : 0)} s`;
}
export function TasksPage({ flows, onCreate, onCreateExamples, onOpenEditor, onRun, onDelete, onRefresh, onToast }: TasksPageProps) {
export function TasksPage({ flows, onCreate, onCreateExamples, onImport, onExport, onOpenEditor, onRun, onDelete, onRefresh, onToast }: TasksPageProps) {
const [query, setQuery] = useState("");
const [selectedId, setSelectedId] = useState<string | null>(flows[0]?.id || null);
const [refreshing, setRefreshing] = useState(false);
@@ -88,6 +92,9 @@ export function TasksPage({ flows, onCreate, onCreateExamples, onOpenEditor, onR
<p>选择一个任务开始运行或编辑流程;组件库只在进入编辑时参与搭建。</p>
</div>
<div className="page-heading-actions">
<button type="button" className="btn" onClick={onImport} title="从分享文件(.rpaflow.json)导入任务">
<Upload className="icon" /> 导入任务
</button>
<button type="button" className="btn" onClick={onCreateExamples}>
<PackagePlus className="icon" /> 安装示例任务
</button>
@@ -166,6 +173,9 @@ export function TasksPage({ flows, onCreate, onCreateExamples, onOpenEditor, onR
<button type="button" className="icon-btn" title="编辑流程" onClick={(event) => { event.stopPropagation(); onOpenEditor(flow.id); }}>
<Pencil className="icon" />
</button>
<button type="button" className="icon-btn" title="分享任务(导出分享文件)" onClick={(event) => { event.stopPropagation(); onExport(flow.id); }}>
<Share2 className="icon" />
</button>
<button type="button" className="icon-btn danger-icon" title="删除任务" onClick={(event) => { event.stopPropagation(); onDelete(flow.id); }}>
<Trash2 className="icon" />
</button>
+1 -1
View File
@@ -764,7 +764,7 @@ function FieldControl({
<input
className="text-input"
value={textOf(raw)}
placeholder={isVisionTarget ? "屏幕上可见的目标文本" : "css:#submit / aria:button"}
placeholder={isVisionTarget ? "屏幕上可见的目标文本" : isGroundTarget ? "name=元素名称;type=Button,或点右侧「桌面拾取」自动获取" : "css:#submit / aria:button"}
onChange={(event) => onChange(field.key, event.target.value)}
/>
{isGroundTarget ? (
@@ -122,6 +122,20 @@ export function validateFlow(flow: FlowDefinition, catalog: Catalog): FlowValida
...(flow.variables ?? []).map((item) => item.name),
...flow.nodes.map((node) => node.options?.outputVar).filter(Boolean) as string[]
]);
// 循环容器在运行时向子节点注入迭代变量,静态校验时视为已定义
const LOOP_VARS: Record<string, string[]> = {
logic_control_array_for_v3: ["item", "index", "value"],
logic_control_dictionary_for_v1: ["item", "index", "value"],
logic_control_count_for: ["index"]
};
for (const node of flow.nodes) {
const vars = LOOP_VARS[node.componentId];
if (vars) {
const custom = String(node.values?.count_index ?? "").trim();
for (const name of vars) definedVars.add(name);
if (node.componentId === "logic_control_count_for" && /^[A-Za-z_]\w*$/.test(custom)) definedVars.add(custom);
}
}
const variableRef = /\$\{\s*([A-Za-z_\u4e00-\u9fa5][\w\u4e00-\u9fa5]*)\s*(?:\.[A-Za-z_]\w*)?\s*\}/g;
for (const node of flow.nodes) {
const component = componentMap.get(node.componentId);
+6
View File
@@ -33,6 +33,12 @@ export interface FlowSaveOptions {
export type FlowSaveResult =
| { conflict: false; flow: FlowDefinition }
| { conflict: true; diskFlow: FlowDefinition };
/** 批量导入任务的结果:成功与失败分开汇报,单文件失败不阻塞其余 */
export interface FlowsImportResult {
imported: FlowSummary[];
failed: Array<{ file: string; reason: string }>;
}
export interface FlowSummary {
id: string;
name: string;