diff --git a/docs/dev-conventions.md b/docs/dev-conventions.md new file mode 100644 index 0000000..fce9bfd --- /dev/null +++ b/docs/dev-conventions.md @@ -0,0 +1,79 @@ +# RPA Studio 开发规范(v1 · 面向 rpa-core 抽离) + +> 生效日期:2026-11-04。 +> 背景:本平台继续作为日常开发载体;所有新代码必须同时满足"当前可用"和"将来可直接搬入 +> `packages/rpa-core`(最终迁入 Agent-Employee)"。迁移路线与任务清单见 +> `docs/rpa-core-refactor-tasks.md`,本规范是它的日常执行约束。 + +## 核心思想 + +**迁移不是将来的改造,而是现在每次写代码时的选择。** 遵守本规范写的能力,抽包时是搬运; +违反规范写的能力,抽包时是返工。 + +--- + +## 一、分层归属(硬性规则) + +| 层 | 目录 | 判别标准 | 未来去向 | +|---|---|---|---| +| **L1 包内层** | `src/main/engine/`、`src/main/capabilities/` 大部分、`src/main/browser/external.ts`、`transport.ts` | 不依赖 Electron 窗口即可运行 | rpa-core 包 | +| **L2 宿主层** | `src/main/index.ts`、`overlay`、`desktop-pick`、`vision-lab`、`fixture/`、`local-vision/`、`flows`、`run/`、`settings`、`logger`、`browser/session.ts` 内置浏览器部分 | 需要 Electron 窗口/应用生命周期 | 宿主适配层 | + +**判别口诀**:这个能力在没有窗口的 CLI 里能跑吗?能 → L1;不能 → L2。 + +--- + +## 二、L1 层禁令(现有代码不追溯,新代码必须遵守) + +1. **禁止 `import ... from "electron"`** + 需要剪贴板/对话框/截屏/屏幕信息时:先看 Node 等价物(fs/os/child_process), + 确实需要宿主能力时在 RunContext 或参数中注入,并在代码处注释 `// HostBridge 候选`。 +2. **禁止横向依赖宿主模块** + L1 内禁止 `import "../settings"`、`"../logger"`、`"../flows"` 等。配置走参数/ctx,日志走 `ctx.log`。 +3. **禁止私起 PowerShell 进程** + 所有 PowerShell 调用必须使用 `capabilities/desktop.ts` 导出的 `powershell()` / `psString()`。 + 禁止自己 `execFile("powershell.exe", ...)`。原因:将来统一切换为常驻 worker(任务 3.5)。 +4. **禁止新增模块级可变全局状态** + 不写模块顶层 `const cache = new Map()` / `let globalXxx`。需要跨调用状态时挂在显式 store + 或 RunContext 上(目标形态:`createEngine()` 实例持有)。 +5. **禁止把 PowerShell 逻辑写成不可拆的内联串** + 复杂脚本提取为独立函数/常量,便于将来搬进 WindowsDriver。 + +## 三、能力编写规范(tool 友好,为 Agent-Employee 准备) + +1. **结构化进出**:参数从 `args` 对象取具名字段,返回结构化对象或原始值; + 禁止返回"为 UI 拼接的字符串"(渲染层需要展示时自行格式化)。 +2. **必有超时**:涉及外部交互(进程/网络/窗口等待)的能力必须有 timeout 参数和默认值。 +3. **错误契约**:失败抛 `RuntimeError(code, message, { recommendation })`; + code 用稳定英文枚举,message 中文,recommendation 给可执行建议。禁止裸 throw string。 +4. **平台标记**:Windows 专用能力(user32/UIA/COM)在 catalog 注册时带 + `platforms: ["win32"]`;能用纯 Node 实现的能力不调 PowerShell。 +5. **可测试**:每个能力有同名 `*.test.ts`(vitest),与实现同目录。 + +## 四、新能力验收 checklist + +- [ ] 归属判定正确(L1/L2 口诀) +- [ ] 无 electron / 宿主模块 import(L1 时) +- [ ] PowerShell 只经 `powershell()` 助手(如需要) +- [ ] 无新增模块级全局状态 +- [ ] 结构化参数/返回 + 超时 + RuntimeError 错误契约 +- [ ] catalog 注册完整(含 platforms 标记) +- [ ] 有测试,`npm run test` + `npm run typecheck` 全绿 + +## 五、渐进收紧节奏 + +规范随改造阶段自动收紧,不必一步到位: + +| 里程碑(见任务清单) | 规范变化 | +|---|---| +| 现在 | 本文第二节禁令生效(靠 review 执行) | +| 阶段 2 完成 | L1 禁 electron 升级为 ESLint no-restricted-imports 强制 | +| 阶段 3 完成 | `powershell()` 收口进 `driver/windows/`,L1 禁止直接引用 | +| 阶段 4 完成 | 无 schema 的新能力不允许进 catalog | + +## 六、明确不动的部分 + +- 现有 77 处 PowerShell 调用点:**不做预防性重构**,统一等阶段 3 的 driver 抽象一次性处理。 + 之前只修 bug、只加能力,不重写。 +- 渲染层(React)按产品需求正常开发,不为其迁移性投资。 +- `local-vision/`、`overlay/` 等宿主层模块正常开发,不受 L1 禁令约束。 diff --git a/docs/rpa-core-refactor-tasks.md b/docs/rpa-core-refactor-tasks.md new file mode 100644 index 0000000..c7c4802 --- /dev/null +++ b/docs/rpa-core-refactor-tasks.md @@ -0,0 +1,116 @@ +# rpa-core 能力层抽离改造任务清单 + +> 目标:把 `src/main/capabilities/` + `src/main/engine/` 抽成零 Electron 依赖的独立包, +> 先在当前 demo 验证,最终以包为单位迁入 `D:\code\Agent-Employee`。 +> 风险评估见对话记录(2026-11-04),任务编号 R1~R10 对应风险清单。 + +## 总体原则 + +1. **单一源**:包建成后 demo 与未来宿主都通过 workspace 引用,任何时候禁止复制代码(R3)。 +2. **行为不变**:抽离期间能力行为保持不变,靠现有 vitest 套件做安全网。 +3. **先断依赖再抽象**:Electron 依赖清零优先于 driver 接口美化。 + +--- + +## 阶段 0 — 基线与护栏 + +- [ ] **0.1 确认基线全绿** + `npm run test` + `npm run typecheck` 全部通过,记录通过数作为回归基线。 +- [ ] **0.2 确定包位置与工作区结构** + 建 `packages/rpa-core/`,demo 的 `package.json` 加 workspaces 配置,源码从 `src/main` 移入后以 `@rpa/core` 引用回主进程。 +- [ ] **0.3 生成能力目录快照** + 用 `npm run catalog` 的产物做一份快照(能力 id / 参数 / 描述清单),抽离完成后 diff 必须为零,作为行为回归基准。 + +## 阶段 1 — 建包与纯 Node 能力搬运 + +- [ ] **1.1 包骨架** + `packages/rpa-core`:`package.json`(deps: exceljs / pdf-lib / nodemailer / basic-ftp / adm-zip / pngjs)、`tsconfig.json`、独立 vitest 配置(R6:包内保留 vitest,不迁 bun test)。 +- [ ] **1.2 领域类型解耦(R2)** + 搬 `engine/`(registry / types / runtime / expression),将 `shared/types` 依赖改为包内自有领域类型(FlowDefinition / FlowNode / CatalogComponent / RunError)。宿主后续做模型映射,不把画布模型塞进包。 +- [ ] **1.3 全局状态收拢为工厂(R4)** + 新增 `createEngine()` 工厂:excel-store 的模块级 `Map` 缓存、session 的 `globalElementWaitMs` 等进程级状态挂到引擎实例上;registry 注册目标从全局单例改为引擎实例。 +- [ ] **1.4 搬运纯 Node 能力** + files / data / data-extended / http / ftp / mail / mssql / mysql / pg / excel(exceljs)/ word / pdf / office-file / time / crypto / data-table 等不依赖 Electron 的能力文件。 +- [ ] **1.5 搬运对应测试** + 上述能力的 `*.test.ts` 一并移入包内,路径修正后全量跑绿。 +- [ ] **1.6 demo 主进程改为引用包** + `src/main` 里被搬走的部分改 import `@rpa/core`,`npm run test` 全绿、catalog 快照 diff 为零。 + +## 阶段 2 — 断 Electron 依赖 + +- [ ] **2.1 定义 HostBridge 接口** + 包内声明 `HostBridge`:clipboard / dialog / screen / userAgentDataPath 等宿主能力;demo 的 Electron 主进程提供实现并注入。 +- [ ] **2.2 改造直接依赖 Electron 的能力(3 处)** + `capabilities/human-dialog.ts`(BrowserWindow)、`capabilities/window-extended.ts`(screen)、`capabilities/ai-service.ts`(settings)→ 走 HostBridge 注入。 +- [ ] **2.3 浏览器子系统切分(R1)** + `browser/external.ts` + `transport.ts` 的外部浏览器 CDP 路径搬进包;`session.ts` 中依赖内置 `BrowserWindow` 的部分留在宿主适配层,包内通过 `BrowserLike` 接口(types.ts 已有)注入。 +- [ ] **2.4 防回归护栏** + 在包内加依赖检查脚本(或 ESLint no-restricted-imports 规则):禁止 `import "electron"` 及对 `src/main` 的相对引用,CI 里跑。 +- [ ] **2.5 Electron 相关能力留守清单** + overlay / desktop-pick / vision-lab / fixture server / human-dialog 的 UI 部分:明确标记"宿主适配层",从包中排除,记录在包 README。 + +## 阶段 3 — Driver 抽象(Windows 自动化能力) + +- [ ] **3.1 拆出 powershell 助手(R5)** + `powershell()` / `psString()` / `USER32` 从 `capabilities/desktop.ts` 移到 `driver/windows/powershell.ts`;修正 11 个引用文件的 import;测试全绿。 +- [ ] **3.2 定义 AutomationDriver 接口** + 按域拆分:`mouse` / `keyboard` / `window` / `element`(UIA)/ `screen`。接口签名以现有 PowerShell 能力的输入输出为准,不重新发明。 +- [ ] **3.3 现有实现包进 WindowsDriver** + desktop / desktop-uia / window-extended / remaining-core / system-extended / common-final / message / utility-extended 中的 user32/UIA 调用改为面向接口;`WindowsDriver` 内部仍用 PowerShell 实现(行为不变)。 +- [ ] **3.4 driver 注入与平台标记** + 引擎创建时注入 driver;能力目录(catalog)加 `platforms: ["win32"]` 字段,渲染层据此置灰。 +- [ ] **3.5(可选,迁移前不阻塞)PowerShell 常驻 worker** + 单进程 JSON 行协议替代每次 spawn;`powershell()` 接口签名不变,内部切换。性能优化项,放在功能等价验收之后。 + +## 阶段 4 — Tool 契约(为 Agent-Employee 做) + +- [ ] **4.1 能力 schema 化** + 每个能力导出结构化定义:参数类型 / 返回值 / 超时 / 错误码 / 是否需要截图回传。新能力一律按此格式写。 +- [ ] **4.2 注册机制显式化** + 取消 import 副作用注册,改为显式 `registerAll(engine)` 导出,宿主可按子集注册。 +- [ ] **4.3 错误契约规范化(R9)** + RuntimeError 错误码枚举固化,中文 message 保留,增加机器可读字段,供 Agent 侧消费。 + +## 阶段 5 — 独立验证 + +- [ ] **5.1 CLI host** + 写 `scripts/rpa-cli.ts`:Node 直接拉起 `createEngine()` + `WindowsDriver`,跑通 5 个代表能力(1 个桌面、1 个 Excel、1 个 HTTP、1 个文件、1 个窗口)。 +- [ ] **5.2 Bun 冒烟(R6/R7)** + 在 Agent-Employee 环境用 `bun test` 跑包内纯逻辑测试,验证 exceljs / pdf-lib 等依赖在 Bun 下正常;onnxruntime 相关暂不进包,另行验证。 +- [ ] **5.3 demo 全量回归** + demo 切到 workspace 引用后:test + typecheck + catalog 快照 diff 三项全绿。 + +## 阶段 6 — 迁入 Agent-Employee + +- [ ] **6.1 ADR 评审** + 按 Agent-Employee 流程提交架构决策记录:包边界 / HostBridge 契约 / driver 生命周期。 +- [ ] **6.2 包迁入 `packages/`** + 整体搬入,改为 bun workspace 成员;demo 侧保留只读引用或解除引用。 +- [ ] **6.3 宿主适配层** + 按 Agent-Employee 的 product-shell 契约写 host 适配(clipboard / dialog / screen / BrowserLike 实现)。 +- [ ] **6.4 权限与治理对接(R10)** + `System_ExecuteCommand` 等危险能力接入其权限模型;能力目录与插件体系对齐。 + +--- + +## 风险 → 任务映射 + +| 风险 | 缓解任务 | +|---|---| +| R1 浏览器子系统耦合 Electron | 2.3 | +| R2 领域模型冲突 | 1.2、阶段 4 | +| R3 代码漂移 | 0.2、总原则 1 | +| R4 模块级全局状态 | 1.3 | +| R5 powershell 助手位置 | 3.1 | +| R6 测试运行器不兼容 | 1.1、5.2 | +| R7 原生依赖 Bun 兼容 | 5.2(能力层已确认为纯 JS) | +| R8 注册副作用 | 4.2 | +| R9 错误契约 | 4.3 | +| R10 权限治理 | 6.4 | + +## 明确不迁入包的部分 + +- `src/renderer/`(React 渲染层,Agent-Employee 用 SolidJS,不迁移) +- `overlay.ts` / `desktop-pick.ts` / `vision-lab.ts` / `fixture/`(Electron 窗口强耦合,留宿主) +- `local-vision/`(onnxruntime-node 原生依赖,Bun 兼容性单独立项验证) +- `flows.ts` / `run/manager.ts` / `settings.ts`(应用级编排与持久化,属宿主职责) diff --git a/resources/catalog-overrides.json b/resources/catalog-overrides.json index 0ec7bb5..771bc2c 100644 --- a/resources/catalog-overrides.json +++ b/resources/catalog-overrides.json @@ -231,24 +231,67 @@ "Image_Move_V2": { "fields": { "window_obj": { - "description": "目标窗口对象;留空则作用于当前活动窗口" + "label": "窗口标题/对象", + "description": "查找范围为「指定窗口」时必填:可填窗口完整标题、标题关键字或进程名;点「选窗口」从可见窗口列表中选择。", + "placeholder": "选择窗口或填写标题/进程名" }, "action": { - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "label": "查找区域", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}" }, "shot_position": { - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "label": "查找区域", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}" }, "image": { - "description": "图片文件的完整路径,用于匹配或识别;点右侧按钮选择文件" + "description": "要查找的小图模板路径;建议只框选按钮、图标或文字块等目标图案,不要框整屏大图。" }, "position": { - "description": "相对目标的点击位置(中心、左上角等)" + "description": "命中目标图像后,鼠标移动到目标区域的哪个位置。", + "options": [ + { + "label": "左上", + "value": "leftTop" + }, + { + "label": "右上", + "value": "rightTop" + }, + { + "label": "左下", + "value": "leftBottom" + }, + { + "label": "右下", + "value": "rightBottom" + }, + { + "label": "中心", + "value": "center" + } + ] }, "show_track": { - "description": "勾选后运行时显示鼠标移动轨迹动画,便于观察执行过程" + "description": "开启后鼠标按轨迹移动到目标位置;关闭后直接跳到目标位置。" + }, + "find_range": { + "description": "选择模板图片的查找范围:整个屏幕、指定窗口,或先框选一块屏幕区域后只在该区域内查找。", + "options": [ + { + "label": "整个屏幕", + "value": "1" + }, + { + "label": "指定窗口", + "value": "2" + }, + { + "label": "指定屏幕范围", + "value": "3" + } + ] } } }, @@ -294,18 +337,43 @@ "Image_Wait_V1": { "fields": { "window_obj": { - "description": "目标窗口对象;留空则作用于当前活动窗口" + "label": "窗口标题/对象", + "description": "查找范围为「指定窗口」时必填:可填窗口完整标题、标题关键字或进程名;点「选窗口」从可见窗口列表中选择。", + "placeholder": "选择窗口或填写标题/进程名" }, "action": { - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "label": "查找区域", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}" }, "position": { - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "label": "查找区域", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}" }, "image": { - "description": "图片文件的完整路径,用于匹配或识别;点右侧按钮选择文件" + "description": "要查找的小图模板路径;建议只框选按钮、图标或文字块等目标图案,不要框整屏大图。" + }, + "find_range": { + "description": "选择模板图片的查找范围:整个屏幕、指定窗口,或先框选一块屏幕区域后只在该区域内查找。", + "options": [ + { + "label": "整个屏幕", + "value": "1" + }, + { + "label": "指定窗口", + "value": "2" + }, + { + "label": "指定屏幕范围", + "value": "3" + } + ] + }, + "time_out": { + "description": "等待目标图像出现的秒数;默认 20 秒。", + "placeholder": "20" } } }, @@ -1229,5 +1297,54 @@ "description": "使用的识别引擎类型" } } + }, + "File_CompressFile": { + "fields": { + "file_type": { + "label": "待压缩内容", + "description": "选择要压缩的对象类型:单个文件或文件夹。" + }, + "file_path": { + "label": "待压缩文件", + "description": "待压缩的单个文件完整路径;仅在「待压缩内容=文件」时使用。", + "placeholder": "选择或输入待压缩文件,例如 C:\\desktop\\report.docx" + }, + "folder_path": { + "label": "待压缩文件夹", + "description": "待压缩的文件夹完整路径;仅在「待压缩内容=文件夹」时使用。", + "placeholder": "选择或输入待压缩文件夹,例如 C:\\desktop\\reports" + }, + "direct_type": { + "label": "压缩包保存位置", + "description": "「源文件路径」表示自动保存到待压缩对象所在目录;「其他路径」表示使用下方指定的输出位置。同名压缩包不会覆盖,会自动生成副本。" + }, + "out_path": { + "label": "压缩包输出位置", + "description": "选择输出文件夹,或填写以 .zip 结尾的完整压缩包路径;仅在「压缩包保存位置=其他路径」时使用。", + "placeholder": "选择输出目录,或输入 C:\\desktop\\result.zip" + } + } + }, + "File_DecompressFile": { + "fields": { + "folder_path": { + "label": "压缩包路径", + "description": "要解压的 .zip 文件完整路径。", + "placeholder": "选择或输入 C:\\desktop\\archive.zip" + }, + "direct_type": { + "label": "解压目标位置", + "description": "「源文件路径」表示解压到压缩包所在目录;「其他路径」表示解压到下方指定的文件夹。" + }, + "out_path": { + "label": "解压目标目录", + "description": "选择或输入解压结果保存的文件夹;仅在「解压目标位置=其他路径」时使用。", + "placeholder": "选择或输入 C:\\desktop\\archive-output" + }, + "compress_tool": { + "label": "解压方式", + "description": "选择解压使用的执行方式;不确定时保持默认。" + } + } } } diff --git a/resources/catalog.json b/resources/catalog.json index 9cfd730..19469da 100644 --- a/resources/catalog.json +++ b/resources/catalog.json @@ -10,7 +10,7 @@ "vendor": 997, "native": 16, "triggers": 3, - "fields": 4296 + "fields": 4298 }, "groups": [ { @@ -58963,8 +58963,8 @@ "displayType": "selectFile", "required": true, "defaultValue": "", - "placeholder": "请选择或输入压缩包路径", - "description": "选择或输入压缩包路径,例如 C:\\desktop\\实在智能.zip", + "placeholder": "选择或输入 C:\\desktop\\archive.zip", + "description": "要解压的 .zip 文件完整路径。", "advanced": false, "valueTypes": [ "rpa_str_file", @@ -58973,14 +58973,14 @@ }, { "key": "direct_type", - "label": "解压至", + "label": "解压目标位置", "widget": "radio", "type": "string", "displayType": "radioGroup", "required": false, "defaultValue": "1", "placeholder": "", - "description": "", + "description": "「源文件路径」表示解压到压缩包所在目录;「其他路径」表示解压到下方指定的文件夹。", "advanced": false, "valueTypes": [], "options": [ @@ -58996,14 +58996,14 @@ }, { "key": "out_path", - "label": "out_path", + "label": "解压目标目录", "widget": "folder", "type": "object", "displayType": "selectFile", "required": false, "defaultValue": "", - "placeholder": "请选择解压文件存放路径", - "description": "", + "placeholder": "选择或输入 C:\\desktop\\archive-output", + "description": "选择或输入解压结果保存的文件夹;仅在「解压目标位置=其他路径」时使用。", "advanced": false, "valueTypes": [ "rpa_str_folder", @@ -59041,14 +59041,14 @@ }, { "key": "compress_tool", - "label": "解压工具", + "label": "解压方式", "widget": "radio", "type": "string", "displayType": "radioGroup", "required": false, "defaultValue": "default", "placeholder": "", - "description": "", + "description": "选择解压使用的执行方式;不确定时保持默认。", "advanced": true, "valueTypes": [], "options": [ @@ -62112,7 +62112,7 @@ "required": false, "defaultValue": "openFile", "placeholder": "", - "description": "选择或输入待压缩文件/文件夹路径,例如 C:\\desktop\\实在智能", + "description": "选择要压缩的对象类型:单个文件或文件夹。", "advanced": false, "valueTypes": [], "options": [ @@ -62128,14 +62128,14 @@ }, { "key": "file_path", - "label": "file_path", + "label": "待压缩文件", "widget": "file", "type": "object", "displayType": "multiSelectFileOrDir", "required": true, "defaultValue": "", - "placeholder": "请选择或输入压缩包路径", - "description": "请选择或输入压缩包路径。", + "placeholder": "选择或输入待压缩文件,例如 C:\\desktop\\report.docx", + "description": "待压缩的单个文件完整路径;仅在「待压缩内容=文件」时使用。", "advanced": false, "valueTypes": [ "rpa_str" @@ -62143,14 +62143,14 @@ }, { "key": "folder_path", - "label": "folder_path", + "label": "待压缩文件夹", "widget": "folder", "type": "object", "displayType": "multiSelectFileOrDir", "required": true, "defaultValue": "", - "placeholder": "请选择或输入压缩包路径", - "description": "请选择或输入压缩包路径。", + "placeholder": "选择或输入待压缩文件夹,例如 C:\\desktop\\reports", + "description": "待压缩的文件夹完整路径;仅在「待压缩内容=文件夹」时使用。", "advanced": false, "valueTypes": [ "rpa_str" @@ -62158,14 +62158,14 @@ }, { "key": "direct_type", - "label": "压缩至", + "label": "压缩包保存位置", "widget": "radio", "type": "string", "displayType": "radioGroup", "required": false, "defaultValue": "1", "placeholder": "", - "description": "默认压缩至当前待压缩文件所在目录,如果压缩文件已存在将生成副本。", + "description": "「源文件路径」表示自动保存到待压缩对象所在目录;「其他路径」表示使用下方指定的输出位置。同名压缩包不会覆盖,会自动生成副本。", "advanced": false, "valueTypes": [], "options": [ @@ -62181,14 +62181,14 @@ }, { "key": "out_path", - "label": "out_path", + "label": "压缩包输出位置", "widget": "folder", "type": "object", "displayType": "selectFile", "required": false, "defaultValue": "", - "placeholder": "请选择或输入压缩至路径", - "description": "", + "placeholder": "选择输出目录,或输入 C:\\desktop\\result.zip", + "description": "选择输出文件夹,或填写以 .zip 结尾的完整压缩包路径;仅在「压缩包保存位置=其他路径」时使用。", "advanced": false, "valueTypes": [ "rpa_str_folder", @@ -66894,7 +66894,7 @@ "required": false, "defaultValue": "1", "placeholder": "", - "description": "请选择查找目标图像的范围", + "description": "选择模板图片的查找范围:整个屏幕、指定窗口,或先框选一块屏幕区域后只在该区域内查找。", "advanced": false, "valueTypes": [], "options": [ @@ -66903,29 +66903,25 @@ "value": "1" }, { - "label": "当前激活窗口", + "label": "指定窗口", "value": "2" }, - { - "label": "指定窗口对象", - "value": "3" - }, { "label": "指定屏幕范围", - "value": "4" + "value": "3" } ] }, { "key": "window_obj", - "label": "窗口对象", + "label": "窗口标题/对象", "widget": "text", "type": "string", "displayType": "text-expression", "required": true, "defaultValue": "", - "placeholder": "", - "description": "目标窗口对象;留空则作用于当前活动窗口", + "placeholder": "选择窗口或填写标题/进程名", + "description": "查找范围为「指定窗口」时必填:可填窗口完整标题、标题关键字或进程名;点「选窗口」从可见窗口列表中选择。", "advanced": false, "valueTypes": [ "rpa_window_new" @@ -66933,27 +66929,27 @@ }, { "key": "action", - "label": "action", + "label": "查找区域", "widget": "region", "type": "object", "displayType": "getScreenShot", "required": false, "defaultValue": "", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}", - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "advanced": false, "valueTypes": [] }, { "key": "shot_position", - "label": "shot_position", + "label": "查找区域", "widget": "region", "type": "object", "displayType": "screenShotValue", "required": false, "defaultValue": "", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}", - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "advanced": false, "valueTypes": [] }, @@ -66966,7 +66962,7 @@ "required": true, "defaultValue": "", "placeholder": "截取或选择图像", - "description": "图片文件的完整路径,用于匹配或识别;点右侧按钮选择文件", + "description": "要查找的小图模板路径;建议只框选按钮、图标或文字块等目标图案,不要框整屏大图。", "advanced": false, "valueTypes": [ "rpa_str" @@ -66981,28 +66977,28 @@ "required": false, "defaultValue": "center", "placeholder": "", - "description": "相对目标的点击位置(中心、左上角等)", + "description": "命中目标图像后,鼠标移动到目标区域的哪个位置。", "advanced": false, "valueTypes": [], "options": [ { "label": "左上", - "value": "left-top" + "value": "leftTop" }, { "label": "右上", - "value": "right-top" + "value": "rightTop" }, { "label": "左下", - "value": "left-bottom" + "value": "leftBottom" }, { "label": "右下", - "value": "right-bottom" + "value": "rightBottom" }, { - "label": "中间", + "label": "中心", "value": "center" } ] @@ -67031,7 +67027,7 @@ "required": false, "defaultValue": "1", "placeholder": "", - "description": "勾选后运行时显示鼠标移动轨迹动画,便于观察执行过程", + "description": "开启后鼠标按轨迹移动到目标位置;关闭后直接跳到目标位置。", "advanced": true, "valueTypes": [] } @@ -67540,7 +67536,7 @@ "required": false, "defaultValue": "1", "placeholder": "", - "description": "请选择查找目标图像的范围", + "description": "选择模板图片的查找范围:整个屏幕、指定窗口,或先框选一块屏幕区域后只在该区域内查找。", "advanced": false, "valueTypes": [], "options": [ @@ -67549,29 +67545,25 @@ "value": "1" }, { - "label": "当前激活窗口", + "label": "指定窗口", "value": "2" }, - { - "label": "指定窗口对象", - "value": "3" - }, { "label": "指定屏幕范围", - "value": "4" + "value": "3" } ] }, { "key": "window_obj", - "label": "窗口对象", + "label": "窗口标题/对象", "widget": "text", "type": "string", "displayType": "text-expression", "required": true, "defaultValue": "", - "placeholder": "", - "description": "目标窗口对象;留空则作用于当前活动窗口", + "placeholder": "选择窗口或填写标题/进程名", + "description": "查找范围为「指定窗口」时必填:可填窗口完整标题、标题关键字或进程名;点「选窗口」从可见窗口列表中选择。", "advanced": false, "valueTypes": [ "rpa_window_new" @@ -67579,27 +67571,27 @@ }, { "key": "action", - "label": "action", + "label": "查找区域", "widget": "region", "type": "object", "displayType": "getScreenShot", "required": false, "defaultValue": "", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}", - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "advanced": false, "valueTypes": [] }, { "key": "position", - "label": "position", + "label": "查找区域", "widget": "region", "type": "object", "displayType": "screenShotValue", "required": false, "defaultValue": "", "placeholder": "{\"x\":0,\"y\":0,\"width\":100,\"height\":50}", - "description": "区域坐标 JSON;点右侧「屏幕框选」在屏幕上框选生成", + "description": "查找范围为「指定屏幕范围」时使用:点右侧「屏幕框选」生成区域坐标 JSON。", "advanced": false, "valueTypes": [] }, @@ -67612,7 +67604,7 @@ "required": true, "defaultValue": "", "placeholder": "截取或选择图像", - "description": "图片文件的完整路径,用于匹配或识别;点右侧按钮选择文件", + "description": "要查找的小图模板路径;建议只框选按钮、图标或文字块等目标图案,不要框整屏大图。", "advanced": false, "valueTypes": [ "rpa_str" @@ -67641,8 +67633,8 @@ "displayType": "text-expression", "required": true, "defaultValue": "20", - "placeholder": "请输入超时等待的时间(秒)", - "description": "输入超时等待的时间,单位:秒,默认20秒,非必填", + "placeholder": "20", + "description": "等待目标图像出现的秒数;默认 20 秒。", "advanced": false, "valueTypes": [ "rpa_num" @@ -125582,4 +125574,4 @@ "slots": [] } ] -} \ No newline at end of file +} diff --git a/src/main/capabilities/files.test.ts b/src/main/capabilities/files.test.ts new file mode 100644 index 0000000..a464102 --- /dev/null +++ b/src/main/capabilities/files.test.ts @@ -0,0 +1,67 @@ +import { existsSync, readdirSync, readFileSync } from "node:fs"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import type { CatalogComponent, FlowNode } from "../../shared/types"; +import type { RunContext } from "../engine/types"; +import { getHandler } from "../engine/registry"; +import "./files"; + +function run(id: string, args: Record): Promise { + const node: FlowNode = { id: "file-test", componentId: id, values: args, options: {}, position: { x: 0, y: 0 } }; + const handler = getHandler(id)!; + return Promise.resolve(handler({ signal: new AbortController().signal, log: vi.fn(), vars: {} } as unknown as RunContext, node, args, {} as CatalogComponent)) as Promise; +} + +const temporary: string[] = []; +afterEach(() => { + for (const dir of temporary.splice(0)) rm(dir, { recursive: true, force: true }); +}); + +describe("file compression path semantics", () => { + it("defaults to a sibling archive and creates a copy instead of overwriting", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-compress-")); + temporary.push(dir); + const source = path.join(dir, "report.txt"); + await writeFile(source, "hello", "utf8"); + const first = await run("File_CompressFile", { file_type: "openFile", file_path: source, direct_type: "1" }); + expect(first).toBe(path.join(dir, "report.txt.zip")); + expect(existsSync(first)).toBe(true); + const second = await run("File_CompressFile", { file_type: "openFile", file_path: source, direct_type: "1" }); + expect(second).toBe(path.join(dir, "report.txt (1).zip")); + expect(existsSync(second)).toBe(true); + }); + + it("accepts an output directory when custom destination is selected", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-compress-")); + temporary.push(dir); + const source = path.join(dir, "logs"); + const output = path.join(dir, "output"); + await Promise.all([mkdir(source), mkdir(output)]); + await writeFile(path.join(source, "app.log"), "log", "utf8"); + const target = await run("File_CompressFile", { + file_type: "openDirectory", folder_path: source, direct_type: "0", out_path: output + }); + expect(target).toBe(path.join(output, "logs.zip")); + expect(existsSync(target)).toBe(true); + }); + + it("extracts beside the archive by default and to the selected folder for custom mode", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-extract-")); + temporary.push(dir); + const source = path.join(dir, "logs.zip"); + const AdmZip = (await import("adm-zip")).default; + const zip = new AdmZip(); + zip.addFile("app.log", Buffer.from("log", "utf8")); + zip.writeZip(source); + const defaultTarget = await run("File_DecompressFile", { folder_path: source, direct_type: "1" }); + expect(defaultTarget).toBe(dir); + expect(readFileSync(path.join(dir, "app.log"), "utf8")).toBe("log"); + + const custom = path.join(dir, "custom-output"); + const customTarget = await run("File_DecompressFile", { folder_path: source, direct_type: "0", out_path: custom }); + expect(customTarget).toBe(custom); + expect(readdirSync(custom)).toContain("app.log"); + }); +}); diff --git a/src/main/capabilities/files.ts b/src/main/capabilities/files.ts index 31387da..aeb7f94 100644 --- a/src/main/capabilities/files.ts +++ b/src/main/capabilities/files.ts @@ -196,10 +196,45 @@ register("File_WaitFile", async (ctx, node, args) => { }); }); +function directoryOutputPath(outputPath: string, sourcePath: string): string { + const output = path.resolve(outputPath); + if (/[\\/]$/.test(outputPath) || path.extname(output).toLowerCase() !== ".zip") { + return path.join(output, path.basename(sourcePath) + ".zip"); + } + return output; +} + +async function uniqueZipPath(targetPath: string): Promise { + try { + await fs.access(targetPath); + } catch { + return targetPath; + } + const dir = path.dirname(targetPath); + const ext = path.extname(targetPath); + const base = path.basename(targetPath, ext); + for (let index = 1; ; index++) { + const candidate = path.join(dir, `${base} (${index})${ext}`); + try { + await fs.access(candidate); + } catch { + return candidate; + } + } +} + register("File_CompressFile", async (_ctx, node, args) => { // catalog 字段为 file_path(文件模式)/folder_path(文件夹模式)+ out_path,兼容旧键 - const source = requirePath(pick(args, ["file_path", "pathSrc", "path", "folder_path"]), "源路径", node.id, node.componentId); - const target = requirePath(pick(args, ["pathDest", "zip_path", "out_path"]), "压缩包路径", node.id, node.componentId); + const contentType = String(args.file_type ?? "openFile"); + const sourceKeys = contentType === "openDirectory" + ? ["folder_path", "pathSrc", "path"] + : ["file_path", "pathSrc", "path"]; + const source = requirePath(pick(args, sourceKeys), contentType === "openDirectory" ? "待压缩文件夹路径" : "待压缩文件路径", node.id, node.componentId); + const saveToSourceDir = String(args.direct_type ?? "1") !== "0"; + const requestedTarget = saveToSourceDir + ? path.join(path.dirname(source), path.basename(source) + ".zip") + : directoryOutputPath(requirePath(pick(args, ["pathDest", "zip_path", "out_path"]), "压缩包输出位置", node.id, node.componentId), source); + const target = await uniqueZipPath(requestedTarget); const zip = new AdmZip(); const stat = await fs.stat(source); if (stat.isDirectory()) zip.addLocalFolder(source); @@ -211,7 +246,10 @@ register("File_CompressFile", async (_ctx, node, args) => { register("File_DecompressFile", async (_ctx, node, args) => { const source = requirePath(pick(args, ["folder_path", "path", "zip_path"]), "压缩包路径", node.id, node.componentId); - const target = requirePath(pick(args, ["out_path", "pathDest", "target"]), "解压目录", node.id, node.componentId); + const saveToSourceDir = String(args.direct_type ?? "1") !== "0"; + const target = saveToSourceDir + ? path.dirname(source) + : requirePath(pick(args, ["out_path", "pathDest", "target"]), "解压目标目录", node.id, node.componentId); await fs.mkdir(target, { recursive: true }); new AdmZip(source).extractAllTo(target, true); return target; diff --git a/src/main/capabilities/vision-helpers.test.ts b/src/main/capabilities/vision-helpers.test.ts index a31c149..a5a6f5b 100644 --- a/src/main/capabilities/vision-helpers.test.ts +++ b/src/main/capabilities/vision-helpers.test.ts @@ -79,3 +79,18 @@ describe("clickPoint", () => { expect(clickPoint(region, "bogus")).toEqual({ x: 130, y: 220 }); }); }); + +describe("legacy position aliases", () => { + it("accepts kebab-case anchors saved by older catalogs", () => { + const region = { x: 100, y: 200, width: 60, height: 40, score: 1 }; + expect(clickPoint(region, "left-top")).toEqual({ x: 100, y: 200 }); + expect(clickPoint(region, "right-top")).toEqual({ x: 160, y: 200 }); + expect(clickPoint(region, "left-bottom")).toEqual({ x: 100, y: 240 }); + expect(clickPoint(region, "right-bottom")).toEqual({ x: 160, y: 240 }); + }); + + it("still falls back to center for unknown anchors", () => { + const region = { x: 100, y: 200, width: 60, height: 40, score: 1 }; + expect(clickPoint(region, "middle")).toEqual({ x: 130, y: 220 }); + }); +}); diff --git a/src/main/capabilities/vision.ts b/src/main/capabilities/vision.ts index ad32bc7..4199afd 100644 --- a/src/main/capabilities/vision.ts +++ b/src/main/capabilities/vision.ts @@ -406,7 +406,7 @@ async function locate(args: Record, evidenceDir: string, nodeId await fs.writeFile(screenshot, PNG.sync.write(cropPng(shot.png, cropRect))); offsetX = rect.x; offsetY = rect.y; - } else if (rangeMode === "3") { + } else if (rangeMode === "3" || rangeMode === "4") { const regionRaw = text(args, ["shot_position", "action", "position"]); if (!regionRaw.trim()) throw new RuntimeError("validation_error", "查找范围为「指定区域」时,需要填写区域 JSON", { nodeId, recommendation: '在 shot_position 填写 {"x":0,"y":0,"width":100,"height":80} 或 [x, y, width, height]' }); // 区域 JSON 是 DIP 坐标(框选工具按 DIP 返回, 支持副屏负坐标),换算物理坐标后减画布原点裁剪 @@ -460,8 +460,8 @@ for (const id of ["Image_Exists", "Image_Exists_V1"]) register(id, async (ctx, n }); // 等待图像: 在超时窗口内每 300ms 重试一次查找, 命中即返回坐标, 超时仍无则抛 not_found for (const id of ["Image_Wait", "Image_Wait_V1"]) register(id, async (ctx, node, args) => { - const rawTimeout = Number(text(args, ["timeout", "time_out"], "10")); - const timeout = Number.isFinite(rawTimeout) && rawTimeout >= 0 ? rawTimeout : 10; + const rawTimeout = Number(text(args, ["timeout", "time_out"], "20")); + const timeout = Number.isFinite(rawTimeout) && rawTimeout >= 0 ? rawTimeout : 20; const deadline = Date.now() + timeout * 1000; for (;;) { try { @@ -476,8 +476,9 @@ for (const id of ["Image_Wait", "Image_Wait_V1"]) register(id, async (ctx, node, // SetCursorPos 传物理坐标, 脚本必须声明 per-monitor DPI 感知(与 injectClick 同理), 否则被系统虚拟化导致偏移 for (const id of ["Image_Move", "Image_Move_V1", "Image_Move_V2"]) register(id, async (ctx, node, args) => { const hit = await locate(args, ctx.evidenceDir, node.id); - const x = Math.round(hit.x + hit.width / 2); - const y = Math.round(hit.y + hit.height / 2); + const point = clickPoint(hit, text(args, ["position"], "center")); + const x = point.x; + const y = point.y; await powershell("Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value); [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y);' -Name RpaImageMoveInput -Namespace RpaImageMove | Out-Null; try { [void][RpaImageMove.RpaImageMoveInput]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}; [RpaImageMove.RpaImageMoveInput]::SetCursorPos(" + x + "," + y + ")"); return { ...hit, x, y }; }); @@ -504,7 +505,7 @@ export function clickPoint(result: MatchResult, position: string): { x: number; // 非法 JSON 落回中心 } } - const anchor = POSITION_ANCHORS[raw] ?? POSITION_ANCHORS[raw.toLowerCase()]; + const anchor = POSITION_ANCHORS[raw] ?? POSITION_ANCHORS[raw.toLowerCase()] ?? POSITION_ANCHORS[raw.replace(/-([a-z])/g, (_: string, char: string) => char.toUpperCase())]; if (anchor) { return { x: Math.round(result.x + result.width * anchor[0]), y: Math.round(result.y + result.height * anchor[1]) }; } diff --git a/src/renderer/src/validation/flowValidation.ts b/src/renderer/src/validation/flowValidation.ts index 0e2f6e5..d2b89af 100644 --- a/src/renderer/src/validation/flowValidation.ts +++ b/src/renderer/src/validation/flowValidation.ts @@ -35,6 +35,30 @@ export function fieldIsActive(componentId: string, key: string, values: Record { expect(fieldIsActive("App_KillV1", "pid", { close_type: "id" })).toBe(true); }); }); + +describe("Image find range conditional fields", () => { + it("Move V2 shows window field only for specified window", () => { + expect(fieldIsActive("Image_Move_V2", "window_obj", { find_range: "1" })).toBe(false); + expect(fieldIsActive("Image_Move_V2", "window_obj", { find_range: "2" })).toBe(true); + expect(fieldIsActive("Image_Move_V2", "window_obj", { find_range: "3" })).toBe(false); + }); + + it("Move V2 shows region fields only for specified screen range", () => { + const values = { find_range: "3" }; + expect(fieldIsActive("Image_Move_V2", "action", values)).toBe(true); + expect(fieldIsActive("Image_Move_V2", "shot_position", values)).toBe(true); + expect(fieldIsActive("Image_Move_V2", "position", values)).toBe(true); + expect(fieldIsActive("Image_Move_V2", "action", { find_range: "1" })).toBe(false); + expect(fieldIsActive("Image_Move_V2", "shot_position", { find_range: "2" })).toBe(false); + }); + + it("Move V2 click anchor remains visible in all modes", () => { + for (const range of ["1", "2", "3", "4"]) { + expect(fieldIsActive("Image_Move_V2", "position", { find_range: range })).toBe(true); + } + }); + + it("Wait V1 treats position as region and legacy range 4 remains compatible", () => { + const values = { find_range: "4" }; + expect(fieldIsActive("Image_Wait_V1", "position", values)).toBe(true); + expect(fieldIsActive("Image_Wait_V1", "window_obj", values)).toBe(false); + expect(fieldIsActive("Image_Wait_V1", "position", { find_range: "1" })).toBe(false); + expect(fieldIsActive("Image_Wait_V1", "window_obj", { find_range: "2" })).toBe(true); + }); +}); + +describe("File compression conditional fields", () => { + it("compress source follows selected content type", () => { + expect(fieldIsActive("File_CompressFile", "file_path", { file_type: "openFile" })).toBe(true); + expect(fieldIsActive("File_CompressFile", "folder_path", { file_type: "openFile" })).toBe(false); + expect(fieldIsActive("File_CompressFile", "file_path", { file_type: "openDirectory" })).toBe(false); + expect(fieldIsActive("File_CompressFile", "folder_path", { file_type: "openDirectory" })).toBe(true); + }); + + it("custom output paths follow destination mode", () => { + expect(fieldIsActive("File_CompressFile", "out_path", { direct_type: "1" })).toBe(false); + expect(fieldIsActive("File_CompressFile", "out_path", { direct_type: "0" })).toBe(true); + expect(fieldIsActive("File_DecompressFile", "out_path", { direct_type: "1" })).toBe(false); + expect(fieldIsActive("File_DecompressFile", "out_path", { direct_type: "0" })).toBe(true); + }); +});