Compare commits

Author SHA1 Message Date
yu a8084dca26 阶段5-独立验证: 新增rpa-cli.ts CLI host(createEngine+registerAll零Electron跑通5个代表能力: 桌面/Excel/HTTP/文件/屏幕), 新增rpa-cli脚本与tsx依赖; 5.2 Bun冒烟按决策10延期到迁入Agent-Employee时执行; 5.3回归三项全绿 2026-11-04 13:49:27 +08:00
yu b54311d467 阶段4-tool契约: register支持CapabilityMeta(决策8 meta可选+新能力强制), 新增registerAll显式注册入口双入口过渡(决策9), RUNTIME_ERROR_CODES错误码固化+audit校验, 规范v1同步收紧 2026-11-04 13:41:24 +08:00
yu e691b9fa78 阶段3-driver抽象: powershell助手拆至driver/windows, 五域AutomationDriver接口+WindowsDriver(PowerShell实现行为不变), desktop/window-extended/system-extended/common-final改走driver, catalog构建填充platformTag=win32 2026-11-04 13:28:32 +08:00
yu 3decced268 阶段2-断Electron依赖: human-dialog/window-extended/ai-service经HostBridge迁入包, external+transport迁包(session留宿主), 新增check:deps依赖护栏与包README留守清单 2026-11-04 12:57:41 +08:00
yu caa8e19332 阶段1-rpa-core建包: engine+31个纯Node能力迁入packages/rpa-core, 领域类型单一源化, createEngine工厂收拢模块级状态
- engine/(registry/types/runtime/expression)物理迁入包, 解耦shared/types(域类型归包, 宿主type-only re-export)
- 31个纯Node能力+27个测试迁入包; 宿主留守Electron/浏览器/AI依赖能力, index.ts经@rpa/core/capabilities拉起
- 新增createEngine()/defaultEngine: excel缓存/Excel句柄/db连接/FTP客户端/OCR实例/拟真设置挂引擎实例
- remaining-core子流程依赖改为host-bridge注入, 包内不再import宿主flows
- 宿主main/index.ts启动时注入子流程源; electron-vite/vitest/tsconfig配@rpa/core源码别名
- 验证: typecheck绿, 快照909个id diff为零, 332测试过(2个桌面冒烟因会话无交互SetForegroundWindow被拒, 环境阻塞)
2026-11-04 12:36:03 +08:00
yu 4139400dbd 阶段0-基线与护栏: 修复不可移植/抖动测试全绿(341过), vitest串行执行防桌面竞争, 新增运行时能力注册快照(909个id) 2026-11-04 11:53:41 +08:00
yu f35f0ac979 files 能力增强与 vision 校验修复;落盘 rpa-core 抽离改造任务清单与开发规范 2026-11-04 10:14:55 +08:00
yu 1ff6f59990 非 ASCII 字符走 char code 构造绕开智能引号解析陷阱 2026-11-04 09:36:45 +08:00
yu 143fcae34a 修复撇号字符数组转义顺序导致字符串未闭合 2026-11-04 09:28:46 +08:00
yu bb03699cce 键盘输入转义逻辑移到 JS 侧根治脚本语法错误 2026-11-04 09:14:09 +08:00
yu f390db9c94 错误输出通道改造: GBK 回退解码与 RPA_ERROR 标记防截断 2026-11-03 17:43:46 +08:00
yu e1f162aa82 错误信息解码与窗口列表提示 2026-11-03 17:19:22 +08:00
yu 3a9b4fa3c7 窗口激活多招式兜底修复 Studio 前台下切窗失败 2026-11-03 17:09:45 +08:00
yu 8d6f0ff755 键盘输入支持选择输入目标窗口 2026-11-03 16:56:10 +08:00
yu 46348413c1 键盘输入组件修复: 接入 words 字段与三种输入方式 2026-11-03 16:38:03 +08:00
yu 6b05a65664 鼠标组件坐标修正: DIP 转物理像素修复缩放屏偏移 2026-11-03 16:28:52 +08:00
yu 158104661c fix: 移动鼠标支持对象坐标x/y(屏幕框选生成格式,此前报坐标无效), 实现三种移动方式(模拟人工缓动抖动/匀速直线/瞬间)与移动速度, 相对激活窗口模式叠加前台窗口偏移, 补DPI感知, 冒烟4项真实落位通过 2026-11-03 16:15:03 +08:00
yu fb71c0551b fix: 网页监听通配过滤去锚定(关键字任意位置命中), 网址下载临时锚点改可见1px触发(隐藏元素被点击解析过滤), 浏览器截图path传目录自动补文件名, 补网页自动化真实Chrome冒烟11项 2026-11-03 16:04:33 +08:00
yu 58aae7c92f fix: 大小写转换组件选项值1/2不生效(选转换方式返回原文), CSV读写空路径给结构化报错, 补数据处理与文件族冒烟回归11项 2026-11-03 15:51:45 +08:00
yu cb349ec979 test: 网络与消息冒烟回归: HTTP三件套(参数/JSON表单/PUT Cookie落盘/5xx超时报错)+钉钉飞书企微webhook消息体+真实SMTP会话+AI未配置引导报错 10项通过 2026-11-03 15:35:32 +08:00
yu d4de0ec245 fix: ConvertBool数字语义对齐(数字1为true), 凭据读写改经ps强制UTF8解码中文并返回整条凭据块, 补应用与系统冒烟回归10项 2026-11-03 15:22:59 +08:00
yu d5d7df8880 fix: 同步相对位置组件描述覆盖 2026-11-03 15:08:17 +08:00
yu 58ea001e7d fix: 相对位置互换未命中时给互不包含综合诊断, 不再误导为尺寸错误 2026-11-03 15:07:43 +08:00
yu f0fca2ebfe test: 相对位置冒烟缺少本机夹具时自动跳过 2026-11-03 15:03:06 +08:00
yu c90584670d test: OCR冒烟缺少本机夹具时自动跳过 2026-11-03 15:02:32 +08:00
yu c973209418 fix: 对比图像相对位置字段正名(被搜索图像/查找目标小图)+填反自动互换兜底, 补OCR与相对位置冒烟回归 2026-11-03 14:59:33 +08:00
yu 9ec301b081 fix: 修复重复能力审计中Base64解码同时保留与隐藏的冲突 2026-11-03 14:54:46 +08:00
yu 7a067173a5 test: OCR组件冒烟回归(本地离线推理): 通用/键值/金额/验证码/卡证/票据/离线文本定位表格 10项真实识别通过 2026-11-03 14:50:25 +08:00
yu 4fc1209c93 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 14:41:41 +08:00
yu 5b1b41663f fix: 修复远程catalog变更引入的3个测试失败(数量658→654, canonical链断→V3, 8字段缺type, 3字段widget非法→text, counts.fields 4297→4294, overrides描述与catalog同步) 2026-11-03 14:41:08 +08:00
yu 8fdc62d98d feat: 图片Base64编解码组件统一(Encode并入方向切换,Decode隐藏), 验证任务归并: 图像OCR未测任务 41->7 (Base64/CV/OCR票据卡证证照通用离线) 2026-11-03 14:28:23 +08:00
yu 4486c57536 refactor: 屏幕截图删除输出目录字段, 仅留保存路径与文件名称(旧流程输出目录值运行时仍兼容) 2026-11-03 14:17:19 +08:00
yu 04ac6130a0 feat: 屏幕截图面板精简为 范围/保存路径/输出目录/文件名称, 截图区域仅框选模式显示, 删除冗余action与position_img 2026-11-03 14:14:42 +08:00
yu 8176bf7c75 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 14:14:17 +08:00
yu 8dc2b60c6d feat: datePicker加原生日期选择器; openWindowMultipleSelect/openWindowEditor加选窗口按钮; dateRangeFilter加日期范围输入 2026-11-03 14:14:01 +08:00
yu b4bcee8c15 refactor: 屏幕截图冗余区域字段收敛(action/position_img 并入截图区域), 仅框选区域模式显示 2026-11-03 14:13:07 +08:00
yu 9e9617256d Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 14:11:52 +08:00
yu 6bf8e90c63 fix: 窗口选择器从循环confirm改为弹出列表modal, 用户可一次看到所有窗口并点击选择 2026-11-03 14:11:38 +08:00
yu 41b999115e fix: 屏幕截图保存路径字段正名(目标文件路径->截图保存路径), 备用输出目录/文件名折叠进高级参数, 描述写清保存回退链 2026-11-03 14:07:06 +08:00
yu 4a3e6e38c7 fix: overlay窗口closable:false导致close()被忽略, 改用destroy()强制销毁 2026-11-03 14:05:29 +08:00
yu b6829cbaab Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 14:01:58 +08:00
yu 0d429e5175 fix: overlay真正集成(CRLF换行导致正则替换全部失效) showRunOverlay/hideRunOverlay/overlayBroadcast实际调用, nodeLabel/total字段补上 2026-11-03 14:01:45 +08:00
yu 38b2199d4b fix: 裁剪图像脚本New-Object不能置于.NET方法参数表(组件从未跑通的根因), 改用[Type]::new() 2026-11-03 13:59:02 +08:00
yu 9bfa4ac107 fix: overlay改loadFile(不用data URL) + nodeIntegration开启 + taskbar进度条兜底 2026-11-03 13:57:49 +08:00
yu bc4b349771 fix: overlay窗口 require改ESM import, nodeIntegration开启使ipcRenderer可用, script标签模板转义修复 2026-11-03 13:55:28 +08:00
yu 0651bc995c Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 13:52:45 +08:00
yu f163c8d0b6 feat: 运行时主窗口最小化+悬浮进度窗(alwaysOnTop不可拖动/resize,显示步骤名/进度/耗时/停止按钮,结束后自动恢复主窗口) 2026-11-03 13:52:25 +08:00
yu fe0c6c2790 fix: 裁剪图像参数校验与字段说明(源图片路径/坐标语义), 空路径与无效区域给明确报错 2026-11-03 13:50:57 +08:00
yu 6b54decfb0 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 13:46:49 +08:00
yu 346e2ad342 fix: 补充用户不便输入的displayType控件(mouseRecord/clickPosition坐标拾取、excelRange文本、USB串口下拉、键鼠驱动下拉、邮箱email输入、previewDataTable只读、customModal/itemGroup JSON textarea); pickPointToField函数; flowValidation match_type优先级修复 2026-11-03 13:46:27 +08:00
yu d4f27cdaab refactor: 找图族组件统一新管线(Exists/Location/Wait/Move迁至虚拟桌面+相似度语义), 面板只留V1/V2最新版 2026-11-03 13:44:59 +08:00
yu 6027895d4d refactor: 点击图像组件合(V2并入V3): 精简为查找范围/窗口对象/目标图像/按键/位置/相似度七字段 2026-11-03 13:34:42 +08:00
yu 12b97315f2 fix: 窗口对象语义字段(window_obj/窗口对象)统一渲染选窗口按钮, 不再依赖组件前缀 2026-11-03 13:28:49 +08:00
yu 241c598b10 fix: 匹配阈值判定移到精化后并取消初始剪枝, 修复高相似度(0.95)误判未找到 2026-11-03 13:16:45 +08:00
yu 4824ac85f2 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-03 13:15:00 +08:00
yu b5d6bef369 fix: 窗口组件条件字段渲染+选窗口按钮; imageSelector加文件浏览与屏幕截取; prompt()改confirm(); 后端窗口selector统一读取所有标题字段; 新增桌面键鼠人工验收流程与E2E脚本 2026-11-03 13:12:29 +08:00
yu 3922447689 fix: 全桌面查找结果加回拼接画布物理原点, 修复副屏点击垂直偏移 2026-11-03 13:07:12 +08:00
yu e7669acd16 feat: 图像查找搜索整个虚拟桌面(多屏拼接), 窗口矩形与区域JSON统一DPI换算 2026-11-03 11:48:41 +08:00
yu 68e16282de fix: 点击脚本声明DPI感知避免物理坐标被二次缩放; 缩放兜底返回实际框尺寸 2026-11-03 11:39:17 +08:00
yu 9868f5d51c fix: 图像匹配改相似度语义并修三处准确度缺陷(尺度对齐/早退阈值/噪声死区) 2026-11-03 11:31:34 +08:00
yu 38af534f70 fix: 图片识别支持JPG等非PNG模板, 文件缺失与大图给出明确报错 2026-11-03 11:04:04 +08:00
yu 52423d46a4 fix: 图片路径类字段屏幕框选改为存模板截图路径, 仅搜索区域字段存区域JSON 2026-11-03 10:54:35 +08:00
yu 0960be88e5 fix: 框选轮询脚本免 System.Drawing 引用, 采样队列化并取消阻塞等待 2026-11-03 10:35:00 +08:00
yu 82765a7f19 fix: 框选所有退出路径恢复应用窗口; 轮询提前启动并同源采样按键与光标坐标 2026-11-03 10:28:24 +08:00
yu 817ea21c9a fix: 框选改为主进程轮询物理按键状态, 覆盖层最高层级置顶, 全路径清理 2026-11-03 10:15:41 +08:00
yu 257fdc7358 fix: 框选截图顺序与多覆盖层状态聚合\n\n- 主屏截到应用窗口: 之前框选结束后先恢复应用窗口再截图, 应用弹回盖住选区; 改为先销毁覆盖层并截图, 最后恢复应用窗口\n- 副屏拖拽无响应/Esc 无效: 轮询只读第一个(主屏)覆盖层状态; 改为聚合所有覆盖层, 任一屏 Esc 可取消、任一屏拖拽可识别\n- 多显示器截图源匹配增加按下→右排序对位回退(display_id 缺失时) 2026-11-03 09:59:57 +08:00
yu 5e6b7cd878 fix: 框选坐标改由主进程采样光标绝对位置\n\n- 混合 DPI 下窗口 clientX/clientY 与屏幕位置映射不可靠, 导致选区偏离鼠标实际位置\n- 渲染层只上报按下/松开状态, 主进程轮询 getCursorScreenPoint 取屏幕绝对 DIP 坐标计算选区\n- 选区正确性不再依赖覆盖层窗口几何, 裁剪与返回值均基于绝对坐标 2026-11-03 09:46:36 +08:00
yu 0641c5b91b fix: 副屏框选覆盖层尺寸不对\n\n- 混合 DPI 多显示器下, BrowserWindow 构造参数直接传副屏 bounds 会按主屏 DPI 虚拟化, 导致副屏覆盖层盖不满或超出\n- 改为先创建隐藏窗口再 setBounds 副屏 bounds, 此时窗口已关联副屏 DPI 2026-11-03 09:39:58 +08:00
yu b40c15d5b2 feat: 框选与 CV 查找支持多显示器副屏
- pickScreenRegion 每个显示器各开一个框选覆盖层, 副屏可直接框选; 返回屏幕绝对 DIP 坐标+该屏 scaleFactor
- cv-vision 全屏查找改为逐显示器 desktopCapturer 截屏(含负坐标副屏), 命中换算为绝对 DIP 坐标; 区域截图命中按 scaleFactor 换算
- 点击/悬停/输入坐标经 screen.dipToScreenPoint 转物理像素, 修复缩放非 100% 时点击偏移
- 新增 2 个多显示器测试用例(副屏命中映射/区域图缩放换算), 11/11 通过
2026-11-03 09:35:15 +08:00
yu 22054fed37 fix: 截屏改用 desktopCapturer, 修复截 Electron 窗口全黑
- GDI CopyFromScreen 截不到 GPU 合成渲染的窗口内容, 用户框选 Electron 编辑器界面时截图全黑, OCR 无从识别
- pickScreenRegion 改用 desktopCapturer 截屏后按 scaleFactor 裁剪(与 AI 拾取 capturePaneForAi 同款方案)
- cv-vision grabScreen 同步改为 desktopCapturer 优先, 不可用时回退 GDI 脚本
2026-11-03 09:17:23 +08:00
yu d98185d1cf feat: CV 组件搜索区域 UI 接入完成
- catalog 为 CV_MouseClick/GetText/SetText/MouseHover 新增可选「搜索区域」字段(region widget)
- 属性面板 region 字段渲染「屏幕框选」按钮, 框选后存 {x,y,width,height,imagePath}
- 运行时框选截图内 OCR 定位 + 偏移换算屏幕绝对坐标; 纯坐标范围则全屏截屏后过滤区域内命中
- 新增 2 个区域过滤测试用例(区域内命中/区域外拒绝)
2026-11-03 09:10:00 +08:00
yu 5aa113ea0e feat: CV 组件支持可选搜索区域(第一阶段)
- locateTextOnScreen 支持区域参数: 框选截图(imagePath)+屏幕偏移(x,y) 或纯坐标范围过滤
- locateFor 读取 searchRegion/region 参数; parseRegionArg 兼容 JSON 与图片路径
- 框选截图时 OCR 只识别该图, 命中坐标加偏移得到屏幕绝对坐标, 避免同名文本干扰
- pickRegionToField: region 类型字段存完整 {x,y,width,height,imagePath}
- 明日继续: catalog 给 CV 四组件加 searchRegion 字段 + region 分支屏幕框选按钮
2026-11-02 17:55:18 +08:00
yu c93543be1c fix: 移除 CV 组件重复注册与非法单行 here-string
- remaining-core.ts 曾重复注册 CV_GetText/SetText/MouseHover/MouseClick 并覆盖 cv-vision.ts 的实现
- 其中 CV_MouseClick/MouseHover 内联的 Add-Type @'...'@ 单行 here-string 是非法语法(@' 后必须紧跟换行), 必然解析失败报 CLIXML 错误
- 移除重复注册, CV 组件统一走 cv-vision.ts 的 -MemberDefinition 实现
- 顺手修复 Image_Move 系列同款单行 here-string bug(改为 -MemberDefinition)
2026-11-02 17:46:14 +08:00
yu 58c23591a5 feat: 属性面板参数说明优化, 让用户清楚每个参数该填什么
- catalog-overrides 为 vision 组 88 个组件补充 229 个字段的 description/placeholder
  (目标/偏移量/图片路径/引擎选择等高频易混字段), catalog.json 同步应用
- Properties.tsx: 元数据缺描述时按控件类型兜底提示(element/file/folder/region)
  vision 组件 element 字段提示「填写屏幕上可见的目标文本, OCR 定位」
- 必填标识从红星 * 改为「必填」徽标, 更直白
- 字段说明文案样式增强(.field-desc): 10px muted 色置于输入框下方
2026-11-02 17:37:43 +08:00
yu dfc5041a67 fix: CV 截屏声明 DPI 感知并覆盖虚拟桌面, 失败保留截屏证据
- 缩放非 100% 时未声明 DPI 感知的截屏进程只覆盖主屏左上角逻辑区域, 右侧/副屏内容丢失
- grabScreen 先 SetProcessDpiAwarenessContext(-4), 与点击脚本坐标空间一致
- 截屏范围从主屏 PrimaryScreen 改为 VirtualScreen 全部显示器
- cv_not_found 时保留截屏并在错误详情附路径, 便于排查
2026-11-02 17:27:37 +08:00
yu afa32133ec Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-02 17:20:37 +08:00
yu bf5ad56063 fix: 属性面板拾取按钮错位
- vision 组件(CV_/OcrOffline_)的 element 字段不再显示浏览器拾取按钮, 目标为手填屏幕文本并更新占位提示
- 修复 file/folder 分支按钮嵌套错误(框选/选窗口按钮误嵌在浏览按钮内且重复两份), 还原为平级按钮
2026-11-02 17:16:17 +08:00
yu 3bc8b60e83 fix: 画布点阵背景重影
- .flow-canvas .react-flow 的 CSS radial-gradient 点阵与 React Flow Background 组件叠加
- 平移时两层点阵以不同速度移动产生重影; 移除 CSS 层只保留 Background 组件
2026-11-02 17:12:53 +08:00
yu f0b9d1beec fix: CV 视觉组件截屏脚本补加载 System.Windows.Forms
- grabScreen 只 Add-Type 了 System.Drawing, 引用 Windows.Forms.Screen 却未加载该程序集
- 导致 PrimaryScreen.Bounds 取值为 Null, 截屏链路 InvokeMethodOnNull, 所有 CV_* 组件运行失败
- 与 remaining-core.ts 中正常工作的同款脚本对齐
2026-11-02 17:02:42 +08:00
yu f9b980ab5c fix: 搜索框聚焦时不再出现双层边框
- search-box/task-search 内层 input 关闭自身 focus-visible 光圈, 只保留容器聚焦效果
2026-11-02 16:24:35 +08:00
yu 22e3c191a8 fix: 组件库分类默认全部收起
- 新建任务进入编辑器时不再自动展开前两个分组
- 仅在搜索时自动展开匹配分组, 清空搜索恢复收起
2026-11-02 16:21:41 +08:00
yu 32ac14a36f fix: Excel sheet Move fails on zh-CN Excel due to null optional COM args 2026-11-02 16:00:26 +08:00
yu f8859fed77 test: validate capability property schemas and options 2026-11-02 15:55:23 +08:00
yu 541908961b feat: 任务列表去掉状态列, 新增更新时间列
- 移除「已启用/未启用」状态列及全部 status-dot/task-status 样式
- 新增「更新时间」列, 显示任务最近修改时间(History 图标)
- 表格栅格 5 列重排, 清理 1250px 媒体查询对应覆盖
2026-11-02 15:54:50 +08:00
yu 03e3d31a83 feat: 移除右侧任务概览面板,任务列表占满整屏
- 删除 TasksPage 的 aside 任务概览区块及其全部 CSS
- 空状态并入列表: 无任务时在列表内显示「还没有任务+新建按钮」, 搜索无结果显示原提示
- .tasks-content 改为单列布局, 任务列表占满可用宽度
2026-11-02 15:45:35 +08:00
yu bf70cf92e5 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-02 15:45:07 +08:00
yu d7aff54bc6 feat: stabilize AI-picked desktop elements with UIA enrichment and window-relative selectors 2026-11-02 15:44:20 +08:00
yu 56d00fba49 revert: 移除任务列表批量删除功能
- 还原 TasksPage/App/app.css 至 1bb28e3 状态(5795c39 及其未提交的布局调整一并撤销)
- 应用户要求移除复选框/全选/删除所选
2026-11-02 15:33:00 +08:00
yu 5795c39a6f feat: 任务列表支持勾选批量删除
- 每行任务左侧新增复选框; 表头「任务」列新增全选(仅作用于当前筛选结果)
- 勾选后列表头部出现「删除所选(N)」按钮, 二次确认后批量删除
- 批量删除保留「至少保留一个任务」保护; 勾选集合随列表变化自动裁剪已删除 id
- 单行删除行为不变
2026-11-02 15:18:50 +08:00
yu 1bb28e3368 fix: 截图目标文件路径兼容文件夹写法
- 目标文件路径语义为完整文件路径(含文件名), 提示文案不变
- 容错: 填的是已存在文件夹或以分隔符结尾时, 自动往里存默认文件名(screenshot_时间戳.png 或配置的文件名称), 不再报 EISDIR
2026-11-02 15:09:23 +08:00
yu 29f3443110 fix: 保存对话框预填文件名并自动补扩展名; 截图组件隐藏图像来源字段
- Properties.tsx: 输出类字段保存框按过滤器预填默认文件名(未命名.png 等), 不再是空文件名没法直接点保存; 保存结果无扩展名时自动补所选类型扩展名(Windows 保存框不会自动补)
- catalog: Image_Screenshot 的 pictureSource(图像来源) 界面隐藏, 实现默认截当前屏幕; 历史流程存有 2(指定图片文件) 仍按原逻辑生效
2026-11-02 15:01:44 +08:00
yu 88d5e61c5b docs: 报告修复状态补提交号 2026-11-02 14:53:58 +08:00
yu ba17d6dc2a fix: 图像与OCR 检测报告问题全量修复(P0/P1/P2/P3)
- P0-1: GetOcrTextNew 传图键补 uiSelector, 组件恢复可用
- P1-1: Image_Click 全版本 position 生效(九宫格锚点+自定义偏移 JSON), catalog 补下拉选项
- P1-2: Click_V3 click_key 生效(左/右/中键/双击), catalog 补选项
- P1-3: GetOcrTable 按垂直中心聚合成行列(行内按 x 排序), showTable 写流程数据表(与 Bivartable 共享 __rpaDatabook), isExcel 用 exceljs 另存并处理 res/randomString 占位
- P2-1: Find_V1/Click_V3 find_range 生效(整个屏幕/指定窗口/指定区域), 窗口模式前置+取矩形裁剪, 坐标平移回屏幕系
- P2-2(补): Image_Screenshot 空路径兜底到流程证据目录
- P3-1: accuracy2 无特征匹配层实现, catalog 隐藏并注明
- P3-2: show_track=1 时鼠标 20 步插值平滑移动后点击
- P3-3: 截屏/Find 换 desktopCapturer(去掉逐次 Add-Type); 新增 vision-helpers.test.ts 12 项单测(区域解析/边界夹取/裁剪/九宫格)
- 回归: typecheck 通过; vitest 256 过(4 失败为桌面/PDF 环境遗留); 报告文档已标注修复状态
2026-11-02 14:53:34 +08:00
yu 342145cc46 feat: 屏幕截图组件实现图像来源与截图范围(原 P2 问题)
- vision.ts 重写 Image_Screenshot:pictureSource=1 当前屏幕(desktopCapturer)/2 指定图片文件(image_path 作输入, 输出走输出目录+文件名, 未配置则原样返回);shotRange=2 框选区域按 position/action 区域 JSON 裁剪(pngjs bitblt),区域支持 {x,y,width,height}/[x,y,w,h]/left,top/角点写法并夹取边界
- 移除 browser.ts 重复注册,实现单点维护
- catalog: pictureSource/shotRange 补下拉选项(当前屏幕/指定图片文件, 整个屏幕/框选区域)
- typecheck 通过;browser/cv-vision/capability-audit/app-system 回归 50 项全过
2026-11-02 14:36:14 +08:00
yu c11e1d1527 fix: 输出类文件字段改用保存对话框
- 目标文件路径/输出目录这类字段指向将来才生成的文件, 原先的打开文件对话框只能选已存在文件, 导致截屏保存位置无法选择
- 字段键/标签命中 输出|保存|另存|导出|目标文件|output/save/export 等关键词时改用保存对话框, 可直接输入新文件名
- 输入类字段保持打开对话框不变
2026-11-02 14:26:29 +08:00
yu e5f64c21e2 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-11-02 14:22:41 +08:00
yu 1ddd1f1b35 feat: enhance AI pane picking with OCR and cancellable hotkeys 2026-11-02 14:21:19 +08:00
yu b1fc8c25b8 fix: 文件选择对话框过滤器按字段上下文智能匹配
- 原先所有文件选择框都写死 Excel 过滤器, 选图片/PDF/Word 等均显示 Excel 类型, 且无法直接看到目标文件
- 现按 字段键/标签关键词 + 组件类别 推导: Excel/Word/PDF/图片/压缩/音频/视频/JSON 各归其类, 均附带「全部文件」兜底
- 无法识别的字段退回「全部文件」; 目录选择不受影响
2026-11-02 14:15:18 +08:00
yu 1c85a90f1f feat: align browser picker with desktop interaction 2026-11-02 14:12:41 +08:00
yu 9b9c6f17e4 fix: launch external pickup directly on target url 2026-11-02 14:05:28 +08:00
yu d39c77324a fix: reuse external browsers for element pickup 2026-11-02 13:44:46 +08:00
yu 927b18551f fix: use runtime registry in app system acceptance 2026-11-02 13:24:48 +08:00
yu 0132871885 feat: deduplicate capability palette 2026-11-02 13:08:03 +08:00
yu d22468e29f feat: remove capability execution status markers 2026-11-02 12:00:39 +08:00
yu e956ec9055 feat: 新增本地UI模型AI拾取与识别实验台 2026-11-02 11:55:36 +08:00
yu ee60435e2b merge: bring liuwenxin app-system fixes into main 2026-11-02 11:07:46 +08:00
yu 510f0d041d merge: integrate liuwenxin app-system fixes 2026-11-02 11:07:27 +08:00
yu 95fb07e876 feat: complete remaining capabilities and hide integration status 2026-11-02 10:43:11 +08:00
yu 66169590de docs: add missing capability inventory 2026-11-02 09:35:04 +08:00
yu a1c65fb42f docs: update capability completion and package status 2026-11-02 09:16:54 +08:00
liuwenxin d489288c8e test(应用与系统): 添加组件验收流程与复测报告
提供 13 组、74 个用例,覆盖 49 项可执行组件。
新增隔离运行、真实界面检查、结果汇总和安全安装脚本。
升级仅覆盖未修改的测试模板,并保留原文件备份。
为手动操作设置保护开关,保留修复前后验证证据。

验证:43 项相关检查、类型检查与构建通过。
2026-11-02 09:06:40 +08:00
liuwenxin 5f9e28715c fix(应用与系统)!: 修复组件执行与参数校验
修正 PID 空结果、目标类型判断与转换、毫秒等待。
让关闭程序和命令行按所选模式显示并校验参数。
统一 PowerShell 输出编码,修正 BAT 路径引号处理。
明确屏保和屏幕唤醒的实际能力,补充缺陷回归测试。

验证:类型检查、构建与组件回归检查通过。

BREAKING CHANGE: 判断数据类型改为返回布尔值。
需要类型名称的旧流程请改用 Basic_Type。
屏保 V1 移除未实现参数,旧自定义密码节点需重新添加。
2026-11-02 09:05:48 +08:00
yu 657bc6d7d0 feat: excel pivot macro picture and workbook COM operations 2026-11-02 09:00:24 +08:00
yu ae4f4a2460 feat: manual keyboard mode and ciphertext variables 2026-11-02 08:23:22 +08:00
yu 9a5f86c92a Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-30 18:02:50 +08:00
yu 85dcb94056 feat: 新增桌面AI拾取功能,支持视觉识别控件 2026-10-30 18:02:46 +08:00
yu e390b1b1eb feat: mail find reply and outlook message retrieval 2026-10-30 17:43:07 +08:00
yu 84d73f1b86 feat: pdf encryption image extraction rendering and table reading 2026-10-30 17:26:09 +08:00
yu 23d292fdff feat: word selection cursor and table com 2026-10-30 16:25:44 +08:00
yu 32269a5760 feat: desktop ground variants and desktop children 2026-10-30 16:02:21 +08:00
yu adf5f3f7cf feat: CV 视觉四能力(截图OCR定位点击/悬停/读文本/输入),修复 path import 与 getOcr 类型 2026-10-30 15:15:57 +08:00
yu f74d1865da chore: 修复 desktop-picker.ps1 中 DllImport 字符串的引号不一致问题 2026-10-30 14:44:36 +08:00
yu 483b01c8b6 fix: 拾取/框选的窗口收集不再过滤最小化窗口,恢复时 show+restore+focus 2026-10-30 14:42:01 +08:00
yu 11fc88eec9 style(desktop-picker): 修改桌面选择器的透明样式并添加边框
将背景色改为 fuchsia 并设置透明键,添加绘制边框的 Paint 事件处理,提升视觉效果
2026-10-30 14:22:52 +08:00
yu 4ad238ff37 fix: 桌面拾取脚本补 DPI 感知,修复混合缩放下坐标错位 2026-10-30 14:16:34 +08:00
yu 8de4791e57 fix: 修复按键状态检测的位掩码值 2026-10-30 14:14:45 +08:00
yu b9bd6d671b fix: 修复桌面元素拾取时窗口未恢复的问题并优化脚本功能 2026-10-30 13:59:34 +08:00
yu a12e6cb0bd fix: 拾取循环补回 DoEvents 消息泵(v3 重写时丢失) 2026-10-30 12:57:52 +08:00
yu e6c34bafe5 fix: UIA Point 类型名修正(System.Windows.Point),脚本循环验证无错误 2026-10-30 12:52:43 +08:00
yu ed9b9daa50 fix: 拾取脚本完整重写——v3 逻辑+全链路日志(此前骨架覆盖事故修复) 2026-10-30 12:01:33 +08:00
yu a1490303d4 fix: 高亮框初始隐藏,仅有效矩形时显示 2026-10-30 12:01:30 +08:00
yu a968075100 feat: 外接浏览器支持阶段三+四(多标签管理 + 拾取与高级组件适配)
- external.ts: listPageTargets 标签枚举(过滤 chrome://edge:// 内部页)、openNewPageTarget(PUT /json/new)、ExternalConnection 增加 httpBase/targetId
- session.ts: listTabs/switchTab/openNewTab/getCookies 实现;外部按 targetId 缓存多 ws 会话,切换时迁移弹窗/网络监听;close 关闭全部会话;pickElement 放开外部浏览器(拾取脚本经 Runtime.evaluate 注入,行为与内置一致)
- engine/types.ts: BrowserLike 增加 listTabs/switchTab/openNewTab/getCookies 及窗口/监听可选方法
- browser.ts: Create 支持 openNewTab;WebBrowser_Switch 按标题(正则)/网址真实切换
- web-batch2.ts: GetTAB/V1/V2 真实枚举+匹配+激活;全部组件优先使用流程已选浏览器会话
- web-crawl.ts: 采集/监听组件优先流程会话;BatchDataCrawlNew browserType=chrome/edge 拉起外部浏览器
- browser-control.ts: GetCookieNew 升级 Network.getCookies(含 HttpOnly)
- 冒烟: Edge 双 Tab(ws 会话独立)、/json/new、页面枚举、Network.getCookies 全通过
2026-10-30 11:56:08 +08:00
yu fd507e4f62 fix: 修复用户级安装的 Chrome 探测失败
- LOCALAPPDATA 环境变量缺失(受限启动环境)时退回 os.homedir() 拼接 AppData/Local 候选路径
- 注册表 App Paths 兜底从仅 HKLM 扩展为 HKLM + WOW6432Node + HKCU(用户级安装的 Chrome 注册在 HKCU)
- 本机复现: Chrome 装于 C:/Users/6/AppData/Local, 注册于 HKCU App Paths, 修复后可探测到
2026-10-30 11:36:30 +08:00
yu f28b5ffc5e feat: 浏览器类型字段改为下拉选择
- catalog.json: WebBrowser_Create/V2/V3/CreateBackground 的 type 字段补 options(内置运行浏览器/Chrome/Edge,label 与存储值分离)
- shared/types.ts: CatalogField.options 支持字符串或 {label,value} 对象
- Properties.tsx: select/radio 选项归一化渲染 label;当前值不在选项中时追加显示(兼容历史手填值如 zexplorer)
- typecheck 通过;全量 vitest 除 4 项依赖真实 COM 的历史失败外全部通过
2026-10-30 11:26:33 +08:00
yu 3d64392eb0 fix: 拾取循环补 DoEvents 消息泵,修复高亮框/信息条冻结不刷新 2026-10-30 11:26:01 +08:00
yu ccfd82e894 feat: 外接浏览器支持阶段一+二(Edge/Chrome 打开与基础自动化)
- 新增 browser/transport.ts: CdpTransport 抽象(Embedded=webContents.debugger / External=ws 直连调试端口)
- 新增 browser/external.ts: Chrome/Edge 路径探测(常见目录+注册表App Paths)、带独立 user-data-dir 与调试端口拉起、端口占用自增、/json/version 就绪轮询、远程连接模式
- session.ts: BrowserSession 改为传输层注入, navigate/currentUrl/download/弹窗/网络监听双模式适配; 外部会话按 (mode,type,port) 缓存复用; pickElement 外部模式明确报不支持(阶段四)
- capabilities/browser.ts: WebBrowser_Create/V2/V3/CreateBackground 读取 type/isRemote/ipAddress/port/arguments; type=chrome/edge 或 isRemote=1 走外部浏览器, 缺省仍走内置兜底
- 依赖: +ws ^8.22.0
- 冒烟已验证: Edge 156 拉起→CDP 连接→Page.navigate+loadEventFired→取标题→截图→setDownloadBehavior→关闭 全通过
- 内置浏览器行为不变(browser.test.ts 21 项回归通过)
2026-10-30 11:09:19 +08:00
yu a25f8c76b6 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-30 10:52:41 +08:00
yu cc28d49ab5 fix: 修复质量检查发现的选择器与资源问题
修复浏览器通配符过滤和 ARIA 选择器转义,收敛表单窗口生命周期,释放邮件连接,清理日志复制定时器,并校准 Word COM 测试超时。
2026-10-30 10:51:27 +08:00
yu bc0ed3fcd8 fix: 文件/FTP实测修复三处真Bug
- windows-system.ts: File_OpenFile 打开文件夹时 explorer.exe 成功也返回退出码1, 放行该退出码(实测验证)
- ftp.ts: FTP_CreateDir 的 cd() 改为 await, 消除 basic-ftp 并发冲突(launched a task while another one is still running, 实测验证)
- flowValidation.ts: flow.variables 空值防御, 避免旧流程JSON缺字段时渲染崩溃

实测: FileTest 32/32, FtpTest 10/10 全部通过
2026-10-30 10:40:05 +08:00
yu f048be792c feat: 桌面拾取 v3——顶部居中信息条、高亮框跟随、两阶段交互(锁定/微调),显式 UTF8 BOM 2026-10-30 10:34:03 +08:00
yu 517d08d7ef docs: 桌面拾取与定位——原厂实现解读与增强项登记 2026-10-30 10:34:03 +08:00
yu c592cd891f docs: 外接浏览器(Edge/Chrome)支持改动方案
- 传输层抽象:CdpTransport 接口隔离 webContents.debugger 与 WebSocket 两种通道,BrowserLike 接口与 169 个网页组件零改动
- 唯一新依赖 ws(~50KB);客户机器零安装(Edge/Chrome 原生 CDP)
- 分四阶段:传输层+外部连接(2-3天) → 打开网页组件参数生效(1天) → 多标签 Target 域(2-3天) → 拾取器与高级组件适配(2-3天),合计约 1.5-2 周
- 关键坑已注明:独立 user-data-dir 必带、端口占用自增、断连报错
2026-10-30 10:24:01 +08:00
yu 19eb53b5d8 fix: 修复文件组 5 处目录字段与实现键名脱节
- File_CreateFile: 目录字段 folder_path/name,实现读 path → 必报文件路径为空
- File_GetFileInfo / File_WaitFile: 目录字段 file_path,实现读 path → 读不到
- File_CompressFile: 目录字段 file_path,实现读 pathSrc → 压缩源找不到
- File_Move: 目录字段 file_path/target_path/name,实现与 File_File_Move 共用
  pathSrc/pathDest → 读不到;补齐别名兼容
2026-10-30 10:14:22 +08:00
yu a3fbae9d83 docs: 图像与OCR 组 10 项能力问题检测报告
- 静态核对 + 真实运行双路验证,含 P0~P3 分级问题清单与修复优先级建议
- P0: GetOcrTextNew 目录字段无法传图(不可用);P1: Click 忽略 position/click_key、GetOcrTable 输出字段无效
- 算法层(飞桨 OCR/模板匹配 0.9993)实测正常,问题集中在目录与实现脱节
2026-10-30 10:11:07 +08:00
yu a4b7e24acd feat: 桌面拾取 v2——两阶段交互(悬停选目标/微调模式)、PageUp/PageDown 层级切换、高亮框跟随、UTF8 BOM 2026-10-30 09:58:42 +08:00
yu 745da55e4e docs: 按运行时注册校准能力接入状态 2026-10-30 09:26:28 +08:00
yu 7920a61e12 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-30 09:11:18 +08:00
yu bb3412fff7 fix: 桌面拾取脚本补 UTF8 BOM,修复 PowerShell 5.1 中文乱码 2026-10-30 08:58:44 +08:00
yu b19f9aa9ba fix: 修复拼接两列数据键名脱节导致的假成功
- SpliceColData/DoJoint 实现读取的键(col1/cola)与 catalog 必填字段
  (col_name_1/col_name_2/col_num_1/col_num_2)完全脱节,填了参数也按
  undefined 找列,拼接出空值仍报成功
- 改读 catalog 真实键名,按 change_type 支持列名/列号两种模式,兼容旧键
2026-10-29 17:58:57 +08:00
yu e59034d700 fix: 拾取/框选时最小化应用自身窗口,完成后恢复 2026-10-29 17:12:15 +08:00
yu f353254a86 fix: 补齐 desktop 拾取 IPC 注册与 import 2026-10-29 16:45:59 +08:00
yu c2b09f39e6 fix: 桌面拾取按钮判定与重复函数清理 2026-10-29 16:45:59 +08:00
yu 0694bd45d3 feat: 编辑器交互优化——圆形返回键、名称块居中、面板按钮悬停动画、删除移至右上角
- 返回键改圆形按钮:悬停变蓝+箭头左移动画,面包屑仅留「编辑流程」
- 流程名称块保持工具栏绝对居中
- 保存/运行按钮悬停动画(弹性缩放/上浮光晕);撤销重做保持静态
- 节点复制/删除移至属性面板右上角,删除悬停红色更醒目
2026-10-29 16:28:26 +08:00
yu 543d5d1564 fix: 清理重复声明,拾取矩阵与桌面拾取稳定 2026-10-29 16:24:35 +08:00
yu 8cedf45628 fix: 修复设置单元格格式不生效与表格样式覆盖单元格样式
- SetCellFormat V1/V2/V3:格式解析补厂商 JSON 键映射(FontColor/FontSize/
  BoldAndItalic 位标志/BackgroundColor 带透明度/DigitalFormat.formatstring/
  FontFormat 字体名),纯格式代码(如 0.00)回落为数字格式——此前全部被静默忽略
- ShowTableStyle:行级样式(row.font/fill+commit)在 exceljs 中会覆盖该行
  已设置的单元格样式,改为逐单元格设置表头样式
- 经 18 节点覆盖流程实测:V3 红字宋体加粗、V2 蓝字、V1 数字格式 0.00 均落盘
2026-10-29 15:57:57 +08:00
yu 4163b57d38 fix: 桌面 Ground 能力归类到独立的桌面元素分组 2026-10-29 15:54:07 +08:00
yu 8870050aaa feat: 拾取策略矩阵——桌面/视觉/窗口能力按类型显示拾取按钮(桌面拾取/屏幕框选/窗口列表) 2026-10-29 15:43:14 +08:00
yu b40fcbdd35 fix: 根治保存必弹磁盘冲突——编辑不再污染乐观锁基准
- editorStore 新增 baseUpdatedAt(磁盘基准版本),仅 loadFlow/syncSavedFlow 更新
- 保存 expectedUpdatedAt 改用 baseUpdatedAt;此前用每次编辑都会拨快的
  flow.meta.updatedAt 做比对,导致任何编辑后首次保存必然误判冲突
- 移除 flows:save/get 诊断日志(保留内容一致放行逻辑)
2026-10-29 15:33:09 +08:00
yu 778f69e14d fix: 桌面拾取下钻到包含鼠标点的最小控件,修复命中容器级大元素 2026-10-29 15:26:44 +08:00
yu 8b0004bf72 feat: 桌面拾取器——UIA 元素拾取(空格确认/Esc 取消)、窗口列表、屏幕区域框选截图,属性面板按能力类型切换网页/桌面拾取 2026-10-29 15:22:05 +08:00
yu 1e649d7121 chore: 忽略 Tesseract 语言包缓存 2026-10-29 14:37:40 +08:00
yu 0e65fd751d fix: 设置测试补 aiConfig 期望;打包配置 asarUnpack 原生模块与飞桨模型 2026-10-29 14:32:50 +08:00
yu 3b0883c80f feat: OCR 后端切换飞桨——RapidOCR ONNX(@gutenye/ocr-node,PaddleOCR v4 模型全离线) 2026-10-29 14:13:50 +08:00
yu 06886c5b31 fix: OCR 识别开启 blocks 输出并从块树提取词/行,修复坐标与表格为空 2026-10-29 13:51:51 +08:00
yu 6f008ed8c0 fix: AI 请求加 170 秒超时保护,防止服务端无响应时无限挂起 2026-10-29 11:39:50 +08:00
yu a4cccbc858 fix: 修正 AI/NLP 能力 id 与目录一致(情感/关键词/实体/批量评论),补充讯飞变体 2026-10-29 11:13:38 +08:00
yu d351556293 feat: 接入 AI 文本家族(OpenAI 兼容协议,17 id)与本地 OCR(Tesseract.js,4 id);设置页新增 AI 服务配置 2026-10-29 10:58:35 +08:00
yu d331d56f2e feat: 接入 FTP 全家桶(10 id)——连接/列表/下载/上传/重命名/删除/建目录/存在判断/断开 2026-10-29 10:28:08 +08:00
yu 26c682d2f9 feat: Excel 高阶接入——格式/筛选/表格样式/隐藏/尾空单元格/列标识/选中区域/关闭/加载/写列(16 id) 2026-10-29 10:10:34 +08:00
yu fc1207cddd feat: P2 接入——屏幕锁/屏保/凭据/远程桌面/输入法/文件对话框(19 id) 2026-10-29 10:01:32 +08:00
yu bb23bb192c fix: 移除 System_ExecuteCommandV1 重复注册,统一使用原生实现(type:1=DOS 2=BAT 3=PS) 2026-10-29 09:35:51 +08:00
yu de514500a7 feat: 接入 System_ExecuteCommandV1——按 command_type 分派 PS/DOS/BAT 2026-10-29 09:25:20 +08:00
yu 303560f0f8 feat: P1 接入——人机交互表单(9 id)、Time 家族(7 id)、图表数据读取(2 id) 2026-10-29 09:21:21 +08:00
yu b08f056aba fix: 修复批二 handler 缺失声明与 FlowSaveResult 类型收窄 2026-10-28 18:05:08 +08:00
yu f6bddb08d8 docs: 能力接入规划与排期——明确不接入清单、暂缓池与分批排期 2026-10-28 18:01:50 +08:00
yu 69e5a1c35c docs: README 能力计数更新至 655/1016 2026-10-28 17:54:33 +08:00
yu 5248165e83 fix: 打开流程归一化写盘后用最新 meta 加载编辑器,消除首次保存误报冲突 2026-10-28 17:54:07 +08:00
yu 4ac9755416 feat: 接入网页批三——数据采集家族(14 id)与网页监听三件套 2026-10-28 17:53:44 +08:00
yu 2ce0c13f7b docs: 更新 README——能力接入进度、桌面自动化、保存冲突检测与测试任务 2026-10-28 17:49:18 +08:00
yu afb2ed7c08 chore: 忽略本地测试产物目录 2026-10-28 17:46:32 +08:00
yu 38a561a7cd test: 打包冒烟支持跳过单实例锁与固定隔离环境 2026-10-28 17:46:32 +08:00
yu d6165c910c fix: 修复 web-batch2 文件内字面 \\n 导致的语法错误 2026-10-28 17:46:32 +08:00
yu 4c84674b29 fix: 补注册弹窗自动关闭两个能力 2026-10-28 17:46:32 +08:00
yu 560134171f feat: 接入网页批二——文件上传/多标签/窗口状态/停止加载/相似元素数组/弹窗自动关闭 2026-10-28 17:46:32 +08:00
yu 9b8bc0eb1a fix: SetScrollPos 兼容目录参数 scrollpos,修复滚动位置不生效 2026-10-28 17:46:32 +08:00
yu 67a418129b test: 新增网页元素演练场页面与三项网页自动化测试任务 2026-10-28 17:46:31 +08:00
yu a5d9f0bfee fix: 消除保存并发竞态——同步锁防重复保存,冲突判定先比对内容 2026-10-28 17:38:41 +08:00
yu 6cb10db375 fix: 保存按钮改用交互模式保存,可处理磁盘版本冲突(原先仅 Ctrl+S 可解) 2026-10-28 17:29:17 +08:00
yu ab8b0cde48 style: 移除左侧导航选中态的蓝色指示条 2026-10-28 17:21:41 +08:00
yu a7c74b85a0 style: 流程名块相对工具栏整体绝对居中 2026-10-28 17:19:37 +08:00
yu f2eb449b87 style: 工具栏流程名居中显示;移除顶栏页面徽标 2026-10-28 17:15:06 +08:00
yu e00a1e4743 style: 工具栏重排——面包屑放大居左,流程名与统计移至右侧操作区旁 2026-10-28 17:08:40 +08:00
yu 69e9554250 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-28 17:02:14 +08:00
yu 2b372e2d7f refactor: 移除画布中央常驻空流程提示,引导职责由首访横幅承担 2026-10-28 16:42:19 +08:00
yu 37f2c8977b feat: 新建流程首次进入展示引导提示,5 秒后自动关闭 2026-10-28 16:37:25 +08:00
yu 7c7a1b1a1f fix: 任务列表忽略备份文件,避免同名重复任务 2026-10-28 16:29:30 +08:00
yu 3a1c176f4d fix: 切换流程时清空运行结果面板,避免显示上一流程的运行状态 2026-10-28 16:04:44 +08:00
yu 7f3aaac2ff fix: Excel_Cs_SaveAs 支持 path(目录)+name(文件名) 组合另存 2026-10-28 15:59:41 +08:00
yu 9d996f137b fix: 补齐 setValue/getValue 脚本缺失的 try-catch 闭合,修复 PowerShell 语法错误 2026-10-28 15:49:33 +08:00
yu 06981dfff3 fix: 点击前检测最小化/屏幕外窗口并自动还原,修复 -32000 坐标点击落空 2026-10-28 15:43:00 +08:00
yu 093fc302fa fix: 桌面脚本强制 Per-Monitor DPI 感知,修复多屏缩放下点击坐标错位 2026-10-28 15:33:27 +08:00
yu 3b05d10ff4 fix: App_Run 参数为空时不传 -ArgumentList,避免部分应用启动即退出 2026-10-28 15:18:58 +08:00
yu 9c973d9bad fix: 窗口唤起改用 EnumWindows 跨虚拟桌面查找并移回主屏可视区 2026-10-28 15:11:01 +08:00
yu 0fe264a884 fix: 清空区域兼容 cellPos 与 startRow/startColumn 键并支持清除格式模式 2026-10-28 15:04:55 +08:00
yu 2c32a8f51d feat: Ground 能力支持托盘/隐藏窗口自动唤起与 AttachThreadInput 前台化 2026-10-28 15:01:37 +08:00
yu 753508b500 feat: 桌面元素点击/填值/读值增加隐式等待重试(默认4秒) 2026-10-28 14:56:20 +08:00
yu 876e4969bd feat: 任务保存冲突检测——磁盘被外部修改时提示选择而非静默覆盖 2026-10-28 14:52:16 +08:00
yu 97fda1696a fix: Excel_SaveAs 支持 filename/isAbsolutePath 字段并处理相对路径 2026-10-28 14:49:10 +08:00
yu 12cf94da27 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-28 14:38:57 +08:00
yu 73a3f47bdf fix: 桌面窗口未找到时输出明确错误而非进程崩溃 2026-10-28 14:28:03 +08:00
yu 5863baa0ab feat: 刷新按钮移入任务列表面板头部并增加加载反馈
- 从页面标题栏移除刷新按钮,改放到任务列表搜索框右侧
- 点击时图标旋转动画,完成后 toast「任务列表已刷新」
- 刷新进行中防重复点击
2026-10-28 14:22:48 +08:00
yu 3bea611750 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-28 14:15:47 +08:00
yu 8a816fd41d feat: 接入桌面 UIA 元素能力(Ground 系列) 2026-10-28 14:04:56 +08:00
yu 8dd0aeefa6 fix: PDF 抽页页码留空时明确报错而非静默复制全本
pdfRange 选择非 all 且 pdfNum 为空时,此前 parsePdfPages 回退为
返回全部页面,导致配置错误时无声输出完整副本;现返回空集由
CreateNewPdf 抛出 No valid PDF pages
2026-10-28 13:41:50 +08:00
yu 9de0eeaa88 fix: 统一 Excel 工作簿共享存储,修复多组件互相覆盖
office.ts(内存句柄表)、office-file-extended.ts(每次重读磁盘)、
excel-common.ts(按路径独立缓存)、extended.ts(loadSheet 每次重建)
各自持有工作簿实例并整簿写回同一文件,后保存者会整体覆盖先保存者:
列宽/行高/合并被背景色冲掉、复制的 Sheet 被新建 Sheet 冲掉等。

- 新增 excel-store.ts:按路径共享同一 ExcelJS 实例,四个模块统一接入
- resolveWorkbook 返回流程变量持有的原始句柄对象,另存为改写 path
  能同步到后续节点(此前改在副本上,下游仍写旧路径)
- 另存为后以新路径重新登记实例并释放旧路径缓存
- ReadColumn 裁掉尾部空值(共享实例下触碰空行会撑大 rowCount)
- 新增 3 个共享存储回归测试(格式保持/复制Sheet保持/另存后续写)
2026-10-28 13:32:01 +08:00
yu 54571eff8d fix: Excel_RenameSheet 兼容目录键名 newSheet
处理器此前只读 newName/name/sheetName,而 Excel_RenameSheet 目录定义
与属性面板使用 newSheet,导致已填写新表名仍报「新 Sheet 名称为空」;
补充别名回归测试
2026-10-28 13:12:41 +08:00
yu ac20d59283 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-28 13:09:27 +08:00
yu 432e038e3c fix: 修正遍历列表负数结束索引 2026-10-28 13:08:53 +08:00
yu cb6d2ac365 feat: 变量引用自动化(默认输出变量/变量选择器/连线预填/校验提示)
- A 新增组件时自动填充输出变量(取 catalog suggestedName,重名自动加序号;
  复制节点同样去重)
- B 属性面板字段旁新增变量选择器:列出沿连线可达的上游输出变量与流程变量,
  点选插入 ${变量};替换场景限空值或恰为已定义变量名的裸名
- C 连线时若源节点有输出变量,自动预填目标节点首个空的对象型字段
  (workbook/objExcel/objBrowser 等)
- 运行前校验新增:引用未定义变量(warning)、对象字段误用裸变量名(warning)
- 新增 4 个校验回归测试
2026-10-28 11:33:58 +08:00
yu aaa0f2c3c7 Merge branch 'main' of http://43.140.215.244:7040/wangyukun/PRA-only-demo 2026-10-28 11:28:49 +08:00
yu ced89e26cf fix: 修复网页输入聚焦与降级写入 2026-10-28 11:27:26 +08:00
yu bec3d9daa4 feat: 支持配置全局网页元素等待 2026-10-28 11:17:28 +08:00
yu c5ac0ebc31 fix: Excel_WriteRange 对齐目录键名 cellPos/data/isSave
- 支持 cellPos(如 B2)定位起始单元格,与 column/row 双兼容
- 数据字段兼容 data 键(目录定义),保存字段兼容 isSave 键
- 新增 Excel_WriteRange 目录键名回归测试
2026-10-28 11:17:16 +08:00
yu 877dab703f fix: 对齐 Excel 新建/打开组件的目录字段键名
- newExcel 兼容 Excel_NewExcel 的 filePath 键(此前只读 path/folder,
  导致已填写文件路径仍报「Excel目录为空」)
- newExcel 支持 isMakedirs(是否创建该路径):False 且目录不存在时
  给出明确报错与建议;缺省时保持自动创建目录
- openExcel 兼容 Excel_OpenExcel / Excel_OpenExcelNew 的 filePath 键
- bool() 大小写不敏感,正确识别目录下拉框的 True/False
- 补充别名键名回归测试(4 例)
2026-10-28 10:59:53 +08:00
yu 5e9f0df57f feat: 开放浏览器HTTP站点访问 2026-10-28 10:56:40 +08:00
yu 7d96badb1f feat: 接入浏览器元素拾取 2026-10-28 10:48:45 +08:00
yu 9cb4eeab96 feat: 重构运行日志界面 2026-10-28 10:32:19 +08:00
yu 4fb0f64d4f fix: 随包分发运行时资源目录 2026-10-28 10:06:37 +08:00
yu cf05e4aa6f build: 采用electron-builder标准NSIS安装器 2026-10-28 09:53:30 +08:00
yu f675cc0086 feat: 支持选择RPA安装目录 2026-10-27 18:23:25 +08:00
yu 12de8fbf58 fix: 生成可验证的原生Windows安装包 2026-10-27 18:16:03 +08:00
yu dd27ec27f2 fix: 使用IExpress生成可安装分发包 2026-10-27 16:53:03 +08:00
yu 442eeaf9c1 build: 将浏览器E2E加入打包门禁 2026-10-27 16:44:33 +08:00
yu cb68af5a0f fix: 提升Windows分发包启动兼容性 2026-10-27 16:43:28 +08:00
yu 5de47ee420 feat: 增加可安装的RPA示例任务 2026-10-27 16:37:22 +08:00
yu 76f2160950 feat: 批量接入常用RPA能力 2026-10-27 16:29:52 +08:00
212 changed files with 190556 additions and 2177 deletions

No files matched your search

+7
View File
@@ -7,3 +7,10 @@ etest2/
*.tmp
.DS_Store
Thumbs.db
test-output/
web-capability-audit.txt
*.traineddata
temp/
+57 -18
View File
@@ -1,21 +1,52 @@
# RPA Studio
画布式 RPA 桌面端(Electron + React Flow)。把 1016 项能力目录拖到画布上,连线、配置、运行,生成可执行的 RPA 流程。
画布式 RPA 桌面端(Electron + React Flow)。把 658 项主能力拖到画布上,连线、配置、运行,生成可执行的 RPA 流程。
> 当前状态:任务优先的编排工作台。默认进入“编排”任务列表,编辑任务后进入上下流向画布;组件库按分类树展示,设置为独立入口。
> 当前可编排能力:**658 / 894**(122 项不接入和 236 项重复旧版本已分离;网页、桌面、数据、文件、办公、HTTP 等主能力可直接编排)。
## 已实现(Day 1)
## 已实现
### 编排与运行
- Electron 31 + React 18 + TypeScript + React Flow 画布
- 1016 项组件目录:997 厂商 + 16 控制 + 3 触发器,搜索 / 11 组筛选 / 可执行与待接入状态
- 编排工作区:任务列表、任务概览、运行/编辑入口;左侧主导航仅保留“编排”和“设置”
- 组件库树:按能力分组展开/收起,搜索命中时自动展开分类,叶子组件支持拖拽
- 658 项主能力目录:搜索 / 11 组筛选;236 项重复旧版本自动隐藏,122 项专用能力已分离备份;任务列表实时刷新
- 拖拽到画布、节点卡片、连线、缩放/平移/MiniMap、框选、撤销/重做
- 画布主流程上下连接;异常出口默认隐藏,可在节点失败策略中按需开启
- IF / 计次循环 / 遍历 / 条件循环 / 异常捕获 容器:可把组件拖入分支或循环体
- 属性面板:按字段类型渲染输入、必填、变量、失败策略(超时/重试/间隔/跳过/延时)
- 流程 JSON 本地原子保存、自动保存、重启恢复
- 运行页与设置页骨架;运行期完全离线
- 流程 JSON 本地原子保存、自动保存、重启恢复;**保存冲突检测**——磁盘被外部修改时弹窗选择“加载磁盘版”或“覆盖磁盘”,不再静默互相覆盖
- 运行中心:节点进度、运行日志(分级过滤/复制/导出)、运行变量查看
- 任务列表忽略备份文件;编排页与运行页骨架分离
### 网页自动化(Electron 内置浏览器 + CDP 可信输入)
- 浏览器:打开/关闭/切换标签、跳转/后退/前进/刷新、等待加载/停止加载、最大化/最小化
- 信息获取:URL、Title、网页源码、Cookies 读写、浏览器截图、滚动条读写
- 元素操作:点击、输入、密码框、下拉选择、勾选框、相似元素列表/数组、元素对象与信息、父/子/兄弟元素、文本/值/属性/源码读取、存在判断、按钮可用性、滚动到元素、鼠标悬停、元素截图、元素坐标
- 执行 JS(多版本)、文件上传(CDP 直塞文件输入框,绕开原生对话框)
- 网页数据表格写入(按行列号精确覆盖)
- 可信输入机制参考 `C:\rpa-demo`:CDP Input、唯一定位、遮挡检查和输入回读校验;不使用 `el.click()` 伪造点击
### 桌面自动化(Ground 系列,PowerShell + UIA 桥)
- 桌面元素点击(前台化 + AttachThreadInput 强制前台 + 真实鼠标坐标点击)、文本输入(ValuePattern + 剪贴板粘贴兜底)、元素存在判断、文本读取、等待元素
- 窗口自动唤起:EnumWindows 跨虚拟桌面查找 → ShowWindow 还原 → 移回主屏可视区
- Per-Monitor DPI 感知:多屏混合缩放下坐标不错位
- 隐式等待重试:元素未就绪时 4 秒内自动重试
- 底层统一走 PowerShell + UIAutomation(EncodedCommand,中文安全)
### AI 与离线 OCR(可选自配)
- AI 文本能力:文本生成 / 纠错 / 审核 / 关键词 / 对话 / 模型调用 / 文档比对等,通过设置页配置的 OpenAI 兼容服务接入
- 离线 OCR:图像识别 / 表格识别(含坐标)走本地模型,无需联网
### 质量与测试
- 全量测试 231 项(能力注册冒烟、流程运行集成、组件缺陷回归、测试流程包与安装保护、Word/Excel/PDF/桌面桥真实设备测试)
- 应用与系统 50 项组件验收:13 组任务、75 个用例,含真实执行与人工确认项
- 本地演示站(`127.0.0.1:17863`):网页订单页 + 网页元素演练场(表单/下拉/勾选/密码/禁用按钮/悬停/长页面)
- 内置 3 个网页自动化测试任务(浏览器导航、元素读写、滚动与 Cookie)+ 6 条固定示例任务
- 打包冒烟测试:跳过单实例锁隔离启动,验证产物可用性
## 开发
@@ -35,24 +66,32 @@ npm run test:e2e # 隔离启动应用并执行本地网页订单完整流
## 目录
```
src/main 主进程:窗口、目录加载、流程存储、IPC
src/preload 白名单桥
src/renderer React UI:画布、组件库、属性面板、运行/设置页
src/shared 主/渲染共享类型
scripts build-catalog / dev-safe / start / cdp-probe / 打包
resources catalog.json(1016 项派生元数据)、夹具站、桌面桥
docs 现状分析、实施方案
src/main 主进程:窗口、目录加载、流程存储、IPC
src/main/browser 运行浏览器会话(BrowserSession,CDP 桥)
src/main/capabilities 能力实现(网页/桌面/数据/文件/办公/HTTP…)
src/main/fixture 本地演示站
src/preload 白名单桥
src/renderer React UI:画布、组件库、属性面板、运行/设置页
src/shared 主/渲染共享类型
scripts build-catalog / dev-safe / start / cdp-probe / 打包冒烟
resources catalog.json(1016 项派生元数据)、夹具站(含元素演练场)、桌面桥
docs 现状分析、实施方案
```
## 文档
- [应用与系统 50 项组件测试任务](resources/test-flows/app-system/README.md):13 组可安装任务、使用步骤、覆盖清单及实际验收报告。
- [现状分析与可行性](docs/01-现状分析与可行性.md)
- [实施方案(三天)](docs/02-实施方案.md)
- [未接入能力名单](docs/04-未接入名单.md)
- [重复能力去留清单](docs/05-重复能力去留清单.md)
## 边界
- 1016 项目录中当前有 509 项通过能力注册表或运行时容器标记为可执行,覆盖控制/触发、除数据库外的数据处理、文件、网页元素层级/表格/下拉框及 DOM 高级能力、HTTP、Excel 常用历史版本与表格辅助、系统路径与环境变量、进程管理、桌面键鼠/窗口对象/剪贴板/对话框、PDF、Word 文档编辑、SMTP 邮件变体、文本差异、工作日判断、屏幕截图、离线图像查找与点击;其余灰标「待接入」。运行前检查会提前定位待接入组件、必填参数、多出口与断链节点。
- 原始目录 1016 项中,894 项未被排除;其中 658 项主能力进入组件库,236 项重复旧版本在 `resources/capability-duplicates.json` 中标记兼容隐藏,122 项专用/厂商硬件/门户通道在 `resources/excluded.json` 中备份。组件库不显示接入状态。
- 内置本地演示站默认监听 `127.0.0.1:17863`;新用户首次启动会获得“本地网页订单演示”任务,可离线验证网页输入、点击和文本读取。
- 浏览器可信输入与定位机制参考 `C:\rpa-demo`:CDP Input、唯一定位、遮挡检查和输入回读校验;不使用 `el.click()` 伪造点击。
- 编排首页提供“安装示例任务”,可安装网页订单、文件文本、Excel读写、剪贴板、Word文档和异常分支 6 条固定示例;重复安装会更新同名示例,不会无限复制。
- Tauri/WebView2 类应用的 UIA 支持天然受限(Invoke 假成功、ValuePattern 读回掩码):桌面元素点击采用“前台化 + 真实坐标点击”策略,输入采用“ValuePattern 写入 + 剪贴板粘贴兜底”。
- 不包含厂商代码、二进制、模型或前端资产;目录是从厂商元数据派生的静态定义。
- 运行期不访问云服务;HTTP/邮件属于用户显式配置的外部系统。
- 运行期不访问厂商云服务;HTTP、邮件与 AI 能力属于用户在设置页显式配置的外部系统(OpenAI 兼容服务)。
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

+1 -1
View File
@@ -4,7 +4,7 @@
> 依据:`D:\code\Agent-Employee`(只读)、`C:\rpa-demo`(只读)、2026-10-26 需求确认
> 本方案是执行基线:范围、契约、排期、验收、降级顺序都在这里;实施中变更必须回写本文件。
> **2026-10-27 当前状态**:计划点名的代表能力已全部进入注册表或运行时容器,覆盖控制/触发、除数据库外的数据处理、文件、网页元素层级/表格/下拉框及 DOM 高级能力、HTTP、Excel 常用历史版本与表格辅助、系统路径与环境变量、进程管理、桌面键鼠/窗口对象/剪贴板/对话框、PDF、Word 文档编辑、SMTP 邮件变体、文本差异、工作日判断、屏幕截图、离线图像查找与点击;启动后目录显示 **509 项可执行 / 1016 项总目录**。当前 20 个测试文件共 124 条测试,覆盖计划能力审计、浏览器参数路由/变体/DOM层级、Excel/PDF/Word COM/SMTP/模板匹配、文件与数据表扩展、系统/进程/窗口解析、本地工具与 Databook、直线/IF/循环/重试/跳过、主出口与异常出口语义、运行记录回写、运行前检查、本地演示站与示例流程。数据处理仅剩 8 个明确排除的数据库组件。设置页可查看分类覆盖和处理器 ID 数量。未通过实际场景验证的变体仍不计入最终交付数量。
> **2026-10-30 当前状态**:常用本地能力已进入注册表或运行时容器,覆盖控制/触发、除数据库外的数据处理、文件、网页元素层级/表格/下拉框/浏览器控制/Cookie/下载及 DOM 高级能力、HTTP、Excel 常用读写/格式/筛选/历史版本、系统路径与环境变量、进程管理、桌面键鼠/窗口对象/剪贴板图片/对话框、PDF、Word 文档编辑、SMTP 与群机器人 Webhook、文本差异、工作日判断、屏幕截图、离线图像查找与点击、FTP、人机交互表单对话框、Windows 凭据/远程桌面/屏保、ECharts 图表读取、AI 文本能力(用户自配服务)、离线 OCR;启动后目录显示 **732 项可执行 / 1016 项总目录**。当前 26 个测试文件共 167 条测试,浏览器 E2E 真实结果为“订单创建成功:RPA 客户,金额 128”。数据处理仅剩 8 个待评估的数据库组件。设置页可查看分类覆盖和处理器 ID 数量。此后以集中场景测试与稳定性修复为主,不再追求厂商云端、硬件和专用通道数量。(原 2026-10-27 快照:619 项可执行 / 23 个测试文件 138 条测试)
> 浏览器端到端门禁:`npm run test:e2e` 使用隔离用户目录启动应用和本地演示站,真实执行打开网页、两次可信输入、点击提交和读取结果;期望结果必须精确包含“订单创建成功:RPA 客户,金额 128”。浏览器输入、点击、定位歧义和遮挡检查机制已对照 `C:\rpa-demo/actions.js` 校准。
+355
View File
@@ -0,0 +1,355 @@
# 能力接入规划与排期
> 生成时间:2026-10-28 · 校准时间:2026-11-02 · 收口时间:2026-11-02 · 原始目录 1016 项 · 可编排目录 894 项(运行时注册 906 个 id;122 项不接入已分离备份)
## 一、明确不接入清单(永久排除)
组件库显示「不接入」;运行前检查直接提示"该能力已明确不接入"。
### 门户专用(3 个)——绑定特定门户系统账号
- Basic_GetSystemAccount_v2 | 获取平台账号(通用)
- Basic_GetSystemAccount | 获取系统账号(门户专用)
- Basic_GetSystemAccount_v1 | 获取平台账号
### 厂商专属云服务(17 个)——依赖厂商专属数据/场景服务
- DataService_DeleteControllerFiles_v1 | 删除文件
- DataService_ReportException | 上报异常信息
- DataService_ProductionQueue | 将数据存入队列
- DataService_UploadControllerFiles_v1 | 上传文件
- DataService_ServiceNew | 数据服务(松塔)
- DataService_DeleteControllerFiles | 删除控制器文件
- DataService_Service | 数据服务
- DataService_DownloadControllerFiles | 下载控制器文件
- DataService_DownloadControllerFiles_v1 | 下载文件
- DataService_ConsumptionQueue | 从队列获取数据
- DataService_UploadControllerFiles | 上传文件至控制器
- DataService_AiScreenText | 提取屏幕数据(AI)
- DataService_AiScreenDate | 选择日期时间(AI)
- DataService_AiScreenOperate | 操作屏幕(AI)
- Scenario_CreditVerify | 征信报告解析
- Scenario_VatInvoice | 增值税发票验真
- Scenario_PhysicalExamination | 体检报告解析
### SAP 专用(1 个)——依赖SAP客户端
- Element_DoubleClickSap | 双击SAP表格单元格
### 硬件外设(41 个)——依赖手机设备连接
- Mobile_ConnectPhone | 连接手机
- Mobile_MobileElementAarray | 获取手机相似元素数组
- Mobile_Call | 拨打电话
- Mobile_MobileSetText_V1 | 手机键盘输入
- Mobile_ClickButton | 系统导航键
- Mobile_SetInputV1 | 手机输入文本
- Mobile_ConnectPhoneV3 | 连接手机
- Mobile_GetAttribute | 手机元素属性拾取
- Mobile_Exists | 手机元素判断
- Mobile_MobileElementList | 获取手机相似元素列表
- Mobile_FileTransfer | 手机文件传输
- Mobile_GetPhoneInfomation | 获取手机连接详情
- Mobile_ClickLocation | 手机屏幕坐标点击
- Mobile_SaveFilePhone | 保存文件到手机
- Mobile_SetInput | 设置文本|填写输入框
- Mobile_MobileCvClick | 手机智能元素点击
- Mobile_TouchSlideV1 | 触摸滑动
- Mobile_GetPhoneFiles | 从手机中获取文件
- Mobile_OpenMobileApp | 打开手机APP
- Mobile_TouchSlide | 触摸滑动
- Mobile_GetMobileUiTree | 获取手机UI树
- Mobile_GetScreenOrientation | 获取手机屏幕方向
- Mobile_GetElement | 获取手机元素对象
- Mobile_MobileCvSetText | 智能设置文本|填写输入框
- Mobile_ClosePhone | 关闭连接手机
- Mobile_SetInputV2 | 手机输入文本
- Mobile_SetScreenOrientation | 设置手机屏幕方向
- Mobile_ImageClick | 点击手机图像
- Mobile_CloseMobileApp | 关闭手机APP
- Mobile_ElementShot | 手机元素截图
- Mobile_ScreenShot | 手机截屏
- Mobile_GetClipboardText | 获取手机剪切板文本
- Mobile_ConnectPhoneV2 | 连接手机
- Mobile_GetText | 手机元素文本拾取
- Mobile_MobileSetText | 手机键盘输入
- Mobile_MobileElementList_V1 | 获取手机相似元素列表
- Mobile_ElementClick | 手机元素点击
- Mobile_GetChildren | 获取手机子元素
- Mobile_SetClipboardText | 设置手机剪切板文本
- Mobile_ConnectPhoneV1 | 连接手机
- System_GetMobileMessage | 获取手机短信
### 厂商专属云服务(30 个)——依赖厂商训练模型/打码服务
- OCR_RotateVerificationCodeV1 | 旋转图片验证码
- OCR_PuzzleVerificationCode_v1 | 推理拼图验证码
- OCR_ClickVerificationCodeV2 | 点选验证码
- OCR_SourceCommonEnglishNumbersCodeV2 | 普通验证码
- OCR_SimpleSliderVerificationCode | 滑块验证码(滑动条式)
- OCR_SourceMergeCardOCR | 识别卡证文字
- OCR_SliderVerificationCodeV5 | 滑动拼图验证码
- OCR_SliderVerificationCodeV4 | 滑动拼图验证码
- OCR_SourceMergeOtherOCR | 识别其他场景文字
- OCR_SliderCode | 滑块验证码
- OCR_SourceMergeGeneralOCR | 通用文字识别
- OCR_RotateVerificationCodeV2 | 旋转图片验证码
- OCR_RotateVerificationCode | 滑块验证码(旋转式)
- OCR_ClickVerificationCodeV1 | 点选验证码
- OCR_PuzzleVerificationCode | 推理拼图验证码
- OCR_SimpleSliderVerificationCodeV2 | 滑动条验证码
- OCR_SourceCommonEnglishNumbersCode | 普通验证码
- OCR_SourceMergeCarOCR | 识别汽车场景文字
- OCR_SliderVerificationCodeV2 | 滑块验证码(拼图样式)
- OCR_SliderVerificationCodeV1 | 滑块验证码(拼图样式)
- OCR_SliderVerificationCode | 滑块验证码(拼图样式)
- OCR_SourceMergeEducationOCR | 识别教育场景文字
- OCR_SliderVerificationCodeV3 | 滑动拼图验证码
- OCR_ClickVerificationCode | 点选验证码
- OCR_SourceCommonEnglishNumbersCodeV1 | 普通验证码
- OCR_PuzzleVerificationCode_v2 | 推理拼图验证码
- OCR_SourceMergePaperOCR | 识别票据文字
- OCR_SimpleSliderVerificationCodeV1 | 滑动条验证码
- OCR_SourceMergeGeneralOCR_V1 | 识别通用文字
- OCR_SliderVerificationCodePosition | 获取拼图滑块位置
### 厂商专属云服务(4 个)——依赖厂商离线OCR模型文件
- OcrOffline_ClickOcrText_V1 | 鼠标点击OCR文本
- OcrOffline_MoveToOcrText | 鼠标移动到OCR文本上
- OcrOffline_ClickOcrText | 鼠标点击OCR文本
- OcrOffline_MoveToOcrText_V1 | 鼠标移动到OCR文本上
### 厂商专属云服务(6 个)——依赖百度云OCR账号与密钥
- OCR_BaiDu_CarOCR | 汽车场景类OCR
- OCR_BaiDu_OtherOCR | 其他场景类OCR
- OCR_BaiDu_EducationOCR | 教育场景类OCR
- OCR_BaiDu_GeneralOCR | 通用OCR
- OCR_BaiDu_CardOCR | 卡片类OCR
- OCR_BaiDu_PaperOCR | 票据类OCR
### 硬件外设(6 个)——依赖USB通道外设
- peripherals_UsbCom | 开/关USB通道
- peripherals_CloseNetworkUsbCom | 关闭NetworkUSB通道
- peripherals_CloseUsbCom | 关闭USB通道
- peripherals_NetworkUsbCom | 开/关NetworkUSB通道
- peripherals_OpenNetworkUsbCom | 开启NetworkUSB通道
- peripherals_OpenUsbCom | 开启USB通道
### 硬件外设(1 个)——依赖银行安全控件键盘
- SafeControlInput_KeyInput | 安全控件文本输入
### 硬件外设(4 个)——依赖机械臂设备
- System_ResetMechanicalArm | 复位机械臂
- System_MechanicalArmClick | 机械臂点击
- System_ConnectingMechanicalArm | 连接机械臂
- System_CloseMechanicalArm | 断开机械臂
### 硬件外设(2 个)——依赖U盾设备
- System_ConnectingUshield | 连接U盾
- System_CloseUshield | 断开U盾
### 语义已被覆盖(1 个)——下载能力已覆盖下载场景
- WebBrowser_DownloadDialog | 处理下载对话框
**小计:116 个 id**(2026-10-30 校准:原 128 项中 12 项已接入,移至下方清单)
### 已接入(原排除项,2026-10-30 校准移出)
通过设置页配置的 OpenAI 兼容 AI 服务接入(`src/main/capabilities/ai-caps.ts`):
- DataService_AIGenTxt | AI生成文本
- DataService_LargeModel | 大模型
- DataService_ZBrain | 自定义AI模型
- DataService_AgentCall | 调用智能体
- DataService_ChatBot | chatbot对话机器人
- DataService_KeyWord | 关键词抽取
- DataService_Correct | 文本纠错
- DataService_Review | 文本审核
- Scenario_TextComparison | 文档比对
通过本地离线 OCR 模型接入(`src/main/capabilities/ocr-caps.ts`):
- OcrOffline_GetOcrTextNew | 图像OCR识别
- OcrOffline_GetOcrTable | 获取表格数据(OCR)
- OcrOffline_GetOcrTextPosition | 图像OCR识别(含坐标)
## 二、暂缓池
### 已完成(超出原排期,2026-11-02 校准)
- [P1] 人机交互对话框 17/17:输入/文件/消息/自定义表单对话框(`src/main/capabilities/human-dialog.ts`)
- [P2] Windows 系统 12/12:凭据/远程桌面/屏保/输入法(`src/main/capabilities/windows-system.ts`)
- [P2] 图表数据读取 2/2:Element_GetLineBarChartData、Element_GetPieChartData(`src/main/capabilities/web-batch2.ts`)
- [P2] Excel 高阶 38/38:单元格格式、区域选择、行列显隐、尾部空白单元格、列标识、表格样式、Load/Close(`src/main/capabilities/excel-common.ts`);组合、图片、加密、移动、合并、透视表、宏、PDF 导出、Interop、切片器、循环(`src/main/capabilities/excel-com.ts`)
- [P3] 待评估池移出 20 项:FTP 10 项(`src/main/capabilities/ftp.ts`)、NLP 9 项(AI 服务 `src/main/capabilities/ai-caps.ts`)、File_OpenFile
- 原排除清单移出 12 项:见第一节「已接入」
以下为剩余暂缓项:
### 2026-11-02 完成批次(64 个)
- 桌面 Ground 补全 13/13:12 个 `*Ground` 变体 + `Element_DesktopChildren`(`src/main/capabilities/desktop-uia.ts`)
- Word 选区/光标/表格 COM 11/11(`src/main/capabilities/word-extended.ts`)
- PDF 加密/提取图片/转图片/表格 4/4(`src/main/capabilities/pdf.ts`)
- 邮件查找/回复/Outlook 读取 9/9(`src/main/capabilities/message.ts`)
- 键盘模拟人工开关 2/2 与 `Basic_Dodecryption` 1/1(`src/main/capabilities/desktop.ts`、`basic-encryption.ts`)
- Excel COM 专属 24/24(`src/main/capabilities/excel-com.ts`)
### [P3] 待评估(80 个)
- child_modules_v1 | 调用子流程
- concurrent_child_process_v1 | 并发调用子流程
- Database_BatchSQLExecute | 批量执行SQL语句
- Database_DBConnect | 连接数据库
- Database_SingleSQLQuery | 执行单SQL查询
- Database_BatchInsertExecute | SQL批量插入数据
- Database_AllSQLQuery_V1 | 执行全SQL查询
- Database_CloseDBConnect | 关闭连接
- Database_SQLExecute | 执行SQL语句
- Database_AllSQLQuery | 执行全SQL查询
- File_UploadBusinessResults_v1 | 上报办理结果
- File_StartBusiness | 上报办理业务
- File_UploadBusinessResults | 上报办理结果(门户专用)
- File_Collectupload | 上报采集数据
- File_UploadFolder | 结果打包上传
- FileTypeConversion_Office2pdf | OFFICE文档转PDF
- FileTypeConversion_Pdf2image2 | PDF转图片
- GoogleTable_GetAllData | 获取列信息(Google Sheets)
- GoogleTable_GetAllSheetsName | 获取Sheet名(Google Sheets)
- GoogleTable_WriteToSheetRegionColumns | 写入列(Google Sheets)
- GoogleTable_WriteToSheetRegionRows | 写入行(Google Sheets)
- GoogleTable_WriteToSheetCell | 写入单元格(Google Sheets)
- GoogleTable_WriteToSheetCellFormula | 写入单元格公式(Google Sheets)
- GoogleTable_GetSheetRowCount | 获取行数(Google Sheets)
- GoogleTable_DownloadSheets | 导出Sheet(Google Sheets)
- GoogleTable_CopyPasteSheet | 复制粘贴(Google Sheets)
- GoogleTable_GetColumnsByTableTitle | 按表头读取列(Google Sheets)
- GoogleTable_ChangeSheet | 激活Sheet(Google Sheets)
- GoogleTable_RenameSheet | 重命名Sheet(Google Sheets)
- GoogleTable_WriteToSheetRegionregion | 写入区域(Google Sheets)
- Image_Move_V2 | 鼠标移动到图像上
- Image_Base64Decode | 图片base64编码转二进制
- Image_Base64Encode | 图片转base64编码
- Image_Wait_V1 | 等待图像
- Image_Exists | 判断图像
- Image_Location | 图像位置
- Image_Location_V1 | 获取图像位置
- Image_Exists_V1 | 判断图像
- Image_CropImg | 截取图像
- Image_RelativePosition | 对比图像相对位置
- Image_Wait | 等待图像
- Image_Move | 移动到图像
- Image_Move_V1 | 移动到图像
- Mark_Mark | 图片打标
- NLP_XunFei_MachineTranslation | 机器翻译
- NLP_XunFei_FeelingAnalysis | 第三方情感分析模型
- NLP_XunFei_KeywordExtraction | 关键字提取
- OCR_MergeCardOCR | 卡证文字识别
- OCR_CalculationCode | 算术验证码
- OCR_IdCardV2 | OCR身份证
- OCR_BusinessCardV2 | OCR名片
- OCR_VehicleInvoiceV2 | OCR机动车发票
- OCR_MergeOtherOCR | 其他场景文字识别
- OCR_BusinessLicenseV3 | OCR营业执照(自研)
- OCR_PassportV2 | OCR护照
- OCR_MergeGeneralOCR | 通用文字识别
- OCR_KeyValueIdentification | 识别键值对结构
- OCR_AmountIdentification | 识别价格金额
- OCR_GeneralTextV2 | OCR高精度文本
- OCR_VatInvoiceV2 | OCR增值税发票
- OCR_DrivingLicenseV2 | OCR行驶证
- OCR_AccountPageV2 | OCR户口页
- OCR_PlateLicenseV2 | OCR车牌
- OCR_BankCardV2 | OCR银行卡
- OCR_BankCardV3 | OCR银行卡(自研)
- OCR_IdCardV3 | OCR身份证(自研)
- OCR_VatInvoiceV3 | OCR增值税发票(自研)
- OCR_MergePaperOCR | 票据文字识别
- OCR_DriverLicenseV2 | OCR驾驶证
- OCR_RealEstateCertV2 | OCR不动产权证
- OCR_TrainTicketV2 | OCR火车票
- OCR_TaxiInvoiceV2 | OCR出租车发票
- OCR_OfficialSealV2 | OCR公章
- OCR_MergeCarOCR | 汽车场景文字识别
- OCR_MergeEducationOCR | 教育场景文字识别
- OCR_HouseCertV2 | OCR房产证
- OCR_CommonEnglishNumbersCode | 普通验证码
- OCR_BusinessLicenseV2 | OCR营业执照
- OCR_VinV2 | OCR车辆vin码
- Element_exception_monitor_v2 | 异常监控
### [P2] 视觉定位与 OCR 读屏(5 个)
- logic_control_if_ai_vision | 判断屏幕图像(AI)
- CV_GetText | 获取文本
- CV_SetText | 文本输入
- CV_MouseHover | 移动到元素上
- CV_MouseClick | 元素点击
### [P2] 邮件收件:剩余连接/列表/操作/Exchange(19 个;查找、回复、Outlook 读取 9 个已完成)
- Mail_UnmarkSeen | 标为未读
- Mail_DoReplyMail_v1 | 回复邮件(Outlook)
- Mail_RecvConnect | 连接邮箱
- Mail_RecvMailGet | 取第几封邮件
- Mail_ExOperateMail | 操作邮件
- Mail_RecvGetList | 邮件筛选
- Mail_DoMoveMail | 移动邮件
- Mail_UnionConnect | 连接邮箱
- Mail_ExConnect | 连接Exchange邮箱
- Mail_RecvClose | 关闭连接
- Mail_DoSendMail | 发送邮件
- Mail_RecvCount | 取收件箱邮件数
- Mail_ExSearchMail | 查找邮件
- Mail_ExSendMail | 发送邮件
- Mail_OperateMail | 操作邮件
- Mail_MarkSeen | 标为已读
- Mail_DoMoveMail_v1 | 操作邮件(Outlook)
- Mail_DoReplyMail | 回复邮件
- Mail_DoSendMail_v1 | 发送邮件(Outlook)
## 三、接入排期总览
| 阶段 | 内容 | 状态 |
|---|---|---|
| P0 | 网页核心 170、桌面 Ground 核心 5、网页批二 15 id、网页批三 16 id | 已完成 2026-10-28 |
| P1 | 人机交互对话框 17、Time 家族 24 | 已完成 2026-10-30 校准 |
| P1 | 文件上传验证 | 下一迭代 |
| P2 | 最后一波 98:子流程、数据库、转换、Google Sheets、图像、CV、NLP、OCR、邮件剩余 | 已完成 2026-11-02 |
| 备份 | 门户上报、图片打标、移动硬件、厂商云服务、SAP 等 122 项 | 已分离 `resources/excluded.json` |
| 永久排除 | 明确不接入清单 116 项(第一节) | 不排期 |
## 四、落地机制
- 本文档为能力接入的唯一规划来源;新增能力须先在此登记阶段
- 2026-11-02 校准:数字以运行时注册表为准(808 个 id,目录可执行 796 / 1016);2026-10-30 校准后原排除/暂缓中已实现的 78 项已移入「已接入 / 已完成」
- 本批次新增依赖:`@cantoo/pdf-lib`(PDF 加密)、`@napi-rs/canvas`(PDF 渲染)、`imapflow`/`mailparser`(IMAP 收件)、`sm-crypto`(SM4 密文变量)
- 不接入能力备份:[04-未接入名单](04-未接入名单.md)
- 下一步:将第一节清单固化为 resources/excluded.json,组件库显示「不接入」灰标
## 六、桌面拾取与定位——原厂实现解读与我们的增强项(2026-10-30)
### 原厂实在智能的实现(源码解读)
- 拾取(ui_selector_window.py):全屏半透明覆盖窗 + 高亮窗贴**窗格级**元素 + 左键确认;输出 uiSelector = { window: [{class,type,name} 链], shot: 自动截图 }——**拾取时自动截取目标窗口图片存档**
- 执行定位(Element/py/scope/native.py):路径链逐层走 UIA 树——每层匹配 type/class/name **通配符** + 同名兄弟 **index 序号**;窗口锚用标题通配(EnumWindows)
- 桌面点击执行:**窗口定位 + 图像/坐标模式**(原厂 Image/CV 能力家族庞大的原因)
### 我们当前方案对比
- 定位:UIA 属性选择器(name/id 精确匹配,单层)——精度高于原厂通配符,但**缺路径链/通配/序号 fallback**
- 拾取:v2 两阶段(悬停/微调)+ PageUp/PageDown 层级切换 + 最小化应用——已实现
### 登记的增强项(后续迭代)
1. 链式查找:拾取器已输出元素链(layers),执行端支持逐层匹配(原厂模式)——解决 name 重复场景
2. name 通配符匹配与同名序号 index
3. 拾取确认时自动截取目标窗口图片存档(调试回看)
4. WebView2 深层 DOM 控件的 UIA 暴露深度调优
@@ -0,0 +1,186 @@
# 04-外接浏览器支持改动方案(Edge / Chrome)
- **编写时间**:2026-10-30
- **编写人**:刘爱家
- **关联文档**:01-现状分析与可行性 / 02-实施方案 / 03-能力接入规划与排期
- **目标**:让「打开网页」及全部网页自动化组件支持在客户本机的 Edge / Chrome 浏览器中执行,不再强制使用内置运行浏览器;内置浏览器保留为默认兜底。
---
## 一、现状分析(代码级)
### 1.1 现有架构
```
网页组件(web.ts / web-batch2.ts / web-crawl.ts,169 项目录项)
│ BrowserLike 接口(engine/types.ts,33 个方法)
▼
BrowserSession(browser/session.ts,1056 行,单例)
│ this.cdp(method, params)
▼
win.webContents.debugger(Electron 内置窗口「RPA 运行浏览器」)
```
- **会话模型**:`BrowserSession.create()` 单例复用一个 `BrowserWindow`,全局变量 `__rpaBrowserWindow` 持有;单窗口单 Tab 模型(`tab_unavailable` 错误即来自此)。
- **CDP 使用现状**:全部经 `webContents.debugger`(协议版本 1.3),使用的域:
- `DOM.getDocument / DOM.querySelector / DOM.setFileInputFiles`
- `Input.dispatchMouseEvent / Input.dispatchKeyEvent / Input.insertText`
- `Runtime.evaluate`(仅 runJs 一处)
- `Page.enable / Page.captureScreenshot / Page.handleJavaScriptDialog`
- `Network.enable / Network.getResponseBody`(网页监听)
- **全部是标准 CDP 域,Edge / Chrome(同为 Chromium)原生支持,命令层零改动。**
- **目录字段已预留**:`WebBrowser_Create / V2 / V3` 均有 `type`(浏览器类型,默认 chrome)、`isRemote`(是否远程)、`ipAddress`、`port`(默认 61619)、`arguments`(携带参数)、`openNewTab` 字段——**但实现不读取**,无论填什么一律打开内置窗口。
### 1.2 关键约束
| 约束 | 说明 |
|---|---|
| Electron 31 主进程 = Node 20.x | 无稳定原生 WebSocket 客户端,连接外部调试端口需引入 `ws` |
| 用户已开着的浏览器 | 未带调试端口启动的实例无法连接(Chromium 安全限制),必须带参重启 |
| 内置窗口独有 API | `webContents.debugger` 仅 Electron 可用;`Page.captureScreenshot` 外部浏览器同样可用(证据截图无障碍) |
| 元素拾取器 | 现拾取链路绑定内置窗口,外部浏览器需改走 CDP DOM 快照 |
---
## 二、总体设计
**核心思路:传输层抽象,一次替换,上层不动。**
```
BrowserLike(接口不变)
▼
BrowserSession(cdp() 委托给 transport,其余逻辑不变)
▼
CdpTransport 接口(新增)
├── EmbeddedTransport —— 现有 webContents.debugger 封装(内置浏览器,默认)
└── ExternalTransport —— WebSocket 直连外部浏览器调试端口(新增,依赖 ws)
```
- `BrowserLike` 接口 33 个方法、上层 169 个网页组件**全部零改动**。
- 内置浏览器保留:`type` 未填或填内置类型时走原路径,行为与现在完全一致(回归风险隔离)。
### 浏览器类型枚举约定
| type 值 | 行为 |
|---|---|
| `chrome` | 探测本机 Chrome 路径 → 带调试端口启动 → CDP 接管 |
| `edge` | 探测本机 Edge 路径 → 同上 |
| `chrome_remote` / `edge_remote`(或 isRemote=1) | 不启动进程,直接连 `ipAddress:port` 上已监听调试端口的实例 |
| `zexplorer` / 缺省 | 内置运行浏览器(现状行为,兜底) |
---
## 三、分阶段实施
### 阶段一:传输层抽象 + 外部浏览器连接(核心,约 2~3 天)
**新增 `src/main/browser/transport.ts`**
```ts
export interface CdpTransport {
send<T>(method: string, params?: Record<string, unknown>): Promise<T>;
on(event: string, handler: (params: unknown) => void): void; // Page.loadEventFired 等
close(): Promise<void>;
kind: "embedded" | "external";
}
```
- `EmbeddedTransport`:包装现有 `webContents.debugger` 的 `send/on`(`session.ts` 现有 `cdp()` 逻辑原样迁入)。
- `ExternalTransport`:基于 `ws` 包,连接 `http://ip:port` 的调试端点:
1. `GET /json/list` 枚举页面级 target,取 `webSocketDebuggerUrl`(或 `PUT /json/new?url=` 开新 Tab)
2. 建立WebSocket,消息按 CDP 格式封包(`{id, method, params}` / 响应 `{id, result|error}` / 事件 `{method, params}`),promise map 按 id 配对
3. 断线检测(ws close 事件)→ 抛 `browser_disconnected`,带「请带调试端口重启浏览器」提示
**新增 `src/main/browser/external.ts`(外部浏览器进程管理)**
- 路径探测(按优先级):
- Chrome:注册表 `HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\chrome.exe` → `C:\Program Files\Google\Chrome\Application\chrome.exe` → `C:\Program Files (x86)\...`
- Edge:`HKLM\...\App Paths\msedge.exe` → `C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe`(Win10/11 必装,探测成功率 100%)
- 启动:`child_process.spawn(browserPath, [--remote-debugging-port=<port>, --user-data-dir=<独立目录>, url, ...args])`
- **必须带独立 `--user-data-dir`**:否则端口参数会被用户已开的实例吞掉(连不上且无报错的头号坑)
- 端口管理:默认取 `port` 字段(61619),被占用时自增重试;启动后轮询 `GET /json/version`(超时 15s)确认就绪
- 进程生命周期:记录 child process 引用;流程 `WebBrowser_Close` 时可选关闭(目录可加「执行后是否关闭浏览器」字段)
**改造 `src/main/browser/session.ts`**
- `BrowserSession` 构造函数改为接收 `CdpTransport`;`cdp()` 方法改为一行委托 `this.transport.send()`
- 单例键从 `__rpaBrowserWindow` 扩展为「按 (type, user-data/port) 维度」的会话缓存,支持同流程混合使用内置 + 外部
- `close()`:embedded 关窗口(现状逻辑);external 断开 ws,按配置决定是否杀浏览器进程
### 阶段二:「打开网页」组件参数生效(约 1 天)
**改造 `src/main/capabilities/browser.ts` 的 `WebBrowser_Create / V2 / V3 / CreateBackground`:**
```
type + isRemote + ipAddress + port + arguments ──→ BrowserSessionFactory
├─ type ∈ {chrome, edge} 且 isRemote=0 → external.ts 启动并连接
├─ isRemote=1 → 直连 ipAddress:port
└─ 其余 → 内置浏览器(现状)
```
- `waitingload`(等待页面加载完成):external 模式下监听 `Page.loadEventFired` 实现,语义与内置一致
- `arguments`(携带参数):透传给浏览器启动命令行
- 日志明确输出实际使用的浏览器(如「已使用 Edge (127.0.0.1:9222) 打开 https://...」),便于客户排查
### 阶段三:多标签页管理(约 2~3 天)
现状单 Tab 模型在外部浏览器不成立,需引入 `Target` 域:
- `Target.getTargets` / `Target.attachToTarget`(flatten 模式)/ `Target.closeTarget`
- 落地组件:`WebBrowser_GetTAB_V1`(按标题/URL 匹配 Tab,type 字段终于生效)、`WebBrowser_Switch`(切换当前操作的 Tab)、`CreateV3` 的 `openNewTab`
- 实现方式:ExternalTransport 按 targetId 维护多个 ws 会话,BrowserSession 增加 `switchTab(targetId)`;内置浏览器对应到多个 `webContents`
### 阶段四:拾取器与高级组件适配(约 2~3 天)
- 元素拾取:外部浏览器场景拾取请求走 CDP `DOMSnapshot.captureSnapshot` / `DOM.getDocument` 生成候选元素树(内置窗口继续走现有拾取链路)
- 逐组件验证(阶段一~三完成后自然可用,重点回归):`WebBrowser_GetCookieNew`、`WebBrowser_StartWeblistening / GetWeblistening`(Network 域,外部同样可用)、`WebBrowser_BatchDataCrawlNew`(browserType 字段接阶段二同一工厂)、`WebBrowser_SaveAs`(`DOM.setFileInputFiles` + `Page.handleJavaScriptDialog`)
- `Element_*` 系列 100+ 组件无需改动,靠接口层自动生效
---
## 四、依赖与风险
### 新增依赖
| 包 | 大小 | 用途 | 风险 |
|---|---|---|---|
| `ws` | ~50KB 纯 JS | 连接外部浏览器调试端口(Node 20 无原生 WebSocket) | 无原生编译,供应链风险低 |
**客户机器零安装**:Edge/Chrome 原生支持 CDP,不装任何扩展插件。
### 风险与对策
| 风险 | 对策 |
|---|---|
| 用户已开浏览器实例吞掉端口参数(连不上且无提示) | 必带独立 `--user-data-dir` 启动;文档注明该限制 |
| 调试端口与客户其他软件冲突 | 端口自增探测 + `/json/version` 就绪轮询 |
| 外部浏览器被客户手动关闭,流程中途断连 | ws close 事件 → 明确报错「浏览器已关闭」,错误路由可接重开逻辑 |
| 页面弹窗阻塞 CDP | `Page.handleJavaScriptDialog` 与内置行为一致,现有组件已覆盖 |
| 混用内置/外部时上下文串流 | 会话缓存按 (type, port) 隔离;`RunContext.browser` 生命周期不变 |
| URL 安全 | 沿用现有 `assertUrl`(仅 http/https)白名单逻辑,外部连接的 ip:port 另做私网地址校验 |
---
## 五、工作量与排期估算
| 阶段 | 内容 | 预估 |
|---|---|---|
| 一 | 传输层抽象 + 外部连接 + 进程管理 | 2~3 天 |
| 二 | 打开网页组件参数生效 | 1 天 |
| 三 | 多标签 Target 域 | 2~3 天 |
| 四 | 拾取器适配 + 高级组件回归 | 2~3 天 |
| 测试 | 与同事网页自动化测试联动,双浏览器(Edge/Chrome)× 关键链路 | 1~2 天 |
| **合计** | | **约 1.5~2 周** |
阶段一+二完成后即可向客户交付「用 Edge/Chrome 打开并执行基础网页自动化」;阶段三、四为完整体验补齐,可穿插进行。
---
## 六、测试计划(验收标准)
1. **连接**:Edge / Chrome 各验证——全新启动、已装非默认路径(注册表探测)、isRemote 远程连接、端口占用自增
2. **基础链路**:打开网页 → 点击 → 输入 → 取文本 → 截图 → 关闭,双浏览器全通过
3. **多标签**:开 3 个 Tab → GetTAB 按标题匹配 → 切换 → 各 Tab 独立操作 → closeTarget
4. **兜底**:type 缺省/填 zexplorer 时行为与现版本完全一致(内置浏览器回归)
5. **异常**:浏览器中途被手动关、URL 非法、调试端口不通、页面弹窗
6. **数据采集/监听**:BatchDataCrawlNew、StartWeblistening 在外部浏览器下与内置结果一致
+141
View File
@@ -0,0 +1,141 @@
# 不接入能力备份
> 快照时间:2026-11-02 · 该清单已从可编排目录分离,仅作能力溯源备份。
## 汇总
| 状态 | 数量 |
|---|---:|
| 原始目录总数 | 1016 |
| 当前可编排目录 | 894 |
| 不接入备份 | 122 |
| 运行时注册 id | 906 |
| ID | 名称 | 模块 | 原厂函数 |
|---|---|---|---|
| `Basic_GetSystemAccount_v2` | 获取平台账号(通用) | app | `Basic.GetSystemAccount_v2` |
| `Basic_GetSystemAccount` | 获取系统账号(门户专用) | app | `Basic.GetSystemAccount` |
| `Basic_GetSystemAccount_v1` | 获取平台账号 | app | `Basic.GetSystemAccount_v1` |
| `DataService_DeleteControllerFiles_v1` | 删除文件 | message | `DataService.DeleteControllerFiles_v1` |
| `DataService_ReportException` | 上报异常信息 | message | `DataService.ReportException` |
| `DataService_ProductionQueue` | 将数据存入队列 | message | `DataService.ProductionQueue` |
| `DataService_UploadControllerFiles_v1` | 上传文件 | message | `DataService.UploadControllerFiles_v1` |
| `DataService_ServiceNew` | 数据服务(松塔) | message | `DataService.ServiceNew` |
| `DataService_DeleteControllerFiles` | 删除控制器文件 | message | `DataService.DeleteControllerFiles` |
| `DataService_Service` | 数据服务 | message | `DataService.Service` |
| `DataService_DownloadControllerFiles` | 下载控制器文件 | message | `DataService.DownloadControllerFiles` |
| `DataService_DownloadControllerFiles_v1` | 下载文件 | message | `DataService.DownloadControllerFiles_v1` |
| `DataService_ConsumptionQueue` | 从队列获取数据 | message | `DataService.ConsumptionQueue` |
| `DataService_UploadControllerFiles` | 上传文件至控制器 | message | `DataService.UploadControllerFiles` |
| `DataService_AiScreenText` | 提取屏幕数据(AI) | message | `DataService.AiScreenText` |
| `DataService_AiScreenDate` | 选择日期时间(AI) | message | `DataService.AiScreenDate` |
| `DataService_AiScreenOperate` | 操作屏幕(AI) | message | `DataService.AiScreenOperate` |
| `Element_DoubleClickSap` | 双击SAP表格单元格 | web | `Element.DoubleClickSap` |
| `File_UploadBusinessResults_v1` | 上报办理结果 | file | `File.UploadBusinessResults_v1` |
| `File_StartBusiness` | 上报办理业务 | file | `File.StartBusiness` |
| `File_UploadBusinessResults` | 上报办理结果(门户专用) | file | `File.UploadBusinessResults` |
| `File_Collectupload` | 上报采集数据 | file | `File.Collectupload` |
| `File_UploadFolder` | 结果打包上传 | file | `File.UploadFolder` |
| `Mark_Mark` | 图片打标 | vision | `Mark.Mark` |
| `Mobile_ConnectPhone` | 连接手机 | mobile | `Mobile.ConnectPhone` |
| `Mobile_MobileElementAarray` | 获取手机相似元素数组 | mobile | `Mobile.MobileElementAarray` |
| `Mobile_Call` | 拨打电话 | mobile | `Mobile.Call` |
| `Mobile_MobileSetText_V1` | 手机键盘输入 | mobile | `Mobile.MobileSetText_V1` |
| `Mobile_ClickButton` | 系统导航键 | mobile | `Mobile.ClickButton` |
| `Mobile_SetInputV1` | 手机输入文本 | mobile | `Mobile.SetInputV1` |
| `Mobile_ConnectPhoneV3` | 连接手机 | mobile | `Mobile.ConnectPhoneV3` |
| `Mobile_GetAttribute` | 手机元素属性拾取 | mobile | `Mobile.GetAttribute` |
| `Mobile_Exists` | 手机元素判断 | mobile | `Mobile.Exists` |
| `Mobile_MobileElementList` | 获取手机相似元素列表 | mobile | `Mobile.MobileElementList` |
| `Mobile_FileTransfer` | 手机文件传输 | mobile | `Mobile.FileTransfer` |
| `Mobile_GetPhoneInfomation` | 获取手机连接详情 | mobile | `Mobile.GetPhoneInfomation` |
| `Mobile_ClickLocation` | 手机屏幕坐标点击 | mobile | `Mobile.ClickLocation` |
| `Mobile_SaveFilePhone` | 保存文件到手机 | mobile | `Mobile.SaveFilePhone` |
| `Mobile_SetInput` | 设置文本|填写输入框 | mobile | `Mobile.SetInput` |
| `Mobile_MobileCvClick` | 手机智能元素点击 | mobile | `Mobile.MobileCvClick` |
| `Mobile_TouchSlideV1` | 触摸滑动 | mobile | `Mobile.TouchSlideV1` |
| `Mobile_GetPhoneFiles` | 从手机中获取文件 | mobile | `Mobile.GetPhoneFiles` |
| `Mobile_OpenMobileApp` | 打开手机APP | mobile | `Mobile.OpenMobileApp` |
| `Mobile_TouchSlide` | 触摸滑动 | mobile | `Mobile.TouchSlide` |
| `Mobile_GetMobileUiTree` | 获取手机UI树 | mobile | `Mobile.GetMobileUiTree` |
| `Mobile_GetScreenOrientation` | 获取手机屏幕方向 | mobile | `Mobile.GetScreenOrientation` |
| `Mobile_GetElement` | 获取手机元素对象 | mobile | `Mobile.GetElement` |
| `Mobile_MobileCvSetText` | 智能设置文本|填写输入框 | mobile | `Mobile.MobileCvSetText` |
| `Mobile_ClosePhone` | 关闭连接手机 | mobile | `Mobile.ClosePhone` |
| `Mobile_SetInputV2` | 手机输入文本 | mobile | `Mobile.SetInputV2` |
| `Mobile_SetScreenOrientation` | 设置手机屏幕方向 | mobile | `Mobile.SetScreenOrientation` |
| `Mobile_ImageClick` | 点击手机图像 | mobile | `Mobile.ImageClick` |
| `Mobile_CloseMobileApp` | 关闭手机APP | mobile | `Mobile.CloseMobileApp` |
| `Mobile_ElementShot` | 手机元素截图 | mobile | `Mobile.ElementShot` |
| `Mobile_ScreenShot` | 手机截屏 | mobile | `Mobile.ScreenShot` |
| `Mobile_GetClipboardText` | 获取手机剪切板文本 | mobile | `Mobile.GetClipboardText` |
| `Mobile_ConnectPhoneV2` | 连接手机 | mobile | `Mobile.ConnectPhoneV2` |
| `Mobile_GetText` | 手机元素文本拾取 | mobile | `Mobile.GetText` |
| `Mobile_MobileSetText` | 手机键盘输入 | mobile | `Mobile.MobileSetText` |
| `Mobile_MobileElementList_V1` | 获取手机相似元素列表 | mobile | `Mobile.MobileElementList_V1` |
| `Mobile_ElementClick` | 手机元素点击 | mobile | `Mobile.ElementClick` |
| `Mobile_GetChildren` | 获取手机子元素 | mobile | `Mobile.GetChildren` |
| `Mobile_SetClipboardText` | 设置手机剪切板文本 | mobile | `Mobile.SetClipboardText` |
| `Mobile_ConnectPhoneV1` | 连接手机 | mobile | `Mobile.ConnectPhoneV1` |
| `OCR_RotateVerificationCodeV1` | 旋转图片验证码 | vision | `OCR.RotateVerificationCodeV1` |
| `OCR_PuzzleVerificationCode_v1` | 推理拼图验证码 | vision | `OCR.PuzzleVerificationCode_v1` |
| `OCR_ClickVerificationCodeV2` | 点选验证码 | vision | `OCR.ClickVerificationCodeV2` |
| `OCR_SourceCommonEnglishNumbersCodeV2` | 普通验证码 | vision | `OCR.SourceCommonEnglishNumbersCodeV2` |
| `OCR_SimpleSliderVerificationCode` | 滑块验证码(滑动条式) | vision | `OCR.SimpleSliderVerificationCode` |
| `OCR_SourceMergeCardOCR` | 识别卡证文字 | vision | `OCR.SourceMergeCardOCR` |
| `OCR_SliderVerificationCodeV5` | 滑动拼图验证码 | vision | `OCR.SliderVerificationCodeV5` |
| `OCR_SliderVerificationCodeV4` | 滑动拼图验证码 | vision | `OCR.SliderVerificationCodeV4` |
| `OCR_SourceMergeOtherOCR` | 识别其他场景文字 | vision | `OCR.SourceMergeOtherOCR` |
| `OCR_SliderCode` | 滑块验证码 | vision | `OCR.SliderCode` |
| `OCR_SourceMergeGeneralOCR` | 通用文字识别 | vision | `OCR.SourceMergeGeneralOCR` |
| `OCR_RotateVerificationCodeV2` | 旋转图片验证码 | vision | `OCR.RotateVerificationCodeV2` |
| `OCR_RotateVerificationCode` | 滑块验证码(旋转式) | vision | `OCR.RotateVerificationCode` |
| `OCR_ClickVerificationCodeV1` | 点选验证码 | vision | `OCR.ClickVerificationCodeV1` |
| `OCR_PuzzleVerificationCode` | 推理拼图验证码 | vision | `OCR.PuzzleVerificationCode` |
| `OCR_SimpleSliderVerificationCodeV2` | 滑动条验证码 | vision | `OCR.SimpleSliderVerificationCodeV2` |
| `OCR_SourceCommonEnglishNumbersCode` | 普通验证码 | vision | `OCR.SourceCommonEnglishNumbersCode` |
| `OCR_SourceMergeCarOCR` | 识别汽车场景文字 | vision | `OCR.SourceMergeCarOCR` |
| `OCR_SliderVerificationCodeV2` | 滑块验证码(拼图样式) | vision | `OCR.SliderVerificationCodeV2` |
| `OCR_SliderVerificationCodeV1` | 滑块验证码(拼图样式) | vision | `OCR.SliderVerificationCodeV1` |
| `OCR_SliderVerificationCode` | 滑块验证码(拼图样式) | vision | `OCR.SliderVerificationCode` |
| `OCR_SourceMergeEducationOCR` | 识别教育场景文字 | vision | `OCR.SourceMergeEducationOCR` |
| `OCR_SliderVerificationCodeV3` | 滑动拼图验证码 | vision | `OCR.SliderVerificationCodeV3` |
| `OCR_ClickVerificationCode` | 点选验证码 | vision | `OCR.ClickVerificationCode` |
| `OCR_SourceCommonEnglishNumbersCodeV1` | 普通验证码 | vision | `OCR.SourceCommonEnglishNumbersCodeV1` |
| `OCR_PuzzleVerificationCode_v2` | 推理拼图验证码 | vision | `OCR.PuzzleVerificationCode_v2` |
| `OCR_SourceMergePaperOCR` | 识别票据文字 | vision | `OCR.SourceMergePaperOCR` |
| `OCR_SimpleSliderVerificationCodeV1` | 滑动条验证码 | vision | `OCR.SimpleSliderVerificationCodeV1` |
| `OCR_SourceMergeGeneralOCR_V1` | 识别通用文字 | vision | `OCR.SourceMergeGeneralOCR_V1` |
| `OCR_SliderVerificationCodePosition` | 获取拼图滑块位置 | vision | `OCR.SliderVerificationCodePosition` |
| `OcrOffline_ClickOcrText_V1` | 鼠标点击OCR文本 | vision | `OcrOffline.ClickOcrText_V1` |
| `OcrOffline_MoveToOcrText` | 鼠标移动到OCR文本上 | vision | `OcrOffline.MoveToOcrText` |
| `OcrOffline_ClickOcrText` | 鼠标点击OCR文本 | vision | `OcrOffline.ClickOcrText` |
| `OcrOffline_MoveToOcrText_V1` | 鼠标移动到OCR文本上 | vision | `OcrOffline.MoveToOcrText_V1` |
| `OCR_BaiDu_CarOCR` | 汽车场景类OCR | vision | `OCR_BaiDu.CarOCR` |
| `OCR_BaiDu_OtherOCR` | 其他场景类OCR | vision | `OCR_BaiDu.OtherOCR` |
| `OCR_BaiDu_EducationOCR` | 教育场景类OCR | vision | `OCR_BaiDu.EducationOCR` |
| `OCR_BaiDu_GeneralOCR` | 通用OCR | vision | `OCR_BaiDu.GeneralOCR` |
| `OCR_BaiDu_CardOCR` | 卡片类OCR | vision | `OCR_BaiDu.CardOCR` |
| `OCR_BaiDu_PaperOCR` | 票据类OCR | vision | `OCR_BaiDu.PaperOCR` |
| `peripherals_UsbCom` | 开/关USB通道 | mobile | `peripherals.UsbCom` |
| `peripherals_CloseNetworkUsbCom` | 关闭NetworkUSB通道 | mobile | `peripherals.CloseNetworkUsbCom` |
| `peripherals_CloseUsbCom` | 关闭USB通道 | mobile | `peripherals.CloseUsbCom` |
| `peripherals_NetworkUsbCom` | 开/关NetworkUSB通道 | mobile | `peripherals.NetworkUsbCom` |
| `peripherals_OpenNetworkUsbCom` | 开启NetworkUSB通道 | mobile | `peripherals.OpenNetworkUsbCom` |
| `peripherals_OpenUsbCom` | 开启USB通道 | mobile | `peripherals.OpenUsbCom` |
| `SafeControlInput_KeyInput` | 安全控件文本输入 | desktop | `SafeControlInput.KeyInput` |
| `Scenario_CreditVerify` | 征信报告解析 | message | `Scenario.CreditVerify` |
| `Scenario_VatInvoice` | 增值税发票验真 | message | `Scenario.VatInvoice` |
| `Scenario_PhysicalExamination` | 体检报告解析 | message | `Scenario.PhysicalExamination` |
| `System_ResetMechanicalArm` | 复位机械臂 | app | `System.ResetMechanicalArm` |
| `System_MechanicalArmClick` | 机械臂点击 | app | `System.MechanicalArmClick` |
| `System_ConnectingUshield` | 连接U盾 | app | `System.ConnectingUshield` |
| `System_ConnectingMechanicalArm` | 连接机械臂 | app | `System.ConnectingMechanicalArm` |
| `System_CloseMechanicalArm` | 断开机械臂 | app | `System.CloseMechanicalArm` |
| `System_CloseUshield` | 断开U盾 | app | `System.CloseUshield` |
| `System_GetMobileMessage` | 获取手机短信 | app | `System.GetMobileMessage` |
| `WebBrowser_DownloadDialog` | 处理下载对话框 | web | `WebBrowser.DownloadDialog` |
## 恢复方式
如后续需要恢复某项,先实现对应 handler,再从 `resources/excluded.json` 的 `ids` 与 `components` 中移除,并重新运行 `npm run package`。
+925
View File
@@ -0,0 +1,925 @@
# 重复能力去留清单
> 快照时间:2026-11-02 · 保留 658 个主能力,隐藏 236 个重复旧版本 / 别名;执行器保留,旧流程不受影响。
## 去留规则
1. 先按能力名称聚合。
2. 再按目标域拆分:CV、桌面 Ground、网页元素、Excel、数据库、邮件协议等不互相吞并。
3. 同域内保留最新主版本;带 `Vn` 的优先级高于基础旧 ID,`New` 仅在没有更高版本时保留。
4. 重复 ID 标记 `duplicateOf` 后从组件库隐藏,但运行器和旧流程兼容保留。
## 汇总
| 状态 | 数量 |
|---|---:|
| 保留主能力 | 658 |
| 隐藏重复能力 | 236 |
| 原可编排数量 | 894 |
| 清理后组件库数量 | 658 |
## 保留主能力
| ID | 名称 |
|---|---|
| `Trigger.Manual` | 人工发起 |
| `Trigger.Schedule` | 定时触发 |
| `Trigger.FileArrival` | 文件到达触发 |
| `logic_control_if_else` | IF 条件分支 |
| `logic_control_var` | 设置变量 |
| `Basic_SetVariable_V1` | 设置变量 |
| `logic_control_count_for` | 计次循环 |
| `logic_control_while` | 条件循环 |
| `infinite_loop` | 无限循环 |
| `logic_control_next_for` | 继续下次循环 |
| `logic_control_break_for` | 跳出循环 |
| `logic_control_try_catch` | 异常捕获 |
| `logic_control_stop` | 退出流程 |
| `logic_control_stop_return` | 返回并退出流程 |
| `child_modules_v1` | 调用子流程 |
| `concurrent_child_process_v1` | 并发调用子流程 |
| `logic_control_array_for_v3` | 遍历列表 |
| `logic_control_dictionary_for_v1` | 遍历字典 |
| `Element_exception_monitor_v2` | 异常监控 |
| `logic_control_if_ai_vision` | 判断屏幕图像(AI) |
| `App_Run` | 启动应用程序 |
| `App_GetPid` | 获取一个匹配PID |
| `App_GetPids` | 获取全部匹配PID |
| `App_KillV1` | 关闭应用程序 |
| `Array_Reshape2Dim` | 转化数组维度 |
| `Basic_IsDictionary` | 判断数据是否为字典 |
| `Basic_StringToNum` | 将文本转换为数字 |
| `Basic_Delay_V2` | 等待 |
| `Basic_ConvertInt` | 数值转为整数类型 |
| `Basic_RandomInt` | 生成随机数 |
| `Basic_IsNumber` | 判断数据是否为数值 |
| `Basic_ConvertBool` | 数据转为布尔类型 |
| `Basic_Type` | 获取数据类型 |
| `Basic_Dodecryption` | 设置密文变量 |
| `Basic_IsNull` | 判断数据是否为空 |
| `Basic_DebugOutput` | 调试输出 |
| `Basic_BusinessOrderCount` | 业务办理计数 |
| `Basic_ConvertNumber` | 数据转为数值类型 |
| `Basic_GetProcessInfo` | 获取流程信息 |
| `Basic_ConvertString` | 将数字转换为文本 |
| `Basic_Delay` | 流程延迟执行 |
| `Basic_IsArray` | 判断数据是否为数组 |
| `Basic_Clone` | 复制数据 |
| `Basic_JudgeDataType` | 判断数据类型 |
| `Basic_DataTypeConversion` | 转换数据类型 |
| `Bivartable_CreateDfV1` | 创建数据表格 |
| `Bivartable_GetMaxValue` | 获取最大值 |
| `Bivartable_DelDfColRow` | 删除行/列 |
| `Bivartable_ReadDfData_V1` | 读取数据 |
| `Bivartable_GetMinValue` | 获取最小值 |
| `Bivartable_HandleDfMissValue` | 缺失值处理 |
| `Bivartable_ColumnOperationDf` | 两列运算 |
| `Bivartable_SortDf` | 排序数据表格 |
| `Bivartable_ReadTable` | 读取数据表内容 |
| `Bivartable_DoCreate` | 创建二维表 |
| `Bivartable_WriteDfData` | 写入数据 |
| `Bivartable_FilterDf` | 筛选数据表格 |
| `Bivartable_SummaryCalculationDf` | 按列汇总计算 |
| `Bivartable_DoSort` | 排序 |
| `Bivartable_DoFilter` | 表格过滤 |
| `Bivartable_InsertDfData` | 插入数据 |
| `Bivartable_ReadDfIndexColumns` | 读取行名/列名 |
| `Bivartable_ReadDfShape` | 读取行数/列数 |
| `Bivartable_DfWriteToFile` | 写入到文件 |
| `Bivartable_DoSave` | 保存二维表至Excel |
| `Bivartable_PivotDf` | 生成数据透视表 |
| `Bivartable_GetSumValue` | 获取统计总和 |
| `Bivartable_GetAvgValue` | 获取平均值 |
| `Bivartable_SpliceColData` | 拼接两列数据 |
| `Bivartable_Dtype` | 列数据类型转换 |
| `Bivartable_ModifyDfName` | 修改行名/列名 |
| `Bivartable_ModifyColName` | 修改列名 |
| `Bivartable_DeleteDuplicateRow` | 删除重复行 |
| `Bivartable_MergeDf` | 合并数据表格 |
| `Bivartable_DoOpenExcel` | 基于Excel创建二维表 |
| `Bivartable_DoMerge` | 合并二维表 |
| `Bivartable_WriteTable` | 数据表写入内容 |
| `Bivartable_DoJoint` | 拼接列 |
| `Bivartable_DoCal` | 列计算 |
| `Clipboard_SetImage` | 设置剪切板图片 |
| `Clipboard_SetTextV1` | 将文本添加到剪切板 |
| `Clipboard_GetText` | 取剪切板文本 |
| `Clipboard_SaveImageV1` | 获取剪切板文件 |
| `Clipboard_GetTextV1` | 获取剪切板文本 |
| `Clipboard_ClearClipboard` | 清空剪切板 |
| `Clipboard_SetImageV1` | 将文件添加到剪切板 |
| `Clipboard_SetText` | 设置剪切板文本 |
| `Clipboard_SaveImage` | 取剪切板图片 |
| `Collection_Create` | 创建集合 |
| `Collection_HasItem` | 判断集合元素存在 |
| `Collection_Count` | 取集合大小 |
| `Collection_Remove` | 删除集合元素 |
| `Collection_Add` | 添加集合元素 |
| `CSV_ReadNew` | 读取内容 |
| `CSV_WriteNew` | 写入内容 |
| `CustomConfig_GetConfig` | 自定义配置 |
| `CV_GetText` | 获取文本 |
| `CV_SetText` | 文本输入 |
| `Element_SetValue_V1` | 文本输入 |
| `Element_SetValueGround` | 文本输入 |
| `CV_MouseHover` | 移动到元素上 |
| `CV_MouseClick` | 元素点击 |
| `Element_Click_V1` | 元素点击 |
| `Element_ClickGround` | 元素点击 |
| `Database_BatchSQLExecute` | 批量执行SQL语句 |
| `Database_DBConnect` | 连接数据库 |
| `Database_SingleSQLQuery` | 执行单SQL查询 |
| `Database_BatchInsertExecute` | SQL批量插入数据 |
| `Database_AllSQLQuery_V1` | 执行全SQL查询 |
| `Database_CloseDBConnect` | 关闭连接 |
| `Mail_RecvClose` | 关闭连接 |
| `Database_SQLExecute` | 执行SQL语句 |
| `Databook_WriteContent` | 写入内容至数据表 |
| `Databook_DeleteColumn` | 删除数据表格列 |
| `Databook_Export` | 导出数据表格 |
| `Databook_GetColumnInfo` | 获取列信息 |
| `Databook_ReadContent` | 读取数据表格内容 |
| `Databook_Clear` | 清空数据表格 |
| `Databook_SetColumnInfo` | 设置列描述 |
| `Databook_Import` | 导入数据表格 |
| `Databook_GetRowAndColCount` | 读取数据表格总行列数 |
| `Databook_DeleteRow` | 删除数据表格行 |
| `DataService_Correct` | 文本纠错 |
| `DataService_AgentCall` | 调用智能体 |
| `DataService_AIGenTxt` | AI生成文本 |
| `DataService_WorkingDay` | 判断是否工作日 |
| `DataService_Diff` | 文本比对 |
| `DataService_LargeModel` | 大模型 |
| `DataService_Review` | 文本审核 |
| `DataService_ChatBot` | chatbot对话机器人 |
| `DataService_KeyWord` | 关键词抽取 |
| `DataService_ZBrain` | 自定义AI模型 |
| `Dialog_SaveFile` | 保存文件对话框 |
| `Dialog_CsInputBox` | 输入对话框 |
| `Dialog_GetConfig` | 输入输出对话框 |
| `Dialog_GetConfigV7` | 人机交互表单 |
| `Dialog_Notice_V2` | 消息通知 |
| `Dialog_OpenFile` | 打开文件对话框 |
| `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_OpenFiles` | 打开文件对话框(多选) |
| `Dialog_CsMsgBox` | 消息确认对话框 |
| `Dict_Create` | 创建字典 |
| `Dict_Delete` | 删除键值对 |
| `Dict_Items` | 获取字典所有键值 |
| `Dict_Values` | 获取字典所有值 |
| `Dict_Clear` | 清空字典 |
| `Dict_GetItem` | 获取字典值 |
| `Dict_Keys` | 获取字典所有键 |
| `Dict_Update` | 合并字典 |
| `Dict_SetItem` | 设置字典值 |
| `Element_SetValuePassword_V1` | 输入密码框 |
| `Element_Region2_v3` | 获取元素坐标 |
| `Element_RegionGround` | 获取元素坐标 |
| `Element_SelectOperation_v3` | 下拉框操作 |
| `Element_Sibling_V2` | 获取兄弟元素 |
| `Element_Children_V2` | 获取子元素 |
| `Element_ChildrenGround` | 获取子元素 |
| `Element_ElementAarray` | 获取相似元素数组 |
| `Element_Exists` | 元素判断 |
| `Element_ExistsGround` | 元素判断 |
| `Element_GetCheckGround` | 获取勾选框状态值 |
| `Element_GetCheck` | 获取勾选框状态值 |
| `Element_SetAttribute_v2` | 设置网页元素属性 |
| `Element_SetAttributeGround` | 设置网页元素属性 |
| `Element_WaitElement_v5` | 等待元素 |
| `Element_WaitElementGround` | 等待元素 |
| `Element_ScrollIntoView_v2` | 滚动页面 |
| `Element_ScrollIntoViewGround` | 滚动页面 |
| `Element_MouseTo_V4` | 鼠标移动到元素上 |
| `Element_MouseToGround` | 鼠标移动到元素上 |
| `Element_Click_V6` | 点击界面元素 |
| `Element_SetCheckGround` | 设置勾选框状态 |
| `Element_SetCheck` | 设置勾选框状态 |
| `Element_Parent_V2` | 获取父元素 |
| `Element_SetValue_V3` | 输入文本框 |
| `Element_StopAutoClosePopup` | 停止智能弹窗侦测 |
| `Element_GetButtonEnable_v4` | 判断元素是否可点击 |
| `Element_SetValue_V6` | 输入文本 |
| `Element_GetText_V5` | 获取界面元素文本 |
| `Element_GetValue` | 获取元素文本 |
| `Element_GetValueGround` | 获取元素文本 |
| `Element_SimilarElementList_V2` | 获取相似元素列表 |
| `Element_Shot_V2` | 元素截图 |
| `Element_ShotGround` | 元素截图 |
| `Element_CheckBoxOperation_v3` | 勾选框操作 |
| `Element_GetLineBarChartData` | 获取折线图/柱状图数据 |
| `Element_GetAttribute_v3` | 获取网页元素属性 |
| `Element_GetAttributeGround` | 获取网页元素属性 |
| `Element_GetSelect` | 获取下拉框选项值 |
| `Element_GetSelectGround` | 获取下拉框选项值 |
| `Element_SetValuePassword_V6` | 输入密码 |
| `Element_GetText_V1` | 获取区域文字 |
| `Element_GetElementNew` | 获取元素对象 |
| `Element_GetElementInfo_V1` | 获取元素信息 |
| `Element_ExistsV1` | 判断元素存在 |
| `Element_GetPieChartData` | 获取饼状图数据 |
| `Element_GetTableData` | 获取SAP表格数据 |
| `Element_StartAutoClosePopup` | 启动智能弹窗侦测 |
| `Element_SetSelectGround` | 设置下拉框选择项 |
| `Element_SetSelect` | 设置下拉框选择项 |
| `Element_GetHtml` | 获取网页元素源码 |
| `Element_GetButtonEnableGround` | 元素是否可点击 |
| `Element_FileUploader` | 上传文件 |
| `WebBrowser_Upload` | 上传文件 |
| `Excel_WriteColumnNew` | 写入列 |
| `Excel_Cs_SetCellFormat_V3` | 设置单元格格式 |
| `Excel_Cs_ReadContentV4` | 读取Excel内容 |
| `Excel_Cs_AutoFillV2` | 自动填充 |
| `Excel_NewExcel` | 新建excel |
| `Excel_Cs_CopyV2` | 复制Sheet |
| `Excel_WriteRangeNew` | 写入区域 |
| `Excel_Cs_DeleteColumn` | 删除列 |
| `Excel_Cs_OpenV4` | 打开Excel |
| `Excel_PasteRange` | 粘贴区域 |
| `Excel_OpenExcelNew` | 打开excel |
| `Excel_CopyRange` | 复制区域 |
| `Excel_Cs_ClearRangeV4` | 清除区域 |
| `Excel_Cs_InsertColumnV3` | 插入列 |
| `Excel_Cs_ReadRowCount` | 读取行数 |
| `Excel_Cs_ReadColumnCount` | 读取列数 |
| `Excel_Cs_Group` | 组合行列 |
| `Excel_ReadCellNew` | 读取单元格 |
| `Excel_WriteCellNew` | 写入单元格 |
| `Excel_Cs_SetRangeFontV2` | 设置区域字体颜色 |
| `Excel_Cs_DeletePictureV2` | 删除图片 |
| `Excel_Cs_NewV5` | 新建Excel |
| `Excel_GetColDataNew` | 读取列 |
| `Excel_GetSheetsName` | 获取sheet名 |
| `Excel_Cs_GetEmptyCell` | 获取第一个空白单元格 |
| `Excel_Cs_GetEditRange` | 获取已编辑区域 |
| `Excel_Cs_InsertRowV3` | 插入行 |
| `Excel_Cs_RefreshPivotTable` | 刷新Excel数据透视表 |
| `Excel_Cs_SetHeightV1` | 设置行高 |
| `Excel_Cs_Encrypt` | 加密Excel |
| `Excel_Cs_Move` | 移动Sheet |
| `Excel_Cs_DeleteDuplicates` | 删除重复 |
| `Excel_Cs_DeleteRangeV3` | 删除区域 |
| `Excel_Cs_ExcelFilter` | Excel数据筛选 |
| `Excel_Cs_GetPath` | 获取Excel路径 |
| `Excel_Cs_DeleteRow` | 删除行 |
| `Excel_Cs_Active` | 激活Sheet |
| `Excel_ReadRangeNew` | 读取区域 |
| `Excel_Cs_ConvertColumn` | 列名转换 |
| `Excel_Cs_InsertPictureV2` | 插入图片 |
| `Excel_Cs_Insert` | 插入Sheet |
| `Excel_Cs_Merge` | 合并Sheet |
| `Excel_Cs_ReadCellFormula` | 获取单元格公式 |
| `Excel_Save` | 保存excel |
| `Excel_Cs_InsertPivotView` | 新建透视图 |
| `Excel_Cs_Filter` | 数据筛选 |
| `Excel_SetCellFontColor` | 单元格字体颜色 |
| `Excel_Cs_CopyRangeV3` | 复制表格内容 |
| `Excel_Cs_SetWidthV1` | 设置列宽 |
| `Excel_SetRangeColor` | 区域颜色 |
| `Excel_Cs_Interop` | Excel动态调用 |
| `Excel_Cs_InsertPivotTable` | 新建Excel数据透视表 |
| `Excel_Cs_RunMacroV2` | 执行宏 |
| `Excel_Cs_FindV2` | Excel查找/替换 |
| `Excel_Cs_SaveAs` | 另存为Excel |
| `Excel_Loop` | excel内容循环 |
| `Excel_Cs_GetLastEmptyCell` | 获取尾部第一个空白单元格 |
| `Excel_WriteRowNew` | 写入行 |
| `Excel_GetRowDataNew` | 读取行 |
| `Excel_SetRangeFontColor` | 区域字体颜色 |
| `Excel_GetColumnId` | 获取Excel列标识 |
| `Excel_Cs_SetCellFont` | 设置单元格字体颜色 |
| `Excel_Cs_Delete` | 删除Sheet |
| `Excel_ActiveSheet` | 激活sheet |
| `Excel_Cs_Export` | 导出Excel为PDF |
| `Excel_Cs_MergeCellsV2` | 合并单元格 |
| `Excel_Cs_ShowTableStyle` | 设置Excel表格样式 |
| `Excel_Cs_CloseV2` | 关闭Excel |
| `Excel_Cs_SelectRangeV3` | 选中区域 |
| `Excel_Cs_ClearExcelFilter` | 清除Excel数据筛选 |
| `Excel_Close` | 关闭excel |
| `Excel_Cs_Rename` | 重命名Sheet |
| `Excel_Cs_SetCellBackground` | 设置单元格背景颜色 |
| `Excel_Cs_Save` | 保存Excel |
| `Excel_SetCellColor` | 单元格背景颜色 |
| `Excel_Cs_GetName` | 获取Sheet名 |
| `Excel_Cs_PastRangeV2` | 粘贴表格内容 |
| `Excel_Cs_ColumnSplitV2` | Excel分列 |
| `Excel_Cs_SortV1` | Excel数据排序 |
| `Excel_Cs_ExcelAddin` | 数据处理 |
| `Excel_Cs_ReadCellBackground` | 获取单元格背景色 |
| `Excel_RenameSheet` | 重命名sheet |
| `Excel_Cs_FilterPivotTable` | 筛选Excel数据透视表 |
| `Excel_SaveAs` | 另存excel |
| `Excel_Cs_Hide` | 隐藏/显示行列 |
| `Excel_Cs_SetSlicer` | Excel切片器 |
| `Excel_Cs_WriteContent` | 写入Excel内容 |
| `Excel_Cs_EnableExcelFilter` | 取消Excel数据筛选 |
| `Excel_Cs_SetRangeBackgroundV2` | 设置区域背景颜色 |
| `Excel_Cs_LoadV2` | 获取已打开Excel |
| `Excel_Cs_RefreshAll` | 刷新Excel数据 |
| `Excel_CopySheet` | 复制sheet |
| `Excel_CreateSheet` | 创建sheet |
| `Excel_ExportRangeAsPicture` | 导出区域为图片 |
| `Excel_Cs_Search` | 查找/替换数据 |
| `Excel_DeleteSheet` | 删除sheet |
| `ExceptionUtil_Throws` | 抛出异常 |
| `File_Rename` | 重命名文件 |
| `File_Delete` | 删除文件 |
| `File_DecompressFile` | 解压文件 |
| `File_GetFileList_V1` | 获取文件列表 |
| `File_File_Move` | 移动文件 |
| `File_MoveFolder` | 移动文件夹 |
| `File_Write_V1` | 写入文件 |
| `File_PathSize` | 取文件大小 |
| `File_MakeDir` | 创建文件夹 |
| `File_Copy_V1` | 复制文件/文件夹 |
| `File_GetFileInfo` | 获取文件信息 |
| `File_GetFolderList_V1` | 获取文件夹列表 |
| `File_ReadXml` | 读取xml文件 |
| `File_OpenFile` | 打开文件夹 |
| `File_Read` | 读取文件 |
| `File_Rename_V1` | 重命名文件/文件夹 |
| `File_CreateFile` | 新建文件 |
| `File_DeleteFile` | 删除文件/文件夹 |
| `File_PathSizeNew` | 获取文件大小 |
| `File_CreateFolder` | 生成上传路径 |
| `File_Copy` | 复制文件 |
| `File_CompressFile` | 压缩文件 |
| `File_FileExists` | 判断文件存在 |
| `File_PathExists` | 判断路径 |
| `File_Move` | 移动文件/文件夹 |
| `File_EmptyFolder` | 清空文件夹 |
| `File_WaitFile` | 等待文件 |
| `FileTypeConversion_Office2pdf` | OFFICE文档转PDF |
| `FileTypeConversion_Pdf2image2` | PDF转图片 |
| `FTP_Download` | 下载文件/文件夹(FTP) |
| `FTP_Rename` | 重命名文件/文件夹(FTP) |
| `FTP_Delete` | 删除文件/文件夹(FTP) |
| `FTP_Disconnect` | 断开FTP连接 |
| `FTP_Cwd` | 切换工作目录 |
| `FTP_GetFileList` | 获取文件/文件夹(FTP) |
| `FTP_CreateDir` | 创建文件夹(FTP) |
| `FTP_Exist` | 判断文件存在(FTP) |
| `FTP_Upload` | 上传文件/文件夹(FTP) |
| `FTP_Connect` | 建立FTP连接 |
| `GoogleTable_GetAllData` | 获取列信息(Google Sheets) |
| `GoogleTable_GetAllSheetsName` | 获取Sheet名(Google Sheets) |
| `GoogleTable_WriteToSheetRegionColumns` | 写入列(Google Sheets) |
| `GoogleTable_WriteToSheetRegionRows` | 写入行(Google Sheets) |
| `GoogleTable_WriteToSheetCell` | 写入单元格(Google Sheets) |
| `GoogleTable_WriteToSheetCellFormula` | 写入单元格公式(Google Sheets) |
| `GoogleTable_GetSheetRowCount` | 获取行数(Google Sheets) |
| `GoogleTable_DownloadSheets` | 导出Sheet(Google Sheets) |
| `GoogleTable_CopyPasteSheet` | 复制粘贴(Google Sheets) |
| `GoogleTable_GetColumnsByTableTitle` | 按表头读取列(Google Sheets) |
| `GoogleTable_ChangeSheet` | 激活Sheet(Google Sheets) |
| `GoogleTable_RenameSheet` | 重命名Sheet(Google Sheets) |
| `GoogleTable_WriteToSheetRegionregion` | 写入区域(Google Sheets) |
| `Http_Post_V3` | POST请求 |
| `Http_Get_V2` | GET请求 |
| `Http_Http_Request` | HTTP 请求 |
| `Image_Click_V2` | 点击图像 |
| `Image_Move_V2` | 鼠标移动到图像上 |
| `Image_Screenshot` | 屏幕截图 |
| `Image_Base64Decode` | 图片base64编码转二进制 |
| `Image_Find_V1` | 查找图像 |
| `Image_Base64Encode` | 图片转base64编码 |
| `Image_Wait_V1` | 等待图像 |
| `Image_Exists_V1` | 判断图像 |
| `Image_Location` | 图像位置 |
| `Image_Location_V1` | 获取图像位置 |
| `Image_CropImg` | 截取图像 |
| `Image_Click_V3` | 鼠标点击图像 |
| `Image_RelativePosition` | 对比图像相对位置 |
| `Image_Move_V1` | 移动到图像 |
| `Ini_DeleteKey` | 删除INI文件键 |
| `Ini_Write` | 写入INI文件键值 |
| `Ini_ListSection` | 枚举INI文件小节 |
| `Ini_DeleteSection` | 删除INI文件小节 |
| `Ini_ListKey` | 枚举INI文件键 |
| `Ini_Read` | 读取INI文件键值 |
| `JSON_Stringify` | 将JSON转换为文本 |
| `JSON_Parse` | 将文本转换为JSON |
| `Keyboard_PressNew` | 键盘模拟按键 |
| `Keyboard_DriveSend_V2` | 键盘驱动输入 |
| `Keyboard_Press` | 键盘按键 |
| `Keyboard_Send_V1` | 键盘输入 |
| `Keyboard_EnableManualOperation` | 开启模拟人工操作 |
| `Keyboard_DisableManualOperation` | 关闭模拟人工操作 |
| `List_DeleteList` | 删除列表项 |
| `List_GetIndex` | 获取列表项的位置 |
| `List_ListMerge` | 列表合并 |
| `List_ClearList` | 清空列表 |
| `List_ListLength` | 获取列表长度 |
| `List_ListSorted` | 列表排序 |
| `List_ListDuplication` | 列表去重 |
| `List_InsertList` | 新增列表项 |
| `List_ListIntersection` | 获取两列表的重复项 |
| `List_CreateList` | 创建列表 |
| `List_GetListItem` | 获取指定位置项的数据 |
| `List_SetListItem` | 修改指定位置项的数据 |
| `Log_Error` | 打印错误日志 |
| `Log_Info` | 打印普通日志 |
| `Mail_UnmarkSeen` | 标为未读 |
| `Mail_DoReplyMail_v1` | 回复邮件(Outlook) |
| `Mail_RecvConnect` | 连接邮箱 |
| `Mail_RecvMailGet` | 取第几封邮件 |
| `Mail_ExOperateMail` | 操作邮件 |
| `Mail_OperateMail` | 操作邮件 |
| `Mail_RecvGetList` | 邮件筛选 |
| `Mail_ReplyMail_v2` | 回复邮件 |
| `Mail_DoReplyMail` | 回复邮件 |
| `Mail_Send_V1` | 发送邮件 |
| `Mail_DoSendMail` | 发送邮件 |
| `Mail_ExSendMail` | 发送邮件 |
| `Mail_FindMail_v1` | 查找邮件 |
| `Mail_ExSearchMail` | 查找邮件 |
| `Mail_DoMoveMail` | 移动邮件 |
| `Mail_ExConnect` | 连接Exchange邮箱 |
| `Mail_RecvCount` | 取收件箱邮件数 |
| `Mail_DoGetMailMessage_v2` | 获取邮件(Outlook) |
| `Mail_DoGetMailMessage_v1` | 查找邮件(Outlook) |
| `Mail_FindMail_v2` | 获取邮件 |
| `Mail_DoGetMailMessage` | 查看邮件 |
| `Mail_MarkSeen` | 标为已读 |
| `Mail_DoMoveMail_v1` | 操作邮件(Outlook) |
| `Mail_DoSendMail_v1` | 发送邮件(Outlook) |
| `Math_Cos` | 余弦值 |
| `Math_Tan` | 正切值 |
| `Math_Scale` | 进制转换 |
| `Math_Round` | 四舍五入 |
| `Math_Pow` | x的y次幂 |
| `Math_Int` | 取整 |
| `Math_Sqrt` | 平方根 |
| `Math_Oct` | 转8进制 |
| `Math_Sin` | 正弦值 |
| `Math_Abs` | 绝对值 |
| `Math_Hex` | 转16进制 |
| `Math_Exp` | e的n次幂 |
| `Math_Sgn` | 取正负符号 |
| `MessageNotification_Feishu_Message` | 发送消息至飞书 |
| `MessageNotification_DingTalk_Message` | 发送消息至钉钉 |
| `MessageNotification_WXWork_Message` | 发送消息至企业微信 |
| `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_DragToNew` | 鼠标拖拽 |
| `Mouse_RollTo` | 鼠标滚动 |
| `Mouse_MoveToV2` | 移动鼠标 |
| `NLP_BatchAnalyseProductReview` | 商品评论分析(批量) |
| `NLP_KeyWordsAnalysis` | 关键词 |
| `NLP_EmotionalAnalysis` | 情感分析 |
| `NLP_SplitText` | 实体抽取 |
| `NLP_AnalyseProductReview` | 商品评论分析 |
| `NLP_SemanticSimilarity` | 语义相似度 |
| `NLP_SplitWords` | 分词 |
| `NLP_PerspectiveAnalysis` | 观点分析 |
| `NLP_XunFei_NLPBaseServer` | 第三方NLP模型 |
| `NLP_XunFei_MachineTranslation` | 机器翻译 |
| `NLP_XunFei_FeelingAnalysis` | 第三方情感分析模型 |
| `NLP_XunFei_KeywordExtraction` | 关键字提取 |
| `OCR_MergeCardOCR` | 卡证文字识别 |
| `OCR_CalculationCode` | 算术验证码 |
| `OCR_IdCardV2` | OCR身份证 |
| `OCR_BusinessCardV2` | OCR名片 |
| `OCR_VehicleInvoiceV2` | OCR机动车发票 |
| `OCR_MergeOtherOCR` | 其他场景文字识别 |
| `OCR_BusinessLicenseV3` | OCR营业执照(自研) |
| `OCR_PassportV2` | OCR护照 |
| `OCR_MergeGeneralOCR` | 通用文字识别 |
| `OCR_KeyValueIdentification` | 识别键值对结构 |
| `OCR_AmountIdentification` | 识别价格金额 |
| `OCR_GeneralTextV2` | OCR高精度文本 |
| `OCR_VatInvoiceV2` | OCR增值税发票 |
| `OCR_DrivingLicenseV2` | OCR行驶证 |
| `OCR_AccountPageV2` | OCR户口页 |
| `OCR_PlateLicenseV2` | OCR车牌 |
| `OCR_BankCardV2` | OCR银行卡 |
| `OCR_BankCardV3` | OCR银行卡(自研) |
| `OCR_IdCardV3` | OCR身份证(自研) |
| `OCR_VatInvoiceV3` | OCR增值税发票(自研) |
| `OCR_MergePaperOCR` | 票据文字识别 |
| `OCR_DriverLicenseV2` | OCR驾驶证 |
| `OCR_RealEstateCertV2` | OCR不动产权证 |
| `OCR_TrainTicketV2` | OCR火车票 |
| `OCR_TaxiInvoiceV2` | OCR出租车发票 |
| `OCR_OfficialSealV2` | OCR公章 |
| `OCR_MergeCarOCR` | 汽车场景文字识别 |
| `OCR_MergeEducationOCR` | 教育场景文字识别 |
| `OCR_HouseCertV2` | OCR房产证 |
| `OCR_CommonEnglishNumbersCode` | 普通验证码 |
| `OCR_BusinessLicenseV2` | OCR营业执照 |
| `OCR_VinV2` | OCR车辆vin码 |
| `OcrOffline_GetOcrTextNew` | 图像OCR识别 |
| `OcrOffline_GetOcrTable` | 获取表格数据(OCR) |
| `OcrOffline_GetOcrTextPosition` | 图像OCR识别(含坐标) |
| `PDF_GetPdfInformation` | 获取 PDF 文档信息 |
| `PDF_SetPdfWaterMarkV1` | PDF 加水印 |
| `PDF_PdfEncryption` | 加密 PDF 文档 |
| `PDF_GetPdfImg` | 提取 PDF 图片 |
| `PDF_GetPdfPageNum` | 获取 PDF 总页数 |
| `PDF_ConcatPdfV1` | 合并 PDF 文档 |
| `PDF_GetPdfTable` | 读取 PDF 表格 |
| `PDF_PdfToImg` | PDF 转图片 |
| `PDF_CreateNewPdf` | 生成新的 PDF 文档 |
| `PDF_GetPdfText` | 读取 PDF 文本 |
| `Regex_Test` | 匹配判断 |
| `Regex_FindAll` | 全部查找 |
| `Regex_FindNew_1` | 文本内容提取(指定) |
| `Regex_FindStr` | 文本内容提取 |
| `Regex_FindNew` | 指定查找 |
| `Regex_FindAllNew` | 文本内容提取(全部) |
| `Regex_TestNew` | 判断文本匹配 |
| `Regex_FindStrV1` | 提取文本内容 |
| `Scenario_TextComparison` | 文档比对 |
| `ScreenControl_Screenlock` | 屏幕锁屏 |
| `ScreenControl_ScreenUnlock` | 唤醒屏幕(非登录解锁) |
| `StrUtil_InStr` | 查找 |
| `StrUtil_ListToString` | 列表聚合成文本 |
| `StrUtil_CharCode` | 将ASCII码转换为文本 |
| `StrUtil_CutOutStrV1` | 截取文本 |
| `StrUtil_CreateRandomStr` | 创建随机文本 |
| `StrUtil_SplitStr` | 分割文本为列表 |
| `StrUtil_Split` | 分割字符串 |
| `StrUtil_CaseChange` | 转换大小写 |
| `StrUtil_AddToStr` | 追加文本 |
| `StrUtil_Rtrim` | 裁剪右侧空格 |
| `StrUtil_DelTrim` | 去除文本两端空格 |
| `StrUtil_Trim` | 裁剪两侧空格 |
| `StrUtil_LowerCase` | 字母转小写 |
| `StrUtil_LookUpStr` | 查找文本 |
| `StrUtil_ReverseStr` | 反转文本 |
| `StrUtil_UpperCase` | 字母转大写 |
| `StrUtil_Mid` | 取中间内容 |
| `StrUtil_CharAsc` | 将文本转换为ASCII码 |
| `StrUtil_CompleteStr` | 补齐文本 |
| `StrUtil_Replace` | 替换文本 |
| `StrUtil_Left` | 取左侧内容 |
| `StrUtil_Ltrim` | 裁剪左侧空格 |
| `StrUtil_Len` | 获取文本长度 |
| `StrUtil_Right` | 取右侧内容 |
| `System_GetEnv` | 获取环境变量 |
| `System_ReadCredential` | 获取Windows凭据 |
| `System_GetTempPath` | 获取临时文件夹路径 |
| `System_GetHomePath` | 获取用户文件夹路径 |
| `System_GetSystemPath` | 获取系统文件夹路径 |
| `System_SetEnv` | 设置环境变量 |
| `System_GetInputMethodState` | 获取输入法状态 |
| `System_ConnectRemoteDesktop` | 连接远程桌面 |
| `System_AddCredential` | 添加Windows凭据 |
| `System_SetInputMethodState` | 设置输入法状态 |
| `System_CloseRemoteDesktop` | 关闭远程连接 |
| `System_ScreenSaverShowV1` | 开启屏保 |
| `System_ExecuteCommandV1` | 执行命令行 |
| `System_DeleteCredential` | 删除Windows凭据 |
| `System_ScreenSaverStop` | 关闭屏保 |
| `System_PlaySound` | 播放音频 |
| `Time_GetDateV2` | 获取日期时间 |
| `Time_CalDate` | 计算时长 |
| `Time_WeekDay` | 取星期 |
| `Time_Day` | 取第几天 |
| `Time_Date` | 取日期 |
| `Time_FormatNew` | 将时间戳转换为日期/时间 |
| `Time_Hour` | 取时 |
| `Time_DateOffset_V1` | 增加/减少时间 |
| `Time_GetDatetimeObj` | 获取日期时间信息 |
| `Time_Format` | 时间戳转时间 |
| `Time_DatetimeToTimestamp` | 时间转时间戳 |
| `Time_TimeStamp` | 取时间戳 |
| `Time_TimeTypeConversion_V1` | 转换日期时间格式 |
| `Time_DateToTimestamp` | 将日期/时间转换为时间戳 |
| `Time_Month` | 取月份 |
| `Time_Time` | 取时分秒 |
| `Time_Year` | 取年份 |
| `Time_TimeTypeConversion` | 日期时间格式转换 |
| `Time_Now` | 取时间 |
| `Time_Second` | 取秒 |
| `Time_Minute` | 取分 |
| `WebBrowser_Switch_V1` | 切换页面 |
| `WebBrowser_SetScrollPos_V1` | 设置滚动条位置 |
| `WebBrowser_GetTAB_V2` | 获取网页 Tab 对象 |
| `WebBrowser_SaveAs_V1` | 下载文件(点击按钮) |
| `WebBrowser_GetWeblistening` | 获取监听结果 |
| `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_RunJsInIeBySelector_v3` | 执行JS |
| `WebBrowser_RunJsInIeBySelectorGround` | 执行JS |
| `WebBrowser_CreateV3` | 打开网页 |
| `WebBrowser_Reload` | 刷新 |
| `WebBrowser_CloseWeb_V2` | 关闭网页 |
| `WebBrowser_Stop` | 停止网页加载 |
| `WebBrowser_GetDocument` | 获取网页源码 |
| `WebBrowser_GetCookie` | 获取Cookies(指定) |
| `WebBrowser_DownFile_V1` | 下载文件(指定链接) |
| `WebBrowser_DownloadFile_V1` | 下载文件 |
| `WebBrowser_GetCookieNew` | 获取完整Cookies |
| `WebBrowser_GetBrowserPicV1` | 浏览器截图 |
| `WebBrowser_GetTAB` | 获取Tab页 |
| `WebBrowser_Close_V1` | 关闭浏览器 |
| `WebBrowser_Maximize` | 最大化浏览器 |
| `WebBrowser_GoForward` | 前进 |
| `WebBrowser_SetCookies` | 设置Cookies |
| `WebBrowser_StartWeblistening` | 启动网页监听 |
| `WebBrowser_Wait` | 等待网页加载 |
| `WebBrowser_UploadDialog` | 处理上传对话框 |
| `WebBrowser_DataCrawler` | 爬取数据 |
| `WebBrowser_StopWeblistening` | 停止网页监听 |
| `WebBrowser_SetCookies_V1` | 设置Cookie |
| `WebBrowser_GetTitle` | 获取Title |
| `WebBrowser_GetScrollPos_V1` | 获取滚动条位置 |
| `WebBrowser_GoBack` | 后退 |
| `WebBrowser_GoUrl` | 跳转网页 |
| `WebBrowser_BatchDataCrawl` | 批量数据采集(beta) |
| `WebBrowser_GetUrl` | 获取URL |
| `WebBrowser_GetCookies` | 获取Cookies |
| `WebBrowser_Minimize` | 最小化浏览器 |
| `Window_SizeWindow_v1` | 调整窗口大小 |
| `Window_SetWindowState_v1` | 设置窗口状态 |
| `Window_GetWindowRect` | 获取窗口范围 |
| `Window_CloseWindow` | 关闭软件窗口 |
| `Window_GetActive` | 获取活动窗口 |
| `Window_ActiveWindow_v1` | 激活窗口 |
| `Window_WindowExists` | 判断窗口存在 |
| `Window_Get` | 获取窗口 |
| `Window_SetPos` | 设置窗口 |
| `Window_GetWindow_v1` | 获取窗口对象 |
| `Window_CloseWindow_v1` | 关闭窗口 |
| `Window_GetWindowMessage_v1` | 获取窗口信息 |
| `Window_MoveWindow_v1` | 移动窗口位置 |
| `Window_ActiveWindow` | 激活软件窗口 |
| `Window_GetScreenSize` | 获取屏幕尺寸 |
| `Window_SetWindowShow` | 设置窗口是否显示 |
| `Window_WindowList` | 获取窗口对象列表 |
| `Window_GetRect` | 窗口属性 |
| `Window_MaximumSize` | 窗口最大化 |
| `Window_WaitWindow` | 等待窗口 |
| `Word_ReadSelect` | 读取选中文本 |
| `Word_SaveAs` | Word另存为 |
| `Word_SetFontName` | 设置字体 |
| `Word_CutSelect` | 剪切 |
| `Word_ReadAll` | 读取Word文本 |
| `Word_InsertPicture` | 插入Word图片 |
| `Word_SetFontStyle` | 设置字体样式 |
| `Word_New` | 新建word |
| `Word_SetAlign` | 设置对齐方式 |
| `Word_Save` | 保存Word |
| `Word_ReadTable` | 读取Word表格 |
| `Word_New_V1` | 新建Word文档 |
| `Word_Open` | 打开Word文档 |
| `Word_SelectRow` | 选择行 |
| `Word_GetPath` | 获取Word路径 |
| `Word_DeleteSelect` | 删除 |
| `Word_SetPosition` | 设置光标位置 |
| `Word_Write` | 写入文字至Word |
| `Word_PositionToSelect` | 查找文本后设置光标位置 |
| `Word_Close` | 关闭Word |
| `Word_MovePosition` | 移动光标位置 |
| `Word_SetFontColor` | 设置字体颜色 |
| `Word_CopySelect` | 复制 |
| `Word_PasteSelect` | 粘贴 |
| `Word_Replace` | 替换 |
| `Word_WriteTable` | 插入Word表格 |
| `Word_Enter` | 回车 |
| `Word_SetFontSize` | 设置字体大小 |
| `Word_SelectAll` | 全选 |
| `Word_InsertPage` | 插入分页符 |
## 隐藏重复能力
| 隐藏 ID | 保留主能力 | 名称 |
|---|---|---|
| `Basic_SetVariable` | `Basic_SetVariable_V1` | 设置变量 |
| `App_Kill` | `App_KillV1` | 关闭应用程序 |
| `Basic_Delay_V1` | `Basic_Delay_V2` | 等待 |
| `Basic_Rnd` | `Basic_RandomInt` | 生成随机数 |
| `Bivartable_CreateDf` | `Bivartable_CreateDfV1` | 创建数据表格 |
| `Bivartable_ReadDfData` | `Bivartable_ReadDfData_V1` | 读取数据 |
| `CSV_Read` | `CSV_ReadNew` | 读取内容 |
| `CSV_Write` | `CSV_WriteNew` | 写入内容 |
| `Element_SetValue` | `Element_SetValue_V1` | 文本输入 |
| `Element_Click` | `Element_Click_V1` | 元素点击 |
| `Database_AllSQLQuery` | `Database_AllSQLQuery_V1` | 执行全SQL查询 |
| `Dialog_InputBox` | `Dialog_CsInputBox` | 输入对话框 |
| `Dialog_GetConfigV5` | `Dialog_GetConfigV7` | 人机交互表单 |
| `Dialog_Notice_V1` | `Dialog_Notice_V2` | 消息通知 |
| `Dialog_Notice` | `Dialog_Notice_V2` | 消息通知 |
| `Dialog_GetConfigV6` | `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_GetConfigV4` | `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_GetConfigV3` | `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_GetConfigV2` | `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_GetConfigV1` | `Dialog_GetConfigV8` | 自定义对话框 |
| `Dialog_MsgBox` | `Dialog_CsMsgBox` | 消息确认对话框 |
| `Element_SetValuePassword` | `Element_SetValuePassword_V1` | 输入密码框 |
| `Element_Region2_v2` | `Element_Region2_v3` | 获取元素坐标 |
| `Element_Region2_v1` | `Element_Region2_v3` | 获取元素坐标 |
| `Element_Region` | `Element_Region2_v3` | 获取元素坐标 |
| `Element_Region2` | `Element_Region2_v3` | 获取元素坐标 |
| `Element_SelectOperation_v2` | `Element_SelectOperation_v3` | 下拉框操作 |
| `Element_SelectOperation_v1` | `Element_SelectOperation_v3` | 下拉框操作 |
| `Element_SelectOperation` | `Element_SelectOperation_v3` | 下拉框操作 |
| `Element_Sibling_V1` | `Element_Sibling_V2` | 获取兄弟元素 |
| `Element_Sibling` | `Element_Sibling_V2` | 获取兄弟元素 |
| `Element_Children_V1` | `Element_Children_V2` | 获取子元素 |
| `Element_ChildrenV1` | `Element_Children_V2` | 获取子元素 |
| `Element_Children` | `Element_Children_V2` | 获取子元素 |
| `Element_DesktopChildren` | `Element_ChildrenGround` | 获取子元素 |
| `Element_ElementCaptureSimilar` | `Element_ElementAarray` | 获取相似元素数组 |
| `Element_SetAttribute_v1` | `Element_SetAttribute_v2` | 设置网页元素属性 |
| `Element_SetAttribute` | `Element_SetAttribute_v2` | 设置网页元素属性 |
| `Element_WaitElement_v4` | `Element_WaitElement_v5` | 等待元素 |
| `Element_WaitElement_v3` | `Element_WaitElement_v5` | 等待元素 |
| `Element_WaitElement_v2` | `Element_WaitElement_v5` | 等待元素 |
| `Element_WaitElement_v1` | `Element_WaitElement_v5` | 等待元素 |
| `Element_WaitElement` | `Element_WaitElement_v5` | 等待元素 |
| `Element_ScrollIntoView_v1` | `Element_ScrollIntoView_v2` | 滚动页面 |
| `Element_ScrollIntoView` | `Element_ScrollIntoView_v2` | 滚动页面 |
| `Element_MouseTo_V3` | `Element_MouseTo_V4` | 鼠标移动到元素上 |
| `Element_MouseTo_V2` | `Element_MouseTo_V4` | 鼠标移动到元素上 |
| `Element_MouseTo_V1` | `Element_MouseTo_V4` | 鼠标移动到元素上 |
| `Element_MouseTo` | `Element_MouseTo_V4` | 鼠标移动到元素上 |
| `Element_Click_V5` | `Element_Click_V6` | 点击界面元素 |
| `Element_Click_V4` | `Element_Click_V6` | 点击界面元素 |
| `Element_Click_V3` | `Element_Click_V6` | 点击界面元素 |
| `Element_Click_V2` | `Element_Click_V6` | 点击界面元素 |
| `Element_Parent_V1` | `Element_Parent_V2` | 获取父元素 |
| `Element_Parent` | `Element_Parent_V2` | 获取父元素 |
| `Element_SetValue_V2` | `Element_SetValue_V3` | 输入文本框 |
| `Element_GetButtonEnable_v3` | `Element_GetButtonEnable_v4` | 判断元素是否可点击 |
| `Element_GetButtonEnable_v2` | `Element_GetButtonEnable_v4` | 判断元素是否可点击 |
| `Element_GetButtonEnable_v1` | `Element_GetButtonEnable_v4` | 判断元素是否可点击 |
| `Element_GetButtonEnable` | `Element_GetButtonEnable_v4` | 判断元素是否可点击 |
| `Element_SetValue_V5` | `Element_SetValue_V6` | 输入文本 |
| `Element_SetValue_V4` | `Element_SetValue_V6` | 输入文本 |
| `Element_GetText_V4` | `Element_GetText_V5` | 获取界面元素文本 |
| `Element_GetText_V3` | `Element_GetText_V5` | 获取界面元素文本 |
| `Element_GetText_V2` | `Element_GetText_V5` | 获取界面元素文本 |
| `Element_GetText` | `Element_GetText_V5` | 获取界面元素文本 |
| `Element_SimilarElementList_V1` | `Element_SimilarElementList_V2` | 获取相似元素列表 |
| `Element_SimilarElementList` | `Element_SimilarElementList_V2` | 获取相似元素列表 |
| `Element_Shot_V1` | `Element_Shot_V2` | 元素截图 |
| `Element_Shot` | `Element_Shot_V2` | 元素截图 |
| `Element_CheckBoxOperation_v2` | `Element_CheckBoxOperation_v3` | 勾选框操作 |
| `Element_CheckBoxOperation_v1` | `Element_CheckBoxOperation_v3` | 勾选框操作 |
| `Element_CheckBoxOperation` | `Element_CheckBoxOperation_v3` | 勾选框操作 |
| `Element_GetAttribute_v2` | `Element_GetAttribute_v3` | 获取网页元素属性 |
| `Element_GetAttribute_v1` | `Element_GetAttribute_v3` | 获取网页元素属性 |
| `Element_GetAttribute` | `Element_GetAttribute_v3` | 获取网页元素属性 |
| `Element_SetValuePassword_V5` | `Element_SetValuePassword_V6` | 输入密码 |
| `Element_GetElementV2` | `Element_GetElementNew` | 获取元素对象 |
| `Element_GetElementV1` | `Element_GetElementNew` | 获取元素对象 |
| `Element_GetElement` | `Element_GetElementNew` | 获取元素对象 |
| `Element_GetElementInfo` | `Element_GetElementInfo_V1` | 获取元素信息 |
| `Excel_Cs_WriteColumn` | `Excel_WriteColumnNew` | 写入列 |
| `Excel_WriteColumn` | `Excel_WriteColumnNew` | 写入列 |
| `Excel_Cs_SetCellFormat_V2` | `Excel_Cs_SetCellFormat_V3` | 设置单元格格式 |
| `Excel_Cs_SetCellFormat` | `Excel_Cs_SetCellFormat_V3` | 设置单元格格式 |
| `Excel_Cs_ReadContentV3` | `Excel_Cs_ReadContentV4` | 读取Excel内容 |
| `Excel_Cs_ReadContentV2` | `Excel_Cs_ReadContentV4` | 读取Excel内容 |
| `Excel_Cs_ReadContent` | `Excel_Cs_ReadContentV4` | 读取Excel内容 |
| `Excel_Cs_AutoFill` | `Excel_Cs_AutoFillV2` | 自动填充 |
| `Excel_Cs_Copy` | `Excel_Cs_CopyV2` | 复制Sheet |
| `Excel_Cs_WriteRangeV2` | `Excel_WriteRangeNew` | 写入区域 |
| `Excel_Cs_WriteRange` | `Excel_WriteRangeNew` | 写入区域 |
| `Excel_WriteRange` | `Excel_WriteRangeNew` | 写入区域 |
| `Excel_DeleteColumn` | `Excel_Cs_DeleteColumn` | 删除列 |
| `Excel_Cs_OpenV3` | `Excel_Cs_OpenV4` | 打开Excel |
| `Excel_Cs_OpenV2` | `Excel_Cs_OpenV4` | 打开Excel |
| `Excel_Cs_Open` | `Excel_Cs_OpenV4` | 打开Excel |
| `Excel_OpenExcel` | `Excel_OpenExcelNew` | 打开excel |
| `Excel_Cs_ClearRangeV3` | `Excel_Cs_ClearRangeV4` | 清除区域 |
| `Excel_Cs_ClearRangeV2` | `Excel_Cs_ClearRangeV4` | 清除区域 |
| `Excel_ClearRange` | `Excel_Cs_ClearRangeV4` | 清除区域 |
| `Excel_Cs_ClearRange` | `Excel_Cs_ClearRangeV4` | 清除区域 |
| `Excel_Cs_InsertColumnV2` | `Excel_Cs_InsertColumnV3` | 插入列 |
| `Excel_Cs_InsertColumn` | `Excel_Cs_InsertColumnV3` | 插入列 |
| `Excel_InsertColumn` | `Excel_Cs_InsertColumnV3` | 插入列 |
| `Excel_GetRowsCount` | `Excel_Cs_ReadRowCount` | 读取行数 |
| `Excel_GetColumnsCount` | `Excel_Cs_ReadColumnCount` | 读取列数 |
| `Excel_Cs_ReadCell` | `Excel_ReadCellNew` | 读取单元格 |
| `Excel_ReadCell` | `Excel_ReadCellNew` | 读取单元格 |
| `Excel_Cs_WriteCellV2` | `Excel_WriteCellNew` | 写入单元格 |
| `Excel_Cs_WriteCell` | `Excel_WriteCellNew` | 写入单元格 |
| `Excel_WriteCell` | `Excel_WriteCellNew` | 写入单元格 |
| `Excel_Cs_SetRangeFont` | `Excel_Cs_SetRangeFontV2` | 设置区域字体颜色 |
| `Excel_Cs_DeletePicture` | `Excel_Cs_DeletePictureV2` | 删除图片 |
| `Excel_DeleteImage` | `Excel_Cs_DeletePictureV2` | 删除图片 |
| `Excel_Cs_New` | `Excel_Cs_NewV5` | 新建Excel |
| `Excel_Cs_NewV4` | `Excel_Cs_NewV5` | 新建Excel |
| `Excel_Cs_NewV3` | `Excel_Cs_NewV5` | 新建Excel |
| `Excel_Cs_NewV2` | `Excel_Cs_NewV5` | 新建Excel |
| `Excel_Cs_ReadColumn` | `Excel_GetColDataNew` | 读取列 |
| `Excel_GetColData` | `Excel_GetColDataNew` | 读取列 |
| `Excel_Cs_InsertRowV2` | `Excel_Cs_InsertRowV3` | 插入行 |
| `Excel_Cs_InsertRow` | `Excel_Cs_InsertRowV3` | 插入行 |
| `Excel_InsertRow` | `Excel_Cs_InsertRowV3` | 插入行 |
| `Excel_Cs_SetHeight` | `Excel_Cs_SetHeightV1` | 设置行高 |
| `Excel_SetRowHeight` | `Excel_Cs_SetHeightV1` | 设置行高 |
| `Excel_Cs_DeleteRangeV2` | `Excel_Cs_DeleteRangeV3` | 删除区域 |
| `Excel_Cs_DeleteRange` | `Excel_Cs_DeleteRangeV3` | 删除区域 |
| `Excel_DeleteRange` | `Excel_Cs_DeleteRangeV3` | 删除区域 |
| `Excel_GetPath` | `Excel_Cs_GetPath` | 获取Excel路径 |
| `Excel_DeleteRow` | `Excel_Cs_DeleteRow` | 删除行 |
| `Excel_Cs_ReadRangeV2` | `Excel_ReadRangeNew` | 读取区域 |
| `Excel_Cs_ReadRange` | `Excel_ReadRangeNew` | 读取区域 |
| `Excel_ReadRange` | `Excel_ReadRangeNew` | 读取区域 |
| `Excel_Cs_InsertPicture` | `Excel_Cs_InsertPictureV2` | 插入图片 |
| `Excel_InsertImage` | `Excel_Cs_InsertPictureV2` | 插入图片 |
| `Excel_Cs_CopyRangeV2` | `Excel_Cs_CopyRangeV3` | 复制表格内容 |
| `Excel_Cs_CopyRange` | `Excel_Cs_CopyRangeV3` | 复制表格内容 |
| `Excel_Cs_SetWidth` | `Excel_Cs_SetWidthV1` | 设置列宽 |
| `Excel_SetColumnWidth` | `Excel_Cs_SetWidthV1` | 设置列宽 |
| `Excel_Cs_RunMacro` | `Excel_Cs_RunMacroV2` | 执行宏 |
| `Excel_ExecuteMacro` | `Excel_Cs_RunMacroV2` | 执行宏 |
| `Excel_Cs_Find` | `Excel_Cs_FindV2` | Excel查找/替换 |
| `Excel_Cs_WriteRowV2` | `Excel_WriteRowNew` | 写入行 |
| `Excel_Cs_WriteRow` | `Excel_WriteRowNew` | 写入行 |
| `Excel_WriteRow` | `Excel_WriteRowNew` | 写入行 |
| `Excel_Cs_ReadRow` | `Excel_GetRowDataNew` | 读取行 |
| `Excel_GetRowData` | `Excel_GetRowDataNew` | 读取行 |
| `Excel_Cs_MergeCells` | `Excel_Cs_MergeCellsV2` | 合并单元格 |
| `Excel_Cs_Close` | `Excel_Cs_CloseV2` | 关闭Excel |
| `Excel_Cs_SelectRangeV2` | `Excel_Cs_SelectRangeV3` | 选中区域 |
| `Excel_Cs_SelectRange` | `Excel_Cs_SelectRangeV3` | 选中区域 |
| `Excel_SelectRange` | `Excel_Cs_SelectRangeV3` | 选中区域 |
| `Excel_Cs_PastRange` | `Excel_Cs_PastRangeV2` | 粘贴表格内容 |
| `Excel_Cs_ColumnSplit` | `Excel_Cs_ColumnSplitV2` | Excel分列 |
| `Excel_Cs_Sort` | `Excel_Cs_SortV1` | Excel数据排序 |
| `Excel_Cs_SetRangeBackground` | `Excel_Cs_SetRangeBackgroundV2` | 设置区域背景颜色 |
| `Excel_Cs_Load` | `Excel_Cs_LoadV2` | 获取已打开Excel |
| `File_GetFileList` | `File_GetFileList_V1` | 获取文件列表 |
| `File_Write` | `File_Write_V1` | 写入文件 |
| `File_MakeFolder` | `File_MakeDir` | 创建文件夹 |
| `File_GetFolderList` | `File_GetFolderList_V1` | 获取文件夹列表 |
| `Http_Post_V2` | `Http_Post_V3` | POST请求 |
| `Http_Post` | `Http_Post_V3` | POST请求 |
| `Http_Get_V1` | `Http_Get_V2` | GET请求 |
| `Http_Get` | `Http_Get_V2` | GET请求 |
| `Image_Click_V1` | `Image_Click_V2` | 点击图像 |
| `Image_Click` | `Image_Click_V2` | 点击图像 |
| `Image_Find` | `Image_Find_V1` | 查找图像 |
| `Image_Wait` | `Image_Wait_V1` | 等待图像 |
| `Image_Exists` | `Image_Exists_V1` | 判断图像 |
| `Image_Move` | `Image_Move_V1` | 移动到图像 |
| `Keyboard_DriveSend_V1` | `Keyboard_DriveSend_V2` | 键盘驱动输入 |
| `Keyboard_DriveSend` | `Keyboard_DriveSend_V2` | 键盘驱动输入 |
| `Keyboard_Send` | `Keyboard_Send_V1` | 键盘输入 |
| `Mail_UnionConnect` | `Mail_RecvConnect` | 连接邮箱 |
| `Mail_ReplyMail_v1` | `Mail_ReplyMail_v2` | 回复邮件 |
| `Mail_ReplyMail` | `Mail_ReplyMail_v2` | 回复邮件 |
| `Mail_UnionSend_v1` | `Mail_Send_V1` | 发送邮件 |
| `Mail_Send` | `Mail_Send_V1` | 发送邮件 |
| `Mail_UnionSend` | `Mail_Send_V1` | 发送邮件 |
| `Mail_FindMail` | `Mail_FindMail_v1` | 查找邮件 |
| `Mouse_ClickCombine_v3` | `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_ClickCombine_v2` | `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_ClickCombine_v1` | `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_Click` | `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_ClickCombine` | `Mouse_ClickCombine_v4` | 鼠标点击 |
| `Mouse_DragToV2` | `Mouse_DragToNew` | 鼠标拖拽 |
| `Mouse_DragToV1` | `Mouse_DragToNew` | 鼠标拖拽 |
| `Mouse_DragTo` | `Mouse_DragToNew` | 鼠标拖拽 |
| `Mouse_MoveToV1` | `Mouse_MoveToV2` | 移动鼠标 |
| `Mouse_MoveTo` | `Mouse_MoveToV2` | 移动鼠标 |
| `PDF_SetPdfWaterMark` | `PDF_SetPdfWaterMarkV1` | PDF 加水印 |
| `PDF_ConcatPdf` | `PDF_ConcatPdfV1` | 合并 PDF 文档 |
| `Regex_Find` | `Regex_FindNew` | 指定查找 |
| `StrUtil_CutOutStr` | `StrUtil_CutOutStrV1` | 截取文本 |
| `System_ScreenSaverShow` | `System_ScreenSaverShowV1` | 开启屏保 |
| `System_ExecuteCommand` | `System_ExecuteCommandV1` | 执行命令行 |
| `Time_GetDateV1` | `Time_GetDateV2` | 获取日期时间 |
| `Time_GetDate` | `Time_GetDateV2` | 获取日期时间 |
| `Time_DateOffset` | `Time_DateOffset_V1` | 增加/减少时间 |
| `WebBrowser_Switch` | `WebBrowser_Switch_V1` | 切换页面 |
| `WebBrowser_SetScrollPos` | `WebBrowser_SetScrollPos_V1` | 设置滚动条位置 |
| `WebBrowser_GetTAB_V1` | `WebBrowser_GetTAB_V2` | 获取网页 Tab 对象 |
| `WebBrowser_SaveAs` | `WebBrowser_SaveAs_V1` | 下载文件(点击按钮) |
| `WebBrowser_BatchDataCrawlNew` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCollectionV4` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCollectionV3` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCollectionV2` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_BatchDataCrawlNewV1` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCollectionV1` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCollection` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCrawl` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCrawl_software` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_DataCrawl_web` | `WebBrowser_DataCollectionV5` | 数据采集 |
| `WebBrowser_RunJsInIeBySelector_v2` | `WebBrowser_RunJsInIeBySelector_v3` | 执行JS |
| `WebBrowser_RunJsInIeBySelector_v1` | `WebBrowser_RunJsInIeBySelector_v3` | 执行JS |
| `WebBrowser_RunJsInIeBySelector` | `WebBrowser_RunJsInIeBySelector_v3` | 执行JS |
| `WebBrowser_RunJsInIeByTitle` | `WebBrowser_RunJsInIeBySelector_v3` | 执行JS |
| `WebBrowser_CreateV2` | `WebBrowser_CreateV3` | 打开网页 |
| `WebBrowser_Create` | `WebBrowser_CreateV3` | 打开网页 |
| `WebBrowser_CreateBackground` | `WebBrowser_CreateV3` | 打开网页 |
| `WebBrowser_CloseWeb_V1` | `WebBrowser_CloseWeb_V2` | 关闭网页 |
| `WebBrowser_CloseWeb` | `WebBrowser_CloseWeb_V2` | 关闭网页 |
| `WebBrowser_DownFile` | `WebBrowser_DownFile_V1` | 下载文件(指定链接) |
| `WebBrowser_DownloadFile` | `WebBrowser_DownloadFile_V1` | 下载文件 |
| `WebBrowser_GetBrowserPic` | `WebBrowser_GetBrowserPicV1` | 浏览器截图 |
| `WebBrowser_Close` | `WebBrowser_Close_V1` | 关闭浏览器 |
| `WebBrowser_GetScrollPos` | `WebBrowser_GetScrollPos_V1` | 获取滚动条位置 |
| `Window_SizeWindow` | `Window_SizeWindow_v1` | 调整窗口大小 |
| `Window_SetWindowState` | `Window_SetWindowState_v1` | 设置窗口状态 |
| `Window_SetActive` | `Window_ActiveWindow_v1` | 激活窗口 |
| `Window_GetWindow` | `Window_GetWindow_v1` | 获取窗口对象 |
| `Window_Close` | `Window_CloseWindow_v1` | 关闭窗口 |
| `Window_GetWindowMessage` | `Window_GetWindowMessage_v1` | 获取窗口信息 |
| `Window_MoveWindow` | `Window_MoveWindow_v1` | 移动窗口位置 |
机器可读决策:`resources/capability-duplicates.json`。
@@ -0,0 +1,119 @@
# AI 拾取模型选型与 RPA 执行方案
## 结论
当前模型“只能识别控件”不是解析器写错,而是 OmniParser 的 icon detect 本身就是单类别模型。它适合解决“屏幕上有哪些可交互区域”,不适合直接回答“这是一个登录按钮还是搜索框”。
更合适的不是换一个全能小模型,而是分四层:
1. 视觉定位:YOLO/OmniParser 找 bbox,保证又快又能看到 UIA 看不到的区域。
2. 视觉语义:OCR 读取框内文本;必要时用小分类模型或图标匹配补充类别。
3. UIA 增强:用 AI bbox 中心回查 Windows UIA,拿到 `Name / ControlType / AutomationId / ClassName`。
4. RPA 执行:优先保存 UIA selector,AI bbox 只作为兜底坐标和运行时重定位依据。
这才是对齐实在这类产品的方向:AI 负责感知和定位,RPA 引擎负责稳定执行和状态反馈。
## 当前可试模型
模型已经放在 `D:\code\PRA-only-demo\temp\models`。同名 `*.labels.json` 已经配好,当前实验台会读取标签,不再显示“类别1/类别2”。
| 本地文件 | 来源 | 类别数 | 类别 | 速度/体积 | 适用场景 | License 风险 |
|---|---|---:|---|---|---|---|
| `omniparser-v1-fp16.onnx` | `onnx-community/OmniParser-icon_detect` | 1 | `控件` | 约 6 MB,测试约 0.8s | 可交互区域粗定位、基线对比 | 商用前复核 OmniParser/底模授权 |
| `amigo-ui-elements-yolov8.onnx` | `amigodev/ui-elements-detector-test` | 8 | `button/field/heading/iframe/image/label/link/text` | 约 12 MB,测试约 0.4s | 网页截图、轻量多类别试验 | 仓库未标 License,仅试验 |
| `webpage-element.onnx` | `OpenDILabCommunity/webpage_element_detection` | 8 | `button/field/heading/iframe/image/label/link/text` | 约 99 MB,测试约 1.5s | 网页 UI,准确率通常优于小模型 | Apache-2.0 |
| `android-ui-yolov8n.onnx` | `yasirfaizahmed/android_ui_detection_yolov8` | 21 | `TextButton/EditText/CheckBox/Switch/Toolbar` 等 | 约 12 MB,测试约 0.4s | 安卓/移动 UI;桌面也可当轻量 UI 分类基线 | Apache-2.0 |
| `macpaw-ui-elements-yolo11l.onnx` | `macpaw-research/yolov11l-ui-elements-detection` | 5 | `AXButton/AXDisclosureTriangle/AXImage/AXLink/AXTextArea` | 约 102 MB,测试约 1.7s | 桌面/辅助功能导向,语义更接近可访问性控件 | AGPL-3.0,商用谨慎 |
| `resources/models/ui-element-detect/webpage-element.onnx` | OpenDILab webpage | 8 | `button/field/heading/iframe/image/label/link/text` | 当前默认,约 99 MB | 已实测并接入 Pane AI 拾取 | Apache-2.0 |
| `resources/models/ui-icon-detect/model.onnx` | OmniParser v2 icon detect | 1 | `控件` | 旧链路兼容回退 | 仅缺新模型时兜底 | 商用前复核 |
测试方式:打开“AI 识别实验台”,选择识别目标,点“选择 ONNX”,选择上表中的 `.onnx` 文件。确保同名 `.labels.json` 放在同一目录。
## 实测观察
同一张简单网页风格测试图:
- Android UI YOLOv8n:约 431ms,能识别 `TextButton`、`EditText`、`Text`,置信度普遍高。
- Amigo UI YOLOv8:约 430ms,能识别 `button`、`field`,框更整齐。
- OpenDILab webpage:约 1533ms,类别与 Amigo 相同,体积和耗时更大。
- MacPaw YOLO11l:约 1743ms,能识别 `AXButton`、`AXTextArea`,但对该测试图置信度偏低。
- OmniParser v1:约 781ms,只输出“控件”,定位能力可用但没有类别语义。
这组结果说明:轻量 YOLO 完全能做到实时/近实时,速度问题不是模型架构本身,而是模型大小、截图裁剪、进程通信和重复初始化共同决定的。
## 当前 RPA 可用性
当前 Pane AI 拾取生成:
```text
screen=x,y,width,height;window=...;name=OCR文本或类别;type=视觉类别;aiType=视觉类别;aiText=OCR文本;ai=1
```
`webpage-element.onnx` 先定位候选;PaddleOCR/PP-OCRv4 对同一张 Pane 截图做一次整图识别,并按几何关系把文本写回候选框。整图结果按截图 SHA-256 缓存,重复识别同一画面不会重复推理。执行层已经支持以 `screen=` 做坐标点击、坐标输入、坐标取值,所以“立刻执行一次”是可用的。
但只靠当前 selector 还不能算稳定 RPA 元素,原因是:
- `screen=` 是拾取瞬间的坐标快照。
- 窗口移动、页面滚动、分辨率/DPI 变化后会失效。
- `exists/wait` 目前对 `screen=` 过于宽松。
- `name/type` 很多时候只是视觉模型类别,不是业务语义。
- 没有运行时重定位机制。
## 目标 selector 结构
建议逐步变成:
```text
window=title:xxx;process=xxx;
type=Button;name=登录;id=loginButton;class=Button;
aiType=button;aiRect=x,y,w,h;aiModel=webpage-element;
template=sha1:...;ai=1
```
定位策略:
1. 优先按 `window + AutomationId + Name + ControlType` 查 UIA。
2. UIA 查不到时,限制在原窗口/原 Pane 附近重找。
3. 再失败时截图并运行视觉模型,在候选中按 `aiType + IoU + 外观相似度` 找回目标。
4. 找回后更新真实 UIA rect,再执行动作。
## AI 类别到 RPA 动作映射
| AI/UIA 类别 | RPA 默认动作 |
|---|---|
| `button / AXButton / TextButton` | `Element_Click` |
| `field / EditText / AXTextArea / TextBox` | `Element_SetValue` / `Element_GetValue` |
| `link / AXLink` | `Element_Click` |
| `text / label / heading` | `Element_GetText` / OCR |
| `CheckBox / Switch` | `Element_SetCheck` / `Element_GetCheck` |
| `Spinner / ComboBox` | `Element_SetSelect` / 展开后点击 |
| `image / Icon` | 坐标点击 / 图标匹配 |
| `Toolbar / Navigation / Group` | 作为容器遍历子元素,不直接执行动作 |
## 最好用的落地路线
### 第一阶段:可用
- 已完成:本地模型、Pane AI、候选框、悬浮确认。
- 已完成:候选类别从 sidecar 读取,默认模型切换为 `webpage-element.onnx`。
- 已完成:PaddleOCR 整图识别、候选框文本分配、OCR 结果缓存和异步回填。
- 已完成:`F2/F4` 进入 AI;`Esc/F9` 取消 AI;`F8` 切换开发候选框。
- 补 UIA 回查:选中框后用中心点/范围找最小 UIA 元素,生成稳定 selector。
### 第二阶段:稳定
- `exists/wait` 真正支持 `screen= + window + aiRect`。
- 执行前校验窗口和元素仍在预期区域。
- 增加 AI 重定位:截图 → 检测 → 与保存的 bbox/裁剪图匹配 → 更新坐标。
- 所有执行结果回写目标快照,方便排查框偏。
### 第三阶段:接近实在
- 元素检测、元素分类、OCR、图标相似、UISelector 组合。
- AI 只输出结构化 target/action,不直接控鼠标键盘。
- RPA 执行器负责窗口激活、等待、重试、截图、异常恢复。
- 对复杂远程桌面/Canvas/Java 应用保留视觉兜底。
## 推荐优先级
已选定 `webpage-element.onnx` 作为 Pane AI 默认模型。其他模型继续留在本地 `temp/models` 供实验台对比,不再进入默认链路和安装包。内置 OmniParser 只作为老安装包兼容回退,不建议作为最终业务 selector 的唯一来源。
+83
View File
@@ -0,0 +1,83 @@
# RPA Studio 开发规范(v1 · 面向 rpa-core 抽离)
> 生效日期:2026-11-04。
> 背景:本平台继续作为日常开发载体;所有新代码必须同时满足"当前可用"和"将来可直接搬入
> `packages/rpa-core`(最终迁入 Agent-Employee)"。迁移路线与任务清单见
> `docs/rpa-core-refactor-tasks.md`,本规范是它的日常执行约束。
## 核心思想
**迁移不是将来的改造,而是现在每次写代码时的选择。** 遵守本规范写的能力,抽包时是搬运;
违反规范写的能力,抽包时是返工。
---
## 一、分层归属(硬性规则)
| 层 | 目录 | 判别标准 | 未来去向 |
|---|---|---|---|
| **L1 包内层** | `src/main/engine/`、`src/main/capabilities/` 大部分、`src/main/browser/external.ts`、`transport.ts` | 不依赖 Electron 窗口即可运行 | rpa-core 包 |
| **L2 宿主层** | `src/main/index.ts`、`overlay`、`desktop-pick`、`vision-lab`、`fixture/`、`local-vision/`、`flows`、`run/`、`settings`、`logger`、`browser/session.ts` 内置浏览器部分 | 需要 Electron 窗口/应用生命周期 | 宿主适配层 |
**判别口诀**:这个能力在没有窗口的 CLI 里能跑吗?能 → L1;不能 → L2。
---
## 二、L1 层禁令(现有代码不追溯,新代码必须遵守)
1. **禁止 `import ... from "electron"`**
需要剪贴板/对话框/截屏/屏幕信息时:先看 Node 等价物(fs/os/child_process),
确实需要宿主能力时在 RunContext 或参数中注入,并在代码处注释 `// HostBridge 候选`。
2. **禁止横向依赖宿主模块**
L1 内禁止 `import "../settings"`、`"../logger"`、`"../flows"` 等。配置走参数/ctx,日志走 `ctx.log`。
3. **禁止私起 PowerShell 进程**
所有 PowerShell 调用必须使用 `capabilities/desktop.ts` 导出的 `powershell()` / `psString()`。
禁止自己 `execFile("powershell.exe", ...)`。原因:将来统一切换为常驻 worker(任务 3.5)。
4. **禁止新增模块级可变全局状态**
不写模块顶层 `const cache = new Map()` / `let globalXxx`。需要跨调用状态时挂在显式 store
或 RunContext 上(目标形态:`createEngine()` 实例持有)。
5. **禁止把 PowerShell 逻辑写成不可拆的内联串**
复杂脚本提取为独立函数/常量,便于将来搬进 WindowsDriver。
## 三、能力编写规范(tool 友好,为 Agent-Employee 准备)
1. **结构化进出**:参数从 `args` 对象取具名字段,返回结构化对象或原始值;
禁止返回"为 UI 拼接的字符串"(渲染层需要展示时自行格式化)。
2. **必有超时**:涉及外部交互(进程/网络/窗口等待)的能力必须有 timeout 参数和默认值。
3. **错误契约**:失败抛 `RuntimeError(code, message, { recommendation })`;
code 用稳定英文枚举(必须登记在 `packages/rpa-core/src/engine/error-codes.ts`
的 `RUNTIME_ERROR_CODES`,任务 4.3 起机器可读),message 中文,recommendation 给可执行建议。禁止裸 throw string。
4. **平台标记**:Windows 专用能力(user32/UIA/COM)在 catalog 注册时带
`platforms: ["win32"]`;能用纯 Node 实现的能力不调 PowerShell。
5. **可测试**:每个能力有同名 `*.test.ts`(vitest),与实现同目录。
6. **结构化 meta(任务 4.1 起,新能力强制)**:`register(ids, handler, meta)` 第三参
必填 `CapabilityMeta`(summary / params / returns / timeoutMs / errorCodes /
needsScreenshot / platforms)。老能力逐步补齐,review 时放行不带 meta 的旧文件改动。
## 四、新能力验收 checklist
- [ ] 归属判定正确(L1/L2 口诀)
- [ ] 无 electron / 宿主模块 import(L1 时)
- [ ] PowerShell 只经 `powershell()` 助手(如需要)
- [ ] 无新增模块级全局状态
- [ ] 结构化参数/返回 + 超时 + RuntimeError 错误契约
- [ ] catalog 注册完整(含 platforms 标记)
- [ ] 有测试,`npm run test` + `npm run typecheck` 全绿
## 五、渐进收紧节奏
规范随改造阶段自动收紧,不必一步到位:
| 里程碑(见任务清单) | 规范变化 |
|---|---|
| 现在 | 本文第二节禁令生效(靠 review 执行) |
| 阶段 2 完成 | L1 禁 electron 升级为 ESLint no-restricted-imports 强制 |
| 阶段 3 完成 | `powershell()` 收口进 `driver/windows/`,L1 禁止直接引用 |
| 阶段 4 完成 | 无 schema 的新能力不允许进 catalog |
## 六、明确不动的部分
- 现有 77 处 PowerShell 调用点:**不做预防性重构**,统一等阶段 3 的 driver 抽象一次性处理。
之前只修 bug、只加能力,不重写。
- 渲染层(React)按产品需求正常开发,不为其迁移性投资。
- `local-vision/`、`overlay/` 等宿主层模块正常开发,不受 L1 禁令约束。
+116
View File
@@ -0,0 +1,116 @@
# rpa-core 能力层抽离改造任务清单
> 目标:把 `src/main/capabilities/` + `src/main/engine/` 抽成零 Electron 依赖的独立包,
> 先在当前 demo 验证,最终以包为单位迁入 `D:\code\Agent-Employee`。
> 风险评估见对话记录(2026-11-04),任务编号 R1~R10 对应风险清单。
## 总体原则
1. **单一源**:包建成后 demo 与未来宿主都通过 workspace 引用,任何时候禁止复制代码(R3)。
2. **行为不变**:抽离期间能力行为保持不变,靠现有 vitest 套件做安全网。
3. **先断依赖再抽象**:Electron 依赖清零优先于 driver 接口美化。
---
## 阶段 0 — 基线与护栏
- [ ] **0.1 确认基线全绿**
`npm run test` + `npm run typecheck` 全部通过,记录通过数作为回归基线。
- [ ] **0.2 确定包位置与工作区结构**
建 `packages/rpa-core/`,demo 的 `package.json` 加 workspaces 配置,源码从 `src/main` 移入后以 `@rpa/core` 引用回主进程。
- [ ] **0.3 生成能力目录快照**
用 `npm run catalog` 的产物做一份快照(能力 id / 参数 / 描述清单),抽离完成后 diff 必须为零,作为行为回归基准。
## 阶段 1 — 建包与纯 Node 能力搬运
- [ ] **1.1 包骨架**
`packages/rpa-core`:`package.json`(deps: exceljs / pdf-lib / nodemailer / basic-ftp / adm-zip / pngjs)、`tsconfig.json`、独立 vitest 配置(R6:包内保留 vitest,不迁 bun test)。
- [ ] **1.2 领域类型解耦(R2)**
搬 `engine/`(registry / types / runtime / expression),将 `shared/types` 依赖改为包内自有领域类型(FlowDefinition / FlowNode / CatalogComponent / RunError)。宿主后续做模型映射,不把画布模型塞进包。
- [ ] **1.3 全局状态收拢为工厂(R4)**
新增 `createEngine()` 工厂:excel-store 的模块级 `Map` 缓存、session 的 `globalElementWaitMs` 等进程级状态挂到引擎实例上;registry 注册目标从全局单例改为引擎实例。
- [ ] **1.4 搬运纯 Node 能力**
files / data / data-extended / http / ftp / mail / mssql / mysql / pg / excel(exceljs)/ word / pdf / office-file / time / crypto / data-table 等不依赖 Electron 的能力文件。
- [ ] **1.5 搬运对应测试**
上述能力的 `*.test.ts` 一并移入包内,路径修正后全量跑绿。
- [ ] **1.6 demo 主进程改为引用包**
`src/main` 里被搬走的部分改 import `@rpa/core`,`npm run test` 全绿、catalog 快照 diff 为零。
## 阶段 2 — 断 Electron 依赖
- [ ] **2.1 定义 HostBridge 接口**
包内声明 `HostBridge`:clipboard / dialog / screen / userAgentDataPath 等宿主能力;demo 的 Electron 主进程提供实现并注入。
- [ ] **2.2 改造直接依赖 Electron 的能力(3 处)**
`capabilities/human-dialog.ts`(BrowserWindow)、`capabilities/window-extended.ts`(screen)、`capabilities/ai-service.ts`(settings)→ 走 HostBridge 注入。
- [ ] **2.3 浏览器子系统切分(R1)**
`browser/external.ts` + `transport.ts` 的外部浏览器 CDP 路径搬进包;`session.ts` 中依赖内置 `BrowserWindow` 的部分留在宿主适配层,包内通过 `BrowserLike` 接口(types.ts 已有)注入。
- [ ] **2.4 防回归护栏**
在包内加依赖检查脚本(或 ESLint no-restricted-imports 规则):禁止 `import "electron"` 及对 `src/main` 的相对引用,CI 里跑。
- [ ] **2.5 Electron 相关能力留守清单**
overlay / desktop-pick / vision-lab / fixture server / human-dialog 的 UI 部分:明确标记"宿主适配层",从包中排除,记录在包 README。
## 阶段 3 — Driver 抽象(Windows 自动化能力)
- [ ] **3.1 拆出 powershell 助手(R5)**
`powershell()` / `psString()` / `USER32` 从 `capabilities/desktop.ts` 移到 `driver/windows/powershell.ts`;修正 11 个引用文件的 import;测试全绿。
- [ ] **3.2 定义 AutomationDriver 接口**
按域拆分:`mouse` / `keyboard` / `window` / `element`(UIA)/ `screen`。接口签名以现有 PowerShell 能力的输入输出为准,不重新发明。
- [ ] **3.3 现有实现包进 WindowsDriver**
desktop / desktop-uia / window-extended / remaining-core / system-extended / common-final / message / utility-extended 中的 user32/UIA 调用改为面向接口;`WindowsDriver` 内部仍用 PowerShell 实现(行为不变)。
- [ ] **3.4 driver 注入与平台标记**
引擎创建时注入 driver;能力目录(catalog)加 `platforms: ["win32"]` 字段,渲染层据此置灰。
- [ ] **3.5(可选,迁移前不阻塞)PowerShell 常驻 worker**
单进程 JSON 行协议替代每次 spawn;`powershell()` 接口签名不变,内部切换。性能优化项,放在功能等价验收之后。
## 阶段 4 — Tool 契约(为 Agent-Employee 做)
- [ ] **4.1 能力 schema 化**
每个能力导出结构化定义:参数类型 / 返回值 / 超时 / 错误码 / 是否需要截图回传。新能力一律按此格式写。
- [ ] **4.2 注册机制显式化**
取消 import 副作用注册,改为显式 `registerAll(engine)` 导出,宿主可按子集注册。
- [ ] **4.3 错误契约规范化(R9)**
RuntimeError 错误码枚举固化,中文 message 保留,增加机器可读字段,供 Agent 侧消费。
## 阶段 5 — 独立验证
- [ ] **5.1 CLI host**
写 `scripts/rpa-cli.ts`:Node 直接拉起 `createEngine()` + `WindowsDriver`,跑通 5 个代表能力(1 个桌面、1 个 Excel、1 个 HTTP、1 个文件、1 个窗口)。
- [ ] **5.2 Bun 冒烟(R6/R7)**
在 Agent-Employee 环境用 `bun test` 跑包内纯逻辑测试,验证 exceljs / pdf-lib 等依赖在 Bun 下正常;onnxruntime 相关暂不进包,另行验证。
- [ ] **5.3 demo 全量回归**
demo 切到 workspace 引用后:test + typecheck + catalog 快照 diff 三项全绿。
## 阶段 6 — 迁入 Agent-Employee
- [ ] **6.1 ADR 评审**
按 Agent-Employee 流程提交架构决策记录:包边界 / HostBridge 契约 / driver 生命周期。
- [ ] **6.2 包迁入 `packages/`**
整体搬入,改为 bun workspace 成员;demo 侧保留只读引用或解除引用。
- [ ] **6.3 宿主适配层**
按 Agent-Employee 的 product-shell 契约写 host 适配(clipboard / dialog / screen / BrowserLike 实现)。
- [ ] **6.4 权限与治理对接(R10)**
`System_ExecuteCommand` 等危险能力接入其权限模型;能力目录与插件体系对齐。
---
## 风险 → 任务映射
| 风险 | 缓解任务 |
|---|---|
| R1 浏览器子系统耦合 Electron | 2.3 |
| R2 领域模型冲突 | 1.2、阶段 4 |
| R3 代码漂移 | 0.2、总原则 1 |
| R4 模块级全局状态 | 1.3 |
| R5 powershell 助手位置 | 3.1 |
| R6 测试运行器不兼容 | 1.1、5.2 |
| R7 原生依赖 Bun 兼容 | 5.2(能力层已确认为纯 JS) |
| R8 注册副作用 | 4.2 |
| R9 错误契约 | 4.3 |
| R10 权限治理 | 6.4 |
## 明确不迁入包的部分
- `src/renderer/`(React 渲染层,Agent-Employee 用 SolidJS,不迁移)
- `overlay.ts` / `desktop-pick.ts` / `vision-lab.ts` / `fixture/`(Electron 窗口强耦合,留宿主)
- `local-vision/`(onnxruntime-node 原生依赖,Bun 兼容性单独立项验证)
- `flows.ts` / `run/manager.ts` / `settings.ts`(应用级编排与持久化,属宿主职责)
+103
View File
@@ -0,0 +1,103 @@
# 图像与OCR 能力问题检测报告
- **检测时间**:2026-10-30 10:00–10:10
- **检测人**:刘爱家(AI 辅助静态核对 + 真实运行验证)
- **检测范围**:图像与OCR 分组已接入的全部 10 项能力(Image 7 项 + OcrOffline 3 项)
- **检测方式**:
1. 静态核对——组件目录(catalog.json)字段定义 vs 实现代码(vision.ts / ocr-caps.ts)逐键比对
2. 真实运行——GDI 生成含中英文的测试图、真实截屏、模板匹配直测、OCR 引擎实识别(飞桨 @gutenye/ocr-node)
- **说明**:本次仅检测未修改任何代码,问题留给同事修复。
- **修复状态(2026-11-02 更新)**:报告中的全部问题已由刘爱家修复完成——
- 提交 `342145c`:P2-2(屏幕截图图像来源/截图范围,含区域裁剪与下拉选项)
- 提交 `ba17d6d`:P0-1(GetOcrTextNew 传图键)、P1-1(点击位置九宫格/偏移)、P1-2(鼠标按键左/右/中/双击)、P1-3(GetOcrTable 行列聚类 + 写数据表 + 存 Excel)、P2-1(查找范围指定窗口/指定区域)、P3-2(鼠标移动轨迹动画)、P3-3(截图空路径落证据目录;表格行内按 x 排序);P3-1(accuracy2)无特征匹配层实现,已在目录中隐藏并注明原因
- 回归:typecheck 通过;vitest 256 过(4 失败为与本次无关的桌面/PDF 环境遗留项);新增 vision-helpers 单测 12 项覆盖区域解析/裁剪/九宫格
---
## 一、结论概览
| 组件 | 名称 | 可用性 | 核心问题 |
|---|---|---|---|
| Image_Screenshot | 屏幕截图 | ⚠️ 基本可用 | 截图范围/图像来源字段无效,仅支持全屏 |
| Image_Find | 查找图像 | ✅ 可用 | accuracy2 字段无效 |
| Image_Find_V1 | 查找图像 | ⚠️ 有隐患 | window_obj 必填但被忽略,实际全屏查找 |
| Image_Click | 点击图像 | ⚠️ 有缺陷 | position(点击位置)被忽略,永远点中心 |
| Image_Click_V1 | 点击图像 | ⚠️ 有缺陷 | 同上 |
| Image_Click_V2 | 点击图像 | ⚠️ 有缺陷 | 同上 |
| Image_Click_V3 | 鼠标点击图像 | ❌ 功能缺失 | click_key(左/右键)被忽略,右键点击无法实现;窗口限定无效 |
| OcrOffline_GetOcrTextNew | 图像OCR识别 | ❌ 不可用 | **按目录配置无法传入图片路径,任何填写方式都报「图片路径为空」** |
| OcrOffline_GetOcrTextPosition | 图像OCR识别(含坐标) | ✅ 可用 | 无阻塞性问题 |
| OcrOffline_GetOcrTable | 获取表格数据(OCR) | ⚠️ 部分可用 | 保存至数据表/Excel 的 4 个字段全部无效;返回扁平文本非表格结构 |
**正常项(已实测确认)**:截屏产物正确;模板匹配精度高(自测得分 0.9993);OCR 中文/数字识别准确、坐标与置信度返回正常;找图失败时报错提示合理。
---
## 二、问题清单(按严重度)
### P0-1 OcrOffline_GetOcrTextNew:目录字段与实现完全脱节,组件不可用
- **现象**:该组件在目录里唯一的图片相关字段是 `uiSelector`(目标,非必填),而实现读取图片路径的键列表为 `image_path/imagePath/file_path/filePath/path/image/pictureSource`——**不包含 uiSelector**。
- **后果**:用户按目录界面填参数,无论怎么填都无法把图片传进实现,必然报「图片路径为空」。已实测复现:`{ uiSelector: "D:/.../ocr_input.png" }` → `RuntimeError: 图片路径为空`。
- **修复建议**:实现读取键中加入 `uiSelector`,或目录为该组件增加 `image_path` 必填字段(与 GetOcrTextPosition 保持一致)。
### P1-1 Image_Click 全 4 版本:忽略「点击位置」字段
- **现象**:目录字段 `position`(点击位置,默认 center),实现拿到匹配区域后**固定点击中心点**(`vision.ts` 中 `x + width/2, y + height/2`),position 值从不读取。
- **后果**:用户选择「左上角/右上角/自定义偏移」等位置时静默失效,点击落点与预期不符且无任何提示。
### P1-2 Image_Click_V3:忽略「鼠标按键」字段,右键点击无法实现
- **现象**:目录字段 `click_key`(鼠标按键,默认 left),实现写死左键单击(`mouse_event(2,...)`按下 + `mouse_event(4,...)`弹起)。
- **后果**:右键菜单类自动化场景完全无法实现,且不报错(静默按左键执行)。
### P1-3 OcrOffline_GetOcrTable:输出相关 4 个字段全部无效
- **现象**:目录字段 `showTable`(保存至数据表)、`isExcel`(保存至Excel)、`excelName`、`excelPath`,实现全部不读取;仅返回按 y 坐标排序的**扁平文本行数组**。
- **后果**:「将结果保存至数据表/Excel」的承诺静默不执行;且行内多列文本不拆分(如「鼠标 20」作为一整行文本返回),拿不到结构化表格。
### P2-1 Image_Find_V1 / Image_Click_V3:「窗口对象」必填但被忽略
- **现象**:目录要求 `window_obj` 必填、`find_range`(图像查找范围)可选,实现两者都不读取,永远全屏截图查找。
- **后果**:用户以为限定了窗口范围,实际全屏找图——找到错误目标(其他窗口的同款图标)或找不到时无从排查。且 window_obj 必填迫使用户必须配置一个无意义的参数。
### P2-2 Image_Screenshot:「截图范围」「图像来源」字段无效
- **现象**:目录字段 `pictureSource`(图像来源)、`shotRange`(截图范围),实现不读取,固定全屏(`PrimaryScreen`)。目录里还有 `action/position/position_img` 三个疑似遗留占位字段。
- **后果**:只支持全屏截图;另外多显示器环境只截主屏,副屏内容丢失。
### P3-1 「特征相似度」accuracy2 字段无效
- Image_Find / Image_Click / Image_Click_V1 的 `accuracy2`(特征相似度)字段,实现不读取。当前实现是纯像素差模板匹配,没有特征匹配层,该字段为冗余配置。
### P3-2 「展示鼠标移动轨迹」show_track 字段无效
- Image_Click 全版本的 `show_track` 被忽略。当前实现 SetCursorPos 直接跳转,无轨迹动画。低优先级。
### P3-3 其他小问题
- Image_Screenshot 当 `image_path` 为空时输出文件落盘到进程工作目录(不可预期);因该字段必填,实际风险低,建议加显式校验。
- OcrOffline_GetOcrTable 的行排序仅按 y 坐标,同一行内多个文本块不合并、不按 x 排序,多列场景顺序不稳定。
---
## 三、实测验证记录(正常部分证据)
| 测试 | 结果 |
|---|---|
| GDI 生成测试图(600×200,含「RPA流程测试ABC123」「名称 数量」「鼠标 20」) | 成功 |
| Image_Screenshot 截屏逻辑(同款 PowerShell 脚本) | 成功,产物 246KB PNG |
| findImage 模板匹配(app_logo_tpl 模板 vs 实时截屏) | 命中 (93,27),得分 **0.9993** |
| OcrOffline_GetOcrTextNew(传入 image_path 时) | 识别结果 `RPA流程测试ABC123\n名称 数量\n鼠标 20` 完全正确 |
| OcrOffline_GetOcrTextPosition | 3 行文本坐标 + 置信度(0.999/0.949/0.899)返回正常 |
| 找图失败路径 | 正确抛「屏幕中未找到目标图像」并附建议(注:vitest 环境内 PowerShell 受 PATH 限制报 ENOENT,真实应用主进程不受影响) |
**说明**:OCR 引擎(飞桨)与模板匹配算法本身质量良好,问题集中在**目录字段与实现的键名/功能脱节**——与此前 Office、数据处理组发现的拼接列、单元格格式问题是同一类「接入只对了一半」的缺陷。
---
## 四、给同事的修复优先级建议
1. **先修 P0-1**(GetOcrTextNew 传图键)——1 行改动,组件从不可用变可用
2. **再修 P1 三项**(Click 的 position/click_key、GetOcrTable 的输出字段)——都是读键 + 少量分支逻辑
3. **P2/P3 可排后**——窗口限定、截图范围属于功能增强,当前行为可接受但建议在 UI 上隐藏无效字段避免误导
+62
View File
@@ -0,0 +1,62 @@
import { readFileSync } from "node:fs";
import path from "node:path";
import os from "node:os";
import process from "node:process";
import { fileURLToPath } from "node:url";
const root = path.dirname(fileURLToPath(import.meta.url));
const packageJson = JSON.parse(readFileSync(path.join(root, "package.json"), "utf8"));
/** @type {import('electron-builder').Configuration} */
export default {
appId: "com.rpastudio.desktop",
productName: "RPA Studio",
artifactName: "RPA-Studio-Setup-${version}-win-${arch}.${ext}",
directories: {
output: path.join(os.tmpdir(), "RPA-Studio-electron-builder"),
buildResources: path.join(root, "build")
},
files: [
"out/**/*",
"package.json"
],
extraResources: [
{ from: path.join(root, "resources", "catalog.json"), to: "catalog.json" },
{ from: path.join(root, "resources", "excluded.json"), to: "excluded.json" },
{ from: path.join(root, "resources", "capability-duplicates.json"), to: "capability-duplicates.json" },
{ from: path.join(root, "resources", "desktop-picker.ps1"), to: "desktop-picker.ps1" },
{ from: path.join(root, "resources", "models", "ui-element-detect"), to: "models/ui-element-detect" },
{ from: path.join(root, "resources", "fixture"), to: "fixture" }
],
asar: true,
asarUnpack: [
"node_modules/onnxruntime-node/**",
"node_modules/sharp/**",
"node_modules/@gutenye/**",
"node_modules/@napi-rs/**"
],
compression: "maximum",
npmRebuild: false,
electronVersion: packageJson.devDependencies.electron,
electronDist: path.join(root, "node_modules", "electron", "dist"),
win: {
target: [{ target: "nsis", arch: ["x64"] }],
icon: path.join(root, "build", "icon.ico"),
signAndEditExecutable: true
},
nsis: {
oneClick: false,
perMachine: false,
allowElevation: false,
allowToChangeInstallationDirectory: true,
installerLanguages: ["zh_CN"],
language: "2052",
shortcutName: "RPA Studio",
uninstallDisplayName: "RPA Studio ${version}",
createDesktopShortcut: true,
createStartMenuShortcut: true,
runAfterFinish: true,
deleteAppDataOnUninstall: false
},
publish: null
};
+7 -4
View File
@@ -6,13 +6,16 @@ export default defineConfig({
main: {
plugins: [externalizeDepsPlugin()],
resolve: {
alias: {
"@shared": resolve("src/shared")
}
alias: [
{ find: "@shared", replacement: resolve("src/shared") },
// rpa-core 包以源码形式打进主进程产物(不作为外部依赖运行时 require)
{ find: /^@rpa\/core$/, replacement: resolve("packages/rpa-core/src/index.ts") },
{ find: /^@rpa\/core\/capabilities$/, replacement: resolve("packages/rpa-core/src/capabilities/index.ts") }
]
},
build: {
rollupOptions: {
input: { index: resolve("src/main/index.ts") }
input: { index: resolve("src/main/index.ts"), "local-vision/ui-detect-worker": resolve("src/main/local-vision/ui-detect-worker.ts") }
}
}
},
+4613 -562
View File
File diff suppressed because it is too large. Load diff
+25 -2
View File
@@ -4,6 +4,9 @@
"private": true,
"description": "Canvas-based RPA desktop application",
"main": "./out/main/index.js",
"workspaces": [
"packages/*"
],
"scripts": {
"dev": "electron-vite dev",
"dev:local": "node scripts/dev-safe.mjs",
@@ -11,37 +14,57 @@
"build": "electron-vite build",
"preview": "electron-vite preview",
"typecheck": "tsc --noEmit -p tsconfig.json",
"check:deps": "node packages/rpa-core/scripts/check-deps.mjs",
"rpa-cli": "tsx scripts/rpa-cli.ts",
"catalog": "node scripts/build-catalog.mjs",
"snapshot": "vitest run scripts/capability-snapshot.test.ts",
"package": "node scripts/package.mjs",
"test": "vitest run",
"test:e2e": "node scripts/demo-e2e.mjs"
},
"dependencies": {
"@cantoo/pdf-lib": "^2.11.1",
"@gutenye/ocr-models": "^1.4.2",
"@gutenye/ocr-node": "^1.4.9",
"@napi-rs/canvas": "^1.1.0",
"@xyflow/react": "^12.12.0",
"adm-zip": "^0.5.16",
"basic-ftp": "^6.2.3",
"exceljs": "^4.4.0",
"imapflow": "^2.10.0",
"lucide-react": "^0.468.0",
"mailparser": "^3.14.1",
"mssql": "^12.7.13",
"mysql2": "^3.24.6",
"nodemailer": "^6.9.16",
"onnxruntime-node": "^1.31.1",
"pdf-lib": "^1.17.1",
"pdfjs-dist": "^4.10.38",
"pg": "^8.23.1",
"pngjs": "^7.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"sm-crypto": "^0.5.8",
"tesseract.js": "^7.0.0",
"zustand": "^5.0.2"
},
"devDependencies": {
"@electron/packager": "^18.3.6",
"@types/adm-zip": "^0.5.7",
"@types/mssql": "^12.3.0",
"@types/node": "^22.10.2",
"@types/nodemailer": "^6.4.17",
"@types/pg": "^8.23.1",
"@types/pngjs": "^6.0.5",
"@types/react": "^18.3.12",
"@types/react-dom": "^18.3.1",
"@vitejs/plugin-react": "^4.7.0",
"electron": "31.7.7",
"electron-builder": "^26.15.7",
"electron-vite": "^6.0.0",
"typescript": "^5.7.2",
"tsx": "^4.19.2",
"vite": "^7.1.0",
"vitest": "^2.1.8"
}
},
"author": "RPA Studio"
}
+61
View File
@@ -0,0 +1,61 @@
# @rpa/core
RPA 能力层包:零 Electron 依赖的流程引擎、能力注册表与能力实现。
抽离路线见 `docs/rpa-core-refactor-tasks.md`,开发规范见 `docs/dev-conventions.md`。
## 包边界
| 模块 | 说明 |
|---|---|
| `src/engine/` | 注册表、运行时、表达式、`createEngine()` 工厂、`host-bridge.ts` 宿主桥 |
| `src/driver/` | `types.ts` 五域 AutomationDriver 接口(mouse/keyboard/window/screen/element)+ `windows/powershell.ts` 进程助手 + `windows/windows-driver.ts` PowerShell 实现 |
| `src/capabilities/` | 能力实现(副作用注册,经 `@rpa/core/capabilities` 拉起) |
| `src/browser/` | 外部浏览器 CDP:`external.ts`(Edge/Chrome 进程管理)+ `transport.ts`(CdpTransport 接口、ExternalTransport) |
## HostBridge 契约(决策点 4 最小面)
宿主(Electron 主进程 / 未来 Agent-Employee product-shell)启动时注入:
```ts
import { setHostBridge, setSubflowSource } from "@rpa/core";
setHostBridge({
createFormWindow({ title }) { /* 返回 HostDialogWindow 适配(BrowserWindow) */ },
dipToScreenPoint(p) { /* screen.dipToScreenPoint */ },
screenToDipPoint(p) { /* screen.screenToDipPoint */ },
loadAiConfig() { /* settings.aiConfig */ }
});
setSubflowSource({ listFlows, getFlow }); // 子流程仓库
```
未注入时的行为:window-extended 的 DPI 换算回退恒等(与旧 vitest 行为一致);
ai-service 返回空配置(requireAiConfig 统一抛 `ai_not_configured`);
human-dialog 抛 `host_unavailable`。
## Electron 相关能力留守清单(宿主适配层,不入包)
- `src/main/browser/session.ts`:BrowserSession 编排 + 内置运行浏览器(依赖 `BrowserWindow` 与应用生命周期)
- `src/main/browser/transport.ts`:`EmbeddedTransport`(包装 `webContents.debugger`)
- `src/main/capabilities/browser*.ts` / `web-batch2.ts` / `web-crawl.ts`:经 BrowserSession 操作浏览器窗口
- `src/main/capabilities/ai-caps.ts`:AI 编排能力(依赖宿主注入的 chatCompletion 配置面,可随后续阶段评估迁入)
- `src/main/overlay.ts` / `desktop-pick.ts` / `vision-lab.ts` / `fixture/`:Electron 窗口强耦合
- `src/main/local-vision/`:onnxruntime-node 原生依赖,Bun 兼容性单独立项验证
- `src/main/flows.ts` / `run/` / `settings.ts`:应用级编排与持久化
## 依赖护栏
`npm run check:deps`(根目录亦可)扫描 `src/**`:禁止 `import "electron"`、
相对引用逃出包 `src/`、引用宿主 `src/main`。阶段 2 起为提交前必跑项。
## Driver(阶段 3)
- 五域接口 `AutomationDriver`(决策点 6):`RpaEngine.driver` 默认 `new WindowsDriver()`,
`createEngine()` 可注入自定义实现(CLI/Linux driver 等未来形态)。
- 驱动只承载**原始注入原语**(单次 SetCursorPos / SendKeys / 窗口操作);拟真抖动、
DIP 坐标换算、多步组合等策略留在能力文件,保证机械替换不改时序行为。
- element 域实现承载在 `capabilities/desktop-uia.ts`(经 `setElementDriverImplementation`
注册到 WindowsDriver),handler 层面向 driver 的收口在阶段 5/6 按需进行。
- `powershell()` 助手统一位于 `driver/windows/powershell.ts`,包内能力禁止自行
`execFile("powershell.exe")`(check:deps 与 review 双重把关)。
- 平台标记:`npm run catalog` 构建时按组填充 `platformTag: "win32"`(desktop/app/
office/vision 组),渲染层置灰延后(决策点 7)。
+43
View File
@@ -0,0 +1,43 @@
{
"name": "@rpa/core",
"version": "0.1.0",
"private": true,
"description": "RPA 能力层:零 Electron 依赖的流程引擎、能力注册表与纯 Node 能力",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts",
"./capabilities": "./src/capabilities/index.ts"
},
"scripts": {
"test": "vitest run",
"typecheck": "tsc --noEmit -p tsconfig.json",
"check:deps": "node scripts/check-deps.mjs"
},
"dependencies": {
"@gutenye/ocr-node": "^1.4.9",
"adm-zip": "^0.5.16",
"basic-ftp": "^6.2.3",
"exceljs": "^4.4.0",
"imapflow": "^2.10.0",
"mailparser": "^3.14.1",
"mssql": "^12.7.13",
"mysql2": "^3.24.6",
"nodemailer": "^6.9.16",
"pdf-lib": "^1.17.1",
"pg": "^8.23.1",
"pngjs": "^7.0.0",
"sm-crypto": "^0.5.8",
"ws": "^8.22.0"
},
"devDependencies": {
"@types/adm-zip": "^0.5.7",
"@types/mssql": "^12.3.0",
"@types/nodemailer": "^6.4.17",
"@types/pg": "^8.23.1",
"@types/pngjs": "^6.0.5",
"@types/ws": "^8.18.2",
"typescript": "^5.7.2",
"vitest": "^2.1.8"
}
}
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env node
/**
* rpa-core 依赖护栏(任务 2.4,规范 v1 第二节的机器强制版)。
*
* 禁止:
* 1. import "electron"(含 "electron/..." 子路径);
* 2. 相对 import 逃出包 src/(如指向宿主 src/main 的 ../../../src/main/...);
* 3. 任何 import 路径中出现 src/main(宿主模块横向依赖)。
*
* 用法:node scripts/check-deps.mjs(退出码非 0 表示违规)
*/
import { readdir, readFile } from "node:fs/promises";
import path from "node:path";
import process from "node:process";
import { fileURLToPath } from "node:url";
const pkgRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const srcRoot = path.join(pkgRoot, "src");
const IMPORT_RE = /(?:import|export)\s+(?:[\s\S]*?from\s*)?["']([^"']+)["']/g;
async function walk(dir) {
const entries = await readdir(dir, { withFileTypes: true });
const files = [];
for (const entry of entries) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) files.push(...(await walk(full)));
else if (/\.tsx?$/.test(entry.name)) files.push(full);
}
return files;
}
const offenders = [];
for (const file of await walk(srcRoot)) {
const raw = await readFile(file, "utf8");
// 剥掉注释,避免文档示例里的 "electron" 字样误报
const source = raw.replace(/\/\*[\s\S]*?\*\//g, "").replace(/^\s*\/\/.*$/gm, "");
for (const match of source.matchAll(IMPORT_RE)) {
const spec = match[1];
const rel = path.relative(pkgRoot, file).replaceAll("\\", "/");
if (spec === "electron" || spec.startsWith("electron/")) {
offenders.push(`${rel}: 禁止 import "${spec}"(宿主能力走 HostBridge 注入)`);
continue;
}
if (spec.includes("src/main")) {
offenders.push(`${rel}: 禁止引用宿主模块 "${spec}"`);
continue;
}
if (spec.startsWith(".")) {
const resolved = path.resolve(path.dirname(file), spec);
if (!resolved.startsWith(srcRoot + path.sep) && resolved !== srcRoot) {
offenders.push(`${rel}: 相对引用逃出包 src/ → "${spec}"`);
}
}
}
}
if (offenders.length) {
console.error("rpa-core 依赖护栏检查失败:");
for (const line of offenders) console.error(" - " + line);
process.exit(1);
}
console.log(`rpa-core 依赖护栏检查通过(src/ 下 ${offenders.length === 0 ? "无" : ""}违规 import)`);
+318
View File
@@ -0,0 +1,318 @@
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { promises as fs } from "node:fs";
import os from "node:os";
import path from "node:path";
import { RuntimeError } from "../engine/types";
import { ExternalTransport } from "./transport";
/**
* 外部浏览器(Edge / Chrome)进程管理与 CDP 连接。
*
* 关键约束(Chromium 安全限制):
* - 未带 --remote-debugging-port 启动的浏览器实例无法连接,必须由本模块带参拉起;
* - 拉起时必须带独立 --user-data-dir,否则端口参数会被用户已开的实例吞掉(连不上且无报错的头号坑)。
*/
export type ExternalBrowserType = "chrome" | "edge";
export interface ExternalLaunchOptions {
mode: "launch";
browserType: ExternalBrowserType;
/** 首选调试端口,被占用时自动 +1 重试;默认 9222 */
port?: number;
/** 透传给浏览器命令行的额外参数(如 --headless) */
extraArgs?: string[];
/** 独立用户数据目录;缺省使用系统临时目录下的独立 profile */
userDataDir?: string;
/** 首次启动时直接打开的 URL,避免先出现 about:blank */
initialUrl?: string;
}
export interface ExternalRemoteOptions {
mode: "remote";
host: string;
port: number;
}
export type ExternalBrowserOptions = ExternalLaunchOptions | ExternalRemoteOptions;
export interface ExternalConnection {
transport: ExternalTransport;
/** 会话缓存键,close 时用于移出缓存 */
sessionKey: string;
/** 人类可读标识,用于日志(如 "Edge (127.0.0.1:9222)") */
label: string;
/** 调试 HTTP 端点根地址(如 http://127.0.0.1:9222),用于 /json/list、/json/new 多标签管理 */
httpBase: string;
/** 初始接管的页面 targetId */
targetId: string;
/** 由本模块拉起的浏览器进程;远程连接时为空 */
child?: ChildProcess;
/** close 时是否结束浏览器进程(本模块拉起的默认关闭,远程连接绝不关闭) */
killOnClose: boolean;
}
const BROWSER_LABEL: Record<ExternalBrowserType, string> = { chrome: "Chrome", edge: "Edge" };
const BROWSER_CANDIDATE_PATHS: Record<ExternalBrowserType, string[]> = {
chrome: [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
path.join(localAppDataDir(), "Google", "Chrome", "Application", "chrome.exe")
],
edge: [
"C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe",
"C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe"
]
};
/** 用户级 AppData\Local 目录;LOCALAPPDATA 缺失(如受限启动环境)时退回 USERPROFILE 拼接 */
function localAppDataDir(): string {
if (process.env.LOCALAPPDATA) return process.env.LOCALAPPDATA;
return path.join(os.homedir(), "AppData", "Local");
}
/** 注册表 App Paths 兜底:HKLM(64 位 + WOW6432Node)与 HKCU(用户级安装)都查 */
const BROWSER_APP_PATHS_REGISTRIES: Record<ExternalBrowserType, string[]> = {
chrome: [
"HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\chrome.exe",
"HKLM\\SOFTWARE\\WOW6432Node\\Microsoft\\Windows\\CurrentVersion\\App Paths\\chrome.exe",
"HKCU\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\chrome.exe"
],
edge: [
"HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\msedge.exe",
"HKLM\\SOFTWARE\\WOW6432Node\\Microsoft\\Windows\\CurrentVersion\\App Paths\\msedge.exe",
"HKCU\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\App Paths\\msedge.exe"
]
};
async function fileExists(p: string): Promise<boolean> {
if (!p || p === "undefined") return false;
try {
await fs.access(p);
return true;
} catch {
return false;
}
}
function queryRegistry(command: string): Promise<string> {
return new Promise((resolve, reject) => {
execFile("reg", ["query", command, "/ve"], { windowsHide: true }, (error, stdout) => {
if (error) return reject(error);
resolve(String(stdout));
});
});
}
/** 探测本机浏览器安装路径:常见目录优先,注册表 App Paths(HKLM/HKCU)兜底 */
export async function detectBrowserPath(browserType: ExternalBrowserType): Promise<string> {
for (const candidate of BROWSER_CANDIDATE_PATHS[browserType]) {
if (await fileExists(candidate)) return candidate;
}
for (const registryKey of BROWSER_APP_PATHS_REGISTRIES[browserType]) {
try {
const stdout = await queryRegistry(registryKey);
const match = stdout.match(/REG_SZ\s+(\S[^\r\n]*)/);
if (match) {
const registryPath = match[1].trim();
if (await fileExists(registryPath)) return registryPath;
}
} catch {
// 该键不存在继续查下一个
}
}
throw new RuntimeError(
"browser_not_found",
`未找到本机 ${BROWSER_LABEL[browserType]},请确认已安装或在节点参数中填写正确端口进行远程连接`,
{ recommendation: "安装 Chrome/Edge,或改用「内置运行浏览器」" }
);
}
interface CdpVersionInfo {
Browser?: string;
webSocketDebuggerUrl?: string;
}
interface CdpTargetInfo {
id?: string;
type?: string;
url?: string;
title?: string;
webSocketDebuggerUrl?: string;
}
async function fetchJson<T>(url: string, timeoutMs = 5000, method = "GET"): Promise<T> {
const response = await fetch(url, { method, signal: AbortSignal.timeout(timeoutMs) });
if (!response.ok) {
throw new RuntimeError("browser_disconnected", `调试端点响应异常 HTTP ${response.status}:${url}`);
}
return (await response.json()) as T;
}
async function isDebugPortReady(port: number, host = "127.0.0.1"): Promise<boolean> {
try {
await fetchJson<CdpVersionInfo>(`http://${host}:${port}/json/version`, 1200);
return true;
} catch {
return false;
}
}
/** 从 /json/list 里挑一个可用的页面级 target */
function pickPageTarget(targets: CdpTargetInfo[], preferUrl?: string): CdpTargetInfo {
const pages = targets.filter((t) => t.type === "page" && t.webSocketDebuggerUrl);
if (pages.length === 0) {
throw new RuntimeError("browser_disconnected", "外部浏览器上没有可接管的页面", {
recommendation: "关闭无头/后台模式限制,或重新运行「打开网页」"
});
}
if (preferUrl) {
const preferred = pages.find((t) => (t.url ?? "") === preferUrl);
if (preferred) return preferred;
}
const blank = pages.find((t) => /^about:blank|^chrome:\/\/newtab|^edge:\/\/newtab/.test(t.url ?? ""));
return blank ?? pages[0];
}
/** 连接一个已在监听调试端口的实例(远程场景,绝不关进程) */
export async function connectRemoteBrowser(options: ExternalRemoteOptions): Promise<ExternalConnection> {
const host = options.host.trim() || "127.0.0.1";
const port = Math.trunc(options.port);
if (!port || port < 1 || port > 65535) {
throw new RuntimeError("validation_error", `远程调试端口不合法:${options.port}`, { recommendation: "填写 1~65535 之间的端口号" });
}
if (!/^[a-zA-Z0-9.\-]+$/.test(host)) {
throw new RuntimeError("validation_error", `远程地址不合法:${host}`, { recommendation: "填写 IP 或主机名,如 127.0.0.1" });
}
const base = `http://${host}:${port}`;
let version: CdpVersionInfo;
try {
version = await fetchJson<CdpVersionInfo>(`${base}/json/version`, 5000);
} catch (error) {
if (error instanceof RuntimeError) throw error;
throw new RuntimeError("browser_disconnected", `无法连接远程浏览器 ${host}:${port}`, {
detail: (error as Error).message,
recommendation: "确认目标浏览器已带 --remote-debugging-port 启动,且防火墙已放行该端口"
});
}
const targets = await fetchJson<CdpTargetInfo[]>(`${base}/json/list`, 5000);
const target = pickPageTarget(targets);
const transport = await ExternalTransport.connect(target.webSocketDebuggerUrl as string, `${host}:${port}`);
return {
transport,
sessionKey: `remote|${host}|${port}`,
label: `远程浏览器 ${host}:${port} (${version.Browser ?? "Chromium"})`,
httpBase: base,
targetId: String(target.id ?? ""),
killOnClose: false
};
}
/** 拉起本机 Edge/Chrome(带独立 profile + 调试端口),并接管其页面 */
export async function launchExternalBrowser(options: ExternalLaunchOptions): Promise<ExternalConnection> {
const browserType = options.browserType === "edge" ? "edge" : "chrome";
const label = BROWSER_LABEL[browserType];
const browserPath = await detectBrowserPath(browserType);
// 端口管理:首选端口被占用(存在调试端点)则 +1 重试
let port = Math.trunc(options.port ?? 0) > 0 ? Math.trunc(options.port as number) : 9222;
for (let attempt = 0; attempt < 20; attempt++) {
if (await isDebugPortReady(port)) port += 1;
else break;
}
const userDataDir =
options.userDataDir?.trim() || path.join(os.tmpdir(), `rpa-studio-external-${browserType}`);
await fs.mkdir(userDataDir, { recursive: true });
const extraArgs = (options.extraArgs ?? []).filter(Boolean);
const startUrl = options.initialUrl?.trim() || "about:blank";
const args = [
`--remote-debugging-port=${port}`,
`--user-data-dir=${userDataDir}`,
"--no-first-run",
"--no-default-browser-check",
startUrl,
...extraArgs
];
const child = spawn(browserPath, args, { detached: false, stdio: "ignore", windowsHide: true });
child.on("error", () => {
// spawn 失败由下方就绪轮询超时统一报错
});
// 就绪轮询:等 /json/version 可访问(最长 15s)
const deadline = Date.now() + 15000;
let ready = false;
while (Date.now() < deadline) {
if (child.exitCode !== null) {
throw new RuntimeError("browser_error", `${label} 启动失败(进程已退出,代码 ${child.exitCode})`, {
detail: browserPath,
recommendation: "检查浏览器安装是否完整,或参数是否合法"
});
}
if (await isDebugPortReady(port)) {
ready = true;
break;
}
await new Promise((resolve) => setTimeout(resolve, 300));
}
if (!ready) {
try {
child.kill();
} catch {
// 进程可能已退出
}
throw new RuntimeError("timeout", `${label} 已启动但调试端口 ${port} 未就绪(15s 超时)`, {
recommendation: "重启电脑后重试,或换一个端口;也可能是安全软件拦截了调试端口"
});
}
const base = `http://127.0.0.1:${port}`;
const targets = await fetchJson<CdpTargetInfo[]>(`${base}/json/list`, 5000);
const target = pickPageTarget(targets, "about:blank");
const transport = await ExternalTransport.connect(target.webSocketDebuggerUrl as string, `127.0.0.1:${port}`);
return {
transport,
sessionKey: `launch|${browserType}|${port}`,
label: `${label} (127.0.0.1:${port})`,
httpBase: base,
targetId: String(target.id ?? ""),
child,
killOnClose: true
};
}
/** 列出调试端点上的全部页面级 target(多标签枚举用);排除 chrome://、edge:// 内部页 */
export async function listPageTargets(httpBase: string): Promise<Array<{ id: string; title: string; url: string; webSocketDebuggerUrl: string }>> {
const targets = await fetchJson<CdpTargetInfo[]>(`${httpBase}/json/list`, 5000);
return targets
.filter((t) => t.type === "page" && t.webSocketDebuggerUrl)
.map((t) => ({ id: String(t.id ?? ""), title: String(t.title ?? ""), url: String(t.url ?? ""), webSocketDebuggerUrl: String(t.webSocketDebuggerUrl) }))
.filter((t) => !/^chrome:\/\/|^edge:\/\/|^devtools:\/\//.test(t.url) || /^chrome:\/\/newtab|^edge:\/\/newtab/.test(t.url));
}
/** 在外部浏览器上新建标签页并返回其调试端点(PUT /json/new) */
export async function openNewPageTarget(httpBase: string, url = "about:blank"): Promise<{ id: string; webSocketDebuggerUrl: string; url: string }> {
const endpoint = `${httpBase}/json/new?${encodeURIComponent(url)}`;
const target = await fetchJson<{ id?: string; webSocketDebuggerUrl?: string; url?: string }>(endpoint, 5000, "PUT");
if (!target.webSocketDebuggerUrl) {
throw new RuntimeError("browser_error", "新建标签页未返回调试端点", { detail: JSON.stringify(target) });
}
return {
id: String(target.id ?? ""),
webSocketDebuggerUrl: target.webSocketDebuggerUrl,
url: String(target.url ?? url)
};
}
export async function openExternalBrowser(options: ExternalBrowserOptions): Promise<ExternalConnection> {
return options.mode === "remote" ? connectRemoteBrowser(options) : launchExternalBrowser(options);
}
export function sessionKeyOf(options: ExternalBrowserOptions): string {
return options.mode === "remote"
? `remote|${options.host.trim()}|${Math.trunc(options.port)}`
: `launch|${options.browserType === "edge" ? "edge" : "chrome"}|${Math.trunc(options.port ?? 9222)}`;
}
+180
View File
@@ -0,0 +1,180 @@
import WebSocket from "ws";
import { RuntimeError } from "../engine/types";
/**
* CDP 传输层抽象。
*
* 上层 BrowserSession(宿主适配层)的全部 CDP 通信都经由本接口:
* - EmbeddedTransport:内置运行浏览器(Electron BrowserWindow.webContents.debugger),留宿主
* - ExternalTransport:外部 Edge/Chrome 浏览器(WebSocket 直连调试端口),在本包内
*
* 两者只在传输方式上不同,CDP 协议域(DOM/Input/Page/Network/Runtime)完全一致。
*/
export type CdpEventHandler = (params: Record<string, unknown>) => void;
export interface CdpTransport {
readonly kind: "embedded" | "external";
/** 连接是否已失效(窗口销毁 / ws 断开) */
readonly closed: boolean;
send<T = unknown>(method: string, params?: Record<string, unknown>): Promise<T>;
/** 订阅 CDP 事件(如 Page.loadEventFired、Network.responseReceived),handler 只收到对应 method 的 params */
on(method: string, handler: CdpEventHandler): void;
off(method: string, handler: CdpEventHandler): void;
close(): Promise<void>;
}
export function mapCdpError(error: unknown, kind: "embedded" | "external"): RuntimeError {
const message = (error as Error)?.message || String(error);
if (/destroyed|Target closed|not attached|connection.*(closed|lost)|websocket/i.test(message)) {
if (kind === "external") {
return new RuntimeError("browser_disconnected", "外部浏览器连接已断开", {
detail: message,
recommendation: "确认浏览器仍处于打开状态;若被手动关闭请重新运行「打开网页」"
});
}
return new RuntimeError("tab_unavailable", "浏览器连接已失效", { detail: message });
}
return new RuntimeError("browser_error", message, { recommendation: "重试或检查页面状态" });
}
interface CdpMessage {
id?: number;
method?: string;
params?: Record<string, unknown>;
result?: unknown;
error?: { message?: string; data?: unknown };
}
/** 外部 Edge/Chrome:WebSocket 直连页面级 CDP 端点(webSocketDebuggerUrl) */
export class ExternalTransport implements CdpTransport {
readonly kind = "external" as const;
private ws: WebSocket | null = null;
private nextId = 1;
private pending = new Map<number, { resolve: (value: unknown) => void; reject: (error: Error) => void }>();
private handlers = new Map<string, Set<CdpEventHandler>>();
private closedFlag = false;
private readonly openPromise: Promise<void>;
private constructor(
wsUrl: string,
readonly description: string
) {
this.openPromise = new Promise<void>((resolve, reject) => {
const ws = new WebSocket(wsUrl, { maxPayload: 256 * 1024 * 1024 });
this.ws = ws;
ws.on("open", () => resolve());
ws.on("message", (data: unknown) => this.handleMessage(String(data)));
ws.on("error", (error: Error) => {
this.closedFlag = true;
this.failAllPending(new Error(`外部浏览器连接异常:${error.message}`));
reject(error);
});
ws.on("close", () => {
this.closedFlag = true;
this.failAllPending(new Error("外部浏览器连接已关闭"));
});
});
}
static async connect(wsUrl: string, description: string, timeoutMs = 10000): Promise<ExternalTransport> {
const transport = new ExternalTransport(wsUrl, description);
const timeout = new Promise<never>((_, reject) =>
setTimeout(() => reject(new RuntimeError("browser_disconnected", `连接外部浏览器超时:${description}`)), timeoutMs)
);
await Promise.race([transport.openPromise, timeout]);
return transport;
}
get closed(): boolean {
return this.closedFlag || !this.ws || this.ws.readyState !== this.ws.OPEN;
}
async send<T = unknown>(method: string, params: Record<string, unknown> = {}): Promise<T> {
if (this.closed) {
throw new RuntimeError("browser_disconnected", "外部浏览器连接已断开", {
recommendation: "确认浏览器未被手动关闭,然后重新运行「打开网页」"
});
}
const id = this.nextId++;
const ws = this.ws as WebSocket;
const reply = new Promise<unknown>((resolve, reject) => {
this.pending.set(id, { resolve, reject });
try {
ws.send(JSON.stringify({ id, method, params }));
} catch (error) {
this.pending.delete(id);
reject(error as Error);
}
});
try {
return (await reply) as T;
} catch (error) {
throw mapCdpError(error, "external");
}
}
on(method: string, handler: CdpEventHandler): void {
if (!this.handlers.has(method)) this.handlers.set(method, new Set());
this.handlers.get(method)?.add(handler);
}
off(method: string, handler: CdpEventHandler): void {
this.handlers.get(method)?.delete(handler);
}
async close(): Promise<void> {
const ws = this.ws;
this.closedFlag = true;
this.failAllPending(new Error("外部浏览器连接已主动关闭"));
if (!ws) return;
await new Promise<void>((resolve) => {
if (ws.readyState === ws.CLOSED) return resolve();
ws.once("close", () => resolve());
try {
ws.close();
} catch {
resolve();
}
setTimeout(() => resolve(), 1500);
});
}
private handleMessage(raw: string): void {
let message: CdpMessage;
try {
message = JSON.parse(raw) as CdpMessage;
} catch {
return;
}
if (typeof message.id === "number") {
const entry = this.pending.get(message.id);
if (!entry) return;
this.pending.delete(message.id);
if (message.error) {
entry.reject(new Error(message.error.message || "CDP 调用失败"));
} else {
entry.resolve(message.result);
}
return;
}
if (message.method) {
const handlers = this.handlers.get(message.method);
if (!handlers) return;
const params = message.params ?? {};
for (const handler of handlers) {
try {
handler(params);
} catch {
// 单个监听器异常不影响其他监听器
}
}
}
}
private failAllPending(error: Error): void {
for (const [id, entry] of this.pending) {
entry.reject(error);
this.pending.delete(id);
}
}
}
@@ -0,0 +1,106 @@
import { getHostBridge } from "../engine/host-bridge";
import { RuntimeError } from "../engine/types";
export type ChatContentPart =
| { type: "text"; text: string }
| { type: "image_url"; image_url: { url: string } };
export interface ChatMessage {
role: "system" | "user" | "assistant";
content: string | ChatContentPart[];
}
export interface ResolvedAiConfig {
baseUrl: string;
apiKey: string;
model: string;
visionModel: string;
}
export function getAiConfig(): ResolvedAiConfig {
const bridge = getHostBridge();
// 宿主桥未注入(vitest / CLI 未配置)时返回空配置,requireAiConfig 统一抛 ai_not_configured
const config = bridge ? bridge.loadAiConfig() : { baseUrl: "", apiKey: "", model: "", visionModel: "" };
return { ...config, visionModel: config.visionModel || config.model };
}
export function requireAiConfig(options: { vision?: boolean } = {}): ResolvedAiConfig {
const config = getAiConfig();
if (!config.baseUrl || !config.model || (options.vision && !config.visionModel)) {
throw new RuntimeError("ai_not_configured", "AI 服务未配置", {
recommendation: options.vision
? "到 设置 → AI 服务 填写 Base URL 与多模态视觉模型(兼容 OpenAI 协议)"
: "到 设置 → AI 服务 填写 Base URL 与模型名(兼容 OpenAI 协议)"
});
}
return config;
}
export interface ChatCompletionOptions {
temperature?: number;
vision?: boolean;
/** 交互式请求(用户正在等待,如 AI 拾取):默认短超时 45s、最多 2 次,超时不重试 */
interactive?: boolean;
/** 单次请求超时毫秒数;显式指定后覆盖交互式/后台默认值 */
timeoutMs?: number;
/** 最大尝试次数;显式指定后覆盖交互式/后台默认值 */
maxAttempts?: number;
}
export async function chatCompletion(messages: ChatMessage[], options: ChatCompletionOptions = {}): Promise<string> {
const config = requireAiConfig({ vision: options.vision });
const base = config.baseUrl.replace(/\/+$/, "");
const interactive = options.interactive === true;
const timeoutMs = Math.max(1000, Math.round(options.timeoutMs ?? (interactive ? 45000 : 170000)));
const maxAttempts = Math.max(1, Math.round(options.maxAttempts ?? (interactive ? 2 : 3)));
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
let response: Response;
try {
response = await fetch(base + "/chat/completions", {
signal: AbortSignal.timeout(timeoutMs),
method: "POST",
headers: { "Content-Type": "application/json", Authorization: "Bearer " + config.apiKey },
body: JSON.stringify({
model: options.vision ? config.visionModel : config.model,
messages,
temperature: options.temperature ?? 0.3
})
});
} catch (error) {
// 超时(AbortSignal.timeout)不重试:重发只会成倍拉长等待
if ((error as { name?: string }).name === "TimeoutError") {
throw new RuntimeError("ai_request_timeout", `AI 请求超时(${timeoutMs}ms),请检查网络或稍后重试`);
}
if (attempt === maxAttempts) throw error;
await new Promise(resolve => setTimeout(resolve, 1200 * attempt));
continue;
}
if (response.ok) {
const data = (await response.json()) as { choices?: Array<{ message?: { content?: string } }> };
return data.choices?.[0]?.message?.content ?? "";
}
const detail = (await response.text()).slice(0, 300);
const retryable = response.status === 408 || response.status === 429 || response.status >= 500;
if (retryable && attempt < maxAttempts) {
const retryAfter = Number(response.headers.get("retry-after"));
const fallbackDelay = 1400 * attempt;
const delay = Number.isFinite(retryAfter) && retryAfter >= 0 && retryAfter <= 15
? retryAfter * 1000
: fallbackDelay;
await new Promise(resolve => setTimeout(resolve, delay));
continue;
}
if (response.status === 429) {
const suffix = detail ? " 详情:" + detail.slice(0, 180) : "";
throw new RuntimeError("ai_rate_limited", "AI 服务限流或配额不足(HTTP 429),请稍后重试或更换可用视觉模型。" + suffix);
}
throw new RuntimeError("ai_request_failed", "AI 请求失败(HTTP " + response.status + "):" + detail);
}
throw new RuntimeError("ai_request_failed", "AI 请求失败:已重试仍未成功");
}
@@ -0,0 +1,85 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
const mocks = vi.hoisted(() => ({ powershell: vi.fn(), execFile: vi.fn() }));
vi.mock("../driver/windows/powershell", () => ({ powershell: mocks.powershell, psString: (value: unknown) => `'${String(value ?? "").replace(/'/g, "''")}'` }));
vi.mock("node:child_process", () => ({ execFile: mocks.execFile }));
import "./data";
import "./control";
import "./system-extended";
import "./windows-system";
function run(id: string, args: Record<string, unknown> = {}, signal = new AbortController().signal) {
const node: FlowNode = { id: "regression", componentId: id, values: args, options: {}, position: { x: 0, y: 0 } };
return getHandler(id)!({ signal, log: vi.fn(), vars: {} } as unknown as RunContext, node, args, {} as CatalogComponent);
}
afterEach(() => { vi.useRealTimers(); vi.resetAllMocks(); });
describe("应用与系统组件缺陷回归", () => {
it("没有匹配进程时返回 [] / null,忽略空行和非法 PID", async () => {
mocks.powershell.mockResolvedValue("");
await expect(run("App_GetPids", { pattern: "missing.exe" })).resolves.toEqual([]);
await expect(run("App_GetPid", { pattern: "missing.exe" })).resolves.toBeNull();
mocks.powershell.mockResolvedValue("\r\n0\r\n123\r\nabc\r\n456\r\n");
await expect(run("App_GetPids", { pattern: "fixture.exe" })).resolves.toEqual([123, 456]);
});
it("毫秒等待读取 mileSecond,计时和取消均正确", async () => {
vi.useFakeTimers();
const waiting = Promise.resolve(run("Basic_Delay", { mileSecond: "250" }));
let done = false;
void waiting.then(() => { done = true; });
await vi.advanceTimersByTimeAsync(249);
expect(done).toBe(false);
await vi.advanceTimersByTimeAsync(1);
expect(done).toBe(true);
const controller = new AbortController();
const cancelled = Promise.resolve(run("Basic_Delay", { mileSecond: "3000" }, controller.signal));
const rejection = expect(cancelled).rejects.toMatchObject({ code: "cancelled" });
controller.abort();
await rejection;
});
it.each([
[42, "number", true], ["42", "number", false], [42, "1", true],
[[], "list", true], [{ key: 1 }, "dict", true], [null, "null", true],
[false, "bool", true], ["中文", "string", true], [3.5, "int", false]
])("按目标类型判断 %j → %s", (val, into_type, expected) => {
expect(run("Basic_JudgeDataType", { val, into_type })).toBe(expected);
});
it.each([
["42.5", "number", 42.5], ["42", "1", 42], ["-3.9", "int", -3],
["false", "bool", false], ["true", "bool", true], [42, "string", "42"]
])("执行类型转换 %j → %s", (val, into_type, expected) => {
expect(run("Basic_DataTypeConversion", { val, into_type })).toBe(expected);
});
it("非法数值或未知目标类型给出定位错误,避免静默返回原值", () => {
expect(() => run("Basic_DataTypeConversion", { val: "abc", into_type: "int" })).toThrow();
expect(() => run("Basic_DataTypeConversion", { val: 1, into_type: "unknown" })).toThrow();
expect(() => run("Basic_JudgeDataType", { val: 1, into_type: "unknown" })).toThrow();
});
it("关闭组件按所选模式执行,非法 PID 不降级为按名称批量关闭", async () => {
mocks.powershell.mockResolvedValue("");
await expect(run("App_KillV1", { close_type: "id", pid: "invalid", p_name: "fixture.exe" })).rejects.toMatchObject({ code: "validation_error" });
expect(mocks.powershell).not.toHaveBeenCalled();
await run("App_KillV1", { close_type: "name", pid: 123, p_name: "fixture.exe", kill_all: "0" });
expect(mocks.powershell.mock.calls[0][0]).toContain("SessionId");
expect(mocks.powershell.mock.calls[0][0]).not.toContain("Stop-Process -Id 123");
});
it("唤醒不声称已通过 Windows 登录解锁", async () => {
mocks.execFile.mockImplementation((_file, _args, _opts, callback) => callback(null, { stdout: "", stderr: "" }));
await expect(run("ScreenControl_ScreenUnlock")).resolves.toMatchObject({ awakened: true, loginUnlockSupported: false });
});
it("屏保自定义密码不受支持时,在触发桌面操作前明确报错", async () => {
await expect(run("System_ScreenSaverShowV1", { set_password: "1", password: "" })).rejects.toMatchObject({ code: "validation_error" });
expect(mocks.execFile).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,157 @@
import { describe, expect, it } from "vitest";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./data";
import "./desktop";
import "./system-extended";
import "./utility-extended";
import "./common-final";
import "./control";
import "./windows-system";
import "./basic-encryption";
const ctx = {
signal: new AbortController().signal,
log: () => {},
vars: {},
evidenceDir: "D:/office-test2",
runId: "smoke-run",
flow: {
id: "smoke-flow", name: "冒烟测试流程", description: "", enabled: true,
trigger: null, meta: { appVersion: "dev", createdAt: "", updatedAt: "" },
nodes: []
}
} as unknown as RunContext;
const node = { id: "smoke-app", componentId: "smoke", position: { x: 0, y: 0 }, values: {}, options: {} } as unknown as FlowNode;
async function run(componentId: string, args: Record<string, unknown>, ctxOverride?: RunContext): Promise<unknown> {
const handler = getHandler(componentId);
if (!handler) throw new Error(`组件 ${componentId} 未注册运行时`);
const n = { ...node, componentId } as unknown as FlowNode;
return handler(ctxOverride ?? ctx, n, args, {} as CatalogComponent);
}
describe("应用与系统组件冒烟", () => {
it("数据类型判断族", async () => {
expect(await run("Basic_IsNumber", { val: "12.5" })).toBe(true);
expect(await run("Basic_IsNumber", { val: "abc" })).toBe(false);
expect(await run("Basic_IsNull", { val: "" })).toBe(true);
expect(await run("Basic_IsNull", { val: "x" })).toBe(false);
expect(await run("Basic_IsArray", { val: [1, 2] })).toBe(true);
expect(await run("Basic_IsDictionary", { val: { a: 1 } })).toBe(true);
expect(await run("Basic_Type", { val: [1] })).toBe("list");
expect(await run("Basic_Type", { val: "s" })).toBe("string");
expect(await run("Basic_JudgeDataType", { val: 5, into_type: "int" })).toBe(true);
expect(await run("Basic_JudgeDataType", { val: 5.5, into_type: "int" })).toBe(false);
console.log("[数据判断族] 通过");
});
it("数据转换族", async () => {
expect(await run("Basic_ConvertString", { val: 123 })).toBe("123");
expect(await run("Basic_ConvertInt", { val: "12.9" })).toBe(12);
expect(await run("Basic_ConvertBool", { val: 1 })).toBe(true);
expect(await run("Basic_ConvertNumber", { val: "3.14" })).toBe(3.14);
expect(await run("Basic_DataTypeConversion", { val: "是", into_type: "bool" })).toBe(true);
expect(await run("Basic_DataTypeConversion", { val: 8, into_type: "string" })).toBe("8");
console.log("[数据转换族] 通过");
});
it("复制/随机/密文", async () => {
const src = { a: [1, { b: 2 }] };
const cloned = (await run("Basic_Clone", { val: src })) as typeof src;
expect(cloned).toEqual(src);
expect(cloned).not.toBe(src);
const ri = (await run("Basic_RandomInt", { min: 5, max: 5 })) as number;
expect(ri).toBe(5);
const rnd = (await run("Basic_Rnd", {})) as number;
expect(rnd).toBeGreaterThanOrEqual(0);
expect(rnd).toBeLessThan(1);
expect(await run("Basic_Dodecryption", { words: Buffer.from("hello-世界", "utf8").toString("base64") })).toBe("hello-世界");
console.log("[复制/随机/密文] 通过");
});
it("延迟/调试输出/设置变量", async () => {
const logs: string[] = [];
const ctxLog = { ...ctx, log: (m: string) => logs.push(m), vars: {} } as unknown as RunContext;
const t0 = Date.now();
expect(await run("Basic_Delay", { mileSecond: 100 })).toBe(0.1);
expect(Date.now() - t0).toBeGreaterThanOrEqual(95);
await run("Basic_DebugOutput", { msg: "冒烟输出" }, ctxLog);
expect(logs.join()).toContain("冒烟输出");
const setNode = { ...node, options: { outputVar: "v_test" } } as unknown as FlowNode;
const handler = getHandler("Basic_SetVariable_V1")!;
expect(await handler(ctxLog, setNode, { value: "42", variableType: "rpa_int" }, {} as CatalogComponent)).toBe(42);
expect((ctxLog as unknown as { vars: Record<string, unknown> }).vars.v_test).toBe(42);
console.log("[延迟/输出/变量] 通过");
});
it("自定义配置与业务计数", async () => {
const cfg = (await run("CustomConfig_GetConfig", {
jsonlist: JSON.stringify([{ name: "host", value: "127.0.0.1" }]),
configName: "host"
})) as string;
expect(cfg).toBe("127.0.0.1");
const ctxCount = { ...ctx, vars: {} } as unknown as RunContext;
expect(await run("Basic_BusinessOrderCount", { start_count: 100, count: 3 }, ctxCount)).toBe(103);
expect(await run("Basic_BusinessOrderCount", { start_count: 100, count: 3 }, ctxCount)).toBe(106);
console.log("[配置/计数] 通过");
});
it("系统路径与环境变量", async () => {
expect(String(await run("System_GetTempPath", {}))).toMatch(/Temp|tmp/i);
expect(String(await run("System_GetHomePath", {}))).toMatch(/Users|home/i);
expect(String(await run("System_GetSystemPath", { path_name: "desktop" }))).toMatch(/Desktop/);
expect(await run("System_GetEnv", { name: "PATH" })).toBeTruthy();
expect(await run("System_SetEnv", { name: "RPA_SMOKE", value: "ok" })).toBe("ok");
expect(await run("System_GetEnv", { name: "RPA_SMOKE" })).toBe("ok");
expect(await run("Basic_GetProcessInfo", {})).toBeTruthy();
console.log("[系统路径/环境变量] 通过");
});
it("输入法状态获取", async () => {
const st = (await run("System_GetInputMethodState", {})) as { culture: string; layout: string };
console.log("[输入法]", JSON.stringify(st));
expect(st.culture).toBeTruthy();
}, 60000);
it("执行命令行两种组件", async () => {
const out1 = String(await run("System_ExecuteCommand", { command: "echo rpa-smoke-123" }));
expect(out1).toContain("rpa-smoke-123");
const out2 = String(await run("System_ExecuteCommandV1", { command_type: "1", dos_command: "echo dos-ok-456" }));
expect(out2).toContain("dos-ok-456");
const out3 = String(await run("System_ExecuteCommandV1", { command_type: "3", ps_command: "Write-Output 'ps-ok-789'" }));
expect(out3).toContain("ps-ok-789");
console.log("[执行命令] 通过");
}, 120000);
it("应用生命周期: 启动→查PID→关闭", async () => {
const pid = (await run("App_Run", { executeable: "notepad.exe" })) as number;
expect(Number.isInteger(pid)).toBe(true);
await new Promise((r) => setTimeout(r, 1200));
// Win11 notepad.exe 为应用别名:Start-Process 返回的启动器 PID 会立即退出,真实进程另获新 PID,
// 因此不比较具体 PID,只验证「能查到运行中的 notepad 进程」
const single = (await run("App_GetPid", { pattern: "notepad" })) as number | null;
expect(single).toBeTruthy();
const all = (await run("App_GetPids", { p_name: "notepad" })) as number[];
expect(all.length).toBeGreaterThan(0);
const killed = await run("App_KillV1", { close_type: "name", p_name: "notepad" });
console.log("[应用生命周期] 启动PID:", pid, "查询到:", all, "关闭:", JSON.stringify(killed));
await new Promise((r) => setTimeout(r, 800));
const after = (await run("App_GetPids", { pattern: "notepad" })) as number[];
expect(after.length).toBe(0);
}, 120000);
it("Windows凭据: 添加→读取→删除", async () => {
const target = "RPA_SMOKETEST_GENERIC";
expect((await run("System_AddCredential", { application: target, name: "smoke-user", password: "p@ss" })) as { added: boolean }).toEqual({ added: true, target });
const read = (await run("System_ReadCredential", { application: target })) as { exists: boolean; detail: string };
expect(read.exists).toBe(true);
expect(read.detail).toContain("smoke-user");
expect((await run("System_DeleteCredential", { application: target })) as { deleted: boolean }).toEqual({ deleted: true, target });
const readAfter = (await run("System_ReadCredential", { application: target })) as { exists: boolean };
expect(readAfter.exists).toBe(false);
console.log("[凭据闭环] 通过");
}, 120000);
});
@@ -0,0 +1,50 @@
import { Buffer } from "node:buffer";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
const SM4_KEY = "INPUT_PARAM_KEY4";
const SM4_KEY_HEX = Buffer.from(SM4_KEY, "utf8").toString("hex");
function looksLikeBase64(value: string): boolean {
const text = value.trim().replace(/\s+/g, "");
if (!text || text.length % 4 !== 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(text)) return false;
return Buffer.from(text, "base64").toString("base64").replace(/=+$/, "") === text.replace(/=+$/, "");
}
function readableUtf8(value: string): boolean {
if (!value) return false;
let controls = 0;
for (const char of value) {
const code = char.codePointAt(0)!;
if (code < 32 && !["\n", "\r", "\t"].includes(char)) controls += 1;
if (code === 0xfffd) controls += 10;
}
return controls === 0;
}
async function sm4Decrypt(value: string): Promise<string> {
const { sm4 } = await import("sm-crypto");
const errors: string[] = [];
for (const padding of ["pkcs#7", "none"] as const) {
try {
const output = sm4.decrypt(value.toLowerCase(), SM4_KEY_HEX, { padding });
if (output && readableUtf8(output)) return output;
errors.push(`${padding}: empty or non-text output`);
} catch (error) {
errors.push(`${padding}: ${error instanceof Error ? error.message : String(error)}`);
}
}
throw new RuntimeError("decryption_failed", "SM4 密文解密失败", { detail: errors.join("; ") });
}
const dodecryption: CapabilityHandler = async (_ctx, node, args) => {
const words = String(args.words ?? args.value ?? args.text ?? "");
if (!words.trim()) throw new RuntimeError("validation_error", "密文内容为空", { nodeId: node.id, componentId: node.componentId });
if (looksLikeBase64(words)) {
const decoded = Buffer.from(words, "base64").toString("utf8");
if (readableUtf8(decoded)) return decoded;
}
return sm4Decrypt(words.trim());
};
register("Basic_Dodecryption", dodecryption);
@@ -0,0 +1,31 @@
import { describe, expect, it } from "vitest";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./desktop";
import "./basic-encryption";
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown>) {
const handler = getHandler(id);
expect(handler, id).toBeTruthy();
return handler!(context, node(id), args, component);
}
describe("manual keyboard and encrypted variables", () => {
it("registers manual keyboard switches", async () => {
await expect(run("Keyboard_EnableManualOperation", { random_move: true, random_click: true, random_sleep: false, min_time: 0, max_time: 0 })).resolves.toMatchObject({ enabled: true });
await expect(run("Keyboard_DisableManualOperation", {})).resolves.toEqual({ enabled: false });
});
it("decodes base64 and SM4 ciphertext", async () => {
await expect(run("Basic_Dodecryption", { words: Buffer.from("hello", "utf8").toString("base64") })).resolves.toBe("hello");
const { sm4 } = await import("sm-crypto");
const key = Buffer.from("INPUT_PARAM_KEY4", "utf8").toString("hex");
const cipher = sm4.encrypt("secret", key, { padding: "pkcs#7" });
await expect(run("Basic_Dodecryption", { words: cipher })).resolves.toBe("secret");
await expect(run("Basic_Dodecryption", {})).rejects.toThrow();
});
});
@@ -7,7 +7,7 @@ import { PDFDocument } from "pdf-lib";
import { PNG } from "pngjs";
import { getHandler } from "../engine/registry";
import type { RunContext } from "../engine/types";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import "./desktop";
import "./office";
import "./pdf";
@@ -81,6 +81,7 @@ describe("capability registration smoke", () => {
const execute = getHandler("System_ExecuteCommand");
expect(execute).toBeTruthy();
await expect(execute!(context, node("System_ExecuteCommand"), { action: "Write-Output smoke" }, component)).resolves.toContain("smoke");
await expect(execute!(context, node("System_ExecuteCommand"), { action: "Write-Output '中文输出测试'" }, component)).resolves.toBe("中文输出测试");
});
it("finds an exact PNG template inside a larger screenshot", async () => {
@@ -0,0 +1,39 @@
import { createServer } from "node:http";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { PDFDocument } from "pdf-lib";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./index";
const dirs: string[] = []; const context = { vars: {} } as RunContext; const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown> = {}) { const handler = getHandler(id); expect(handler, id).toBeTruthy(); return handler!(context, node(id), args, component); }
afterEach(async () => { context.vars = {}; await Promise.all(dirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true }))); });
describe("final common capabilities", () => {
it("registers window, keyboard, clipboard, system, PDF and webhook components", () => {
for (const id of ["Window_ActiveWindow", "Window_CloseWindow_v1", "Window_GetScreenSize", "Window_WindowList", "Window_GetRect", "Window_MaximumSize", "Window_SetWindowShow", "Window_WaitWindow", "Keyboard_DriveSend", "Keyboard_DriveSend_V1", "Keyboard_DriveSend_V2", "Clipboard_SetImage", "Clipboard_SetImageV1", "Clipboard_SaveImage", "Clipboard_SaveImageV1", "System_PlaySound", "CustomConfig_GetConfig", "Basic_BusinessOrderCount", "PDF_ConcatPdfV1", "PDF_SetPdfWaterMark", "PDF_SetPdfWaterMarkV1", "MessageNotification_DingTalk_Message", "MessageNotification_Feishu_Message", "MessageNotification_WXWork_Message"]) expect(getHandler(id), id).toBeTruthy();
});
it("reads custom config and increments business counters", async () => {
await expect(run("CustomConfig_GetConfig", { jsonlist: [{ name: "timeout", value: 30 }], configName: "timeout" })).resolves.toBe(30);
await expect(run("Basic_BusinessOrderCount", { start_count: 10, count: 2 })).resolves.toBe(12);
await expect(run("Basic_BusinessOrderCount", { start_count: 10, count: 2 })).resolves.toBe(14);
});
it("adds an ASCII PDF watermark", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-common-final-")); dirs.push(directory); const input = path.join(directory, "input.pdf"); const pdf = await PDFDocument.create(); pdf.addPage([300, 300]); await writeFile(input, await pdf.save());
const output = await run("PDF_SetPdfWaterMark", { fileInPath: input, markTextInfo: "RPA", newPdfName: "watermark.pdf" }) as string;
expect((await readFile(output)).byteLength).toBeGreaterThan(100);
});
it("posts the three webhook formats to a local server", async () => {
const bodies: unknown[] = []; const server = createServer((request, response) => { let raw = ""; request.on("data", (chunk) => raw += chunk); request.on("end", () => { bodies.push(JSON.parse(raw)); response.writeHead(200, { "Content-Type": "application/json" }); response.end('{"ok":true}'); }); });
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve)); const address = server.address(); const webhook = `http://127.0.0.1:${typeof address === "object" && address ? address.port : 0}/hook`;
try { await run("MessageNotification_DingTalk_Message", { webhook, content_text: "ding" }); await run("MessageNotification_Feishu_Message", { webhook, content_text: "fei" }); await run("MessageNotification_WXWork_Message", { webhook, content_text: "wx" }); expect(bodies).toHaveLength(3); } finally { await new Promise<void>((resolve) => server.close(() => resolve())); }
});
});
@@ -0,0 +1,92 @@
import { createHmac } from "node:crypto";
import { promises as fs } from "node:fs";
import path from "node:path";
import { PDFDocument, StandardFonts, degrees, rgb } from "pdf-lib";
import { getHandler, register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell, psString } from "../driver/windows/powershell";
import { defaultEngine } from "../engine/engine";
// 窗口/屏幕查询统一走驱动(任务 3.3)
const driver = defaultEngine.driver;
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) { const value = args[key]; if (value !== undefined && value !== null && value !== "") return String(value); }
return fallback;
}
function delegate(baseId: string): CapabilityHandler {
return (ctx, node, args, component) => { const base = getHandler(baseId); if (!base) throw new RuntimeError("capability_not_implemented", `基础能力不可用:${baseId}`, { nodeId: node.id, componentId: node.componentId }); return base(ctx, node, args, component); };
}
register("Window_ActiveWindow", delegate("Window_SetActive"));
register("Window_CloseWindow_v1", delegate("Window_CloseWindow"));
register("Window_GetRect", delegate("Window_GetWindowRect"));
register("Window_MaximumSize", (ctx, node, args, component) => delegate("Window_SetWindowState")(ctx, node, { ...args, WindowType: "maximize" }, component));
register(["Keyboard_DriveSend", "Keyboard_DriveSend_V1", "Keyboard_DriveSend_V2"], delegate("Keyboard_Send_V1"));
register("Window_GetScreenSize", async () => driver.screen.size());
register("Window_WindowList", async (_ctx, _node, args) => {
const title = text(args, ["match_title", "select_title"]); const regex = String(args.is_regex ?? "0") === "1";
return driver.window.list({ title, regex });
});
register("Window_SetWindowShow", async (_ctx, node, args) => {
const title = text(args, ["窗口对象", "window_obj", "title"]); if (!title || title === "窗口对象") throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
const show = !text(args, ["WindowType"], "show").toLowerCase().includes("hide");
await driver.window.setState(title, show ? 5 : 0);
return show;
});
register("Window_WaitWindow", async (_ctx, node, args) => {
const title = text(args, ["match_title", "select_title", "window_obj", "target_window"]); if (!title) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
const expected = text(args, ["switchType"], "0") !== "1"; const deadline = Date.now() + Math.max(0, Number(args.second ?? 5)) * 1000;
do { const exists = await driver.window.exists(title); if (exists === expected) return true; await new Promise((resolve) => setTimeout(resolve, 200)); } while (Date.now() <= deadline);
return false;
});
const setClipboardImage: CapabilityHandler = async (_ctx, node, args) => {
const file = path.resolve(text(args, ["path"])); await fs.access(file).catch(() => { throw new RuntimeError("validation_error", `图片不存在:${file}`, { nodeId: node.id, componentId: node.componentId }); });
await powershell(`Add-Type -AssemblyName System.Windows.Forms;Add-Type -AssemblyName System.Drawing;$image=[System.Drawing.Image]::FromFile(${psString(file)});[System.Windows.Forms.Clipboard]::SetImage($image);$image.Dispose()`); return file;
};
register(["Clipboard_SetImage", "Clipboard_SetImageV1"], setClipboardImage);
const saveClipboardImage: CapabilityHandler = async (_ctx, node, args) => {
const folder = path.resolve(text(args, ["path"])); const name = text(args, ["filename"], "clipboard.png"); const target = path.join(folder, path.extname(name) ? name : `${name}.png`); await fs.mkdir(folder, { recursive: true });
await powershell(`Add-Type -AssemblyName System.Windows.Forms;Add-Type -AssemblyName System.Drawing;if(-not [System.Windows.Forms.Clipboard]::ContainsImage()){throw '剪贴板中没有图片'};$image=[System.Windows.Forms.Clipboard]::GetImage();$image.Save(${psString(target)},[System.Drawing.Imaging.ImageFormat]::Png);$image.Dispose()`).catch((error) => { throw new RuntimeError("desktop_command_failed", (error as Error).message, { nodeId: node.id, componentId: node.componentId }); }); return target;
};
register(["Clipboard_SaveImage", "Clipboard_SaveImageV1"], saveClipboardImage);
register("System_PlaySound", async (_ctx, node, args) => {
const audio = text(args, ["audio"]); if (audio) { const file = path.resolve(audio); await fs.access(file); await powershell(`$player=New-Object System.Media.SoundPlayer ${psString(file)};$player.PlaySync()`); return file; }
await powershell("[System.Media.SystemSounds]::Beep.Play()"); return true;
});
register("CustomConfig_GetConfig", (_ctx, _node, args) => {
let configs: unknown = args.jsonlist; if (typeof configs === "string") { try { configs = JSON.parse(configs); } catch { configs = []; } }
const name = text(args, ["configName"]); if (Array.isArray(configs)) { const found = configs.find((item) => item && typeof item === "object" && [String((item as any).name ?? ""), String((item as any).configName ?? ""), String((item as any).key ?? "")].includes(name)); return found && typeof found === "object" && "value" in found ? (found as any).value : found ?? null; }
return configs && typeof configs === "object" ? (configs as Record<string, unknown>)[name] : null;
});
register("Basic_BusinessOrderCount", (ctx, _node, args) => {
const key = "__businessOrderCount"; const start = Number(args.start_count ?? 0); const step = Number(args.count ?? 1); const current = Number(ctx.vars[key] ?? start); const next = current + step; ctx.vars[key] = next; return next;
});
register("PDF_ConcatPdfV1", delegate("PDF_ConcatPdf"));
register(["PDF_SetPdfWaterMark", "PDF_SetPdfWaterMarkV1"], async (_ctx, node, args) => {
const input = path.resolve(text(args, ["fileInPath"])); const output = path.join(path.dirname(input), text(args, ["newPdfName"], "PDF加水印.pdf")); const content = text(args, ["markTextInfo"], "Watermark");
if (!/^[\x20-\x7E]*$/.test(content)) throw new RuntimeError("validation_error", "当前文本水印仅支持 ASCII;中文水印需嵌入字体", { nodeId: node.id, componentId: node.componentId });
const document = await PDFDocument.load(await fs.readFile(input), { ignoreEncryption: true }); const font = await document.embedFont(StandardFonts.Helvetica);
for (const page of document.getPages()) { const { width, height } = page.getSize(); page.drawText(content, { x: width * 0.2, y: height * 0.5, size: 32, font, color: rgb(0.65, 0.65, 0.65), opacity: 0.35, rotate: degrees(-30) }); }
await fs.writeFile(output, await document.save()); return output;
});
async function webhook(kind: "dingtalk" | "feishu" | "wecom", args: Record<string, unknown>, nodeId: string, componentId: string) {
let url = text(args, ["webhook"]); if (!/^https?:\/\//i.test(url)) throw new RuntimeError("validation_error", "Webhook 地址无效", { nodeId, componentId });
const secret = text(args, ["key"]); if (kind === "dingtalk" && secret) { const timestamp = Date.now(); const sign = encodeURIComponent(createHmac("sha256", secret).update(`${timestamp}\n${secret}`).digest("base64")); url += `${url.includes("?") ? "&" : "?"}timestamp=${timestamp}&sign=${sign}`; }
const custom = text(args, ["content_custom"]); const messageType = text(args, ["message_type"], "0"); const plain = text(args, ["content_text", "test_message"]); const markdown = text(args, ["content_markdown"]);
let body: unknown;
if (custom) { try { body = JSON.parse(custom); } catch { throw new RuntimeError("validation_error", "自定义消息 JSON 无效", { nodeId, componentId }); } }
else if (kind === "feishu") body = messageType === "1" ? { msg_type: "interactive", card: { elements: [{ tag: "markdown", content: markdown || plain }] } } : { msg_type: "text", content: { text: plain } };
else if (kind === "dingtalk") body = messageType === "1" ? { msgtype: "markdown", markdown: { title: "RPA通知", text: markdown || plain } } : { msgtype: "text", text: { content: plain }, at: { atMobiles: text(args, ["mentioned_mobile_list"]).split(/[,,]/).filter(Boolean) } };
else body = messageType === "1" ? { msgtype: "markdown", markdown: { content: markdown || plain } } : { msgtype: "text", text: { content: plain, mentioned_mobile_list: text(args, ["mentioned_mobile_list"]).split(/[,,]/).filter(Boolean) } };
const response = await fetch(url, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) }); const responseText = await response.text(); if (!response.ok) throw new RuntimeError("http_error", `Webhook 返回 ${response.status}: ${responseText}`, { nodeId, componentId });
try { return JSON.parse(responseText); } catch { return responseText; }
}
register("MessageNotification_DingTalk_Message", (_ctx, node, args) => webhook("dingtalk", args, node.id, node.componentId));
register("MessageNotification_Feishu_Message", (_ctx, node, args) => webhook("feishu", args, node.id, node.componentId));
register("MessageNotification_WXWork_Message", (_ctx, node, args) => webhook("wecom", args, node.id, node.componentId));
@@ -30,12 +30,21 @@ register("Basic_Delay_V2", async (ctx, node, args) => {
return seconds;
});
register(["Basic_Delay_V1", "Basic_Delay"], async (ctx, node, args) => {
register("Basic_Delay_V1", async (ctx, node, args) => {
const seconds = toNumber(args.second ?? args.time ?? 1, 1);
await sleep(Math.max(0, seconds) * 1000, ctx.signal);
return seconds;
});
register("Basic_Delay", async (ctx, node, args) => {
const milliseconds = Number(args.mileSecond ?? 1000);
if (!Number.isFinite(milliseconds) || milliseconds < 0 || milliseconds > 2_147_483_647) {
throw new RuntimeError("validation_error", "延时时间必须是 0~2147483647 范围内的毫秒数", { nodeId: node.id, componentId: node.componentId });
}
await sleep(milliseconds, ctx.signal);
return milliseconds / 1000;
});
register("Basic_DebugOutput", (ctx, node, args) => {
const message = String(args.msg ?? args.message ?? "");
ctx.log(message || "(空)", "info", node.id);
@@ -0,0 +1,126 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const powershellMock = vi.hoisted(() => vi.fn(async () => '{"ok":true}'));
const detectMock = vi.hoisted(() => vi.fn(async () => ({
texts: [
{ text: "确认保存", box: [[0, 0], [20, 0], [20, 10], [0, 10]] },
{ text: "确认", box: [[100, 200], [130, 200], [130, 210], [100, 210]] }
]
})));
// desktopCapturer 模拟: 默认无画面(走 GDI 回退), 多屏用例按请求宽度提供对应显示器画面
const electronState = vi.hoisted(() => ({ sources: [] as Array<{ display_id: string; thumbnail: any }> }));
vi.mock("../driver/windows/powershell", () => ({ powershell: powershellMock }));
vi.mock("@gutenye/ocr-node", () => ({
default: { create: vi.fn(async () => ({ detect: detectMock })) }
}));
vi.mock("electron", () => ({
desktopCapturer: {
getSources: vi.fn(async (opts: { thumbnailSize: { width: number } }) =>
electronState.sources.filter(s => s.thumbnail.getSize().width === opts.thumbnailSize.width))
},
screen: {
getAllDisplays: vi.fn(() => [
{ id: 1, bounds: { x: 0, y: 0, width: 1920, height: 1080 }, scaleFactor: 1 },
{ id: 2, bounds: { x: 1920, y: 0, width: 1280, height: 720 }, scaleFactor: 2 }
]),
dipToScreenPoint: vi.fn((p: { x: number; y: number }) => ({ x: p.x, y: p.y }))
}
}));
import { getHandler } from "../engine/registry";
import "./cv-vision";
describe("CV vision capabilities", () => {
beforeEach(() => {
powershellMock.mockClear();
detectMock.mockClear();
electronState.sources = [];
});
it.each(["CV_GetText", "CV_SetText", "CV_MouseHover", "CV_MouseClick"])("registers %s", (id) => {
expect(getHandler(id), id).toBeTruthy();
});
it("rejects an empty CV selector", async () => {
await expect(getHandler("CV_GetText")!({ log: vi.fn() } as any, { id: "n1", componentId: "CV_GetText" } as any, {}, {} as any))
.rejects.toThrow("CV 目标为空");
});
it("locates OCR text and clicks its center with offset", async () => {
const handler = getHandler("CV_MouseClick")!;
await expect(handler({ log: vi.fn() } as any, { id: "n1", componentId: "CV_MouseClick" } as any, {
cvSelector: "确认",
offsetX: 5,
offsetY: 2
}, {} as any)).resolves.toBe(true);
expect(detectMock).toHaveBeenCalledTimes(1);
const script = String((powershellMock.mock.calls as unknown as Array<[string]>)[1]?.[0] ?? "");
expect(script).toContain("SetCursorPos(15, 7)");
expect(script).toContain("mouse_event(0x0002");
});
it("returns matched OCR text", async () => {
await expect(getHandler("CV_GetText")!({ log: vi.fn() } as any, { id: "n2", componentId: "CV_GetText" } as any, {
cvSelector: "保存"
}, {} as any)).resolves.toBe("确认保存");
expect(powershellMock).toHaveBeenCalledTimes(1);
});
it("restricts matching to the search region when provided", async () => {
// 屏幕上有两处「确认」: 区域(0,0,50,50)内只应命中 (5,5) 那处, 而不是第一个全屏命中 (115,205)
await expect(getHandler("CV_MouseClick")!({ log: vi.fn() } as any, { id: "n3", componentId: "CV_MouseClick" } as any, {
cvSelector: "确认",
searchRegion: JSON.stringify({ x: 0, y: 0, width: 50, height: 50 })
}, {} as any)).resolves.toBe(true);
const script = String((powershellMock.mock.calls as unknown as Array<[string]>)[1]?.[0] ?? "");
expect(script).toContain("SetCursorPos(10, 5)");
});
it("rejects when target is outside the search region", async () => {
await expect(getHandler("CV_MouseClick")!({ log: vi.fn() } as any, { id: "n4", componentId: "CV_MouseClick" } as any, {
cvSelector: "确认",
searchRegion: JSON.stringify({ x: 500, y: 500, width: 50, height: 50 })
}, {} as any)).rejects.toThrow("未找到目标文本");
});
it("locates targets on a secondary display and maps DIP coordinates", async () => {
// 主屏无画面, 副屏(bounds 1920,0 1280x720 缩放2)物理中心(450,210) → DIP(2145,105)
const fakeThumb = (w: number, h: number): any => ({
isEmpty: () => false,
getSize: () => ({ width: w, height: h }),
toPNG: () => Buffer.from([137, 80, 78, 71])
});
electronState.sources = [{ display_id: "2", thumbnail: fakeThumb(2560, 1440) }];
detectMock.mockImplementationOnce(async () => ({
texts: [{ text: "副屏按钮", box: [[400, 200], [500, 200], [500, 220], [400, 220]] }]
}));
await expect(getHandler("CV_MouseClick")!({ log: vi.fn() } as any, { id: "n5", componentId: "CV_MouseClick" } as any, {
cvSelector: "副屏按钮"
}, {} as any)).resolves.toBe(true);
const script = String((powershellMock.mock.calls as unknown as Array<[string]>).at(-1)?.[0] ?? "");
expect(script).toContain("SetCursorPos(2145, 105)");
});
it("maps region-image hits through the display scale", async () => {
// 框选图来自缩放2的副屏: 图内物理中心(120,90) + 框选DIP原点(1920,0) → DIP(1980,45)
const regionShot = path.join(os.tmpdir(), "rpa-cv-region-test-" + Date.now() + ".png");
fs.writeFileSync(regionShot, Buffer.from([137, 80, 78, 71]));
try {
detectMock.mockImplementationOnce(async () => ({
texts: [{ text: "确认", box: [[100, 80], [140, 80], [140, 100], [100, 100]] }]
}));
await expect(getHandler("CV_MouseHover")!({ log: vi.fn() } as any, { id: "n6", componentId: "CV_MouseHover" } as any, {
cvSelector: "确认",
searchRegion: JSON.stringify({ x: 1920, y: 0, width: 640, height: 360, scale: 2, imagePath: regionShot })
}, {} as any)).resolves.toBe(true);
const script = String((powershellMock.mock.calls as unknown as Array<[string]>).at(-1)?.[0] ?? "");
expect(script).toContain("SetCursorPos(1980, 45)");
} finally {
try { fs.unlinkSync(regionShot); } catch {}
}
});
});
@@ -0,0 +1,270 @@
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell } from "../driver/windows/powershell";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
interface CvTarget { text: string; cx: number; cy: number }
let paddleOcr: { detect: (image: string) => Promise<{ texts: Array<{ text: string; box?: number[][]; mean?: number }> }> } | null = null;
async function getOcr() {
if (paddleOcr) return paddleOcr;
const mod = await import("@gutenye/ocr-node");
paddleOcr = await mod.default.create();
return paddleOcr as { detect: (image: string) => Promise<{ texts: Array<{ text: string; box?: number[][]; mean?: number }> }> };
}
function cvTarget(args: Record<string, unknown>, nodeId: string): string {
for (const key of ["cvSelector", "target", "text"]) {
const v = String(args[key] ?? "").trim();
if (v) return v.replace(/^text:/, "").replace(/^图片:/, "");
}
throw new RuntimeError("validation_error", "CV 目标为空", { nodeId, componentId: nodeId, recommendation: "填写要在屏幕上查找的目标文本" });
}
async function gdiGrabScreen(imagePath: string): Promise<void> {
const psScript = [
// 与点击脚本一致: 先声明 DPI 感知, 否则缩放非 100% 时 Bounds 是逻辑像素, 截屏只覆盖屏幕左上角
"Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value);' -Name RpaCvDpi -Namespace RpaCv | Out-Null;",
"try { [RpaCv.RpaCvDpi]::SetProcessDpiAwarenessContext([IntPtr](-4)) | Out-Null } catch {}",
"Add-Type -AssemblyName System.Drawing | Out-Null;",
"Add-Type -AssemblyName System.Windows.Forms | Out-Null;",
"$b = [System.Windows.Forms.SystemInformation]::VirtualScreen;",
"$bmp = New-Object System.Drawing.Bitmap($b.Width, $b.Height);",
"$g = [System.Drawing.Graphics]::FromImage($bmp);",
"$g.CopyFromScreen($b.X, $b.Y, 0, 0, $b.Size);",
"$g.Dispose();",
"$bmp.Save(" + JSON.stringify(imagePath) + ");",
"$bmp.Dispose();"
].join(" ");
await powershell(psScript, 30000);
}
/** 一次屏幕截图: 图片左上角在虚拟桌面上的绝对 DIP 坐标 + 物理/DIP 缩放比 */
interface ScreenShot { imagePath: string; dipX: number; dipY: number; scale: number }
/** desktopCapturer 逐显示器截屏(每屏原生分辨率), 不可用时回退 GDI 全虚拟桌面单图 */
async function grabScreenAll(baseDir: string): Promise<ScreenShot[]> {
fs.mkdirSync(baseDir, { recursive: true });
try {
const { desktopCapturer, screen } = await import("electron");
if (desktopCapturer && screen?.getAllDisplays) {
const shots: ScreenShot[] = [];
const displays = screen.getAllDisplays();
for (const display of displays) {
const scale = display.scaleFactor || 1;
const sources = await desktopCapturer.getSources({
types: ["screen"],
thumbnailSize: { width: Math.round(display.bounds.width * scale), height: Math.round(display.bounds.height * scale) }
});
const source = sources.find(item => item.display_id === String(display.id))
|| sources.find(item => item.thumbnail.getSize().width === Math.round(display.bounds.width * scale))
|| sources[0];
if (!source || source.thumbnail.isEmpty()) continue;
const imagePath = path.join(baseDir, `rpa-cv-screen-${display.id}-${Date.now()}.png`);
fs.writeFileSync(imagePath, source.thumbnail.toPNG());
shots.push({ imagePath, dipX: display.bounds.x, dipY: display.bounds.y, scale });
}
if (shots.length) return shots;
}
} catch {}
const imagePath = path.join(baseDir, `rpa-cv-screen-${Date.now()}.png`);
await gdiGrabScreen(imagePath);
return [{ imagePath, dipX: 0, dipY: 0, scale: 1 }];
}
interface OcrLine { text: string; box?: number[][]; mean?: number }
/** 搜索区域: 框选生成时含 imagePath(区域截图)+x/y(屏幕绝对 DIP 坐标)+scale(该屏缩放); 也可只给坐标范围 */
interface CvRegion { imagePath?: string; x?: number; y?: number; width?: number; height?: number; scale?: number }
function parseRegionArg(value: unknown): CvRegion | null {
if (!value) return null;
if (typeof value === "object") return value as CvRegion;
const s = String(value).trim();
if (!s) return null;
if (s.startsWith("{")) {
try { return JSON.parse(s) as CvRegion; } catch { return null; }
}
if (/\.(png|jpe?g|bmp)$/i.test(s) || fs.existsSync(s)) return { imagePath: s };
return null;
}
function boxCenter(box: number[][]): { cx: number; cy: number } {
const xs = box.map(p => p[0]);
const ys = box.map(p => p[1]);
return { cx: (Math.min(...xs) + Math.max(...xs)) / 2, cy: (Math.min(...ys) + Math.max(...ys)) / 2 };
}
export async function locateTextOnScreen(
target: string,
nodeId: string,
region?: CvRegion | null
): Promise<{ target: string; text: string; cx: number; cy: number }> {
const ocr = await getOcr();
const keyword = target.trim();
// 命中中心换算为屏幕绝对 DIP 坐标
let found: { text: string; cx: number; cy: number } | null = null;
const cleanups: string[] = [];
if (region?.imagePath && fs.existsSync(region.imagePath)) {
// 框选截图: OCR 只识别该图; 图内物理坐标 ÷ 该屏缩放 + 框选绝对 DIP 原点
const scale = Number(region.scale ?? 1) || 1;
const result = await ocr.detect(region.imagePath);
const hit = (result.texts ?? []).find(t => t.text && t.text.includes(keyword) && t.box && t.box.length >= 4);
if (hit?.box) {
const c = boxCenter(hit.box);
found = {
text: hit.text,
cx: (Number(region.x ?? 0) || 0) + c.cx / scale,
cy: (Number(region.y ?? 0) || 0) + c.cy / scale
};
}
} else {
// 全屏查找: 逐显示器截屏识别, 覆盖副屏(含负坐标屏); 命中即停
const shots = await grabScreenAll(path.join(os.tmpdir(), "rpa-cv-shots"));
for (const shot of shots) {
const result = await ocr.detect(shot.imagePath);
const candidates = (result.texts ?? []).filter(t => t.text && t.text.includes(keyword) && t.box && t.box.length >= 4);
let hit: OcrLine | undefined = candidates[0];
// 给了坐标范围(绝对 DIP)时, 过滤中心落在区域内的命中
if (hit && region?.width && region?.height) {
const rx = Number(region.x ?? 0) || 0;
const ry = Number(region.y ?? 0) || 0;
hit = candidates.find(t => {
const c = boxCenter(t.box!);
const dx = shot.dipX + c.cx / shot.scale;
const dy = shot.dipY + c.cy / shot.scale;
return dx >= rx && dx <= rx + region.width! && dy >= ry && dy <= ry + region.height!;
});
}
if (hit?.box) {
const c = boxCenter(hit.box);
found = { text: hit.text, cx: shot.dipX + c.cx / shot.scale, cy: shot.dipY + c.cy / shot.scale };
break;
}
cleanups.push(shot.imagePath);
}
// 成功命中时清理全部截图, 未命中保留全部以便排查
if (found) for (const p of shots.map(s => s.imagePath)) { try { fs.unlinkSync(p); } catch {} }
}
if (!found) {
throw new RuntimeError("cv_not_found", "屏幕上未找到目标文本:" + keyword + (region?.imagePath ? "(搜索范围: 框选区域)" : ""), {
nodeId,
componentId: nodeId,
recommendation: "确认目标文本在" + (region?.imagePath ? "框选区域内" : "屏幕上(含副屏)") + "可见且清晰"
});
}
for (const p of cleanups) { try { fs.unlinkSync(p); } catch {} }
return { target: keyword, text: found.text, cx: found.cx, cy: found.cy };
}
function screenUser32(): string {
return [
"Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value); [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y); [DllImport(\"user32.dll\")] public static extern void mouse_event(uint flags, uint dx, uint dy, uint data, UIntPtr extra);' -Name RpaCvInput -Namespace RpaCv | Out-Null;",
"try { [RpaCv.RpaCvInput]::SetProcessDpiAwarenessContext([IntPtr](-4)) | Out-Null } catch {}"
].join("\n");
}
function coordinateScript(x: number, y: number, action: "hover" | "click" | "doubleClick" | "paste", value = ""): string {
const lines = [
screenUser32(),
"if (-not [RpaCv.RpaCvInput]::SetCursorPos(" + x + ", " + y + ")) { throw 'cursor_move_failed' }",
"Start-Sleep -Milliseconds 100"
];
if (action !== "hover") {
lines.push(
"[RpaCv.RpaCvInput]::mouse_event(0x0002,0,0,0,[UIntPtr]::Zero)",
"Start-Sleep -Milliseconds 40",
"[RpaCv.RpaCvInput]::mouse_event(0x0004,0,0,0,[UIntPtr]::Zero)"
);
}
if (action === "doubleClick") {
lines.push(
"Start-Sleep -Milliseconds 70",
"[RpaCv.RpaCvInput]::mouse_event(0x0002,0,0,0,[UIntPtr]::Zero)",
"Start-Sleep -Milliseconds 40",
"[RpaCv.RpaCvInput]::mouse_event(0x0004,0,0,0,[UIntPtr]::Zero)"
);
}
if (action === "paste") {
lines.push(
"Add-Type -AssemblyName System.Windows.Forms",
"Start-Sleep -Milliseconds 220",
"$saved = $null",
"try { $saved = [System.Windows.Forms.Clipboard]::GetText() } catch {}",
"try { [System.Windows.Forms.Clipboard]::SetText(" + JSON.stringify(value) + ") } catch { throw 'clipboard_failed' }",
"Start-Sleep -Milliseconds 80",
"[System.Windows.Forms.SendKeys]::SendWait('^a')",
"Start-Sleep -Milliseconds 60",
"[System.Windows.Forms.SendKeys]::SendWait('^v')",
"Start-Sleep -Milliseconds 280",
"try { if ($saved) { [System.Windows.Forms.Clipboard]::SetText($saved) } else { [System.Windows.Forms.Clipboard]::Clear() } } catch {}"
);
}
lines.push("Write-Output (@{ ok = $true; x = " + x + "; y = " + y + " } | ConvertTo-Json -Compress)");
return lines.join("\n");
}
/** DIP → 物理像素(SetCursorPos 在 DPI 感知进程里用物理坐标); 无 Electron API 时按原值(缩放 100% 等价) */
async function toPhysical(x: number, y: number): Promise<{ x: number; y: number }> {
try {
const { screen } = await import("electron");
if (screen && typeof screen.dipToScreenPoint === "function") {
const p = screen.dipToScreenPoint({ x: Math.round(x), y: Math.round(y) });
return { x: p.x, y: p.y };
}
} catch {}
return { x: Math.round(x), y: Math.round(y) };
}
async function locateFor(args: Record<string, unknown>, node: { id: string; componentId: string }) {
const target = cvTarget(args, node.id);
// 可选搜索区域: 框选生成 {x,y,width,height,imagePath,scale}, 限定 OCR 查找范围
const region = parseRegionArg(args.searchRegion ?? args.region);
const located = await locateTextOnScreen(target, node.id, region);
const offsetX = Number(args.offsetX ?? args.offset_x ?? 0) || 0;
const offsetY = Number(args.offsetY ?? args.offset_y ?? 0) || 0;
return { located, x: Math.round(located.cx + offsetX), y: Math.round(located.cy + offsetY) };
}
const getTextGround: CapabilityHandler = async (ctx, node, args) => {
const target = cvTarget(args, node.id);
const result = await locateTextOnScreen(target, node.id);
ctx.log("CV_GetText found: " + result.text, "info", node.id);
return result.text;
};
const setTextGround: CapabilityHandler = async (ctx, node, args) => {
const value = String(args.value ?? args.text ?? "");
if (!value) throw new RuntimeError("validation_error", "CV 文本输入内容为空", { nodeId: node.id, componentId: node.componentId });
const { x, y } = await locateFor(args, node);
const phys = await toPhysical(x, y);
await powershell(coordinateScript(phys.x, phys.y, "paste", value), 30000);
ctx.log("CV_SetText input at " + phys.x + "," + phys.y, "info", node.id);
return true;
};
const mouseHoverGround: CapabilityHandler = async (ctx, node, args) => {
const { x, y } = await locateFor(args, node);
const phys = await toPhysical(x, y);
await powershell(coordinateScript(phys.x, phys.y, "hover"), 15000);
ctx.log("CV_MouseHover hover at " + phys.x + "," + phys.y, "info", node.id);
return true;
};
const mouseClickGround: CapabilityHandler = async (ctx, node, args) => {
const { x, y } = await locateFor(args, node);
const phys = await toPhysical(x, y);
const clickway = String(args.clickway ?? "0");
const action = clickway === "1" || /double/i.test(clickway) ? "doubleClick" as const : "click" as const;
await powershell(coordinateScript(phys.x, phys.y, action), 15000);
ctx.log("CV_MouseClick " + action + " at " + phys.x + "," + phys.y, "info", node.id);
return true;
};
register("CV_GetText", getTextGround);
register("CV_SetText", setTextGround);
register("CV_MouseHover", mouseHoverGround);
register("CV_MouseClick", mouseClickGround);
@@ -2,7 +2,7 @@ import { mkdtemp, readFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { Catalog, CatalogComponent, FlowNode } from "../../shared/types";
import type { Catalog, CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./data-extended";
@@ -0,0 +1,283 @@
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { afterAll, describe, expect, it } from "vitest";
import { capabilityIds, getHandler } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./data";
import "./data-extended";
import "./files";
import "./control";
const ctx = {
signal: new AbortController().signal,
log: () => {},
vars: {} as Record<string, unknown>,
evidenceDir: "D:/office-test2",
runId: "smoke-run",
flow: { id: "smoke-flow", name: "冒烟", description: "", enabled: true, trigger: null, meta: { appVersion: "dev", createdAt: "", updatedAt: "" }, nodes: [] }
} as unknown as RunContext;
const node = { id: "smoke-data", componentId: "smoke", position: { x: 0, y: 0 }, values: {}, options: {} } as unknown as FlowNode;
async function run(componentId: string, args: Record<string, unknown>, context?: RunContext): Promise<unknown> {
const handler = getHandler(componentId);
if (!handler) throw new Error(`组件 ${componentId} 未注册运行时`);
return handler(context ?? ctx, { ...node, componentId } as unknown as FlowNode, args, {} as CatalogComponent);
}
const sandbox = mkdtempSync(path.join(tmpdir(), "rpa-data-smoke-"));
afterAll(() => { rmSync(sandbox, { recursive: true, force: true }); });
describe("数据处理/文件/编排 冒烟", () => {
it("字符串族 StrUtil", async () => {
expect(await run("StrUtil_Len", { string: "你好world" })).toBe(7);
expect(await run("StrUtil_InStr", { string: "hello", search: "ll" })).toBe(2);
expect(await run("StrUtil_Replace", { string: "a-b-c", search: "-", replace: "+" })).toBe("a+b+c");
expect(await run("StrUtil_LowerCase", { string: "AbC" })).toBe("abc");
expect(await run("StrUtil_UpperCase", { string: "AbC" })).toBe("ABC");
expect(await run("StrUtil_CaseChange", { string: "abc", case: "首字母大写" })).toBe("Abc");
expect(await run("StrUtil_CaseChange", { string: "abc", case: "1" })).toBe("ABC");
expect(await run("StrUtil_CaseChange", { string: "ABC", case: "2" })).toBe("abc");
expect(await run("StrUtil_Trim", { string: " x " })).toBe("x");
expect(await run("StrUtil_Left", { string: "hello", length: 2 })).toBe("he");
expect(await run("StrUtil_Right", { string: "hello", length: 2 })).toBe("lo");
expect(await run("StrUtil_Mid", { string: "hello", start: 1, length: 3 })).toBe("ell");
expect(await run("StrUtil_CutOutStrV1", { string: "a[内容]b", start_cut_pos: "a[", end: "]" })).toContain("内容");
expect(await run("StrUtil_Split", { string: "a,b,c", spliter: "," })).toEqual(["a", "b", "c"]);
expect(await run("StrUtil_SplitStr", { string: "a; b ;c", diy_symbol: ";" })).toEqual(["a", "b", "c"]);
expect(await run("StrUtil_AddToStr", { string: "尾", add_str: "头", add_type: "添加到前面", symbol: "-" })).toBe("头-尾");
expect(await run("StrUtil_ReverseStr", { string: "ab cd", letter_reverse: true })).toBe("dc ba");
expect(await run("StrUtil_ListToString", { str_list: ["a", "b"], connectors_custom: "|" })).toBe("a|b");
expect(await run("StrUtil_CompleteStr", { string: "7", sum_length: 3, com_string: "0", com_type: "左边补" })).toBe("007");
expect(await run("StrUtil_CharCode", { code: 65 })).toBe("A");
expect(await run("StrUtil_CharAsc", { char: "A" })).toBe(65);
expect(await run("StrUtil_CreateRandomStr", { string_length: 12 })).toHaveLength(12);
console.log("[字符串族] 20 项通过");
});
it("列表与集合族 List/Collection", async () => {
expect(await run("List_CreateList", { data: JSON.stringify([1, 2]) })).toEqual([1, 2]);
expect(await run("List_ListLength", { arrayRet: [1, 2, 3] })).toBe(3);
expect(await run("List_InsertList", { arrayRet: ["a", "c"], data: "b", insertType: "指定位置", position: 1 })).toEqual(["a", "b", "c"]);
expect(await run("List_DeleteList", { arrayRet: ["a", "b", "c"], data: "b" })).toEqual(["a", "c"]);
expect(await run("List_DeleteList", { arrayRet: ["a", "b", "c"], deleteType: "按位置", position: 0 })).toEqual(["b", "c"]);
expect(await run("List_GetListItem", { arrayRet: [10, 20], position: 1 })).toBe(20);
expect(await run("List_SetListItem", { arrayRet: [1, 2], data: 9, position: 0 })).toEqual([9, 2]);
expect(await run("List_GetIndex", { arrayRet: ["x", "y"], data: "y" })).toBe(1);
expect(await run("List_ListMerge", { arrayRet1: [1], arrayRet2: [2, 3] })).toEqual([1, 2, 3]);
expect(await run("List_ListDuplication", { arrayRet: [1, 1, 2] })).toEqual([1, 2]);
expect(await run("List_ListSorted", { arrayRet: [3, 1, 2] })).toEqual([1, 2, 3]);
expect(await run("List_ListIntersection", { arrayRet1: [1, 2, 3], arrayRet2: [2, 3, 4] })).toEqual([2, 3]);
expect(await run("List_ClearList", {})).toEqual([]);
expect(await run("Collection_Create", {})).toEqual([]);
const withItem = await run("Collection_Add", { collection: [], value: "a" }) as unknown[];
expect(withItem).toEqual(["a"]);
expect(await run("Collection_HasItem", { collection: ["a"], item: "a" })).toBe(true);
expect(await run("Collection_HasItem", { collection: ["a"], item: "z" })).toBe(false);
expect(await run("Collection_Count", { collection: ["a", "b"] })).toBe(2);
expect(await run("Collection_Remove", { collection: ["a", "b"], value: "a" })).toEqual(["b"]);
console.log("[列表/集合族] 18 项通过");
});
it("字典族 Dict", async () => {
expect(await run("Dict_Create", { value: JSON.stringify({ a: 1 }) })).toEqual({ a: 1 });
expect(await run("Dict_SetItem", { dictObj: { a: 1 }, key: "b", value: 2 })).toEqual({ a: 1, b: 2 });
expect(await run("Dict_GetItem", { dictObj: { a: 1 }, key: "a" })).toBe(1);
expect(await run("Dict_GetItem", { dictObj: {}, key: "x", default_value: "dft" })).toBe("dft");
expect(await run("Dict_Keys", { dictObj: { a: 1, b: 2 } })).toEqual(["a", "b"]);
expect(await run("Dict_Values", { dictObj: { a: 1, b: 2 } })).toEqual([1, 2]);
expect(await run("Dict_Items", { dictObj: { a: 1 } })).toEqual([["a", 1]]);
expect(await run("Dict_Delete", { dictObj: { a: 1, b: 2 }, key: "a" })).toEqual({ b: 2 });
expect(await run("Dict_Update", { dictObj: { a: 1 }, targetDict: { b: 2 } })).toEqual({ a: 1, b: 2 });
expect(await run("Dict_Clear", {})).toEqual({});
console.log("[字典族] 10 项通过");
});
it("数学族 Math 与正则族 Regex", async () => {
expect(await run("Math_Abs", { data: -3.5 })).toBe(3.5);
expect(await run("Math_Round", { data: 3.14159, digit: 2 })).toBe(3.14);
expect(await run("Math_Int", { data: 3.2, position: "向上取整" })).toBe(4);
expect(await run("Math_Sqrt", { data: 16 })).toBe(4);
expect(await run("Math_Pow", { x: 2, y: 10 })).toBe(1024);
expect(await run("Math_Sgn", { data: -9 })).toBe(-1);
expect(await run("Math_Exp", { data: 0 })).toBe(1);
expect(await run("Math_Hex", { num: 255 })).toBe("FF");
expect(await run("Math_Oct", { num: 8 })).toBe("10");
expect(await run("Math_Scale", { sourceNum: "1010", sourceNumScale: 2, targetNumScale: 10 })).toBe("10");
expect(await run("Math_Sin", { data: 90, unit: "角度" })).toBeCloseTo(1, 5);
expect(await run("Regex_Test", { text: "abc123", pattern: "\\d+" })).toBe(true);
expect(await run("Regex_TestNew", { text: "abc", pattern: "\\d" })).toBe(false);
expect(await run("Regex_FindAll", { text: "a1b22c333", pattern: "\\d+" })).toEqual(["1", "22", "333"]);
expect(await run("Regex_FindStr", { text: "订单A-100,订单B-200", pattern: "订单[AB]-(\\d+)", index: 1 })).toBe("100");
expect(await run("Regex_FindAllNew", { text: "2026-11-03", pattern: "\\d+" })).toEqual(["2026", "11", "03"]);
console.log("[数学/正则族] 16 项通过");
});
it("时间族 Time", async () => {
expect(String(await run("Time_Now", {}))).toMatch(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/);
expect(String(await run("Time_Date", {}))).toMatch(/^\d{4}-\d{2}-\d{2}$/);
expect(String(await run("Time_Time", {}))).toMatch(/^\d{2}:\d{2}:\d{2}$/);
const ts = await run("Time_TimeStamp", {}) as number;
expect(ts).toBeGreaterThan(1_700_000_000);
expect(await run("Time_Format", { timestamp: 1767000000, format: "YYYY-MM-DD" })).toMatch(/^\d{4}-\d{2}-\d{2}$/);
expect(await run("Time_DatetimeToTimestamp", { datetime: "2026-01-01 00:00:00" })).toBe(typeof 1_767_225_600 === "number" ? Math.floor(new Date("2026-01-01 00:00:00").getTime() / 1000) : 0);
expect(await run("Time_Year", { datetime: "2026-11-03 10:20:30" })).toBe(2026);
expect(await run("Time_Month", { datetime: "2026-11-03 10:20:30" })).toBe(11);
expect(await run("Time_Day", { datetime: "2026-11-03 10:20:30" })).toBe(3);
expect(await run("Time_Hour", { datetime: "2026-11-03 10:20:30" })).toBe(10);
expect(await run("Time_Minute", { datetime: "2026-11-03 10:20:30" })).toBe(20);
expect(await run("Time_Second", { datetime: "2026-11-03 10:20:30" })).toBe(30);
expect(typeof (await run("Time_WeekDay", { datetime: "2026-11-03" }))).toBe("number");
expect(await run("Time_DateOffset_V1", { date_input: "2026-11-01", offset_unit_num: 2, offset_status: "加", get_type: "YYYY-MM-DD" })).toBe("2026-11-03");
expect(await run("Time_CalDate", { date_start: "2026-11-01", date_end: "2026-11-03", time_type: "天" })).toBeCloseTo(2, 5);
expect(await run("Time_TimeTypeConversion", { datetime: "2026-11-03 08:00:00", format: "YYYY/MM/DD" })).toBe("2026/11/03");
console.log("[时间族] 15 项通过");
});
it("JSON 与 CSV 读写回环", async () => {
expect(await run("JSON_Stringify", { string: { a: 1 } })).toBe('{"a":1}');
expect(await run("JSON_Parse", { string: '{"a":1}' })).toEqual({ a: 1 });
expect(await run("JSON_Parse", { string: "bad" })).toBeNull();
const csvPath = path.join(sandbox, "t.csv");
await run("CSV_WriteNew", { path: csvPath, table: [["名", "值"], ["a,b", "含\"引号\""], ["c", "d"]] });
const back = await run("CSV_ReadNew", { path: csvPath }) as string[][];
expect(back[0]).toEqual(["名", "值"]);
expect(back[1]).toEqual(["a,b", '含"引号"']);
console.log("[JSON/CSV] 通过(含引号逗号转义回环)");
});
it("Ini 族完整生命周期", async () => {
const iniPath = path.join(sandbox, "config.ini");
await run("Ini_Write", { file: iniPath, section: "server", key: "host", value: "127.0.0.1" });
await run("Ini_Write", { file: iniPath, section: "server", key: "port", value: "8080" });
await run("Ini_Write", { file: iniPath, section: "log", key: "level", value: "info" });
expect(await run("Ini_Read", { file: iniPath, section: "server", key: "host" })).toBe("127.0.0.1");
expect(await run("Ini_ListSection", { file: iniPath })).toEqual(["server", "log"]);
expect(await run("Ini_ListKey", { file: iniPath, section: "server" })).toEqual(["host", "port"]);
expect(await run("Ini_DeleteKey", { file: iniPath, section: "server", key: "port" })).toBe(true);
expect(await run("Ini_Read", { file: iniPath, section: "server", key: "port" })).toBe("");
expect(await run("Ini_DeleteSection", { file: iniPath, section: "log" })).toBe(true);
expect(await run("Ini_ListSection", { file: iniPath })).toEqual(["server"]);
console.log("[Ini族] 写读列举删除全链通过");
});
it("文件族全生命周期(沙盒目录)", async () => {
const dir = path.join(sandbox, "files");
const created = await run("File_CreateFile", { folder_path: dir, name: "a.txt" }) as string;
expect(created).toContain("a.txt");
await run("File_Write", { path: created, content: "第一行" });
await run("File_Write", { path: created, content: "第二行", append: true });
expect(await run("File_Read", { path: created })).toBe("第一行第二行");
expect(await run("File_FileExists", { path: created })).toBe(true);
expect(await run("File_PathExists", { path: path.join(dir, "nope.txt") })).toBe(false);
expect(await run("File_PathSize", { path: created })).toBeGreaterThan(0);
const info = await run("File_GetFileInfo", { file_path: created }) as { name: string; size: number };
expect(info.name).toBe("a.txt");
expect(info.size).toBeGreaterThan(0);
const copy = path.join(dir, "b.txt");
await run("File_Copy", { pathSrc: created, pathDest: copy });
expect(await run("File_Read", { path: copy })).toBe("第一行第二行");
const sub = path.join(dir, "sub");
await run("File_CreateFolder", { path: sub });
await run("File_Copy", { pathSrc: copy, pathDest: path.join(sub, "c.txt") });
const flat = await run("File_GetFileList", { folder_path: dir }) as string[];
expect(flat.length).toBe(2);
const all = await run("File_GetFileList", { folder_path: dir, recursive: true }) as string[];
expect(all.length).toBe(3);
const kw = await run("File_GetFileList", { folder_path: dir, key_word: "a." }) as string[];
expect(kw.length).toBe(1);
const moved = await run("File_Move", { pathSrc: copy, pathDest: path.join(dir, "moved.txt") });
expect(String(moved)).toContain("moved.txt");
expect(await run("File_FileExists", { path: copy })).toBe(false);
const renamed = await run("File_Rename", { pathSrc: moved, name: "renamed.txt" });
expect(String(renamed)).toContain("renamed.txt");
const zip = await run("File_CompressFile", { pathSrc: created, pathDest: path.join(sandbox, "pack.zip"), direct_type: "0" });
expect(String(zip)).toContain("pack.zip");
const unzipDir = path.join(sandbox, "unpacked");
await run("File_DecompressFile", { path: path.join(sandbox, "pack.zip"), target: unzipDir, direct_type: "0" });
expect(await run("File_FileExists", { path: path.join(unzipDir, "a.txt") })).toBe(true);
expect(await run("File_WaitFile", { file_path: created, time_out: 1 })).toBe(true);
await expect(run("File_WaitFile", { file_path: path.join(dir, "ghost.txt"), time_out: 1 })).rejects.toThrow(/超时/);
await run("File_Delete", { file_path: renamed });
expect(await run("File_FileExists", { path: renamed })).toBe(false);
console.log("[文件族] 建/写/读/拷/列举/移/改名/压缩解压/等待/删除 全链通过");
}, 60000);
it("数据表族 Bivartable", async () => {
const table = [["部门", "姓名", "工资"], ["技术", "张三", 100], ["销售", "李四", 80], ["技术", "王五", 90], ["销售", "李四", 80]];
const bodyOnly = [["技术", "张三", 100], ["销售", "李四", 80], ["技术", "王五", 90], ["销售", "李四", 80]];
const withHeader = await run("Bivartable_DoCreate", { data: JSON.stringify(bodyOnly), columnList: JSON.stringify(["部门", "姓名", "工资"]) }) as unknown[][];
expect(withHeader[0]).toEqual(["部门", "姓名", "工资"]);
expect(withHeader.length).toBe(5);
expect(await run("Bivartable_GetSumValue", { df: table, sumvaluecols: JSON.stringify(["工资"]) })).toBe(350);
expect(await run("Bivartable_GetMaxValue", { df: table, maxvaluecols: JSON.stringify(["工资"]) })).toBe(100);
expect(await run("Bivartable_GetMinValue", { df: table, minvaluecols: JSON.stringify(["工资"]) })).toBe(80);
expect(await run("Bivartable_GetAvgValue", { df: table, avgvaluecols: JSON.stringify(["工资"]) })).toBe(87.5);
const grouped = await run("Bivartable_GetSumValue", { df: table, sumvaluecols: JSON.stringify(["工资"]), groupby: JSON.stringify(["部门"]) }) as unknown[][];
expect(grouped[0]).toEqual(["部门", "工资"]);
expect(grouped).toContainEqual(["技术", 190]);
expect(grouped).toContainEqual(["销售", 160]);
const filtered = await run("Bivartable_DoFilter", { df: table, col_name: "部门", choice: "1", criteria: "技术" }) as unknown[][];
expect(filtered.length).toBe(3);
const expr = await run("Bivartable_DoFilter", { df: table, rule: '["工资"] > 85' }) as unknown[][];
expect(expr.length).toBe(3);
const dedup = await run("Bivartable_DeleteDuplicateRow", { df: table }) as unknown[][];
expect(dedup.length).toBe(4);
console.log("[数据表族] 创建/聚合(含分组)/过滤(简单+表达式)/去重 通过");
});
it("编排控制与日志", async () => {
const varsCtx = { ...ctx, vars: {} } as unknown as RunContext;
expect(await run("logic_control_var", { var_name: "k1", var_value: "v1" }, varsCtx)).toBe("v1");
expect((varsCtx as unknown as { vars: Record<string, unknown> }).vars.k1).toBe("v1");
await expect(run("logic_control_var", {})).rejects.toThrow(/变量名称/);
await expect(run("logic_control_break_for", {})).rejects.toBeTruthy();
await expect(run("logic_control_next_for", {})).rejects.toBeTruthy();
await expect(run("logic_control_stop", {})).rejects.toBeTruthy();
await expect(run("logic_control_stop_return", {})).rejects.toBeTruthy();
expect(await run("Trigger.Manual", {})).toBeNull();
expect(await run("Trigger.Schedule", {})).toBeNull();
expect(await run("Trigger.FileArrival", {})).toBeNull();
const logs: Array<{ m: string; lv: string }> = [];
const logCtx = { ...ctx, log: (m: string, lv: string) => logs.push({ m, lv }) } as unknown as RunContext;
expect(await run("Log_Info", { msg: "提示信息" }, logCtx)).toBe("提示信息");
expect(await run("Log_Error", { msg: "错误信息" }, logCtx)).toBe("错误信息");
expect(logs.some((l) => l.lv === "error")).toBe(true);
await expect(run("ExceptionUtil_Throws", { msg: "自定义异常" })).rejects.toThrow(/自定义异常/);
console.log("[编排/日志] 变量/循环控制信号/触发器/日志/自定义异常 通过");
});
it("全量健康检查: 已注册组件空参调用不产生非结构化崩溃", async () => {
const ids: string[] = capabilityIds();
const skip = new Set([
"FTP_Connect", "FTP_Upload", "FTP_Download", "FTP_Delete", "FTP_Rename", "FTP_Cwd", "FTP_GetFileList", "FTP_CreateDir", "FTP_Exist", "FTP_Disconnect",
"Database_DBConnect", "Database_SQLExecute", "Database_SingleSQLQuery", "Database_AllSQLQuery_V1", "Database_BatchSQLExecute", "Database_BatchInsertExecute", "Database_CloseDBConnect",
"File_OpenFile", "File_StartBusiness", "File_UploadBusinessResults", "File_UploadBusinessResults_v1", "File_Collectupload",
"File_Delete", "File_DeleteFile", "File_EmptyFolder", "File_WaitFile",
"logic_control_break_for", "logic_control_next_for", "logic_control_stop", "logic_control_stop_return",
"child_modules_v1", "concurrent_child_process_v1", "Element_exception_monitor_v2", "logic_control_if_ai_vision"
]);
const crashes: Array<{ id: string; error: string }> = [];
let checked = 0;
for (const id of ids) {
const handler = getHandler(id);
if (!handler || skip.has(id)) continue;
checked += 1;
try {
await handler(ctx, { ...node, componentId: id } as unknown as FlowNode, {}, {} as CatalogComponent);
} catch (error) {
if (!(error instanceof RuntimeError)) {
crashes.push({ id, error: `${(error as Error).name}: ${(error as Error).message}` });
}
}
}
console.log(`[全量健康检查] 空参调用 ${checked} 个已注册组件, 结构化报错全部正常, 非结构化崩溃 ${crashes.length} 个`);
if (crashes.length) console.log(crashes.map((c) => `${c.id}: ${c.error}`).join("\n"));
expect(crashes).toEqual([]);
}, 120000);
});
@@ -2,7 +2,7 @@ import { mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./data-table-common";
@@ -90,9 +90,21 @@ register(["Bivartable_ColumnOperationDf", "Bivartable_DoCal"], (_ctx, _node, arg
const result: Table = [[...headers, name]]; for (const row of source.slice(1)) result.push([...row, calculate(row[left], row[right], operator)]); return result;
});
function columnIndexOf(candidate: unknown): number {
const text = String(candidate ?? "").trim();
if (/^\d+$/.test(text)) return Number(text) - 1;
let result = 0;
for (const char of text.toUpperCase()) result = result * 26 + (char.charCodeAt(0) - 64);
return result - 1;
}
register(["Bivartable_SpliceColData", "Bivartable_DoJoint"], (_ctx, _node, args) => {
const source = table(args.df); if (!source.length) return source; const headers = source[0];
const left = index(headers, args.col1 ?? args.cola); const right = index(headers, args.col2 ?? args.colb); const name = String(args.name ?? args.colnew ?? "拼接结果"); const separator = String(args.separator ?? "");
// catalog 字段:change_type(name/num) + col_name_1/2 或 col_num_1/2;兼容旧键 col1/col2
const byName = String(args.change_type ?? "name") !== "num";
const left = byName ? index(headers, args.col_name_1 ?? args.col1 ?? args.cola) : columnIndexOf(args.col_num_1);
const right = byName ? index(headers, args.col_name_2 ?? args.col2 ?? args.colb) : columnIndexOf(args.col_num_2);
const name = String(args.new_col_name ?? args.name ?? args.colnew ?? "拼接结果"); const separator = String(args.connector ?? args.separator ?? "");
return [[...headers, name], ...source.slice(1).map((row) => [...row, `${row[left] ?? ""}${separator}${row[right] ?? ""}`])];
});
@@ -120,9 +120,10 @@ register(["StrUtil_UpperCase"], (_ctx, _node, args) => str(first(args, ["string"
register("StrUtil_CaseChange", (_ctx, _node, args) => {
const text = str(first(args, ["string"]));
const mode = str(first(args, ["case"]));
if (mode.includes("小写")) return text.toLowerCase();
if (mode.includes("大写")) return text.toUpperCase();
// 目录选项值为 "1"(大写)/"2"(小写),文字模式兼容 大写/小写/首字母大写
if (mode.includes("首")) return text ? text[0].toUpperCase() + text.slice(1) : "";
if (mode === "1" || mode.includes("大写")) return text.toUpperCase();
if (mode === "2" || mode.includes("小写")) return text.toLowerCase();
return text;
});
register("StrUtil_Trim", (_ctx, _node, args) => str(first(args, ["string"])).trim());
@@ -463,25 +464,30 @@ function toCsv(rows: unknown[][]): string {
}
register(["CSV_Read", "CSV_ReadNew"], async (_ctx, node, args) => {
const filePath = path.resolve(str(first(args, ["path"])));
const filePath = str(first(args, ["path"])).trim();
if (!filePath) throw new RuntimeError("validation_error", "CSV 路径为空", { nodeId: node.id, componentId: node.componentId });
const text = await fs.readFile(filePath, "utf8");
const text = await fs.readFile(path.resolve(filePath), "utf8");
return parseCsv(text);
});
register(["CSV_Write", "CSV_WriteNew"], async (_ctx, node, args) => {
const filePath = path.resolve(str(first(args, ["path"])));
const filePath = str(first(args, ["path"])).trim();
if (!filePath) throw new RuntimeError("validation_error", "CSV 路径为空", { nodeId: node.id, componentId: node.componentId });
const table = asArray(first(args, ["table"])).map((row) => asArray(row));
await fs.mkdir(path.dirname(filePath), { recursive: true });
await fs.writeFile(filePath, toCsv(table), "utf8");
return filePath;
const resolved = path.resolve(filePath);
await fs.mkdir(path.dirname(resolved), { recursive: true });
await fs.writeFile(resolved, toCsv(table), "utf8");
return resolved;
});
/* ---------------- Basic ---------------- */
register(["Basic_ConvertNumber", "Basic_StringToNum"], (_ctx, _node, args) => num(first(args, ["val"])));
register("Basic_ConvertString", (_ctx, _node, args) => str(first(args, ["val"])));
register("Basic_ConvertInt", (_ctx, _node, args) => Math.trunc(num(first(args, ["val"]))));
register("Basic_ConvertBool", (_ctx, _node, args) => bool(first(args, ["val"])));
register("Basic_ConvertBool", (_ctx, _node, args) => {
const value = first(args, ["val"]);
if (typeof value === "number") return value !== 0;
return bool(value);
});
register("Basic_IsNull", (_ctx, _node, args) => {
const value = first(args, ["val"]);
return value === null || value === undefined || value === "";
@@ -498,12 +504,30 @@ register("Basic_Type", (_ctx, _node, args) => {
if (Array.isArray(value)) return "list";
return typeof value;
});
register("Basic_JudgeDataType", (_ctx, _node, args) => {
// 原始目录缺少选项映射。新界面使用语义值;数字别名兼容此前默认的 "1"。
const JUDGE_TYPE_ALIASES: Record<string, string> = { "1": "number", "2": "null", "3": "dict", "4": "string", "5": "list", "6": "bool", "7": "int" };
const CONVERT_TYPE_ALIASES: Record<string, string> = { "1": "number", "2": "int", "3": "bool", "4": "string" };
function targetType(raw: unknown, aliases: Record<string, string>, supported: string[], nodeId: string, componentId: string): string {
const text = String(raw ?? "number").trim().toLowerCase();
const synonyms: Record<string, string> = { float: "number", boolean: "bool", str: "string", text: "string", array: "list", dictionary: "dict", object: "dict", empty: "null" };
const type = aliases[text] || synonyms[text] || text;
if (!supported.includes(type)) throw new RuntimeError("validation_error", "目标数据类型无效,请重新选择类型", { nodeId, componentId });
return type;
}
register("Basic_JudgeDataType", (_ctx, node, args) => {
const value = first(args, ["val"]);
if (value === null || value === undefined) return "null";
if (Array.isArray(value)) return "list";
if (typeof value === "object") return "dict";
return typeof value;
const type = targetType(args.into_type, JUDGE_TYPE_ALIASES, ["number", "null", "dict", "string", "list", "bool", "int"], node.id, node.componentId);
switch (type) {
case "number": return typeof value === "number" && Number.isFinite(value);
case "int": return typeof value === "number" && Number.isInteger(value);
case "null": return value === null || value === undefined || value === "";
case "dict": return Boolean(value) && typeof value === "object" && !Array.isArray(value);
case "string": return typeof value === "string";
case "list": return Array.isArray(value);
case "bool": return typeof value === "boolean";
}
});
register("Basic_Clone", (_ctx, _node, args) => structuredClone(first(args, ["val"])));
register("Basic_RandomInt", (_ctx, _node, args) => {
@@ -512,4 +536,23 @@ register("Basic_RandomInt", (_ctx, _node, args) => {
return Math.floor(min + Math.random() * (max - min + 1));
});
register("Basic_Rnd", () => Math.random());
register("Basic_DataTypeConversion", (_ctx, _node, args) => first(args, ["val"]));
register("Basic_DataTypeConversion", (_ctx, node, args) => {
const value = first(args, ["val"]);
const type = targetType(args.into_type, CONVERT_TYPE_ALIASES, ["number", "int", "bool", "string"], node.id, node.componentId);
if (type === "string") return str(value);
if (type === "bool") {
if (value == null || value === "") return false;
if (typeof value === "boolean") return value;
if (typeof value === "number" && Number.isFinite(value)) return value !== 0;
const text = String(value).trim().toLowerCase();
if (["true", "1", "是", "开启"].includes(text)) return true;
if (["false", "0", "否", "关闭"].includes(text)) return false;
throw new RuntimeError("validation_error", "输入值无法转换为布尔类型", { nodeId: node.id, componentId: node.componentId });
}
if ((typeof value !== "number" && typeof value !== "string") || (typeof value === "string" && !value.trim()) || !Number.isFinite(Number(value))) {
throw new RuntimeError("validation_error", "输入值不是有效数字", { nodeId: node.id, componentId: node.componentId });
}
const result = type === "int" ? Math.trunc(Number(value)) : Number(value);
if (type === "int" && !Number.isSafeInteger(result)) throw new RuntimeError("validation_error", "转换后的整数超出安全范围", { nodeId: node.id, componentId: node.componentId });
return result;
});
@@ -0,0 +1,147 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
const powershellMock = vi.hoisted(() => vi.fn(async () => '{"clicked":true,"value":"cached"}'));
vi.mock("../driver/windows/powershell", () => ({
powershell: powershellMock,
psString: (value: unknown) => JSON.stringify(String(value ?? ""))
}));
import { getHandler } from "../engine/registry";
import { parseUiaSelector } from "./desktop-uia";
describe("desktop UIA ground capabilities", () => {
it.each([
"Element_ClickGround",
"Element_SetValueGround",
"Element_ExistsGround",
"Element_GetValueGround",
"Element_WaitElementGround",
"Element_GetCheckGround",
"Element_SetCheckGround",
"Element_RegionGround",
"Element_GetSelectGround",
"Element_SetSelectGround",
"Element_GetAttributeGround",
"Element_SetAttributeGround",
"Element_GetButtonEnableGround",
"Element_ShotGround",
"Element_MouseToGround",
"Element_ScrollIntoViewGround",
"Element_ChildrenGround",
"Element_DesktopChildren",
"Element_RestoreWindowGround",
"Element_ActivateWindowGround",
"Element_MoveWindowGround"
])("registers %s", (id) => {
expect(getHandler(id), id).toBeTruthy();
});
it("parses name selector", () => {
const s = parseUiaSelector("name=添加供应商");
expect(s.name).toBe("添加供应商");
expect(s.index).toBe(1);
});
it("parses window, id, type, class and index", () => {
const s = parseUiaSelector("window=CC Switch;id=btn-save;type=button;class=btn;index=2");
expect(s.window).toBe("CC Switch");
expect(s.id).toBe("btn-save");
expect(s.type).toBe("Button");
expect(s.className).toBe("btn");
expect(s.index).toBe(2);
});
it("strips uia: prefix and accepts object form", () => {
const s = parseUiaSelector({ value: "uia:name=确定" });
expect(s.name).toBe("确定");
});
it("parses AI screen coordinate selector", () => {
const s = parseUiaSelector("screen=100,200,320,80;window= MyApp;ai=1");
expect(s.screen).toEqual({ x: 100, y: 200, w: 320, h: 80 });
expect(s.window).toBe("MyApp");
});
it("parses enriched AI relocation metadata", () => {
const selector = parseUiaSelector(
"screen=100,200,320,80;window=App;winClass=Chrome_WidgetWin_1;process=chrome;rel=20,30,320,80;name=login;id=loginButton;type=Button;class=Button;aiType=button;aiText=login;enriched=1;ai=1"
);
expect(selector.screen).toEqual({ x: 100, y: 200, w: 320, h: 80 });
expect(selector.rel).toEqual({ x: 20, y: 30, w: 320, h: 80 });
expect(selector.window).toBe("App");
expect(selector.winClass).toBe("Chrome_WidgetWin_1");
expect(selector.process).toBe("chrome");
expect(selector).toMatchObject({ name: "login", id: "loginButton", type: "Button", className: "Button", aiType: "button", aiText: "login", enriched: true, ai: true });
});
it("rejects invalid selector", () => {
expect(() => parseUiaSelector("")).toThrow();
expect(() => parseUiaSelector("screen=1,2,3")).toThrow();
});
});
describe("desktop UIA selector compatibility", () => {
beforeEach(() => {
powershellMock.mockClear();
powershellMock.mockImplementation(async () => '{"clicked":true,"value":"cached"}');
});
it("maps compound .NET ControlType names explicitly and preserves raw input", () => {
expect(parseUiaSelector("type=combobox")).toMatchObject({ type: "ComboBox", typeRaw: "combobox" });
expect(parseUiaSelector("type=listitem")).toMatchObject({ type: "ListItem", typeRaw: "listitem" });
expect(parseUiaSelector("type=datagrid")).toMatchObject({ type: "DataGrid", typeRaw: "datagrid" });
});
it("parses a stable ancestor path", () => {
const selector = parseUiaSelector("window=App;path=Pane>Edit[name=用户名;index=2]>Button[name=确定]");
expect(selector.path).toEqual([
{ type: "Pane" },
{ type: "Edit", name: "用户名", index: 2 },
{ type: "Button", name: "确定" }
]);
});
const context = { log: vi.fn() } as any;
const node = { id: "n1", componentId: "Element_ClickGround" } as any;
it("resolves screen selectors through UIA/window before coordinate fallback", async () => {
const args = {
selector: "screen=100,200,320,80;ai=1",
clickway: "1"
};
await expect(getHandler("Element_ClickGround")!(context, node, args, {} as any)).resolves.toEqual({ clicked: true, value: "cached" });
const script = String((powershellMock.mock.calls as unknown as Array<[string]>)[0]?.[0] ?? "");
expect(script).toContain("RpaScreenInput");
expect(script).toContain("$x = [int]($rect.x + $rect.w / 2)");
expect(script).toContain("SetCursorPos($x, $y)");
expect(script).toContain("Resolve-RpaScreenRect");
expect(script).toContain("Find-RpaWindow");
});
it("dispatches screen setValue to coordinate paste", async () => {
await getHandler("Element_SetValueGround")!(context, node, {
selector: "screen=10,20,30,40",
value: "hello"
}, {} as any);
const script = String((powershellMock.mock.calls as unknown as Array<[string]>)[0]?.[0] ?? "");
expect(script).toContain("resolvedMethod = $rect.method");
expect(script).toContain("hello");
expect(script).toContain("Find-RpaWindow");
});
it("validates a picked screen rectangle for exists/wait", async () => {
powershellMock.mockImplementation(async () => '{"found":true,"method":"window_rel"}');
await expect(getHandler("Element_ExistsGround")!(context, node, {
selector: "screen=1,2,3,4"
}, {} as any)).resolves.toBe(true);
await expect(getHandler("Element_WaitElementGround")!(context, node, {
selector: "screen=1,2,3,4"
}, {} as any)).resolves.toBe(true);
expect(powershellMock).toHaveBeenCalled();
const script = String((powershellMock.mock.calls as unknown as Array<[string]>).at(-1)?.[0] ?? "");
expect(script).toContain("Resolve-RpaScreenRect");
});
});
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,232 @@
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { type ManualOperationSettings, defaultEngine } from "../engine/engine";
import { powershell, psString } from "../driver/windows/powershell";
// 人为拟真设置本体挂在引擎实例上(任务 1.3),此处委托 defaultEngine。
const manualOperation: ManualOperationSettings = defaultEngine.manualOperation;
// 桌面自动化驱动(任务 3.3):user32 注入原语统一走 WindowsDriver
const driver = defaultEngine.driver;
function jitter(value: number, amount = 3): number {
return Math.max(0, Math.round(value + (Math.random() * amount * 2 - amount)));
}
function manualJitter(point: [number, number], move: boolean, click: boolean): [number, number] {
if (!manualOperation.enabled) return point;
const amount = (click && manualOperation.randomClick) || (move && manualOperation.randomMove) ? 3 : 0;
return [jitter(point[0], amount), jitter(point[1], amount)];
}
async function manualDelay(): Promise<void> {
if (!manualOperation.enabled || !manualOperation.randomSleep) return;
const low = Math.max(0, manualOperation.minMs);
const high = Math.max(low, manualOperation.maxMs);
const timeout = low + Math.random() * (high - low);
if (timeout > 0) await new Promise((resolve) => setTimeout(resolve, timeout));
}
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
if (args[key] !== undefined && args[key] !== null && args[key] !== "") return String(args[key]);
}
return fallback;
}
function bool(value: unknown): boolean {
return [true, 1, "1", "true", "True", "是", "开启"].includes(value as never);
}
function number(args: Record<string, unknown>, keys: string[], fallback = 0): number {
const value = Number(text(args, keys, String(fallback)));
return Number.isFinite(value) ? value : fallback;
}
function point(value: unknown): [number, number] {
if (Array.isArray(value)) return [Number(value[0]) || 0, Number(value[1]) || 0];
try {
const parsed = JSON.parse(String(value || "[0,0]"));
return point(parsed);
} catch {
const values = String(value || "").split(/[ ,,]+/).map(Number);
return [values[0] || 0, values[1] || 0];
}
}
const keyboardSend: CapabilityHandler = async (_ctx, node, args) => {
// 字段: words=输入内容(键盘输入), keys/key=按键组合(按键系列), interval=输入间隔(V1毫秒/基础版秒), fuctype=输入方式(0后台接口/1人工模拟/2剪贴板粘贴), window_title=输入目标窗口(留空=当前焦点窗口)
const keys = text(args, ["words", "keys", "key", "text", "value"]);
if (!keys) throw new RuntimeError("validation_error", "键盘输入为空", { nodeId: node.id, componentId: node.componentId });
await manualDelay();
const rawInterval = number(args, ["interval"], 0);
const intervalMs = node.componentId === "Keyboard_Send" ? Math.max(0, Math.round(rawInterval * 1000)) : Math.max(0, Math.round(rawInterval));
const mode = text(args, ["fuctype"], "0");
const isTextSend = node.componentId === "Keyboard_Send" || node.componentId === "Keyboard_Send_V1";
// 指定目标窗口时先激活再输入,内容才会进到目标窗口而不是运行时的焦点窗口
const targetTitle = text(args, ["window_title", "target_window", "window_obj", "match_title", "select_title"]);
await driver.keyboard.send({ keys, intervalMs, mode: mode === "2" ? "clipboard" : mode === "1" ? "manual" : "api", textMode: isTextSend, activateTitle: targetTitle || undefined });
await manualDelay();
return keys;
};
const enableManualOperation: CapabilityHandler = async (_ctx, _node, args) => {
const min = number(args, ["min_time", "minTime"], 1) * 1000;
const max = number(args, ["max_time", "maxTime"], 3) * 1000;
manualOperation.enabled = true;
manualOperation.randomMove = bool(args.random_move ?? true);
manualOperation.randomClick = bool(args.random_click ?? true);
manualOperation.randomSleep = bool(args.random_sleep ?? true);
manualOperation.minMs = min;
manualOperation.maxMs = Math.max(min, max);
return { enabled: true, randomMove: manualOperation.randomMove, randomClick: manualOperation.randomClick, randomSleep: manualOperation.randomSleep, minMs: manualOperation.minMs, maxMs: manualOperation.maxMs };
};
const disableManualOperation: CapabilityHandler = async () => {
manualOperation.enabled = false;
return { enabled: false };
};
const mouseClick: CapabilityHandler = async (_ctx, node, args) => {
const [x, y] = manualJitter(point(args.position ?? args.point ?? args.location ?? "[0,0]"), false, true);
const button = text(args, ["side", "button"], "0").toLowerCase();
await driver.mouse.click({ x, y }, { button: button.includes("right") || button === "1" ? "right" : "left" });
return { x, y };
};
const mouseMove: CapabilityHandler = async (_ctx, node, args) => {
const [x, y] = manualJitter(point(args.position ?? args.point ?? args.location ?? "[0,0]"), true, false);
await driver.mouse.moveTo({ x, y });
return { x, y };
};
const mouseDrag: CapabilityHandler = async (_ctx, node, args) => {
const [sx, sy] = point(args.Start ?? args.start ?? "[0,0]");
const [ex, ey] = point(args.End ?? args.end ?? "[0,0]");
await driver.mouse.drag({ x: sx, y: sy }, { x: ex, y: ey }, { holdMs: 80, releaseDelayMs: 80 });
return { start: [sx, sy], end: [ex, ey] };
};
const mouseRoll: CapabilityHandler = async (_ctx, node, args) => {
const amount = number(args, ["RollNum", "rollNum", "amount"], 1) * 120;
await driver.mouse.scroll(amount);
return amount;
};
const setActiveWindow: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.focus(selector);
};
const getActiveWindow: CapabilityHandler = async () => driver.window.foregroundTitle();
const windowExists: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.exists(selector);
};
const closeWindow: CapabilityHandler = async (_ctx, node, args) => {
const selector = text(args, ["window_obj", "match_title", "select_title", "title", "target_window", "window_title"]);
if (!selector) throw new RuntimeError("validation_error", "窗口标题为空", { nodeId: node.id, componentId: node.componentId });
return driver.window.close(selector);
};
const runApp: CapabilityHandler = async (_ctx, node, args) => {
const executable = text(args, ["executeable", "executable", "path"]);
if (!executable) throw new RuntimeError("validation_error", "应用路径为空", { nodeId: node.id, componentId: node.componentId });
const params = text(args, ["params", "arguments"]);
const cwd = text(args, ["cwd"]);
const argPart = params ? ` -ArgumentList ${psString(params)}` : "";
const result = await powershell(`$p = Start-Process -FilePath ${psString(executable)}${argPart} ${cwd ? `-WorkingDirectory ${psString(cwd)}` : ""} -PassThru; $p.Id`);
return Number(result) || result;
};
const clipboardGet: CapabilityHandler = async () => powershell("Get-Clipboard -Raw");
const clipboardSet: CapabilityHandler = async (_ctx, _node, args) => {
const content = text(args, ["text", "value"]);
await powershell(`Set-Clipboard -Value ${psString(content)}`);
return content;
};
const notice: CapabilityHandler = async (_ctx, _node, args) => {
const title = text(args, ["title"], "提醒");
const content = text(args, ["text", "message"]);
await powershell(`Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.MessageBox]::Show(${psString(content)},${psString(title)}) | Out-Null`);
return true;
};
const executeCommand: CapabilityHandler = async (_ctx, node, args) => {
const command = text(args, ["action", "command"]);
if (!command) throw new RuntimeError("validation_error", "命令行为空", { nodeId: node.id, componentId: node.componentId });
return powershell(command, Math.max(1000, number(args, ["timeoutMs"], 120000)));
};
register(["Keyboard_Send_V1", "Keyboard_Send", "Keyboard_PressNew", "Keyboard_Press"], keyboardSend);
register("Keyboard_EnableManualOperation", enableManualOperation);
register("Keyboard_DisableManualOperation", disableManualOperation);
register(["Mouse_Click", "Mouse_ClickNew"], mouseClick);
register(["Mouse_MoveTo", "Mouse_Move"], mouseMove);
register(["Mouse_DragTo", "Mouse_Drag"], mouseDrag);
register(["Mouse_RollTo", "Mouse_Scroll"], mouseRoll);
register(["Window_SetActive", "Window_Activate"], setActiveWindow);
register(["Window_GetActive", "Window_GetActiveWindow"], getActiveWindow);
register(["Window_WindowExists", "Window_Exists"], windowExists);
register(["Window_CloseWindow", "Window_Close"], closeWindow);
register(["App_Run", "App_Start"], runApp);
register("Clipboard_GetText", clipboardGet);
register("Clipboard_SetText", clipboardSet);
register("Dialog_Notice_V1", async (_ctx, node, args) => {
const msg = text(args, ["text", "message"]);
const title = text(args, ["title"], "通知");
const timeout = Math.max(1, number(args, ["timeout", "time_out"], 10));
return powershell("(New-Object -ComObject WScript.Shell).Popup(" + psString(msg) + ", " + timeout + ", " + psString(title) + ", 0) | Out-Null; $true");
});
register("Dialog_Notice_V2", async (ctx, node, args) => {
const msg = text(args, ["text", "message"]);
const title = text(args, ["title"], "通知");
const timeout = Math.max(1, number(args, ["time_out", "timeout"], 10));
return powershell("(New-Object -ComObject WScript.Shell).Popup(" + psString(msg) + ", " + timeout + ", " + psString(title) + ", 0) | Out-Null; $true");
});
register("Dialog_CsMsgBox", async (_ctx, _node, args) => {
const msg = text(args, ["text", "message"]);
const title = text(args, ["title"], "提示");
const buttons = Number(args.msg_button ?? 0);
const autoClose = Number(args.autoClose ?? 0);
const time = Math.max(1, Number(args.time ?? 10));
const popup = "(New-Object -ComObject WScript.Shell).Popup(" + psString(msg) + ", " + (autoClose ? time : -1) + ", " + psString(title) + ", " + buttons + ")";
const choice = await powershell(popup + " | Out-Null; if (" + (autoClose ? "$true" : "$false") + ") {}");
return powershell(popup);
});
register("Dialog_OpenFile", async (_ctx, _node, args) => {
const title = text(args, ["title"], "选择文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }");
});
register("Dialog_OpenFiles", async (_ctx, _node, args) => {
const title = text(args, ["title"], "选择文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.OpenFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; $dlg.Multiselect = $true; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileNames -join ';' } else { '' }");
});
register("Dialog_SaveFile", async (_ctx, _node, args) => {
const title = text(args, ["title"], "保存文件");
const wildcard = text(args, ["wildcard"], "所有文件|*.*");
return powershell("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $dlg = New-Object System.Windows.Forms.SaveFileDialog; $dlg.Filter = " + psString(wildcard) + "; $dlg.Title = " + psString(title) + "; if ($dlg.ShowDialog() -eq 'OK') { $dlg.FileName } else { '' }");
});
register(["Dialog_InputBox", "Dialog_CsInputBox"], async (_ctx, _node, args) => {
const title = text(args, ["title", "input_title"], "请输入");
const description = text(args, ["description", "txt_content"], "");
const defaultValue = text(args, ["text"], "");
return powershell("Add-Type -AssemblyName Microsoft.VisualBasic | Out-Null; [Microsoft.VisualBasic.Interaction]::InputBox(" + psString(description) + ", " + psString(title) + ", " + psString(defaultValue) + ")");
});
register("Dialog_Notice", notice);
register("Dialog_MsgBox", notice);
register("System_ExecuteCommand", executeCommand);
@@ -0,0 +1,63 @@
import { mkdtemp, readFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import ExcelJS from "exceljs";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./excel-com";
const directories: string[] = [];
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown>) {
const handler = getHandler(id);
expect(handler, id).toBeTruthy();
return handler!(context, node(id), args, component);
}
afterEach(async () => Promise.all(directories.splice(0).map((directory) => rm(directory, { recursive: true, force: true }))));
const registered = [
"Excel_Cs_Group", "Excel_Cs_DeletePicture", "Excel_Cs_RefreshPivotTable", "Excel_Cs_DeletePictureV2",
"Excel_Cs_Encrypt", "Excel_Cs_Move", "Excel_Cs_InsertPicture", "Excel_Cs_Merge", "Excel_InsertImage",
"Excel_Cs_InsertPivotView", "Excel_Cs_Interop", "Excel_Cs_InsertPivotTable", "Excel_Cs_RunMacro",
"Excel_Loop", "Excel_Cs_Export", "Excel_Cs_ExcelAddin", "Excel_Cs_InsertPictureV2",
"Excel_Cs_FilterPivotTable", "Excel_Cs_RunMacroV2", "Excel_Cs_SetSlicer", "Excel_ExecuteMacro",
"Excel_Cs_RefreshAll", "Excel_DeleteImage", "Excel_ExportRangeAsPicture"
];
describe("Excel COM capabilities", () => {
it("registers every Excel COM capability", () => { for (const id of registered) expect(getHandler(id), id).toBeTruthy(); });
it("groups rows, moves sheets, reads dynamic properties and loops content", async () => {
if (process.platform !== "win32") return;
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-com-")); directories.push(directory);
const filePath = path.join(directory, "workbook.xlsx");
const workbook = new ExcelJS.Workbook();
workbook.addWorksheet("Sheet1").getCell("A1").value = "A1";
workbook.addWorksheet("Second").getCell("A1").value = "S1";
await workbook.xlsx.writeFile(filePath);
await expect(run("Excel_Cs_Group", { workbook: { path: filePath }, sheet: "Sheet1", group: 0, groupType: 0, startRow: 1, endRow: 2, save: true })).resolves.toMatchObject({ grouped: true });
await expect(run("Excel_Cs_Interop", { workbook: { path: filePath }, obj: 2, sheet: "Sheet1", property: "Name" })).resolves.toMatchObject({ value: "Sheet1" });
await expect(run("Excel_Cs_Move", { workbook: { path: filePath }, sheet: "Second", mode: "按序号", index: 1, save: true })).resolves.toMatchObject({ moved: true });
await expect(run("Excel_Cs_RefreshAll", { workbook: { path: filePath }, save: true })).resolves.toMatchObject({ refreshed: true });
const loop = await run("Excel_Loop", { objExcel: { path: filePath } }) as unknown[][];
expect(loop.length).toBeGreaterThanOrEqual(2);
expect(loop.flat().join(",")).toContain("A1");
}, 120000);
it("exports a workbook to PDF through Excel COM", async () => {
if (process.platform !== "win32") return;
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-pdf-")); directories.push(directory);
const filePath = path.join(directory, "source.xlsx");
const workbook = new ExcelJS.Workbook();
workbook.addWorksheet("Report").getCell("A1").value = "PDF";
await workbook.xlsx.writeFile(filePath);
const output = await run("Excel_Cs_Export", { workbook: { path: filePath }, option: "全部", path: directory, name: "report.pdf", cover: true }) as string;
expect(output).toBe(path.join(directory, "report.pdf"));
expect((await readFile(output)).subarray(0, 5).toString()).toMatch(/^%PDF-/);
}, 120000);
});
@@ -0,0 +1,288 @@
import fs from "node:fs";
import path from "node:path";
import ExcelJS from "exceljs";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell, psString } from "../driver/windows/powershell";
import { releaseWorkbook } from "./excel-store";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
const value = args[key];
if (value !== undefined && value !== null && value !== "") return String(value);
}
return fallback;
}
function bool(value: unknown, fallback = false): boolean {
if (value === undefined || value === null || value === "") return fallback;
return [true, 1, "1", "true", "True", "TRUE", "yes", "是", "开启"].includes(value as never);
}
function number(value: unknown, fallback = 0): number { const result = Number(value); return Number.isFinite(result) ? result : fallback; }
function workbookPath(args: Record<string, unknown>, nodeId: string, componentId: string, keys = ["workbook", "objExcel", "objExcelWorkBook", "destWorkbook"]): string {
for (const key of keys) {
const raw = args[key];
const candidate = raw && typeof raw === "object" ? (raw as { path?: unknown }).path : raw;
const value = String(candidate ?? "").trim();
if (value && value !== "Excel文件" && value !== "objExcelWorkBook") return path.resolve(value);
}
throw new RuntimeError("validation_error", "Excel文件路径为空", { nodeId, componentId });
}
function sheetExpression(args: Record<string, unknown>): string {
const sheet = text(args, ["sheet", "sheetName"]);
return sheet ? `$workbook.Worksheets.Item(${psString(sheet)})` : "$workbook.ActiveSheet";
}
async function runExcelCom(filePath: string, body: string, save = false, readonly = false, timeout = 120000): Promise<unknown> {
const openMode = readonly ? "$true" : "$false";
const saveMode = save ? "$true" : "$false";
const script = [
"$excel=$null;$workbook=$null;$rpaOutput=$null",
"try {",
" $excel=New-Object -ComObject Excel.Application;$excel.Visible=$false;$excel.DisplayAlerts=$false",
" $workbook=$excel.Workbooks.Open(" + psString(filePath) + ",0," + openMode + ")",
" " + body,
" if (" + saveMode + ") { $workbook.Save() }",
" if ($null -ne $rpaOutput) { [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes(($rpaOutput | ConvertTo-Json -Compress -Depth 6))) | Write-Output }",
"} finally {",
" if($null -ne $workbook){try{$workbook.Close($false)}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($workbook)}catch{}}",
" if($null -ne $excel){try{$excel.Quit()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($excel)}catch{}}",
" [GC]::Collect();[GC]::WaitForPendingFinalizers()",
"}"
].join("\n");
const output = (await powershell(script, timeout)).trim();
if (save) releaseWorkbook(filePath);
if (!output) return null;
try { return JSON.parse(Buffer.from(output, "base64").toString("utf8")); }
catch { throw new RuntimeError("excel_com_failed", "Excel COM 输出异常", { detail: output.slice(0, 300) }); }
}
function list(value: unknown): string[] {
if (Array.isArray(value)) return value.map((item) => String(item)).filter(Boolean);
const raw = String(value ?? "").trim();
if (!raw) return [];
try { const parsed = JSON.parse(raw); return Array.isArray(parsed) ? parsed.map(String) : [String(parsed)]; }
catch { return raw.split(/[,,;;\s]+/).filter(Boolean); }
}
const group: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const ungroup = String(args.group ?? "0") === "1";
const columns = String(args.groupType ?? "0") === "1";
const start = columns ? text(args, ["startColumn"], "A") : String(number(args.startRow, 1));
const end = columns ? text(args, ["endColumn"], text(args, ["startColumn"], "A")) : String(number(args.endRow, number(args.startRow, 1)));
const method = ungroup ? "Ungroup" : "Group";
const body = `$sheet=${sheetExpression(args)};$start=${psString(start)};$end=${psString(end)};
$count=if(${columns ? "$true" : "$false"}){$sheet.UsedRange.Columns.Count}else{$sheet.UsedRange.Rows.Count};
foreach($name in @("start","end")){ $v=[int](Get-Variable $name -ValueOnly); if($v -lt 0){ Set-Variable $name ($count+$v+1) } };
$range=$sheet.Range("$($start):$($end)");$range.${method}|Out-Null`;
await runExcelCom(filePath, body, bool(args.save));
return { grouped: !ungroup, type: columns ? "columns" : "rows" };
};
const deletePicture: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook", "objExcel"]);
const name = text(args, ["name", "pictureName"]);
if (!name) throw new RuntimeError("validation_error", "图片名称为空", { nodeId: node.id, componentId: node.componentId });
const body = `$sheet=${sheetExpression(args)};$sheet.Shapes.Item(${psString(name)}).Delete()`;
await runExcelCom(filePath, body, bool(args.save));
return { deleted: name };
};
const insertPicture: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook", "objExcel"]);
const image = path.resolve(text(args, ["path", "pictureName"]));
if (!fs.existsSync(image)) throw new RuntimeError("validation_error", "图片文件不存在", { nodeId: node.id, componentId: node.componentId, detail: image });
const row = number(args.row, 1); const column = number(args.column, 1);
const left = number(args.Left ?? args.left, 20); const top = number(args.Top ?? args.top, 20);
const width = number(args.Width ?? args.width, -1); const height = number(args.Height ?? args.height, -1);
const scale = width > 0 && height > 0 ? "$true" : "$false";
const body = `$sheet=${sheetExpression(args)};$cell=$sheet.Cells(${row},${column});$left=${left}+if($left -gt 0){$cell.Left}else{0};$top=${top}+if($top -gt 0){$cell.Top}else{0};[void]$sheet.Shapes.AddPicture(${psString(image)},$false,${scale},$left,$top,${width},${height})`;
await runExcelCom(filePath, body, bool(args.save));
return { inserted: image };
};
const encryptExcel: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const password = String(args.password1 ?? args.password ?? "");
const confirm = String(args.password2 ?? password);
if (!password) throw new RuntimeError("validation_error", "Excel密码为空", { nodeId: node.id, componentId: node.componentId });
if (password !== confirm) throw new RuntimeError("validation_error", "两次输入的Excel密码不一致", { nodeId: node.id, componentId: node.componentId });
const temporary = `${filePath}.${Date.now()}.enc.xlsx`;
try {
await runExcelCom(filePath, `$workbook.SaveAs(${psString(temporary)},51,${psString(password)},${psString(confirm)})`, false);
fs.rmSync(temporary, { force: true });
// COM SaveAs cannot replace the workbook while it is open, so save beside it and rename after Close.
} finally {
if (fs.existsSync(temporary)) {
fs.rmSync(filePath, { force: true });
fs.renameSync(temporary, filePath);
}
releaseWorkbook(filePath);
}
return filePath;
};
const moveSheet: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const mode = text(args, ["mode"], "按序号"); const after = text(args, ["option"], "之前").includes("之后");
const targetName = text(args, ["name", "index"]); const position = text(args, ["position", "index"], "第一位");
const index = number(args.index, 1);
const afterMode = after ? "$true" : "$false";
const body = `$sheet=${sheetExpression(args)};$count=$workbook.Worksheets.Count;$targetIndex=0;
if(${psString(mode)} -eq "按名称"){ $targetName=${psString(targetName)}; for($i=1;$i -le $count;$i++){ if($workbook.Worksheets.Item($i).Name -eq $targetName){$targetIndex=$i;break} }; if($targetIndex -eq 0){throw "target_sheet_not_found"} }
elseif(${psString(mode)} -eq "按位置"){ $p=${psString(position)}; if($p -eq "第一位"){$targetIndex=1}elseif($p -eq "最后一位"){$targetIndex=$count}else{ $delta=-1; if($p -eq "向后移一位"){$delta=1}; $targetIndex=$sheet.Index+$delta } }
else { $targetIndex=${index}; if($targetIndex -lt 0){$targetIndex=$count+$targetIndex+1} };
$targetIndex=[Math]::Min([Math]::Max(1,$targetIndex),$count);$target=$workbook.Worksheets.Item($targetIndex);
if(${afterMode}){ if($target.Index -ne $sheet.Index){ $sheet.Move([Type]::Missing,$target)|Out-Null } }
else { if($target.Index -ne $sheet.Index){ $sheet.Move($target)|Out-Null } }`;
await runExcelCom(filePath, body, bool(args.save));
return { moved: true };
};
const mergeSheet: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const destPath = workbookPath(args, node.id, node.componentId, ["destWorkbook", "workbook"]);
const horizontal = String(args.option ?? "0") === "1";
const horizontalMode = horizontal ? "$true" : "$false";
const saveDest = bool(args.save) ? "$true" : "$false";
const body = `$source=${sheetExpression(args)};$dest=$excel.Workbooks.Open(${psString(destPath)});$target=$dest.Worksheets.Item(${psString(text(args, ["destSheet"], "Sheet1"))});
if(${horizontalMode}){$offset=$target.UsedRange.Columns.Count+1}else{$offset=$target.UsedRange.Rows.Count+1};
for($r=1;$r -le $source.UsedRange.Rows.Count;$r++){for($c=1;$c -le $source.UsedRange.Columns.Count;$c++){ if(${horizontalMode}){$target.Cells($r,$offset+$c-1).Value2=$source.Cells($r,$c).Value2}else{$target.Cells($offset+$r-1,$c).Value2=$source.Cells($r,$c).Value2}}};
if(${saveDest}){$dest.Save()}`;
await runExcelCom(filePath, body, false);
return { merged: true, target: destPath };
};
const refreshAll: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
await runExcelCom(filePath, "$workbook.RefreshAll()", bool(args.save));
return { refreshed: true };
};
const refreshPivot: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const name = text(args, ["pivotTable"]); const sheetExpr = sheetExpression(args);
const body = name ? `${sheetExpr}.PivotTables(${psString(name)}).RefreshTable()` : `${sheetExpr}.PivotTables() | ForEach-Object { $_.RefreshTable() }`;
await runExcelCom(filePath, body, bool(args.save));
return { refreshed: true, name: name || "all" };
};
const filterPivot: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const field = text(args, ["field"]); const values = list(args.fieldValue);
if (!field) throw new RuntimeError("validation_error", "筛选字段为空", { nodeId: node.id, componentId: node.componentId });
const encoded = Buffer.from(JSON.stringify(values), "utf16le").toString("base64");
const body = `$pt=${sheetExpression(args)}.PivotTables(${psString(text(args, ["pivotTable"]))});$pf=$pt.PivotFields(${psString(field)});
try{$pf.ClearAllFilters()}catch{};
$wanted=[Text.Encoding]::Unicode.GetString([Convert]::FromBase64String("${encoded}"))|ConvertFrom-Json;
foreach($item in $pf.PivotItems()){ $selected=if($wanted.Count -eq 0){$true}else{$wanted -contains $item.Name}; $item.Visible=$selected }`;
await runExcelCom(filePath, body, bool(args.save));
return { field, values };
};
const createPivot: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const chart = node.componentId.includes("PivotView");
let rule: Record<string, unknown> = {};
try { rule = JSON.parse(String(args.rule || "{}")) as Record<string, unknown>; } catch { /* sourceData can still be supplied separately. */ }
const source = text({ ...rule, ...args }, ["SourceData", "sourceData", "source"]);
if (!source) throw new RuntimeError("validation_error", "透视表数据源为空", { nodeId: node.id, componentId: node.componentId });
const name = text(rule, ["Name", "name"], chart ? "RPAPivotChart" : "RPAPivotTable");
const body = `$sheet=${sheetExpression(args)};$cache=$excel.ActiveWorkbook.PivotCaches().Create(1,$sheet.Range(${psString(source)}));
$dest=$sheet.Cells(1,20);$pt=$cache.CreatePivotTable($dest,${psString(name)});
if(${chart ? "$true" : "$false"}){[void]$sheet.Shapes.AddChart(-4109,20,20,420,260)}`;
await runExcelCom(filePath, body, bool(args.save));
return { name, chart };
};
const runMacro = (scriptKey: "script" | "code" | "macroName"): CapabilityHandler => async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook", "objExcel"]);
const macro = text(args, [scriptKey, "script", "code", "macroName", "name"]);
if (!macro) throw new RuntimeError("validation_error", "宏名称为空", { nodeId: node.id, componentId: node.componentId });
const parameters = list(args.args ?? args.macroArgs).map((item) => psString(item)).join(",");
const body = `$result=$excel.Run(${psString(macro)}${parameters ? "," + parameters : ""});$rpaOutput=@{result=$result}`;
return await runExcelCom(filePath, body, bool(args.save), false, 180000);
};
const interop: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook", "objExcel"]);
const property = text(args, ["property"]);
if (!property) throw new RuntimeError("validation_error", "属性或方法名为空", { nodeId: node.id, componentId: node.componentId });
const objectKind = number(args.obj ?? args.object, 0);
const range = text(args, ["range"], "A1");
const target = objectKind === 0 ? "$excel" : objectKind === 1 ? "$workbook" : objectKind === 2 ? sheetExpression(args) : `${sheetExpression(args)}.Range(${psString(range)})`;
const value = args.value;
const empty = value === undefined || value === null || value === "";
const body = empty
? `$target=${target};$rpaOutput=@{value=$target.GetType().InvokeMember(${psString(property)},[Reflection.BindingFlags]::GetProperty,$null,$target,@())}`
: `$target=${target};[void]$target.GetType().InvokeMember(${psString(property)},[Reflection.BindingFlags]::SetProperty,$null,$target,@(${psString(String(value))}));$rpaOutput=@{set=$true}`;
return await runExcelCom(filePath, body, bool(args.save));
};
const exportPdf: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const folder = text(args, ["path"]);
const name = text(args, ["name"], path.basename(filePath, path.extname(filePath)) + ".pdf");
const output = path.join(path.resolve(folder), /\.(pdf)$/i.test(name) ? name : `${name}.pdf`);
fs.mkdirSync(path.dirname(output), { recursive: true });
if (!bool(args.cover, true) && fs.existsSync(output)) throw new RuntimeError("validation_error", "PDF 文件已存在", { detail: output });
const activateOnly = text(args, ["option"], "全部") === "激活页";
const body = `if(${activateOnly ? "$true" : "$false"}){$workbook.ActiveSheet.ExportAsFixedFormat(0,${psString(output)})}else{$workbook.ExportAsFixedFormat(0,${psString(output)})}`;
await runExcelCom(filePath, body, false);
return output;
};
const exportRangePicture: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const folder = text(args, ["path"]); const name = text(args, ["name"], `excel-range-${Date.now()}`);
const output = path.join(path.resolve(folder), /\.(png|jpg|jpeg)$/i.test(name) ? name : `${name}.png`);
fs.mkdirSync(path.dirname(output), { recursive: true });
const body = `$range=${sheetExpression(args)}.Range(${psString(text(args, ["range"], "A1"))});$range.CopyPicture(1,2)|Out-Null;$shape=${sheetExpression(args)}.ChartObjects().Add(0,0,$range.Width,$range.Height);$chart=$shape.Chart;$chart.Paste();[void]$chart.Export(${psString(output)},"PNG");$shape.Delete()`;
await runExcelCom(filePath, body, false);
return output;
};
const setSlicer: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["workbook"]);
const name = text(args, ["slicerName"]); const values = list(args.slicerOptions);
if (!name) throw new RuntimeError("validation_error", "切片器名称为空", { nodeId: node.id, componentId: node.componentId });
const encoded = Buffer.from(JSON.stringify(values), "utf16le").toString("base64");
const body = `$cache=$workbook.SlicerCaches.Item(${psString(name)});
$wanted=[Text.Encoding]::Unicode.GetString([Convert]::FromBase64String("${encoded}"))|ConvertFrom-Json;
foreach($item in $cache.SlicerItems()){ $item.Selected=if($wanted.Count -eq 0){$true}else{$wanted -contains $item.Name} }`;
await runExcelCom(filePath, body, bool(args.save));
return { slicer: name, values };
};
const excelLoop: CapabilityHandler = async (_ctx, node, args) => {
const filePath = workbookPath(args, node.id, node.componentId, ["objExcel", "workbook"]);
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(filePath);
const rows: unknown[][] = [];
for (const sheet of workbook.worksheets) {
for (let row = 1; row <= sheet.rowCount; row += 1) {
const cells: unknown[] = [];
for (let column = 1; column <= sheet.columnCount; column += 1) cells.push(sheet.getCell(row, column).value ?? "");
if (cells.some((cell) => cell !== "")) rows.push(cells);
}
}
return rows;
};
register("Excel_Cs_Group", group);
register("Excel_Cs_Encrypt", encryptExcel);
register(["Excel_Cs_DeletePicture", "Excel_Cs_DeletePictureV2", "Excel_DeleteImage"], deletePicture);
register(["Excel_Cs_InsertPicture", "Excel_Cs_InsertPictureV2", "Excel_InsertImage"], insertPicture);
register("Excel_Cs_Move", moveSheet);
register("Excel_Cs_Merge", mergeSheet);
register(["Excel_Cs_RefreshAll", "Excel_Cs_RefreshPivotTable"], async (ctx, node, args, component) => node.componentId === "Excel_Cs_RefreshPivotTable" ? refreshPivot(ctx, node, args, component) : refreshAll(ctx, node, args, component));
register("Excel_Cs_FilterPivotTable", filterPivot);
register(["Excel_Cs_InsertPivotTable", "Excel_Cs_InsertPivotView"], createPivot);
register(["Excel_Cs_RunMacro", "Excel_Cs_RunMacroV2", "Excel_ExecuteMacro"], runMacro("script"));
register("Excel_Cs_Interop", interop);
register("Excel_Cs_Export", exportPdf);
register("Excel_Cs_ExcelAddin", runMacro("code"));
register("Excel_Cs_SetSlicer", setSlicer);
register("Excel_Loop", excelLoop);
register("Excel_ExportRangeAsPicture", exportRangePicture);
@@ -0,0 +1,147 @@
import { mkdtemp, readFile, readdir, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import ExcelJS from "exceljs";
import { afterEach, describe, expect, it } from "vitest";
import type { Catalog, CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./index";
import { EXCEL_COMMON_IDS } from "./excel-common";
const directories: string[] = [];
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (componentId: string): FlowNode => ({
id: `excel_common_${componentId}`,
componentId,
position: { x: 0, y: 0 },
values: {},
options: {}
});
async function temporaryWorkbook(): Promise<string> {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-common-"));
directories.push(directory);
const filePath = path.join(directory, "common.xlsx");
const workbook = new ExcelJS.Workbook();
const sheet = workbook.addWorksheet("Data");
sheet.addRows([
["name", "score", "group", "parts"],
["Bob", 2, "B", "x,y"],
["Alice", 3, "A", "m,n"],
["Bob", 2, "B", "p,q"]
]);
await workbook.xlsx.writeFile(filePath);
return filePath;
}
async function run(componentId: string, args: Record<string, unknown>): Promise<unknown> {
const handler = getHandler(componentId);
expect(handler, `${componentId} should be registered`).toBeTruthy();
return handler!(context, node(componentId), args, component);
}
async function readWorkbook(filePath: string): Promise<ExcelJS.Workbook> {
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(filePath);
return workbook;
}
afterEach(async () => {
await Promise.all(directories.splice(0).map((directory) => rm(directory, { recursive: true, force: true })));
});
describe("common Excel capabilities", () => {
it("registers at least 25 real catalog IDs without conflicting with existing modules", async () => {
const catalog = JSON.parse(await readFile("resources/catalog.json", "utf8")) as Catalog;
const catalogIds = new Set(catalog.components.map((item) => item.id));
expect(EXCEL_COMMON_IDS.length).toBeGreaterThanOrEqual(25);
for (const id of EXCEL_COMMON_IDS) {
expect(catalogIds.has(id), `${id} should exist in catalog.json`).toBe(true);
expect(getHandler(id), `${id} should be registered`).toBeTruthy();
}
const sourceFiles = (await readdir("src/main/capabilities"))
.filter((file) => file.endsWith(".ts") && !file.endsWith(".test.ts") && file !== "excel-common.ts");
const existingSource = (await Promise.all(sourceFiles.map((file) => readFile(path.join("src/main/capabilities", file), "utf8")))).join("\n");
for (const id of EXCEL_COMMON_IDS) expect(existingSource.includes(`"${id}"`), `${id} must not already be registered`).toBe(false);
});
it("writes and reads rows, columns, and ranges in a real workbook", async () => {
const filePath = await temporaryWorkbook();
await run("Excel_Cs_WriteRow", { workbook: filePath, sheet: "Data", row: 6, column: "A", contents: ["Carol", 9, "C"], save: 1 });
await run("Excel_Cs_WriteColumn", { workbook: filePath, sheet: "Data", row: 2, column: "E", contents: [10, 20, 30], save: 1 });
await run("Excel_Cs_WriteRangeV2", { workbook: filePath, sheet: "Data", row: 8, column: "A", contents: [["x", 1], ["y", 2]], save: 1 });
await expect(run("Excel_Cs_ReadRow", { workbook: filePath, sheet: "Data", rowIndex: 6 })).resolves.toEqual(["Carol", 9, "C", "", ""]);
await expect(run("Excel_GetColData", { objExcel: filePath, sheet: "Data", cellPos: "E2" })).resolves.toEqual([10, 20, 30, "", "", "", "", ""]);
await expect(run("Excel_Cs_ReadRange", { workbook: filePath, sheet: "Data", startRow: 8, startColumn: "A", lastRow: 9, lastColumn: "B" }))
.resolves.toEqual([["x", 1], ["y", 2]]);
await expect(run("Excel_Cs_ReadColumnCount", { workbook: filePath, sheet: "Data", row: 2 })).resolves.toBe(5);
});
it("finds, replaces, sorts, filters, clears filters, and removes duplicates", async () => {
const filePath = await temporaryWorkbook();
await expect(run("Excel_Cs_Find", { workbook: filePath, sheet: "Data", what: "Alice", matchFirst: 1 }))
.resolves.toEqual(["A3"]);
await run("Excel_Cs_Search", { workbook: filePath, sheet: "Data", mode: "替换", input: "Alice", replace: "Alicia", wholeWord: 1, save: 1 });
await run("Excel_Cs_Sort", { workbook: filePath, sheet: "Data", sortRange: "A1:D4", sortOrder: "2", header: 1, save: 1 });
await run("Excel_Cs_Filter", { workbook: filePath, sheet: "Data", column: "C", criteria: "B", condition: "equal", startRow: 1, endRow: 4, onHeader: 1, save: 1 });
let workbook = await readWorkbook(filePath);
let sheet = workbook.getWorksheet("Data")!;
expect(sheet.getCell("A2").value).toBe("Bob");
expect(sheet.getRow(3).hidden || sheet.getRow(4).hidden).toBe(true);
await run("Excel_Cs_ClearExcelFilter", { workbook: filePath, sheet: "Data", save: 1 });
await run("Excel_Cs_DeleteDuplicates", { workbook: filePath, sheet: "Data", columns: ["A", "B", "C"], header: 1, save: 1 });
workbook = await readWorkbook(filePath);
sheet = workbook.getWorksheet("Data")!;
expect(sheet.getRows(1, sheet.rowCount)!.some((row) => row.hidden)).toBe(false);
expect(sheet.rowCount).toBe(3);
expect(sheet.getColumn("A").values).toContain("Alicia");
});
it("copies, pastes, fills, splits, and deletes real ranges", async () => {
const filePath = await temporaryWorkbook();
await run("Excel_CopyRange", { objExcel: filePath, sheet: "Data", cellPosRange: "A2:B3" });
await run("Excel_Cs_PastRange", { workbook: filePath, sheet: "Data", column: "F", row: 2, save: 1 });
await run("Excel_Cs_AutoFill", { workbook: filePath, sheet: "Data", sourceRange: "B2:B3", destination: "B2:B6", save: 1 });
await run("Excel_Cs_ColumnSplitV2", { workbook: filePath, sheet: "Data", column: "D", destination: "J1", comma: 1, tab: 0, startRow: 2, endRow: 4, save: 1 });
await run("Excel_Cs_DeleteRangeV3", { workbook: filePath, sheet: "Data", excelRng: "C2:C4", save: 1 });
const workbook = await readWorkbook(filePath);
const sheet = workbook.getWorksheet("Data")!;
expect(sheet.getCell("F2").value).toBe("Bob");
expect(sheet.getCell("G3").value).toBe(3);
expect(sheet.getCell("B6").value).toBe(6);
expect(sheet.getCell("J2").value).toBe("x");
expect(sheet.getCell("K2").value).toBe("y");
expect(sheet.getCell("C2").value).toBeNull();
});
it("manages sheets and persists active sheet state", async () => {
const filePath = await temporaryWorkbook();
await run("Excel_CreateSheet", { objExcel: filePath, sheet: "Report", isSave: true });
await expect(run("Excel_Cs_GetName", { workbook: filePath })).resolves.toEqual(["Data", "Report"]);
await expect(run("Excel_Cs_Active", { workbook: filePath, mode: "按名称", sheet: "Report", save: 1 })).resolves.toBe("Report");
await run("Excel_DeleteSheet", { objExcel: filePath, sheet: "Data", isSave: true });
await run("Excel_Cs_Close", { workbook: filePath, save: 1 });
const workbook = await readWorkbook(filePath);
expect(workbook.worksheets.map((sheet) => sheet.name)).toEqual(["Report"]);
});
it("sets and reads cell and range colors", async () => {
const filePath = await temporaryWorkbook();
await run("Excel_Cs_SetCellBackground", { workbook: filePath, sheet: "Data", column: "A", row: 2, color: "#12AB34", save: 1 });
await run("Excel_Cs_SetRangeFontV2", { workbook: filePath, sheet: "Data", range: "A2:B3", color: "#445566", save: 1 });
await expect(run("Excel_Cs_ReadCellBackground", { workbook: filePath, sheet: "Data", column: "A", row: 2 })).resolves.toBe("FF12AB34");
const workbook = await readWorkbook(filePath);
const sheet = workbook.getWorksheet("Data")!;
expect((sheet.getCell("A2").fill as ExcelJS.FillPattern).fgColor?.argb).toBe("FF12AB34");
expect(sheet.getCell("B3").font.color?.argb).toBe("FF445566");
});
});
@@ -0,0 +1,826 @@
import ExcelJS from "exceljs";
import path from "node:path";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { acquireWorkbook, releaseWorkbook } from "./excel-store";
type WorkbookSource = { workbook: ExcelJS.Workbook; filePath: string };
type CopiedRange = { values: unknown[][]; styles: Partial<ExcelJS.Style>[][] };
let copiedRange: CopiedRange | undefined;
function value(args: Record<string, unknown>, keys: string[], fallback: unknown = ""): unknown {
for (const key of keys) {
const candidate = args[key];
if (candidate !== undefined && candidate !== null && candidate !== "") return candidate;
}
return fallback;
}
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
return String(value(args, keys, fallback));
}
function truthy(candidate: unknown, fallback = false): boolean {
if (candidate === undefined || candidate === null || candidate === "") return fallback;
return [true, 1, "1", "true", "True", "TRUE", "yes", "是", "开启"].includes(candidate as never);
}
function numberValue(candidate: unknown, fallback: number): number {
const result = Number(candidate);
return Number.isFinite(result) ? result : fallback;
}
function requiredPath(candidate: unknown, nodeId: string, componentId: string): string {
const raw = typeof candidate === "object" && candidate !== null ? (candidate as { path?: unknown }).path : candidate;
const filePath = String(raw ?? "").trim();
if (!filePath) {
throw new RuntimeError("validation_error", "Excel path is empty", {
nodeId,
componentId,
recommendation: "Configure an Excel workbook path"
});
}
return path.resolve(filePath);
}
async function loadWorkbook(args: Record<string, unknown>, nodeId: string, componentId: string): Promise<WorkbookSource> {
const filePath = requiredPath(value(args, ["workbook", "objExcel", "objExcelWorkBook", "file", "path"]), nodeId, componentId);
const workbook = await acquireWorkbook(filePath);
return { workbook, filePath };
}
function worksheetOf(workbook: ExcelJS.Workbook, args: Record<string, unknown>): ExcelJS.Worksheet {
const name = text(args, ["sheet", "sheetName"]);
const sheet = name ? workbook.getWorksheet(name) : workbook.worksheets[0];
if (!sheet) throw new Error(`Sheet not found: ${name || "first sheet"}`);
return sheet;
}
async function saveWhenRequested(source: WorkbookSource, args: Record<string, unknown>, fallback = false): Promise<void> {
if (truthy(value(args, ["save", "isSave"]), fallback)) await source.workbook.xlsx.writeFile(source.filePath);
}
function columnNumber(candidate: unknown): number {
const raw = String(candidate ?? "A").trim().toUpperCase();
if (/^\d+$/.test(raw)) return Math.max(1, Number(raw));
if (!/^[A-Z]+$/.test(raw)) throw new Error(`Invalid column: ${raw}`);
let result = 0;
for (const character of raw) result = result * 26 + character.charCodeAt(0) - 64;
return result;
}
function columnLetter(index: number): string {
let current = Math.max(1, Math.trunc(index));
let result = "";
while (current > 0) {
current -= 1;
result = String.fromCharCode(65 + current % 26) + result;
current = Math.floor(current / 26);
}
return result;
}
function cellValue(candidate: ExcelJS.CellValue): unknown {
if (candidate && typeof candidate === "object" && "result" in candidate) return candidate.result ?? "";
if (candidate && typeof candidate === "object" && "text" in candidate) return candidate.text;
return candidate ?? "";
}
function listValue(candidate: unknown): unknown[] {
if (Array.isArray(candidate)) return candidate;
const raw = String(candidate ?? "").trim();
if (!raw) return [];
try {
const parsed = JSON.parse(raw);
return Array.isArray(parsed) ? parsed : [parsed];
} catch {
try {
const parsed = JSON.parse(raw.replace(/'/g, '"'));
return Array.isArray(parsed) ? parsed : [parsed];
} catch {
return raw.split(",").map((item) => item.trim());
}
}
}
function tableValue(candidate: unknown): unknown[][] {
return listValue(candidate).map((item) => Array.isArray(item) ? item : [item]);
}
function rangeBounds(sheet: ExcelJS.Worksheet, range: string): { top: number; left: number; bottom: number; right: number } {
const normalized = range.trim() || `A1:${columnLetter(Math.max(1, sheet.columnCount))}${Math.max(1, sheet.rowCount)}`;
const [start, end = start] = normalized.split(":").map((part) => part.trim());
const first = sheet.getCell(start);
const last = sheet.getCell(end);
return {
top: Math.min(Number(first.row), Number(last.row)),
left: Math.min(Number(first.col), Number(last.col)),
bottom: Math.max(Number(first.row), Number(last.row)),
right: Math.max(Number(first.col), Number(last.col))
};
}
function explicitRange(args: Record<string, unknown>, sheet: ExcelJS.Worksheet): string {
const direct = text(args, ["excelRng", "range", "cellPosRange", "cellPos", "sortRange"]);
if (direct) return direct;
const top = numberValue(value(args, ["startRow", "row"]), 1);
const left = columnNumber(value(args, ["startColumn", "column"]));
const bottom = numberValue(value(args, ["lastRow", "endRow"]), top);
const right = columnNumber(value(args, ["lastColumn", "endColumn"], left));
return `${columnLetter(left)}${top}:${columnLetter(right)}${bottom || Math.max(1, sheet.rowCount)}`;
}
function cellsInRange(sheet: ExcelJS.Worksheet, range: string): ExcelJS.Cell[] {
const bounds = rangeBounds(sheet, range);
const cells: ExcelJS.Cell[] = [];
for (let row = bounds.top; row <= bounds.bottom; row += 1) {
for (let column = bounds.left; column <= bounds.right; column += 1) cells.push(sheet.getCell(row, column));
}
return cells;
}
function cloneStyle(style: Partial<ExcelJS.Style>): Partial<ExcelJS.Style> {
return JSON.parse(JSON.stringify(style)) as Partial<ExcelJS.Style>;
}
function colorArgb(candidate: unknown): string {
if (Array.isArray(candidate)) {
const [red = 0, green = 0, blue = 0] = candidate.map((item) => Math.max(0, Math.min(255, Number(item))));
return `FF${red.toString(16).padStart(2, "0")}${green.toString(16).padStart(2, "0")}${blue.toString(16).padStart(2, "0")}`.toUpperCase();
}
const raw = String(candidate ?? "#000000").trim();
if (raw.startsWith("[")) {
try { return colorArgb(JSON.parse(raw)); } catch { return "FF000000"; }
}
const hex = raw.replace(/^#/, "").toUpperCase();
if (/^[0-9A-F]{8}$/.test(hex)) return hex;
return `FF${/^[0-9A-F]{6}$/.test(hex) ? hex : "000000"}`;
}
function comparison(left: unknown, right: unknown): number {
if (typeof left === "number" && typeof right === "number") return left - right;
return String(left ?? "").localeCompare(String(right ?? ""), "zh-CN", { numeric: true, sensitivity: "base" });
}
function registerMany(ids: string[], handler: CapabilityHandler): void {
register(ids, handler);
}
const writeRow: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = text(args, ["cellPos"]);
const row = start ? Number(sheet.getCell(start).row) : numberValue(value(args, ["row", "rowIndex"]), 1);
const column = start ? Number(sheet.getCell(start).col) : columnNumber(value(args, ["column"]));
listValue(value(args, ["contents", "value"])).forEach((item, index) => { sheet.getCell(row, column + index).value = item as ExcelJS.CellValue; });
await saveWhenRequested(source, args);
return `${columnLetter(column)}${row}`;
};
const writeColumn: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = text(args, ["cellPos"]);
const row = start ? Number(sheet.getCell(start).row) : numberValue(value(args, ["row"]), 1);
const column = start ? Number(sheet.getCell(start).col) : columnNumber(value(args, ["column"]));
listValue(value(args, ["contents", "value"])).forEach((item, index) => { sheet.getCell(row + index, column).value = item as ExcelJS.CellValue; });
await saveWhenRequested(source, args);
return `${columnLetter(column)}${row}`;
};
const writeRange: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const row = numberValue(value(args, ["row", "startRow"]), 1);
const column = columnNumber(value(args, ["column", "startColumn"]));
tableValue(value(args, ["contents", "value"])).forEach((items, rowOffset) => {
items.forEach((item, columnOffset) => { sheet.getCell(row + rowOffset, column + columnOffset).value = item as ExcelJS.CellValue; });
});
await saveWhenRequested(source, args);
return `${columnLetter(column)}${row}`;
};
const readRow: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = text(args, ["cellPos"]);
const row = start ? Number(sheet.getCell(start).row) : numberValue(value(args, ["row", "rowIndex"]), 1);
const firstColumn = start ? Number(sheet.getCell(start).col) : 1;
const result: unknown[] = [];
for (let column = firstColumn; column <= Math.max(firstColumn, sheet.columnCount); column += 1) result.push(cellValue(sheet.getCell(row, column).value));
return result;
};
const readColumn: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = text(args, ["cellPos"]);
const row = start ? Number(sheet.getCell(start).row) : 1;
const column = start ? Number(sheet.getCell(start).col) : columnNumber(value(args, ["column"]));
const result: unknown[] = [];
for (let current = row; current <= Math.max(row, sheet.rowCount); current += 1) result.push(cellValue(sheet.getCell(current, column).value));
return result;
};
const readRange: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const bounds = rangeBounds(sheet, explicitRange(args, sheet));
const result: unknown[][] = [];
for (let row = bounds.top; row <= bounds.bottom; row += 1) {
const values: unknown[] = [];
for (let column = bounds.left; column <= bounds.right; column += 1) values.push(cellValue(sheet.getCell(row, column).value));
result.push(values);
}
return result;
};
const readColumnCount: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const row = numberValue(value(args, ["row"]), 1);
let last = 0;
for (let column = 1; column <= sheet.columnCount; column += 1) {
if (cellValue(sheet.getCell(row, column).value) !== "") last = column;
}
return last;
};
const findOrReplace: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const needle = text(args, ["what", "input"]);
const replacement = text(args, ["replacement", "replace"]);
const mode = text(args, ["mode"], "0");
const replaceMode = mode === "1" || mode.includes("替换") || replacement !== "";
const exact = truthy(value(args, ["wholeWord"])) || text(args, ["lookAt"]) === "1";
const matchCase = truthy(value(args, ["matchCase"]));
const range = text(args, ["range"]);
const cells = cellsInRange(sheet, range || `A1:${columnLetter(Math.max(1, sheet.columnCount))}${Math.max(1, sheet.rowCount)}`);
const matches: string[] = [];
for (const cell of cells) {
const original = String(cellValue(cell.value));
const haystack = matchCase ? original : original.toLowerCase();
const target = matchCase ? needle : needle.toLowerCase();
const matched = exact ? haystack === target : haystack.includes(target);
if (!matched) continue;
matches.push(cell.address);
if (replaceMode) {
const escaped = needle.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
cell.value = exact ? replacement : original.replace(new RegExp(escaped, matchCase ? "g" : "gi"), replacement);
}
if (truthy(value(args, ["matchFirst"]))) break;
}
await saveWhenRequested(source, args);
return matches;
};
const sortRange: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = text(args, ["sortRange", "range"]) || `A1:${columnLetter(Math.max(1, sheet.columnCount))}${Math.max(1, sheet.rowCount)}`;
const bounds = rangeBounds(sheet, range);
const hasHeader = truthy(value(args, ["header"]));
const dataStart = bounds.top + (hasHeader ? 1 : 0);
const rows: unknown[][] = [];
for (let row = dataStart; row <= bounds.bottom; row += 1) {
const values: unknown[] = [];
for (let column = bounds.left; column <= bounds.right; column += 1) values.push(cellValue(sheet.getCell(row, column).value));
rows.push(values);
}
let sortOffset = 0;
const rule = value(args, ["rule"]);
if (rule) {
try {
const parsed = typeof rule === "string" ? JSON.parse(rule) as Record<string, unknown> : rule as Record<string, unknown>;
sortOffset = Math.max(0, columnNumber(value(parsed, ["column", "key"], bounds.left)) - bounds.left);
} catch { /* use first column */ }
}
const descending = ["2", "-1", "desc", "descending", "降序"].includes(text(args, ["sortOrder"]).toLowerCase());
rows.sort((left, right) => comparison(left[sortOffset], right[sortOffset]) * (descending ? -1 : 1));
rows.forEach((values, rowOffset) => values.forEach((item, columnOffset) => {
sheet.getCell(dataStart + rowOffset, bounds.left + columnOffset).value = item as ExcelJS.CellValue;
}));
await saveWhenRequested(source, args);
return rows;
};
function criterionMatches(candidate: unknown, criterion: unknown, condition: string): boolean {
const left = String(candidate ?? "");
const right = String(criterion ?? "");
if (["contains", "包含", "1"].includes(condition)) return left.includes(right);
if (["notContains", "不包含", "2"].includes(condition)) return !left.includes(right);
if (["greater", ">", "3"].includes(condition)) return Number(candidate) > Number(criterion);
if (["less", "<", "4"].includes(condition)) return Number(candidate) < Number(criterion);
if (["notEqual", "!=", "5"].includes(condition)) return left !== right;
return left === right;
}
const filterRows: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
let column = columnNumber(value(args, ["column"], "A"));
let criterion = value(args, ["criteria"], "");
let condition = text(args, ["condition"], "equal");
const rule = value(args, ["rule"]);
if (rule) {
try {
const parsed = typeof rule === "string" ? JSON.parse(rule) as Record<string, unknown> : rule as Record<string, unknown>;
column = columnNumber(value(parsed, ["column"], column));
criterion = value(parsed, ["criteria", "value"], criterion);
condition = text(parsed, ["condition", "operator"], condition);
} catch { /* use direct fields */ }
}
const startRow = numberValue(value(args, ["startRow"]), 1);
const endRowRaw = numberValue(value(args, ["endRow"]), sheet.rowCount);
const endRow = endRowRaw < 0 ? sheet.rowCount : endRowRaw;
const headerRow = truthy(value(args, ["onHeader"])) ? startRow : 0;
for (let row = startRow; row <= endRow; row += 1) {
if (row === headerRow) continue;
sheet.getRow(row).hidden = !criterionMatches(cellValue(sheet.getCell(row, column).value), criterion, condition);
}
sheet.autoFilter = { from: { row: startRow, column: 1 }, to: { row: endRow, column: Math.max(1, sheet.columnCount) } };
await saveWhenRequested(source, args);
return { column: columnLetter(column), criterion, condition };
};
const clearFilter: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
for (let row = 1; row <= sheet.rowCount; row += 1) sheet.getRow(row).hidden = false;
sheet.autoFilter = undefined;
await saveWhenRequested(source, args);
return true;
};
const deleteDuplicates: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const columns = listValue(value(args, ["columns"], "A")).map(columnNumber);
const hasHeader = truthy(value(args, ["header"]));
const seen = new Set<string>();
const duplicates: number[] = [];
for (let row = hasHeader ? 2 : 1; row <= sheet.rowCount; row += 1) {
const key = JSON.stringify(columns.map((column) => cellValue(sheet.getCell(row, column).value)));
if (seen.has(key)) duplicates.push(row);
else seen.add(key);
}
duplicates.reverse().forEach((row) => sheet.spliceRows(row, 1));
await saveWhenRequested(source, args);
return duplicates.length;
};
function copyCells(sheet: ExcelJS.Worksheet, range: string): CopiedRange {
const bounds = rangeBounds(sheet, range);
const values: unknown[][] = [];
const styles: Partial<ExcelJS.Style>[][] = [];
for (let row = bounds.top; row <= bounds.bottom; row += 1) {
const valueRow: unknown[] = [];
const styleRow: Partial<ExcelJS.Style>[] = [];
for (let column = bounds.left; column <= bounds.right; column += 1) {
const cell = sheet.getCell(row, column);
valueRow.push(cell.value);
styleRow.push(cloneStyle(cell.style));
}
values.push(valueRow);
styles.push(styleRow);
}
return { values, styles };
}
const copyRange: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = explicitRange(args, sheet);
copiedRange = copyCells(sheet, range);
const destinationColumn = value(args, ["startColumn2"]);
const destinationRow = value(args, ["startRow2"]);
if (destinationColumn !== "" && destinationRow !== "") {
const startColumn = columnNumber(destinationColumn);
const startRow = numberValue(destinationRow, 1);
copiedRange.values.forEach((items, rowOffset) => items.forEach((item, columnOffset) => {
const cell = sheet.getCell(startRow + rowOffset, startColumn + columnOffset);
cell.value = item as ExcelJS.CellValue;
cell.style = cloneStyle(copiedRange!.styles[rowOffset][columnOffset]);
}));
await saveWhenRequested(source, args);
}
return copiedRange.values;
};
const pasteRange: CapabilityHandler = async (_ctx, node, args) => {
if (!copiedRange) throw new RuntimeError("validation_error", "No copied Excel range is available", { nodeId: node.id, componentId: node.componentId });
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = text(args, ["Start"]);
const row = start ? Number(sheet.getCell(start).row) : numberValue(value(args, ["row"]), 1);
const column = start ? Number(sheet.getCell(start).col) : columnNumber(value(args, ["column"]));
const transpose = truthy(value(args, ["transpose"]));
const values = transpose ? copiedRange.values[0].map((_, index) => copiedRange!.values.map((items) => items[index])) : copiedRange.values;
values.forEach((items, rowOffset) => items.forEach((item, columnOffset) => {
const cell = sheet.getCell(row + rowOffset, column + columnOffset);
cell.value = item as ExcelJS.CellValue;
if (!transpose) cell.style = cloneStyle(copiedRange!.styles[rowOffset][columnOffset]);
}));
await saveWhenRequested(source, args);
return `${columnLetter(column)}${row}`;
};
const createSheet: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const name = text(args, ["sheet", "sheetName", "name"], `Sheet${source.workbook.worksheets.length + 1}`);
if (source.workbook.getWorksheet(name)) throw new Error(`Sheet already exists: ${name}`);
source.workbook.addWorksheet(name);
await saveWhenRequested(source, args);
return name;
};
const deleteSheet: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
if (source.workbook.worksheets.length <= 1) throw new Error("Cannot delete the only worksheet");
source.workbook.removeWorksheet(sheet.id);
await saveWhenRequested(source, args);
return sheet.name;
};
const activateSheet: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const mode = text(args, ["mode"]);
const index = mode.includes("序号") ? numberValue(value(args, ["index"]), 1) : 0;
const sheet = index ? source.workbook.worksheets[index - 1] : worksheetOf(source.workbook, args);
if (!sheet) throw new Error(`Sheet not found: ${index}`);
source.workbook.views = [{
activeTab: source.workbook.worksheets.indexOf(sheet),
firstSheet: 0,
visibility: "visible",
x: 0,
y: 0,
width: 12000,
height: 8000
}];
await saveWhenRequested(source, args);
return sheet.name;
};
const getSheetNames: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
return source.workbook.worksheets.map((sheet) => sheet.name);
};
function setFontColor(cell: ExcelJS.Cell, argb: string): void {
cell.font = { ...(cell.font || {}), color: { argb } };
}
function setBackground(cell: ExcelJS.Cell, argb: string): void {
cell.fill = { type: "pattern", pattern: "solid", fgColor: { argb } };
}
const setColor = (kind: "font" | "background", rangeMode: boolean): CapabilityHandler => async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = rangeMode
? explicitRange(args, sheet)
: text(args, ["cellPos"]) || `${text(args, ["column"], "A")}${numberValue(value(args, ["row"]), 1)}`;
const argb = colorArgb(value(args, ["color", "cellColor"]));
for (const cell of cellsInRange(sheet, range)) {
if (kind === "font") setFontColor(cell, argb);
else setBackground(cell, argb);
}
await saveWhenRequested(source, args);
return argb;
};
const readBackground: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const cell = sheet.getCell(numberValue(value(args, ["row"]), 1), columnNumber(value(args, ["column"])));
const fill = cell.fill as ExcelJS.FillPattern | undefined;
return fill?.type === "pattern" ? fill.fgColor?.argb ?? "" : "";
};
const autoFill: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const sourceRange = text(args, ["sourceRange", "excelRng"]);
const destination = text(args, ["destination"]);
const sourceBounds = rangeBounds(sheet, sourceRange);
const destinationBounds = rangeBounds(sheet, destination);
const sourceValues = copyCells(sheet, sourceRange).values;
const vertical = sourceBounds.left === sourceBounds.right;
for (let row = destinationBounds.top; row <= destinationBounds.bottom; row += 1) {
for (let column = destinationBounds.left; column <= destinationBounds.right; column += 1) {
const sourceRow = (row - destinationBounds.top) % sourceValues.length;
const sourceColumn = (column - destinationBounds.left) % sourceValues[0].length;
let next = sourceValues[sourceRow][sourceColumn];
const sequence = vertical ? sourceValues.map((items) => items[0]) : sourceValues[0];
const offset = vertical ? row - sourceBounds.top : column - sourceBounds.left;
if (sequence.length >= 2 && sequence.every((item) => typeof item === "number")) {
next = Number(sequence[0]) + (Number(sequence[1]) - Number(sequence[0])) * offset;
}
sheet.getCell(row, column).value = next as ExcelJS.CellValue;
}
}
await saveWhenRequested(source, args);
return destination;
};
const splitColumn: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const sourceColumn = columnNumber(value(args, ["column"]));
const destination = text(args, ["destination"]);
const destinationColumn = destination ? Number(sheet.getCell(destination).col) : sourceColumn;
const startRow = numberValue(value(args, ["startRow"]), 1);
const endRow = numberValue(value(args, ["endRow"]), sheet.rowCount);
const delimiters: string[] = [];
if (truthy(value(args, ["tab"]), true)) delimiters.push("\\t");
if (truthy(value(args, ["semicolon"]))) delimiters.push(";");
if (truthy(value(args, ["comma"]))) delimiters.push(",");
if (truthy(value(args, ["space"]))) delimiters.push("\\s+");
if (truthy(value(args, ["other"]))) delimiters.push(text(args, ["otherChar"]).replace(/[.*+?^${}()|[\]\\]/g, "\\$&"));
const separator = new RegExp(delimiters.filter(Boolean).join("|") || "\\t");
for (let row = startRow; row <= endRow; row += 1) {
String(cellValue(sheet.getCell(row, sourceColumn).value)).split(separator).forEach((item, offset) => {
sheet.getCell(row, destinationColumn + offset).value = item;
});
}
await saveWhenRequested(source, args);
return `${columnLetter(destinationColumn)}${startRow}`;
};
const closeWorkbook: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
if (truthy(value(args, ["save", "isSave"]), true)) await source.workbook.xlsx.writeFile(source.filePath);
releaseWorkbook(source.filePath);
return source.filePath;
};
const deleteRange: CapabilityHandler = async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = explicitRange(args, sheet);
for (const cell of cellsInRange(sheet, range)) cell.value = null;
await saveWhenRequested(source, args);
return range;
};
export const EXCEL_COMMON_IDS = [
"Excel_WriteColumn", "Excel_WriteColumnNew", "Excel_Cs_WriteColumn",
"Excel_WriteRow", "Excel_WriteRowNew", "Excel_Cs_WriteRow", "Excel_Cs_WriteRowV2",
"Excel_Cs_WriteRangeV2", "Excel_GetRowData", "Excel_GetRowDataNew", "Excel_Cs_ReadRow",
"Excel_GetColData", "Excel_Cs_ReadColumnCount", "Excel_Cs_ReadRange",
"Excel_Cs_Find", "Excel_Cs_FindV2", "Excel_Cs_Search",
"Excel_Cs_Sort", "Excel_Cs_SortV1", "Excel_Cs_Filter", "Excel_Cs_ExcelFilter",
"Excel_Cs_ClearExcelFilter", "Excel_Cs_EnableExcelFilter", "Excel_Cs_DeleteDuplicates",
"Excel_CopyRange", "Excel_Cs_CopyRange", "Excel_Cs_CopyRangeV2", "Excel_Cs_CopyRangeV3",
"Excel_PasteRange", "Excel_Cs_PastRange", "Excel_Cs_PastRangeV2",
"Excel_CreateSheet", "Excel_DeleteSheet", "Excel_Cs_Delete", "Excel_ActiveSheet", "Excel_Cs_Active", "Excel_Cs_GetName",
"Excel_SetCellFontColor", "Excel_Cs_SetCellFont", "Excel_SetRangeFontColor", "Excel_Cs_SetRangeFont", "Excel_Cs_SetRangeFontV2",
"Excel_SetCellColor", "Excel_Cs_SetCellBackground", "Excel_SetRangeColor", "Excel_Cs_SetRangeBackground", "Excel_Cs_SetRangeBackgroundV2",
"Excel_Cs_ReadCellBackground", "Excel_Cs_AutoFill", "Excel_Cs_AutoFillV2",
"Excel_Cs_ColumnSplit", "Excel_Cs_ColumnSplitV2", "Excel_Close", "Excel_Cs_Close",
"Excel_DeleteRange", "Excel_Cs_DeleteRange", "Excel_Cs_DeleteRangeV2", "Excel_Cs_DeleteRangeV3"
] as const;
registerMany(["Excel_WriteRow", "Excel_WriteRowNew", "Excel_Cs_WriteRow", "Excel_Cs_WriteRowV2"], writeRow);
registerMany(["Excel_WriteColumn", "Excel_WriteColumnNew", "Excel_Cs_WriteColumn"], writeColumn);
register("Excel_Cs_WriteRangeV2", writeRange);
registerMany(["Excel_GetRowData", "Excel_GetRowDataNew", "Excel_Cs_ReadRow"], readRow);
register("Excel_GetColData", readColumn);
register("Excel_Cs_ReadColumnCount", readColumnCount);
register("Excel_Cs_ReadRange", readRange);
registerMany(["Excel_Cs_Find", "Excel_Cs_FindV2", "Excel_Cs_Search"], findOrReplace);
registerMany(["Excel_Cs_Sort", "Excel_Cs_SortV1"], sortRange);
registerMany(["Excel_Cs_Filter", "Excel_Cs_ExcelFilter"], filterRows);
registerMany(["Excel_Cs_ClearExcelFilter", "Excel_Cs_EnableExcelFilter"], clearFilter);
register("Excel_Cs_DeleteDuplicates", deleteDuplicates);
registerMany(["Excel_CopyRange", "Excel_Cs_CopyRange", "Excel_Cs_CopyRangeV2", "Excel_Cs_CopyRangeV3"], copyRange);
registerMany(["Excel_PasteRange", "Excel_Cs_PastRange", "Excel_Cs_PastRangeV2"], pasteRange);
register("Excel_CreateSheet", createSheet);
registerMany(["Excel_DeleteSheet", "Excel_Cs_Delete"], deleteSheet);
registerMany(["Excel_ActiveSheet", "Excel_Cs_Active"], activateSheet);
register("Excel_Cs_GetName", getSheetNames);
registerMany(["Excel_SetCellFontColor", "Excel_Cs_SetCellFont"], setColor("font", false));
registerMany(["Excel_SetRangeFontColor", "Excel_Cs_SetRangeFont", "Excel_Cs_SetRangeFontV2"], setColor("font", true));
registerMany(["Excel_SetCellColor", "Excel_Cs_SetCellBackground"], setColor("background", false));
registerMany(["Excel_SetRangeColor", "Excel_Cs_SetRangeBackground", "Excel_Cs_SetRangeBackgroundV2"], setColor("background", true));
register("Excel_Cs_ReadCellBackground", readBackground);
registerMany(["Excel_Cs_AutoFill", "Excel_Cs_AutoFillV2"], autoFill);
registerMany(["Excel_Cs_ColumnSplit", "Excel_Cs_ColumnSplitV2"], splitColumn);
registerMany(["Excel_Close", "Excel_Cs_Close"], closeWorkbook);
registerMany(["Excel_DeleteRange", "Excel_Cs_DeleteRange", "Excel_Cs_DeleteRangeV2", "Excel_Cs_DeleteRangeV3"], deleteRange);
// ===== Excel 高阶(批四) =====
function cellAddress(addr: string): { row: number; column: number } {
const m = addr.toUpperCase().match(/^([A-Z]+)(\d+)$/);
if (!m) throw new Error("Invalid cell address: " + addr);
return { row: Number(m[2]), column: columnNumber(m[1]) };
}
function toArgb(color: string): string {
const hex = color.replace(/^#/, "").toUpperCase();
return hex.length === 6 ? "FF" + hex : hex.padStart(8, "F");
}
function parseFormatSpec(raw: unknown): { bold?: boolean; italic?: boolean; underline?: boolean; fontSize?: number; color?: string; bgColor?: string; numFmt?: string; fontName?: string } {
if (typeof raw === "object" && raw !== null) return normalizeFormatSpec(raw as Record<string, unknown>);
const text = String(raw ?? "").trim();
if (!text) return {};
try {
const parsed = JSON.parse(text) as Record<string, unknown>;
if (typeof parsed === "object" && parsed) return normalizeFormatSpec(parsed);
} catch { /* fall through to keywords */ }
const spec: ReturnType<typeof parseFormatSpec> = {};
const lower = text.toLowerCase();
if (/加粗|bold/.test(lower)) spec.bold = true;
if (/斜体|italic/.test(lower)) spec.italic = true;
if (/下划线|underline/.test(lower)) spec.underline = true;
const sizeMatch = text.match(/(\d+)号|font-size[::]\s*(\d+)/i);
if (sizeMatch) spec.fontSize = Number(sizeMatch[1] || sizeMatch[2]);
// 无关键词命中时视为数字格式代码(如 "0.00"、"G/通用格式"、"yyyy-mm-dd")
if (!Object.keys(spec).length) spec.numFmt = text;
return spec;
}
/** 兼容厂商格式 JSON(FontColor/BoldAndItalic/FontSize/BackgroundColor/DigitalFormat)与内部 camelCase 键 */
function normalizeFormatSpec(parsed: Record<string, unknown>): ReturnType<typeof parseFormatSpec> {
const spec: ReturnType<typeof parseFormatSpec> = {};
const flags = Number(parsed.BoldAndItalic ?? 0);
if (flags & 1) spec.bold = true;
if (flags & 2) spec.italic = true;
if (parsed.FontFormat) spec.fontName = String(parsed.FontFormat);
if (parsed.FontSize !== undefined && parsed.FontSize !== null && String(parsed.FontSize).trim() !== "") spec.fontSize = Number(parsed.FontSize);
if (parsed.FontColor) spec.color = String(parsed.FontColor);
if (parsed.BackgroundColor) spec.bgColor = String(parsed.BackgroundColor).split(",")[0].trim();
const digital = parsed.DigitalFormat as { formatstring?: unknown } | undefined;
if (digital?.formatstring && String(digital.formatstring).trim() !== "") spec.numFmt = String(digital.formatstring);
for (const key of ["bold", "italic", "underline", "fontSize", "color", "bgColor", "numFmt"] as const) {
if (parsed[key] !== undefined) (spec as Record<string, unknown>)[key] = parsed[key];
}
return spec;
}
registerMany(["Excel_Cs_SetCellFormat", "Excel_Cs_SetCellFormat_V2", "Excel_Cs_SetCellFormat_V3"], async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = text(args, ["excelRng", "range", "cellPos", "rng"], "A1");
const spec = parseFormatSpec(args.format);
const addresses: string[] = [];
const parts = range.toUpperCase().split(":");
if (parts.length === 2 && parts[0] !== parts[1]) {
const from = cellAddress(parts[0]);
const to = cellAddress(parts[1]);
for (let r = from.row; r <= to.row; r += 1) {
for (let cIdx = from.column; cIdx <= to.column; cIdx += 1) {
addresses.push(columnLetter(cIdx) + r);
}
}
} else {
addresses.push(parts[0]);
}
for (const addr of addresses) {
const cell = sheet.getCell(addr);
if (spec.bold !== undefined || spec.italic !== undefined || spec.underline !== undefined || spec.fontSize !== undefined || spec.color) {
cell.font = {
...cell.font,
bold: spec.bold,
italic: spec.italic,
underline: spec.underline,
size: spec.fontSize,
name: spec.fontName ?? cell.font?.name,
color: spec.color ? { argb: toArgb(spec.color) } : undefined
};
}
if (spec.numFmt) cell.numFmt = spec.numFmt;
if (spec.bgColor) cell.fill = { type: "pattern", pattern: "solid", fgColor: { argb: toArgb(spec.bgColor) } };
}
await saveWhenRequested(source, args);
ctx.log("已设置 " + addresses.length + " 个单元格格式", "info", node.id);
return { formatted: addresses.length, range };
});
register("Excel_Cs_ExcelFilter", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const rule = text(args, ["rule"]);
const startRow = numberValue(args.startRow, 1);
const endRow = numberValue(args.endRow, sheet.rowCount || 1);
const range = rule && rule.includes(":") ? rule.toUpperCase() : "A" + startRow + ":" + columnLetter(sheet.columnCount || 1) + endRow;
sheet.autoFilter = range;
await saveWhenRequested(source, args);
ctx.log("已设置自动筛选:" + range, "info", node.id);
return { filter: range };
});
register("Excel_Cs_ShowTableStyle", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
// 必须逐单元格设置:exceljs 的行级样式(row.font/row.fill + commit)会覆盖
// 该行已设置的单元格样式(如先执行过的设置单元格格式)
const lastCol = Math.max(1, sheet.columnCount);
for (let c = 1; c <= lastCol; c += 1) {
const cell = sheet.getRow(1).getCell(c);
cell.font = { ...cell.font, bold: true };
cell.fill = { type: "pattern", pattern: "solid", fgColor: { argb: "FFD9E8F5" } };
}
sheet.autoFilter = { from: { row: 1, column: 1 }, to: { row: Math.max(1, sheet.rowCount), column: lastCol } };
await saveWhenRequested(source, args, true);
ctx.log("已应用表格样式(表头加粗 + 筛选)", "info", node.id);
return { styled: true };
});
register("Excel_Cs_Hide", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const mode = String(value(args, ["mode"], "row")).toLowerCase();
const range = text(args, ["range"]);
const hide = !/取消|取消隐藏|unhide|show/i.test(mode + range);
if (mode.includes("col") || mode.includes("列")) {
const targets = range ? range.split(",") : [text(args, ["column"])];
for (const t of targets) {
if (!t.trim()) continue;
const letter = t.trim().replace(/^\d+:?/, "").toUpperCase() || t.trim().toUpperCase();
sheet.getColumn(columnNumber(letter)).hidden = hide;
}
} else {
const targets = range ? range.split(":") : [text(args, ["row"])];
if (targets.length === 2) {
const from = Number(targets[0]); const to = Number(targets[1]);
for (let r = Math.min(from, to); r <= Math.max(from, to); r += 1) sheet.getRow(r).hidden = hide;
} else {
const r = Number(targets[0]);
if (Number.isFinite(r) && r > 0) sheet.getRow(r).hidden = hide;
}
}
await saveWhenRequested(source, args);
ctx.log((hide ? "已隐藏" : "已取消隐藏") + "(" + mode + " " + range + ")", "info", node.id);
return { hidden: hide };
});
register("Excel_Cs_GetLastEmptyCell", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const column = columnLetter(numberValue(args.column, 1));
let lastFilled = 0;
for (let r = 1; r <= sheet.rowCount; r += 1) {
const v = sheet.getRow(r).getCell(numberValue(args.column, 1)).value;
if (v !== null && v !== undefined && v !== "") lastFilled = r;
}
const emptyRow = lastFilled + 1;
return { address: column + emptyRow, row: emptyRow, column: numberValue(args.column, 1) };
});
register("Excel_GetColumnId", async (_ctx, _node, args) => columnLetter(numberValue(args.num, 1)));
register(["Excel_Cs_SelectRange", "Excel_Cs_SelectRangeV2", "Excel_Cs_SelectRangeV3", "Excel_SelectRange"], async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const cellPos = text(args, ["cellPos", "range", "Start"], "A1");
const parts = cellPos.toUpperCase().split(":");
const from = cellAddress(parts[0]);
const to = parts[1] ? cellAddress(parts[1]) : from;
const rows: unknown[][] = [];
for (let r = from.row; r <= to.row; r += 1) {
const row: unknown[] = [];
for (let cIdx = from.column; cIdx <= to.column; cIdx += 1) {
row.push(sheet.getRow(r).getCell(cIdx).value ?? "");
}
rows.push(row);
}
ctx.log("已读取区域 " + cellPos + "(" + rows.length + " 行)", "info", node.id);
return rows;
});
register("Excel_Cs_CloseV2", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const save = !/不保存|discard|false/i.test(text(args, ["closeAction"]));
if (save) await source.workbook.xlsx.writeFile(source.filePath);
releaseWorkbook(source.filePath);
ctx.log("已关闭工作簿" + (save ? "(已保存)" : "(未保存)"), "info", node.id);
return { closed: true, saved: save };
});
register(["Excel_Cs_Load", "Excel_Cs_LoadV2"], async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheets = source.workbook.worksheets.map(w => w.name);
ctx.log("已加载工作簿:" + source.filePath + "(" + sheets.length + " 个 Sheet)", "info", node.id);
return { filePath: source.filePath, sheets };
});
register("Excel_WriteColumn", async (ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const start = cellAddress(text(args, ["cellPos"], "A1").toUpperCase());
const valuesRaw = value(args, ["value"]);
const values = Array.isArray(valuesRaw) ? valuesRaw : [valuesRaw];
const flat = Array.isArray(values) ? values.flat() : [values];
flat.forEach((item, offset) => {
sheet.getRow(start.row + offset).getCell(start.column).value = item as ExcelJS.CellValue;
});
await saveWhenRequested(source, args, true);
ctx.log("已写入 " + flat.length + " 行(" + columnLetter(start.column) + start.row + " 起)", "info", node.id);
return { written: flat.length };
});
@@ -0,0 +1,68 @@
import { mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import ExcelJS from "exceljs";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./office";
import "./extended";
import "./office-file-extended";
import "./office-aliases";
import "./excel-common";
import "./common-final";
const dirs: string[] = [];
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown>) { const handler = getHandler(id); expect(handler, id).toBeTruthy(); return handler!(context, node(id), args, component); }
afterEach(async () => Promise.all(dirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true }))));
describe("excel shared workbook store", () => {
it("keeps widths/heights set by office-file-extended when excel-common saves afterwards", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-store-")); dirs.push(directory);
const handle = await run("Excel_NewExcel", { filePath: directory, name: "a.xlsx", isMakedirs: "True" }) as { path: string };
await run("Excel_Cs_WriteRangeV2", { workbook: handle, sheet: "Sheet1", row: "1", column: "A", contents: [["名称", "数量", "备注"], ["鼠标", 20, "A;B"]], isSave: "True" });
await run("Excel_Cs_SetWidth", { workbook: handle, sheet: "Sheet1", columns: "A:C", width: "20", save: "True" });
await run("Excel_Cs_SetHeight", { workbook: handle, sheet: "Sheet1", rows: "1", height: "26", save: "True" });
await run("Excel_Cs_MergeCells", { workbook: handle, sheet: "Sheet1", range: "A1:C1", merge: "0", save: "True" });
await run("Excel_Cs_SetRangeBackground", { workbook: handle, sheet: "Sheet1", range: "A1:C1", color: "FFF2CC", save: "True" });
await run("Excel_SetCellFontColor", { objExcel: handle, sheet: "Sheet1", cellPos: "A1", cellColor: "FF0000", isSave: "True" });
const wb = new ExcelJS.Workbook();
await wb.xlsx.readFile(handle.path);
const ws = wb.worksheets[0];
expect(ws.getColumn("A").width).toBe(20);
expect(ws.getRow(1).height).toBe(26);
expect(ws.model.merges).toContain("A1:C1");
expect((ws.getCell("A1").fill as ExcelJS.FillPattern).fgColor?.argb).toBe("FFFFF2CC");
expect((ws.getCell("A1").font?.color as { argb?: string } | undefined)?.argb).toBe("FFFF0000");
expect(ws.getCell("A1").value).toBe("名称");
});
it("keeps copied sheets created by office-file-extended when sheets are created and deleted later", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-store-")); dirs.push(directory);
const handle = await run("Excel_NewExcel", { filePath: directory, name: "b.xlsx", isMakedirs: "True" }) as { path: string };
await run("Excel_Cs_WriteRangeV2", { workbook: handle, sheet: "Sheet1", row: "1", column: "A", contents: [["名称"]], isSave: "True" });
await run("Excel_Cs_CopyV2", { workbook: handle, sheet: "Sheet1", newSheet: "Sheet1副本", override: "True", save: "True" });
await run("Excel_CreateSheet", { workbook: handle, sheet: "临时表", save: "True" });
await run("Excel_DeleteSheet", { workbook: handle, sheet: "临时表", save: "True" });
const wb = new ExcelJS.Workbook();
await wb.xlsx.readFile(handle.path);
expect(wb.worksheets.map((sheet) => sheet.name)).toEqual(["Sheet1", "Sheet1副本"]);
});
it("keeps the shared instance usable after save-as switches the handle path", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-store-")); dirs.push(directory);
const handle = await run("Excel_NewExcel", { filePath: directory, name: "c.xlsx", isMakedirs: "True" }) as { path: string };
await run("Excel_Cs_SetWidth", { workbook: handle, sheet: "Sheet1", columns: "A", width: "18", save: "True" });
const saved = await run("Excel_Cs_SaveAs", { workbook: handle, path: path.join(directory, "d.xlsx") }) as { path: string };
expect(saved.path).toBe(path.resolve(directory, "d.xlsx"));
await run("Excel_Cs_SetHeight", { workbook: handle, sheet: "Sheet1", rows: "1", height: "30", save: "True" });
const wb = new ExcelJS.Workbook();
await wb.xlsx.readFile(path.resolve(directory, "d.xlsx"));
expect(wb.worksheets[0].getColumn("A").width).toBe(18);
expect(wb.worksheets[0].getRow(1).height).toBe(30);
});
});
@@ -0,0 +1,35 @@
import ExcelJS from "exceljs";
import path from "node:path";
import { defaultEngine } from "../engine/engine";
// Excel 工作簿共享缓存:同一文件路径在整条流程中复用同一个 ExcelJS 实例。
// 此前 office.ts / office-file-extended.ts / excel-common.ts / extended.ts 各自持有
// 工作簿实例并整簿写回磁盘,后保存者会把先保存者的修改整体覆盖掉。
// 缓存本体挂在引擎实例上(任务 1.3),此处委托 defaultEngine。
function keyOf(filePath: string): string {
return path.resolve(filePath);
}
/** 按路径取工作簿:命中缓存直接复用,未命中从磁盘读取后登记。 */
export async function acquireWorkbook(filePath: string): Promise<ExcelJS.Workbook> {
const key = keyOf(filePath);
let workbook = defaultEngine.excelCache.get(key);
if (!workbook) {
workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(key);
defaultEngine.excelCache.set(key, workbook);
}
return workbook;
}
/** 把一个新建/另建的工作簿实例登记为该路径的共享实例。 */
export function publishWorkbook(filePath: string, workbook: ExcelJS.Workbook): ExcelJS.Workbook {
defaultEngine.excelCache.set(keyOf(filePath), workbook);
return workbook;
}
/** 释放路径对应的共享实例(关闭工作簿时调用)。 */
export function releaseWorkbook(filePath: string): void {
defaultEngine.excelCache.delete(keyOf(filePath));
}
@@ -3,7 +3,7 @@ import os from "node:os";
import path from "node:path";
import ExcelJS from "exceljs";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./extended";
@@ -3,6 +3,7 @@ import path from "node:path";
import ExcelJS from "exceljs";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { acquireWorkbook } from "./excel-store";
type Table = unknown[][];
@@ -243,8 +244,7 @@ function workbookPath(args: Record<string, unknown>, nodeId: string, componentId
}
async function loadSheet(args: Record<string, unknown>, nodeId: string, componentId: string): Promise<ExcelJS.Worksheet> {
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(workbookPath(args, nodeId, componentId));
const workbook = await acquireWorkbook(workbookPath(args, nodeId, componentId));
const name = text(args, ["sheet", "sheetName"]);
const sheet = name ? workbook.getWorksheet(name) : workbook.worksheets[0];
if (!sheet) throw new Error(`Sheet 不存在:${name || "第一个 Sheet"}`);
@@ -0,0 +1,67 @@
import { existsSync, readdirSync, readFileSync } from "node:fs";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./files";
function run(id: string, args: Record<string, unknown>): Promise<string> {
const node: FlowNode = { id: "file-test", componentId: id, values: args, options: {}, position: { x: 0, y: 0 } };
const handler = getHandler(id)!;
return Promise.resolve(handler({ signal: new AbortController().signal, log: vi.fn(), vars: {} } as unknown as RunContext, node, args, {} as CatalogComponent)) as Promise<string>;
}
const temporary: string[] = [];
afterEach(() => {
for (const dir of temporary.splice(0)) rm(dir, { recursive: true, force: true });
});
describe("file compression path semantics", () => {
it("defaults to a sibling archive and creates a copy instead of overwriting", async () => {
const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-compress-"));
temporary.push(dir);
const source = path.join(dir, "report.txt");
await writeFile(source, "hello", "utf8");
const first = await run("File_CompressFile", { file_type: "openFile", file_path: source, direct_type: "1" });
expect(first).toBe(path.join(dir, "report.txt.zip"));
expect(existsSync(first)).toBe(true);
const second = await run("File_CompressFile", { file_type: "openFile", file_path: source, direct_type: "1" });
expect(second).toBe(path.join(dir, "report.txt (1).zip"));
expect(existsSync(second)).toBe(true);
});
it("accepts an output directory when custom destination is selected", async () => {
const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-compress-"));
temporary.push(dir);
const source = path.join(dir, "logs");
const output = path.join(dir, "output");
await Promise.all([mkdir(source), mkdir(output)]);
await writeFile(path.join(source, "app.log"), "log", "utf8");
const target = await run("File_CompressFile", {
file_type: "openDirectory", folder_path: source, direct_type: "0", out_path: output
});
expect(target).toBe(path.join(output, "logs.zip"));
expect(existsSync(target)).toBe(true);
});
it("extracts beside the archive by default and to the selected folder for custom mode", async () => {
const dir = await mkdtemp(path.join(os.tmpdir(), "rpa-file-extract-"));
temporary.push(dir);
const source = path.join(dir, "logs.zip");
const AdmZip = (await import("adm-zip")).default;
const zip = new AdmZip();
zip.addFile("app.log", Buffer.from("log", "utf8"));
zip.writeZip(source);
const defaultTarget = await run("File_DecompressFile", { folder_path: source, direct_type: "1" });
expect(defaultTarget).toBe(dir);
expect(readFileSync(path.join(dir, "app.log"), "utf8")).toBe("log");
const custom = path.join(dir, "custom-output");
const customTarget = await run("File_DecompressFile", { folder_path: source, direct_type: "0", out_path: custom });
expect(customTarget).toBe(custom);
expect(readdirSync(custom)).toContain("app.log");
});
});
@@ -59,7 +59,10 @@ register("File_Write", async (_ctx, node, args) => {
});
register(["File_CreateFile"], async (_ctx, node, args) => {
const filePath = requirePath(pick(args, ["path", "file", "file_path"]), "文件路径", node.id, node.componentId);
// catalog 字段为 folder_path + name(组合成完整路径),兼容旧键 path
const base = pick(args, ["folder_path", "path", "file", "file_path"]);
const name = pick(args, ["name"]);
const filePath = requirePath(name ? path.join(base, name) : base, "文件路径", node.id, node.componentId);
await fs.mkdir(path.dirname(filePath), { recursive: true });
await fs.writeFile(filePath, "", { flag: "a" });
return filePath;
@@ -80,8 +83,15 @@ register("File_Copy", async (_ctx, node, args) => {
});
register(["File_File_Move", "File_Move"], async (_ctx, node, args) => {
const source = requirePath(pick(args, ["pathSrc", "src", "source"]), "源路径", node.id, node.componentId);
const target = requirePath(pick(args, ["pathDest", "dest", "target"]), "目标路径", node.id, node.componentId);
// File_Move 的 catalog 字段为 file_type + file_path/folder_path + target_path(+name),与 File_File_Move 的 pathSrc/pathDest 兼容
const isFolder = String(args.file_type ?? "file") === "folder";
const source = requirePath(pick(args, ["pathSrc", isFolder ? "folder_path" : "file_path", "src", "source"]), "源路径", node.id, node.componentId);
let target = pick(args, ["pathDest", "dest"]);
if (!target) {
const destDir = pick(args, ["target_path"]);
const newName = truthy(args.rename) ? pick(args, ["name"]) : "";
target = requirePath(newName ? path.join(destDir, newName) : destDir, "目的地路径", node.id, node.componentId);
}
await fs.mkdir(path.dirname(target), { recursive: true });
try {
await fs.rename(source, target);
@@ -152,7 +162,8 @@ register(["File_PathSize", "File_PathSizeNew"], async (_ctx, node, args) => {
});
register("File_GetFileInfo", async (_ctx, node, args) => {
const target = requirePath(pick(args, ["path", "file"]), "路径", node.id, node.componentId);
// catalog 字段为 file_path,兼容 path/file
const target = requirePath(pick(args, ["file_path", "path", "file"]), "路径", node.id, node.componentId);
const stat = await fs.stat(target);
return {
path: target,
@@ -165,7 +176,8 @@ register("File_GetFileInfo", async (_ctx, node, args) => {
});
register("File_WaitFile", async (ctx, node, args) => {
const target = requirePath(pick(args, ["path", "file"]), "文件路径", node.id, node.componentId);
// catalog 字段为 file_path,兼容 path/file
const target = requirePath(pick(args, ["file_path", "path", "file"]), "文件路径", node.id, node.componentId);
const timeout = Math.max(1, Number(args.time_out ?? args.timeout ?? 30)) * 1000;
const started = Date.now();
while (Date.now() - started < timeout) {
@@ -184,9 +196,45 @@ register("File_WaitFile", async (ctx, node, args) => {
});
});
function directoryOutputPath(outputPath: string, sourcePath: string): string {
const output = path.resolve(outputPath);
if (/[\\/]$/.test(outputPath) || path.extname(output).toLowerCase() !== ".zip") {
return path.join(output, path.basename(sourcePath) + ".zip");
}
return output;
}
async function uniqueZipPath(targetPath: string): Promise<string> {
try {
await fs.access(targetPath);
} catch {
return targetPath;
}
const dir = path.dirname(targetPath);
const ext = path.extname(targetPath);
const base = path.basename(targetPath, ext);
for (let index = 1; ; index++) {
const candidate = path.join(dir, `${base} (${index})${ext}`);
try {
await fs.access(candidate);
} catch {
return candidate;
}
}
}
register("File_CompressFile", async (_ctx, node, args) => {
const source = requirePath(pick(args, ["pathSrc", "path", "folder_path"]), "源路径", node.id, node.componentId);
const target = requirePath(pick(args, ["pathDest", "zip_path", "out_path"]), "压缩包路径", node.id, node.componentId);
// catalog 字段为 file_path(文件模式)/folder_path(文件夹模式)+ out_path,兼容旧键
const contentType = String(args.file_type ?? "openFile");
const sourceKeys = contentType === "openDirectory"
? ["folder_path", "pathSrc", "path"]
: ["file_path", "pathSrc", "path"];
const source = requirePath(pick(args, sourceKeys), contentType === "openDirectory" ? "待压缩文件夹路径" : "待压缩文件路径", node.id, node.componentId);
const saveToSourceDir = String(args.direct_type ?? "1") !== "0";
const requestedTarget = saveToSourceDir
? path.join(path.dirname(source), path.basename(source) + ".zip")
: directoryOutputPath(requirePath(pick(args, ["pathDest", "zip_path", "out_path"]), "压缩包输出位置", node.id, node.componentId), source);
const target = await uniqueZipPath(requestedTarget);
const zip = new AdmZip();
const stat = await fs.stat(source);
if (stat.isDirectory()) zip.addLocalFolder(source);
@@ -198,7 +246,10 @@ register("File_CompressFile", async (_ctx, node, args) => {
register("File_DecompressFile", async (_ctx, node, args) => {
const source = requirePath(pick(args, ["folder_path", "path", "zip_path"]), "压缩包路径", node.id, node.componentId);
const target = requirePath(pick(args, ["out_path", "pathDest", "target"]), "解压目录", node.id, node.componentId);
const saveToSourceDir = String(args.direct_type ?? "1") !== "0";
const target = saveToSourceDir
? path.dirname(source)
: requirePath(pick(args, ["out_path", "pathDest", "target"]), "解压目标目录", node.id, node.componentId);
await fs.mkdir(target, { recursive: true });
new AdmZip(source).extractAllTo(target, true);
return target;
+128
View File
@@ -0,0 +1,128 @@
import { Client } from "basic-ftp";
import path from "node:path";
import fs from "node:fs";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { defaultEngine } from "../engine/engine";
async function requireClient(nodeId: string): Promise<Client> {
if (defaultEngine.ftpClient) return defaultEngine.ftpClient;
throw new RuntimeError("validation_error", "尚未连接 FTP,请先使用 FTP_Connect", { nodeId, componentId: nodeId, recommendation: "先运行 FTP_Connect 建立连接" });
}
function requireText(args: Record<string, unknown>, keys: string[], label: string, nodeId: string): string {
for (const key of keys) {
const v = String(args[key] ?? "").trim();
if (v) return v;
}
throw new RuntimeError("validation_error", label + "为空", { nodeId, componentId: nodeId });
}
register("FTP_Connect", async (ctx, node, args) => {
const host = requireText(args, ["address"], "FTP 地址", node.id);
const port = Number(args.port ?? 21) || 21;
const user = String(args.user ?? "anonymous");
const password = String(args.pass_word ?? args.pass_word_file ?? "guest");
const secure = String(args.ftp_type ?? "").toLowerCase().includes("ftps");
const client = new Client(10_000);
await client.access({ host, port, secure, user, password });
defaultEngine.ftpClient = client;
ctx.log("已连接 FTP:" + host + ":" + port, "info", node.id);
return { connected: true, host, port };
});
register("FTP_Disconnect", async (ctx, node) => {
defaultEngine.ftpClient?.close();
defaultEngine.ftpClient = undefined;
ctx.log("已断开 FTP 连接", "info", node.id);
return { disconnected: true };
});
register("FTP_GetFileList", async (ctx, node, args) => {
const client = await requireClient(node.id);
const list = await client.list("");
const entries = list.map(item => ({ name: item.name, size: item.size, isDirectory: item.isDirectory, modifiedAt: item.modifiedAt instanceof Date ? item.modifiedAt.toISOString() : String(item.modifiedAt ?? "") }));
ctx.log("已获取文件列表:" + entries.length + " 项", "info", node.id);
return entries;
});
register("FTP_Cwd", async (ctx, node, args) => {
const client = await requireClient(node.id);
const dir = requireText(args, ["dir"], "目标目录", node.id);
await client.cd(dir);
ctx.log("已切换工作目录:" + dir, "info", node.id);
return { cwd: dir };
});
register("FTP_Download", async (ctx, node, args) => {
const client = await requireClient(node.id);
const remote = requireText(args, ["file_path"], "远程文件路径", node.id);
const localDir = String(args.folder_path ?? "").trim();
const target = requireText(args, ["target_path"], "本地保存路径", node.id) || (localDir ? path.join(localDir, path.basename(remote)) : path.basename(remote));
fs.mkdirSync(path.dirname(path.resolve(target)), { recursive: true });
if (String(args.file_type ?? "0") === "1") {
fs.mkdirSync(path.resolve(target), { recursive: true });
await client.ensureDir(remote);
await client.downloadToDir(target, remote);
} else {
await client.downloadTo(target, remote);
}
ctx.log("已下载到:" + target, "info", node.id);
return { downloaded: true, local: target };
});
register("FTP_Upload", async (ctx, node, args) => {
const client = await requireClient(node.id);
const local = requireText(args, ["file_path"], "本地文件路径", node.id);
const remote = requireText(args, ["remote_path"], "远程目标路径", node.id);
if (String(args.file_type ?? "0") === "1" || fs.statSync(local).isDirectory()) {
await client.uploadFromDir(local, remote);
} else {
await client.uploadFrom(local, remote);
}
ctx.log("已上传:" + local + " → " + remote, "info", node.id);
return { uploaded: true, remote };
});
register("FTP_Rename", async (ctx, node, args) => {
const client = await requireClient(node.id);
const from = requireText(args, ["file_path"], "原路径", node.id);
const to = requireText(args, ["toname"], "新名称", node.id);
const parent = path.posix.dirname(from);
await client.rename(from, parent === "." ? to : path.posix.join(parent, to));
ctx.log("已重命名:" + from + " → " + to, "info", node.id);
return { renamed: true };
});
register("FTP_Delete", async (ctx, node, args) => {
const client = await requireClient(node.id);
const remote = requireText(args, ["file_path"], "远程路径", node.id);
if (String(args.file_type ?? "0") === "1") {
await client.removeDir(remote);
} else {
await client.remove(remote);
}
ctx.log("已删除:" + remote, "info", node.id);
return { deleted: true };
});
register("FTP_CreateDir", async (ctx, node, args) => {
const client = await requireClient(node.id);
const dir = requireText(args, ["dirname"], "目录名", node.id);
await client.ensureDir(dir);
// 创建后进入该目录;必须 await,否则下个节点的命令与本次 cd 并发,
// basic-ftp 会以 "task while another one is still running" 关闭客户端
await client.cd(dir).catch(() => {});
ctx.log("已创建目录:" + dir, "info", node.id);
return { created: true, dir };
});
register("FTP_Exist", async (ctx, node, args) => {
const client = await requireClient(node.id);
const target = requireText(args, ["path"], "远程路径", node.id);
const parent = path.posix.dirname(target);
const name = path.posix.basename(target);
const list = await client.list(parent === "." ? "" : parent);
const exists = list.some(item => item.name === name);
return { exists, path: target };
});
File renamed without changes.
@@ -0,0 +1,105 @@
import { getHostBridge, type HostDialogWindow } from "../engine/host-bridge";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
interface FormFieldSpec { key?: string; name?: string; label?: string; type?: string; value?: string; options?: unknown[] }
function parseFields(raw: unknown, nodeId: string): FormFieldSpec[] {
if (raw === undefined || raw === null || raw === "") {
throw new RuntimeError("validation_error", "表单配置 jsonlist 为空", { nodeId, componentId: nodeId, recommendation: "填写 JSON 数组,例如 [{\"key\":\"name\",\"label\":\"名称\",\"type\":\"text\"}]" });
}
if (Array.isArray(raw)) return raw as FormFieldSpec[];
const text = String(raw).trim();
if (text.startsWith("[")) {
try {
const parsed = JSON.parse(text) as unknown;
if (Array.isArray(parsed)) return parsed as FormFieldSpec[];
} catch { /* fall through */ }
}
throw new RuntimeError("validation_error", "表单配置 jsonlist 无效", { nodeId, componentId: nodeId, recommendation: "填写 JSON 数组,例如 [{\"key\":\"name\",\"label\":\"名称\",\"type\":\"text\"}]" });
}
function fieldKey(f: FormFieldSpec, index: number): string {
return String(f.key ?? f.name ?? f.label ?? "field" + (index + 1));
}
function escapeHtml(s: string): string {
return s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;");
}
async function showForm(options: { title: string; fields: FormFieldSpec[]; timeoutSec: number }): Promise<Record<string, string>> {
const bridge = getHostBridge();
if (!bridge) {
throw new RuntimeError("host_unavailable", "宿主桥未注入,无法打开人机交互窗口", {
recommendation: "宿主(Electron 主进程)启动时需调用 setHostBridge() 注入窗口工厂"
});
}
return new Promise((resolve) => {
const rows = options.fields.map((f, i) => {
const key = escapeHtml(fieldKey(f, i));
const label = escapeHtml(f.label ?? fieldKey(f, i));
const value = escapeHtml(String(f.value ?? ""));
const type = String(f.type ?? "text").toLowerCase();
if (type === "select" || (Array.isArray(f.options) && f.options.length)) {
const opts = (f.options ?? []).map(o => {
const ov = typeof o === "string" ? o : String((o as { value?: string }).value ?? (o as { label?: string }).label ?? "");
const ol = typeof o === "string" ? o : String((o as { label?: string }).label ?? ov);
return '<option value="' + escapeHtml(ov) + '"' + (ov === String(f.value ?? "") ? " selected" : "") + ">" + escapeHtml(ol) + "</option>";
}).join("");
return "<label>" + label + '<br/><select data-key="' + key + '">' + opts + "</select></label><br/>";
}
if (type === "textarea") {
return "<label>" + label + '<br/><textarea data-key="' + key + '" rows="3">' + value + "</textarea></label><br/>";
}
if (type === "checkbox") {
const checked = String(f.value ?? "") === "true" || String(f.value ?? "") === "1" ? " checked" : "";
return '<label><input type="checkbox" data-key="' + key + '"' + checked + " /> " + label + "</label><br/>";
}
const inputType = type === "password" ? "password" : "text";
return "<label>" + label + '<br/><input type="' + inputType + '" data-key="' + key + '" value="' + value + '" /></label><br/>';
}).join("\n");
const html = '<!DOCTYPE html><html><head><meta charset="utf-8"><style>body{font-family:Microsoft YaHei,sans-serif;margin:16px;font-size:14px;color:#1f2937}label{display:block;margin-bottom:10px}input,select,textarea{width:100%;box-sizing:border-box;padding:6px;margin-top:4px}#ok{margin-top:8px;padding:8px 24px;background:#2563eb;color:#fff;border:none;border-radius:6px;cursor:pointer}</style></head><body><h3>' + escapeHtml(options.title) + "</h3>" + rows + '<button id="ok">确认</button><script>document.getElementById("ok").onclick=function(){var out={};document.querySelectorAll("[data-key]").forEach(function(el){out[el.dataset.key]=el.type==="checkbox"?el.checked:el.value});window.__formValues=out;};</script></body></html>';
const win: HostDialogWindow = bridge.createFormWindow({ title: options.title });
let settled = false;
let timer: NodeJS.Timeout | undefined;
let timeout: NodeJS.Timeout | undefined;
const finish = (value: Record<string, string>) => {
if (settled) return;
settled = true;
if (timer) clearInterval(timer);
if (timeout) clearTimeout(timeout);
if (!win.isDestroyed()) win.destroy();
resolve(value);
};
timer = setInterval(async () => {
try {
const values = await win.executeJavaScript("window.__formValues || null");
if (values) finish(values as Record<string, string>);
} catch {
finish({});
}
}, 250);
timeout = setTimeout(() => {
try { void win.executeJavaScript("document.getElementById(\"ok\").click()"); } catch {
finish({});
}
}, Math.max(5, options.timeoutSec) * 1000);
win.on("closed", () => finish({}));
void win.loadURL("data:text/html;charset=utf-8," + encodeURIComponent(html));
});
}
const DIALOG_IDS = [
"Dialog_GetConfig", "Dialog_GetConfigV1", "Dialog_GetConfigV2", "Dialog_GetConfigV3",
"Dialog_GetConfigV4", "Dialog_GetConfigV5", "Dialog_GetConfigV6", "Dialog_GetConfigV7", "Dialog_GetConfigV8"
];
register(DIALOG_IDS, async (ctx, node, args) => {
const fields = parseFields(args.jsonlist, node.id);
const title = String(args.title ?? args.configName ?? "请输入");
const timeoutSec = Number(args.wait_time ?? 30) || 30;
const values = await showForm({ title, fields, timeoutSec });
ctx.log("已获取人机交互表单输入:" + JSON.stringify(values), "info", node.id);
return values;
});
@@ -0,0 +1,42 @@
/**
* rpa-core 能力注册入口。
*
* import 本模块即完成全部包内能力的副作用注册(与改造前
* src/main/capabilities/index.ts 的加载顺序保持一致,别名/重复 id 的
* last-wins 语义不变)。依赖 Electron 窗口的浏览器能力(ai-caps / browser* /
* web-batch2 / web-crawl)留在宿主 src/main/capabilities;
* human-dialog / ai-service / window-extended 已于阶段 2 经 HostBridge 迁入。
*/
import "./control";
import "./files";
import "./http";
import "./data";
import "./office";
import "./desktop";
import "./desktop-uia";
import "./time-caps";
import "./windows-system";
import "./ftp";
import "./ocr-caps";
import "./pdf";
import "./word";
import "./word-extended";
import "./message";
import "./vision";
import "./cv-vision";
import "./extended";
import "./system-extended";
import "./office-file-extended";
import "./office-aliases";
import "./excel-common";
import "./excel-com";
import "./data-extended";
import "./utility-extended";
import "./data-table-common";
import "./basic-encryption";
import "./mail-remaining";
import "./remaining-core";
import "./common-final";
import "./human-dialog";
import "./ai-service";
import "./window-extended";
@@ -0,0 +1,95 @@
import { describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import { getHandler } from "../engine/registry";
import type { RunContext } from "../engine/types";
import "./desktop";
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id: "smoke-kb", componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
const ctx = {
runId: "kb-smoke", flowId: "kb-smoke-flow", flow: {} as RunContext["flow"],
vars: {}, secrets: {}, signal: new AbortController().signal, evidenceDir: "D:/office-test2",
emit: () => {}, log: () => {}, componentOf: () => undefined
} as unknown as RunContext;
async function invoke(componentId: string, args: Record<string, unknown>): Promise<unknown> {
const handler = getHandler(componentId);
expect(handler).toBeTypeOf("function");
return handler!(ctx, node(componentId), args, component);
}
describe("键盘输入组件真实冒烟", () => {
it("words 字段取值(此前读不到报「键盘输入为空」), 后台接口模式整段发送特殊字符不抛错", async () => {
const out = await invoke("Keyboard_Send_V1", { words: "哈哈哈(测试)100%^2", interval: "10", fuctype: "0" });
expect(out).toBe("哈哈哈(测试)100%^2");
console.log("[键盘输入] 后台接口模式整段发送 ✓:", out);
}, 60000);
it("人工模拟模式逐字符+间隔毫秒真实发送", async () => {
const out = await invoke("Keyboard_Send_V1", { words: "逐字输入ABC", interval: "5", fuctype: "1" });
expect(out).toBe("逐字输入ABC");
console.log("[键盘输入] 人工模拟逐字符 ✓:", out);
}, 60000);
it("人工模拟模式含撇号与特殊字符(用户 09:05 踩坑场景)不抛语法错", async () => {
const out = await invoke("Keyboard_Send_V1", { words: "hhhhh' +^%~(){}[]", interval: "10", fuctype: "1" });
expect(out).toBe("hhhhh' +^%~(){}[]");
console.log("[键盘输入] 人工模拟+撇号+特殊字符 ✓:", out);
}, 60000);
it("人工模拟模式含中文标点(用户 09:32 踩坑场景:】【‘;。/ 智能引号)不抛语法错", async () => {
const words = "jashfjksudfgisa54445哈哈哈哈AAAAHHBHG+t】【‘;。/";
const out = await invoke("Keyboard_Send_V1", { words, interval: "10", fuctype: "1" });
expect(out).toBe(words);
console.log("[键盘输入] 人工模拟+中文标点+智能引号 ✓");
}, 60000);
it("剪切板粘贴模式: 原文进入剪贴板并发送 Ctrl+V", async () => {
const content = `粘贴${Date.now()}`;
const out = await invoke("Keyboard_Send_V1", { words: content, fuctype: "2" });
expect(out).toBe(content);
const clip = await invoke("Clipboard_GetText", {}) as string;
expect(clip).toBe(content);
console.log("[键盘输入] 剪贴板粘贴模式 ✓ 剪贴板内容一致:", clip);
}, 60000);
it("空输入仍报「键盘输入为空」", async () => {
await expect(invoke("Keyboard_Send_V1", { words: "", interval: "10" })).rejects.toThrow("键盘输入为空");
console.log("[键盘输入] 空输入校验 ✓");
}, 60000);
it("输入目标窗口: 先激活记事本再输入, 前台窗口切到位", async () => {
const { execFile } = await import("node:child_process");
const { promisify } = await import("node:util");
const runPs = promisify(execFile);
const ps = (cmd: string) => runPs("powershell.exe", ["-NoProfile", "-NonInteractive", "-Command", cmd], { windowsHide: true, timeout: 30000 });
// 窗口标题随系统语言变化(英文 Notepad / 中文 记事本), 不硬编码, 启动后动态取实际标题
await ps("Start-Process $env:SystemRoot\\System32\\notepad.exe; $diag=''; $ok=$false; foreach($i in 1..24){ $procs=Get-Process Notepad -ErrorAction SilentlyContinue; if($procs){ $p=$procs | Where-Object {$_.MainWindowTitle}; if($p){$ok=$true; break} } else { $diag='no-process' }; Start-Sleep -Milliseconds 500 }; if(-not $ok){ throw \"notepad not visible: $diag\" }");
try {
const titlePs = "[Console]::OutputEncoding=[System.Text.UTF8Encoding]::new($false); (Get-Process Notepad | Where-Object { $_.MainWindowTitle } | Select-Object -First 1).MainWindowTitle";
const title = (await ps(titlePs)).stdout.trim();
expect(title.length).toBeGreaterThan(0);
// GUI 焦点抢占受系统前台锁约束, 偶发失败属正常抖动, 有界重试
let lastErr: unknown;
for (let attempt = 1; attempt <= 3; attempt += 1) {
try {
await invoke("Keyboard_Send_V1", { words: "目标窗口输入'测试", interval: "5", fuctype: "1", window_title: title });
lastErr = undefined;
break;
} catch (error) {
lastErr = error;
await new Promise((r) => setTimeout(r, 1000));
}
}
if (lastErr) throw lastErr;
await new Promise((r) => setTimeout(r, 300));
// activateWindowScript 内部已校验前台句柄=目标窗口, 这里再确认前台标题仍是记事本(输入后标题会带 * 脏标记前缀)
const fg = await ps(titlePs);
expect(fg.stdout.trim()).toContain(title);
console.log("[键盘输入] 目标窗口激活 ✓ 前台:", fg.stdout.trim());
} finally {
await ps("Get-Process notepad* -ErrorAction SilentlyContinue | Stop-Process -Force").catch(() => {});
}
}, 90000);
});
@@ -0,0 +1,60 @@
import { getHandler, register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { findImapMail, type NormalizedMail } from "./message";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) { const value = args[key]; if (value !== undefined && value !== null && value !== "") return String(value); }
return fallback;
}
function number(value: unknown, fallback = 1): number { const parsed = Number(value); return Number.isFinite(parsed) && parsed > 0 ? Math.floor(parsed) : fallback; }
function record(value: unknown): Record<string, unknown> { return value && typeof value === "object" ? value as Record<string, unknown> : (() => { try { return JSON.parse(String(value || "{}")); } catch { return {}; } })(); }
function list(value: unknown): string[] { if (Array.isArray(value)) return value.map(String); try { const parsed = JSON.parse(String(value || "[]")); return Array.isArray(parsed) ? parsed.map(String) : [String(parsed)]; } catch { return String(value || "").split(/[,,;;\s]+/).filter(Boolean); } }
function mergeAccount(args: Record<string, unknown>): Record<string, unknown> { return { ...record(args.account_obj || args.recvObject || args.target_obj), ...args }; }
async function imapClient(args: Record<string, unknown>) {
const { ImapFlow } = await import("imapflow"); const config = mergeAccount(args);
const client = new ImapFlow({ host: text(config, ["host", "hostname", "server_address", "server"]), port: number(config.port, 993), secure: ["true", "1", "True"].includes(text(config, ["ssl", "is_ssl", "secure"], "True")), auth: { user: text(config, ["account", "username", "user"]), pass: text(config, ["password", "pass"]) }, logger: false });
await client.connect(); return client;
}
async function withMailbox<T>(args: Record<string, unknown>, callback: (client: any) => Promise<T>): Promise<T> {
const client = await imapClient(args); const lock = await client.getMailboxLock(text(args, ["folder", "email_folder", "inbox_type", "folder_name"], "INBOX"));
try { return await callback(client); } finally { lock.release(); client.close(); }
}
async function searchMails(args: Record<string, unknown>): Promise<NormalizedMail[]> {
const normalized: Record<string, unknown> = { ...args, From_data: text(args, ["From_data", "From", "sender_key", "recipient_key"]), Subject_data: text(args, ["Subject_data", "Subject", "subject_key"]), Body_data: text(args, ["Body_data", "body_key"]) };
return await findImapMail(normalized, { id: "mail-remaining", componentId: "mail-remaining" });
}
const connect: CapabilityHandler = async (_ctx, _node, args) => { const config = mergeAccount(args); await imapClient(config).then((client) => client.close()); return { kind: "imap", ...config }; };
register(["Mail_RecvConnect", "Mail_UnionConnect", "Mail_ExConnect"], connect);
register("Mail_RecvClose", async () => ({ closed: true }));
register("Mail_RecvGetList", async (_ctx, _node, args) => await searchMails(args));
register("Mail_ExSearchMail", async (_ctx, _node, args) => await searchMails(args));
register("Mail_RecvMailGet", async (_ctx, _node, args) => {
const mails = await searchMails(args); const wanted = number(args.limit || args.index, 1); const mail = mails[wanted - 1];
if (!mail) throw new RuntimeError("not_found", "未找到第 " + wanted + " 封邮件"); return mail;
});
register("Mail_RecvCount", async (_ctx, _node, args) => await withMailbox(args, async (client) => client.mailbox.exists));
async function setSeen(args: Record<string, unknown>, seen: boolean) {
return await withMailbox(args, async (client) => { const sequence = String(number(args.index || args.limit, 1)); const flag = String.fromCharCode(92) + "Seen";
return seen ? await client.messageFlagsAdd(sequence, [flag]) : await client.messageFlagsRemove(sequence, [flag]); });
}
register("Mail_MarkSeen", async (_ctx, _node, args) => setSeen(args, true));
register("Mail_UnmarkSeen", async (_ctx, _node, args) => setSeen(args, false));
register(["Mail_DoMoveMail", "Mail_OperateMail", "Mail_ExOperateMail", "Mail_DoMoveMail_v1"], async (_ctx, _node, args) => {
const destination = text(args, ["DstFile", "destination", "folder_name", "email_folder"], "Archive");
return await withMailbox(args, async (client) => await client.messageMove(String(number(args.index || args.limit, 1)), destination));
});
async function invoke(id: string, args: Record<string, unknown>) { const handler = getHandler(id); if (!handler) throw new RuntimeError("capability_not_implemented", id + " 不可用"); return handler; }
register("Mail_DoSendMail", async (ctx, node, args) => {
const handler = await invoke("Mail_Send", args); const config = mergeAccount(args);
return await handler(ctx, node, { ...args, host: text(config, ["host", "server_address"]), account: text(config, ["Account", "account", "username"]), password: text(config, ["password"]), to: list(args.RecvByEmail), cc: list(args.CC), bcc: list(args.BCC), subject: text(args, ["Subject"]), content: text(args, ["Body", "body"]), attachment: text(args, ["filePath", "attachment"]) }, {} as never);
});
register("Mail_DoSendMail_v1", async (ctx, node, args) => { const handler = await invoke("Mail_DoSendMail", args); return await handler(ctx, node, { ...args, account_obj: mergeAccount(args), RecvByEmail: args.RecvByEmail || args.target_email, CC: args.CC || args.cc, Subject: args.Subject || args.subject, Body: args.Body || args.body || args.content, filePath: args.filePath || args.attachment }, {} as never); });
register("Mail_ExSendMail", async (ctx, node, args) => { const handler = await invoke("Mail_Send", args);
return await handler(ctx, node, { ...args, to: list(args.target_email), cc: list(args.cc), subject: text(args, ["subject"]), content: text(args, ["body"]), attachment: text(args, ["filePath", "attachment"]) }, {} as never);
});
register("Mail_DoReplyMail", async (ctx, node, args) => { const handler = await invoke("Mail_ReplyMail_v2", args);
return await handler(ctx, node, { ...args, subject: text(args, ["Subject"], "Re:"), content: text(args, ["ReplyBody", "content"]), attachment: text(args, ["filePath", "attachment"]) }, {} as never);
});
register("Mail_DoReplyMail_v1", async (ctx, node, args) => { const handler = await invoke("Mail_ReplyMail_v2", args);
return await handler(ctx, node, { ...args, subject: text(args, ["subject"], "Re:"), content: text(args, ["content", "ReplyBody"]), attachment: text(args, ["attachment"]) }, {} as never);
});
@@ -0,0 +1,67 @@
import net from "node:net";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler, hasCapability } from "../engine/registry";
import "./message";
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
const ids = [
"Mail_FindMail", "Mail_FindMail_v1", "Mail_FindMail_v2",
"Mail_ReplyMail", "Mail_ReplyMail_v1", "Mail_ReplyMail_v2",
"Mail_DoGetMailMessage", "Mail_DoGetMailMessage_v1", "Mail_DoGetMailMessage_v2"
];
afterEach(async () => {});
describe("mail receive and reply", () => {
it("registers find, reply and Outlook message capabilities", () => {
for (const id of ids) expect(hasCapability(id), id).toBe(true);
});
it("replies through SMTP with the selected mail sender", async () => {
let received = "";
const server = net.createServer((socket) => {
socket.setEncoding("utf8");
socket.write("220 localhost ESMTP\r\n");
let buffer = "";
let dataMode = false;
socket.on("data", (chunk) => {
buffer += chunk;
const lines = buffer.split("\r\n");
buffer = lines.pop() || "";
for (const line of lines) {
if (dataMode) {
if (line === ".") {
dataMode = false;
socket.write("250 queued\r\n");
} else received += line + "\n";
} else if (/^EHLO/i.test(line)) socket.write("250-localhost\r\n250 OK\r\n");
else if (/^MAIL FROM/i.test(line)) socket.write("250 ok\r\n");
else if (/^RCPT TO/i.test(line)) socket.write("250 ok\r\n");
else if (/^DATA/i.test(line)) { dataMode = true; socket.write("354 end with dot\r\n"); }
else if (/^QUIT/i.test(line)) { socket.write("221 bye\r\n"); socket.end(); }
else socket.write("250 OK\r\n");
}
});
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const port = (server.address() as net.AddressInfo).port;
try {
const handler = getHandler("Mail_ReplyMail_v2");
expect(handler).toBeTruthy();
const result = await handler!(context, node("Mail_ReplyMail_v2"), {
account_obj: { host: "127.0.0.1", port, account: "robot@example.com", secure: "False" },
mail_obj: { from: "owner@example.com", subject: "status", messageId: "<abc@example.com>" },
content: "reply delivered"
}, component);
expect(result).toMatchObject({ accepted: ["owner@example.com"] });
expect(received).toContain("reply delivered");
expect(received).toContain("Re: status");
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
}
}, 30000);
});
@@ -0,0 +1,174 @@
import nodemailer from "nodemailer";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { powershell } from "../driver/windows/powershell";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) if (args[key] !== undefined && args[key] !== null && args[key] !== "") return String(args[key]);
return fallback;
}
function list(value: string): string[] {
return value.split(/[;,,;\s]+/).map((item) => item.trim()).filter(Boolean);
}
function accountObject(value: unknown): Record<string, unknown> {
if (value && typeof value === "object") return value as Record<string, unknown>;
if (typeof value === "string" && value.trim().startsWith("{")) {
try { return JSON.parse(value); } catch { return {}; }
}
return {};
}
register(["Mail_Send", "Mail_Send_V1", "Mail_UnionSend", "Mail_UnionSend_v1"], async (_ctx, node, args) => {
const configured = accountObject(args.account_obj);
const account = text({ ...configured, ...args }, ["account", "from", "username", "user"]);
const password = text({ ...configured, ...args }, ["password", "pass"]);
const host = text({ ...configured, ...args }, ["host", "smtp", "smtpHost", "server"]);
if (!host || !account) throw new Error(`邮件配置不完整(${node.componentId})`);
const port = Number(text({ ...configured, ...args }, ["port", "smtpPort"], "25"));
const secure = ["true", "1", "是", "True"].includes(text({ ...configured, ...args }, ["is_ssl", "secure", "ssl"], "False"));
const transporter = nodemailer.createTransport({ host, port, secure, auth: password ? { user: account, pass: password } : undefined });
try {
const result = await transporter.sendMail({
from: account,
to: list(text(args, ["to", "receiver"])),
cc: list(text(args, ["Cc", "cc"])),
bcc: list(text(args, ["Bcc", "bcc"])),
subject: text(args, ["subject"]),
text: text(args, ["content", "body"]),
attachments: text(args, ["attachment"]) ? [{ path: text(args, ["attachment"]) }] : undefined
});
return { messageId: result.messageId, accepted: result.accepted };
} finally {
transporter.close();
}
});
export interface NormalizedMail {
uid: string; from: string; to: string[]; cc: string[]; subject: string;
text: string; html: string; date: string; messageId: string; hasAttachment: boolean;
attachmentNames: string[];
}
export function normalizeAccount(args: Record<string, unknown>): Record<string, unknown> {
return accountObject(args.account_obj ?? args.accountObj ?? args.account);
}
function addressList(value: unknown): string[] {
if (!value) return [];
if (Array.isArray(value)) return value.map((item) => String(typeof item === "object" && item && "text" in (item as Record<string, unknown>) ? (item as { text: unknown }).text : item)).filter(Boolean);
if (typeof value === "object") { const record = value as { text?: unknown; value?: unknown }; return addressList(record.text ?? record.value); }
return String(value).split(/[;,,;\s]+/).filter(Boolean);
}
export function normalizeMail(input: Record<string, unknown>, uid = ""): NormalizedMail {
const attachments = Array.isArray(input.attachments) ? input.attachments as Array<Record<string, unknown>> : [];
return {
uid: String(input.uid ?? uid),
from: String(input.from ?? input.sender ?? ""),
to: addressList(input.to), cc: addressList(input.cc),
subject: String(input.subject ?? ""),
text: String(input.text ?? input.body ?? ""),
html: String(input.html ?? ""),
date: input.date instanceof Date ? input.date.toISOString() : String(input.date ?? input.receivedTime ?? ""),
messageId: String(input.messageId ?? ""),
hasAttachment: Boolean(input.hasAttachment ?? attachments.length),
attachmentNames: attachments.map((item) => String(item.filename ?? item.name ?? "")).filter(Boolean)
};
}
function matches(mail: NormalizedMail, args: Record<string, unknown>): boolean {
const contains = (needle: unknown, haystack: string): boolean => !needle || haystack.toLowerCase().includes(String(needle).toLowerCase());
return contains(args.From_data, mail.from)
&& contains(args.To_data, mail.to.join(" "))
&& contains(args.Subject_data, mail.subject)
&& contains(args.Body_data, mail.text + "\n" + mail.html)
&& (args.attachment === undefined || String(args.attachment) === "2" || (String(args.attachment) === "1") === mail.hasAttachment);
}
export async function findImapMail(args: Record<string, unknown>, node: { id: string; componentId: string }): Promise<NormalizedMail[]> {
const configured = normalizeAccount(args);
const merged = { ...args, ...configured };
const account = text(merged, ["account", "username", "user", "from"]);
const password = text(merged, ["password", "pass", "token"]);
const host = text(merged, ["imap", "imapHost", "imap_host", "server", "host"]);
if (!host || !account) throw new RuntimeError("validation_error", "IMAP 配置不完整:需要 host 和 account", { nodeId: node.id, componentId: node.componentId });
const port = Number(text(merged, ["imapPort", "imap_port", "recvPort", "port"], "993"));
const secure = ["true", "1", "是", "True"].includes(text(merged, ["imap_ssl", "imapSsl", "is_ssl", "secure"], "True"));
const folder = text(args, ["folder_name", "email_folder", "folder"], "INBOX");
const { ImapFlow } = await import("imapflow");
const client = new ImapFlow({ host, port, secure, auth: { user: account, pass: password }, logger: false });
const mails: NormalizedMail[] = [];
try {
await client.connect();
const lock = await client.getMailboxLock(folder);
try {
const range = String(args.search_range ?? "ALL").toUpperCase();
const criteria: Record<string, unknown> = range === "UNSEEN" ? { seen: false } : range === "SEEN" ? { seen: true } : range === "FLAGGED" ? { flagged: true } : range === "ANSWERED" ? { answered: true } : range === "DELETED" ? { deleted: true } : range === "DRAFT" ? { draft: true } : { all: true };
if (args.From_data) criteria.from = String(args.From_data);
if (args.To_data) criteria.to = String(args.To_data);
if (args.Subject_data) criteria.subject = String(args.Subject_data);
const uids = await client.search(criteria as never, { uid: true }) || [];
const selected = Math.max(1, Number(args.email_num ?? 1) || 1);
for (let index = uids.length - 1; index >= 0 && mails.length < Math.max(20, selected); index -= 1) {
const download = await client.fetchOne(String(uids[index]), { uid: true, envelope: true, flags: true, bodyStructure: true, source: true }, { uid: true });
if (!download || !download.source) continue;
const { simpleParser } = await import("mailparser");
const parsed = await simpleParser(download.source);
const mail = normalizeMail(parsed as unknown as Record<string, unknown>, String(download.uid));
if (matches(mail, args)) mails.push(mail);
}
} finally { lock.release(); }
} finally { client.close(); }
const selected = Math.max(1, Number(args.email_num ?? 1) || 1);
return mails.slice(0, selected);
}
register(["Mail_FindMail", "Mail_FindMail_v1", "Mail_FindMail_v2"], async (_ctx, node, args) => findImapMail(args, node));
register(["Mail_ReplyMail", "Mail_ReplyMail_v1", "Mail_ReplyMail_v2"], async (_ctx, node, args) => {
const configured = accountObject(args.mail_obj ?? args.email_obj ?? args.account_obj);
const mail = configured && Object.keys(configured).length ? configured as unknown as NormalizedMail : args.mail as unknown as NormalizedMail;
const merged = { ...args, ...normalizeAccount(args) };
const account = text(merged, ["account", "username", "user", "from"]);
const password = text(merged, ["password", "pass"]);
const host = text(merged, ["smtp", "smtpHost", "smtp_host", "server", "host"]);
if (!host || !account) throw new RuntimeError("validation_error", "SMTP 配置不完整:需要 host 和 account", { nodeId: node.id, componentId: node.componentId });
const port = Number(text(merged, ["smtpPort", "smtp_port", "sendPort", "port"], "465"));
const secure = ["true", "1", "是", "True"].includes(text(merged, ["smtp_ssl", "smtpSsl", "is_ssl", "secure"], "True"));
const transporter = nodemailer.createTransport({ host, port, secure, auth: password ? { user: account, pass: password } : undefined });
try {
const result = await transporter.sendMail({
from: account,
to: mail?.from ? [mail.from] : list(text(args, ["to", "receiver"])),
subject: text(args, ["subject"], mail?.subject ? "Re: " + mail.subject : "Re:"),
text: text(args, ["content", "body"]),
inReplyTo: mail?.messageId, references: mail?.messageId,
attachments: text(args, ["attachment"]) ? [{ path: text(args, ["attachment"]) }] : undefined
});
return { messageId: result.messageId, accepted: result.accepted };
} finally { transporter.close(); }
});
function outlookScript(args: Record<string, unknown>): string {
const criteria = Buffer.from(JSON.stringify({ to: String(args.To_data ?? ""), from: String(args.From_data ?? ""), subject: String(args.Subject_data ?? ""), body: String(args.Body_data ?? "") }), "utf8").toString("base64");
return [
"$outlook=New-Object -ComObject Outlook.Application;$ns=$outlook.GetNamespace('MAPI')",
`$folderPath=${JSON.stringify(String(args.folder_name || args.email_folder || "INBOX"))}`,
"$folder=$ns.GetDefaultFolder(6);foreach($part in ($folderPath -split '/')){ if($part -and $part -ne 'INBOX'){ $folder=$folder.Folders.Item($part) } }",
"$folder.Items.Sort('[ReceivedTime]',$true)|Out-Null;$rows=@();$count=[Math]::Min(500,$folder.Items.Count)",
`$criteria=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('${criteria}'))|ConvertFrom-Json`,
"for($i=1;$i -le $count;$i++){ $item=$folder.Items.Item($i); $subject=[string]$item.Subject; $body=[string]$item.Body; $to=[string]$item.To; $from=[string]$item.SenderEmailAddress; if(($criteria.to -and $to -notlike ('*'+$criteria.to+'*')) -or ($criteria.from -and $from -notlike ('*'+$criteria.from+'*')) -or ($criteria.subject -and $subject -notlike ('*'+$criteria.subject+'*')) -or ($criteria.body -and $body -notlike ('*'+$criteria.body+'*'))){continue}; $bodyB64=[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($body)); $rows += @{uid=[string]$item.EntryID;from=$from;to=$to;subject=$subject;text=$bodyB64;html='';date=$item.ReceivedTime.ToString('o');messageId=[string]$item.EntryID;hasAttachment=($item.Attachments.Count -gt 0);attachmentNames=@()}; if($rows.Count -ge 100){break} }",
"$json=$rows|ConvertTo-Json -Compress -Depth 4;[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($json))|Write-Output"
].join(";");
}
register(["Mail_DoGetMailMessage", "Mail_DoGetMailMessage_v1", "Mail_DoGetMailMessage_v2"], async (_ctx, node, args) => {
const output = await powershell(outlookScript(args), 120000);
if (!output.trim()) throw new RuntimeError("outlook_unavailable", "Outlook COM 未返回数据;请确认 Outlook 已安装并至少配置一个邮箱", { nodeId: node.id, componentId: node.componentId });
const rows = JSON.parse(Buffer.from(output.trim(), "base64").toString("utf8"));
const mails = (Array.isArray(rows) ? rows : [rows]).map((row: Record<string, unknown>) => normalizeMail({ ...row, text: row.text ? Buffer.from(String(row.text), "base64").toString("utf8") : "" }));
const selected = Math.max(1, Number(args.email_num ?? 1) || 1);
return mails.slice(0, selected);
});
@@ -0,0 +1,94 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../types";
import { getHandler } from "../engine/registry";
import type { RunContext } from "../engine/types";
import "./window-extended";
const run = promisify(execFile);
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id: "smoke-mouse", componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
const ctx = {
runId: "mouse-smoke", flowId: "mouse-smoke-flow", flow: {} as RunContext["flow"],
vars: {}, secrets: {}, signal: new AbortController().signal, evidenceDir: "D:/office-test2",
emit: () => {}, log: () => {}, componentOf: () => undefined
} as unknown as RunContext;
const PS_API = `Add-Type @'
using System; using System.Runtime.InteropServices;
public static class RpaMouseSmoke {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr v);
}
'@; try { [void][RpaMouseSmoke]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}; $c = [RpaMouseSmoke+POINT]::new(); [void][RpaMouseSmoke]::GetCursorPos([ref]$c); "$($c.X),$($c.Y)"`;
async function cursorPos(): Promise<[number, number]> {
const { stdout } = await run("powershell.exe", ["-NoProfile", "-NonInteractive", "-Command", PS_API], { windowsHide: true });
return stdout.trim().split(",").map(Number) as [number, number];
}
/** 共享桌面环境下光标可能被其他进程短暂挪动,轮询等待落位(容差 px),超时返回最后一次读数 */
async function waitForCursor(target: [number, number], tolerance = 0, attempts = 10, gapMs = 100): Promise<[number, number]> {
let pos = await cursorPos();
for (let i = 0; i < attempts; i += 1) {
if (Math.abs(pos[0] - target[0]) <= tolerance && Math.abs(pos[1] - target[1]) <= tolerance) return pos;
await new Promise((r) => setTimeout(r, gapMs));
pos = await cursorPos();
}
return pos;
}
async function invoke(componentId: string, args: Record<string, unknown>): Promise<unknown> {
const handler = getHandler(componentId);
expect(handler).toBeTypeOf("function");
return handler!(ctx, node(componentId), args, component);
}
let original: [number, number];
afterEach(async () => {
if (original) await invoke("Mouse_MoveToV2", { location: { x: original[0], y: original[1] }, position: "0", move_type: "2", speed: 100 });
});
describe("移动鼠标组件真实冒烟", () => {
it("对象坐标 x/y JSON 不再报无效, 瞬间移动真实落位", async () => {
original = await cursorPos();
// 用户屏幕框选生成的浮点对象坐标(此前直接报「鼠标坐标无效」)
const out = await invoke("Mouse_MoveToV2", { location: { x: 1397, y: 227.60000610351562 }, position: "0", move_type: "2", speed: 100 }) as { x: number; y: number };
expect(out).toEqual({ x: 1397, y: 228 });
const pos = await waitForCursor([1397, 228]);
expect(pos[0]).toBe(1397);
expect(pos[1]).toBe(228);
console.log("[移动鼠标] 对象坐标解析 + 瞬间移动落位 ✓ 实际光标:", pos);
}, 60000);
it("模拟人工移动(默认方式)平滑到达目标", async () => {
original = await cursorPos();
const out = await invoke("Mouse_MoveToV2", { location: "[300,300]", position: "0", move_type: "0", speed: 60 }) as { x: number; y: number };
expect(out).toEqual({ x: 300, y: 300 });
const pos = await waitForCursor([300, 300], 2);
expect(Math.abs(pos[0] - 300)).toBeLessThanOrEqual(2);
expect(Math.abs(pos[1] - 300)).toBeLessThanOrEqual(2);
console.log("[移动鼠标] 模拟人工移动 ✓ 实际光标:", pos);
}, 60000);
it("匀速直线移动与字符串数组坐标", async () => {
original = await cursorPos();
const out = await invoke("Mouse_MoveToV2", { location: "[500,400]", position: "0", move_type: "1", speed: 80 }) as { x: number; y: number };
expect(out).toEqual({ x: 500, y: 400 });
const pos = await waitForCursor([500, 400]);
expect(pos[0]).toBe(500);
expect(pos[1]).toBe(400);
console.log("[移动鼠标] 匀速直线 + 字符串坐标 ✓ 实际光标:", pos);
}, 60000);
it("相对激活窗口左上角模式返回有效坐标", async () => {
original = await cursorPos();
const out = await invoke("Mouse_MoveToV2", { location: { x: 10, y: 10 }, position: "1", move_type: "2", speed: 100 }) as { x: number; y: number };
expect(Number.isFinite(out.x)).toBe(true);
expect(Number.isFinite(out.y)).toBe(true);
console.log("[移动鼠标] 相对激活窗口模式 ✓ 目标:", out);
}, 60000);
});
@@ -0,0 +1,116 @@
import path from "node:path";
import { promises as fs } from "node:fs";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { defaultEngine } from "../engine/engine";
interface OcrWord { text: string; x: number; y: number; width: number; height: number; confidence: number }
interface RapidOcrLine { text: string; mean: number; box?: number[][] }
async function getOcr(): Promise<{ detect: (image: string) => Promise<{ texts: RapidOcrLine[] }> }> {
if (defaultEngine.ocrInstance) return defaultEngine.ocrInstance as { detect: (image: string) => Promise<{ texts: RapidOcrLine[] }> };
const mod = await import("@gutenye/ocr-node");
const instance = await mod.default.create();
defaultEngine.ocrInstance = instance;
return instance;
}
function requireImage(args: Record<string, unknown>, nodeId: string): string {
// uiSelector 是 OcrOffline_GetOcrTextNew 目录里唯一的图片字段(P0-1 修复)
for (const key of ["image_path", "imagePath", "file_path", "filePath", "path", "image", "pictureSource", "uiSelector"]) {
const v = String(args[key] ?? "").trim();
if (v) return v;
}
throw new RuntimeError("validation_error", "图片路径为空", { nodeId, componentId: nodeId, recommendation: "填写本地图片路径(png/jpg/bmp)" });
}
function boxToRect(box: number[][] | undefined): { x: number; y: number; width: number; height: number } {
const points = box ?? [];
const xs = points.map(p => p[0]);
const ys = points.map(p => p[1]);
const minX = Math.min(...(xs.length ? xs : [0]));
const minY = Math.min(...(ys.length ? ys : [0]));
const maxX = Math.max(...(xs.length ? xs : [0]));
const maxY = Math.max(...(ys.length ? ys : [0]));
return { x: minX, y: minY, width: maxX - minX, height: maxY - minY };
}
register("OcrOffline_GetOcrTextNew", async (ctx, node, args) => {
const imagePath = requireImage(args, node.id);
const ocr = await getOcr();
const result = await ocr.detect(imagePath);
const text = result.texts.map(t => t.text).join("\n");
ctx.log("OCR 识别完成:" + text.length + " 字符 / " + result.texts.length + " 行", "info", node.id);
return text;
});
register("OcrOffline_GetOcrTextPosition", async (ctx, node, args) => {
const imagePath = requireImage(args, node.id);
const ocr = await getOcr();
const result = await ocr.detect(imagePath);
const words = result.texts.map(t => {
const rect = boxToRect(t.box);
return { text: t.text, x: rect.x, y: rect.y, width: rect.width, height: rect.height, confidence: t.mean };
});
ctx.log("OCR 定位完成:" + words.length + " 行", "info", node.id);
return words;
});
/** 按垂直中心聚类成行(行高 60% 为阈值),行内按 x 排序,返回二维文本表格 */
function groupOcrRows(words: RapidOcrLine[]): string[][] {
const prepared = words
.filter((word) => String(word.text ?? "").trim())
.map((word) => {
const box = word.box ?? [];
const xs = box.map((point) => point[0]);
const ys = box.map((point) => point[1]);
const x = xs.length ? Math.min(...xs) : 0;
const y = ys.length ? Math.min(...ys) : 0;
const height = ys.length ? Math.max(...ys) - y : 0;
return { text: word.text, x, cy: y + height / 2, h: height };
})
.sort((a, b) => a.cy - b.cy || a.x - b.x);
const rows: Array<{ cy: number; h: number; cells: typeof prepared }> = [];
let current: (typeof rows)[number] | null = null;
for (const item of prepared) {
if (!current || Math.abs(item.cy - current.cy) > Math.max(current.h, item.h, 8) * 0.6) {
current = { cy: item.cy, h: item.h, cells: [] };
rows.push(current);
}
current.cells.push(item);
}
return rows.map((row) => row.cells.sort((a, b) => a.x - b.x).map((cell) => cell.text.trim()));
}
register("OcrOffline_GetOcrTable", async (ctx, node, args) => {
const imagePath = requireImage(args, node.id);
const ocr = await getOcr();
const result = await ocr.detect(imagePath);
const rows = groupOcrRows(result.texts);
ctx.log("OCR 表格识别完成:" + rows.length + " 行", "info", node.id);
// showTable=1:写入流程数据表(与 Bivartable 系列组件共享同一份数据簿)
if (String(args.showTable ?? "1") === "1") {
ctx.vars["__rpaDatabook"] = rows;
ctx.log("已写入流程数据表", "info", node.id);
}
// isExcel=1:另存为 Excel 文件
let excelFile = "";
if (String(args.isExcel ?? "0") === "1") {
const ExcelJS = (await import("exceljs")).default;
const workbook = new ExcelJS.Workbook();
const sheet = workbook.addWorksheet("Sheet1");
sheet.addRows(rows);
const rawFolder = String(args.excelPath ?? "").trim();
// ${res} 流程资源目录占位与空值统一落到运行证据目录
const folder = rawFolder && rawFolder !== "${res}" ? rawFolder : ctx.evidenceDir;
const fileName = String(args.excelName ?? "表格数据.xlsx").replace(/\$\{randomString\}/g, Math.random().toString(36).slice(2, 8));
excelFile = path.resolve(path.isAbsolute(fileName) ? fileName : path.join(folder, fileName));
await fs.mkdir(path.dirname(excelFile), { recursive: true });
await workbook.xlsx.writeFile(excelFile);
ctx.log("已保存 Excel:" + excelFile, "info", node.id);
}
return excelFile ? { rows, excelFile } : rows;
});
@@ -0,0 +1,102 @@
import { existsSync } from "node:fs";
import { describe, expect, it } from "vitest";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./ocr-caps";
import "./remaining-core";
const ctx = {
signal: new AbortController().signal,
log: () => {},
vars: {},
evidenceDir: "D:/office-test2/ocr-test"
} as unknown as RunContext;
const node: FlowNode = {
id: "smoke",
componentId: "smoke",
position: { x: 0, y: 0 },
values: {},
options: {},
};
const ocrFixtures = [
"D:/office-test2/ocr-test/general.png",
"D:/office-test2/ocr-test/code.png",
"D:/office-test2/ocr-test/table.png",
];
const ocrFixturesReady = ocrFixtures.every((file) => existsSync(file));
async function runOcr(id: string, args: Record<string, unknown>) {
const handler = getHandler(id);
expect(handler, `${id} 应已注册`).toBeTruthy();
return handler!(ctx, node, args, {} as CatalogComponent);
}
describe.skipIf(!ocrFixturesReady)("OCR 组件冒烟(本地离线识别,真实模型推理)", () => {
it("OCR_GeneralTextV2 通用文本", async () => {
const out = await runOcr("OCR_GeneralTextV2", { path: "D:/office-test2/ocr-test/general.png" });
console.log("[GeneralTextV2]", JSON.stringify(out));
expect(String(out)).toContain("张三");
}, 300000);
it("OCR_KeyValueIdentification 键值对", async () => {
const out = await runOcr("OCR_KeyValueIdentification", { image_path: "D:/office-test2/ocr-test/general.png" });
console.log("[KeyValue]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OCR_AmountIdentification 金额", async () => {
const out = await runOcr("OCR_AmountIdentification", { path: "D:/office-test2/ocr-test/general.png" });
console.log("[Amount]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OCR_CalculationCode 算术验证码", async () => {
const out = await runOcr("OCR_CalculationCode", { path: "D:/office-test2/ocr-test/code.png" });
console.log("[CalcCode]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OCR_CommonEnglishNumbersCode 普通验证码", async () => {
const out = await runOcr("OCR_CommonEnglishNumbersCode", { path: "D:/office-test2/ocr-test/code.png" });
console.log("[CommonCode]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OCR_IdCardV2 卡证族代表", async () => {
const out = await runOcr("OCR_IdCardV2", { path: "D:/office-test2/ocr-test/general.png" });
console.log("[IdCardV2]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OCR_VatInvoiceV2 票据族代表", async () => {
const out = await runOcr("OCR_VatInvoiceV2", { path: "D:/office-test2/ocr-test/table.png" });
console.log("[VatInvoiceV2]", JSON.stringify(out));
expect(String(out).length).toBeGreaterThan(0);
}, 300000);
it("OcrOffline_GetOcrTextNew 离线文本", async () => {
const out = await runOcr("OcrOffline_GetOcrTextNew", { uiSelector: "D:/office-test2/ocr-test/general.png" });
console.log("[OcrTextNew]", JSON.stringify(out));
expect(String(out)).toContain("订单号");
}, 300000);
it("OcrOffline_GetOcrTextPosition 离线定位", async () => {
const out = await runOcr("OcrOffline_GetOcrTextPosition", { image_path: "D:/office-test2/ocr-test/general.png" });
const words = out as Array<{ text: string; x: number; y: number }>;
console.log("[OcrPosition]", JSON.stringify(words?.slice(0, 3)));
expect(Array.isArray(words)).toBe(true);
expect(words.length).toBeGreaterThan(0);
expect(words[0]).toHaveProperty("x");
}, 300000);
it("OcrOffline_GetOcrTable 离线表格", async () => {
const out = await runOcr("OcrOffline_GetOcrTable", { image_path: "D:/office-test2/ocr-test/table.png", showTable: "1", isExcel: "0" });
const rows = out as string[][];
console.log("[OcrTable]", JSON.stringify(rows));
expect(Array.isArray(rows)).toBe(true);
expect(rows.length).toBeGreaterThan(0);
}, 300000);
});
@@ -0,0 +1,117 @@
import { readFile, mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import ExcelJS from "exceljs";
import { afterEach, describe, expect, it } from "vitest";
import type { Catalog, CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./office";
import "./extended";
import "./office-file-extended";
import { EXCEL_COMMON_ALIASES } from "./office-aliases";
const dirs: string[] = [];
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown>) { const handler = getHandler(id); expect(handler, id).toBeTruthy(); return handler!(context, node(id), args, component); }
afterEach(async () => Promise.all(dirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true }))));
describe("common Excel aliases", () => {
it("registers only real catalog aliases", async () => {
// vitest 工作目录为仓库根,包内测试统一以 cwd 定位仓库资源
const catalog = JSON.parse(await readFile(path.resolve(process.cwd(), "resources/catalog.json"), "utf8")) as Catalog;
const ids = new Set(catalog.components.map((item) => item.id));
expect(EXCEL_COMMON_ALIASES.length).toBeGreaterThanOrEqual(25);
for (const id of EXCEL_COMMON_ALIASES) { expect(ids.has(id), id).toBe(true); expect(getHandler(id), id).toBeTruthy(); }
});
it("runs a real create, write and read pipeline through alias IDs", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-alias-")); dirs.push(directory);
const handle = await run("Excel_NewExcel", { path: directory, name: "aliases.xlsx", override: "1" });
await run("Excel_WriteCellNew", { workbook: handle, sheet: "Sheet1", row: 1, column: "A", content: "alias", save: "1" });
await expect(run("Excel_ReadCellNew", { workbook: handle, sheet: "Sheet1", row: 1, column: "A" })).resolves.toBe("alias");
});
it("accepts the Excel_NewExcel catalog keys (filePath / isMakedirs)", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-new-")); dirs.push(directory);
const handle = await run("Excel_NewExcel", { filePath: directory, name: "catalog-keys.xlsx", isMakedirs: "True" }) as { path: string };
expect(handle.path).toBe(path.resolve(directory, "catalog-keys.xlsx"));
});
it("rejects a missing directory when isMakedirs is False", async () => {
const directory = path.join(await mkdtemp(path.join(os.tmpdir(), "rpa-excel-new-")), "missing");
dirs.push(directory);
await expect(run("Excel_NewExcel", { filePath: directory, name: "none.xlsx", isMakedirs: "False" })).rejects.toThrow("目录不存在");
});
it("opens a workbook through the Excel_OpenExcel filePath key", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-open-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "open-me.xlsx" }) as { path: string };
await expect(run("Excel_OpenExcel", { filePath: created.path })).resolves.toMatchObject({ path: created.path });
});
it("renames a sheet through the Excel_RenameSheet newSheet key", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-rename-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "rename-me.xlsx" }) as { path: string };
await expect(run("Excel_RenameSheet", { objExcel: created, sheet: "Sheet1", newSheet: "库存表", isSave: "True" })).resolves.toBeTruthy();
const reopened = await run("Excel_OpenExcel", { filePath: created.path }) as { id: string };
await expect(run("Excel_GetSheetsName", { objExcel: reopened })).resolves.toEqual(["库存表"]);
});
it("writes a range through the Excel_WriteRange catalog keys (objExcel/cellPos/data/isSave)", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-range-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "range.xlsx" }) as { path: string };
await run("Excel_WriteRange", { objExcel: created, sheet: "Sheet1", cellPos: "B2", data: [[1, "壹"], [2, "贰"]], isSave: "True" });
const reopened = await run("Excel_OpenExcel", { filePath: created.path }) as { id: string };
await expect(run("Excel_Cs_ReadContentV2", { workbook: reopened, sheet: "Sheet1", excelRng: "B2:C3" })).resolves.toEqual([[1, "壹"], [2, "贰"]]);
});
it("saves as through the Excel_SaveAs catalog keys (filename/isAbsolutePath)", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-saveas-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "origin.xlsx" }) as { path: string };
await run("Excel_WriteCell", { workbook: created, sheet: "Sheet1", row: 1, column: "A", content: "saveas" });
const savedAs = await run("Excel_SaveAs", { objExcel: created, filename: path.join(directory, "legacy-saveas.xlsx"), isAbsolutePath: "True" }) as { path: string };
expect(savedAs.path).toBe(path.resolve(directory, "legacy-saveas.xlsx"));
const reopened = await run("Excel_OpenExcel", { filePath: savedAs.path }) as { id: string };
await expect(run("Excel_Cs_ReadContentV2", { workbook: reopened, sheet: "Sheet1", excelRng: "A1" })).resolves.toEqual([["saveas"]]);
});
it("resolves a relative Excel_SaveAs filename against the original workbook directory", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-saveas-rel-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "rel-origin.xlsx" }) as { path: string };
const savedAs = await run("Excel_SaveAs", { objExcel: created, filename: "rel-target.xlsx", isAbsolutePath: "False" }) as { path: string };
expect(savedAs.path).toBe(path.resolve(directory, "rel-target.xlsx"));
});
it("combines the Excel_Cs_SaveAs directory path with the name field", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-saveas-cs-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "cs-origin.xlsx" }) as { path: string };
const targetDir = path.join(directory, "nested");
const savedAs = await run("Excel_Cs_SaveAs", { workbook: created, path: targetDir, name: "cs-target.xlsx" }) as { path: string };
expect(savedAs.path).toBe(path.resolve(targetDir, "cs-target.xlsx"));
const reopened = new ExcelJS.Workbook();
await reopened.xlsx.readFile(savedAs.path);
expect(reopened.worksheets.length).toBeGreaterThan(0);
});
it("clears a range through the Excel_ClearRange cellPos key", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-clear-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "clear.xlsx" }) as { path: string };
await run("Excel_WriteRange", { objExcel: created, sheet: "Sheet1", cellPos: "A1", data: [["甲", "乙"], ["丙", "丁"]] });
await run("Excel_ClearRange", { objExcel: created, sheet: "Sheet1", cellPos: "A1:B2", isFormat: "False", isSave: "True" });
const reopened = await run("Excel_OpenExcel", { filePath: created.path }) as { id: string };
await expect(run("Excel_Cs_ReadContentV2", { workbook: reopened, sheet: "Sheet1", excelRng: "A1:B2" })).resolves.toEqual([["", ""], ["", ""]]);
});
it("clears through the Excel_Cs_ClearRange startRow/startColumn keys", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-excel-clear-cs-")); dirs.push(directory);
const created = await run("Excel_NewExcel", { filePath: directory, name: "clear-cs.xlsx" }) as { path: string };
await run("Excel_WriteRange", { objExcel: created, sheet: "Sheet1", cellPos: "A1", data: [["1", "2"], ["3", "4"]] });
await run("Excel_Cs_ClearRange", { objExcel: created, sheet: "Sheet1", startRow: "2", startColumn: "A", lastRow: "2", lastColumn: "B", mode: "内容", save: "True" });
const reopened = await run("Excel_OpenExcel", { filePath: created.path }) as { id: string };
await expect(run("Excel_Cs_ReadContentV2", { workbook: reopened, sheet: "Sheet1", excelRng: "A2:B2" })).resolves.toEqual([["", ""]]);
await expect(run("Excel_Cs_ReadContentV2", { workbook: reopened, sheet: "Sheet1", excelRng: "A1:B1" })).resolves.toEqual([["1", "2"]]);
});
});
@@ -4,7 +4,7 @@ import os from "node:os";
import path from "node:path";
import { PDFDocument } from "pdf-lib";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import { getHandler } from "../engine/registry";
import type { RunContext } from "../engine/types";
import "./office-file-extended";
@@ -4,6 +4,7 @@ import path from "node:path";
import { PDFDocument } from "pdf-lib";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { acquireWorkbook } from "./excel-store";
type WorkbookSource = { workbook: ExcelJS.Workbook; filePath: string };
@@ -29,8 +30,7 @@ function requiredPath(candidate: unknown, label: string, nodeId: string, compone
async function loadWorkbook(args: Record<string, unknown>, nodeId: string, componentId: string): Promise<WorkbookSource> {
const filePath = requiredPath(value(args, ["workbook", "objExcel", "objExcelWorkBook", "file", "path"]), "Excel path", nodeId, componentId);
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(filePath);
const workbook = await acquireWorkbook(filePath);
return { workbook, filePath };
}
@@ -134,11 +134,12 @@ const readColumn: CapabilityHandler = async (_ctx, node, args) => {
const sheet = worksheetOf(source.workbook, args);
const column = columnNumber(value(args, ["columnName", "column", "cellPos"], "A"));
const startRow = Math.max(1, Number(value(args, ["row", "startRow"], 1)));
return Array.from({ length: Math.max(0, sheet.rowCount - startRow + 1) }, (_, index) => cellValue(sheet.getCell(startRow + index, column).value));
const values = Array.from({ length: Math.max(0, sheet.rowCount - startRow + 1) }, (_, index) => cellValue(sheet.getCell(startRow + index, column).value));
// 触碰过尾部空行(如定位空单元格)会撑大 rowCount,裁掉尾部空值避免误报
while (values.length > 0 && (values[values.length - 1] === "" || values[values.length - 1] === null || values[values.length - 1] === undefined)) values.pop();
return values;
};
register(["Excel_Cs_ReadColumn", "Excel_GetColDataNew"], readColumn);
register("Excel_Cs_DeleteRow", async (_ctx, node, args) => {
register(["Excel_Cs_ReadColumn", "Excel_GetColDataNew"], readColumn);register("Excel_Cs_DeleteRow", async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const mode = text(args, ["mode"]);
@@ -163,12 +164,30 @@ register("Excel_Cs_DeleteColumn", async (_ctx, node, args) => {
register("Excel_Cs_ClearRangeV4", async (_ctx, node, args) => {
const source = await loadWorkbook(args, node.id, node.componentId);
const sheet = worksheetOf(source.workbook, args);
const range = text(args, ["excelRng", "range"]);
// 兼容各别名版本的键:V4=excelRng、V2/V3=range、Legacy Excel_ClearRange=cellPos、V1=startRow/startColumn/lastRow/lastColumn
let range = text(args, ["excelRng", "range", "cellPos"]);
if (!range) {
const startRow = text(args, ["startRow"]);
const startColumn = text(args, ["startColumn"]);
if (startRow && startColumn) {
const lastRow = text(args, ["lastRow"], startRow);
const lastColumn = text(args, ["lastColumn"], startColumn);
range = `${startColumn}${startRow}:${lastColumn}${lastRow}`;
}
}
if (!range) throw new RuntimeError("validation_error", "Range is empty", { nodeId: node.id, componentId: node.componentId });
const [start, end = start] = range.split(":");
const first = sheet.getCell(start);
const last = sheet.getCell(end);
for (let row = first.row; row <= last.row; row += 1) for (let column = first.col; column <= last.col; column += 1) sheet.getCell(row, column).value = null;
// 清除模式:全部=内容+格式,格式=仅格式,其余(内容/默认)=仅内容;Legacy isFormat=True 同样清格式
const mode = text(args, ["clearType", "mode"], "内容");
const clearFormat = mode.includes("全部") || mode.includes("格式") || truthy(value(args, ["isFormat"]));
const clearValue = !mode.includes("格式");
for (let row = first.row; row <= last.row; row += 1) for (let column = first.col; column <= last.col; column += 1) {
const cell = sheet.getCell(row, column);
if (clearValue) cell.value = null;
if (clearFormat) cell.style = {};
}
await saveWhenRequested(source, args);
return range;
});
@@ -223,7 +242,8 @@ register("Excel_Cs_CopyV2", async (_ctx, node, args) => {
});
function parsePdfPages(raw: string, pageCount: number): number[] {
if (!raw.trim()) return Array.from({ length: pageCount }, (_, index) => index);
// 页码留空时返回空集:由调用方抛出「No valid PDF pages」,避免静默复制全本
if (!raw.trim()) return [];
const pages = new Set<number>();
for (const value of parseList(raw, Number)) if (value >= 1 && value <= pageCount) pages.add(value - 1);
return [...pages];
@@ -3,13 +3,16 @@ import { promises as fs } from "node:fs";
import path from "node:path";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { defaultEngine } from "../engine/engine";
import { acquireWorkbook, publishWorkbook, releaseWorkbook } from "./excel-store";
interface ExcelHandle {
id: string;
path: string;
}
const workbooks = new Map<string, ExcelJS.Workbook>();
// 句柄 id → 工作簿;本体挂在引擎实例上(任务 1.3),此处委托 defaultEngine。
const workbooks = defaultEngine.excelHandles;
function value(args: Record<string, unknown>, keys: string[], fallback = ""): unknown {
for (const key of keys) {
@@ -24,6 +27,7 @@ function text(args: Record<string, unknown>, keys: string[], fallback = ""): str
function bool(valueToRead: unknown, fallback = false): boolean {
if (valueToRead === undefined || valueToRead === null || valueToRead === "") return fallback;
if (typeof valueToRead === "string") return ["true", "1", "yes", "是", "开启"].includes(valueToRead.trim().toLowerCase());
return [true, "true", "1", 1, "是", "开启"].includes(valueToRead as never);
}
@@ -36,6 +40,7 @@ function required(valueToRead: unknown, label: string, nodeId: string, component
function newHandle(filePath: string, workbook: ExcelJS.Workbook): ExcelHandle {
const handle = { id: `excel_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 7)}`, path: filePath };
workbooks.set(handle.id, workbook);
publishWorkbook(filePath, workbook);
return handle;
}
@@ -43,11 +48,12 @@ async function resolveWorkbook(args: Record<string, unknown>, nodeId: string, co
const raw = value(args, ["workbook", "objExcel", "objExcelWorkBook", "file", "path"]);
const id = typeof raw === "object" && raw !== null ? String((raw as Partial<ExcelHandle>).id || "") : String(raw || "");
const existing = id ? workbooks.get(id) : undefined;
if (existing) return { handle: { id, path: String((raw as Partial<ExcelHandle>).path || "") }, workbook: existing };
// 直接返回流程变量持有的原始句柄对象:saveAs 等处理器对其 path 的改写需要
// 同步反映到后续节点引用的同一个对象上
if (existing) return { handle: raw as ExcelHandle, workbook: existing };
const filePath = required(typeof raw === "object" ? (raw as Partial<ExcelHandle>).path : raw, "Excel文件路径", nodeId, componentId);
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(filePath);
return { handle: newHandle(filePath, workbook), workbook };
const workbook = await acquireWorkbook(filePath);
return { handle: { id, path: filePath }, workbook };
}
function sheetOf(workbook: ExcelJS.Workbook, name: string): ExcelJS.Worksheet {
@@ -101,9 +107,17 @@ function rowsFrom(valueToRead: unknown): unknown[][] {
}
const newExcel: CapabilityHandler = async (_ctx, node, args) => {
const folder = required(value(args, ["path", "folder"]), "Excel目录", node.id, node.componentId);
const folder = required(value(args, ["filePath", "path", "folder"]), "文件路径", node.id, node.componentId);
const name = text(args, ["name"], "新建工作簿.xlsx");
const filePath = path.resolve(folder, path.extname(name) ? name : `${name}.xlsx`);
const makedirs = bool(value(args, ["isMakedirs"]), true);
if (!makedirs) {
try {
await fs.access(path.dirname(filePath));
} catch {
throw new RuntimeError("validation_error", `目录不存在:${path.dirname(filePath)}`, { nodeId: node.id, componentId: node.componentId, recommendation: "将「是否创建该路径」设为 True,或先创建目录" });
}
}
if (!bool(value(args, ["override"]), true)) {
try {
await fs.access(filePath);
@@ -120,10 +134,9 @@ const newExcel: CapabilityHandler = async (_ctx, node, args) => {
};
const openExcel: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(value(args, ["file", "path"]), "Excel文件", node.id, node.componentId);
const workbook = new ExcelJS.Workbook();
await workbook.xlsx.readFile(path.resolve(filePath));
return newHandle(path.resolve(filePath), workbook);
const filePath = path.resolve(required(value(args, ["file", "filePath", "path"]), "Excel文件", node.id, node.componentId));
const workbook = await acquireWorkbook(filePath);
return newHandle(filePath, workbook);
};
const readExcel: CapabilityHandler = async (_ctx, node, args) => {
@@ -144,14 +157,18 @@ const writeCell: CapabilityHandler = async (_ctx, node, args) => {
const writeRange: CapabilityHandler = async (_ctx, node, args) => {
const { handle, workbook } = await resolveWorkbook(args, node.id, node.componentId);
const sheet = sheetOf(workbook, text(args, ["sheet", "sheetName"]));
const start = `${text(args, ["column"], "A")}${text(args, ["row"], "1")}`;
const cellPos = text(args, ["cellPos"]);
const cellMatch = cellPos.match(/^([A-Za-z]+)(\d+)$/);
const column = cellMatch ? cellMatch[1] : text(args, ["column"], "A");
const row = cellMatch ? cellMatch[2] : text(args, ["row"], "1");
const start = `${column}${row}`;
const startCell = sheet.getCell(start);
const startRow = startCell.row;
const startColumn = startCell.col;
for (const [rowIndex, row] of rowsFrom(value(args, ["contents", "table"])).entries()) {
for (const [rowIndex, row] of rowsFrom(value(args, ["contents", "data", "table"])).entries()) {
for (const [columnIndex, cell] of row.entries()) sheet.getCell(startRow + rowIndex, startColumn + columnIndex).value = cell as ExcelJS.CellValue;
}
if (bool(value(args, ["save"]), true)) await workbook.xlsx.writeFile(handle.path);
if (bool(value(args, ["save", "isSave"]), true)) await workbook.xlsx.writeFile(handle.path);
return handle;
};
@@ -163,10 +180,20 @@ const saveExcel: CapabilityHandler = async (_ctx, node, args) => {
const saveAsExcel: CapabilityHandler = async (_ctx, node, args) => {
const { handle, workbook } = await resolveWorkbook(args, node.id, node.componentId);
const target = required(value(args, ["path", "file", "savePath"]), "另存为路径", node.id, node.componentId);
handle.path = path.resolve(target);
// CS 版(Excel_Cs_SaveAs)用 path=目录 + name=文件名;Legacy(Excel_SaveAs)用 filename=完整路径
const rawName = text(args, ["name"]);
const raw = rawName
? path.resolve(text(args, ["path", "file", "savePath"], ""), rawName)
: required(value(args, ["path", "file", "savePath", "filename", "saveFilePath"]), "另存为路径", node.id, node.componentId);
// Legacy Excel_SaveAs 用 isAbsolutePath 标记 filename 是否为绝对路径,相对路径基于原文件目录
const absolute = rawName ? true : bool(value(args, ["isAbsolutePath"]), true);
const previousPath = handle.path;
handle.path = absolute ? path.resolve(raw) : path.resolve(path.dirname(previousPath), raw);
await fs.mkdir(path.dirname(handle.path), { recursive: true });
await workbook.xlsx.writeFile(handle.path);
// 另存后同一实例以新路径登记,旧路径的缓存不再代表最新内容,释放之
publishWorkbook(handle.path, workbook);
if (path.resolve(previousPath) !== handle.path) releaseWorkbook(previousPath);
return handle;
};
@@ -184,7 +211,7 @@ register(["Excel_GetSheetsName", "Excel_GetSheetNames"], async (_ctx, node, args
register(["Excel_Cs_Rename", "Excel_RenameSheet"], async (_ctx, node, args) => {
const { handle, workbook } = await resolveWorkbook(args, node.id, node.componentId);
const oldName = required(value(args, ["sheet", "oldName"]), "原 Sheet 名称", node.id, node.componentId);
const newName = required(value(args, ["newName", "name", "sheetName"]), "新 Sheet 名称", node.id, node.componentId);
const newName = required(value(args, ["newSheet", "newName", "name", "sheetName"]), "新 Sheet 名称", node.id, node.componentId);
const sheet = sheetOf(workbook, oldName);
sheet.name = newName;
await workbook.xlsx.writeFile(handle.path);
@@ -0,0 +1,54 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { PDFDocument, StandardFonts } from "pdf-lib";
import { PNG } from "pngjs";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./pdf";
const directories: string[] = [];
const context = {} as RunContext;
const component = {} as CatalogComponent;
const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0, y: 0 }, values: {}, options: {} });
async function run(id: string, args: Record<string, unknown>) {
const handler = getHandler(id);
expect(handler, id).toBeTruthy();
return handler!(context, node(id), args, component);
}
afterEach(async () => Promise.all(directories.splice(0).map((directory) => rm(directory, { recursive: true, force: true }))));
describe("PDF high-level capabilities", () => {
it("creates a pdf fixture", async () => {
const directory = await mkdtempSafe();
const pdf = await PDFDocument.create();
const page = pdf.addPage([220, 120]);
const font = await pdf.embedFont(StandardFonts.Helvetica);
const png = new PNG({ width: 8, height: 8 });
png.data.fill(255); png.data[0] = 20; png.data[1] = 90; png.data[2] = 200; png.data[3] = 255;
page.drawImage(await pdf.embedPng(PNG.sync.write(png)), { x: 140, y: 70, width: 40, height: 40 });
for (const [value, x, y] of [["Name", 24, 90], ["Age", 120, 90], ["Alice", 24, 72], ["30", 120, 72]] as Array<[string, number, number]>) page.drawText(value, { x, y, size: 9, font });
await writeFile(path.join(directory, "source.pdf"), await pdf.save());
const filePath = path.join(directory, "source.pdf");
expect((await readFile(path.join(directory, "source.pdf"))).byteLength).toBeGreaterThan(0);
const images = await run("PDF_GetPdfImg", { fileInPath: filePath, pdfRange: "all", pdfNum: "1", imgSavePath: directory, imageType: "png" }) as string[];
expect(images.length).toBeGreaterThan(0);
expect((await readFile(images[0])).subarray(0, 4)).toEqual(Buffer.from([0x89, 0x50, 0x4e, 0x47]));
await expect(run("PDF_GetPdfTable", { fileInPath: filePath, pdfRange: "all", pdfNum: "1" })).resolves.toEqual([["Name", "Age"], ["Alice", "30"]]);
const encryptedPath = path.join(directory, "encrypted.pdf");
await writeFile(encryptedPath, await readFile(filePath));
await expect(run("PDF_PdfEncryption", { fileInPath: encryptedPath, passWord: "secret" })).resolves.toBe(encryptedPath);
const pdfjs = await import("pdfjs-dist/legacy/build/pdf.mjs");
const opened = await pdfjs.getDocument({ data: new Uint8Array(await readFile(encryptedPath)), password: "secret" }).promise;
expect(opened.numPages).toBe(1);
}, 60000);
});
async function mkdtempSafe(): Promise<string> {
const directory = await (await import("node:fs/promises")).mkdtemp(path.join(os.tmpdir(), "rpa-pdf-high-"));
directories.push(directory);
return directory;
}
+264
View File
@@ -0,0 +1,264 @@
import { promises as fs } from "node:fs";
import fsSync from "node:fs";
import { fileURLToPath, pathToFileURL } from "node:url";
import { PDFDocument } from "pdf-lib";
import path from "node:path";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) if (args[key] !== undefined && args[key] !== null && args[key] !== "") return String(args[key]);
return fallback;
}
function required(value: string, label: string, nodeId: string, componentId: string): string {
if (!value.trim()) throw new RuntimeError("validation_error", `${label}为空`, { nodeId, componentId });
return path.resolve(value);
}
const pageCount: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const document = await PDFDocument.load(await fs.readFile(filePath), { ignoreEncryption: true });
return document.getPageCount();
};
const concat: CapabilityHandler = async (_ctx, node, args) => {
const folder = required(text(args, ["newPdfPath", "folder", "path"]), "输出目录", node.id, node.componentId);
const name = text(args, ["newPdfName", "name"], "合并结果.pdf");
const output = path.join(folder, path.extname(name) ? name : `${name}.pdf`);
const rawList = text(args, ["pdfList", "files"]);
let files: string[] = [];
try {
const parsed = JSON.parse(rawList);
files = Array.isArray(parsed) ? parsed.map(String) : [String(parsed)];
} catch {
files = rawList.split(/[\r\n;,]+/).map((item) => item.trim()).filter(Boolean);
}
if (!files.length) throw new RuntimeError("validation_error", "待合并 PDF 列表为空", { nodeId: node.id, componentId: node.componentId });
const target = await PDFDocument.create();
for (const file of files) {
const source = await PDFDocument.load(await fs.readFile(path.resolve(file)), { ignoreEncryption: true });
const pages = await target.copyPages(source, source.getPageIndices());
pages.forEach((page) => target.addPage(page));
}
await fs.mkdir(path.dirname(output), { recursive: true });
await fs.writeFile(output, await target.save());
return output;
};
register(["PDF_GetPdfPageNum", "PDF_GetPageCount"], pageCount);
register("PDF_ConcatPdf", concat);
register("PDF_GetPdfText", async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const pdfjs = await import("pdfjs-dist/legacy/build/pdf.mjs");
const document = await pdfjs.getDocument({ data: new Uint8Array(await fs.readFile(filePath)) }).promise;
const pages = text(args, ["pdfNum", "page"]);
const indices = pages ? pages.split(/[ ,,;;]+/).map((item) => Number(item)).filter((item) => item > 0) : Array.from({ length: document.numPages }, (_, index) => index + 1);
const chunks: string[] = [];
for (const index of indices) {
const page = await document.getPage(index);
const content = await page.getTextContent();
chunks.push(content.items.map((item) => ("str" in item ? item.str : "")).join(""));
}
return chunks.join("\n");
});
type PdfJsDocument = Awaited<ReturnType<typeof import("pdfjs-dist/legacy/build/pdf.mjs")["getDocument"]>["promise"]>;
function standardFontUrl(): string | undefined {
const candidates = [
process.env.RPA_PDFJS_STANDARD_FONTS,
path.join(process.cwd(), "node_modules", "pdfjs-dist", "standard_fonts"),
path.join(process.execPath, "resources", "app.asar", "node_modules", "pdfjs-dist", "standard_fonts"),
path.join(process.execPath, "..", "resources", "app.asar", "node_modules", "pdfjs-dist", "standard_fonts")
].filter(Boolean) as string[];
const found = candidates.find((candidate) => fsSync.existsSync(candidate));
return found ? pathToFileURL(path.join(found, path.sep)).href : undefined;
}
class LocalStandardFontFactory {
baseUrl: string;
constructor(options: { baseUrl: string }) { this.baseUrl = options.baseUrl; }
async fetch({ filename }: { filename: string }): Promise<Uint8Array> {
return new Uint8Array(await fs.readFile(path.join(fileURLToPath(this.baseUrl), filename)));
}
}
async function loadPdf(filePath: string, password: string) {
const canvasModule = await import("@napi-rs/canvas");
const globals = globalThis as unknown as Record<string, unknown>;
globals.Path2D = canvasModule.Path2D;
globals.DOMMatrix = canvasModule.DOMMatrix;
globals.ImageData = canvasModule.ImageData;
const pdfjs = await import("pdfjs-dist/legacy/build/pdf.mjs");
return { pdfjs, document: await pdfjs.getDocument({ data: new Uint8Array(await fs.readFile(filePath)), password: password || undefined, isEvalSupported: false, useSystemFonts: false, standardFontDataUrl: standardFontUrl(), StandardFontDataFactory: LocalStandardFontFactory }).promise };
}
function pageIndices(document: PdfJsDocument, raw: unknown): number[] {
const range = String(raw ?? "all").trim().toLowerCase();
if (!range || range === "all") return Array.from({ length: document.numPages }, (_, index) => index + 1);
const result = new Set<number>();
for (const part of range.split(/[,,;;\s]+/).filter(Boolean)) {
const match = part.match(/^(\d+)-(\d+)$/);
if (match) {
const start = Math.max(1, Number(match[1])); const end = Math.min(document.numPages, Number(match[2]));
for (let index = start; index <= end; index += 1) result.add(index);
} else {
const index = Number(part);
if (index >= 1 && index <= document.numPages) result.add(Math.floor(index));
}
}
const values = [...result].sort((left, right) => left - right);
if (!values.length) throw new Error(`PDF 页码范围无效:${range}`);
return values;
}
async function renderPage(page: Awaited<ReturnType<PdfJsDocument["getPage"]>>, scale = 2): Promise<Buffer> {
const canvasModule = await import("@napi-rs/canvas");
const viewport = page.getViewport({ scale });
const canvas = canvasModule.createCanvas(Math.ceil(viewport.width), Math.ceil(viewport.height));
const context = canvas.getContext("2d");
context.fillStyle = "white"; context.fillRect(0, 0, canvas.width, canvas.height);
await page.render({ canvasContext: context as unknown as CanvasRenderingContext2D, viewport }).promise;
return canvas.toBuffer("image/png");
}
function saveImage(folder: string, prefix: string, suffix: string, data: Buffer, type: string): string {
const ext = /jpe?g/i.test(type) ? "jpg" : "png";
const output = path.join(path.resolve(folder), `${prefix}_${suffix}_${Date.now()}.${ext}`);
fsSync.mkdirSync(path.dirname(output), { recursive: true });
fsSync.writeFileSync(output, data);
return output;
}
async function rawImageToBuffer(image: Record<string, unknown>, type: string): Promise<Buffer> {
const width = Number(image.width || 0); const height = Number(image.height || 0);
const data = image.data;
if (!width || !height || !(data instanceof Uint8Array) || data.length === 0) throw new Error("PDF 图片数据无效");
if (data[0] === 0xff && data[1] === 0xd8) return Buffer.from(data);
const canvasModule = await import("@napi-rs/canvas");
const canvas = canvasModule.createCanvas(width, height); const context = canvas.getContext("2d");
const target = context.createImageData(width, height); const kind = Number(image.kind || 0);
if (kind === 1) {
const bytesPerRow = Math.ceil(width / 8);
for (let y = 0; y < height; y += 1) for (let x = 0; x < width; x += 1) {
const bit = (data[y * bytesPerRow + (x >> 3)] >> (7 - (x & 7))) & 1; const output = (y * width + x) * 4; const color = bit ? 255 : 0;
target.data[output] = color; target.data[output + 1] = color; target.data[output + 2] = color; target.data[output + 3] = 255;
}
} else if (kind === 3) { target.data.set(data.subarray(0, Math.min(data.length, target.data.length))); }
else {
for (let source = 0, targetIndex = 0; source < data.length && targetIndex < target.data.length; source += 3, targetIndex += 4) {
target.data[targetIndex] = data[source]; target.data[targetIndex + 1] = data[source + 1] ?? 0; target.data[targetIndex + 2] = data[source + 2] ?? 0; target.data[targetIndex + 3] = 255;
}
}
context.putImageData(target, 0, 0);
return /jpe?g/i.test(type) ? canvas.toBuffer("image/jpeg") : canvas.toBuffer("image/png");
}
async function objectImage(page: Awaited<ReturnType<PdfJsDocument["getPage"]>>, objId: unknown): Promise<Record<string, unknown> | null> {
if (typeof objId !== "string") return null;
const objects = objId.startsWith("g_") ? page.commonObjs : page.objs;
try { if (objects.has(objId)) return objects.get(objId) as Record<string, unknown>; } catch {}
return await new Promise((resolve) => { try { objects.get(objId, (value: unknown) => resolve(value as Record<string, unknown>)); } catch { resolve(null); } });
}
const encryption: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const password = String(args.passWord ?? args.password ?? "");
if (!password) throw new RuntimeError("validation_error", "PDF密码为空", { nodeId: node.id, componentId: node.componentId });
const { PDFDocument } = await import("@cantoo/pdf-lib");
const document = await PDFDocument.load(await fs.readFile(filePath), { ignoreEncryption: true });
document.encrypt({ userPassword: password, ownerPassword: password, permissions: { printing: "highResolution", copying: true, modifying: false } });
const temporary = `${filePath}.${Date.now()}.enc`;
await fs.writeFile(temporary, await document.save());
await fs.rename(temporary, filePath);
return filePath;
};
const extractImages: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const folder = required(text(args, ["imgSavePath", "pdfSavePath", "folder", "path"]), "图片保存路径", node.id, node.componentId);
const type = text(args, ["imageType", "type"], "png");
const prefix = text(args, ["imageName", "name"], "pdf-image");
const { pdfjs, document } = await loadPdf(filePath, String(args.pdfPassWord ?? args.password ?? ""));
const outputs: string[] = [];
for (const pageNumber of pageIndices(document, args.pdfNum)) {
const page = await document.getPage(pageNumber);
const opList = await page.getOperatorList();
let sequence = 0;
for (let index = 0; index < opList.fnArray.length; index += 1) {
const op = opList.fnArray[index]; const commandArgs = opList.argsArray[index] as unknown[];
let image: Record<string, unknown> | null = null;
if (op === pdfjs.OPS.paintImageXObject || op === pdfjs.OPS.paintImageXObjectRepeat) image = await objectImage(page, commandArgs[0]);
else if (op === pdfjs.OPS.paintInlineImageXObject) image = commandArgs[0] as Record<string, unknown>;
else if (op === pdfjs.OPS.paintInlineImageXObjectGroup) image = commandArgs[0] as Record<string, unknown>;
if (!image) continue;
sequence += 1;
outputs.push(saveImage(folder, `${prefix}-p${pageNumber}`, String(sequence), await rawImageToBuffer(image, type), type));
}
}
return outputs;
};
const pdfToImages: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const folder = required(text(args, ["pdfSavePath", "imgSavePath", "folder", "path"]), "图片保存路径", node.id, node.componentId);
const type = text(args, ["imageType", "type"], "png"); const prefix = text(args, ["imageName", "name"], "pdf-page");
const { document } = await loadPdf(filePath, String(args.pdfPassWord ?? args.password ?? ""));
const outputs: string[] = [];
for (const pageNumber of pageIndices(document, args.pdfNum)) {
const buffer = await renderPage(await document.getPage(pageNumber));
outputs.push(saveImage(folder, prefix, String(pageNumber), buffer, type));
}
return outputs;
};
interface TextToken { x: number; y: number; value: string; }
async function extractTableRows(page: Awaited<ReturnType<PdfJsDocument["getPage"]>>): Promise<string[][]> {
const content = await page.getTextContent();
const tokens = content.items.flatMap((item) => {
if (!("str" in item) || !item.str.trim()) return [];
const transform = (item as { transform?: number[] }).transform ?? [1,0,0,1,0,0];
return [{ x: Number(transform[4] || 0), y: Number(transform[5] || 0), value: item.str.trim() }];
});
if (tokens.length < 2) return [];
const lines = new Map<number, TextToken[]>();
for (const token of [...tokens].sort((left, right) => right.y - left.y || left.x - right.x)) {
const lineKey = [...lines.keys()].find((key) => Math.abs(key - token.y) <= 3.5) ?? token.y;
if (!lines.has(lineKey)) lines.set(lineKey, []);
lines.get(lineKey)!.push(token);
}
const lineList = [...lines.entries()].map(([y, values]) => ({ y, values: values.sort((left, right) => left.x - right.x) })).sort((left, right) => right.y - left.y);
const counts = new Map<number, number>();
for (const line of lineList) for (const token of line.values) {
const key = Math.round(token.x / 5) * 5;
counts.set(key, (counts.get(key) || 0) + 1);
}
const columns = [...counts.entries()].filter(([, count]) => count >= Math.min(2, lineList.length)).sort((left, right) => left[0] - right[0]).map(([key]) => key);
if (columns.length < 2) return [];
const rows: string[][] = [];
for (const line of lineList) {
const cells = columns.map(() => "");
for (const token of line.values) {
let nearest = 0;
for (let index = 1; index < columns.length; index += 1) if (Math.abs(token.x - columns[index]) < Math.abs(token.x - columns[nearest])) nearest = index;
cells[nearest] = cells[nearest] ? `${cells[nearest]} ${token.value}` : token.value;
}
if (cells.filter(Boolean).length >= 2) rows.push(cells);
}
return rows.length >= 2 ? rows : [];
}
const readTables: CapabilityHandler = async (_ctx, node, args) => {
const filePath = required(text(args, ["fileInPath", "path", "file"]), "PDF文件路径", node.id, node.componentId);
const { document } = await loadPdf(filePath, String(args.pdfPassWord ?? args.password ?? ""));
const rows: string[][] = [];
for (const pageNumber of pageIndices(document, args.pdfNum)) rows.push(...await extractTableRows(await document.getPage(pageNumber)));
return rows;
};
register("PDF_PdfEncryption", encryption);
register("PDF_GetPdfImg", extractImages);
register("PDF_PdfToImg", pdfToImages);
register("PDF_GetPdfTable", readTables);
@@ -0,0 +1,35 @@
import { describe, expect, it } from "vitest";
import { createEngine } from "../engine/engine";
import { capabilityCount, capabilityIds, getCapabilityMeta } from "../engine/registry";
import { isKnownErrorCode } from "../engine/error-codes";
import { capabilityDomains, registerAll } from "./register-all";
import "./index";
describe("registerAll 显式注册入口", () => {
it("副作用入口已注册全量包内能力,registerAll 全量幂等", async () => {
// 包内 capabilities/index 注册 719 个 id(909 快照 = 包内 + 宿主浏览器能力)
const before = capabilityCount();
expect(before).toBeGreaterThan(700);
const engine = await registerAll(createEngine());
expect(engine).toBeDefined();
expect(capabilityCount()).toBe(before);
});
it("按子集注册:只拉起指定域,未知域报错", async () => {
expect(capabilityDomains).toContain("excelCom");
await expect(registerAll(createEngine(), ["excelCom", "no-such-domain"])).rejects.toThrow("未知能力域");
await registerAll(createEngine(), ["files"]);
expect(getCapabilityMeta).toBeDefined();
});
it("meta(任务 4.1):老能力允许缺省;已声明 meta 的能力错误码必须在稳定清单内", () => {
// 全量注册后的每个 meta,其 errorCodes 都必须落在 RUNTIME_ERROR_CODES 清单内
for (const id of capabilityIds()) {
const meta = getCapabilityMeta(id);
if (!meta) continue;
for (const code of meta.errorCodes ?? []) {
expect(isKnownErrorCode(code), `${id} 的错误码 ${code} 未登记 error-codes.ts`).toBe(true);
}
}
});
});
@@ -0,0 +1,59 @@
import type { RpaEngine } from "../engine/engine";
/**
* 显式注册入口(任务 4.2,决策点 9:双入口过渡)。
*
* - 旧入口:`import "@rpa/core/capabilities"` 副作用注册全量(宿主/测试继续使用);
* - 新入口:`await registerAll(engine, groups?)` 按 subset 显式拉起,供
* Agent-Employee 等宿主按需注册。阶段 6 迁移时切显式入口、删副作用入口。
*
* 引擎当前持有全局注册表(阶段 1 决策 2),拉起模块即完成对全局表的注册,
* 返回 engine 以便链式使用。
*/
const DOMAINS: Record<string, () => Promise<unknown>> = {
control: () => import("./control"),
files: () => import("./files"),
http: () => import("./http"),
data: () => import("./data"),
office: () => import("./office"),
desktop: () => import("./desktop"),
desktopUia: () => import("./desktop-uia"),
time: () => import("./time-caps"),
windowsSystem: () => import("./windows-system"),
ftp: () => import("./ftp"),
ocr: () => import("./ocr-caps"),
pdf: () => import("./pdf"),
word: () => import("./word"),
wordExtended: () => import("./word-extended"),
message: () => import("./message"),
vision: () => import("./vision"),
cvVision: () => import("./cv-vision"),
extended: () => import("./extended"),
systemExtended: () => import("./system-extended"),
officeFileExtended: () => import("./office-file-extended"),
officeAliases: () => import("./office-aliases"),
excelCommon: () => import("./excel-common"),
excelCom: () => import("./excel-com"),
dataExtended: () => import("./data-extended"),
utilityExtended: () => import("./utility-extended"),
dataTableCommon: () => import("./data-table-common"),
basicEncryption: () => import("./basic-encryption"),
mailRemaining: () => import("./mail-remaining"),
remainingCore: () => import("./remaining-core"),
commonFinal: () => import("./common-final"),
humanDialog: () => import("./human-dialog"),
aiService: () => import("./ai-service"),
windowExtended: () => import("./window-extended")
};
export async function registerAll(engine: RpaEngine, groups?: string[]): Promise<RpaEngine> {
const wanted = groups && groups.length ? groups : Object.keys(DOMAINS);
for (const group of wanted) {
const load = DOMAINS[group];
if (!load) throw new Error(`未知能力域:${group}(可选:${Object.keys(DOMAINS).join(", ")})`);
await load();
}
return engine;
}
export const capabilityDomains = Object.keys(DOMAINS);
@@ -0,0 +1,59 @@
import { existsSync } from "node:fs";
import { describe, expect, it } from "vitest";
import { getHandler } from "../engine/registry";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import "./vision";
import "./remaining-core";
const ctx = {
signal: new AbortController().signal,
log: () => {},
vars: {},
evidenceDir: "D:/office-test2/ocr-test"
} as unknown as RunContext;
const node = { id: "smoke-rp", componentId: "Image_RelativePosition", position: { x: 0, y: 0 }, values: {}, options: {} };
const relativePositionFixtures = [
"D:/office-test2/ocr-test/general.png",
"D:/office-test2/ocr-test/order-template.png",
];
const relativePositionFixturesReady = relativePositionFixtures.every((file) => existsSync(file));
describe.skipIf(!relativePositionFixturesReady)("对比图像相对位置冒烟", () => {
it("小图在大图中命中并返回坐标", async () => {
const handler = getHandler("Image_RelativePosition");
expect(handler).toBeTruthy();
const out = await handler!(
ctx,
node,
{
source: "D:/office-test2/ocr-test/general.png",
template: "D:/office-test2/ocr-test/order-template.png"
},
{} as CatalogComponent
);
console.log("[RelativePosition 正向]", JSON.stringify(out));
const hit = out as { x: number; y: number; width: number; height: number; score: number };
expect(hit.x).toBeGreaterThanOrEqual(0);
expect(hit.y).toBeGreaterThan(0);
expect(hit.score).toBeGreaterThan(0.9);
}, 120000);
it("填反(大图填进查找目标)自动互换后仍命中", async () => {
const logs: string[] = [];
const ctxWithLog = { ...ctx, log: (m: string) => logs.push(m) } as unknown as RunContext;
const handler = getHandler("Image_RelativePosition")!;
const out = await handler!(
ctxWithLog,
node,
{
source: "D:/office-test2/ocr-test/order-template.png",
template: "D:/office-test2/ocr-test/general.png"
},
{} as CatalogComponent
);
console.log("[RelativePosition 互换]", JSON.stringify(out), "logs:", logs.join(" | "));
expect(out).toBeTruthy();
}, 120000);
});
@@ -0,0 +1,272 @@
import fs from "node:fs";
import path from "node:path";
import { getHandler, register } from "../engine/registry";
import { runFlow } from "../engine/runtime";
import { RuntimeError, type CapabilityHandler, type RunContext } from "../engine/types";
import { defaultEngine } from "../engine/engine";
import { getSubflowSource } from "../engine/host-bridge";
import { powershell, psString } from "../driver/windows/powershell";
import { findImage } from "./vision";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) { const value = args[key]; if (value !== undefined && value !== null && value !== "") return String(value); }
return fallback;
}
function num(args: Record<string, unknown>, keys: string[], fallback = 0): number { const value = Number(text(args, keys, String(fallback))); return Number.isFinite(value) ? value : fallback; }
function bool(value: unknown, fallback = false): boolean { if (value === undefined || value === null || value === "") return fallback; return [true, 1, "1", "true", "True", "是", "开启"].includes(value as never); }
function list(value: unknown): string[] {
if (Array.isArray(value)) return value.map(String);
const raw = String(value ?? "").trim(); if (!raw) return [];
try { const parsed = JSON.parse(raw); return Array.isArray(parsed) ? parsed.map(String) : [String(parsed)]; }
catch { return raw.split(/[,,;;\n]+/).map((item) => item.trim()).filter(Boolean); }
}
function record(value: unknown): Record<string, unknown> { return value && typeof value === "object" ? value as Record<string, unknown> : (() => { try { return JSON.parse(String(value || "{}")); } catch { return {}; } })(); }
function required(value: string, label: string, node: { id: string; componentId: string }): string {
if (!value.trim()) throw new RuntimeError("validation_error", label + "为空", { nodeId: node.id, componentId: node.componentId });
return value;
}
// 连接 id → 数据库连接;本体挂在引擎实例上(任务 1.3)。
const dbConnections = defaultEngine.dbConnections as Map<string, any>;
function dbConfig(args: Record<string, unknown>): Record<string, unknown> {
const inline = record(args.DatabaseConfig || args.config || args.connectionID);
const merged = { ...inline, ...args };
return { ...inline, type: text(merged, ["dtype", "type"], "mysql"), host: text(merged, ["host", "server", "server_address"], "127.0.0.1"), port: num(merged, ["port"], 0), user: text(merged, ["user", "username", "account"], ""), password: text(merged, ["password", "pass"], ""), database: text(merged, ["database", "db", "catalog"], "") };
}
async function databaseClient(config: Record<string, unknown>) {
const type = String(config.type || "mysql").toLowerCase();
if (type.includes("mysql")) { const mysql = await import("mysql2/promise"); return await mysql.createConnection(config as never); }
if (type.includes("postgres") || type.includes("pgsql")) { const { Client } = await import("pg"); const client = new Client({ host: String(config.host), port: Number(config.port || 5432), user: String(config.user), password: String(config.password), database: String(config.database) }); await client.connect(); return client; }
if (type.includes("sqlserver") || type.includes("mssql")) { const sql = await import("mssql"); return await sql.connect(config as never); }
throw new RuntimeError("validation_error", "暂不支持的数据库类型:" + type);
}
async function rowsOf(client: any, sql: string, params: string[]) {
const result = await client.query(sql, params);
return result.rows || result.recordset || [];
}
async function runDb(args: Record<string, unknown>, sql: string) {
const key = String(args.connectionID || "inline");
let client = dbConnections.get(key);
if (!client) { client = await databaseClient(dbConfig(args)); dbConnections.set(key, client); }
return await rowsOf(client, sql, list(args.paras));
}
register("Database_DBConnect", async (_ctx, _node, args) => {
const id = "db_" + Date.now().toString(36); const client = await databaseClient(dbConfig(args)); dbConnections.set(id, client);
return { connectionID: id };
});
register(["Database_SingleSQLQuery", "Database_AllSQLQuery", "Database_AllSQLQuery_V1"], async (_ctx, node, args) => runDb(args, required(text(args, ["sql"]), "SQL", node)));
register(["Database_SQLExecute", "Database_BatchSQLExecute"], async (_ctx, node, args) => {
const statements = list(args.sql).length ? list(args.sql) : [required(text(args, ["sql"]), "SQL", node)];
for (const statement of statements) await runDb(args, statement); return { executed: statements.length };
});
register("Database_BatchInsertExecute", async (_ctx, node, args) => {
const parsed = record(args.data); const rows = Array.isArray(parsed) ? parsed : Array.isArray(parsed.value) ? parsed.value : [];
for (const row of rows) await runDb(args, required(text(args, ["sql"]), "SQL", node)); return { inserted: rows.length };
});
register("Database_CloseDBConnect", async (_ctx, _node, args) => {
const key = String(args.connectionID || ""); const client = dbConnections.get(key);
if (client) { if (client.close) await client.close(); if (client.end) await client.end(); }
dbConnections.delete(key); return { closed: true };
});
async function callSubflow(ctx: RunContext, node: { id: string; componentId: string }, args: Record<string, unknown>) {
// 子流程仓库属宿主职责:经 host-bridge 注入(阶段 2 正式化),包内不 import "../flows"
const source = getSubflowSource();
if (!source) throw new RuntimeError("validation_error", "子流程源未注入", { nodeId: node.id, componentId: node.componentId, recommendation: "宿主需在启动时调用 setSubflowSource()" });
const wanted = text(args, ["module_name", "module", "flow"]);
const rows = await source.listFlows(); const found = rows.find((row) => row.id === wanted || row.name === wanted);
if (!found) throw new RuntimeError("validation_error", "子流程不存在:" + wanted, { nodeId: node.id, componentId: node.componentId });
const { randomUUID } = await import("node:crypto");
const child = await source.getFlow(found.id); const inputs = record(args.args || args.input);
return await runFlow(child, { runId: randomUUID(), evidenceDir: ctx.evidenceDir, signal: ctx.signal, emit: ctx.emit, log: ctx.log, componentOf: ctx.componentOf, browser: ctx.browser, variables: { ...ctx.vars, ...inputs } });
}
register("child_modules_v1", async (ctx, node, args) => callSubflow(ctx, node, args));
register("concurrent_child_process_v1", async (ctx, node, args) => {
const tasks = Array.from({ length: Math.max(1, num(args, ["count"], 2)) }, () => callSubflow(ctx, node, args));
return bool(args.is_async) ? { started: true } : await Promise.all(tasks);
});
register("Element_exception_monitor_v2", async (ctx, node, args) => {
ctx.log("异常监控已启用:" + text(args, ["event_type", "monitor_type"], "element"), bool(args.is_exits) ? "info" : "warn", node.id);
return { monitoring: true };
});
register("FileTypeConversion_Office2pdf", async (_ctx, node, args) => {
const input = path.resolve(text(args, ["fileInPath", "path"])); const folder = path.resolve(text(args, ["fileOutPath", "folder"], path.dirname(input)));
fs.mkdirSync(folder, { recursive: true }); const output = path.join(folder, path.basename(input, path.extname(input)) + ".pdf");
const ext = path.extname(input).toLowerCase();
if ([".doc", ".docx", ".rtf"].includes(ext)) await powershell(`$w=New-Object -ComObject Word.Application;$w.Visible=$false;$d=$w.Documents.Open(${psString(input)});$d.SaveAs([ref]${psString(output)},[ref]17);$d.Close($false);$w.Quit()`);
else if ([".xls", ".xlsx", ".csv"].includes(ext)) await powershell(`$e=New-Object -ComObject Excel.Application;$e.Visible=$false;$w=$e.Workbooks.Open(${psString(input)});$w.ExportAsFixedFormat(0,${psString(output)});$w.Close($false);$e.Quit()`);
else if ([".ppt", ".pptx"].includes(ext)) await powershell(`$p=New-Object -ComObject PowerPoint.Application;$pre=$p.Presentations.Open(${psString(input)},$true,$false,$false);$pre.SaveAs(${psString(output)},32);$pre.Close()`);
else throw new RuntimeError("validation_error", "不支持的 Office 文件:" + ext);
return output;
});
register("FileTypeConversion_Pdf2image2", async (ctx, node, args) => {
const pdf = path.resolve(text(args, ["pdfPath", "path"])); const folder = path.resolve(text(args, ["imageDir", "folder"], path.dirname(pdf)));
const handler = getHandler("PDF_PdfToImg"); if (!handler) throw new RuntimeError("capability_not_implemented", "PDF 渲染能力不可用");
return await handler(ctx, node, { ...args, fileInPath: pdf, pdfSavePath: folder, imageName: path.basename(pdf, path.extname(pdf)), imageType: "png" }, {} as never);
});
function sheetId(args: Record<string, unknown>): string {
const url = text(args, ["url", "spreadsheetUrl"]); const match = url.match(/\/spreadsheets\/d\/([a-zA-Z0-9-_]+)/);
if (!match) throw new RuntimeError("validation_error", "无效 Google Sheets URL:" + url); return match[1];
}
function sheetRange(args: Record<string, unknown>): string {
const sheet = encodeURIComponent(text(args, ["sheetName"], "Sheet1"));
const start = text(args, ["startPosition", "start"], "A1"); const end = text(args, ["endPosition", "end"], "Z1000");
return `${sheet}!${start}:${end}`;
}
async function googleRequest(args: Record<string, unknown>, pathUrl: string, init: RequestInit = {}) {
const id = sheetId(args); const token = text(args, ["accessToken", "access_token", "token"]);
const key = text(args, ["apiKey", "api_key", "key"]); const headers: Record<string,string> = {};
if (token) headers.Authorization = "Bearer " + token;
const separator = pathUrl.includes("?") ? "&" : "?";
const response = await fetch(`https://sheets.googleapis.com/v4/spreadsheets/${id}${pathUrl}${key ? separator + "key=" + key : ""}`, { ...init, headers: { ...headers, ...(init.headers || {}) } });
if (!response.ok) throw new RuntimeError("external_service_failed", "Google Sheets API " + response.status + ": " + await response.text());
return await response.json();
}
register("GoogleTable_GetAllData", async (_ctx, node, args) => { const out = await googleRequest(args, `/values/${sheetRange(args)}?majorDimension=ROWS`); return out.values || []; });
register("GoogleTable_GetAllSheetsName", async (_ctx, _node, args) => { const out = await googleRequest(args, "?fields=sheets.properties.title"); return (out.sheets || []).map((sheet: any) => sheet.properties.title); });
async function googleWrite(args: Record<string, unknown>) {
if (!text(args, ["accessToken", "access_token", "token"])) throw new RuntimeError("validation_error", "Google Sheets 写入需要 accessToken");
return await googleRequest(args, `/values/${sheetRange(args)}?valueInputOption=USER_ENTERED`, { method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ values: (() => { const value = args.Values || args.values; if (Array.isArray(value)) return value.map((row) => Array.isArray(row) ? row : [row]); try { const parsed = JSON.parse(String(value || "[]")); return Array.isArray(parsed) ? parsed.map((row: any) => Array.isArray(row) ? row : [row]) : [[parsed]]; } catch { return [[value]]; } })() }) });
}
for (const id of ["GoogleTable_WriteToSheetRegionColumns", "GoogleTable_WriteToSheetRegionRows", "GoogleTable_WriteToSheetRegionregion"]) register(id, async (_ctx, _node, args) => googleWrite(args));
function googleCell(args: Record<string, unknown>) { return `${encodeURIComponent(text(args, ["sheetName"], "Sheet1"))}!${text(args, ["cellName", "cell"], "A1")}`; }
register("GoogleTable_WriteToSheetCell", async (_ctx, _node, args) => googleRequest(args, `/values/${googleCell(args)}?valueInputOption=RAW`, { method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ values: [[args.value]] }) }));
register("GoogleTable_WriteToSheetCellFormula", async (_ctx, _node, args) => {
const value = String(args.value || ""); return googleRequest(args, `/values/${googleCell(args)}?valueInputOption=USER_ENTERED`, { method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ values: [[value.startsWith("=") ? value : "=" + value]] }) });
});
register("GoogleTable_GetSheetRowCount", async (_ctx, _node, args) => { const out = await googleRequest(args, `/values/${encodeURIComponent(text(args, ["sheetName"], "Sheet1"))}`); return (out.values || []).length; });
register("GoogleTable_DownloadSheets", async (_ctx, _node, args) => {
const format = /xlsx/i.test(text(args, ["format"], "csv")) ? "xlsx" : "csv";
const response = await fetch(`https://docs.google.com/spreadsheets/d/${sheetId(args)}/export?format=${format}`);
if (!response.ok) throw new RuntimeError("external_service_failed", "Google 导出失败:" + response.status);
const file = path.resolve(process.cwd(), `google-sheet-${Date.now()}.${format}`);
fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, Buffer.from(await response.arrayBuffer())); return file;
});
register("GoogleTable_CopyPasteSheet", async (_ctx, _node, args) => {
const source = `${encodeURIComponent(text(args, ["sourceSheetName"], "Sheet1"))}!${text(args, ["sourceStart"], "A1")}:${text(args, ["sourceEnd"], "Z1000")}`;
const target = `${encodeURIComponent(text(args, ["destSheetName"], "Copy"))}!${text(args, ["destStart"], "A1")}:${text(args, ["destEnd"], "Z1000")}`;
const values = await googleRequest(args, `/values/${source}`);
return await googleRequest(args, `/values/${target}?valueInputOption=USER_ENTERED`, { method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ values: values.values || [] }) });
});
register("GoogleTable_GetColumnsByTableTitle", async (_ctx, _node, args) => {
const titles = list(args.tableTitles); const values = await googleRequest(args, `/values/${encodeURIComponent(text(args, ["sheetName"], "Sheet1"))}`);
const row = (values.values || [])[num(args, ["rowNum"], 0)] || [];
return titles.map((title) => ({ title, value: row[Number(title)] ?? "" }));
});
register("GoogleTable_ChangeSheet", async (_ctx, _node, args) => {
const out = await googleRequest(args, "?fields=sheets.properties"); const wanted = text(args, ["sheetName"], "Sheet1");
const sheet = (out.sheets || []).find((item: any) => item.properties.title === wanted); if (!sheet) throw new RuntimeError("not_found", "Google Sheet 不存在:" + wanted); return sheet.properties;
});
register("GoogleTable_RenameSheet", async (_ctx, _node, args) => {
const out = await googleRequest(args, "?fields=sheets.properties.sheetId"); const oldName = text(args, ["originalSheetName"]);
const sheet = (out.sheets || []).find((item: any) => item.properties.title === oldName); if (!sheet) throw new RuntimeError("not_found", "Google Sheet 不存在:" + oldName);
return await googleRequest(args, ":batchUpdate", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ requests: [{ updateSheet: { properties: { sheetId: sheet.properties.sheetId, title: text(args, ["modifySheetName"]) }, fields: "title" } }] }) });
});
// Image_Exists / Image_Location / Image_Wait / Image_Move 统一由 vision.ts 注册到新查找管线
// (虚拟桌面多屏拼接 + 相似度语义 + JPG 模板)。此处旧实现用 GDI CopyFromScreen 只截主屏
// 且 accuracy 语义颠倒, 已整体迁出删除。
async function captureScreen(file: string) {
fs.mkdirSync(path.dirname(file), { recursive: true });
await powershell(`Add-Type -AssemblyName System.Drawing;Add-Type -AssemblyName System.Windows.Forms;$bounds=[System.Windows.Forms.Screen]::PrimaryScreen.Bounds;$bitmap=New-Object System.Drawing.Bitmap $bounds.Width,$bounds.Height;$graphics=[System.Drawing.Graphics]::FromImage($bitmap);$graphics.CopyFromScreen($bounds.Location,[System.Drawing.Point]::Empty,$bounds.Size);$bitmap.Save(${psString(file)});$graphics.Dispose();$bitmap.Dispose()`);
}
// 图片Base64编解码: 统一组件按 mode 分流(encode=图片→Base64, decode=Base64→图片)。
// Image_Base64Decode 已隐藏进该组件,但保留注册以防旧流程直接引用旧 id。
register("Image_Base64Encode", async (_ctx, node, args) => {
const mode = text(args, ["mode"], "encode");
if (mode === "decode") {
const value = text(args, ["val", "value"]).replace(/^data:[^;]+;base64,/, "");
if (!value.trim()) throw new RuntimeError("validation_error", "Base64编码不能为空", { nodeId: node.id, componentId: node.componentId, recommendation: "填写 base64 字符串,或用 ${变量} 引用上游编码输出" });
const custom = text(args, ["output"]).trim();
const file = custom ? path.resolve(custom) : path.resolve(process.cwd(), `image-${Date.now()}.png`);
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, Buffer.from(value, "base64"));
return file;
}
const imageArg = text(args, ["path", "image"]);
if (!imageArg.trim()) throw new RuntimeError("validation_error", "图片对象不能为空", { nodeId: node.id, componentId: node.componentId, recommendation: "填写源图片路径,或用 ${变量} 引用上游截图输出" });
const source = path.resolve(imageArg);
if (!fs.existsSync(source)) throw new RuntimeError("validation_error", `找不到图片文件:${source}`, { nodeId: node.id, componentId: node.componentId, recommendation: "确认路径正确,或先用「屏幕截图」组件生成图片" });
return fs.readFileSync(source).toString("base64");
});
register("Image_Base64Decode", async (_ctx, _node, args) => {
const value = text(args, ["val", "value"]).replace(/^data:[^;]+;base64,/, ""); const file = path.resolve(process.cwd(), `image-${Date.now()}.png`);
fs.mkdirSync(path.dirname(file), { recursive: true }); fs.writeFileSync(file, Buffer.from(value, "base64")); return file;
});
register("Image_CropImg", async (_ctx, node, args) => {
const rawInput = text(args, ["file_path", "imagePath", "path"]);
if (!rawInput.trim()) throw new RuntimeError("validation_error", "图片地址不能为空", { nodeId: node.id, componentId: node.componentId, recommendation: "填写要裁剪的源图片完整路径(如 D:\\images\\shot.png),该组件是从已有图片文件里裁剪区域,不是截屏" });
const input = path.resolve(rawInput);
if (!fs.existsSync(input)) throw new RuntimeError("validation_error", `找不到图片文件:${input}`, { nodeId: node.id, componentId: node.componentId, recommendation: "确认路径正确;若要截取屏幕内容,请改用「截图图像 (Image_Screenshot)」组件" });
const output = path.resolve(text(args, ["output"], input.replace(/(\.[^.]+)$/, "-crop$1")));
const x = num(args, ["xstart"], 0), y = num(args, ["ystart"], 0), width = num(args, ["xend"], 100) - x, height = num(args, ["yend"], 100) - y;
if (width <= 0 || height <= 0) throw new RuntimeError("validation_error", `裁剪区域无效(宽 ${width} × 高 ${height})`, { nodeId: node.id, componentId: node.componentId, recommendation: "终点坐标需要大于起点坐标(单位是图片上的像素)" });
fs.mkdirSync(path.dirname(output), { recursive: true });
// 注意: .NET 方法的参数表里只能放表达式, 不能放 New-Object 语句——
// DrawImage($img, New-Object Rectangle(...), ...) 会解析报错"','后面缺少表达式"(组件此前从未跑通的原因);
// 统一改用 [Type]::new() 表达式写法
await powershell(`Add-Type -AssemblyName System.Drawing;$image=[System.Drawing.Image]::FromFile(${psString(input)});$rect=[System.Drawing.Rectangle]::new(${x},${y},${width},${height});$bitmap=[System.Drawing.Bitmap]::new($rect.Width,$rect.Height);$graphics=[System.Drawing.Graphics]::FromImage($bitmap);$graphics.DrawImage($image,[System.Drawing.Rectangle]::new(0,0,$rect.Width,$rect.Height),$rect,[System.Drawing.GraphicsUnit]::Pixel);$bitmap.Save(${psString(output)});$graphics.Dispose();$bitmap.Dispose();$image.Dispose()`);
return output;
});
register("Image_RelativePosition", async (ctx, node, args) => {
const source = path.resolve(text(args, ["source", "image"]));
const template = path.resolve(text(args, ["template", "template_path"]));
if (!source.trim() || !template.trim()) throw new RuntimeError("validation_error", "被搜索图像与查找目标都需填写图片路径", { nodeId: node.id, componentId: node.componentId });
if (!fs.existsSync(source)) throw new RuntimeError("validation_error", `找不到被搜索图像:${source}`, { nodeId: node.id, componentId: node.componentId });
if (!fs.existsSync(template)) throw new RuntimeError("validation_error", `找不到查找目标图片:${template}`, { nodeId: node.id, componentId: node.componentId });
// 兼容填反:模板比源图大时(首次查找直接抛"还大"错误)自动互换,
// 大图当被搜索图、小图当查找目标,语义仍是「小图在大图中的位置」
const accuracy = num(args, ["accuracy"], 0.2);
let hit: Awaited<ReturnType<typeof findImage>> = null;
let firstError: unknown = null;
try {
hit = await findImage(template, source, accuracy);
} catch (error) {
firstError = error;
try {
hit = await findImage(source, template, accuracy);
if (hit) ctx.log("两张图疑似填反,已自动按「小图在大图中」查找并命中", "warn", node.id);
} catch {
// 互换也失败则回落到原始方向的错误信息
}
}
if (!hit) {
// 互换也没命中:两张图内容互不包含(或真的不存在),给出综合诊断而不是误导性的尺寸错误
throw new RuntimeError("not_found", "查找目标未出现在被搜索图像中(已自动尝试反向查找仍未命中,两张图内容互不包含)", {
nodeId: node.id,
componentId: node.componentId,
detail: firstError instanceof Error ? firstError.message : String(firstError ?? ""),
recommendation: "「被搜索图像」应是较大的底图(如整屏/整窗截图),「查找目标」应是底图中实际出现的小图案(按钮/图标/文字块)的截图;两张完全无关的图无法匹配"
});
}
return hit;
});
register("logic_control_if_ai_vision", async (ctx, node, args) => {
const screenshot = path.join(ctx.evidenceDir, `ai-vision-${Date.now()}.png`); await captureScreen(screenshot);
return { image: screenshot, prompt: text(args, ["input_data", "prompt", "condition"], "描述屏幕内容并判断条件是否成立"), matched: true };
});
// CV_GetText / CV_SetText / CV_MouseHover / CV_MouseClick 统一由 cv-vision.ts 注册:
// 此处曾重复注册并覆盖 cv-vision 的实现,其中内联 here-string(@'...'@ 单行)是
// 非法 PowerShell 语法,导致 CV_MouseClick/CV_MouseHover 必然解析失败(报 CLIXML 错误)。
// 如需调整 CV 行为请修改 cv-vision.ts,不要在此重复注册。
async function xunfeiRequest(args: Record<string, unknown>, operation: string, payload: Record<string, unknown>) {
const endpoint = text(args, ["endpoint"]); if (!endpoint) throw new RuntimeError("validation_error", "需要配置讯飞 API endpoint");
const response = await fetch(endpoint, { method: "POST", headers: { "Content-Type": "application/json", "X-Appid": text(args, ["APPID", "appId"]), Authorization: "Bearer " + text(args, ["APIKey", "apiKey"]) }, body: JSON.stringify({ operation, ...payload }) });
if (!response.ok) throw new RuntimeError("external_service_failed", "讯飞 API " + response.status + ": " + await response.text()); return await response.json();
}
register("NLP_XunFei_MachineTranslation", async (_ctx, _node, args) => xunfeiRequest(args, "translation", { text: text(args, ["text"]), from: text(args, ["BeforLanguage"], "auto"), to: text(args, ["AfterLanguage"], "en") }));
register("NLP_XunFei_FeelingAnalysis", async (_ctx, _node, args) => xunfeiRequest(args, "sentiment", { text: text(args, ["text"]) }));
register("NLP_XunFei_KeywordExtraction", async (_ctx, _node, args) => xunfeiRequest(args, "keyword", { text: text(args, ["text"]) }));
async function ocrImage(args: Record<string, unknown>, node: { id: string; componentId: string }) {
const image = path.resolve(text(args, ["path", "image_url", "data", "pictureSource", "image_path", "image"]));
if (!fs.existsSync(image)) throw new RuntimeError("validation_error", "OCR 图片不存在", { nodeId: node.id, componentId: node.componentId, detail: image });
const module = await import("@gutenye/ocr-node"); const ocr = await module.default.create(); const result = await ocr.detect(image);
return (result.texts || []).map((line: any) => line.text).join("\n");
}
const ocrIds = ["OCR_MergeCardOCR", "OCR_CalculationCode", "OCR_IdCardV2", "OCR_BusinessCardV2", "OCR_VehicleInvoiceV2", "OCR_MergeOtherOCR", "OCR_BusinessLicenseV3", "OCR_PassportV2", "OCR_MergeGeneralOCR", "OCR_KeyValueIdentification", "OCR_AmountIdentification", "OCR_GeneralTextV2", "OCR_VatInvoiceV2", "OCR_DrivingLicenseV2", "OCR_AccountPageV2", "OCR_PlateLicenseV2", "OCR_BankCardV2", "OCR_BankCardV3", "OCR_IdCardV3", "OCR_VatInvoiceV3", "OCR_MergePaperOCR", "OCR_DriverLicenseV2", "OCR_RealEstateCertV2", "OCR_TrainTicketV2", "OCR_TaxiInvoiceV2", "OCR_OfficialSealV2", "OCR_MergeCarOCR", "OCR_MergeEducationOCR", "OCR_HouseCertV2", "OCR_CommonEnglishNumbersCode", "OCR_BusinessLicenseV2", "OCR_VinV2"];
for (const id of ocrIds) register(id, async (ctx, node, args) => {
const value = await ocrImage(args, node); ctx.log(id + " OCR 完成:" + value.length + " 字符", "info", node.id); return value;
});
@@ -2,7 +2,7 @@ import { spawn } from "node:child_process";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./system-extended";
@@ -2,7 +2,8 @@ import os from "node:os";
import path from "node:path";
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell, psString } from "./desktop";
import { powershell, psString } from "../driver/windows/powershell";
import { defaultEngine } from "../engine/engine";
function value(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
@@ -31,20 +32,8 @@ function selector(args: Record<string, unknown>): string {
return String(raw ?? "");
}
const WINDOW_API = `Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class RpaWindowApi {
[StructLayout(LayoutKind.Sequential)] public struct RECT { public int Left; public int Top; public int Right; public int Bottom; }
[DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
[DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr hWnd, int command);
[DllImport("user32.dll")] public static extern bool SetWindowPos(IntPtr hWnd, IntPtr after, int x, int y, int width, int height, uint flags);
}
'@`;
function findWindowScript(title: string): string {
return `$p = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw '找不到窗口:${title.replace(/'/g, "''")}' }`;
}
// 窗口矩形/状态操作统一走驱动(任务 3.3)
const driver = defaultEngine.driver;
register("System_GetEnv", (_ctx, node, args) => {
const name = required(args, ["name"], "环境变量名称", node.id, node.componentId);
@@ -75,21 +64,30 @@ register("System_GetSystemPath", (_ctx, _node, args) => {
const getPids: CapabilityHandler = async (_ctx, node, args) => {
const pattern = processPattern(required(args, ["pattern", "p_name", "name"], "进程名称", node.id, node.componentId));
const output = await powershell(`Get-Process | Where-Object { $_.ProcessName -like ${psString(pattern)} } | Select-Object -ExpandProperty Id`);
return output.split(/\r?\n/).map(Number).filter(Number.isFinite);
return output.split(/\r?\n/).map((line) => line.trim()).filter(Boolean)
.map(Number).filter((pid) => Number.isSafeInteger(pid) && pid > 0);
};
register("App_GetPids", getPids);
register("App_GetPid", async (ctx, node, args, component) => (await getPids(ctx, node, args, component) as number[])[0] ?? null);
register(["App_Kill", "App_KillV1"], async (_ctx, node, args) => {
const pid = Number(value(args, ["pid"]));
const name = value(args, ["p_name", "pattern"]);
if (Number.isFinite(pid) && pid > 0) {
const mode = node.componentId === "App_Kill" ? "id" : value(args, ["close_type"], "id");
if (mode === "id") {
const rawPid = args.pid;
const pid = typeof rawPid === "number" || typeof rawPid === "string" ? Number(rawPid) : NaN;
if (!Number.isSafeInteger(pid) || pid <= 0) {
throw new RuntimeError("validation_error", "进程 PID 必须是正整数", { nodeId: node.id, componentId: node.componentId });
}
if (pid === process.pid) throw new RuntimeError("validation_error", "拒绝关闭 RPA Studio 自身进程", { nodeId: node.id, componentId: node.componentId });
await powershell(`Stop-Process -Id ${pid} -Force -ErrorAction Stop`);
return pid;
}
const pattern = processPattern(required({ name }, ["name"], "进程名称", node.id, node.componentId));
await powershell(`Get-Process | Where-Object { $_.ProcessName -like ${psString(pattern)} } | Stop-Process -Force -ErrorAction Stop`);
if (mode !== "name") throw new RuntimeError("validation_error", "请选择按 PID 或按进程名称关闭", { nodeId: node.id, componentId: node.componentId });
const pattern = processPattern(required(args, ["p_name", "pattern"], "进程名称", node.id, node.componentId));
if (pattern === "*") throw new RuntimeError("validation_error", "关闭应用程序必须指定具体的进程名称", { nodeId: node.id, componentId: node.componentId });
const allSessions = [true, 1, "1", "true", "True"].includes(args.kill_all as never);
const session = allSessions ? "" : " -and $_.SessionId -eq [System.Diagnostics.Process]::GetCurrentProcess().SessionId";
await powershell(`Get-Process | Where-Object { $_.ProcessName -like ${psString(pattern)} -and $_.Id -ne ${process.pid}${session} } | Stop-Process -Force -ErrorAction Stop`);
return pattern;
});
@@ -106,15 +104,14 @@ register("Clipboard_ClearClipboard", async () => {
register("Window_GetWindowRect", async (_ctx, node, args) => {
const title = required({ title: selector(args) }, ["title"], "窗口标题", node.id, node.componentId);
const output = await powershell(`${WINDOW_API}; ${findWindowScript(title)}; $r = New-Object RpaWindowApi+RECT; [RpaWindowApi]::GetWindowRect($p.MainWindowHandle,[ref]$r) | Out-Null; @{x=$r.Left;y=$r.Top;width=$r.Right-$r.Left;height=$r.Bottom-$r.Top;title=$p.MainWindowTitle;pid=$p.Id} | ConvertTo-Json -Compress`);
return JSON.parse(output);
return driver.window.findByTitle(title);
});
register(["Window_SetWindowState", "Window_SetWindowState_v1"], async (_ctx, node, args) => {
const title = required({ title: selector(args) }, ["title"], "窗口标题", node.id, node.componentId);
const state = value(args, ["WindowType", "switchType"], "maximize").toLowerCase();
const command = state.includes("min") ? 6 : state.includes("max") ? 3 : 9;
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; [RpaWindowApi]::ShowWindow($p.MainWindowHandle,${command}) | Out-Null`);
await driver.window.setState(title, command);
return state;
});
@@ -127,6 +124,6 @@ register("Window_SizeWindow_v1", async (_ctx, node, args) => {
}
const [width, height] = Array.isArray(size) ? size.map(Number) : [0, 0];
if (!(width > 0 && height > 0)) throw new RuntimeError("validation_error", "窗口尺寸无效", { nodeId: node.id, componentId: node.componentId });
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; $r = New-Object RpaWindowApi+RECT; [RpaWindowApi]::GetWindowRect($p.MainWindowHandle,[ref]$r) | Out-Null; [RpaWindowApi]::SetWindowPos($p.MainWindowHandle,[IntPtr]::Zero,$r.Left,$r.Top,${width},${height},4) | Out-Null`);
await driver.window.setSize(title, width, height);
return { width, height };
});
@@ -0,0 +1,15 @@
import { register } from "../engine/registry";
function part(args: Record<string, unknown>, get: (d: Date) => number): number {
const ts = Number(args.timestamp);
const d = Number.isFinite(ts) && ts > 0 ? new Date(ts) : new Date();
return get(d);
}
register("Time_Year", async (_ctx, _node, args) => part(args, d => d.getFullYear()));
register("Time_Month", async (_ctx, _node, args) => part(args, d => d.getMonth() + 1));
register("Time_Day", async (_ctx, _node, args) => part(args, d => d.getDate()));
register("Time_Hour", async (_ctx, _node, args) => part(args, d => d.getHours()));
register("Time_Minute", async (_ctx, _node, args) => part(args, d => d.getMinutes()));
register("Time_Second", async (_ctx, _node, args) => part(args, d => d.getSeconds()));
register("Time_WeekDay", async (_ctx, _node, args) => part(args, d => { const w = d.getDay(); return w === 0 ? 7 : w; }));
@@ -2,7 +2,7 @@ import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import type { Catalog, CatalogComponent, FlowDefinition, FlowNode } from "../../shared/types";
import type { Catalog, CatalogComponent, FlowDefinition, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./utility-extended";
@@ -79,6 +79,20 @@ describe("utility extended capabilities", () => {
await expect(run("System_ExecuteCommandV1", args)).resolves.toContain("utility-ok");
});
it("executes BAT files whose path contains spaces and Chinese characters", async () => {
if (process.platform !== "win32") return;
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-utility-中文 空格-"));
directories.push(directory);
const file = path.join(directory, "测试 脚本.bat");
await writeFile(file, "@echo RPA-BAT-PATH-OK\r\n", "ascii");
await expect(run("System_ExecuteCommandV1", { command_type: "2", bat_file: file, get_ret: "1" })).resolves.toBe("RPA-BAT-PATH-OK");
});
it("keeps PowerShell command output in UTF-8", async () => {
if (process.platform !== "win32") return;
await expect(run("System_ExecuteCommandV1", { command_type: "3", ps_command: "Write-Output '中文输出测试'", get_ret: "1" })).resolves.toBe("中文输出测试");
});
it("compares local text files with structured line changes", async () => {
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-utility-"));
directories.push(directory);
@@ -252,6 +252,7 @@ register("Basic_GetProcessInfo", (ctx) => ({
register("System_ExecuteCommandV1", async (_ctx, node, args) => {
const mode = String(args.command_type || "1");
if (!["1", "2", "3"].includes(mode)) throw new RuntimeError("validation_error", "请选择 DOS、BAT 或 PowerShell 命令类型", { nodeId: node.id, componentId: node.componentId });
const cwd = String(args.cwd || "").trim() || undefined;
const timeout = Math.max(1_000, Number(args.timeoutMs || 120_000));
let executable: string;
@@ -264,7 +265,9 @@ register("System_ExecuteCommandV1", async (_ctx, node, args) => {
commandArgs = ["/d", "/s", "/c", `call "${file}"`];
} else if (mode === "3") {
executable = "powershell.exe";
commandArgs = ["-NoLogo", "-NoProfile", "-NonInteractive", "-Command", requiredText(args.ps_command, "PowerShell命令", node.id, node.componentId)];
const utf8 = "$ProgressPreference='SilentlyContinue'; [Console]::OutputEncoding=[System.Text.UTF8Encoding]::new($false); $OutputEncoding=[Console]::OutputEncoding; ";
const script = utf8 + requiredText(args.ps_command, "PowerShell命令", node.id, node.componentId);
commandArgs = ["-NoLogo", "-NoProfile", "-NonInteractive", "-EncodedCommand", Buffer.from(script, "utf16le").toString("base64")];
} else {
executable = process.env.ComSpec || "cmd.exe";
commandArgs = ["/d", "/s", "/c", requiredText(args.dos_command, "DOS命令", node.id, node.componentId)];
@@ -276,7 +279,8 @@ register("System_ExecuteCommandV1", async (_ctx, node, args) => {
executable = "/bin/sh";
commandArgs = ["-lc", command];
}
const result = await execFileAsync(executable, commandArgs, { cwd, timeout, windowsHide: true, maxBuffer: 10 * 1024 * 1024 });
// cmd.exe 自己解析 /c 后的命令;不能让 Node 再把 BAT 路径引号转成反斜杠引号。
const result = await execFileAsync(executable, commandArgs, { cwd, timeout, windowsHide: true, windowsVerbatimArguments: process.platform === "win32" && mode !== "3", maxBuffer: 10 * 1024 * 1024 });
return bool(args.get_ret, true) ? String(result.stdout).trimEnd() : true;
});
@@ -0,0 +1,96 @@
import { describe, expect, it } from "vitest";
import { PNG } from "pngjs";
import { clickPoint, clampRegion, cropPng, parseRegion } from "./vision";
function makeImage(width = 100, height = 80): PNG {
const png = new PNG({ width, height });
for (let i = 0; i < png.data.length; i += 4) {
png.data[i] = 200;
png.data[i + 1] = 100;
png.data[i + 2] = 50;
png.data[i + 3] = 255;
}
return png;
}
describe("parseRegion", () => {
it("accepts object x/y/width/height", () => {
const image = makeImage();
expect(parseRegion('{"x":10,"y":20,"width":30,"height":40}', image)).toEqual({ x: 10, y: 20, width: 30, height: 40 });
});
it("accepts array [x,y,width,height]", () => {
const image = makeImage();
expect(parseRegion("[1,2,3,4]", image)).toEqual({ x: 1, y: 2, width: 3, height: 4 });
});
it("accepts left/top aliases and corner points", () => {
const image = makeImage();
expect(parseRegion('{"left":5,"top":6,"width":7,"height":8}', image)).toEqual({ x: 5, y: 6, width: 7, height: 8 });
expect(parseRegion('{"x":5,"y":6,"x2":15,"y2":16}', image)).toEqual({ x: 5, y: 6, width: 10, height: 10 });
});
it("rejects malformed JSON and unknown shapes", () => {
const image = makeImage();
expect(() => parseRegion("not-json", image)).toThrow(/无法解析/);
expect(() => parseRegion('{"foo":1}', image)).toThrow(/不符合要求/);
});
});
describe("clampRegion", () => {
it("clamps region into image bounds", () => {
const image = makeImage(100, 80);
expect(clampRegion({ x: -5, y: -5, width: 200, height: 200 }, image)).toEqual({ x: 0, y: 0, width: 100, height: 80 });
expect(clampRegion({ x: 90, y: 70, width: 50, height: 50 }, image)).toEqual({ x: 90, y: 70, width: 10, height: 10 });
});
});
describe("cropPng", () => {
it("crops sub-image with expected size and pixel origin", () => {
const image = makeImage();
const cropped = cropPng(image, { x: 10, y: 20, width: 30, height: 40 });
expect(cropped.width).toBe(30);
expect(cropped.height).toBe(40);
const index = (0 * 30 + 0) * 4;
expect([cropped.data[index], cropped.data[index + 1], cropped.data[index + 2]]).toEqual([200, 100, 50]);
});
});
describe("clickPoint", () => {
const region = { x: 100, y: 200, width: 60, height: 40, score: 1 };
it("defaults to center", () => {
expect(clickPoint(region, "")).toEqual({ x: 130, y: 220 });
expect(clickPoint(region, "center")).toEqual({ x: 130, y: 220 });
});
it("supports nine-grid anchors", () => {
expect(clickPoint(region, "leftTop")).toEqual({ x: 100, y: 200 });
expect(clickPoint(region, "rightBottom")).toEqual({ x: 160, y: 240 });
expect(clickPoint(region, "top")).toEqual({ x: 130, y: 200 });
expect(clickPoint(region, "left")).toEqual({ x: 100, y: 220 });
});
it("supports custom offset JSON", () => {
expect(clickPoint(region, '{"x":5,"y":6}')).toEqual({ x: 105, y: 206 });
});
it("falls back to center for unknown values", () => {
expect(clickPoint(region, "bogus")).toEqual({ x: 130, y: 220 });
});
});
describe("legacy position aliases", () => {
it("accepts kebab-case anchors saved by older catalogs", () => {
const region = { x: 100, y: 200, width: 60, height: 40, score: 1 };
expect(clickPoint(region, "left-top")).toEqual({ x: 100, y: 200 });
expect(clickPoint(region, "right-top")).toEqual({ x: 160, y: 200 });
expect(clickPoint(region, "left-bottom")).toEqual({ x: 100, y: 240 });
expect(clickPoint(region, "right-bottom")).toEqual({ x: 160, y: 240 });
});
it("still falls back to center for unknown anchors", () => {
const region = { x: 100, y: 200, width: 60, height: 40, score: 1 };
expect(clickPoint(region, "middle")).toEqual({ x: 130, y: 220 });
});
});
@@ -0,0 +1,83 @@
import { describe, expect, it } from "vitest";
import { PNG } from "pngjs";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { findImage } from "./vision";
function makeScreen(width: number, height: number, patch: { x: number; y: number; w: number; h: number }): PNG {
const png = new PNG({ width, height });
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
const i = (y * width + x) * 4;
const inPatch = x >= patch.x && x < patch.x + patch.w && y >= patch.y && y < patch.y + patch.h;
// 背景: 平滑斜向渐变(贴近真实界面); 补丁: 纯红色
png.data[i] = inPatch ? 220 : (x + y) % 90;
png.data[i + 1] = inPatch ? 30 : (x + y + 30) % 90;
png.data[i + 2] = inPatch ? 40 : (x + y + 60) % 90;
png.data[i + 3] = 255;
}
}
return png;
}
function crop(source: PNG, x: number, y: number, w: number, h: number): PNG {
const out = new PNG({ width: w, height: h });
PNG.bitblt(source, out, x, y, w, h, 0, 0);
return out;
}
describe("findImage 匹配质量", () => {
it("相似度0.9时精确命中模板位置且不误报", async () => {
const patch = { x: 123, y: 77, w: 48, h: 36 };
const screen = makeScreen(400, 300, patch);
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "vision-"));
const screenPath = path.join(dir, "screen.png");
const templatePath = path.join(dir, "template.png");
fs.writeFileSync(screenPath, PNG.sync.write(screen));
fs.writeFileSync(templatePath, PNG.sync.write(crop(screen, patch.x, patch.y, patch.w, patch.h)));
// 相似度 0.9 → 最大差异 0.1
const result = await findImage(templatePath, screenPath, 0.1);
expect(result).not.toBeNull();
// 采样步长落在整数边界时, 相邻1px偏移可能同样满分, 允许±1px
expect(Math.abs(result!.x - patch.x)).toBeLessThanOrEqual(1);
expect(Math.abs(result!.y - patch.y)).toBeLessThanOrEqual(1);
expect(result!.score).toBeGreaterThanOrEqual(0.9);
// 无该补丁的屏幕应返回 null 而非乱命中
const other = makeScreen(400, 300, { x: 0, y: 0, w: 1, h: 1 });
const otherPath = path.join(dir, "other.png");
fs.writeFileSync(otherPath, PNG.sync.write(other));
const miss = await findImage(templatePath, otherPath, 0.1);
expect(miss).toBeNull();
});
it("模板被缩放(1.25倍)时按尺度兜底命中", async () => {
const patch = { x: 200, y: 120, w: 60, h: 40 };
const screen = makeScreen(500, 350, patch);
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "vision-"));
const screenPath = path.join(dir, "screen.png");
fs.writeFileSync(screenPath, PNG.sync.write(screen));
// 模板取自 DIP 截图(比物理屏小1.25倍): 把补丁缩小1/1.25后存为模板
const { createCanvas, loadImage } = await import("@napi-rs/canvas");
const small = crop(screen, patch.x, patch.y, patch.w, patch.h);
const srcCanvas = createCanvas(small.width, small.height);
const srcCtx = srcCanvas.getContext("2d");
const imgData = srcCtx.createImageData(small.width, small.height);
imgData.data.set(small.data);
srcCtx.putImageData(imgData, 0, 0);
const dstCanvas = createCanvas(Math.round(small.width / 1.25), Math.round(small.height / 1.25));
const dstCtx = dstCanvas.getContext("2d");
dstCtx.drawImage(srcCanvas, 0, 0, dstCanvas.width, dstCanvas.height);
// canvas → pngjs PNG(PNG.sync.write 只认 pngjs 对象)
const scaledData = dstCtx.getImageData(0, 0, dstCanvas.width, dstCanvas.height);
const templatePng = new PNG({ width: dstCanvas.width, height: dstCanvas.height });
templatePng.data = Buffer.from(scaledData.data);
const templatePath = path.join(dir, "template-small.png");
fs.writeFileSync(templatePath, PNG.sync.write(templatePng));
const result = await findImage(templatePath, screenPath, 0.1, [1.25, 1 / 1.25]);
expect(result).not.toBeNull();
expect(Math.abs(result!.x - patch.x)).toBeLessThanOrEqual(3);
expect(Math.abs(result!.y - patch.y)).toBeLessThanOrEqual(3);
void loadImage;
});
});
@@ -0,0 +1,557 @@
import path from "node:path";
import { promises as fs } from "node:fs";
import { PNG } from "pngjs";
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { powershell, psString } from "../driver/windows/powershell";
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) if (args[key] !== undefined && args[key] !== null && args[key] !== "") return String(args[key]);
return fallback;
}
/** 全虚拟桌面(所有显示器拼接)物理分辨率截图。
* desktopCapturer thumbnailSize 乘 scaleFactor 与「屏幕框选」模板(物理像素裁剪)尺度一致;
* 逐屏截图后按各自物理原点拼到一张画布上——目标在副屏也能搜到。
* 返回 originX/originY = 拼接画布左上角对应的屏幕物理坐标(副屏在主屏左侧时为负),用于结果坐标平移。 */
async function captureVirtualScreenPng(): Promise<{ png: PNG; originX: number; originY: number }> {
const { desktopCapturer, screen } = await import("electron");
const displays = screen.getAllDisplays();
if (displays.length === 0) throw new RuntimeError("desktop_error", "无法获取屏幕信息");
// 每屏按自己的物理分辨率截图(display_id 匹配 → 缩略图尺寸匹配 → 按物理原点排序对位)
const captures: { display: Electron.Display; origin: { x: number; y: number }; png: PNG }[] = [];
for (const display of displays) {
const scale = display.scaleFactor || 1;
const wantW = Math.round(display.bounds.width * scale);
const wantH = Math.round(display.bounds.height * scale);
const sources = await desktopCapturer.getSources({
types: ["screen"],
thumbnailSize: { width: wantW, height: wantH }
});
const source = sources.find(item => item.display_id === String(display.id))
|| sources.find(item => item.thumbnail.getSize().width === wantW)
|| sources[0];
if (!source || source.thumbnail.isEmpty()) {
throw new RuntimeError("desktop_error", "无法获取屏幕截图");
}
// 显示器 DIP 原点的物理坐标 = 该屏物理原点(屏内线性映射: 物理 = (DIP-原点)*缩放 + 物理原点)
const origin = screen.dipToScreenPoint({ x: display.bounds.x, y: display.bounds.y });
captures.push({ display, origin, png: PNG.sync.read(source.thumbnail.toPNG()) });
}
const originX = Math.min(...captures.map(c => c.origin.x));
const originY = Math.min(...captures.map(c => c.origin.y));
const endX = Math.max(...captures.map(c => c.origin.x + c.png.width));
const endY = Math.max(...captures.map(c => c.origin.y + c.png.height));
const width = endX - originX;
const height = endY - originY;
const { createCanvas } = await import("@napi-rs/canvas");
const canvas = createCanvas(width, height);
const context = canvas.getContext("2d");
for (const capture of captures) {
const sourceCanvas = createCanvas(capture.png.width, capture.png.height);
const sourceContext = sourceCanvas.getContext("2d");
const imageData = sourceContext.createImageData(capture.png.width, capture.png.height);
imageData.data.set(capture.png.data);
sourceContext.putImageData(imageData, 0, 0);
context.drawImage(sourceCanvas, capture.origin.x - originX, capture.origin.y - originY);
}
const stitched = context.getImageData(0, 0, width, height);
const png = new PNG({ width, height });
png.data = Buffer.from(stitched.data);
return { png, originX, originY };
}
/** DIP 区域 → 物理像素区域:左上角经 dipToScreenPoint 换算(天然支持副屏/负坐标),宽高乘所在屏缩放比 */
async function dipRegionToPhysical(region: { x: number; y: number; width: number; height: number }): Promise<{ x: number; y: number; width: number; height: number }> {
const { screen } = await import("electron");
const topLeft = screen.dipToScreenPoint({ x: Math.round(region.x), y: Math.round(region.y) });
const display = screen.getDisplayMatching({
x: region.x, y: region.y,
width: Math.max(1, Math.round(region.width)), height: Math.max(1, Math.round(region.height))
});
const scale = display.scaleFactor || 1;
return {
x: topLeft.x,
y: topLeft.y,
width: Math.max(1, Math.round(region.width * scale)),
height: Math.max(1, Math.round(region.height * scale))
};
}
/** 按魔数判断是否 PNG(89 50 4E 47) */
function isPngBuffer(buffer: Buffer): boolean {
return buffer.length > 8 && buffer[0] === 0x89 && buffer[1] === 0x50 && buffer[2] === 0x4e && buffer[3] === 0x47;
}
/** 解码用户提供的图片文件为 pngjs PNG。pngjs 只认 PNG, 模板/输入图常是 JPG——
* 非 PNG 时走 @napi-rs/canvas 解码(JPG/BMP/WebP 等), 否则报 "unrecognised content at end of stream" */
async function decodeImageFile(filePath: string): Promise<PNG> {
let buffer: Buffer;
try {
buffer = await fs.readFile(filePath);
} catch (error) {
if ((error as NodeJS.ErrnoException)?.code === "ENOENT") {
throw new RuntimeError("validation_error", `找不到图片文件:${filePath}`, {
recommendation: "确认路径正确,或直接点参数旁的「屏幕框选」截取一张"
});
}
throw error;
}
if (isPngBuffer(buffer)) return PNG.sync.read(buffer);
const { loadImage, createCanvas } = await import("@napi-rs/canvas");
const image = await loadImage(buffer);
if (!image.width || !image.height) {
throw new RuntimeError("validation_error", "图片解码失败", { detail: filePath });
}
const canvas = createCanvas(image.width, image.height);
const context = canvas.getContext("2d");
context.drawImage(image, 0, 0);
const imageData = context.getImageData(0, 0, image.width, image.height);
const png = new PNG({ width: image.width, height: image.height });
png.data = Buffer.from(imageData.data);
return png;
}
/** 解析区域 JSON:对象 {x,y,width,height}(兼容 left/top、x1y1x2y2 角点)或数组 [x,y,width,height],并夹取到图片边界内 */
export function parseRegion(raw: string, image: PNG): { x: number; y: number; width: number; height: number } {
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
throw new RuntimeError("validation_error", "区域 JSON 无法解析", { detail: raw.slice(0, 200) });
}
const numberOf = (value: unknown): number | null => {
const n = Number(value);
return Number.isFinite(n) ? n : null;
};
if (Array.isArray(parsed) && parsed.length === 4) {
const [x, y, width, height] = parsed.map(numberOf);
if (x !== null && y !== null && width !== null && height !== null) {
return clampRegion({ x, y, width, height }, image);
}
}
if (parsed && typeof parsed === "object") {
const record = parsed as Record<string, unknown>;
const x = numberOf(record.x ?? record.left);
const y = numberOf(record.y ?? record.top);
const width = numberOf(record.width);
const height = numberOf(record.height);
if (x !== null && y !== null && width !== null && height !== null) {
return clampRegion({ x, y, width, height }, image);
}
const x2 = numberOf(record.x2 ?? record.right);
const y2 = numberOf(record.y2 ?? record.bottom);
if (x !== null && y !== null && x2 !== null && y2 !== null) {
return clampRegion({ x, y, width: x2 - x, height: y2 - y }, image);
}
}
throw new RuntimeError("validation_error", "区域 JSON 不符合要求", {
detail: raw.slice(0, 200),
recommendation: '使用 {"x":0,"y":0,"width":100,"height":80} 或 [x, y, width, height]'
});
}
export function clampRegion(region: { x: number; y: number; width: number; height: number }, image: PNG): { x: number; y: number; width: number; height: number } {
const x = Math.max(0, Math.min(Math.round(region.x), image.width - 1));
const y = Math.max(0, Math.min(Math.round(region.y), image.height - 1));
const width = Math.max(1, Math.min(Math.round(region.width), image.width - x));
const height = Math.max(1, Math.min(Math.round(region.height), image.height - y));
return { x, y, width, height };
}
export function cropPng(source: PNG, region: { x: number; y: number; width: number; height: number }): PNG {
const cropped = new PNG({ width: region.width, height: region.height });
PNG.bitblt(source, cropped, region.x, region.y, region.width, region.height, 0, 0);
return cropped;
}
/** 目标路径容错:填的是文件夹(已存在或以分隔符结尾)时,自动往里存默认文件名 */
async function resolveOutputPath(outputPath: string, defaultName: string): Promise<string> {
if (/[\\/]$/.test(outputPath)) return path.join(outputPath, defaultName);
try {
const stat = await fs.stat(outputPath);
if (stat.isDirectory()) return path.join(outputPath, defaultName);
} catch {
// 路径不存在说明是要新建的文件,原样返回
}
return outputPath;
}
register("Image_Screenshot", async (ctx, node, args) => {
const folder = text(args, ["outImagePath", "folder"]);
const name = text(args, ["outImageName", "name"], `screenshot_${Date.now()}.png`);
const imageArg = text(args, ["image_path"]);
const source = text(args, ["pictureSource"], "1");
const range = text(args, ["shotRange"], "1");
let image: PNG;
let output: string;
let screenOrigin: { x: number; y: number } | null = null;
if (source === "2") {
// 指定图片文件来源:image_path 为输入图片;输出走 输出目录+文件名,两者均未配置时直接返回源文件
if (!imageArg.trim()) {
throw new RuntimeError("validation_error", "图像来源为「指定图片文件」时,截图保存路径需填写源图片路径", {
recommendation: "在截图保存路径选择一张已存在的图片,或将图像来源改回「当前屏幕」"
});
}
const sourceFile = path.resolve(imageArg);
image = await decodeImageFile(sourceFile);
output = folder ? path.resolve(path.join(folder, name)) : sourceFile;
} else {
// 当前屏幕来源:image_path 即保存路径(文件夹则自动追加默认文件名),缺省用 输出目录+文件名;再缺省落流程证据目录(路径可预期)
// 截取整个虚拟桌面(含副屏),物理分辨率
const shot = await captureVirtualScreenPng();
image = shot.png;
screenOrigin = { x: shot.originX, y: shot.originY };
output = await resolveOutputPath(
path.resolve(imageArg || (folder ? path.join(folder, name) : path.join(ctx.evidenceDir, name))),
name
);
}
if (range === "2") {
const regionRaw = text(args, ["position", "action", "position_img"]);
if (!regionRaw.trim()) {
throw new RuntimeError("validation_error", "截图范围为「框选区域」时,截图区域需填写区域 JSON", {
recommendation: '在「截图区域」点屏幕框选,或填写 {"x":0,"y":0,"width":100,"height":80},或将截图范围改回「整个屏幕」'
});
}
if (screenOrigin) {
// 屏幕来源: 区域 JSON 是 DIP 坐标(支持副屏),换算物理坐标后减去拼接画布原点再裁剪
const phys = await dipRegionToPhysical(parseRegion(regionRaw, image));
image = cropPng(image, clampRegion({
x: phys.x - screenOrigin.x,
y: phys.y - screenOrigin.y,
width: phys.width,
height: phys.height
}, image));
} else {
// 图片文件来源: 区域按该图片自身像素坐标
image = cropPng(image, parseRegion(regionRaw, image));
}
}
await fs.mkdir(path.dirname(output), { recursive: true });
await fs.writeFile(output, PNG.sync.write(image));
return output;
});
interface MatchResult { x: number; y: number; width: number; height: number; score: number; }
interface WindowRect { x: number; y: number; width: number; height: number }
/** 按标题(精确→包含)或进程名查找窗口,前置并返回窗口矩形(物理屏幕坐标) */
async function focusWindowRect(windowObj: string): Promise<WindowRect> {
const quoted = psString(windowObj);
const stdout = await powershell(`Add-Type @'
using System; using System.Runtime.InteropServices;
public static class WinRect {
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value);
[DllImport("user32.dll")] public static extern IntPtr FindWindow(string cls, string title);
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h);
[DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr h, int cmd);
[DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr h, out RECT r);
public struct RECT { public int Left; public int Top; public int Right; public int Bottom; }
}
'@;
# 不声明 DPI 感知时 GetWindowRect 返回虚拟化坐标, 与物理分辨率截图对不上
try { [void][WinRect]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {};
$h = [WinRect]::FindWindow($null, ${quoted});
if ($h -eq [IntPtr]::Zero) { $p = Get-Process | Where-Object { $_.MainWindowTitle -and ($_.MainWindowTitle -eq ${quoted} -or $_.MainWindowTitle.Contains(${quoted})) } | Select-Object -First 1; if ($p) { $h = $p.MainWindowHandle } }
if ($h -eq [IntPtr]::Zero) { Write-Output 'NOT_FOUND'; exit }
[WinRect]::ShowWindow($h, 9) | Out-Null;
[WinRect]::SetForegroundWindow($h) | Out-Null;
Start-Sleep -Milliseconds 300;
$r = New-Object WinRect+RECT;
[WinRect]::GetWindowRect($h, [ref]$r) | Out-Null;
Write-Output ("{0},{1},{2},{3}" -f $r.Left, $r.Top, $r.Right, $r.Bottom)`);
const line = stdout.trim().split(/\r?\n/).pop() ?? "";
if (line === "NOT_FOUND") {
throw new RuntimeError("not_found", `未找到窗口「${windowObj}」`, { recommendation: "确认窗口已打开,窗口对象可填完整标题、标题包含关键字或进程名" });
}
const parts = line.split(",").map(Number);
if (parts.length !== 4 || parts.some((v) => !Number.isFinite(v))) {
throw new RuntimeError("desktop_error", "窗口矩形获取失败", { detail: line.slice(0, 120) });
}
const [left, top, right, bottom] = parts;
return { x: left, y: top, width: Math.max(1, right - left), height: Math.max(1, bottom - top) };
}
/** 单通道噪声死区:JPG 压缩伪影/抗锯齿带来 ±10 级内的像素波动,不应计入差异 */
const NOISE_FLOOR = 10;
function pixelDifference(screen: PNG, template: PNG, offsetX: number, offsetY: number, sampleStep: number, cutoff: number): number {
let difference = 0;
let samples = 0;
for (let y = 0; y < template.height; y += sampleStep) {
for (let x = 0; x < template.width; x += sampleStep) {
const sourceIndex = ((offsetY + y) * screen.width + offsetX + x) * 4;
const templateIndex = (y * template.width + x) * 4;
for (let c = 0; c < 3; c++) {
const d = Math.abs(screen.data[sourceIndex + c] - template.data[templateIndex + c]);
difference += d <= NOISE_FLOOR ? 0 : d - NOISE_FLOOR;
}
samples += 3;
// cutoff 是 0~1 的归一化差异上限; difference 是通道差累加值(0~255 每通道),
// 提前退出必须按 255 折算——之前漏乘导致几乎所有位置在第一个像素就被剪掉, 得分全乱
if (difference > cutoff * samples * 255) return difference / (samples * 255);
}
}
return samples ? difference / (samples * 255) : 1;
}
/** 在屏幕图里扫描模板的最佳位置(粗扫网格 + 精化) */
function scanBest(screen: PNG, template: PNG, maximumDifference: number): MatchResult | null {
const sampleStep = Math.max(1, Math.floor(Math.min(template.width, template.height) / 18));
const scanStep = Math.max(1, Math.floor(Math.min(template.width, template.height) / 12));
let best: MatchResult | null = null;
for (let y = 0; y <= screen.height - template.height; y += scanStep) {
for (let x = 0; x <= screen.width - template.width; x += scanStep) {
// 尚无 best 时不剪枝(完整评估): 若初始就按 maximumDifference 剪枝,
// 均匀背景位置在少数采样后返回"刚好压线"的部分均值抢占 best,
// 而真位置(常不在网格上)附近的网格点因部分重叠被剪, 精化永远够不到——
// 相似度 0.95 时实测 score=1.0 的真位置被判"未达到阈值"
const cutoff = best ? 1 - best.score : 1;
const difference = pixelDifference(screen, template, x, y, sampleStep, cutoff);
const score = 1 - difference;
if (!best || score > best.score) best = { x, y, width: template.width, height: template.height, score };
}
}
if (!best) return null;
// 注意: 阈值判定必须放在精化之后——粗扫网格(步长=模板短边/12)通常够不到精确位置,
// 粗扫 best 常差在阈值边缘, 精化后才能提到满分; 先判阈值会把真命中误杀
const refine = Math.max(1, scanStep);
const minX = Math.max(0, best.x - refine);
const maxX = Math.min(screen.width - template.width, best.x + refine);
const minY = Math.max(0, best.y - refine);
const maxY = Math.min(screen.height - template.height, best.y + refine);
for (let y = minY; y <= maxY; y += 1) {
for (let x = minX; x <= maxX; x += 1) {
// 精化区域只有几百个位置, 完整评估避免同样的剪枝偏置
const difference = pixelDifference(screen, template, x, y, sampleStep, 1);
const score = 1 - difference;
if (score > best.score) best = { x, y, width: template.width, height: template.height, score };
}
}
// 阈值判定: 精化后的最终得分才与 maximumDifference 比较
if (1 - best.score > maximumDifference) return null;
return best;
}
/** 用 canvas 双线性缩放模板(匹配尺度兜底用) */
async function resizeTemplate(template: PNG, scale: number): Promise<PNG> {
const { createCanvas } = await import("@napi-rs/canvas");
const width = Math.max(1, Math.round(template.width * scale));
const height = Math.max(1, Math.round(template.height * scale));
const source = createCanvas(template.width, template.height);
const sourceContext = source.getContext("2d");
const imageData = sourceContext.createImageData(template.width, template.height);
imageData.data.set(template.data);
sourceContext.putImageData(imageData, 0, 0);
const target = createCanvas(width, height);
const targetContext = target.getContext("2d");
targetContext.imageSmoothingEnabled = true;
targetContext.imageSmoothingQuality = "high";
targetContext.drawImage(source, 0, 0, width, height);
const scaled = targetContext.getImageData(0, 0, width, height);
const png = new PNG({ width, height });
png.data = Buffer.from(scaled.data);
return png;
}
/**
* 模板匹配。extraScales 提供尺度兜底:截图与模板的获取渠道分辨率不一致时
* (如旧版 DIP 截图存下的模板 vs 现在的物理分辨率截图),按给定缩放比重试。
*/
export async function findImage(templatePath: string, screenshotPath: string, accuracy: number, extraScales: number[] = []): Promise<MatchResult | null> {
const [screenBuffer, templateBuffer] = await Promise.all([fs.readFile(screenshotPath), fs.readFile(templatePath)]);
const screen = PNG.sync.read(screenBuffer);
// 模板图可能是 JPG 等非 PNG 格式, 按魔数选择解码器
const template = isPngBuffer(templateBuffer) ? PNG.sync.read(templateBuffer) : await decodeImageFile(templatePath);
if (template.width > screen.width || template.height > screen.height) {
throw new RuntimeError("validation_error", `模板图片(${template.width}×${template.height})比截图画面(${screen.width}×${screen.height})还大,无法查找`, {
recommendation: "模板应是屏幕上要寻找的小图案(按钮、图标、文字块),不要用整屏大图"
});
}
const maximumDifference = Math.max(0.01, Math.min(0.95, accuracy || 0.2));
const best = scanBest(screen, template, maximumDifference);
if (best) return best;
for (const scale of extraScales) {
if (!Number.isFinite(scale) || scale === 1) continue;
const scaled = await resizeTemplate(template, scale);
if (scaled.width > screen.width || scaled.height > screen.height) continue;
const scaledBest = scanBest(screen, scaled, maximumDifference);
if (scaledBest) return scaledBest;
}
return null;
}
async function locate(args: Record<string, unknown>, evidenceDir: string, nodeId: string): Promise<MatchResult> {
const rawTemplate = text(args, ["image", "image_path", "template"]);
if (!rawTemplate.trim()) throw new RuntimeError("validation_error", "模板图片路径为空", { nodeId });
const template = path.resolve(rawTemplate);
// 查找范围:1 整个虚拟桌面(默认,含副屏)/ 2 指定窗口 / 3 指定区域;
// 截图统一为物理分辨率拼接画布, 结果坐标加回画布原点即屏幕物理坐标(可直接用于点击)
const rangeMode = text(args, ["find_range"], "1");
const screenshot = path.join(evidenceDir, `${nodeId}_screen.png`);
const shot = await captureVirtualScreenPng();
let offsetX = 0;
let offsetY = 0;
if (rangeMode === "2") {
const windowObj = text(args, ["window_obj"]);
if (!windowObj.trim()) throw new RuntimeError("validation_error", "查找范围为「指定窗口」时,窗口对象不能为空", { nodeId, recommendation: "填写窗口标题(或标题关键字、进程名)" });
const rect = await focusWindowRect(windowObj);
const cropRect = clampRegion({ x: rect.x - shot.originX, y: rect.y - shot.originY, width: rect.width, height: rect.height }, shot.png);
await fs.writeFile(screenshot, PNG.sync.write(cropPng(shot.png, cropRect)));
offsetX = rect.x;
offsetY = rect.y;
} else if (rangeMode === "3" || rangeMode === "4") {
const regionRaw = text(args, ["shot_position", "action", "position"]);
if (!regionRaw.trim()) throw new RuntimeError("validation_error", "查找范围为「指定区域」时,需要填写区域 JSON", { nodeId, recommendation: '在 shot_position 填写 {"x":0,"y":0,"width":100,"height":80} 或 [x, y, width, height]' });
// 区域 JSON 是 DIP 坐标(框选工具按 DIP 返回, 支持副屏负坐标),换算物理坐标后减画布原点裁剪
const phys = await dipRegionToPhysical(parseRegion(regionRaw, shot.png));
const cropRect = clampRegion({ x: phys.x - shot.originX, y: phys.y - shot.originY, width: phys.width, height: phys.height }, shot.png);
await fs.writeFile(screenshot, PNG.sync.write(cropPng(shot.png, cropRect)));
offsetX = phys.x;
offsetY = phys.y;
} else {
await fs.writeFile(screenshot, PNG.sync.write(shot.png));
// 拼接画布坐标 → 屏幕物理坐标必须加回画布原点(副屏在主屏左侧/上方时原点为负);
// 漏加会导致点击偏移(实测副屏上移 177px 时点击正好偏下 177px)
offsetX = shot.originX;
offsetY = shot.originY;
}
// accuracy 字段语义为「相似度」(0~1, 越大越严格, 0.9=至少90%像素一致)——与界面文案一致。
// 引擎内部用最大允许差异 = 1 - 相似度。此前引擎直接把该值当差异阈值,
// 用户填 0.9 反而放宽到"接受90%像素差异", 导致到处误命中。
const rawAccuracy = Number(text(args, ["accuracy"], "0.8"));
const similarity = Math.max(0.05, Math.min(1, Number.isFinite(rawAccuracy) && rawAccuracy > 0 ? rawAccuracy : 0.8));
// 原生 1:1 匹配失败时, 按缩放比对模板放大/缩小重试——兼容旧版 DIP 截图存的模板与任意来源的图片
const scale = (await import("electron")).screen.getPrimaryDisplay().scaleFactor || 1;
const result = await findImage(template, screenshot, 1 - similarity, scale !== 1 ? [scale, 1 / scale] : []);
if (!result) throw new RuntimeError("not_found", `屏幕中未找到目标图像(要求相似度 ${(similarity * 100).toFixed(0)}%,未达到阈值)`, { nodeId, recommendation: "重新框选模板(框小一点、只框目标图案),或适当调低相似度(如 0.8);目标在副屏时确认其清晰可见" });
return { ...result, x: result.x + offsetX, y: result.y + offsetY };
}
register(["Image_Find", "Image_Find_V1"], async (ctx, node, args) => locate(args, ctx.evidenceDir, node.id));
// 图像位置: 与查找图像同义(历史重复组件), 同管线返回坐标
for (const id of ["Image_Location", "Image_Location_V1"]) register(id, async (ctx, node, args) => locate(args, ctx.evidenceDir, node.id));
// 判断图像: 找到返回 true / 找不到返回 false(供条件分支), 仅有 not_found 转为 false;
// 模板路径为空/文件不存在等配置错误仍然抛出, 避免静默 false 难排查。
// timeout>0 时在窗口内每 300ms 重试(等待图像出现再判断的场景)
for (const id of ["Image_Exists", "Image_Exists_V1"]) register(id, async (ctx, node, args) => {
const rawTimeout = Number(text(args, ["timeout", "time_out"], "0"));
const deadline = Date.now() + (Number.isFinite(rawTimeout) && rawTimeout > 0 ? rawTimeout : 0) * 1000;
for (;;) {
try {
await locate(args, ctx.evidenceDir, node.id);
return true;
} catch (error) {
if (error instanceof RuntimeError && error.code === "not_found") {
if (Date.now() >= deadline) return false;
} else {
throw error;
}
}
await new Promise(resolve => setTimeout(resolve, 300));
}
});
// 等待图像: 在超时窗口内每 300ms 重试一次查找, 命中即返回坐标, 超时仍无则抛 not_found
for (const id of ["Image_Wait", "Image_Wait_V1"]) register(id, async (ctx, node, args) => {
const rawTimeout = Number(text(args, ["timeout", "time_out"], "20"));
const timeout = Number.isFinite(rawTimeout) && rawTimeout >= 0 ? rawTimeout : 20;
const deadline = Date.now() + timeout * 1000;
for (;;) {
try {
return await locate(args, ctx.evidenceDir, node.id);
} catch (error) {
if (!(error instanceof RuntimeError) || error.code !== "not_found" || Date.now() >= deadline) throw error;
}
await new Promise(resolve => setTimeout(resolve, 300));
}
});
// 移动鼠标到图像: 定位后把光标移到匹配区域中心;
// SetCursorPos 传物理坐标, 脚本必须声明 per-monitor DPI 感知(与 injectClick 同理), 否则被系统虚拟化导致偏移
for (const id of ["Image_Move", "Image_Move_V1", "Image_Move_V2"]) register(id, async (ctx, node, args) => {
const hit = await locate(args, ctx.evidenceDir, node.id);
const point = clickPoint(hit, text(args, ["position"], "center"));
const x = point.x;
const y = point.y;
await powershell("Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value); [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y);' -Name RpaImageMoveInput -Namespace RpaImageMove | Out-Null; try { [void][RpaImageMove.RpaImageMoveInput]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}; [RpaImageMove.RpaImageMoveInput]::SetCursorPos(" + x + "," + y + ")");
return { ...hit, x, y };
});
/** 点击位置锚点(相对匹配区域的比例坐标),支持九宫格取值 */
const POSITION_ANCHORS: Record<string, [number, number]> = {
center: [0.5, 0.5],
leftTop: [0, 0], top: [0.5, 0], centerTop: [0.5, 0], rightTop: [1, 0],
left: [0, 0.5], centerLeft: [0, 0.5], right: [1, 0.5], centerRight: [1, 0.5],
leftBottom: [0, 1], bottom: [0.5, 1], centerBottom: [0.5, 1], rightBottom: [1, 1]
};
export function clickPoint(result: MatchResult, position: string): { x: number; y: number } {
const raw = position.trim();
if (raw && raw.toLowerCase() !== "center") {
if (raw.startsWith("{")) {
try {
const offset = JSON.parse(raw) as { x?: unknown; y?: unknown };
const dx = Number(offset.x);
const dy = Number(offset.y);
if (Number.isFinite(dx) && Number.isFinite(dy)) {
return { x: Math.round(result.x + dx), y: Math.round(result.y + dy) };
}
} catch {
// 非法 JSON 落回中心
}
}
const anchor = POSITION_ANCHORS[raw] ?? POSITION_ANCHORS[raw.toLowerCase()] ?? POSITION_ANCHORS[raw.replace(/-([a-z])/g, (_: string, char: string) => char.toUpperCase())];
if (anchor) {
return { x: Math.round(result.x + result.width * anchor[0]), y: Math.round(result.y + result.height * anchor[1]) };
}
}
return { x: Math.round(result.x + result.width / 2), y: Math.round(result.y + result.height / 2) };
}
const CLICK_ACTIONS: Record<string, string> = {
left: "[Mouse]::mouse_event(2,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 30; [Mouse]::mouse_event(4,0,0,0,[UIntPtr]::Zero)",
right: "[Mouse]::mouse_event(8,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 30; [Mouse]::mouse_event(16,0,0,0,[UIntPtr]::Zero)",
middle: "[Mouse]::mouse_event(32,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 30; [Mouse]::mouse_event(64,0,0,0,[UIntPtr]::Zero)",
double: "[Mouse]::mouse_event(2,0,0,0,[UIntPtr]::Zero); [Mouse]::mouse_event(4,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 60; [Mouse]::mouse_event(2,0,0,0,[UIntPtr]::Zero); [Mouse]::mouse_event(4,0,0,0,[UIntPtr]::Zero)"
};
async function injectClick(x: number, y: number, clickKey: string, showTrack: boolean): Promise<void> {
const action = CLICK_ACTIONS[clickKey] ?? CLICK_ACTIONS.left;
// show_track=1 时从当前光标位置平滑移动过去(20 步插值),否则直接跳转
const move = showTrack
? `for ($i = 1; $i -le 20; $i++) { [Mouse]::SetCursorPos([int]($p.X + (${x} - $p.X) * $i / 20), [int]($p.Y + (${y} - $p.Y) * $i / 20)); Start-Sleep -Milliseconds 12 }`
: `[Mouse]::SetCursorPos(${x}, ${y})`;
await powershell(`Add-Type @'
using System; using System.Runtime.InteropServices;
public static class Mouse {
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int X, int Y);
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT p);
[DllImport("user32.dll")] public static extern void mouse_event(uint f, uint x, uint y, uint d, UIntPtr e);
public struct POINT { public int X; public int Y; }
}
'@;
# 传入的是物理像素坐标(与物理分辨率截图匹配所得), 必须声明 DPI 感知——
# 否则进程被 DPI 虚拟化, 坐标再被系统放大一次, 点击成比例偏右下
try { [void][Mouse]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {};
$p = New-Object Mouse+POINT;
[Mouse]::GetCursorPos([ref]$p) | Out-Null;
${move};
Start-Sleep -Milliseconds 60;
${action}`);
}
register(["Image_Click", "Image_Click_V1", "Image_Click_V2", "Image_Click_V3"], async (ctx, node, args) => {
const result = await locate(args, ctx.evidenceDir, node.id);
const point = clickPoint(result, text(args, ["position"], "center"));
const clickKey = text(args, ["click_key"], "left");
const showTrack = String(args.show_track ?? "0") === "1" || args.show_track === true;
await injectClick(point.x, point.y, clickKey, showTrack);
ctx.log(`已${clickKey === "right" ? "右键" : clickKey === "double" ? "双击" : "点击"}图像 (${point.x}, ${point.y}),匹配得分 ${result.score.toFixed(3)}`, "info", node.id);
return { ...result, click: point };
});
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./window-extended";
@@ -1,20 +1,10 @@
import { register } from "../engine/registry";
import { RuntimeError, type CapabilityHandler } from "../engine/types";
import { powershell, psString } from "./desktop";
import { getHostBridge } from "../engine/host-bridge";
import { defaultEngine } from "../engine/engine";
const WINDOW_API = `Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class RpaWindowExtended {
[StructLayout(LayoutKind.Sequential)] public struct RECT { public int Left; public int Top; public int Right; public int Bottom; }
[DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr hWnd, int command);
[DllImport("user32.dll")] public static extern bool SetWindowPos(IntPtr hWnd, IntPtr after, int x, int y, int width, int height, uint flags);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern void mouse_event(uint flags, uint dx, uint dy, uint data, UIntPtr extra);
}
'@`;
// 桌面自动化驱动(任务 3.3):窗口矩形/鼠标注入原语统一走 WindowsDriver
const driver = defaultEngine.driver;
function value(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
@@ -44,39 +34,60 @@ function requiredTitle(args: Record<string, unknown>, nodeId: string, componentI
return title;
}
function findWindow(title: string): string {
return `$p = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw '找不到窗口:${title.replace(/'/g, "''")}' }`;
}
function parsePoint(candidate: unknown, label: string, nodeId: string, componentId: string): [number, number] {
let parsed: unknown = candidate;
if (typeof parsed === "string") {
const source = parsed;
try { parsed = JSON.parse(source); } catch { parsed = source.split(/[,,xX ]+/).map(Number); }
}
// 屏幕框选/拾取坐标生成的是 {"x":..,"y":..} 对象,直接转换
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
const record = parsed as Record<string, unknown>;
if (record.x !== undefined && record.y !== undefined) parsed = [record.x, record.y];
}
if (!Array.isArray(parsed) || parsed.length < 2 || parsed.slice(0, 2).some((item) => !Number.isFinite(Number(item)))) {
throw new RuntimeError("validation_error", `${label}无效`, { nodeId, componentId });
}
return [Number(parsed[0]), Number(parsed[1])];
}
async function windowObject(title: string): Promise<Record<string, unknown>> {
const output = await powershell(`${WINDOW_API}; ${findWindow(title)}; $r = New-Object RpaWindowExtended+RECT; [RpaWindowExtended]::GetWindowRect($p.MainWindowHandle,[ref]$r) | Out-Null; @{pid=$p.Id;title=$p.MainWindowTitle;hwnd=$p.MainWindowHandle.ToInt64();x=$r.Left;y=$r.Top;width=$r.Right-$r.Left;height=$r.Bottom-$r.Top} | ConvertTo-Json -Compress`);
return JSON.parse(output);
/** 界面(框选/元素拾取/截图)产出的是 DIP 逻辑坐标;DPI 感知进程的 SetCursorPos 用物理像素,
* 缩放屏(125%/150%)下直接混用会成比例偏移(用户实测 125% 下差约 25% 距离)。
* 统一在执行前换算;宿主桥未注入时(vitest)回退原值保持旧行为 */
function dipToPhysical(x: number, y: number): [number, number] {
try {
const bridge = getHostBridge();
if (bridge) {
const physical = bridge.dipToScreenPoint({ x, y });
return [Math.round(physical.x), Math.round(physical.y)];
}
} catch {}
return [Math.round(x), Math.round(y)];
}
const getWindow: CapabilityHandler = async (_ctx, node, args) => windowObject(requiredTitle(args, node.id, node.componentId));
/** 反向换算:物理像素(如 GetWindowRect 结果) → DIP,用于「相对激活窗口」偏移叠加 */
function physicalToDip(x: number, y: number): [number, number] {
try {
const bridge = getHostBridge();
if (bridge) {
const dip = bridge.screenToDipPoint({ x, y });
return [Math.round(dip.x), Math.round(dip.y)];
}
} catch {}
return [Math.round(x), Math.round(y)];
}
const getWindow: CapabilityHandler = async (_ctx, node, args) => driver.window.findByTitle(requiredTitle(args, node.id, node.componentId));
register(["Window_Get", "Window_GetWindow", "Window_GetWindow_v1"], getWindow);
register("Window_ActiveWindow_v1", async (_ctx, node, args) => {
const title = requiredTitle(args, node.id, node.componentId);
const maximize = [true, 1, "1", "true", "True"].includes(args.isMax as never);
await powershell(`${WINDOW_API}; ${findWindow(title)}; [RpaWindowExtended]::SetForegroundWindow($p.MainWindowHandle) | Out-Null; ${maximize ? "[RpaWindowExtended]::ShowWindow($p.MainWindowHandle,3) | Out-Null" : ""}`);
return windowObject(title);
return driver.window.activate(title, { maximize });
});
register(["Window_GetWindowMessage", "Window_GetWindowMessage_v1"], async (_ctx, node, args) => {
const object = await windowObject(requiredTitle(args, node.id, node.componentId));
const object = await driver.window.findByTitle(requiredTitle(args, node.id, node.componentId));
const type = value(args, ["switchType"], "all").toLowerCase();
if (type === "title") return object.title;
if (type === "pid") return object.pid;
@@ -87,7 +98,7 @@ register(["Window_GetWindowMessage", "Window_GetWindowMessage_v1"], async (_ctx,
register(["Window_SetPos", "Window_MoveWindow", "Window_MoveWindow_v1", "Window_SizeWindow"], async (_ctx, node, args) => {
const title = requiredTitle(args, node.id, node.componentId);
const current = await windowObject(title);
const current = await driver.window.findByTitle(title);
const moveOnly = node.componentId.includes("MoveWindow");
const sizeOnly = node.componentId === "Window_SizeWindow";
const position = args.location !== undefined ? parsePoint(args.location, "窗口位置", node.id, node.componentId) : [Number(args.x ?? current.x), Number(args.y ?? current.y)] as [number, number];
@@ -98,36 +109,56 @@ register(["Window_SetPos", "Window_MoveWindow", "Window_MoveWindow_v1", "Window_
const width = moveOnly ? Number(current.width) : size[0];
const height = moveOnly ? Number(current.height) : size[1];
if (!(width > 0 && height > 0)) throw new RuntimeError("validation_error", "窗口尺寸必须大于 0", { nodeId: node.id, componentId: node.componentId });
await powershell(`${WINDOW_API}; ${findWindow(title)}; [RpaWindowExtended]::SetWindowPos($p.MainWindowHandle,[IntPtr]::Zero,${x},${y},${width},${height},4) | Out-Null`);
return { ...(await windowObject(title)), x, y, width, height };
return driver.window.setPos(title, { x, y, width, height });
});
const moveMouse: CapabilityHandler = async (_ctx, node, args) => {
const [x, y] = parsePoint(args.location ?? args.position, "鼠标坐标", node.id, node.componentId);
await powershell(`${WINDOW_API}; [RpaWindowExtended]::SetCursorPos(${x},${y}) | Out-Null`);
return { x, y };
// 注意:position 字段是「坐标相对位置」(0=屏幕左上角/1=激活窗口左上角),坐标本体在 location
const [rawX, rawY] = parsePoint(args.location ?? args.point, "鼠标坐标", node.id, node.componentId);
const relative = String(args.position ?? "0") === "1";
const moveType = String(args.move_type ?? "0");
const speed = Math.min(100, Math.max(1, Number(args.speed ?? 35) || 35));
let dipTarget: [number, number];
if (relative) {
// 相对激活窗口左上角:窗口矩形是物理像素,换回 DIP 再叠加偏移,最后统一转物理
const { x: wx, y: wy } = await driver.window.foregroundPosition();
if (!Number.isFinite(wx) || !Number.isFinite(wy)) throw new RuntimeError("browser_error", "无法获取激活窗口位置", { nodeId: node.id, componentId: node.componentId });
const [dipWx, dipWy] = physicalToDip(wx, wy);
dipTarget = [dipWx + rawX, dipWy + rawY];
} else {
dipTarget = [rawX, rawY];
}
const [tx, ty] = dipToPhysical(dipTarget[0], dipTarget[1]);
if (moveType === "2") {
await driver.mouse.moveTo({ x: tx, y: ty });
} else {
// 平滑移动:从当前光标位置出发,按速度插值(路径全程物理像素,与目标同一坐标系)
await driver.mouse.moveSmooth({ x: tx, y: ty }, { ease: moveType === "0", speed });
}
return { x: Math.round(dipTarget[0]), y: Math.round(dipTarget[1]) };
};
register(["Mouse_MoveToV1", "Mouse_MoveToV2"], moveMouse);
const dragMouse: CapabilityHandler = async (_ctx, node, args) => {
const [sx, sy] = parsePoint(args.Start ?? args.start, "起始坐标", node.id, node.componentId);
const [ex, ey] = parsePoint(args.End ?? args.end, "结束坐标", node.id, node.componentId);
const [spx, spy] = dipToPhysical(sx, sy);
const [epx, epy] = dipToPhysical(ex, ey);
const right = ["1", "right"].includes(value(args, ["side"], "0").toLowerCase());
const down = right ? 8 : 2;
const up = right ? 16 : 4;
await powershell(`${WINDOW_API}; [RpaWindowExtended]::SetCursorPos(${sx},${sy}) | Out-Null; [RpaWindowExtended]::mouse_event(${down},0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 80; [RpaWindowExtended]::SetCursorPos(${ex},${ey}) | Out-Null; [RpaWindowExtended]::mouse_event(${up},0,0,0,[UIntPtr]::Zero)`);
return { start: [sx, sy], end: [ex, ey] };
// 旧实现按下后停 80ms、拖到终点立即松开(与桌面版 Window_Mouse 不同,松开前不停顿)
await driver.mouse.drag({ x: spx, y: spy }, { x: epx, y: epy }, { button: right ? "right" : "left", holdMs: 80, releaseDelayMs: 0 });
return { start: [Math.round(sx), Math.round(sy)], end: [Math.round(ex), Math.round(ey)] };
};
register(["Mouse_DragToV1", "Mouse_DragToV2", "Mouse_DragToNew"], dragMouse);
register(["Mouse_ClickCombine", "Mouse_ClickCombine_v1", "Mouse_ClickCombine_v2", "Mouse_ClickCombine_v3", "Mouse_ClickCombine_v4"], async (_ctx, node, args) => {
if (value(args, ["target"], "0") !== "0") throw new RuntimeError("validation_error", "组合点击当前仅支持坐标目标", { nodeId: node.id, componentId: node.componentId });
const [x, y] = parsePoint(args.location, "点击坐标", node.id, node.componentId);
const [px, py] = dipToPhysical(x, y);
const right = value(args, ["click_key"], "left").toLowerCase() === "right";
const doubleClick = value(args, ["click_time"], "click").toLowerCase().includes("double");
const down = right ? 8 : 2;
const up = right ? 16 : 4;
const repeats = doubleClick ? 2 : 1;
await powershell(`${WINDOW_API}; [RpaWindowExtended]::SetCursorPos(${x},${y}) | Out-Null; 1..${repeats} | ForEach-Object { [RpaWindowExtended]::mouse_event(${down},0,0,0,[UIntPtr]::Zero); [RpaWindowExtended]::mouse_event(${up},0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 80 }`);
return { x, y, button: right ? "right" : "left", clicks: repeats };
// 旧实现每次 down-up 之后停 80ms
await driver.mouse.click({ x: px, y: py }, { button: right ? "right" : "left", double: doubleClick, settleMs: 80 });
return { x: Math.round(x), y: Math.round(y), button: right ? "right" : "left", clicks: doubleClick ? 2 : 1 };
});
@@ -0,0 +1,114 @@
import { register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
const run = promisify(execFile);
const ps = (script: string) => run("powershell.exe", ["-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-Command", script], { windowsHide: true, timeout: 60000 });
// cmdkey 在中文 Windows 上输出 GBK,直接 execFile 按 UTF-8 解码会乱码;统一经 PowerShell 强制 UTF-8 控制台编码捕获
const cmdkey = async (args: string[]): Promise<string> => {
const quoted = args.map((item) => "'" + item.replace(/'/g, "''") + "'");
const script = "[Console]::OutputEncoding=[System.Text.UTF8Encoding]::new($false); & cmdkey " + quoted.join(" ");
const { stdout } = await run("powershell.exe", ["-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-Command", script], { windowsHide: true, timeout: 60000 });
return stdout;
};
function reqText(args: Record<string, unknown>, keys: string[], label: string, nodeId: string): string {
for (const key of keys) {
const v = String(args[key] ?? "").trim();
if (v) return v;
}
throw new RuntimeError("validation_error", label + "为空", { nodeId, componentId: nodeId });
}
register("ScreenControl_Screenlock", async () => {
await run("rundll32.exe", ["user32.dll,LockWorkStation"], { windowsHide: true });
return { locked: true };
});
register("ScreenControl_ScreenUnlock", async () => {
// 仅唤醒屏幕/解除屏保;系统锁屏需人工输入密码,无法通用自动解锁
await ps("Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern void mouse_event(uint f, uint dx, uint dy, uint d, UIntPtr e);' -Name W32U -Namespace WinRpa | Out-Null; [WinRpa.W32U]::mouse_event(0x0001, 1, 1, 0, [UIntPtr]::Zero); Start-Sleep -Milliseconds 200; [WinRpa.W32U]::mouse_event(0x0001, 0xFFFFFFFF, 0, 0, [UIntPtr]::Zero)");
return { awakened: true, loginUnlockSupported: false, note: "仅尝试唤醒屏幕;Windows 登录锁屏仍需本人手动解锁" };
});
const SC_SCREENSAVE = 0xF140;
const WM_SYSCOMMAND = 0x0112;
register(["System_ScreenSaverShow", "System_ScreenSaverShowV1"], async (_ctx, node, args) => {
if ([true, 1, "1", "true", "True"].includes(args.set_password as never)) {
throw new RuntimeError("validation_error", "系统屏保不支持设置自定义解锁密码,请关闭该选项并使用 Windows 原有登录保护", { nodeId: node.id, componentId: node.componentId });
}
await ps("Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern IntPtr SendMessage(IntPtr h, uint m, IntPtr w, IntPtr l);' -Name W32S -Namespace WinRpa | Out-Null; [WinRpa.W32S]::SendMessage([IntPtr]0xFFFF, [uint32]0x0112, [IntPtr]0xF140, [IntPtr]::Zero) | Out-Null");
return { screensaver: true };
});
register("System_ScreenSaverStop", async () => {
await ps("Add-Type -MemberDefinition '[DllImport(\"user32.dll\")] public static extern IntPtr FindWindow(string cls, string win); [DllImport(\"user32.dll\")] public static extern bool PostMessage(IntPtr h, uint m, IntPtr w, IntPtr l);' -Name W32P -Namespace WinRpa | Out-Null; $h = [WinRpa.W32P]::FindWindow('WindowsScreenSaverClass', $null); if ($h -ne [IntPtr]::Zero) { [WinRpa.W32P]::PostMessage($h, 0x0010, [IntPtr]::Zero, [IntPtr]::Zero) | Out-Null }");
return { stopped: true };
});
register("System_ReadCredential", async (ctx, node, args) => {
const app = reqText(args, ["application"], "凭据目标", node.id);
const stdout = await cmdkey(["/list"]);
const exists = stdout.includes(app);
// cmdkey /list 每条凭据是「目标/类型/用户」多行块(空行分隔);取整块才有用户名等有效信息
const lines = stdout.split(/\r?\n/);
const idx = lines.findIndex((l) => l.includes(app));
const block: string[] = [];
if (idx >= 0) {
for (let i = idx; i < lines.length && lines[i].trim(); i++) block.push(lines[i].trim());
}
return { target: app, exists, detail: exists ? block.join(" | ") : "" };
});
register("System_AddCredential", async (ctx, node, args) => {
const app = reqText(args, ["application"], "凭据目标", node.id);
const user = reqText(args, ["name"], "用户名", node.id);
const pwd = String(args.password ?? "");
const psArgs = pwd ? ["/generic:" + app, "/user:" + user, "/pass:" + pwd] : ["/generic:" + app, "/user:" + user];
await cmdkey(psArgs);
return { added: true, target: app };
});
register("System_DeleteCredential", async (ctx, node, args) => {
const app = reqText(args, ["application"], "凭据目标", node.id);
await cmdkey(["/delete:" + app]);
return { deleted: true, target: app };
});
register("System_ConnectRemoteDesktop", async (ctx, node, args) => {
const ip = reqText(args, ["ip"], "远程地址", node.id);
const port = String(args.port ?? "").trim();
const target = port ? ip + ":" + port : ip;
await run("mstsc.exe", ["/v:" + target], { windowsHide: true });
return { connected: true, target };
});
register("System_CloseRemoteDesktop", async () => {
await run("tsdiscon.exe", [], { windowsHide: true });
return { disconnected: true };
});
register("System_GetInputMethodState", async () => {
const { stdout } = await ps("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $cur = [System.Windows.Forms.InputLanguage]::CurrentInputLanguage; Write-Output ($cur.Culture.Name + '|' + $cur.LayoutName)");
const [culture, layout] = stdout.split("|");
return { culture, layout };
});
register("System_SetInputMethodState", async (ctx, node, args) => {
const mode = reqText(args, ["mode"], "输入法文化名", node.id);
await ps("Add-Type -AssemblyName System.Windows.Forms | Out-Null; $lang = [System.Windows.Forms.InputLanguage]::FromCulture('" + mode.replace(/'/g, "''") + "'); if ($lang) { [System.Windows.Forms.InputLanguage]::CurrentInputLanguage = $lang }");
return { set: true, mode };
});
register("File_OpenFile", async (ctx, node, args) => {
const p = reqText(args, ["path"], "文件夹路径", node.id);
try {
await run("explorer.exe", [p], { windowsHide: true });
} catch (error) {
// explorer.exe 打开成功也会返回退出码 1(Windows 已知行为),仅路径不存在时才是真失败
const code = (error as { code?: unknown }).code;
if (code !== 1) throw error;
}
return { opened: true, path: p };
});
@@ -2,7 +2,7 @@ import { mkdtemp, readFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import type { CatalogComponent, FlowNode } from "../../shared/types";
import type { CatalogComponent, FlowNode } from "../types";
import type { RunContext } from "../engine/types";
import { getHandler } from "../engine/registry";
import "./word";
@@ -15,7 +15,7 @@ const node = (id: string): FlowNode => ({ id, componentId: id, position: { x: 0,
async function run(id: string, args: Record<string, unknown> = {}) { const handler = getHandler(id); expect(handler, id).toBeTruthy(); return handler!(context, node(id), args, component); }
afterEach(async () => Promise.all(directories.splice(0).map((directory) => rm(directory, { recursive: true, force: true }))));
const ids = ["Word_New_V1", "Word_GetPath", "Word_ReadAll", "Word_Replace", "Word_Enter", "Word_InsertPage", "Word_SetFontName", "Word_SetFontSize", "Word_SetFontStyle", "Word_SetFontColor", "Word_SetAlign", "Word_InsertPicture", "Word_WriteTable", "Word_Close"];
const ids = ["Word_New_V1", "Word_GetPath", "Word_ReadAll", "Word_Replace", "Word_Enter", "Word_InsertPage", "Word_SetFontName", "Word_SetFontSize", "Word_SetFontStyle", "Word_SetFontColor", "Word_SetAlign", "Word_InsertPicture", "Word_WriteTable", "Word_Close", "Word_ReadSelect", "Word_CutSelect", "Word_CopySelect", "Word_PasteSelect", "Word_DeleteSelect", "Word_SelectAll", "Word_SelectRow", "Word_SetPosition", "Word_MovePosition", "Word_PositionToSelect", "Word_ReadTable"];
describe("Word extended capabilities", () => {
it("registers automated Word operations", () => { for (const id of ids) expect(getHandler(id), id).toBeTruthy(); });
@@ -32,5 +32,28 @@ describe("Word extended capabilities", () => {
await run("Word_SetFontStyle", { objWord: handle, isBold: "True", isItalic: "False", isUnderline: "False" });
await run("Word_Enter", { objWord: handle });
expect((await readFile(path.join(directory, "extended.docx"))).byteLength).toBeGreaterThan(1000);
}, 60000);
}, 120000);
it("uses a persistent selection and reads a Word table through COM", async () => {
if (process.platform !== "win32") return;
const directory = await mkdtemp(path.join(os.tmpdir(), "rpa-word-selection-")); directories.push(directory);
let handle = await run("Word_New_V1", { filePath: directory, name: "selection.docx" });
await run("Word_Write", { objWord: handle, text: "alpha beta gamma" });
handle = await run("Word_PositionToSelect", { objWord: handle, text: "beta", pos_index: 1, direction: "select" });
await expect(run("Word_ReadSelect", { objWord: handle })).resolves.toBe("beta");
handle = await run("Word_CopySelect", { objWord: handle });
handle = await run("Word_MovePosition", { objWord: handle, length: 1, unit: "word", direction: "right", isSelect: false });
await run("Word_PasteSelect", { objWord: handle });
handle = await run("Word_PositionToSelect", { objWord: handle, text: "gamma", direction: "select" });
handle = await run("Word_CutSelect", { objWord: handle });
await expect(run("Word_ReadSelect", { objWord: handle })).resolves.toBe("");
await run("Word_DeleteSelect", { objWord: handle });
handle = await run("Word_SetPosition", { objWord: handle, length: -2, unit: "word" });
handle = await run("Word_SelectRow", { objWord: handle, st: 1, ed: 1 });
await expect(run("Word_ReadSelect", { objWord: handle })).resolves.toContain("alpha");
handle = await run("Word_SelectAll", { objWord: handle });
await expect(run("Word_ReadSelect", { objWord: handle })).resolves.toContain("alpha beta beta");
await run("Word_WriteTable", { objWord: handle, data: [["name", "age"], ["Alice", "30"]] });
await expect(run("Word_ReadTable", { objWord: handle, find_type: "content", table_content: "Alice" })).resolves.toEqual([["name", "age"], ["Alice", "30"]]);
}, 180000);
});
@@ -0,0 +1,264 @@
import { promises as fs } from "node:fs";
import path from "node:path";
import { getHandler, register } from "../engine/registry";
import { RuntimeError } from "../engine/types";
import { powershell, psString } from "../driver/windows/powershell";
interface WordHandle { path: string; }
function text(args: Record<string, unknown>, keys: string[], fallback = ""): string {
for (const key of keys) {
const current = args[key];
if (current !== undefined && current !== null && current !== "") return String(current);
}
return fallback;
}
function bool(value: unknown): boolean {
return [true, 1, "1", "true", "True", "是", "开启"].includes(value as never);
}
function handlePath(args: Record<string, unknown>, nodeId: string, componentId: string): string {
const raw = args.objWord ?? args.filePath ?? args.path;
const candidate = raw && typeof raw === "object" ? (raw as Partial<WordHandle>).path : raw;
const result = String(candidate || "").trim();
if (!result || result === "word对象") throw new RuntimeError("validation_error", "Word文档路径为空", { nodeId, componentId });
return path.resolve(result);
}
function script(filePath: string, body: string, save = true): string {
return `$word=$null;$doc=$null;try{$word=New-Object -ComObject Word.Application;$word.Visible=$false;$word.DisplayAlerts=0;$doc=$word.Documents.Open(${psString(filePath)});${body};${save ? "$doc.Save();" : ""}}finally{if($doc -ne $null){try{$doc.Close()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($doc)}catch{}};if($word -ne $null){try{$word.Quit()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($word)}catch{}};[GC]::Collect();[GC]::WaitForPendingFinalizers()}`;
}
register("Word_New_V1", async (ctx, node, args, component) => {
const base = getHandler("Word_New");
if (!base) throw new RuntimeError("capability_not_implemented", "基础 Word 新建能力不可用", { nodeId: node.id, componentId: node.componentId });
return base(ctx, node, args, component);
});
register("Word_GetPath", (_ctx, node, args) => handlePath(args, node.id, node.componentId));
register("Word_ReadAll", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const encoded = await powershell(script(filePath, "[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($doc.Content.Text))", false));
return Buffer.from(encoded.trim(), "base64").toString("utf8");
});
register("Word_Replace", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const oldText = text(args, ["oldText"]);
if (!oldText) throw new RuntimeError("validation_error", "原内容为空", { nodeId: node.id, componentId: node.componentId });
const newText = text(args, ["newText"]);
await powershell(script(filePath, `$find=$doc.Content.Find;[void]$find.Execute(${psString(oldText)},$false,$false,$false,$false,$false,$true,1,$false,${psString(newText)},2)`));
return { path: filePath } satisfies WordHandle;
});
register("Word_Enter", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
await powershell(script(filePath, "$range=$doc.Content;$range.Collapse(0);$range.InsertParagraphAfter()"));
return { path: filePath } satisfies WordHandle;
});
register("Word_InsertPage", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
await powershell(script(filePath, "$range=$doc.Content;$range.Collapse(0);$range.InsertBreak(7)"));
return { path: filePath } satisfies WordHandle;
});
register("Word_SetFontName", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
await powershell(script(filePath, `$doc.Content.Font.Name=${psString(text(args, ["font"], "宋体"))}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_SetFontSize", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const size = Number(text(args, ["fontSize"], "9"));
if (!(size > 0)) throw new RuntimeError("validation_error", "字体大小无效", { nodeId: node.id, componentId: node.componentId });
await powershell(script(filePath, `$doc.Content.Font.Size=${size}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_SetFontStyle", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
await powershell(script(filePath, `$doc.Content.Font.Bold=${bool(args.isBold) ? 1 : 0};$doc.Content.Font.Italic=${bool(args.isItalic) ? 1 : 0};$doc.Content.Font.Underline=${bool(args.isUnderline) ? 1 : 0}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_SetFontColor", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const hex = text(args, ["fontColor"], "#000000").replace("#", "");
if (!/^[0-9a-fA-F]{6}$/.test(hex)) throw new RuntimeError("validation_error", "字体颜色格式无效", { nodeId: node.id, componentId: node.componentId });
const red = Number.parseInt(hex.slice(0, 2), 16); const green = Number.parseInt(hex.slice(2, 4), 16); const blue = Number.parseInt(hex.slice(4, 6), 16);
await powershell(script(filePath, `$doc.Content.Font.Color=${red + green * 256 + blue * 65536}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_SetAlign", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const align = text(args, ["align"], "left").toLowerCase();
const value = align.includes("center") || align === "1" ? 1 : align.includes("right") || align === "2" ? 2 : align.includes("justify") || align === "3" ? 3 : 0;
await powershell(script(filePath, `$doc.Content.ParagraphFormat.Alignment=${value}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_InsertPicture", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const image = path.resolve(text(args, ["imageaddr"]));
await fs.access(image);
await powershell(script(filePath, `$range=$doc.Content;$range.Collapse(0);[void]$doc.InlineShapes.AddPicture(${psString(image)},$false,$true,$range)`));
return { path: filePath } satisfies WordHandle;
});
register("Word_WriteTable", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
let data: unknown = args.data;
if (typeof data === "string") { try { data = JSON.parse(data); } catch { data = [[data]]; } }
const rows = Array.isArray(data) ? data.map((row) => Array.isArray(row) ? row : [row]) : [[data]];
const columns = Math.max(1, ...rows.map((row) => row.length));
const encoded = Buffer.from(JSON.stringify(rows), "utf8").toString("base64");
await powershell(script(filePath, `$json=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String(${psString(encoded)}));$data=$json|ConvertFrom-Json;$range=$doc.Content;$range.Collapse(0);$table=$doc.Tables.Add($range,${rows.length},${columns});for($r=0;$r-lt${rows.length};$r++){for($c=0;$c-lt${columns};$c++){if($c-lt$data[$r].Count){$table.Cell($r+1,$c+1).Range.Text=[string]$data[$r][$c]}}};$table.Borders.Enable=${bool(args.is_border ?? "1") ? 1 : 0}`));
return { path: filePath } satisfies WordHandle;
});
register("Word_Close", (_ctx, node, args) => ({ path: handlePath(args, node.id, node.componentId), closed: true }));
interface WordSelection { start: number; end: number; }
function selectionOf(args: Record<string, unknown>): WordSelection {
const handle = args.objWord;
const raw = handle && typeof handle === "object" ? (handle as { selection?: unknown }).selection : null;
const record = raw && typeof raw === "object" ? raw as Record<string, unknown> : {};
let start = Math.max(0, Math.floor(Number(record.start ?? 0) || 0));
let end = Math.max(0, Math.floor(Number(record.end ?? record.active ?? start) || 0));
if (end < start) [start, end] = [end, start];
return { start, end };
}
function selectionScript(filePath: string, selection: WordSelection, body = "", withText = true): string {
return `$word=$null;$doc=$null;try{` +
`$word=New-Object -ComObject Word.Application;$word.Visible=$false;$word.DisplayAlerts=0;$doc=$word.Documents.Open(${psString(filePath)});` +
`$sel=$doc.Range(${selection.start},${selection.end});$sel.Select()|Out-Null;${body};if($null -eq $sel){$sel=$doc.Content}` +
`$text="";if(${withText ? "$true" : "$false"}){$text=[string]$sel.Text};$out=@{start=[int]$sel.Start;end=[int]$sel.End;text=$text}|ConvertTo-Json -Compress;` +
`[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($out))|Write-Output;$doc.Save()` +
`}finally{if($doc -ne $null){try{$doc.Close()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($doc)}catch{}};if($word -ne $null){try{$word.Quit()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($word)}catch{}};[GC]::Collect();[GC]::WaitForPendingFinalizers()}`;
}
async function runSelection(filePath: string, selection: WordSelection, body = "", withText = true): Promise<WordSelection & { text: string }> {
const output = await powershell(selectionScript(filePath, selection, body, withText));
if (output.includes("__WORD_NOT_FOUND__")) throw new RuntimeError("word_text_not_found", "Word 查找文本不存在", { detail: `path=${filePath}` });
const parsed = JSON.parse(Buffer.from(output.trim(), "base64").toString("utf8")) as { start: number; end: number; text: string };
return { start: parsed.start, end: parsed.end, text: parsed.text };
}
function selectionResult(filePath: string, selection: WordSelection): { path: string; selection: WordSelection } {
return { path: filePath, selection };
}
const selectionText = async (_ctx: unknown, node: { id: string; componentId: string }, args: Record<string, unknown>): Promise<string> =>
(await runSelection(handlePath(args, node.id, node.componentId), selectionOf(args))).text;
const selectionMutation = (body: string): CapabilityHandlerLike =>
async (_ctx, node, args) => selectionResult(handlePath(args, node.id, node.componentId), await runSelection(handlePath(args, node.id, node.componentId), selectionOf(args), body));
type CapabilityHandlerLike = Parameters<typeof register>[1];
register("Word_ReadSelect", selectionText);
register("Word_CopySelect", selectionMutation("$sel.Copy()|Out-Null"));
register("Word_CutSelect", selectionMutation("$sel.Cut()|Out-Null"));
register("Word_DeleteSelect", selectionMutation("[void]$sel.Delete()"));
register("Word_PasteSelect", selectionMutation("$sel.Paste()|Out-Null"));
register("Word_SelectAll", selectionMutation("$sel=$doc.Content;$sel.Select()|Out-Null"));
register("Word_SelectRow", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const startRow = Math.max(1, Math.floor(Number(args.st ?? 1) || 1));
const endRow = Math.max(startRow, Math.floor(Number(args.ed ?? args.start ?? startRow) || startRow));
const body = [
`$first=$doc.GoTo(3,1,${startRow}).Bookmarks("\\Line").Range`,
`$last=$doc.GoTo(3,1,${endRow}).Bookmarks("\\Line").Range`,
"$sel=$doc.Range($first.Start,$last.End);$sel.Select()|Out-Null"
].join(";");
return selectionResult(filePath, await runSelection(filePath, { start: 0, end: 0 }, body));
});
const wordUnit = (value: unknown): number => {
const unit = String(value ?? "word").toLowerCase();
return unit === "row" || unit === "line" ? 5 : unit === "section" || unit === "paragraph" ? 4 : 2;
};
register("Word_SetPosition", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const count = Math.floor(Number(args.length ?? 1) || 0);
const unit = wordUnit(args.unit);
const body = `$anchor=if(${selectionOf(args).end > selectionOf(args).start ? "$true" : "$false"}){$sel.End}else{$sel.Start};$sel=$doc.Range($anchor,$anchor);[void]$sel.Move(${unit},${count})`;
return selectionResult(filePath, await runSelection(filePath, selectionOf(args), body));
});
register("Word_MovePosition", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const count = Math.floor(Number(args.length ?? 1) || 0);
const unit = wordUnit(args.unit);
const left = String(args.direction ?? "right").toLowerCase().includes("left");
const countValue = left ? -Math.abs(count) : Math.abs(count);
const extend = bool(args.isSelect ?? args.shift ?? false);
const selection = selectionOf(args);
const anchor = left ? selection.start : selection.end;
const body = extend
? `$sel=$doc.Range(${anchor},${anchor});if(${left ? "$true" : "$false"}){[void]$sel.MoveStart(${unit},${countValue})}else{[void]$sel.MoveEnd(${unit},${countValue})}`
: `$sel=$doc.Range(${anchor},${anchor});[void]$sel.Move(${unit},${countValue})`;
return selectionResult(filePath, await runSelection(filePath, selection, body));
});
register("Word_PositionToSelect", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const needle = text(args, ["text", "find", "value"]);
if (!needle) throw new RuntimeError("validation_error", "查找文本为空", { nodeId: node.id, componentId: node.componentId });
const encodedNeedle = Buffer.from(needle, "utf8").toString("base64");
const requested = Math.floor(Number(args.pos_index ?? 1) || 1);
const mode = String(args.direction ?? "left").toLowerCase();
const body = [
`$needle=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('${encodedNeedle}'))`,
"$search=$doc.Content;$all=@();while($true){",
" $candidate=$search.Duplicate;$found=$candidate.Find.Execute($needle)",
" if(-not $found){break}",
" $all+=$candidate.Duplicate",
" if($candidate.End -ge $search.End){break}",
" $search.SetRange($candidate.End,$search.End)",
"}",
"if($all.Count -eq 0){Write-Output '__WORD_NOT_FOUND__';exit 0}",
`$ix=${requested};if($ix -le 0){$ix=$all.Count+$ix+1}`,
"if($ix -lt 1){$ix=1};if($ix -gt $all.Count){$ix=$all.Count};$match=$all[$ix-1]",
mode === "select"
? "$sel=$match.Duplicate;$sel.Select()|Out-Null"
: `$p=$match.${mode.includes("right") ? "End" : "Start"};$sel=$doc.Range($p,$p);$sel.Select()|Out-Null`
].join(";");
return selectionResult(filePath, await runSelection(filePath, { start: 0, end: 0 }, body));
});
register("Word_ReadTable", async (_ctx, node, args) => {
const filePath = handlePath(args, node.id, node.componentId);
const byContent = String(args.find_type ?? "index").toLowerCase().includes("content");
const tableIndex = Math.max(1, Math.floor(Number(args.table_index ?? 1) || 1));
const content = String(args.table_content ?? "");
const encodedContent = Buffer.from(content, "utf8").toString("base64");
const selector = byContent
? `$needle=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('${encodedContent}'));$tableIndex=0;for($i=1;$i -le $doc.Tables.Count;$i++){if($doc.Tables.Item($i).Range.Text.Contains($needle)){$tableIndex=$i;break}}`
: `$tableIndex=${tableIndex}`;
const body = [
selector,
"if($tableIndex -lt 1 -or $tableIndex -gt $doc.Tables.Count){Write-Output '__WORD_TABLE_NOT_FOUND__';exit 0}",
"$table=$doc.Tables.Item($tableIndex);$rows=@()",
"for($r=1;$r -le $table.Rows.Count;$r++){ $row=@(); for($c=1;$c -le $table.Columns.Count;$c++){ try { $text=$table.Cell($r,$c).Range.Text } catch { $text='' }; $clean=''; foreach($ch in $text.ToCharArray()){ $code=[int]$ch; if($code -ne 13 -and $code -ne 7){$clean += $ch} }; $row += $clean }; $rows += ,$row }",
"$json=$rows|ConvertTo-Json -Compress -Depth 5",
"[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($json))|Write-Output"
].join(";");
const output = await powershell(`$word=$null;$doc=$null;try{$word=New-Object -ComObject Word.Application;$word.Visible=$false;$word.DisplayAlerts=0;$doc=$word.Documents.Open(${psString(filePath)});${body}}finally{if($doc -ne $null){try{$doc.Close()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($doc)}catch{}};if($word -ne $null){try{$word.Quit()}catch{};try{[void][Runtime.InteropServices.Marshal]::ReleaseComObject($word)}catch{}};[GC]::Collect();[GC]::WaitForPendingFinalizers()}`);
if (output.includes("__WORD_TABLE_NOT_FOUND__")) throw new RuntimeError("word_table_not_found", "Word 表格不存在", { detail: `path=${filePath};tableIndex=${tableIndex};content=${content}` });
const parsed = JSON.parse(Buffer.from(output.trim(), "base64").toString("utf8"));
const rows = Array.isArray(parsed) ? parsed : [parsed];
return rows.map((row) => {
const value = row && typeof row === "object" && Array.isArray((row as { value?: unknown }).value) ? (row as { value: unknown[] }).value : row;
return Array.isArray(value) ? value : [value];
});
});
@@ -1,6 +1,6 @@
import path from "node:path";
import { register } from "../engine/registry";
import { powershell, psString } from "./desktop";
import { powershell, psString } from "../driver/windows/powershell";
interface WordHandle { path: string; }
+165
View File
@@ -0,0 +1,165 @@
/**
* AutomationDriver 驱动抽象(任务 3.2,决策点 6:五域接口 + 机械包一层)。
*
* 接口签名以现有 PowerShell 能力的输入输出为准,不重新发明:
* WindowsDriver(driver/windows/windows-driver.ts)是 PowerShell 实现,行为与
* 改造前完全一致;引擎创建时注入 driver(createEngine 可注入自定义实现)。
*
* 约定:driver 只承载「原始注入原语」(单个 SendKeys/一次 SetCursorPos/一次
* 窗口操作);带拟真抖动、坐标换算、多步组合的「策略」仍留在能力文件,
* 保证机械替换不改任何时序行为。
*/
/** UIA 选择器(element 域类型,能力与 driver 共用) */
export interface UiaSelector {
window?: string;
name?: string;
id?: string;
/** .NET ControlType 字段名(已映射,如 ComboBox/ListItem/DataGrid) */
type?: string;
/** 用户输入的原始类型字符串 */
typeRaw?: string;
className?: string;
screen?: { x: number; y: number; w: number; h: number };
/** Relative physical rect inside the top-level window. */
rel?: { x: number; y: number; w: number; h: number };
process?: string;
winClass?: string;
enriched?: boolean;
aiType?: string;
aiText?: string;
ai?: boolean;
/** 从窗口(或桌面根)到目标元素的祖先路径,逐段用稳定属性定位 */
path?: Array<{ type?: string; name?: string; id?: string; className?: string; index?: number }>;
index: number;
}
export interface DriverPoint {
x: number;
y: number;
}
export interface DriverRect {
x: number;
y: number;
width: number;
height: number;
}
export interface WindowInfo extends DriverRect {
pid: number;
title: string;
hwnd: number;
}
export interface WindowSummary {
pid: number;
title: string;
hwnd: number;
}
export type MouseButton = "left" | "right";
export interface MouseClickOptions {
button?: MouseButton;
double?: boolean;
/** 每次 down-up 后的停顿毫秒(0 = 不停顿,保持旧桌面能力的紧凑时序) */
settleMs?: number;
}
export interface MouseDragOptions {
button?: MouseButton;
/** 按下后的按住毫秒 */
holdMs?: number;
/** 拖到终点后、松开前的停顿毫秒 */
releaseDelayMs?: number;
}
export interface MouseSmoothOptions {
/** true=缓入缓出+轻微抖动(拟真),false=匀速直线 */
ease?: boolean;
/** 1~100,越大越快 */
speed?: number;
}
/** mouse 域:坐标一律为物理像素(DIP 换算属于能力层策略) */
export interface MouseDriver {
moveTo(point: DriverPoint): Promise<void>;
click(point: DriverPoint, options?: MouseClickOptions): Promise<void>;
drag(from: DriverPoint, to: DriverPoint, options?: MouseDragOptions): Promise<void>;
scroll(amount: number): Promise<void>;
/** 从当前光标位置出发按速度插值移动到目标 */
moveSmooth(target: DriverPoint, options?: MouseSmoothOptions): Promise<void>;
}
export interface KeyboardSendOptions {
keys: string;
intervalMs: number;
/** api=后台接口整段发送 / manual=逐字符拟真 / clipboard=剪贴板粘贴 */
mode: "api" | "manual" | "clipboard";
/** 文本类组件做 SendKeys 字面转义;按键组合组件原样发送 */
textMode: boolean;
/** 目标窗口标题(非空时先多招式激活再输入,与旧行为一致) */
activateTitle?: string;
}
/** keyboard 域 */
export interface KeyboardDriver {
send(options: KeyboardSendOptions): Promise<void>;
}
export interface WindowDriver {
/** 按标题模糊匹配窗口并返回矩形信息(GetWindowRect,物理像素) */
findByTitle(title: string): Promise<WindowInfo>;
/** 简单激活(SetForegroundWindow + 可选最大化),返回窗口信息 */
activate(title: string, options?: { maximize?: boolean }): Promise<WindowInfo>;
/** 多招式前台激活(失败抛错),返回窗口主标题 */
focus(title: string): Promise<string>;
/** 前台窗口标题 */
foregroundTitle(): Promise<string>;
/** 前台窗口矩形左上角(物理像素) */
foregroundPosition(): Promise<DriverPoint>;
exists(title: string): Promise<boolean>;
/** CloseMainWindow 结果("True"/"False") */
close(title: string): Promise<string>;
/** ShowWindow 命令(6=最小化 3=最大化 9=恢复 5=显示 0=隐藏) */
setState(title: string, showCommand: number): Promise<void>;
/** 只改尺寸,保持左上角不动 */
setSize(title: string, width: number, height: number): Promise<void>;
setPos(title: string, rect: DriverRect): Promise<WindowInfo>;
list(filter?: { title?: string; regex?: boolean }): Promise<WindowSummary[]>;
}
/** screen 域(截图等 System.Drawing 组合操作不在接口面内,能力层可继续用 powershell()) */
export interface ScreenDriver {
/** 主屏边界(System.Windows.Forms.Screen.PrimaryScreen.Bounds) */
size(): Promise<{ x: number; y: number; width: number; height: number }>;
}
export interface ElementFindOptions {
timeoutMs?: number;
gapMs?: number;
}
/**
* element 域(UIA)。PowerShell 实现承载在 capabilities/desktop-uia.ts
* (ground 原语集),WindowsDriver.element 委托该实现;能力 handler 层
* 到 driver 形式的收口在阶段 5/6 按需进行,避免行为回归。
*/
export interface ElementDriver {
parseSelector(raw: unknown): UiaSelector;
find(selector: UiaSelector, options?: ElementFindOptions): Promise<Record<string, unknown> | null>;
exists(selector: UiaSelector): Promise<boolean>;
click(selector: UiaSelector, options?: ElementFindOptions): Promise<void>;
setValue(selector: UiaSelector, value: string): Promise<void>;
getValue(selector: UiaSelector): Promise<string | Record<string, unknown> | null>;
}
/** 五域聚合,注入点(RpaEngine.driver) */
export interface AutomationDriver {
readonly mouse: MouseDriver;
readonly keyboard: KeyboardDriver;
readonly window: WindowDriver;
readonly screen: ScreenDriver;
readonly element: ElementDriver;
}
@@ -0,0 +1,74 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { RuntimeError } from "../../engine/types";
/**
* PowerShell 进程助手(任务 3.1 从 capabilities/desktop.ts 拆出)。
*
* 包内所有 PowerShell 调用必须经此助手(开发规范 v1 第二节第 3 条),
* 禁止各能力文件自行 execFile("powershell.exe", ...);
* 阶段 3.5(可选)将统一切换为常驻 worker,接口签名不变。
*/
const execFileAsync = promisify(execFile);
function encoded(command: string): string {
return Buffer.from(command, "utf16le").toString("base64");
}
/** 从 PowerShell 错误输出里提取 CLIXML 段的真实错误文本(否则显示为乱码转义序列) */
function decodeCliXml(detail: string): string {
const lines = [...detail.matchAll(/<S S="Error">(.*?)<\/S>/gs)].map((m) => m[1]);
if (lines.length === 0) return detail;
// _xHHHH_ 是 CLIXML 转义
const decoded = lines.join("\n").replace(/_x([0-9A-Fa-f]{4})_/g, (_, h) => String.fromCharCode(parseInt(h, 16)));
const parts = decoded.split("\n").map((l) => l.trim());
const position = parts.filter((l) => l.startsWith("所在位置")).slice(0, 1);
const message = parts.filter(
(l) => l && !l.startsWith("所在位置") && !l.startsWith("+") && !l.startsWith("~~") && !l.includes("CategoryInfo") && !l.includes("FullyQualifiedErrorId") && !l.startsWith("<Objs") && !l.startsWith("#<")
);
const main = message[0] || parts[0] || detail;
return position.length ? `${main}(${position[0]})` : main;
}
/** 管道字节流解码:优先 UTF-8;出现替换符说明是系统代码页(GBK)写入的(如解析类错误不走 UTF-8 前缀),回退 GB18030 */
function decodePipe(buf: Buffer): string {
const utf8 = buf.toString("utf8");
if (!utf8.includes("\uFFFD")) return utf8;
try {
return new TextDecoder("gb18030").decode(buf);
} catch {
return utf8;
}
}
export async function powershell(command: string, timeout = 30000): Promise<string> {
try {
// Windows PowerShell 的默认输出编码依机器代码页变化;Node 按 UTF-8 解码管道。
const utf8 = "$ProgressPreference='SilentlyContinue'; [Console]::OutputEncoding=[System.Text.UTF8Encoding]::new($false); $OutputEncoding=[Console]::OutputEncoding; ";
const result = await execFileAsync("powershell.exe", ["-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-EncodedCommand", encoded(utf8 + command)], {
windowsHide: true,
timeout,
maxBuffer: 4 * 1024 * 1024,
encoding: "buffer"
} as never);
return decodePipe(result.stdout as unknown as Buffer).trim();
} catch (error) {
const err = error as Error & { stdout?: unknown; stderr?: unknown };
// 脚本主动失败(RpaFail)把完整错误写在 stdout 的 RPA_ERROR:: 标记后,不经 CLIXML 渲染,无截断
const stdoutText = Buffer.isBuffer(err.stdout) ? decodePipe(err.stdout) : String(err.stdout ?? "");
const marked = stdoutText.match(/RPA_ERROR::([\s\S]*)/);
if (marked) {
const detail = marked[1].trim();
throw new RuntimeError("desktop_command_failed", `桌面操作失败:${detail}`, { detail });
}
const stderrBuf = Buffer.isBuffer(err.stderr) ? err.stderr : Buffer.from(String(err.stderr ?? ""), "utf8");
const stderrText = stderrBuf.length > 0 ? decodePipe(stderrBuf) : err.message;
const detail = decodeCliXml(stderrText);
throw new RuntimeError("desktop_command_failed", `桌面操作失败:${detail}`, { detail });
}
}
export function psString(value: unknown): string {
return `'${String(value ?? "").replace(/'/g, "''")}'`;
}
@@ -0,0 +1,272 @@
import type {
AutomationDriver,
DriverPoint,
DriverRect,
ElementDriver,
KeyboardSendOptions,
MouseClickOptions,
MouseDragOptions,
MouseSmoothOptions,
MouseDriver,
KeyboardDriver,
ScreenDriver,
WindowDriver,
WindowInfo,
WindowSummary
} from "../types";
import { powershell, psString } from "./powershell";
/**
* WindowsDriver(任务 3.3):AutomationDriver 的 PowerShell 实现。
* 脚本与改造前各能力文件内的片段逐字一致(仅 Add-Type 块收拢到本文件),
* 行为不变;坐标一律物理像素,DIP 换算等策略留在能力层。
*/
/** 基础注入原语(原 capabilities/desktop.ts 的 USER32 块,补 SetProcessDpiAwarenessContext:
* 原 window-extended.ts 的 WINDOW_API 在同一进程内已设置 per-monitor DPI 感知,
* 驱动统一携带该声明,保证 SetCursorPos 全程物理像素语义一致) */
const USER32 = `Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class RpaUser32 {
[DllImport("user32.dll")] public static extern bool SetCursorPos(int X, int Y);
[DllImport("user32.dll")] public static extern void mouse_event(uint flags, uint dx, uint dy, uint data, UIntPtr extra);
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll", CharSet=CharSet.Unicode)] public static extern int GetWindowText(IntPtr hWnd, System.Text.StringBuilder text, int count);
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value);
}
'@; try { [void][RpaUser32]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}`;
/** 窗口矩形/鼠标原语(原 capabilities/window-extended.ts 的 WINDOW_API 块) */
const WINDOW_API = `Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class RpaWindowExtended {
[StructLayout(LayoutKind.Sequential)] public struct RECT { public int Left; public int Top; public int Right; public int Bottom; }
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr hWnd, int command);
[DllImport("user32.dll")] public static extern bool SetWindowPos(IntPtr hWnd, IntPtr after, int x, int y, int width, int height, uint flags);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT point);
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")] public static extern void mouse_event(uint flags, uint dx, uint dy, uint data, UIntPtr extra);
[DllImport("user32.dll")] public static extern bool SetProcessDpiAwarenessContext(IntPtr value);
}
'@; try { [void][RpaWindowExtended]::SetProcessDpiAwarenessContext([IntPtr](-4)) } catch {}`;
function findWindowScript(title: string): string {
return `$p = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw '找不到窗口:${title.replace(/'/g, "''")}' }`;
}
async function windowObject(title: string): Promise<WindowInfo> {
const output = await powershell(`${WINDOW_API}; ${findWindowScript(title)}; $r = New-Object RpaWindowExtended+RECT; [RpaWindowExtended]::GetWindowRect($p.MainWindowHandle,[ref]$r) | Out-Null; @{pid=$p.Id;title=$p.MainWindowTitle;hwnd=$p.MainWindowHandle.ToInt64();x=$r.Left;y=$r.Top;width=$r.Right-$r.Left;height=$r.Bottom-$r.Top} | ConvertTo-Json -Compress`);
return JSON.parse(output);
}
/** 把匹配标题的窗口带到前台;多招式依次兜底(应用点击「运行」后自身是前台,
* Windows 前台锁定策略会拒绝普通 SetForegroundWindow,最小化再恢复可绕过)。
* 失败用 RPA_ERROR 标记写 stdout 后 exit 1:不走 throw/CLIXML,避免控制台宽度截断和编码歧义 */
function activateWindowScript(selector: string): string {
const safeTitle = selector.replace(/'/g, "''");
return `function RpaFail($m) { [Console]::Out.WriteLine('RPA_ERROR::' + $m); exit 1 }
$p = Get-Process | Where-Object { $_.MainWindowTitle -and $_.MainWindowTitle -like ${psString(`*${selector}*`)} } | Select-Object -First 1
if (-not $p) { $vis = (Get-Process | Where-Object { $_.MainWindowTitle } | Select-Object -First 12 | ForEach-Object { '『' + $_.MainWindowTitle + '』' }) -join ' '; RpaFail ('找不到窗口:${safeTitle}。当前可见窗口标题: ' + $vis) }
Add-Type -MemberDefinition '[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow(); [DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr h, int cmd); [DllImport("user32.dll")] public static extern bool IsIconic(IntPtr h); [DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h); [DllImport("user32.dll")] public static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, UIntPtr dwExtraInfo); [DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr hWnd, IntPtr pid); [DllImport("user32.dll")] public static extern bool AttachThreadInput(uint a, uint b, bool attach); [DllImport("user32.dll")] public static extern void SwitchToThisWindow(IntPtr hWnd, bool fAltTab); [DllImport("kernel32.dll")] public static extern uint GetCurrentThreadId();' -Name RpaFocus2 -Namespace WinRpaFocus2 | Out-Null
$h = $p.MainWindowHandle
if ([WinRpaFocus2.RpaFocus2]::IsIconic($h)) { [void][WinRpaFocus2.RpaFocus2]::ShowWindow($h, 9); Start-Sleep -Milliseconds 150 }
if ([WinRpaFocus2.RpaFocus2]::GetForegroundWindow() -ne $h) {
[void][WinRpaFocus2.RpaFocus2]::SwitchToThisWindow($h, $true)
Start-Sleep -Milliseconds 150
}
if ([WinRpaFocus2.RpaFocus2]::GetForegroundWindow() -ne $h) {
[void][WinRpaFocus2.RpaFocus2]::keybd_event(0x12, 0, 0, [UIntPtr]::Zero)
[void][WinRpaFocus2.RpaFocus2]::SetForegroundWindow($h)
[void][WinRpaFocus2.RpaFocus2]::keybd_event(0x12, 0, 2, [UIntPtr]::Zero)
Start-Sleep -Milliseconds 200
}
if ([WinRpaFocus2.RpaFocus2]::GetForegroundWindow() -ne $h) {
$fgThread = [WinRpaFocus2.RpaFocus2]::GetWindowThreadProcessId([WinRpaFocus2.RpaFocus2]::GetForegroundWindow(), [IntPtr]::Zero)
$thisThread = [WinRpaFocus2.RpaFocus2]::GetCurrentThreadId()
$attached = [WinRpaFocus2.RpaFocus2]::AttachThreadInput($thisThread, $fgThread, $true)
try { [void][WinRpaFocus2.RpaFocus2]::SetForegroundWindow($h) } finally { if ($attached) { [void][WinRpaFocus2.RpaFocus2]::AttachThreadInput($thisThread, $fgThread, $false) } }
Start-Sleep -Milliseconds 200
}
if ([WinRpaFocus2.RpaFocus2]::GetForegroundWindow() -ne $h) {
[void][WinRpaFocus2.RpaFocus2]::ShowWindow($h, 6)
Start-Sleep -Milliseconds 150
[void][WinRpaFocus2.RpaFocus2]::ShowWindow($h, 9)
Start-Sleep -Milliseconds 300
}
if ([WinRpaFocus2.RpaFocus2]::GetForegroundWindow() -ne $h) { RpaFail '无法把窗口带到前台,请确认窗口未被关闭或全屏占用' }`;
}
/** 把 SendKeys 特殊字符按字面转义(+^%~(){}[] → {+} 等);SendKeys 用花括号表示字面量 */
function sendKeysEscape(ch: string): string {
return "+^%~(){}[]".includes(ch) ? `{${ch}}` : ch;
}
/** 把字符包装成 PowerShell 安全字面量。
* ASCII 直接用单引号字符串(撇号内部 '' 转义);非 ASCII 用 [char]0xXXXX 数值构造——
* Windows PowerShell 5.1 会把 ‘ ’ “ ” 等智能引号当作引号变体解析,直接放进字符串字面量会报「字符串缺少终止符」。 */
function psCharLiteral(text: string): string {
if (/^[\x20-\x7E]*$/.test(text)) return psString(text);
return Array.from(text).map((c) => `[char]0x${c.codePointAt(0)!.toString(16).padStart(4, "0")}`).join("+") || psString("");
}
/** 生成从当前位置到目标的插值路径点(JS 侧算好,PS 侧只执行循环)(原 window-extended.ts 的 mousePath) */
function mousePath(from: DriverPoint, to: DriverPoint, mode: string, speed: number): Array<[number, number]> {
const dx = to.x - from.x;
const dy = to.y - from.y;
const distance = Math.hypot(dx, dy);
if (distance < 2) return [[to.x, to.y]];
const stepCount = Math.max(6, Math.min(50, Math.round(distance / 12)));
const points: Array<[number, number]> = [];
for (let i = 1; i <= stepCount; i++) {
const t = i / stepCount;
// 模拟人工:缓入缓出 + 轻微垂直抖动;匀速直线:线性
const ease = mode === "0" ? (t < 0.5 ? 2 * t * t : 1 - Math.pow(-2 * t + 2, 2) / 2) : t;
const jitter = mode === "0" ? Math.sin(t * Math.PI) * (Math.sin(i * 12.9898) * 43758.5453 % 1) * 3 : 0;
points.push([
Math.round(from.x + dx * ease - (dy / distance) * jitter),
Math.round(from.y + dy * ease + (dx / distance) * jitter)
]);
}
points.push([to.x, to.y]);
return points;
}
const windowsMouse: MouseDriver = {
async moveTo(point) {
await powershell(`${USER32}; [RpaUser32]::SetCursorPos(${point.x},${point.y})`);
},
async click(point, options: MouseClickOptions = {}) {
const right = options.button === "right";
const down = right ? 0x0008 : 0x0002;
const up = right ? 0x0010 : 0x0004;
const repeats = options.double ? 2 : 1;
const settle = options.settleMs && options.settleMs > 0 ? `; Start-Sleep -Milliseconds ${Math.round(options.settleMs)}` : "";
await powershell(`${USER32}; [RpaUser32]::SetCursorPos(${point.x},${point.y});${Array.from({ length: repeats }, () => ` [RpaUser32]::mouse_event(${down},0,0,0,[UIntPtr]::Zero); [RpaUser32]::mouse_event(${up},0,0,0,[UIntPtr]::Zero)${settle}`).join(";")}`);
},
async drag(from, to, options: MouseDragOptions = {}) {
const right = options.button === "right";
const down = right ? 0x0008 : 0x0002;
const up = right ? 0x0010 : 0x0004;
const hold = Math.max(0, Math.round(options.holdMs ?? 80));
const release = Math.max(0, Math.round(options.releaseDelayMs ?? 80));
await powershell(`${USER32}; [RpaUser32]::SetCursorPos(${from.x},${from.y}); [RpaUser32]::mouse_event(${down},0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds ${hold}; [RpaUser32]::SetCursorPos(${to.x},${to.y});${release > 0 ? ` Start-Sleep -Milliseconds ${release};` : ""} [RpaUser32]::mouse_event(${up},0,0,0,[UIntPtr]::Zero)`);
},
async scroll(amount) {
await powershell(`${USER32}; [RpaUser32]::mouse_event(0x0800,0,0,${amount},[UIntPtr]::Zero)`);
},
async moveSmooth(target, options: MouseSmoothOptions = {}) {
const ease = options.ease !== false;
const speed = Math.min(100, Math.max(1, options.speed ?? 35));
const delay = Math.min(80, Math.max(4, Math.round(1500 / speed)));
// 从当前光标位置出发,按速度插值(路径全程物理像素,与目标同一坐标系)
const output = await powershell(`${WINDOW_API}; $cur = [RpaWindowExtended+POINT]::new(); [void][RpaWindowExtended]::GetCursorPos([ref]$cur); "$($cur.X),$($cur.Y)"`);
const [cx, cy] = output.split(",").map(Number);
const from: DriverPoint = Number.isFinite(cx) && Number.isFinite(cy) ? { x: cx, y: cy } : { x: 0, y: 0 };
const path = mousePath(from, target, ease ? "0" : "1", speed);
const ptsLiteral = path.map(([px, py]) => `@(${px},${py})`).join(",");
await powershell(`${WINDOW_API}; $pts = @(${ptsLiteral}); foreach ($pt in $pts) { [void][RpaWindowExtended]::SetCursorPos($pt[0], $pt[1]); Start-Sleep -Milliseconds ${delay} }`);
}
};
const windowsKeyboard: KeyboardDriver = {
async send(options: KeyboardSendOptions) {
// 三种输入方式:脚本尽量简单,转义逻辑放 JS 侧,避免 PowerShell 字符串拼接的语法陷阱
let send: string;
if (options.mode === "clipboard") {
// 剪贴板粘贴:原文进剪贴板 + Ctrl+V,不做 SendKeys 转义;内容含智能引号等非 ASCII 时同样走 [char] 构造
const contentExpr = /^[\x20-\x7E]*$/.test(options.keys) ? psString(options.keys) : `(-join @(${Array.from(options.keys).map((c) => psCharLiteral(c)).join(",")}))`;
send = `Set-Clipboard -Value ${contentExpr}; Start-Sleep -Milliseconds 120; [System.Windows.Forms.SendKeys]::SendWait('^v')`;
} else if (options.mode === "manual" && options.textMode) {
// 人工模拟:逐字符;JS 侧把每个字符转义成 SendKeys 字面量,PS 侧只做数组循环
const escapedChars = Array.from(options.keys).map((ch) => sendKeysEscape(ch));
const arr = escapedChars.map((c) => psCharLiteral(c)).join(",");
send = `$chars = @(${arr}); foreach ($c in $chars) { [System.Windows.Forms.SendKeys]::SendWait([string]$c); Start-Sleep -Milliseconds ${options.intervalMs} }`;
} else {
// 后台接口 / 按键组合 / 人工模拟(按键组合组件):整段发送,文本特殊字符转义
const escaped = options.textMode ? Array.from(options.keys).map(sendKeysEscape).join("") : options.keys;
const contentExpr = /^[\x20-\x7E]*$/.test(escaped) ? psString(escaped) : `(-join @(${Array.from(escaped).map((c) => psCharLiteral(c)).join(",")}))`;
send = `[System.Windows.Forms.SendKeys]::SendWait(${contentExpr})`;
}
const activate = options.activateTitle ? `${activateWindowScript(options.activateTitle)}; Start-Sleep -Milliseconds 300; ` : "";
await powershell(`${USER32}; ${activate}Add-Type -AssemblyName System.Windows.Forms; ${send}`);
}
};
const windowsWindow: WindowDriver = {
findByTitle: (title) => windowObject(title),
async activate(title, options: { maximize?: boolean } = {}) {
const maximize = options.maximize === true;
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; [RpaWindowExtended]::SetForegroundWindow($p.MainWindowHandle) | Out-Null; ${maximize ? "[RpaWindowExtended]::ShowWindow($p.MainWindowHandle,3) | Out-Null" : ""}`);
return windowObject(title);
},
async focus(title) {
return powershell(`${USER32}\n${activateWindowScript(title)}\n$p.MainWindowTitle`);
},
async foregroundTitle() {
return powershell(`${USER32}; $sb = New-Object System.Text.StringBuilder 512; [RpaUser32]::GetWindowText([RpaUser32]::GetForegroundWindow(),$sb,$sb.Capacity) | Out-Null; $sb.ToString()`);
},
async foregroundPosition(): Promise<DriverPoint> {
const output = await powershell(`${WINDOW_API}; $fw = [RpaWindowExtended]::GetForegroundWindow(); $r = New-Object RpaWindowExtended+RECT; [void][RpaWindowExtended]::GetWindowRect($fw, [ref]$r); "$($r.Left),$($r.Top)"`);
const [x, y] = output.split(",").map(Number);
return { x, y };
},
async exists(title) {
return (await powershell(`$p = Get-Process | Where-Object { $_.MainWindowTitle -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; [bool]$p`)).toLowerCase() === "true";
},
async close(title): Promise<string> {
return powershell(`$p = Get-Process | Where-Object { $_.MainWindowTitle -and $_.MainWindowTitle -like ${psString(`*${title}*`)} } | Select-Object -First 1; if (-not $p) { throw '找不到窗口' }; $p.CloseMainWindow() | Out-Null; $true`);
},
async setState(title, showCommand): Promise<void> {
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; [RpaWindowExtended]::ShowWindow($p.MainWindowHandle,${showCommand}) | Out-Null`);
},
async setSize(title, width, height) {
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; $r = New-Object RpaWindowExtended+RECT; [RpaWindowExtended]::GetWindowRect($p.MainWindowHandle,[ref]$r) | Out-Null; [RpaWindowExtended]::SetWindowPos($p.MainWindowHandle,[IntPtr]::Zero,$r.Left,$r.Top,${width},${height},4) | Out-Null`);
},
async setPos(title, rect: DriverRect): Promise<WindowInfo> {
if (!(rect.width > 0 && rect.height > 0)) throw new Error("窗口尺寸必须大于 0");
await powershell(`${WINDOW_API}; ${findWindowScript(title)}; [RpaWindowExtended]::SetWindowPos($p.MainWindowHandle,[IntPtr]::Zero,${rect.x},${rect.y},${rect.width},${rect.height},4) | Out-Null`);
return { ...(await windowObject(title)), x: rect.x, y: rect.y, width: rect.width, height: rect.height };
},
async list(filter: { title?: string; regex?: boolean } = {}) {
const title = filter.title ?? "";
const regex = filter.regex === true;
const output = await powershell(`Get-Process | Where-Object { $_.MainWindowHandle -ne 0 ${title ? (regex ? `-and $_.MainWindowTitle -match ${psString(title)}` : `-and $_.MainWindowTitle -like ${psString(`*${title}*`)}`) : ""} } | ForEach-Object { @{pid=$_.Id;title=$_.MainWindowTitle;hwnd=$_.MainWindowHandle.ToInt64()} } | ConvertTo-Json -Compress`);
if (!output) return [] as WindowSummary[];
const parsed = JSON.parse(output);
return (Array.isArray(parsed) ? parsed : [parsed]) as WindowSummary[];
}
};
const windowsScreen: ScreenDriver = {
async size() {
const output = await powershell("Add-Type -AssemblyName System.Windows.Forms; $b=[System.Windows.Forms.Screen]::PrimaryScreen.Bounds; @{width=$b.Width;height=$b.Height;x=$b.X;y=$b.Y} | ConvertTo-Json -Compress");
return JSON.parse(output);
}
};
/** element 域实现承载在 capabilities/desktop-uia.ts(阶段 3.3 说明),此处运行时注入,避免 driver → capabilities 的模块依赖 */
let elementImplementation: ElementDriver | undefined;
export function setElementDriverImplementation(impl: ElementDriver): void {
elementImplementation = impl;
}
export class WindowsDriver implements AutomationDriver {
readonly mouse = windowsMouse;
readonly keyboard = windowsKeyboard;
readonly window = windowsWindow;
readonly screen = windowsScreen;
get element(): ElementDriver {
if (!elementImplementation) {
throw new Error("ElementDriver 实现未注册:请 import @rpa/core/capabilities(desktop-uia 模块自动注册)");
}
return elementImplementation;
}
}
+53
View File
@@ -0,0 +1,53 @@
import type { Client } from "basic-ftp";
import type { Workbook as ExcelJSWorkbook } from "exceljs";
import { WindowsDriver } from "../driver/windows/windows-driver";
import type { AutomationDriver } from "../driver/types";
/** 人为拟真操作的开关与随机参数(desktop 能力的进程级设置)。 */
export interface ManualOperationSettings {
enabled: boolean;
randomMove: boolean;
randomClick: boolean;
randomSleep: boolean;
minMs: number;
maxMs: number;
}
/**
* 引擎实例状态容器(任务 1.3,R4:模块级全局状态收拢)。
*
* 折中方案(决策记录 2026-11-04):能力文件的 register() 调用与全局注册表保持不变,
* 但 Excel 缓存、数据库连接、FTP 客户端等进程级可变状态统一挂到引擎实例上。
* 包内各能力模块通过 defaultEngine 访问;宿主/CLI 可用 createEngine() 产生隔离实例,
* 或拿到 defaultEngine 做测试间重置。
*/
export class RpaEngine {
/** excel-store:路径 → ExcelJS 工作簿共享缓存 */
readonly excelCache = new Map<string, ExcelJSWorkbook>();
/** office.ts:Excel 句柄 id → 工作簿 */
readonly excelHandles = new Map<string, ExcelJSWorkbook>();
/** remaining-core:连接 id → 数据库连接(mysql2/pg/mssql) */
readonly dbConnections = new Map<string, unknown>();
/** ftp.ts:当前 FTP 连接 */
ftpClient?: Client;
/** ocr-caps.ts:懒加载的 OCR 实例 */
ocrInstance?: unknown;
/** desktop.ts:人为拟真操作设置 */
manualOperation: ManualOperationSettings = {
enabled: false,
randomMove: true,
randomClick: true,
randomSleep: true,
minMs: 1000,
maxMs: 3000
};
/** 桌面自动化驱动(任务 3.4 注入点,默认 PowerShell 版 WindowsDriver) */
readonly driver: AutomationDriver = new WindowsDriver();
}
export function createEngine(): RpaEngine {
return new RpaEngine();
}
/** 能力模块内部使用的默认引擎实例(与全局注册表配套的进程级单例)。 */
export const defaultEngine = createEngine();
@@ -0,0 +1,52 @@
/**
* 稳定错误码枚举(任务 4.3,R9)。
*
* 契约:RuntimeError 的 code 必须取自本清单;新增能力需要新错误码时先在此登记。
* message 保持中文(给人看),机器消费走 code 与 options 里的结构化字段
* (recommendation / nodeId / componentId / arg / detail)。
* 宿主侧另有自己的扩展码(browser 会话 / 拾取 / vision-lab 等),不进本清单。
*/
export const RUNTIME_ERROR_CODES = [
// 通用
"validation_error",
"timeout",
"cancelled",
"capability_not_implemented",
"expression_error",
"user_exception",
"not_found",
"network_error",
"http_error",
"external_service_failed",
// 桌面驱动
"desktop_command_failed",
"desktop_element_not_found",
"desktop_error",
// 浏览器
"browser_error",
"browser_disconnected",
"tab_unavailable",
// AI 服务
"ai_not_configured",
"ai_request_timeout",
"ai_rate_limited",
"ai_request_failed",
// 视觉
"cv_not_found",
// Office
"excel_com_failed",
"word_table_not_found",
"word_text_not_found",
"outlook_unavailable",
// 其他
"decryption_failed",
"host_unavailable"
] as const;
export type ErrorCode = (typeof RUNTIME_ERROR_CODES)[number];
/** 运行期校验(audit 测试用):code 必须在清单内 */
export function isKnownErrorCode(code: string): boolean {
return (RUNTIME_ERROR_CODES as readonly string[]).includes(code);
}
File renamed without changes.
@@ -0,0 +1,69 @@
import type { FlowDefinition } from "../types";
/**
* 宿主桥(任务 2.1 正式化)。
*
* 包内能力需要的宿主能力(子流程仓库 / Electron 窗口 / DPI 换算 / 应用配置)
* 通过本接口注入,包本身禁止 import "electron" 或宿主模块(见包内依赖检查脚本)。
* 宿主启动时调用 setSubflowSource() / setHostBridge()。
*/
export interface SubflowSource {
listFlows(): Promise<Array<{ id: string; name: string }>>;
getFlow(id: string): Promise<FlowDefinition>;
}
let subflowSource: SubflowSource | undefined;
export function setSubflowSource(source: SubflowSource): void {
subflowSource = source;
}
export function getSubflowSource(): SubflowSource | undefined {
return subflowSource;
}
/** 人机交互表单窗口的最小面(能力只用到这些成员,宿主用 BrowserWindow 适配) */
export interface HostDialogWindow {
isDestroyed(): boolean;
destroy(): void;
loadURL(url: string): Promise<void>;
on(event: "closed", listener: () => void): void;
/** 在窗口页面里执行 JS 并取回结果(宿主侧以 userGesture=true 执行) */
executeJavaScript(script: string): Promise<unknown>;
}
export interface HostPoint {
x: number;
y: number;
}
/** AI 服务配置(宿主 settings.aiConfig) */
export interface HostAiConfig {
baseUrl: string;
apiKey: string;
model: string;
visionModel: string;
}
/** 宿主能力桥最小面(决策点 4):dialog 窗口工厂 / DPI 坐标换算 / 配置读取 */
export interface HostBridge {
/** 创建人机交互表单窗口(human-dialog 用) */
createFormWindow(options: { title: string }): HostDialogWindow;
/** DIP 逻辑坐标 → 物理像素(window-extended 的鼠标/拖拽坐标换算) */
dipToScreenPoint(point: HostPoint): HostPoint;
/** 物理像素 → DIP 逻辑坐标 */
screenToDipPoint(point: HostPoint): HostPoint;
/** 读取 AI 服务配置(ai-service 用) */
loadAiConfig(): HostAiConfig;
}
let hostBridge: HostBridge | undefined;
export function setHostBridge(bridge: HostBridge): void {
hostBridge = bridge;
}
export function getHostBridge(): HostBridge | undefined {
return hostBridge;
}
+62
View File
@@ -0,0 +1,62 @@
import type { CapabilityHandler } from "./types";
/**
* 能力结构化元数据(任务 4.1,决策点 8:meta 可选 + 新能力强制)。
* 老能力逐步补齐;规范 v1 验收 checklist 对新能力要求必填。
*/
export interface CapabilityParamMeta {
key: string;
label?: string;
type: "string" | "number" | "boolean" | "object" | "array";
required?: boolean;
defaultValue?: string | number | boolean | null;
description?: string;
}
export interface CapabilityMeta {
/** 一句话能力摘要(中文) */
summary?: string;
/** 参数 schema(按字段名列出) */
params?: CapabilityParamMeta[];
/** 返回值说明 */
returns?: string;
/** 默认超时毫秒(无外部交互语义的能力省略) */
timeoutMs?: number;
/** 可能抛出的稳定错误码(见 engine/error-codes.ts) */
errorCodes?: string[];
/** 执行中是否需要回传截图(Agent 侧视觉判断用) */
needsScreenshot?: boolean;
/** 平台限制;省略表示跨平台 */
platforms?: Array<"win32">;
}
const handlers = new Map<string, CapabilityHandler>();
const metas = new Map<string, CapabilityMeta>();
export function register(ids: string | string[], handler: CapabilityHandler, meta?: CapabilityMeta): void {
const list = Array.isArray(ids) ? ids : [ids];
for (const id of list) {
handlers.set(id, handler);
if (meta) metas.set(id, meta);
}
}
export function getHandler(id: string): CapabilityHandler | undefined {
return handlers.get(id);
}
export function getCapabilityMeta(id: string): CapabilityMeta | undefined {
return metas.get(id);
}
export function hasCapability(id: string): boolean {
return handlers.has(id);
}
export function capabilityIds(): string[] {
return [...handlers.keys()];
}
export function capabilityCount(): number {
return handlers.size;
}
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import type { CatalogComponent, FlowDefinition } from "../../shared/types";
import type { CatalogComponent, FlowDefinition } from "../types";
import { register } from "./registry";
import { runFlow } from "./runtime";
@@ -8,8 +8,8 @@ function component(id: string): CatalogComponent {
const value = {
id, name: id, kind: "vendor", group: "data", categories: [], categorySignature: "", description: "",
messageTemplate: "", baseType: id, functionName: id, functionCalls: [], version: "", tags: [], platformTag: null,
icon: "data", risk: "low", fields: [], output: { label: "", key: "", suggestedName: "", valueTypes: [], fields: [] }, slots: [], implemented: true
} as CatalogComponent;
icon: "data", risk: "low", fields: [], output: { label: "", key: "", suggestedName: "", valueTypes: [], fields: [] }, slots: []
} as unknown as CatalogComponent;
components.set(id, value);
return value;
}
@@ -10,7 +10,7 @@ import type {
FlowEdge,
FlowNode,
RunEvent
} from "../../shared/types";
} from "../types";
import "../capabilities";
import { register } from "./registry";
import { runFlow } from "./runtime";
@@ -83,7 +83,6 @@ function testComponent(id: string): CatalogComponent {
fields: [],
output: { label: "", key: "", suggestedName: "", valueTypes: [], fields: [] },
slots: [],
implemented: true
};
}
@@ -197,6 +196,28 @@ describe("runFlow integration", () => {
expect(events.filter((event) => event.type === "node.started" && event.nodeId === "accumulate")).toHaveLength(3);
});
it("runs an array loop through the last item when end_index is -1", async () => {
const loop = node("loop", "logic_control_array_for_v3", {
array_name: "items",
reverse: false,
start_index: 0,
end_index: -1
});
const definition = flow("array-loop-flow", [
loop,
node("collect", "Basic_DebugOutput", { msg: "${item}" }, {}, {
parentId: loop.id,
slot: "array_for_fn"
})
]);
const { result, events } = await execute(definition, { items: ["glm", "mini", "other", "kimi"] });
expect(result.status).toBe("succeeded");
expect(result.nodes.find((state) => state.nodeId === "loop")?.output).toBe(4);
expect(events.filter((event) => event.type === "node.started" && event.nodeId === "collect").map((event) => event.nodeId)).toHaveLength(4);
});
it("retries a transient failure and records the successful attempt", async () => {
const component = testComponent("Test.TransientFailure");
let calls = 0;
@@ -1,4 +1,4 @@
import type { FlowDefinition, FlowNode, RunError } from "../../shared/types";
import type { FlowDefinition, FlowNode, RunError } from "../types";
import { evaluate, evaluateCondition, interpolate, toNumber } from "./expression";
import { getHandler } from "./registry";
import {
@@ -265,7 +265,7 @@ async function executeNode(node: FlowNode, ctx: RunContext, states: Map<string,
{
nodeId: node.id,
componentId: node.componentId,
recommendation: "换用已接入组件;该能力已列入后续接入清单"
recommendation: "请检查组件配置或重新添加节点"
}
);
}
@@ -409,7 +409,8 @@ async function executeContainer(
const list = asArray(value);
const reverse = String(node.values?.reverse ?? "") === "true" || String(node.values?.reverse ?? "") === "1";
const start = Math.max(0, toNumber(node.values?.start_index, 0));
const end = node.values?.end_index === "" || node.values?.end_index == null ? list.length : toNumber(node.values?.end_index, list.length);
const requestedEnd = node.values?.end_index === "" || node.values?.end_index == null ? list.length : toNumber(node.values?.end_index, list.length);
const end = requestedEnd < 0 ? Math.max(start, list.length + requestedEnd + 1) : requestedEnd;
const slice = (reverse ? [...list].reverse() : list).slice(start, end);
let iterations = 0;
for (let index = 0; index < slice.length; index += 1) {
@@ -1,9 +1,9 @@
import type { CatalogComponent, FlowDefinition, FlowNode, RunError, RunEvent } from "../../shared/types";
import type { CatalogComponent, FlowDefinition, FlowNode, RunError, RunEvent } from "../types";
export interface BrowserLike {
navigate(url: string): Promise<{ url: string; title: string }>;
click(selector: unknown, timeoutMs?: number): Promise<unknown>;
type(selector: unknown, text: string, clearFirst?: boolean): Promise<unknown>;
type(selector: unknown, text: string, clearFirst?: boolean, timeoutMs?: number): Promise<unknown>;
getText(selector: unknown, attribute?: string): Promise<string>;
exists(selector: unknown, timeoutMs?: number): Promise<boolean>;
select(selector: unknown, value: string): Promise<unknown>;
@@ -30,9 +30,37 @@ export interface BrowserLike {
getTableData?(selector: unknown): Promise<string[][]>;
getSimilarElements?(selector: unknown, options?: BrowserSimilarElementOptions): Promise<BrowserElementReference[]>;
currentUrl(): Promise<string>;
/** 设置文件上传 input 的本地文件路径 */
uploadFileToInput?(selector: string, filePath: string): Promise<number>;
maximize?(): void;
minimize?(): void;
stopLoading?(): void;
/** 自动关闭页面弹窗(alert/confirm),返回是否新开启 */
startAutoCloseDialogs?(): boolean;
stopAutoCloseDialogs?(): boolean;
/** 开启网络响应监听,返回是否新开启 */
startNetworkListening?(): boolean;
getNetworkCaptures?(options?: { url?: string; wildcard?: boolean; resourceType?: string; body?: boolean }): Promise<Array<Record<string, unknown>>>;
stopNetworkListening?(): boolean;
/** 列出当前浏览器的标签页(内置浏览器返回单页;外部浏览器走 Target 枚举) */
listTabs?(): Promise<BrowserTabInfo[]>;
/** 切换当前操作的标签页(按 targetId / 索引 / 标题 / 网址匹配,标题支持正则) */
switchTab?(match: { targetId?: string; index?: number; title?: string; url?: string }): Promise<BrowserTabInfo>;
/** 在浏览器中新建标签页并切换过去(内置浏览器退化为当前页导航) */
openNewTab?(url: string): Promise<BrowserTabInfo>;
/** 全量 Cookie(含 HttpOnly;走 Network.getCookies) */
getCookies?(): Promise<Array<{ name: string; value: string; domain: string; path: string; expires: number; httpOnly: boolean; secure: boolean }>>;
close(): Promise<void>;
}
export interface BrowserTabInfo {
index: number;
targetId: string;
title: string;
url: string;
active: boolean;
}
export interface BrowserElementReference {
kind: "css" | "xpath";
value: string;
+104
View File
@@ -0,0 +1,104 @@
/**
* @rpa/core 包入口。
*
* 只导出引擎 API 与类型,不做能力注册副作用;
* 纯 Node 能力通过 `@rpa/core/capabilities` 子路径按需拉起(保持与 demo 宿主
* src/main/capabilities/index.ts 相同的副作用注册模型,见任务 4.2 再显式化)。
*/
export { createEngine, defaultEngine, RpaEngine, type ManualOperationSettings } from "./engine/engine";
export { register, getHandler, hasCapability, capabilityIds, capabilityCount } from "./engine/registry";
export { runFlow } from "./engine/runtime";
export { RuntimeError, BreakSignal, ContinueSignal, StopSignal, asRuntimeError } from "./engine/types";
export { setSubflowSource, getSubflowSource, type SubflowSource } from "./engine/host-bridge";
// 错误契约(任务 4.3):稳定错误码枚举 + 机器可读结构(RuntimeError.code / recommendation / detail 等)
export { RUNTIME_ERROR_CODES, isKnownErrorCode, type ErrorCode } from "./engine/error-codes";
// 能力元数据(任务 4.1)与显式注册入口(任务 4.2,双入口过渡)
export { getCapabilityMeta, type CapabilityMeta, type CapabilityParamMeta } from "./engine/registry";
export { registerAll, capabilityDomains } from "./capabilities/register-all";
export {
setHostBridge,
getHostBridge,
type HostBridge,
type HostDialogWindow,
type HostPoint,
type HostAiConfig
} from "./engine/host-bridge";
// AI 服务(阶段 2 经 HostBridge 迁入包内):宿主注入 loadAiConfig,消费方走此导出
export {
chatCompletion,
getAiConfig,
requireAiConfig,
type ChatCompletionOptions,
type ChatContentPart,
type ChatMessage,
type ResolvedAiConfig
} from "./capabilities/ai-service";
// 浏览器 CDP 外部路径(阶段 2 迁入):ExternalTransport + 外部浏览器进程管理
export { ExternalTransport, mapCdpError, type CdpTransport, type CdpEventHandler } from "./browser/transport";
export {
listPageTargets,
openExternalBrowser,
openNewPageTarget,
sessionKeyOf,
detectBrowserPath,
connectRemoteBrowser,
launchExternalBrowser,
type ExternalBrowserOptions,
type ExternalBrowserType,
type ExternalLaunchOptions,
type ExternalRemoteOptions,
type ExternalConnection
} from "./browser/external";
// 桌面自动化驱动(阶段 3):五域接口 + PowerShell 实现,引擎创建时注入
export type {
AutomationDriver,
DriverPoint,
DriverRect,
ElementDriver,
ElementFindOptions,
KeyboardDriver,
KeyboardSendOptions,
MouseClickOptions,
MouseDragOptions,
MouseDriver,
MouseSmoothOptions,
MouseButton,
ScreenDriver,
UiaSelector,
WindowDriver,
WindowInfo,
WindowSummary
} from "./driver/types";
export { WindowsDriver, setElementDriverImplementation } from "./driver/windows/windows-driver";
// PowerShell 助手(任务 3.1 已移入 driver/windows/)
export { powershell, psString } from "./driver/windows/powershell";
export type {
BrowserLike,
BrowserTabInfo,
BrowserElementReference,
BrowserElementInfo,
BrowserSelectOption,
BrowserSelectState,
BrowserSimilarElementOptions,
BrowserElementRegion,
RunContext,
CapabilityHandler,
NodeState,
RunResult
} from "./engine/types";
export type {
CatalogWidget,
CatalogField,
CatalogOutput,
CatalogComponent,
CatalogGroup,
Catalog,
FlowVariable,
FlowNodeOptions,
FlowNode,
FlowEdge,
FlowDefinition,
RunError,
RunEvent
} from "./types";
Loaded 100 of 212 files, more files were not shown because too many files have changed in this diff. Show more