PaperActive

OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrations

quality89

Updated 2 h ago · first seen 12 Sept 2026

paper_01M29X35BFQXQW6R44JCZ8HXXA

Published: 12 Sept 2026; T1 · 2 h ago
arXiv: 2609.02149; T1 · 2 h ago
Category: cs.HC; T1 · 2 h ago

Abstract

-cross Abstract: Large language models (LLMs) are increasingly evolving from conversational assistants into agents capable of operating external digital environments. Graphical user interface (GUI) agents play an important role in this transition, as many real-world workflows remain accessible only through user-facing software interfaces. However, despite recent progress on general computer-use benchmarks, domain-specific professional standard operating procedures (SOPs) remain challenging for GUI agents because they often involve implicit domain knowledge, software-specific conventions, and task-level verification requirements. We introduce OmegaUse-SOP, a human-in-the-loop SOP Engineering system for transforming human demonstrations of professional computer use into reusable SOP skills for GUI agents. Analogous to prompt engineering, SOP Engineering iteratively refines demonstrations, execution rules, and domain knowledge to convert professional SOPs into reusable GUI-agent skills. OmegaUse-SOP consists of four modules: Observe, Reason, Configure, and Execute. Together, these modules record expert operations as multimodal GUI traces, abstract low-level events into semantic step-level instructions, incorporate domain rules and task-specific parameters, and execute the resulting skills in live GUI environments through step-wise grounding, action generation, and verification. To demonstrate its effectiveness, we collaborate with a power-sector client and test OmegaUse-SOP on photovoltaic simulation workflows in PVsyst 7.2. The results suggest that OmegaUse-SOP can improve GUI-agent reliability on professional SOP tasks, highlighting a practical path toward deploying GUI agents in domain-specific professional software environments.

Authors 12

Hua Wu, Hucheng Yang, Jingbo Zhou, Jingjia Cao, Lang An, Pinxue Ma, Siqi Bao, Ting Liu, Ting Wang, Yixiong Xiao, Yongquan Chen, Yusai Zhao

Specification

Official page: arxiv.org/abs/2609.02149
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Arxiv announce type: replace
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
arXiv id: 2609.02149
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Categories: cs.AI, cs.HC
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
DOI: 10.21203/rs.3.rs-10741604/v1
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
PDF: arxiv.org/pdf/2609.02149
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Primary category: cs.HC
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Published: 12 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

Source tiers

T110

Freshest observation

2 h ago

Conflicts

None

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · PDF

1 claims · 1 propertiesShow all properties

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2609.02149	12 Sept 2026 04:12 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	2 h ago	2

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.