Agent S：像人一样操作电脑的开源计算机使用代理框架（Agent-S）

💡 Introduction Agent S is an open source computer use agent framework from Simular .

It takes a natural-language task, looks at the screen, and completes the task by clicking, typing, and scrolling in ordinary desktop and web applications — no API integration and no per-app scripting required.

Agent S3 is the third generation of the framework and was the first computer use agent to surpass human performance on the OSWorld benchmark, at 72.60% .

It runs on macOS, Windows, and Linux, and works with models from OpenAI, Anthropic, and open-weight providers.

The research continues in production: in August 2026, Sai — Simular's hosted computer use agent, built on the ideas in this repo — reached a 73% success rate on OSWorld 2.0 , ahead of GPT-5.6 Sol at 62.57% as reported by OpenAI, and at lower cost.

If you are looking for a hosted, production version of this technology rather than a research framework, see Sai . Whether you're researching OS agents, automating your own desktop, or contributing to open source computer use, we're excited to have you here.

🎯 OSWorld Benchmark Results August 2026 — Sai tops OSWorld 2.0 (73%) Our flagship product Sai reaches a 73% success rate on OSWorld 2.0 , a 108-task benchmark of long professional and everyday tasks that take skilled humans more than an hour to complete.

That places Sai ahead of GPT-5.6 Sol at 62.57% as reported by OpenAI, and it does so at lower cost per task. Full write-up: Sai Tops OSWorld 2.0 .

Agent S3 on OSWorld, WindowsAgentArena and AndroidWorld On OSWorld, Agent S3 alone reaches 66% in the 100-step setting, already exceeding the previous state of the art of 63.4% (GTA1 w/ GPT-5).

With the addition of Behavior Best-of-N, performance climbs even higher to 72.6%, surpassing human-level performance on OSWorld (~72%)! Agent S3 also demonstrates strong zero-shot generalization! On WindowsAgentArena, accuracy rises from 50.

2% using only Agent S3 to 56.6% by selecting from 3 rollouts. Similarly on AndroidWorld, performance improves from 68.1% to 71.6% 🛠️ Installation & Setup Prerequisites

Installation To install Agent S3 without cloning the repository, run

• 🛠️ Installation & Setup
• 🚀 Usage
• 🎬 See Sai in Action
• 🤝 Acknowledgements
• 💬 Citation
• Single Monitor : Our agent is designed for single monitor screens
• Security : The agent runs Python code to control your computer - use with care
• Supported Platforms : Linux, Mac, and Windows

pip install gui-agents

If you would like to test Agent S3 while making changes, clone the repository and install using

pip install -e .

Don't forget to also brew install tesseract ! Pytesseract requires this extra installation to work. API Configuration Option 1: Environment Variables Add to your .bashrc (Linux) or .zshrc (MacOS):

export OPENAI_API_KEY= < YOUR_API_KEY >

export ANTHROPIC_API_KEY= < YOUR_ANTHROPIC_API_KEY >

export HF_TOKEN= < YOUR_HF_TOKEN >

Option 2

Python Script

import os os . environ [ "OPENAI_API_KEY" ] = "<YOUR_API_KEY>" Supported Models We support Azure OpenAI, Anthropic, Gemini, Open Router, and vLLM inference. See models.md for details. Grounding Models (Required) For optimal performance, we recommend UI-TARS-1.

5-7B hosted on Hugging Face Inference Endpoints or another provider. See Hugging Face Inference Endpoints for setup instructions.

🚀 Usage ⚡️ Recommended Setup: For the best configuration, we recommend using OpenAI gpt-5-2025-08-07 as the main model, paired with UI-TARS-1.5-7B for grounding. CLI Note, this is running Agent S3, our improved agent, without bBoN.

Run Agent S3 with the required parameters: agent_s \ --provider openai \ --model gpt-5-2025-08-07 \ --ground_provider huggingface \ --ground_url http://localhost:8080 \ --ground_model ui-tars-1.

5-7b \ --grounding_width 1920 \ --grounding_height 1080 Local Coding Environment (Optional) For tasks that require code execution (e.g.

, data processing, file manipulation, system automation), you can enable the local coding environment: agent_s \ --provider openai \ --model gpt-5-2025-08-07 \ --ground_provider huggingface \ --ground_url http://localhost:8080 \ --ground_model ui-tars-1.

5-7b \ --grounding_width 1920 \ --grounding_height 1080 \ --enable_local_env ⚠️ WARNING : The local coding environment executes arbitrary Python and Bash code locally on your machine. Only use this feature in trusted environments and with trusted inputs.

Required Parameters

Optional Parameters

Grounding Model Dimensions The grounding width and height should match the output coordinate resolution of your grounding model:

Optional Parameters

Local Coding Environment Details The local coding environment enables Agent S3 to execute Python and Bash code directly on your machine. This is particularly useful for:

• --provider : Main generation model provider (e.g., openai, anthropic, etc.) - Default: "openai"
• --model : Main generation model name (e.g., gpt-5-2025-08-07) - Default: "gpt-5-2025-08-07"
• --ground_provider : The provider for the grounding model - Required
• --ground_url : The URL of the grounding model - Required
• --ground_model : The model name for the grounding model - Required
• --grounding_width : Width of the output coordinate resolution from the grounding model - Required
• --grounding_height : Height of the output coordinate resolution from the grounding model - Required
• --model_temperature : The temperature to fix all model calls to (necessary to set to 1.0 for models like o3 but can be left blank for other models)
• UI-TARS-1.5-7B : Use --grounding_width 1920 --grounding_height 1080
• UI-TARS-72B : Use --grounding_width 1000 --grounding_height 1000
• --model_url : Custom API URL for main generation model - Default: ""
• --model_api_key : API key for main generation model - Default: ""
• --ground_api_key : API key for grounding model endpoint - Default: ""
• --max_trajectory_length : Maximum number of image turns to keep in trajectory - Default: 8
• --enable_reflection : Enable reflection agent to assist the worker agent - Default: True
• --enable_local_env : Enable local coding environment for code execution (WARNING: Executes arbitrary code locally) - Default: False
• Data Processing : Manipulating spreadsheets, CSV files, or databases
• File Operations : Bulk file processing, content extraction, or file organization