{
"canonical_name": "microsoft/magentic-ui",
"compilation_id": "pack_d14d605fd18149bcbb796b25651a578c",
"created_at": "2026-05-18T02:21:03.961610+00:00",
"created_by": "project-pack-compiler",
"feedback": {
"carrier_selection_notes": [
"viable_asset_types=skill, recipe, host_instruction, eval, preflight",
"recommended_asset_types=skill, recipe, host_instruction, eval, preflight"
],
"evidence_delta": {
"confirmed_claims": [
"identity_anchor_present",
"capability_and_host_targets_present",
"install_path_declared_or_better"
],
"missing_required_fields": [],
"must_verify_forwarded": [
"Run or inspect `pip install magentic-ui` in an isolated environment.",
"Confirm the project exposes the claimed capability to at least one target host."
],
"quickstart_execution_scope": "allowlisted_sandbox_smoke",
"sandbox_command": "pip install magentic-ui",
"sandbox_container_image": "python:3.12-slim",
"sandbox_execution_backend": "docker",
"sandbox_planner_decision": "llm_execute_isolated_install",
"sandbox_validation_id": "sbx_cdc662fd813443d3a2587e4e7300f97f"
},
"feedback_event_type": "project_pack_compilation_feedback",
"learning_candidate_reasons": [],
"template_gaps": []
},
"identity": {
"canonical_id": "project_84171691db23a3d14db7f72565c5f863",
"canonical_name": "microsoft/magentic-ui",
"homepage_url": null,
"license": "unknown",
"repo_url": "https://github.com/microsoft/magentic-ui",
"slug": "magentic-ui",
"source_packet_id": "phit_d663dfedde334fb69f4f57cc96742549",
"source_validation_id": "dval_36857acfbba94767984ed9af003c2ae8"
},
"merchandising": {
"best_for": "需要信息检索与知识管理能力，并使用 local_cli的用户",
"github_forks": 978,
"github_stars": 9810,
"one_liner_en": "A research prototype of a human-centered web agent",
"one_liner_zh": "A research prototype of a human-centered web agent",
"primary_category": {
"category_id": "research-knowledge",
"confidence": "medium",
"name_en": "Research & Knowledge",
"name_zh": "信息检索与知识管理",
"reason": "matched_keywords:research, search"
},
"target_user": "使用 local_cli 等宿主 AI 的用户",
"title_en": "magentic-ui",
"title_zh": "magentic-ui 能力包",
"visible_tags": [
{
"label_en": "Browser Agents",
"label_zh": "浏览器 Agent",
"source": "repo_evidence_project_characteristics",
"tag_id": "product_domain-browser-agents",
"type": "product_domain"
},
{
"label_en": "Web Task Automation",
"label_zh": "网页任务自动化",
"source": "repo_evidence_project_characteristics",
"tag_id": "user_job-web-task-automation",
"type": "user_job"
},
{
"label_en": "Browser Automation",
"label_zh": "浏览器自动化",
"source": "repo_evidence_project_characteristics",
"tag_id": "core_capability-browser-automation",
"type": "core_capability"
},
{
"label_en": "Checkpoint Resume",
"label_zh": "断点恢复流程",
"source": "repo_evidence_project_characteristics",
"tag_id": "workflow_pattern-checkpoint-resume",
"type": "workflow_pattern"
},
{
"label_en": "Evaluation Suite",
"label_zh": "评测体系",
"source": "repo_evidence_project_characteristics",
"tag_id": "selection_signal-evaluation-suite",
"type": "selection_signal"
}
]
},
"packet_id": "phit_d663dfedde334fb69f4f57cc96742549",
"page_model": {
"artifacts": {
"artifact_slug": "magentic-ui",
"files": [
"PROJECT_PACK.json",
"QUICK_START.md",
"PROMPT_PREVIEW.md",
"HUMAN_MANUAL.md",
"AI_CONTEXT_PACK.md",
"BOUNDARY_RISK_CARD.md",
"PITFALL_LOG.md",
"REPO_INSPECTION.json",
"REPO_INSPECTION.md",
"CAPABILITY_CONTRACT.json",
"EVIDENCE_INDEX.json",
"CLAIM_GRAPH.json"
],
"required_files": [
"PROJECT_PACK.json",
"QUICK_START.md",
"PROMPT_PREVIEW.md",
"HUMAN_MANUAL.md",
"AI_CONTEXT_PACK.md",
"BOUNDARY_RISK_CARD.md",
"PITFALL_LOG.md",
"REPO_INSPECTION.json"
]
},
"detail": {
"capability_source": "Project Hit Packet + DownstreamValidationResult",
"commands": [
{
"command": "pip install magentic-ui",
"label": "Python / pip · 官方安装入口",
"source": "https://github.com/microsoft/magentic-ui#readme",
"verified": true
}
],
"display_tags": [
"浏览器 Agent",
"网页任务自动化",
"浏览器自动化",
"断点恢复流程",
"评测体系"
],
"eyebrow": "信息检索与知识管理",
"glance": [
{
"body": "判断自己是不是目标用户。",
"label": "最适合谁",
"value": "需要信息检索与知识管理能力，并使用 local_cli的用户"
},
{
"body": "先理解能力边界，再决定是否继续。",
"label": "核心价值",
"value": "A research prototype of a human-centered web agent"
},
{
"body": "未完成验证前保持审慎。",
"label": "继续前",
"value": "publish to Doramagic.ai project surfaces"
}
],
"guardrail_source": "Boundary & Risk Card",
"guardrails": [
{
"body": "Prompt Preview 只展示流程，不证明项目已安装或运行。",
"label": "Check 1",
"value": "不要把试用当真实运行"
},
{
"body": "local_cli",
"label": "Check 2",
"value": "确认宿主兼容"
},
{
"body": "publish to Doramagic.ai project surfaces",
"label": "Check 3",
"value": "先隔离验证"
}
],
"mode": "skill, recipe, host_instruction, eval, preflight",
"pitfall_log": {
"items": [
{
"body": "GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Create tutorials and documentation for the codebase",
"category": "安装坑",
"evidence": [
"community_evidence:github | cevd_c0979f7ebb064422a6a8095561f6a9bd | https://github.com/microsoft/magentic-ui/issues/154 | 来源类型 github_issue 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Create tutorials and documentation for the codebase",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Support Podman in place of Docker",
"category": "安装坑",
"evidence": [
"community_evidence:github | cevd_f88231cc4cad442ca53d92ea3a40a655 | https://github.com/microsoft/magentic-ui/issues/312 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Support Podman in place of Docker",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个安装相关的待验证问题：magentic-ui can't display all the html element",
"category": "安装坑",
"evidence": [
"community_evidence:github | cevd_21f19953edd74379ab2d25cedc37ca1b | https://github.com/microsoft/magentic-ui/issues/362 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：magentic-ui can't display all the html element",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：Refreshing or restart the web app will make the current Session unavailable",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_83a9cafd59254028853ef84cd1ccc756 | https://github.com/microsoft/magentic-ui/issues/336 | 来源讨论提到 node 相关条件，需在安装/试用前复核。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Refreshing or restart the web app will make the current Session unavailable",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "README/documentation is current enough for a first validation pass.",
"category": "能力坑",
"evidence": [
"capability.assumptions | github_repo:978331188 | https://github.com/microsoft/magentic-ui | README/documentation is current enough for a first validation pass."
],
"severity": "medium",
"suggested_check": "将假设转成下游验证清单。",
"title": "能力判断依赖假设",
"user_impact": "假设不成立时，用户拿不到承诺的能力。"
},
{
"body": "GitHub 社区证据显示该项目存在一个运行相关的待验证问题：Why not conduct a requirement analysis before the plan?",
"category": "运行坑",
"evidence": [
"community_evidence:github | cevd_6003a9c2194f40c0865145385cf98c32 | https://github.com/microsoft/magentic-ui/issues/321 | 来源类型 github_issue 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Why not conduct a requirement analysis before the plan?",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：Sticked at click the “Shopping Cart” icon and cannot goto check out page",
"category": "维护坑",
"evidence": [
"community_evidence:github | cevd_7e754869326e42e1a7c57f3a1962ef9e | https://github.com/microsoft/magentic-ui/issues/360 | 来源类型 github_issue 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Sticked at click the “Shopping Cart” icon and cannot goto check out page",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "未记录 last_activity_observed。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | last_activity_observed missing"
],
"severity": "medium",
"suggested_check": "补 GitHub 最近 commit、release、issue/PR 响应信号。",
"title": "维护活跃度未知",
"user_impact": "新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。"
},
{
"body": "no_demo",
"category": "安全/权限坑",
"evidence": [
"downstream_validation.risk_items | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium"
],
"severity": "medium",
"suggested_check": "进入安全/权限治理复核队列。",
"title": "下游验证发现风险项",
"user_impact": "下游已经要求复核，不能在页面中弱化。"
},
{
"body": "no_demo",
"category": "安全/权限坑",
"evidence": [
"risks.scoring_risks | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium"
],
"severity": "medium",
"suggested_check": "把风险写入边界卡，并确认是否需要人工复核。",
"title": "存在评分风险",
"user_impact": "风险会影响是否适合普通用户安装。"
},
{
"body": "GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：Settings redesign",
"category": "安全/权限坑",
"evidence": [
"community_evidence:github | cevd_6a2eeae98b6d4fdab476464d57e64e1d | https://github.com/microsoft/magentic-ui/issues/227 | 来源类型 github_issue 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：Settings redesign",
"user_impact": "可能影响授权、密钥配置或安全边界。"
},
{
"body": "issue_or_pr_quality=unknown。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | issue_or_pr_quality=unknown"
],
"severity": "low",
"suggested_check": "抽样最近 issue/PR，判断是否长期无人处理。",
"title": "issue/PR 响应质量未知",
"user_impact": "用户无法判断遇到问题后是否有人维护。"
},
{
"body": "release_recency=unknown。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | release_recency=unknown"
],
"severity": "low",
"suggested_check": "确认最近 release/tag 和 README 安装命令是否一致。",
"title": "发布节奏不明确",
"user_impact": "安装命令和文档可能落后于代码，用户踩坑概率升高。"
}
],
"source": "ProjectPitfallLog + ProjectHitPacket + validation + community signals",
"summary": "发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：安装坑 - 来源证据：Create tutorials and documentation for the codebase。",
"title": "踩坑日志"
},
"snapshot": {
"contributors": 30,
"forks": 978,
"license": "unknown",
"note": "站点快照，非实时质量证明；用于开工前背景判断。",
"stars": 9810
},
"source_url": "https://github.com/microsoft/magentic-ui",
"steps": [
{
"body": "不安装项目，先体验能力节奏。",
"code": "preview",
"title": "先试 Prompt"
},
{
"body": "理解输入、输出、失败模式和边界。",
"code": "manual",
"title": "读说明书"
},
{
"body": "把上下文交给宿主 AI 继续工作。",
"code": "context",
"title": "带给 AI"
},
{
"body": "进入主力环境前先完成安装入口与风险边界验证。",
"code": "verify",
"title": "沙箱验证"
}
],
"subtitle": "A research prototype of a human-centered web agent",
"title": "magentic-ui 能力包",
"trial_prompt": "# magentic-ui - Prompt Preview\n\n> Copy the prompt below into your AI host before installing anything.\n> Its purpose is to let you safely feel the project's workflow, not to claim the project has already run.\n\n## Copy this prompt\n\n```text\nYou are using an independent Doramagic capability pack for microsoft/magentic-ui.\n\nProject:\n- Name: magentic-ui\n- Repository: https://github.com/microsoft/magentic-ui\n- Summary: A research prototype of a human-centered web agent\n- Host target: local_cli\n\nGoal:\nHelp me evaluate this project for the following task without installing it yet: A research prototype of a human-centered web agent\n\nBefore taking action:\n1. Restate my task, success standard, and boundary.\n2. Identify whether the next step requires tools, browser access, network access, filesystem access, credentials, package installation, or host configuration.\n3. Use only the Doramagic Project Pack, the upstream repository, and the source-linked evidence listed below.\n4. If a real command, install step, API call, file write, or host integration is required, mark it as \"requires post-install verification\" and ask for approval first.\n5. If evidence is missing, say \"evidence is missing\" instead of filling the gap.\n\nPreviewable capabilities:\n- Capability 1: A research prototype of a human-centered web agent\n\nCapabilities that require post-install verification:\n- Capability 1: Use the source-backed project context to guide one small, checkable workflow step.\n\nCore service flow:\n1. getting-started: Getting Started with Magentic-UI. Produce one small intermediate artifact and wait for confirmation.\n2. key-concepts: Key Concepts. Produce one small intermediate artifact and wait for confirmation.\n3. high-level-architecture: High-Level Architecture. Produce one small intermediate artifact and wait for confirmation.\n4. agent-system: Agent System. Produce one small intermediate artifact and wait for confirmation.\n5. team-orchestration: Team Orchestration. Produce one small intermediate artifact and wait for confirmation.\n\nSource-backed evidence to keep in mind:\n- https://github.com/microsoft/magentic-ui\n- https://github.com/microsoft/magentic-ui#readme\n- README.md\n- pyproject.toml\n- TROUBLESHOOTING.md\n- src/magentic_ui/approval_guard.py\n- src/magentic_ui/guarded_action.py\n- src/magentic_ui/learning/learner.py\n- src/magentic_ui/backend/web/app.py\n- src/magentic_ui/backend/teammanager/teammanager.py\n\nFirst response rules:\n1. Start Step 1 only.\n2. Explain the one service action you will perform first.\n3. Ask exactly three questions about my target workflow, success standard, and sandbox boundary.\n4. Stop and wait for my answers.\n\nStep 1 follow-up protocol:\n- After I answer the first three questions, stay in Step 1.\n- Produce six parts only: clarified task, success standard, boundary conditions, two or three options, tradeoffs for each option, and one recommendation.\n- End by asking whether I confirm the recommendation.\n- Do not move to Step 2 until I explicitly confirm.\n\nConversation rules:\n- Advance one step at a time and wait for confirmation after each small artifact.\n- Write outputs as recommendations or planned checks, not as completed execution.\n- Do not claim tests passed, files changed, commands ran, APIs were called, or the project was installed.\n- If the user asks for execution, first provide the sandbox setup, expected output, rollback, and approval checkpoint.\n```\n",
"voices": [
{
"body": "来源平台：github。github/github_issue: Create tutorials and documentation for the codebase（https://github.com/microsoft/magentic-ui/issues/154）；github/github_issue: Settings redesign（https://github.com/microsoft/magentic-ui/issues/227）；github/github_issue: Support Podman in place of Docker（https://github.com/microsoft/magentic-ui/issues/312）；github/github_issue: Why not conduct a requirement analysis before the plan?（https://github.com/microsoft/magentic-ui/issues/321）；github/github_issue: Refreshing or restart the web app will make the current Session unavaila（https://github.com/microsoft/magentic-ui/issues/336）；github/github_issue: Sticked at click the “Shopping Cart” icon and cannot goto check out page（https://github.com/microsoft/magentic-ui/issues/360）；github/github_issue: magentic-ui can't display all the html element（https://github.com/microsoft/magentic-ui/issues/362）；github/github_release: Magentic-UI 0.1.5: \"Tell Me When\" tasks enabled by SentinelSteps（https://github.com/microsoft/magentic-ui/releases/tag/0.1.5）；github/github_release: Magentic-UI 0.1.2（https://github.com/microsoft/magentic-ui/releases/tag/v0.1.2）；github/github_release: Magentic-UI 0.1.1（https://github.com/microsoft/magentic-ui/releases/tag/v0.1.1）；github/github_release: Magentic-UI 0.1.0（https://github.com/microsoft/magentic-ui/releases/tag/v0.1.0）；github/github_release: Magentic-UI v0.0.6（https://github.com/microsoft/magentic-ui/releases/tag/v0.0.6）。这些是项目级外部声音，不作为单独质量证明。",
"items": [
{
"kind": "github_issue",
"source": "github",
"title": "Create tutorials and documentation for the codebase",
"url": "https://github.com/microsoft/magentic-ui/issues/154"
},
{
"kind": "github_issue",
"source": "github",
"title": "Settings redesign",
"url": "https://github.com/microsoft/magentic-ui/issues/227"
},
{
"kind": "github_issue",
"source": "github",
"title": "Support Podman in place of Docker",
"url": "https://github.com/microsoft/magentic-ui/issues/312"
},
{
"kind": "github_issue",
"source": "github",
"title": "Why not conduct a requirement analysis before the plan?",
"url": "https://github.com/microsoft/magentic-ui/issues/321"
},
{
"kind": "github_issue",
"source": "github",
"title": "Refreshing or restart the web app will make the current Session unavaila",
"url": "https://github.com/microsoft/magentic-ui/issues/336"
},
{
"kind": "github_issue",
"source": "github",
"title": "Sticked at click the “Shopping Cart” icon and cannot goto check out page",
"url": "https://github.com/microsoft/magentic-ui/issues/360"
},
{
"kind": "github_issue",
"source": "github",
"title": "magentic-ui can't display all the html element",
"url": "https://github.com/microsoft/magentic-ui/issues/362"
},
{
"kind": "github_release",
"source": "github",
"title": "Magentic-UI 0.1.5: \"Tell Me When\" tasks enabled by SentinelSteps",
"url": "https://github.com/microsoft/magentic-ui/releases/tag/0.1.5"
},
{
"kind": "github_release",
"source": "github",
"title": "Magentic-UI 0.1.2",
"url": "https://github.com/microsoft/magentic-ui/releases/tag/v0.1.2"
},
{
"kind": "github_release",
"source": "github",
"title": "Magentic-UI 0.1.1",
"url": "https://github.com/microsoft/magentic-ui/releases/tag/v0.1.1"
},
{
"kind": "github_release",
"source": "github",
"title": "Magentic-UI 0.1.0",
"url": "https://github.com/microsoft/magentic-ui/releases/tag/v0.1.0"
},
{
"kind": "github_release",
"source": "github",
"title": "Magentic-UI v0.0.6",
"url": "https://github.com/microsoft/magentic-ui/releases/tag/v0.0.6"
}
],
"status": "已收录 12 条来源",
"title": "社区讨论"
}
]
},
"homepage_card": {
"category": "信息检索与知识管理",
"desc": "A research prototype of a human-centered web agent",
"effort": "安装已验证",
"forks": 978,
"icon": "search",
"name": "magentic-ui 能力包",
"risk": "可发布",
"slug": "magentic-ui",
"stars": 9810,
"tags": [
"浏览器 Agent",
"网页任务自动化",
"浏览器自动化",
"断点恢复流程",
"评测体系"
],
"thumb": "blue",
"type": "Skill Pack"
},
"manual": {
"markdown": "# https://github.com/microsoft/magentic-ui 项目说明书\n\n生成时间：2026-05-18 02:19:23 UTC\n\n## 目录\n\n- [Getting Started with Magentic-UI](#getting-started)\n- [Key Concepts](#key-concepts)\n- [High-Level Architecture](#high-level-architecture)\n- [Agent System](#agent-system)\n- [Team Orchestration](#team-orchestration)\n- [Backend API](#backend-api)\n- [Frontend UI](#frontend-ui)\n- [Browser Automation](#browser-automation)\n- [Docker Containers](#docker-containers)\n- [Configuration](#configuration)\n\n\n\n## Getting Started with Magentic-UI\n\n### 相关页面\n\n相关主题：[Configuration](#configuration), [Docker Containers](#docker-containers)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [README.md](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n- [TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n- [CONTRIBUTING.md](https://github.com/microsoft/magentic-ui/blob/main/CONTRIBUTING.md)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [pyproject.toml](https://github.com/microsoft/magentic-ui/blob/main/pyproject.toml)\n \n\n# Getting Started with Magentic-UI\n\nMagentic-UI is a Microsoft open-source project that provides a multi-agent framework for building AI-powered user interfaces. It enables developers to create intelligent agents that can browse the web, execute plans, handle file uploads, and interact with users through a web-based chat interface.\n\n## Prerequisites\n\nBefore getting started with Magentic-UI, ensure your system meets the following requirements:\n\n| Requirement | Version/Details |\n|-------------|-----------------|\n| Python | 3.10 or higher |\n| Docker | Latest stable version |\n| Node.js | For frontend development |\n| pip | Latest version |\n\n资料来源：[README.md:1-20](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n### System Dependencies\n\n- **Docker**: Required for running agent containers that provide browser automation capabilities\n- **Node.js**: Needed only if you plan to modify the frontend code\n\n## Installation\n\n### Standard Installation\n\nThe simplest way to install Magentic-UI is via pip:\n\n```bash\npip install magentic-ui\n```\n\n资料来源：[README.md:25-30](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n### Installation with Fara-7B Support\n\nTo use the Fara-7B model locally, install with the fara extras:\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\npip install magentic-ui[fara]\n```\n\n资料来源：[README.md:55-60](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Quick Start with Docker\n\nAfter installation, start Magentic-UI using Docker:\n\n```bash\nmagentic-ui --port 8081\n```\n\n> **Note**: Running this command for the first time will pull two Docker images required for the Magentic-UI agents. If you encounter problems, you can build them directly with:\n\n```bash\ncd docker\nsh build-all.sh\n```\n\nOnce the server is running, access the UI at [http://localhost:8081](http://localhost:8081).\n\n资料来源：[README.md:30-45](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Local Development Setup\n\n### Backend Development\n\nFor local backend development, clone the repository and set up the environment:\n\n```bash\ngit clone https://github.com/microsoft/magentic-ui.git\ncd magentic-ui\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -e .\n```\n\nRun the development server:\n\n```bash\nmagentic ui --port 8081\n```\n\n资料来源：[TROUBLESHOOTING.md:1-20](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n### Frontend Development\n\nThe frontend is located in the `frontend/` directory of the repository. To set up for development:\n\n1. Navigate to the frontend directory:\n\n```bash\ncd frontend\n```\n\n2. Create environment configuration:\n\n```bash\ncp .env.default .env.development\n```\n\n3. Configure the API URL:\n\nEdit `.env.development` and set:\n\n```\nGATSBY_API_URL=http://localhost:8081/api\n```\n\n资料来源：[frontend/README.md:1-15](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Connecting Frontend to Backend\n\nThe frontend makes requests to the backend API expecting responses at `http://localhost:8081/api`. Ensure `GATSBY_API_URL` is correctly set in your environment configuration.\n\n资料来源：[frontend/README.md:10-12](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## Using Fara-7B Locally\n\nTo run Magentic-UI with a local Fara-7B model:\n\n### Step 1: Serve the Model\n\nIn a separate process, serve the Fara-7B model using vLLM:\n\n```bash\nvllm serve \"microsoft/Fara-7B\" --port 5000 --dtype auto \n```\n\n### Step 2: Create Configuration\n\nCreate a `fara_config.yaml` file with the following content:\n\n```yaml\nmodel_config_local_surfer: &client_surfer\n provider: OpenAIChatCompletionClient\n config:\n model: \"microsoft/Fara-7B\"\n base_url: http://localhost:5000/v1\n api_key: not-needed\n model_info:\n vision: true\n function_calling: true\n json_output: false\n family: \"unknown\" \n structured_output: false\n multiple_system_messages: false\n\norchestrator_client: *client\n```\n\n资料来源：[README.md:60-80](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Core Features\n\n### Web Surfer Agent\n\nThe web surfer agent enables browsing and interacting with web pages. Available actions include:\n\n| Action | Description |\n|--------|-------------|\n| `key` | Performs key presses (Enter, Alt, Shift, Tab, etc.) |\n| `type` | Types text into input fields |\n| `mouse_move` | Moves cursor to specified pixel coordinates |\n| `left_click` | Clicks the left mouse button |\n| `scroll` | Scrolls the mouse wheel |\n| `visit_url` | Navigates to a specified URL |\n| `web_search` | Performs a web search |\n| `history_back` | Goes back in browser history |\n| `pause_and_memorize_fact` | Stores information for future reference |\n| `wait` | Waits for specified seconds |\n| `terminate` | Ends the current task |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-60](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### Plans System\n\nMagentic-UI supports creating and managing reusable plans:\n\n- **Create Plans**: Build step-by-step task plans through the UI\n- **Attach Plans**: Attach saved plans to queries for execution\n- **Learn Plans**: Save successful conversation patterns as reusable plans\n- **Import/Export**: Import existing plans or export your library\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-50](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### File Handling\n\nThe system supports:\n\n- Arbitrary file uploads\n- File preview and download\n- Multiple file type support (images, documents, etc.)\n\n资料来源：[frontend/src/components/common/filerenderer.tsx:1-80](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n### MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers with multiple connection types:\n\n| Connection Type | Description |\n|-----------------|-------------|\n| SSE | Server-Sent Events connection |\n| Stdio | Standard input/output connection |\n| JSON Config | JSON-based configuration file |\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-60](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## Project Structure\n\n```\nmagentic-ui/\n├── frontend/ # React frontend application\n│ ├── src/\n│ │ ├── components/ # Reusable UI components\n│ │ │ ├── features/ # Feature-specific components\n│ │ │ ├── views/ # View components\n│ │ │ └── common/ # Common utilities\n│ │ └── pages/ # Page components\n│ └── README.md # Frontend development guide\n├── src/\n│ └── magentic_ui/ # Main Python package\n│ └── agents/ # Agent implementations\n├── docker/ # Docker configuration\n├── README.md # Main documentation\n├── CONTRIBUTING.md # Contribution guidelines\n└── TROUBLESHOOTING.md # Issue resolution guide\n```\n\n## Troubleshooting\n\n### Common Issues\n\n#### Port Already in Use\n\nIf port 8081 is occupied, either stop the existing service or use a different port:\n\n```bash\nmagentic ui --port 8082\n```\n\n#### Virtual Environment Activation\n\nIf you installed in a virtual environment but it didn't activate:\n\n```bash\ndeactivate\nsource .venv/bin/activate\nmagentic ui --port 8081\n```\n\n#### Wrong Package Installed\n\nEnsure you installed `magentic-ui` (not the unrelated `magentic` package):\n\n```bash\npip install magentic-ui\n```\n\n### Getting Help\n\nIf issues persist:\n\n1. Double-check all prerequisites in the README\n2. Search [GitHub Issues](https://github.com/microsoft/magentic-ui/issues) for similar problems\n3. Open a new issue with:\n - Detailed problem description\n - System information (OS, Docker version)\n - Steps to replicate\n\n资料来源：[TROUBLESHOOTING.md:1-50](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Contributing\n\nWe welcome community contributions:\n\n1. **Find an Issue**: Browse [All Issues](https://github.com/microsoft/magentic-ui/issues) and look for `help-wanted` labeled items\n2. **Fork and Clone**: Fork the repository and clone locally\n3. **Create a Branch**: Use descriptive names (e.g., `fix/session-bug` or `feature/file-upload`)\n4. **Write Code and Tests**: Include tests for new features\n5. **Run Checks**: Before submitting PR, run:\n\n```bash\npoe check\n```\n\n6. **Submit PR**: Open against `main` branch and reference the issue number\n\n### Top \"Help Wanted\" Issues\n\n| Issue | Description |\n|-------|-------------|\n| [#132](https://github.com/microsoft/magentic-ui/issues/132) | Allow MAGUI to understand video and audio |\n| [#128](https://github.com/microsoft/magentic-ui/issues/128) | Enable arbitrary file upload in UI |\n| [#126](https://github.com/microsoft/magentic-ui/issues/126) | Add streaming of final answer and coder messages |\n| [#123](https://github.com/microsoft/magentic-ui/issues/123) | Add unit tests |\n| [#124](https://github.com/microsoft/magentic-ui/issues/124) | Allow websurfer to scroll inside containers |\n\n资料来源：[CONTRIBUTING.md:1-40](https://github.com/microsoft/magentic-ui/blob/main/CONTRIBUTING.md)\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n A[User Interface] --> B[Frontend React App]\n B --> C[Backend API]\n C --> D[Magentic-UI Agents]\n D --> E[Web Surfer Agent]\n D --> F[Plans Executor]\n D --> G[MCP Servers]\n E --> H[Docker Containers]\n H --> I[Browser Automation]\n F --> J[Task Execution]\n G --> K[External Tools]\n```\n\n## Configuration Reference\n\n### Environment Variables (Frontend)\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `GATSBY_API_URL` | Backend API endpoint | `http://localhost:8081/api` |\n\n### Docker Ports\n\n| Port | Service |\n|------|---------|\n| 8081 | Main application (configurable) |\n| 5000 | vLLM model server (Fara-7B) |\n\n---\n\n\n\n## Key Concepts\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Agent System](#agent-system)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/approval_guard.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/approval_guard.py)\n- [src/magentic_ui/guarded_action.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/guarded_action.py)\n- [src/magentic_ui/learning/learner.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/learning/learner.py)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/hooks/store.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/store.tsx)\n \n\n# Key Concepts\n\nMagentic-UI is an interactive UI framework that enables AI agents to execute tasks with human oversight. This document explains the fundamental concepts that power the system's architecture, including approval workflows, guarded actions, plan management, and the learning system.\n\n---\n\n## 1. Approval Guard System\n\nThe Approval Guard System is a security mechanism that ensures AI agents require explicit human authorization before executing sensitive or irreversible actions. It acts as a gatekeeper between agent decision-making and action execution.\n\n### 1.1 Purpose and Scope\n\nThe approval guard intercepts actions proposed by agents and pauses execution until a human user provides approval or denial. This allows users to:\n\n- Review actions before they are executed\n- Modify parameters before approval\n- Deny dangerous or unintended operations\n- Maintain control over agent behavior in automated workflows\n\n### 1.2 Architecture Overview\n\n```mermaid\ngraph TD\n A[Agent Decision] --> B{Approval Required?}\n B -->|Yes| C[Approval Guard]\n B -->|No| D[Execute Action]\n C --> E[Pause Execution]\n E --> F[User Notification]\n F --> G{User Decision}\n G -->|Approve| H[Execute Action]\n G -->|Deny| I[Cancel Action]\n G -->|Modify| J[Update Parameters] --> H\n```\n\n### 1.3 Run Status States\n\nThe system uses a status-based workflow to track the state of agent runs. The following table documents the primary run states:\n\n| Status | Description |\n|--------|-------------|\n| `created` | Run has been initialized but not started |\n| `active` | Run is currently executing |\n| `awaiting_input` | Run is paused waiting for user input |\n| `paused` | Run has been paused by user or system |\n| `completed` | Run finished successfully |\n| `failed` | Run encountered an error |\n\n资料来源：[frontend/src/components/views/chat/runview.tsx:1-50]()\n\n### 1.4 Approval Guard Configuration\n\nThe approval guard supports different policy configurations:\n\n| Policy | Behavior |\n|--------|----------|\n| `AutoApprove` | All actions are automatically approved |\n| `RequireApproval` | All actions require explicit user approval |\n| `HybridPolicy` | Some actions auto-approved, others require approval |\n\n---\n\n## 2. Guarded Actions\n\nGuarded Actions represent the atomic operations that agents can perform. Each action has a type, parameters, and metadata about whether approval is required.\n\n### 2.1 Action Structure\n\nEach guarded action contains:\n\n```python\n{\n \"action\": str, # Action type identifier\n \"parameters\": dict, # Action-specific parameters\n \"requires_approval\": bool, # Whether approval is needed\n \"timestamp\": datetime, # When action was created\n \"status\": str # pending, approved, denied, executed\n}\n```\n\n### 2.2 Action Types\n\nThe system supports multiple action categories:\n\n| Category | Examples | Description |\n|----------|----------|-------------|\n| Navigation | `visit_url`, `history_back` | Browser navigation actions |\n| Input | `type`, `key`, `mouse_move` | User input simulation |\n| Control | `scroll`, `left_click` | Mouse and scroll interactions |\n| Search | `web_search` | Information retrieval |\n| System | `wait`, `terminate`, `pause_and_memorize_fact` | System-level operations |\n\n资料来源：[src/magentic_ui/guarded_action.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/guarded_action.py)\n\n### 2.3 Action Execution Flow\n\n```mermaid\nsequenceDiagram\n participant Agent\n participant Guard as Approval Guard\n participant User\n participant Executor\n \n Agent->>Guard: Propose Action\n Guard->>Guard: Check Approval Requirement\n alt Requires Approval\n Guard->>User: Display Action for Review\n User->>Guard: Approve/Deny/Modify\n alt Denied\n Guard->>Agent: Action Cancelled\n else Approved\n Guard->>Executor: Execute Action\n end\n else No Approval Required\n Guard->>Executor: Execute Action\n end\n Executor-->>Agent: Action Result\n```\n\n---\n\n## 3. Plans System\n\nThe Plans system enables users to create, save, and reuse task workflows. Plans consist of structured steps that guide agent behavior through complex multi-step tasks.\n\n### 3.1 Plan Structure\n\nA Plan consists of:\n\n| Component | Type | Description |\n|-----------|------|-------------|\n| `id` | string | Unique identifier |\n| `task` | string | High-level task description |\n| `steps` | array | Ordered list of execution steps |\n| `created_at` | datetime | Creation timestamp |\n| `metadata` | object | Additional configuration |\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### 3.2 Plan Execution\n\nWhen a plan is attached to a query:\n\n1. The plan is retrieved from the plan library\n2. Steps are displayed to the user for confirmation\n3. The agent follows the plan's structured workflow\n4. Progress is tracked through the progress bar component\n\n```mermaid\ngraph LR\n A[Create/Load Plan] --> B[Attach to Query]\n B --> C[Display Plan Steps]\n C --> D{User Approval?}\n D -->|Yes| E[Execute Plan]\n D -->|No| F[Cancel]\n E --> G[Track Progress]\n G --> H[Complete Task]\n```\n\n### 3.3 Plan Management\n\nPlans can be managed through the PlanCard component:\n\n- **Creation**: Users can create new plans from scratch\n- **Editing**: Existing plans can be modified through a modal interface\n- **Deletion**: Plans can be removed from the library\n- **Attachment**: Plans can be attached to queries via the chat input\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n---\n\n## 4. Learning System\n\nThe Learning System allows Magentic-UI to learn reusable plans from completed conversations. This enables knowledge transfer between sessions and automation of recurring tasks.\n\n### 4.1 Learn Plan Workflow\n\n```mermaid\ngraph TD\n A[Completed Conversation] --> B[Learn Plan Button]\n B --> C[Extract Task Structure]\n C --> D[Generate Plan Steps]\n D --> E[User Review]\n E --> F[Save to Library]\n F --> G[Available for Reuse]\n```\n\n### 4.2 Learning Process\n\n| Step | Description |\n|------|-------------|\n| 1 | Conversation completes with final answer |\n| 2 | User clicks \"Learn Plan\" button |\n| 3 | System analyzes conversation history |\n| 4 | Extracts reusable workflow patterns |\n| 5 | Presents plan for user confirmation |\n| 6 | Saves plan to user's plan library |\n\n资料来源：[src/magentic_ui/learning/learner.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/learning/learner.py)\n\n### 4.3 Learn Plan Button States\n\nThe LearnPlanButton component has multiple states:\n\n| State | Visual Indicator | Behavior |\n|-------|------------------|----------|\n| Loading | Spinner + \"Learning Plan...\" | Analyzing conversation |\n| Disabled | Opacity 50% | No session or user ID |\n| Ready | Blue button + \"Learn Plan\" | Ready to learn |\n| Dark Mode | Blue-400 text | Light theme variant |\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n---\n\n## 5. Chat Interface Components\n\nThe chat interface serves as the primary interaction point between users and the agent system.\n\n### 5.1 Chat Input Component\n\nThe ChatInput component handles user input and supports multiple input types:\n\n| Feature | Description |\n|---------|-------------|\n| Text Input | Free-form text queries |\n| File Attachment | Upload files via dropdown menu |\n| Plan Attachment | Attach predefined plans |\n| MCP Server Selection | Configure Model Context Protocol servers |\n| Pause Control | Pause active runs |\n\n资料来源：[frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n\n### 5.2 Run View Component\n\nThe RunView component displays the current state of an agent run:\n\n- Status indicators (icons for each run state)\n- Approval buttons for pending actions\n- Detail viewer for visual feedback\n- Input response handling\n\n```mermaid\ngraph TD\n A[User Input] --> B{ChatInput Component}\n B --> C{Run Status}\n C -->|awaiting_input| D[Show Input Request]\n C -->|paused| E[Show Pause Controls]\n C -->|active| F[Show Progress]\n D --> G[Approval Buttons]\n E --> H[Resume/Cancel]\n F --> I[Progress Bar]\n```\n\n资料来源：[frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n\n### 5.3 Progress Bar\n\nThe progress bar visualizes task completion:\n\n| Segment | Color | Description |\n|---------|-------|-------------|\n| Completed | Green (`#22c55e`) | Finished steps |\n| Current | Magenta (`#861657`) | Active step |\n| Remaining | Gray | Pending steps |\n\nWhen a final answer is available, the progress bar displays at 100% with a \"Task Completed\" status indicator.\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/progressbar.tsx)\n\n---\n\n## 6. State Management\n\nThe frontend uses a Zustand-based store for centralized state management.\n\n### 6.1 Store Structure\n\n| State Category | Key Properties |\n|----------------|----------------|\n| Session | `session`, `sessions`, `connectionId` |\n| Messages | `messages`, `setMessages` |\n| Configuration | `version`, `setVersion` |\n| Header | `title`, `breadcrumbs` |\n| Agent Flow | `direction`, `showLabels`, `showGrid`, `showTokens` |\n\n资料来源：[frontend/src/hooks/store.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/store.tsx)\n\n### 6.2 Agent Flow Settings\n\nAgent flow visualization settings:\n\n| Setting | Default | Description |\n|---------|---------|-------------|\n| `direction` | `\"TB\"` | Flow layout direction (TB=Top-Bottom) |\n| `showLabels` | `true` | Display node labels |\n| `showGrid` | `true` | Show background grid |\n| `showTokens` | `true` | Display token counts |\n| `showMessages` | `true` | Show message nodes |\n| `showMiniMap` | `false` | Enable minimap navigation |\n\n---\n\n## 7. MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers for extended agent capabilities.\n\n### 7.1 Server Configuration Types\n\n| Type | Use Case |\n|------|----------|\n| SSE | Server-Sent Events communication |\n| Stdio | Standard I/O subprocess communication |\n| JSON | Raw JSON configuration |\n\n### 7.2 Server Management\n\nServers can be added, updated, and removed through the MCPConfigModal component. Each server configuration includes:\n\n- Server name (alphanumeric, max 50 characters)\n- Connection type selection\n- Type-specific configuration parameters\n- Validation for duplicates and required fields\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n---\n\n## 8. File Handling\n\nThe system supports file uploads and rendering through the file renderer component.\n\n### 8.1 Supported File Operations\n\n| Operation | Description |\n|-----------|-------------|\n| Upload | Attach files to messages |\n| Preview | Display file thumbnails |\n| Download | Retrieve uploaded files |\n| Metadata | Extract and display file information |\n\n### 8.2 File Card Component\n\nFile cards display uploaded files with:\n\n- Icon representation based on file type\n- Truncated filename with full name tooltip\n- Download button for file retrieval\n- Click handler for file preview\n\n资料来源：[frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n---\n\n## 9. Browser Automation\n\nThe web surfer module provides browser automation capabilities through the FARA (Firefox Automation with Remote Access) system.\n\n### 9.1 Supported Browser Actions\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `key` | `keys[]` | Perform key press sequences |\n| `type` | `text`, `press_enter`, `delete_existing_text` | Type text input |\n| `mouse_move` | `coordinate[x,y]` | Move cursor to position |\n| `left_click` | `coordinate[x,y]` | Click at position |\n| `scroll` | `pixels` | Scroll wheel (positive=up, negative=down) |\n| `visit_url` | `url` | Navigate to URL |\n| `web_search` | `query` | Execute web search |\n| `history_back` | - | Navigate browser back |\n| `wait` | `time` (seconds) | Wait for page changes |\n| `terminate` | `status` | End task with status |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### 9.2 Display Configuration\n\n| Parameter | Description |\n|-----------|-------------|\n| `display_width_px` | Browser viewport width |\n| `display_height_px` | Browser viewport height |\n| `include_input_text_key_args` | Enable text input key arguments |\n\n---\n\n## 10. User Interface Layout\n\nThe application layout follows a structured component hierarchy.\n\n### 10.1 Layout Structure\n\n```mermaid\ngraph TD\n A[MagenticUILayout] --> B[ConfigProvider]\n A --> C[SessionManager]\n A --> D[Warning Banner]\n B --> C\n C --> E[Chat/RunView]\n C --> F[DetailViewer]\n E --> G[ChatInput]\n F --> H[Browser Modal]\n F --> I[Fullscreen Overlay]\n```\n\n### 10.2 Theme Support\n\nThe layout supports both light and dark themes through Ant Design's theme configuration system.\n\n### 10.3 Warning Banner\n\nA persistent disclaimer informs users about the system's limitations:\n\n> Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n---\n\n\n\n## High-Level Architecture\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [Team Orchestration](#team-orchestration), [Backend API](#backend-api)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/views/chat/detail_viewer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/detail_viewer.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n \n\n# High-Level Architecture\n\n## Overview\n\nMagentic-UI is a web-based interface that enables users to create, manage, and execute AI-driven task automation workflows. The system combines a React-based frontend with a Python backend to provide an interactive chat interface where users can execute multi-step plans, browse the web autonomously, and manage reusable workflow templates.\n\nThe architecture follows a **client-server model** where:\n- The **frontend** handles UI rendering, user interaction, and real-time display of task execution\n- The **backend** processes AI requests, manages agent execution, and coordinates with external tools\n- Communication occurs via RESTful API endpoints\n\n## System Components\n\n### Frontend Architecture\n\nThe frontend is a React application using TypeScript, organized into a modular component structure. It communicates with the backend API at `http://localhost:8081/api` as specified in the environment configuration.\n\n资料来源：[frontend/README.md:1-7]()\n\n```mermaid\ngraph TD\n A[User Browser] --> B[React Frontend]\n B --> C[Components]\n C --> D[Views/Chat]\n C --> E[Features]\n E --> F[Plans]\n E --> G[McpServersConfig]\n C --> H[Layout]\n B --> I[Backend API
localhost:8081/api]\n I --> J[Python Backend]\n J --> K[AI Agents]\n J --> L[Team Manager]\n```\n\n### Core Frontend Components\n\n| Component | File Path | Purpose |\n|-----------|-----------|---------|\n| `Chat` | `frontend/src/components/views/chat/chat.tsx` | Main chat interface container |\n| `RunView` | `frontend/src/components/views/chat/runview.tsx` | Manages run status and detail viewer |\n| `RenderMessage` | `frontend/src/components/views/chat/rendermessage.tsx` | Renders different message types |\n| `ChatInput` | `frontend/src/components/views/chat/chatinput.tsx` | User input with file/plan attachment |\n| `DetailViewer` | `frontend/src/components/views/chat/detail_viewer.tsx` | Browser view, screenshots, live tabs |\n| `ProgressBar` | `frontend/src/components/views/chat/progressbar.tsx` | Task progress visualization |\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:1-50]()\n\n## Frontend View Architecture\n\n### Chat System\n\nThe chat system is the primary user interaction point. It manages:\n- Display of conversation messages\n- Real-time run status updates\n- Progress tracking for multi-step tasks\n- Detail viewer integration for visual feedback\n\n```mermaid\ngraph LR\n A[ChatInput] -->|User Input| B[Chat Container]\n B --> C[RenderMessage]\n C -->|Multi-modal| D[PlanView]\n C -->|File| E[FileRenderer]\n B -->|Active Run| F[RunView]\n F --> G[DetailViewer]\n G --> H[Screenshots]\n G --> I[Live View]\n G --> J[BrowserModal]\n```\n\nThe chat component handles multiple run states:\n- `active` - Task is currently executing\n- `awaiting_input` - Waiting for user response\n- `paused` - Task temporarily paused\n- `pausing` - Pause operation in progress\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:30-40]()\n\n### Plan Management System\n\nPlans are reusable workflow templates that define multi-step task sequences. The plan system consists of:\n\n| Component | Function |\n|-----------|----------|\n| `PlanList` | Displays all saved plans with search/filter |\n| `PlanCard` | Individual plan summary with quick actions |\n| `PlanView` | Detailed plan editing and viewing |\n| `LearnPlanButton` | Creates new plans from conversation history |\n\n```mermaid\ngraph TD\n A[User Conversation] --> B[LearnPlanButton]\n B --> C[Plan Created]\n C --> D[PlanList]\n D --> E[PlanCard]\n E --> F[PlanView
Edit/View]\n F --> G[Save Plan]\n G --> D\n A --> H[Attach Plan to Chat]\n H --> I[Execute Plan]\n```\n\nPlans can be attached to chat queries using the dropdown menu in `ChatInput`, allowing users to:\n1. Create new empty plans\n2. Import plans from JSON files\n3. Search through existing plans\n4. Execute attached plans with the current query\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-80]()\n资料来源：[frontend/src/components/features/Plans/PlanList.tsx:1-50]()\n\n### Detail Viewer System\n\nThe detail viewer provides visual feedback during task execution through multiple tabs:\n\n| Tab | Purpose |\n|-----|---------|\n| Screenshots | Static screenshots captured during execution |\n| Live | Real-time browser view via noVNC |\n| Browser Modal | Full browser view in modal overlay |\n\nThe system supports control handover, allowing users to take control during autonomous browsing:\n\n```mermaid\ngraph TD\n A[Agent Execution] --> B[DetailViewer]\n B --> C[ScreenshotsTab]\n B --> D[LiveTab]\n D --> E[noVNC Connection]\n E --> F[User Control
Handover]\n F --> G[FullscreenOverlay]\n G --> H[User Input Response]\n H --> A\n```\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx:1-100]()\n资料来源：[frontend/src/components/views/chat/runview.tsx:1-50]()\n\n## MCP Server Configuration\n\nMagentic-UI supports the Model Context Protocol (MCP) for extending functionality through external servers. The configuration modal supports three connection types:\n\n| Connection Type | Description |\n|-----------------|-------------|\n| SSE | Server-Sent Events for streaming responses |\n| Stdio | Standard input/output process communication |\n| JSON Config | Direct JSON configuration import |\n\n```typescript\ninterface MCPConfig {\n serverName: string; // Unique identifier\n connectionType: 'sse' | 'stdio' | 'json';\n // Additional config based on type...\n}\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-100]()\n\n## Progress Tracking System\n\nThe progress bar component provides real-time feedback on task execution:\n\n```mermaid\ngraph TD\n A[Progress Update] --> B{Has Final Answer?}\n B -->|Yes| C[100% Complete
Green Bar]\n B -->|No| D[Calculate Percentage]\n D --> E[Current Step Highlight
Magenta]\n E --> F[Remaining Steps
Gray]\n C --> G[Status: Task Completed]\n F --> H[Status: Step X of Y]\n```\n\nThe progress system tracks:\n- `currentStep` - Current execution step index\n- `totalSteps` - Total number of steps in the plan\n- `plan.steps` - Array of step definitions with titles\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx:1-80]()\n\n## Message Rendering System\n\nMessages are rendered based on their content type:\n\n```mermaid\ngraph TD\n A[Raw Message] --> B{Parse Content}\n B --> C{Multi-Modal?}\n C -->|Yes| D[Map Each Item]\n C -->|No| E[Render as Text]\n D --> F{Is String?}\n F -->|Yes| G[Parse & Display]\n F -->|No| H[Skip/Empty]\n G --> I{Has Plan?}\n I -->|Yes| J[PlanView Component]\n J --> K[Final Output]\n H --> K\n E --> K\n```\n\nSupported content types:\n- Plain text with markdown rendering\n- Multi-step plans\n- File attachments with download capability\n- Code blocks\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx:1-100]()\n\n## Web Surfer Agent\n\nThe web surfer agent enables autonomous web browsing. It uses a structured action system:\n\n| Action | Purpose | Required Parameters |\n|--------|---------|---------------------|\n| `visit_url` | Navigate to URL | `url` |\n| `web_search` | Search the web | `query` |\n| `scroll` | Scroll page | `pixels` |\n| `click` | Click element | `coordinate` |\n| `type` | Input text | `text`, `coordinate` |\n| `pause_and_memorize_fact` | Store information | `fact` |\n| `wait` | Wait for page | `time` |\n| `terminate` | End task | `status` |\n\n```python\nparameters = {\n \"action\": {\n \"type\": \"string\",\n \"enum\": [\"visit_url\", \"web_search\", \"scroll\", \"click\", ...]\n },\n \"coordinate\": {\n \"description\": \"(x, y): The x and y coordinates for mouse actions\",\n \"type\": \"array\"\n }\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-80]()\n\n## Data Flow\n\n### User Query Flow\n\n```mermaid\nsequenceDiagram\n participant User\n participant ChatInput\n participant Backend\n participant Agent\n participant DetailViewer\n \n User->>ChatInput: Submit query\n ChatInput->>Backend: POST /api/run\n Backend->>Agent: Execute task\n Agent->>DetailViewer: Stream screenshots\n Agent->>Backend: Progress updates\n Backend->>ChatInput: Status updates\n Agent->>Backend: Final response\n Backend->>User: Display result\n```\n\n### Plan Attachment Flow\n\n```mermaid\ngraph TD\n A[User clicks attach] --> B[Dropdown shows plans]\n B --> C[Select plan]\n C --> D[PlanView Modal opens]\n D --> E[Confirm attachment]\n E --> F[Plan attached to input]\n F --> G[Submit with plan context]\n```\n\n## State Management\n\nThe frontend manages several key state objects:\n\n```typescript\ninterface ChatState {\n currentRun: Run | null;\n runStatus: 'idle' | 'active' | 'paused' | 'awaiting_input';\n progress: {\n currentStep: number;\n totalSteps: number;\n plan?: Plan;\n };\n hasFinalAnswer: boolean;\n}\n\ninterface PlanState {\n task: string;\n steps: Step[];\n created_at?: string;\n}\n```\n\n## Security Considerations\n\nThe layout includes a disclaimer for user awareness:\n\n> Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\n\nControl handover features allow users to take control from autonomous agents during execution, ensuring human oversight of automated tasks.\n\n资料来源：[frontend/src/components/layout.tsx:1-50]()\n\n## Configuration\n\n### Environment Variables\n\n| Variable | Default | Purpose |\n|----------|---------|---------|\n| `GATSBY_API_URL` | `http://localhost:8081/api` | Backend API endpoint |\n| API requests target | `http://localhost:8081/api` | Frontend-backend communication |\n\n### Theme Support\n\nThe application supports light and dark themes using Ant Design's theme algorithm system, dynamically switching between `darkAlgorithm` and `defaultAlgorithm` based on user preference.\n\n资料来源：[frontend/README.md:1-10]()\n资料来源：[frontend/src/components/layout.tsx:1-30]()\n\n---\n\n\n\n## Agent System\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Team Orchestration](#team-orchestration), [Browser Automation](#browser-automation)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n \n\n# Agent System\n\n## Overview\n\nThe Agent System in Magentic-UI is a multi-agent orchestration framework that enables autonomous task execution through specialized agents. The system coordinates various agent types including web surfers, file surfers, coders, and user proxies to accomplish complex tasks requested by users.\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n User[User Request] --> Orchestrator[Orchestrator Agent]\n Orchestrator --> WebSurfer[Web Surfer Agent]\n Orchestrator --> FileSurfer[File Surfer Agent]\n Orchestrator --> Coder[Coder Agent]\n Orchestrator --> UserProxy[User Proxy Agent]\n \n WebSurfer --> Browser[Browser Control]\n FileSurfer --> FileSystem[File System]\n Coder --> CodeExecution[Code Executor]\n UserProxy --> UserApproval[User Approval]\n \n Browser --> StateUpdate[State Update]\n FileSystem --> StateUpdate\n CodeExecution --> StateUpdate\n UserApproval --> StateUpdate\n \n StateUpdate --> Orchestrator\n```\n\n## Agent Types\n\n### Web Surfer Agent\n\nThe Web Surfer Agent enables autonomous web browsing by controlling a browser instance. It handles various browsing actions including navigation, interaction, and information retrieval.\n\n#### Supported Actions\n\n| Action | Description | Required Parameters |\n|--------|-------------|---------------------|\n| `visit_url` | Navigate to a specific URL | `url` |\n| `web_search` | Perform a web search | `query` |\n| `left_click` | Click at coordinates | `coordinate` |\n| `right_click` | Right-click at coordinates | `coordinate` |\n| `mouse_move` | Move mouse to coordinates | `coordinate` |\n| `type` | Type text into an element | `text`, `coordinate` |\n| `scroll` | Scroll the page | `pixels` |\n| `key` | Press keyboard keys | `keys` |\n| `pause_and_memorize_fact` | Store information for later use | `fact` |\n| `wait` | Wait for specified duration | `time` |\n| `history_back` | Navigate back in browser history | - |\n| `terminate` | End the browsing session | `status` |\n\n#### Configuration Parameters\n\nThe Web Surfer Agent supports the following configuration options:\n\n```python\n{\n \"display_width_px\": int, # Browser viewport width\n \"display_height_px\": int, # Browser viewport height\n \"include_input_text_key_args\": bool # Include type-specific arguments\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-80]()\n\n### File Surfer Agent\n\nThe File Surfer Agent provides file system navigation and file content interaction capabilities. It allows agents to read, write, and manage files within the project workspace.\n\n### Coder Agent\n\nThe Coder Agent handles code generation, analysis, and execution tasks. It works in conjunction with the orchestrator to implement requested functionality.\n\n### User Proxy Agent\n\nThe User Proxy Agent acts as an intermediary between the autonomous agent system and human users. It handles:\n\n- Requesting user confirmation for sensitive operations\n- Presenting information that requires human judgment\n- Managing user input during interactive sessions\n\n## Agent Communication Flow\n\n```mermaid\nsequenceDiagram\n participant User\n participant Frontend\n participant Orchestrator\n participant Agent\n \n User->>Frontend: Submit Task\n Frontend->>Orchestrator: Send Request\n Orchestrator->>Agent: Delegate Subtask\n Agent->>Agent: Execute Action\n Agent-->>Orchestrator: Return Result\n \n alt Requires Approval\n Orchestrator->>User: Request Approval\n User-->>Orchestrator: Approval/Denial\n end\n \n Orchestrator-->>Frontend: Final Response\n Frontend-->>User: Display Result\n```\n\n## Task Execution Workflow\n\nWhen a user submits a task through the chat interface, the system follows this execution model:\n\n1. **Task Submission**: User enters a query via `ChatInput` component\n2. **Agent Selection**: The orchestrator determines which agent(s) to invoke\n3. **Execution**: Selected agents perform their designated actions\n4. **Progress Tracking**: The system displays execution progress via `ProgressBar`\n5. **State Updates**: Real-time updates are rendered via `RenderMessage`\n6. **Completion**: Final results are presented with option to save plans\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:50-120]()\n\n## Plan System Integration\n\nThe Agent System integrates with a Plan System that breaks down complex tasks into executable steps:\n\n```mermaid\ngraph LR\n Task[User Task] --> Plan[Generated Plan]\n Plan --> Step1[Step 1]\n Plan --> Step2[Step 2]\n Plan --> Step3[Step 3]\n \n Step1 --> Execute1[Execute]\n Step2 --> Execute2[Execute]\n Step3 --> Execute3[Execute]\n \n Execute1 --> Result1[Result]\n Execute2 --> Result2[Result]\n Execute3 --> Result3[Result]\n \n Result1 --> Aggregate[Aggregate Results]\n Result2 --> Aggregate\n Result3 --> Aggregate\n```\n\n### Plan Components\n\n| Component | File | Purpose |\n|-----------|------|---------|\n| `PlanCard` | `PlanCard.tsx` | Displays individual plan summary |\n| `PlanList` | `PlanList.tsx` | Lists all available plans |\n| `PlanView` | `PlanView.tsx` | Interactive plan editor/viewer |\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-100]()\n\n## Message Rendering System\n\nThe Agent System communicates results through a structured message rendering system:\n\n```mermaid\ngraph TD\n Message[Agent Message] --> Parse[Parse Content]\n Parse --> Type{Message Type}\n \n Type -->|Text| TextRender[Text Renderer]\n Type -->|Plan| PlanRender[Plan View]\n Type -->|File| FileRender[File Renderer]\n Type -->|Image| ImageRender[Image Renderer]\n \n TextRender --> Display[UI Display]\n PlanRender --> Display\n FileRender --> Display\n ImageRender --> Display\n```\n\nThe `RenderMessage` component handles the display of agent outputs, supporting:\n\n- Multi-modal content rendering\n- Plan visualization\n- File previews and downloads\n- Image galleries\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx:1-100]()\n\n## Browser Control Details\n\nThe Web Surfer Agent uses coordinate-based browser control:\n\n### Coordinate System\n\n- **X-axis**: Pixels from the left edge of the viewport\n- **Y-axis**: Pixels from the top edge of the viewport\n- **Scroll**: Positive values scroll up, negative values scroll down\n\n### Type Action Parameters\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `text` | string | Text to type |\n| `coordinate` | [x, y] | Target element position |\n| `press_enter` | boolean | Submit after typing |\n| `delete_existing_text` | boolean | Clear before typing |\n\n## Run Status States\n\nThe agent execution maintains the following status states:\n\n| Status | Description |\n|--------|-------------|\n| `active` | Agent is currently executing |\n| `paused` | Execution paused, awaiting resume |\n| `pausing` | Pause is in progress |\n| `awaiting_input` | Waiting for user input or approval |\n| `completed` | Task finished successfully |\n| `failed` | Task execution failed |\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:30-60]()\n\n## Configuration Management\n\nThe Agent System configuration is managed through the frontend store:\n\n```typescript\ninterface IAgentFlowSettings {\n direction: \"TB\" | \"LR\"; // Flow chart orientation\n showLabels: boolean; // Display edge labels\n showGrid: boolean; // Show background grid\n showTokens: boolean; // Display token counts\n showMessages: boolean; // Show message nodes\n showMiniMap: boolean; // Show navigation minimap\n}\n```\n\n资料来源：[frontend/src/hooks/store.tsx:1-80]()\n\n## MCP Server Integration\n\nThe system supports Model Context Protocol (MCP) servers for extended agent capabilities:\n\n- SSE-based server connections\n- Stdio-based server connections\n- JSON configuration import\n\nMCP servers are configured via the `McpConfigModal` component and integrated into the agent selection process during task execution.\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-100]()\n\n## Detail Viewer\n\nThe `DetailViewer` component provides real-time visualization of agent activities:\n\n- **Screenshots Tab**: Periodic screenshots of browser state\n- **Live Tab**: Real-time browser view via noVNC\n- **Control Mode**: Fullscreen overlay for manual intervention\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx:1-100]()\n[frontend/src/components/views/chat/runview.tsx:1-80]()\n\n## Summary\n\nThe Agent System in Magentic-UI provides a comprehensive framework for autonomous task execution through:\n\n- **Specialized Agents**: Web Surfer, File Surfer, Coder, User Proxy\n- **Orchestration Layer**: Coordinates multi-agent collaboration\n- **Plan System**: Breaks tasks into executable steps\n- **Real-time Visualization**: Browser screenshots and live view\n- **Human-in-the-Loop**: User proxy for approval and input\n- **MCP Integration**: Extensible server architecture\n\nThis architecture enables complex, multi-step task automation while maintaining user oversight and control throughout the execution process.\n\n---\n\n\n\n## Team Orchestration\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [High-Level Architecture](#high-level-architecture)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n- [src/magentic_ui/agents/_coder.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/_coder.py)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n \n\n# Team Orchestration\n\n## Overview\n\nTeam Orchestration is a core system in Magentic-UI that coordinates multiple AI agents to work together on complex tasks. It provides the infrastructure for orchestrating agent teams, managing communication between agents, and handling task distribution and execution flow.\n\nThe orchestration system enables Magentic-UI to:\n\n- Coordinate multiple specialized agents (coders, web surfers, planners)\n- Manage agent collaboration through structured message passing\n- Handle approval policies for sensitive actions\n- Support dynamic task execution with planning and reflection capabilities\n\n## Architecture\n\n### Core Components\n\n| Component | File Path | Purpose |\n|-----------|-----------|---------|\n| Orchestrator | `src/magentic_ui/teams/orchestrator/_orchestrator.py` | Central coordinator for agent teams |\n| Group Chat | `src/magentic_ui/teams/orchestrator/_group_chat.py` | Manages multi-agent message passing |\n| Prompts | `src/magentic_ui/teams/orchestrator/_prompts.py` | Prompt templates for orchestration |\n| Sentinel Prompts | `src/magentic_ui/teams/orchestrator/_sentinel_prompts.py` | Safety and monitoring prompts |\n\n### Agent Types\n\nMagentic-UI supports several specialized agent types that can be orchestrated:\n\n| Agent Type | Purpose | Key Actions |\n|------------|---------|-------------|\n| Coder Agent | Execute Python/code tasks | Write, debug, execute code |\n| Web Surfer Agent | Browse and interact with web content | scroll, visit_url, web_search, wait |\n| Planner Agent | Create and manage execution plans | Task decomposition, step planning |\n| MCP Server Agents | External tool integrations | Configurable via SSE/stdio protocols |\n\n## Web Surfer Agent Actions\n\nThe web surfer agent supports a comprehensive set of actions for web interaction:\n\n```python\nparameters = {\n \"properties\": {\n \"scroll\": {\n \"description\": \"The number of pixels to scroll. Positive scrolls down, negative scrolls up.\",\n \"type\": \"number\",\n },\n \"url\": {\n \"description\": \"The URL to visit. Required only by `action=visit_url`.\",\n \"type\": \"string\",\n },\n \"query\": {\n \"description\": \"The query to search for. Required only by `action=web_search`.\",\n \"type\": \"string\",\n },\n \"fact\": {\n \"description\": \"The fact to remember for the future. Required only by `action=pause_and_memorize_fact`.\",\n \"type\": \"string\",\n },\n \"time\": {\n \"description\": \"The seconds to wait. Required only by `action=wait`.\",\n \"type\": \"number\",\n },\n \"status\": {\n \"description\": \"The status of the task. Required only by `action=terminate`.\",\n \"type\": \"string\",\n \"enum\": [\"success\", \"failure\"],\n },\n },\n \"required\": [\"action\"],\n \"type\": \"object\",\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-50](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### Action Types\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `scroll` | `scroll` (pixels) | Scrolls the viewport |\n| `visit_url` | `url` | Navigate to a URL |\n| `web_search` | `query` | Search the web |\n| `pause_and_memorize_fact` | `fact` | Store information for context |\n| `wait` | `time` (seconds) | Wait before continuing |\n| `terminate` | `status` (success/failure) | End the task |\n\n## Function Call Handling\n\nThe system uses a specialized function call prompt system for agent communication:\n\n```python\ntool_descs = [{\"type\": \"function\", \"function\": f} for f in functions]\ntool_names = [\n function.get(\"name_for_model\", function.get(\"name\", \"\"))\n for function in functions\n]\ntool_descs = \"\\n\".join([json.dumps(f, ensure_ascii=False) for f in tool_descs])\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py:1-60](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py)\n\n### Message Processing Flow\n\n```\ngraph TD\n A[User Input] --> B[Orchestrator]\n B --> C{Agent Selection}\n C -->|Planning| D[Planner Agent]\n C -->|Execution| E[Coder Agent]\n C -->|Web Tasks| F[Web Surfer Agent]\n D --> G[Execution Plan]\n E --> H[Code Execution]\n F --> I[Web Actions]\n G --> B\n H --> B\n I --> B\n B --> J[User Response]\n```\n\n### Role-Based Message Handling\n\n| Role | Processing | Content Format |\n|------|------------|----------------|\n| `ASSISTANT` | Appends tool calls to last message | `` XML blocks |\n| `FUNCTION` | Processes tool responses | `` XML blocks |\n| `USER` | Standard user messages | Plain text or structured |\n\n## MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers for extended functionality:\n\n```typescript\ninterface McpServerConfig {\n serverName: string;\n agentName: string;\n agentDescription: string;\n connectionType: 'sse' | 'stdio' | 'json';\n}\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n### MCP Configuration Modes\n\n| Mode | Description | Use Case |\n|------|-------------|----------|\n| SSE | Server-Sent Events | Remote server connections |\n| Stdio | Standard I/O | Local process communication |\n| JSON Config | Raw JSON configuration | Advanced users |\n\n### Agent Description Requirements\n\nEach MCP server requires a description that helps the orchestrator decide when to invoke it:\n\n> \"Describe how and when this server should be used. This helps the orchestrator decide when to call this agent.\"\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-120](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## Code Execution Flow\n\nThe coder agent provides secure code execution with output capture:\n\n```python\nasync def _summarize_coding(\n agent_name: str,\n model_client: ChatCompletionClient,\n thread: Sequence[BaseChatMessage | BaseAgentEvent],\n cancellation_token: CancellationToken,\n model_context: ChatCompletionContext,\n) -> TextMessage:\n```\n\n资料来源：[src/magentic_ui/agents/_coder.py:1-100](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/_coder.py)\n\n### Code Execution States\n\n| State | Description | Exit Code |\n|-------|-------------|-----------|\n| Success | Code executed without errors | `0` |\n| Timeout | Execution exceeded time limit | N/A |\n| Error | Runtime exception occurred | Non-zero |\n\n```python\n# Break if all code executions were successful\nif all([code_output == 0 for code_output in exit_code_list]):\n break\n```\n\n## CLI Configuration\n\nThe orchestration system is configured through the CLI entry point:\n\n```python\ndef main() -> None:\n \"\"\"\n Entry point for the magentic-cli command.\n Called from pyproject.toml's [project.scripts] section.\n \"\"\"\n app()\n```\n\n资料来源：[src/magentic_ui/_cli.py:1-50](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n\n### CLI Parameters\n\n| Parameter | Type | Purpose |\n|-----------|------|---------|\n| `mcp_agents` | List | External MCP server agents |\n| `run_without_docker` | bool | Run without container isolation |\n| `browser_headless` | bool | Run browser in headless mode |\n| `browser_local` | bool | Use local browser instead of remote |\n| `sentinel_tasks` | List | Background monitoring tasks |\n| `dynamic_sentinel_sleep` | int | Sleep interval for sentinel checks |\n\n## Approval Policies\n\nThe orchestration system supports configurable approval policies for controlling agent actions:\n\n| Policy | Behavior |\n|--------|----------|\n| `Auto Approve` | All actions execute automatically |\n| `Manual Approval` | User must approve each action |\n| `Policy Based` | Rules determine approval based on action type |\n\n## Error Handling\n\n### Timeout Handling\n\n```python\nexcept asyncio.TimeoutError:\n executor_msg = TextMessage(\n source=agent_name + \"-executor\",\n metadata={\"internal\": \"yes\"},\n content=\"Code execution timed out.\",\n )\n delta.append(executor_msg)\n yield executor_msg\n```\n\n## Summary\n\nTeam Orchestration in Magentic-UI provides a flexible framework for coordinating multiple AI agents. Key features include:\n\n1. **Multi-Agent Coordination**: Specialized agents work together through the orchestrator\n2. **Flexible Communication**: Role-based message passing with XML-formatted tool calls\n3. **MCP Integration**: Extensible architecture through Model Context Protocol servers\n4. **Safe Execution**: Code execution with timeout handling and error capture\n5. **Approval Workflows**: Configurable policies for sensitive operations\n\nThe system is designed to be modular, allowing new agent types and capabilities to be added through well-defined interfaces.\n\n---\n\n\n\n## Backend API\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Frontend UI](#frontend-ui)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n \n\n# Backend API\n\n## Overview\n\nThe Magentic-UI Backend API is a FastAPI-based REST/WebSocket service that orchestrates multi-agent workflows, manages conversation sessions, and provides real-time execution capabilities for the frontend interface. It serves as the central hub for all backend operations including session management, run execution, agent coordination, and MCP (Model Context Protocol) integration.\n\nThe API is accessible at `http://localhost:8081/api` for local development and expects all frontend requests to be directed to this endpoint. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## Architecture\n\n### High-Level Architecture\n\n```mermaid\ngraph TB\n subgraph \"Frontend Client\"\n UI[UI Components]\n end\n \n subgraph \"Backend API - FastAPI\"\n App[Main Application]\n Routers[Route Handlers]\n Managers[Connection Managers]\n end\n \n subgraph \"Data Layer\"\n DB[(Database)]\n StaticFiles[Static Files]\n end\n \n UI --> |HTTP/WS| App\n App --> Routers\n Routers --> Managers\n Managers --> DB\n App --> StaticFiles\n```\n\n### API Router Structure\n\nThe backend organizes its functionality into modular routers, each handling a specific domain:\n\n| Router | Prefix | Purpose |\n|--------|--------|---------|\n| Teams Router | `/teams` | Multi-agent team coordination |\n| WebSocket Router | `/ws` | Real-time bidirectional communication |\n| Validation Router | `/validate` | Input validation endpoints |\n| Settings Router | `/settings` | User and system configuration |\n| MCP Router | `/mcp` | Model Context Protocol integration |\n| Sessions Router | `/sessions` | Conversation session management |\n| Runs Router | `/runs` | Execution run tracking and control |\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## Core Endpoints\n\n### Health Check\n\n```\nGET /api/health\n```\n\nReturns the health status of the API service.\n\n**Response:**\n```json\n{\n \"status\": true,\n \"message\": \"Service is healthy\"\n}\n```\n\n### Version Information\n\n```\nGET /api/version\n```\n\nRetrieves the current API version.\n\n**Response:**\n```json\n{\n \"status\": true,\n \"message\": \"Version retrieved successfully\",\n \"data\": {\n \"version\": \"\"\n }\n}\n```\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## API Configuration\n\n### Environment Variables\n\nThe frontend must be configured with the correct API URL. Create a `.env.development` file based on `.env.default`:\n\n```bash\ncp .env.default .env.development\n```\n\nThe primary configuration variable is `GATSBY_API_URL` which should be set to `http://localhost:8081/api` for local development. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Static File Serving\n\nThe backend mounts two static file directories:\n\n| Mount Path | Directory | Purpose |\n|------------|-----------|---------|\n| `/files` | `static_root` | File downloads with HTML fallback |\n| `/` | `ui_root` | Frontend UI assets |\n\n```python\napp.mount(\n \"/files\",\n StaticFiles(directory=initializer.static_root, html=True),\n name=\"files\",\n)\napp.mount(\"/\", StaticFiles(directory=initializer.ui_root, html=True), name=\"ui\")\n```\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## Error Handling\n\n### Internal Server Error Handler\n\nThe API includes a global exception handler for 500 errors:\n\n```python\n@app.exception_handler(500)\nasync def internal_error_handler(request: Request, exc: Exception):\n logger.error(f\"Internal error: {str(exc)}\")\n return {\n \"status\": False,\n \"message\": \"Internal server error\",\n \"detail\": str(exc) if settings.debug else None\n }\n```\n\nThis handler:\n- Logs the full error details server-side\n- Returns sanitized error messages to clients (hiding details in production)\n- Uses `settings.debug` to control error visibility 资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## WebSocket Communication\n\nThe WebSocket router (`/ws`) enables real-time bidirectional communication between the frontend and backend. This is essential for:\n\n- Live agent execution progress updates\n- Streaming intermediate results\n- Real-time user input responses during agent runs\n- Session state synchronization\n\n## Agent Actions and Tool Integration\n\nThe backend exposes agent capabilities through structured action parameters. Agents support the following action types:\n\n| Action | Description | Required Parameters |\n|--------|-------------|---------------------|\n| `key` | Perform keyboard key presses | `keys` (array) |\n| `type` | Type text into input fields | `text`, `press_enter`, `delete_existing_text` |\n| `mouse_move` | Move cursor to coordinates | `coordinate` [x, y] |\n| `left_click` | Click at coordinates | `coordinate` [x, y] |\n| `scroll` | Scroll mouse wheel | `pixels` |\n| `visit_url` | Navigate to URL | `url` |\n| `web_search` | Execute web search | `query` |\n| `history_back` | Go to previous page | - |\n| `pause_and_memorize_fact` | Store information | `fact` |\n| `wait` | Pause execution | `time` (seconds) |\n| `terminate` | End task | `status` (success/failure) |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n## Running the Backend\n\n### Prerequisites\n\nEnsure all prerequisites are installed before running the backend. The system requires:\n\n- Python environment with dependencies installed\n- Node.js and npm for frontend development (if building from source)\n- nvm for Node version management\n\n### Starting the Server\n\n```bash\nmagentic-ui --port 8081\n```\n\nThe server will:\n1. Initialize the FastAPI application\n2. Mount all routers under `/api` prefix\n3. Establish database connections\n4. Start listening on the specified port\n\n### Development Mode\n\nFor frontend development with hot-reloading:\n\n1. Start frontend in development mode:\n```bash\ncd frontend\nnpm run start\n```\n\n2. Run the backend:\n```bash\nmagentic-ui --port 8081\n```\n\nThe frontend development server runs at `http://localhost:8000`, while the compiled frontend is available at `http://localhost:8081`. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## API Response Format\n\nAll API responses follow a consistent format:\n\n```json\n{\n \"status\": true | false,\n \"message\": \"Human-readable status message\",\n \"data\": { ... } | null,\n \"detail\": \"Error details (optional, debug mode only)\"\n}\n```\n\nThis standardization allows the frontend to handle all responses uniformly regardless of which router handled the request.\n\n## Related Documentation\n\nFor troubleshooting and setup issues, refer to the [TROUBLESHOOTING.md](TROUBLESHOOTING.md) file in the repository root.\n\n---\n\n\n\n## Frontend UI\n\n### 相关页面\n\n相关主题：[Backend API](#backend-api), [High-Level Architecture](#high-level-architecture)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/manager.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/manager.tsx)\n- [frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n- [frontend/src/components/features/McpServersConfig/McpServerCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpServerCard.tsx)\n- [frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n- [frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n \n\n# Frontend UI\n\n## Overview\n\nThe Frontend UI of magentic-ui is a React-based web application that provides an interactive interface for users to interact with AI agents. The frontend communicates with the backend API at `http://localhost:8081/api` and enables features such as chat conversations, plan management, MCP server configuration, and real-time task execution visualization.\n\nThe UI is built using:\n- **React** with TypeScript for component architecture\n- **Ant Design** as the primary UI component library\n- **Tailwind CSS** for custom styling\n- **React Markdown** for rendering markdown content\n\n资料来源：[frontend/package.json](https://github.com/microsoft/magentic-ui/blob/main/frontend/package.json)\n\n## Architecture Overview\n\nThe frontend application follows a component-based architecture with clear separation between layout, views, features, and common components.\n\n```mermaid\ngraph TD\n A[App Entry] --> B[MagenticUILayout]\n B --> C[SessionManager]\n C --> D[Views]\n C --> E[SubMenus]\n D --> F[ChatView]\n D --> G[RunView]\n D --> H[PlanList]\n E --> I[SessionList]\n E --> J[PlanLibrary]\n F --> K[ChatInput]\n F --> L[RenderMessage]\n F --> M[ProgressBar]\n F --> N[DetailViewer]\n G --> K\n G --> L\n G --> M\n G --> N\n```\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n## Application Layout\n\n### MagenticUILayout\n\nThe main layout wrapper component that provides theme configuration and session management context to all child components.\n\n| Prop | Type | Description |\n|------|------|-------------|\n| restricted | boolean | Whether to restrict access to authenticated users only |\n| children | ReactNode | Child components to render within the layout |\n\nKey responsibilities:\n- Applies theme algorithms (dark/light mode) via Ant Design's `ConfigProvider`\n- Wraps content in `AppContext` for global state access\n- Displays a disclaimer footer: \"Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\"\n\n资料来源：[frontend/src/components/layout.tsx:1-100](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n### SessionManager\n\nThe central orchestrator component that manages the overall application state including sessions, plans, and navigation between different views.\n\n```mermaid\ngraph LR\n A[SessionManager] --> B[PlanList]\n A --> C[ChatView]\n A --> D[SessionEditor]\n B --> E[PlanCard]\n C --> F[ChatInput]\n C --> G[MessageList]\n C --> H[RunView]\n```\n\nState management includes:\n- `activeSubMenuItem`: Current navigation state\n- `sessions`: List of user sessions\n- `currentRun`: Active task execution state\n- `selectedMcpServers`: Selected MCP server configurations\n- `editingSession`: Session being edited\n\n资料来源：[frontend/src/components/views/manager.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/manager.tsx)\n\n## Chat System\n\n### ChatView\n\nThe primary chat interface where users interact with AI agents through messages and file uploads.\n\n```mermaid\nsequenceDiagram\n User->>ChatInput: Enter message\n ChatInput->>ChatView: handleSubmit(query, files, plan)\n ChatView->>Backend: runTask() via API\n Backend-->>ChatView: CurrentRun status\n ChatView->>RunView: Pass run data\n RunView->>MessageList: Display messages\n MessageList->>RenderMessage: Render each message\n```\n\n**Key Features:**\n- Message submission with text, files, and attached plans\n- Real-time run status display (running, paused, awaiting_input, completed)\n- MCP server selection for task execution\n- Plan execution control (approve, deny, pause, cancel)\n- Sample tasks for quick start\n\n资料来源：[frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n\n### ChatInput\n\nA rich text input component supporting multi-line input, file attachments, and plan attachments.\n\n**Props:**\n| Prop | Type | Description |\n|------|------|-------------|\n| onSubmit | Function | Callback when message is submitted |\n| onCancel | Function | Callback to cancel current operation |\n| runStatus | string | Current run status |\n| inputRequest | object | Request for user input |\n| isPlanMessage | boolean | Whether input is for plan response |\n| onPause | Function | Callback to pause execution |\n| onExecutePlan | Function | Callback to execute a plan |\n| enable_upload | boolean | Enable file uploads |\n| selectedMcpServers | array | Selected MCP servers |\n\n**Features:**\n- File drag-and-drop and paste support\n- File list display with remove capability\n- Plan attachment modal for viewing attached plans\n- Auto-resizing textarea\n- Submit button with loading state\n\n资料来源：[frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n\n### RenderMessage\n\nComponent responsible for rendering different types of messages including user messages, AI responses, plans, and file previews.\n\n**Rendering Logic:**\n1. Checks if message is from user or assistant\n2. Parses content using `parseContent` utility\n3. Handles multi-modal content (text arrays)\n4. Renders plans via `PlanView` component\n5. Applies appropriate styling based on message type\n\n```typescript\n// Message type detection based on metadata\nif (message?.metadata?.type === \"file\" && message?.metadata?.files) {\n // File message handling\n const parsedFiles = JSON.parse(message.metadata.files);\n}\n```\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n\n### ProgressBar\n\nVisual indicator for task execution progress with step-by-step status display.\n\n**Display States:**\n- **Task Completed**: Green progress bar at 100% with \"Task Completed\" text\n- **In Progress**: Shows current step (e.g., \"Step 2 of 5\") with progress bar\n- **Planning**: Shows \"Planning...\" text when plan is being generated\n\n**Progress Calculation:**\n```javascript\n// Completed section width\nwidth: hasFinalAnswer ? \"100%\" : (currentStep / totalSteps) * 100 + \"%\"\n\n// Current step indicator position\nleft: (currentStep / totalSteps) * 100 + \"%\"\nwidth: (1 / totalSteps) * 100 + \"%\"\n```\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/progressbar.tsx)\n\n### RunView\n\nContainer component that manages the detail viewer and message display during task execution.\n\n**Layout:**\n- Split view with message list on the left\n- Detail viewer on the right (collapsible/expandable)\n- Manages image gallery, VNC preview, and input responses\n\n资料来源：[frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n\n## Plan Management\n\n### PlanList\n\nDisplays the user's saved plans library with search, create, import, and management capabilities.\n\n**Features:**\n| Feature | Description |\n|---------|-------------|\n| Create | Create a new empty plan |\n| Import | Import plan from JSON file |\n| Search | Filter plans by name |\n| Export | Download plan as JSON |\n| Delete | Remove plan from library |\n\n**Plan Card Actions:**\n- **Run Plan**: Create new session with plan loaded\n- **Edit Plan**: Modify plan title and steps in modal\n\n资料来源：[frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n\n### PlanCard\n\nIndividual plan display card showing plan metadata and quick actions.\n\n**Displayed Information:**\n- Plan title (truncated to 40 characters)\n- Step count summary (showing first 3 steps)\n- Creation timestamp with relative time formatting\n- Hover actions for export and delete\n\n**Modal Editing:**\n- Editable plan title\n- Full plan step editor via `PlanView` component\n- Save and cancel functionality\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### LearnPlanButton\n\nButton component that allows users to extract and save a reusable plan from the current conversation.\n\n**States:**\n| State | Appearance |\n|-------|------------|\n| Disabled | Opacity 50%, cursor not-allowed |\n| Learning | Spinner with \"Learning Plan...\" text |\n| Ready | Normal button with \"Learn Plan\" label |\n\n**Behavior:**\n- Disabled when no `sessionId` or `effectiveUserId`\n- Triggers plan extraction from conversation history\n- Saves extracted plan to user's plan library\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n## File Handling\n\n### RenderFile\n\nComponent for displaying and managing file attachments in messages.\n\n**Features:**\n- Detects file type from metadata\n- Renders appropriate preview based on file type\n- Provides download functionality\n- Supports modal view for detailed file inspection\n\n**File Type Detection:**\n```typescript\nif (message?.metadata?.type === \"file\" && message?.metadata?.files) {\n const parsedFiles = JSON.parse(message.metadata.files);\n // Process files to ensure correct type detection\n}\n```\n\n### FileCard\n\nDisplays individual file with icon, name, and download button.\n\n**Interactions:**\n- Click to open file in modal\n- Hover to show download button\n- Drag-and-drop zone support\n\n资料来源：[frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n## Markdown Rendering\n\n### MarkdownRender\n\nComponent for rendering markdown content with syntax highlighting and GitHub Flavored Markdown support.\n\n**Features:**\n- Syntax highlighting via language detection from file extensions\n- Configurable text truncation\n- Indentation indicator support\n- Dark/light mode compatible styling\n\n**Configuration:**\n| Option | Type | Description |\n|--------|------|-------------|\n| truncate | boolean | Enable content truncation |\n| maxLength | number | Maximum character length |\n| indented | boolean | Show indentation indicator |\n| isFilePreview | boolean | Wrap in code block |\n\n资料来源：[frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n\n## MCP Server Configuration\n\n### McpServerCard\n\nCard component for displaying MCP (Model Context Protocol) server configurations.\n\n**Displayed Information:**\n- Server name\n- Agent description (truncated to 2 lines)\n- Availability status\n\n**Actions:**\n| Action | Description |\n|--------|-------------|\n| Edit | Modify server configuration |\n| Remove | Delete server from configuration |\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpServerCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpServerCard.tsx)\n\n## Relevant Plans\n\n### RelevantPlans\n\nComponent for displaying plans relevant to the current conversation context.\n\n**Features:**\n- Shows top 3 most relevant plans based on current query\n- Plan attachment to query\n- Play action to attach and run plan\n\n资料来源：[frontend/src/components/views/chat/relevant_plans.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/relevant_plans.tsx)\n\n## State Management\n\nThe frontend uses React Context for global state management:\n\n```mermaid\ngraph TD\n A[AppContext] --> B[User State]\n A --> C[Session State]\n A --> D[Theme State]\n A --> E[Provider State]\n \n B --> F[userId]\n B --> G[username]\n \n C --> H[sessions]\n C --> I[currentRun]\n C --> J[plans]\n \n E --> K[mcpServers]\n E --> L[selectedMcpServers]\n```\n\n### Provider Hook\n\nCustom hooks for accessing and manipulating application state:\n\n| Hook | Purpose |\n|------|---------|\n| useAppContext | Access global app context |\n| useSessions | Manage session list and operations |\n| usePlans | Manage saved plans |\n| useMcpServers | Manage MCP server configurations |\n\n资料来源：[frontend/src/hooks/provider.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/provider.tsx)\n\n## API Integration\n\nThe frontend communicates with the backend API at `http://localhost:8081/api`.\n\n**Environment Configuration:**\n```bash\n# In .env.development\nGATSBY_API_URL=http://localhost:8081/api\n```\n\n资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Key API Operations\n\n| Operation | Description |\n|-----------|-------------|\n| runTask | Start a new task execution |\n| handleInputResponse | Respond to input requests |\n| handlePause | Pause current execution |\n| handleCancel | Cancel running task |\n| handleApprove | Approve pending action |\n| handleDeny | Deny pending action |\n| handleAcceptPlan | Accept proposed plan |\n| handleRegeneratePlan | Regenerate plan suggestions |\n\n## Component Hierarchy Summary\n\n```mermaid\ngraph TD\n Root[MagenticUILayout] --> SessionManager\n SessionManager --> Header\n SessionManager --> Sidebar\n SessionManager --> Content\n \n Sidebar --> PlanList\n Sidebar --> SessionList\n \n Content --> ChatView\n Content --> RunView\n \n ChatView --> ChatInput\n ChatView --> MessageList\n ChatView --> RelevantPlans\n \n MessageList --> RenderMessage\n RenderMessage --> PlanView\n RenderMessage --> RenderFile\n RenderMessage --> MarkdownRender\n \n ChatInput --> FileList\n ChatInput --> PlanModal\n \n RunView --> DetailViewer\n RunView --> ProgressBar\n```\n\n## Development Guidelines\n\n### Adding New Routes\n\nTo add a new route (e.g., `/about`):\n1. Create folder `src/pages/about`\n2. Add `index.tsx` file\n3. Follow content style from `src/pages/index.tsx`\n4. Place core logic in `src/components` folder\n\n### Key Dependencies\n\n| Package | Version | Purpose |\n|---------|---------|---------|\n| react | ^18.x | UI framework |\n| antd | ^5.x | Component library |\n| @ant-design/icons | ^5.x | Icon library |\n| react-markdown | ^9.x | Markdown rendering |\n| remark-gfm | ^4.x | GitHub Flavored Markdown |\n\n资料来源：[frontend/package.json](https://github.com/microsoft/magentic-ui/blob/main/frontend/package.json)\n\n---\n\n\n\n## Browser Automation\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [Docker Containers](#docker-containers)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/browser/local_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/local_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/playwright_controller.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_controller.py)\n- [src/magentic_ui/tools/playwright/playwright_state.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_state.py)\n- [src/magentic_ui/tools/playwright/_set_of_mark.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/_set_of_mark.py)\n- [src/magentic_ui/agents/web_surfer/_web_surfer.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/_web_surfer.py)\n \n\n# Browser Automation\n\n## Overview\n\nThe Browser Automation system in Magentic-UI provides a comprehensive framework for controlling web browsers through programmatic interactions. Built on top of Playwright, this system enables AI agents to navigate websites, interact with UI elements, extract content, and perform complex browsing tasks autonomously.\n\nThe system supports multiple browser deployment modes including local execution, headless Docker containers, and VNC-enabled Docker containers for visual debugging. This flexibility allows the system to operate in various environments from development machines to cloud deployments.\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n A[WebSurfer Agent] --> B[PlaywrightController]\n B --> C[Browser Implementations]\n C --> D[LocalPlaywrightBrowser]\n C --> E[HeadlessDockerPlaywrightBrowser]\n C --> F[VncDockerPlaywrightBrowser]\n G[PlaywrightState] --> B\n H[SetOfMark] --> B\n I[Playwright API] --> C\n```\n\n## Browser Implementations\n\nThe system implements a base `PlaywrightBrowser` class with three specialized implementations:\n\n### Base Architecture\n\nAll browser implementations inherit from `PlaywrightBrowser` which provides the core interface for browser operations. This design pattern allows for consistent API usage across different deployment scenarios.\n\n### LocalPlaywrightBrowser\n\nThe `LocalPlaywrightBrowser` provides direct browser control on the local machine. This implementation offers:\n\n- Full browser lifecycle management (launch, close, context management)\n- Synchronous and asynchronous operation support\n- Download folder configuration\n- Viewport customization\n- Screenshot capture capabilities\n\n**Key Features:**\n- Direct Playwright API access without containerization overhead\n- Ideal for development and testing environments\n- Supports all Playwright browser types (Chromium, Firefox, WebKit)\n\n资料来源：[src/magentic_ui/tools/playwright/browser/local_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/local_playwright_browser.py)\n\n### HeadlessDockerPlaywrightBrowser\n\nThe `HeadlessDockerPlaywrightBrowser` runs browsers inside headless Docker containers. This approach provides:\n\n- Isolated browser execution environment\n- Consistent behavior across different host systems\n- No visual rendering overhead\n- Enhanced security through containerization\n\n**Docker Integration:**\n- Automatic container image pulling on first use\n- Graceful container lifecycle management\n- Resource-efficient headless operation\n\n资料来源：[src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py)\n\n### VncDockerPlaywrightBrowser\n\nThe `VncDockerPlaywrightBrowser` extends headless Docker support with VNC connectivity, enabling:\n\n- Real-time visual browser observation\n- Interactive debugging capabilities\n- NoVNC support for browser-based VNC access\n- Remote control handover to human operators\n\n**Port Configuration:**\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `port` | Main VNC port for container communication | 5900 |\n| `novnc_port` | WebSocket port for noVNC browser access | 6080 |\n\n资料来源：[src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py)\n\n## PlaywrightController\n\nThe `PlaywrightController` serves as the central orchestrator for browser interactions. It abstracts the complexities of browser automation into a clean, agent-friendly interface.\n\n### Core Responsibilities\n\n**Page Navigation and Content Extraction:**\n- Visit URLs with configurable timeouts\n- Extract visible text content from pages\n- Capture full-page or viewport screenshots\n- Analyze DOM structure for interactive elements\n\n**User Interaction Simulation:**\n- Mouse movements to specific coordinates\n- Left-click and hover actions\n- Text input via keyboard typing\n- Keyboard shortcuts and key presses\n- Scroll operations with configurable pixels\n\n**Tab Management:**\n- Create new browser tabs\n- Switch between existing tabs\n- Close tabs\n- Refresh page content\n\n### Action Schema\n\nThe controller defines a structured JSON schema for all available actions:\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `visit_url` | `url` | Navigate to specified URL |\n| `web_search` | `query` | Execute web search |\n| `type` | `text`, `coordinate`, `press_enter`, `delete_existing_text` | Type text or interact with elements |\n| `key` | `keys` | Press keyboard keys |\n| `mouse_move` | `coordinate` | Move mouse cursor |\n| `left_click` | `coordinate` | Click at coordinate |\n| `hover` | `coordinate` | Hover over element |\n| `scroll` | `pixels` | Scroll page (positive=up, negative=down) |\n| `select_option` | `element`, `value` | Select dropdown option |\n| `create_tab` | `url` | Open new tab |\n| `switch_tab` | `tab_id` | Switch to specific tab |\n| `refresh_page` | - | Reload current page |\n| `history_back` | - | Navigate browser history back |\n| `sleep` | `time` | Wait specified seconds |\n| `stop_action` | - | Stop current action sequence |\n\n资料来源：[src/magentic_ui/tools/playwright/playwright_controller.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_controller.py)\n\n## State Management\n\n### PlaywrightState\n\nThe `PlaywrightState` module handles serialization and persistence of browser session state.\n\n**State Components:**\n\n| Component | Description |\n|-----------|-------------|\n| `BrowserState` | Complete snapshot of browser context |\n| `save_browser_state()` | Serialize current state to storage |\n| `load_browser_state()` | Restore state from storage |\n\n**State Data Structure:**\n- Current page URL and title\n- Tab information and active tab ID\n- Scroll position\n- Cookie and local storage data\n- Form input values\n- Screenshot history\n\nThis enables:\n- Session recovery after interruptions\n- Parallel agent execution with shared state\n- Checkpoint creation for long-running tasks\n\n资料来源：[src/magentic_ui/tools/playwright/playwright_state.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_state.py)\n\n## Interactive Element Marking\n\n### Set of Mark (_set_of_mark)\n\nThe `_set_of_mark` module enhances web pages with visual markers that identify interactive elements. This is crucial for LLM-based agents to accurately identify and target UI elements.\n\n**Marking Strategy:**\n- Assigns unique numeric identifiers to interactive elements\n- Overlays clickable numbers on buttons, links, inputs\n- Uses sequential numbering for easy reference\n- Provides coordinate mappings for action targeting\n\n**Benefits:**\n- Enables precise element targeting by AI agents\n- Reduces ambiguity in element selection\n- Supports visual debugging and verification\n- Works across different page layouts and frameworks\n\n资料来源：[src/magentic_ui/tools/playwright/_set_of_mark.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/_set_of_mark.py)\n\n## WebSurfer Agent\n\nThe `WebSurfer` agent is the primary consumer of the browser automation system. It combines the browser implementations with an LLM to make intelligent browsing decisions.\n\n### Agent Capabilities\n\n**Autonomous Navigation:**\n- Follow links and navigate between pages\n- Complete multi-step web forms\n- Search the web and process results\n- Extract structured information from pages\n\n**Content Processing:**\n- Optical Character Recognition (OCR) for image content\n- Visual question answering on screenshots\n- Markdown rendering of page content\n- File download handling\n\n**Interaction Modes:**\n- Automatic execution with configurable action limits\n- Step-by-step mode with human approval\n- Control handover for human intervention\n- Pause and resume capabilities\n\n### Configuration Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `start_page` | str | Google | Initial page on browser launch |\n| `animate_actions` | bool | False | Enable action animation |\n| `save_screenshots` | bool | False | Persist screenshots to disk |\n| `max_actions_per_step` | int | 5 | Maximum actions per reasoning step |\n| `resize_viewport` | bool | True | Auto-resize viewport |\n| `url_statuses` | dict | None | URL allow/reject rules |\n| `single_tab_mode` | bool | False | Restrict to single tab |\n\n资料来源：[src/magentic_ui/agents/web_surfer/_web_surfer.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/_web_surfer.py)\n\n## Usage Patterns\n\n### Local Browser Usage\n\n```python\nfrom magentic_ui.tools.playwright.browser import LocalPlaywrightBrowser\n\nbrowser = LocalPlaywrightBrowser(\n headless=False,\n downloads_folder=\"./downloads\"\n)\nawait browser.start()\n```\n\n### Docker-based Browser with VNC\n\n```python\nfrom magentic_ui.tools.playwright.browser import VncDockerPlaywrightBrowser\n\nbrowser = VncDockerPlaywrightBrowser(\n port=5900,\n novnc_port=8080\n)\nawait browser.start()\n# Access via browser at http://localhost:8080/vnc.html\n```\n\n### Controller Integration\n\n```python\nfrom magentic_ui.tools.playwright.playwright_controller import PlaywrightController\n\ncontroller = PlaywrightController(browser)\nawait controller.async_setup()\n\n# Execute actions\nresult = await controller(\n {\"action\": \"visit_url\", \"url\": \"https://example.com\"}\n)\n```\n\n## Workflow Diagram\n\n```mermaid\nsequenceDiagram\n participant Agent\n participant Controller\n participant Browser\n participant Page\n \n Agent->>Controller: Execute action\n Controller->>Controller: Validate parameters\n Controller->>Browser: Get page instance\n Browser->>Page: Perform action\n Page-->>Browser: Action result\n Browser-->>Controller: Browser response\n Controller->>Controller: Process result\n Controller->>Agent: Action result\n \n Note over Agent,Page: Repeat until task complete\n```\n\n## Frontend Integration\n\nThe browser automation system integrates with the Magentic-UI frontend through:\n\n**DetailViewer Component:**\n- Real-time browser view in the UI\n- Screenshot gallery display\n- Live action feed\n- Control mode overlay for human takeover\n\n**Modal Components:**\n- BrowserModal for full-screen viewing\n- VNC connection handling\n- Pause and resume controls\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/detail_viewer.tsx)\n\n## Security Considerations\n\n**URL Filtering:**\n- `UrlStatusManager` validates navigation targets\n- Configurable allow/reject lists\n- Prevents navigation to untrusted domains\n\n**Sandbox Isolation:**\n- Docker containers provide process isolation\n- Restricted network access when needed\n- Resource limits prevent runaway processes\n\n**Approval Workflows:**\n- Human approval for sensitive actions\n- Configurable approval thresholds\n- Audit logging of all actions\n\n## Conclusion\n\nThe Browser Automation system provides a robust, flexible foundation for web interaction in Magentic-UI. By combining Playwright's powerful browser control with thoughtful abstractions and multiple deployment options, it enables AI agents to perform complex web-based tasks reliably and safely.\n\n---\n\n\n\n## Docker Containers\n\n### 相关页面\n\n相关主题：[Getting Started with Magentic-UI](#getting-started), [Browser Automation](#browser-automation)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [docker/magentic-ui-browser-docker/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/Dockerfile)\n- [docker/magentic-ui-browser-docker/supervisord.conf](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/supervisord.conf)\n- [docker/magentic-ui-browser-docker/playwright-server.js](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/playwright-server.js)\n- [docker/magentic-ui-python-env/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-python-env/Dockerfile)\n- [docker/build-all.sh](https://github.com/microsoft/magentic-ui/blob/main/docker/build-all.sh)\n- [src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n \n\n# Docker Containers\n\nMagentic-UI leverages Docker containers to provide isolated, reproducible environments for running browser automation and code execution tasks. This architecture enables the application to execute complex multi-agent workflows while maintaining system-level isolation and consistent runtime dependencies.\n\n## Architecture Overview\n\nMagentic-UI uses two primary Docker images working in tandem to deliver its functionality:\n\n```mermaid\ngraph TB\n subgraph \"Magentic-UI Architecture\"\n A[\"Frontend UI
(localhost:8081)\"] --> B[\"Backend API
(Python/FastAPI)\"]\n B --> C[\"Browser Container
(VNC + Playwright)\"]\n B --> D[\"Python Environment Container
(Code Execution)\"]\n end\n \n subgraph \"Container Communication\"\n C <-->|\"WebSocket/REST\"| B\n D <-->|\"STDIO/REST\"| B\n end\n```\n\n## Docker Image Types\n\n| Image Type | Purpose | Key Components |\n|------------|---------|----------------|\n| `magentic-ui-browser` | Browser automation and web interaction | VNC Server, noVNC, Playwright, Chromium |\n| `magentic-ui-python-env` | Safe Python code execution | Python runtime, isolated environment |\n\n资料来源：[docker/build-all.sh:1-20]()\n\n### Browser Docker Container\n\nThe browser container provides a full graphical environment for web surfing agents. It includes:\n\n- **VNC Server**: Provides virtual display access\n- **noVNC**: Web-based VNC client for browser access\n- **Playwright**: Browser automation framework for programmatic control\n- **Chromium**: Headless-capable web browser\n\n资料来源：[docker/magentic-ui-browser-docker/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/Dockerfile)\n\n### Python Environment Docker Container\n\nThe Python environment container provides a sandboxed environment for executing user-generated Python code safely:\n\n- Isolated Python runtime\n- Restricted file system access\n- Controlled network access\n- Independent package management\n\n资料来源：[docker/magentic-ui-python-env/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-python-env/Dockerfile)\n\n## Docker Initialization Workflow\n\n```mermaid\nsequenceDiagram\n participant User\n participant CLI\n participant Docker Daemon\n participant Registry\n \n User->>CLI: magentic-ui --port 8081\n CLI->>Docker Daemon: Check if Docker is running\n Docker Daemon-->>CLI: Docker Status\n \n alt Docker not running\n CLI->>User: Error: Please start Docker\n else Docker running\n CLI->>Docker Daemon: Check browser image exists\n Docker Daemon-->>CLI: Image Status\n \n alt Image missing\n CLI->>Registry: Pull browser image\n Registry-->>Docker Daemon: Image layers\n Docker Daemon->>Docker Daemon: Build image\n end\n \n CLI->>Docker Daemon: Check Python image exists\n Docker Daemon-->>CLI: Image Status\n \n alt Image missing\n CLI->>Registry: Pull Python image\n Registry-->>Docker Daemon: Image layers\n Docker Daemon->>Docker Daemon: Build image\n end\n \n CLI->>User: Magentic-UI ready\n end\n```\n\n资料来源：[src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n\n## Container Management Functions\n\nThe `src/magentic_ui/_docker.py` module provides core Docker management functionality:\n\n| Function | Purpose |\n|----------|---------|\n| `check_docker_running()` | Verifies Docker daemon is accessible |\n| `check_browser_image()` | Checks if browser Docker image exists locally |\n| `check_python_image()` | Checks if Python environment Docker image exists locally |\n| `pull_browser_image()` | Pulls/updates the browser Docker image |\n| `pull_python_image()` | Pulls/updates the Python environment Docker image |\n\n资料来源：[src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n\n## Build Process\n\nThe build script `docker/build-all.sh` constructs both Docker images:\n\n```bash\n# Build browser Docker image\ndocker build -t magentic-ui-browser ./magentic-ui-browser-docker\n\n# Build Python environment Docker image\ndocker build -t magentic-ui-python-env ./magentic-ui-python-env\n```\n\n资料来源：[docker/build-all.sh](https://github.com/microsoft/magentic-ui/blob/main/docker/build-all.sh)\n\n## Browser Container Services\n\nThe browser container runs multiple services managed by supervisord:\n\n```mermaid\ngraph LR\n subgraph \"Browser Container Services\"\n A[\"supervisord
(Process Manager)\"]\n A --> B[\"Xvfb
(Virtual Display)\"]\n A --> C[\"x11vnc
(VNC Server)\"]\n A --> D[\"noVNC
(Web VNC)\"]\n A --> E[\"Playwright
(Browser Control)\"]\n end\n```\n\n### Service Configuration\n\nThe browser container uses `supervisord.conf` for service orchestration:\n\n- **Process Management**: Supervisord manages all background services\n- **Auto-restart**: Services automatically restart on failure\n- **Log Management**: Centralized logging configuration\n\n资料来源：[docker/magentic-ui-browser-docker/supervisord.conf](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/supervisord.conf)\n\n### Playwright Server\n\nThe Playwright server (`playwright-server.js`) provides HTTP API access to browser automation:\n\n```javascript\n// Server initialization with browser configuration\n// Handles browser launching, page creation, and element interaction\n```\n\n资料来源：[docker/magentic-ui-browser-docker/playwright-server.js](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/playwright-server.js)\n\n## Running Without Docker\n\nFor environments where Docker is unavailable, Magentic-UI supports a limited mode:\n\n```bash\nmagentic-ui --run-without-docker --port 8081\n```\n\n**Limitations in No-Docker Mode**:\n\n| Feature | With Docker | Without Docker |\n|---------|-------------|----------------|\n| Web Surfing | Full browser automation | Not available |\n| Code Execution | Isolated sandbox | Not available |\n| File Handling | Enhanced isolation | Basic support |\n| Agent Capabilities | Complete | Reduced |\n\n资料来源：[README.md](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Prerequisites\n\n### System Requirements\n\n| Requirement | Minimum | Recommended |\n|-------------|---------|-------------|\n| Docker Version | Latest stable | Latest stable |\n| Python | 3.10+ | 3.11+ |\n| RAM | 4GB | 8GB+ |\n| Disk Space | 2GB | 5GB+ |\n\n### Platform Support\n\n- **Linux**: Full support with native Docker\n- **macOS**: Full support with Docker Desktop\n- **Windows**: WSL2 required for Docker support\n\n资料来源：[TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Troubleshooting\n\n### Common Docker Issues\n\n| Issue | Symptom | Solution |\n|-------|---------|----------|\n| Docker not running | \"Docker is not running\" error | Start Docker Desktop/daemon |\n| Image pull failure | Timeout during first run | Run `docker/build-all.sh` manually |\n| Port conflict | Container fails to start | Change port with `--port` flag |\n\n### Verification Commands\n\n```bash\n# Check Docker is running\ndocker info\n\n# Verify images exist\ndocker images | grep magentic-ui\n\n# Manually build images\ncd docker && sh build-all.sh\n```\n\n资料来源：[TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Configuration\n\n### Environment Variables\n\n| Variable | Purpose | Default |\n|----------|---------|---------|\n| `NOVNC_PORT` | noVNC web interface port | 6080 |\n| `PLAYWRIGHT_PORT` | Playwright API port | 8080 |\n| `PYTHON_ENV_PORT` | Python execution port | 8082 |\n\n### Workspace Configuration\n\nThe CLI manages workspace paths passed to containers:\n\n```python\nworkspace_config = {\n \"internal_workspace_root\": \"/path/to/internal\",\n \"external_workspace_root\": \"/path/to/external\",\n \"inside_docker\": True,\n \"config\": {...},\n \"run_without_docker\": False\n}\n```\n\n资料来源：[src/magentic_ui/backend/cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/cli.py)\n\n## Security Considerations\n\n### Container Isolation\n\n- **Network Isolation**: Containers communicate via internal bridge network\n- **File System Isolation**: Read-only base images with volume mounts for data\n- **Process Isolation**: Separate PID namespaces\n\n### Best Practices\n\n1. Always run Docker with non-root user when possible\n2. Keep Docker images updated with latest security patches\n3. Use the provided workspace paths for file operations\n4. Monitor container resource usage\n\n---\n\n\n\n## Configuration\n\n### 相关页面\n\n相关主题：[Getting Started with Magentic-UI](#getting-started)\n\n\nRelated Source Files
\n\nThe following source files were used to generate this documentation:\n\n- [src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n- [src/magentic_ui/backend/cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/cli.py)\n- [src/magentic_ui/backend/web/config.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/config.py)\n- [src/magentic_ui/backend/teammanager/teammanager.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/teammanager/teammanager.py)\n- [fara_config.yaml](https://github.com/microsoft/magentic-ui/blob/main/fara_config.yaml)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n \n\n# Configuration\n\nMagentic-UI provides a multi-layered configuration system that spans both the backend (Python) and frontend (React/TypeScript) layers. The system handles environment-based settings, agent parameters, UI theming, and server configurations through YAML files, environment variables, and component-level props.\n\n## Overview\n\nThe configuration architecture in Magentic-UI can be visualized as follows:\n\n```mermaid\ngraph TD\n A[Configuration Sources] --> B[Backend CLI]\n A --> C[Environment Variables]\n A --> D[YAML Config Files]\n A --> E[Frontend React Components]\n \n B --> F[Server Initialization]\n C --> G[API URL Configuration]\n D --> H[Agent Parameters]\n E --> I[UI Theme & Modal Settings]\n \n F --> J[Backend Server Running on Port 8081]\n G --> K[Frontend Dev Server]\n H --> L[Web Surfer Agent]\n I --> M[User Interface]\n```\n\n## Backend Configuration\n\n### CLI Entry Point\n\nThe main CLI entry point in `src/magentic_ui/_cli.py` serves as the primary configuration bootstrap for the backend server. It handles argument parsing and delegates to the backend CLI module.\n\n**Key configuration parameters supported:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `--host` | string | Server host address |\n| `--port` | integer | Server port number (default: 8081) |\n| `--config` | string | Path to YAML configuration file |\n| `--debug` | boolean | Enable debug mode |\n\n资料来源：[src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n\n### Web Server Configuration\n\nThe web server configuration module (`src/magentic_ui/backend/web/config.py`) defines the core server settings used by the FastAPI-based backend.\n\n```python\nclass ServerConfig:\n host: str = \"0.0.0.0\"\n port: int = 8081\n cors_origins: list[str] = [\"http://localhost:8000\"]\n debug: bool = False\n```\n\n**Configuration Options:**\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `host` | `\"0.0.0.0\"` | Bind address for the server |\n| `port` | `8081` | HTTP port for the backend API |\n| `cors_origins` | `[\"http://localhost:8000\"]` | Allowed CORS origins |\n| `debug` | `false` | Enable verbose logging and hot reload |\n\n资料来源：[src/magentic_ui/backend/web/config.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/config.py)\n\n### Team Manager Configuration\n\nThe `teammanager.py` module handles multi-agent orchestration configuration. It manages agent teams and their coordination settings.\n\n**Key configuration aspects:**\n\n- Agent pool sizing\n- Maximum concurrent agents\n- Communication protocols between agents\n- Timeout settings for agent operations\n\n资料来源：[src/magentic_ui/backend/teammanager/teammanager.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/teammanager/teammanager.py)\n\n## YAML Configuration Files\n\n### fara_config.yaml\n\nThe `fara_config.yaml` file contains configuration for the web surfer agent, including display settings and browser automation parameters.\n\n```yaml\ndisplay_width_px: 1280\ndisplay_height_px: 720\ninclude_input_text_key_args: false\n```\n\n**Web Surfer Agent Parameters:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `display_width_px` | integer | Browser viewport width in pixels |\n| `display_height_px` | integer | Browser viewport height in pixels |\n| `include_input_text_key_args` | boolean | Include text input keyboard shortcuts |\n\n资料来源：[fara_config.yaml](https://github.com/microsoft/magentic-ui/blob/main/fara_config.yaml)\n\n### Agent Parameter Handling\n\nThe `_prompts.py` module in the web surfer agent demonstrates how configuration is consumed:\n\n```python\ndef __init__(self, cfg=None):\n self.display_width_px = cfg[\"display_width_px\"]\n self.display_height_px = cfg[\"display_height_px\"]\n include_input_text_key_args = cfg.pop(\"include_input_text_key_args\", False)\n if not include_input_text_key_args:\n self.parameters[\"properties\"].pop(\"press_enter\", None)\n self.parameters[\"properties\"].pop(\"delete_existing_text\", None)\n super().__init__(cfg)\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n## Frontend Configuration\n\n### Environment Variables\n\nThe frontend uses environment variables configured through a `.env` file structure. The development environment requires specific settings to connect to the backend API.\n\n**Setup Instructions:**\n\n1. Copy `.env.default` to `.env.development`\n2. Set the required variables in the new file\n\n| Variable | Required Value | Description |\n|----------|---------------|-------------|\n| `GATSBY_API_URL` | `http://localhost:8081/api` | Backend API endpoint |\n| `GATSBY_WS_URL` | `ws://localhost:8081/ws` | WebSocket endpoint (if applicable) |\n\n资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### API Configuration Flow\n\n```mermaid\nsequenceDiagram\n participant FE as Frontend (React)\n participant API as Backend API\n participant WS as WebSocket\n \n FE->>FE: Load .env.development\n FE->>API: HTTP requests to GATSBY_API_URL\n FE->>WS: WebSocket connections\n API-->>FE: JSON responses\n WS-->>FE: Real-time updates\n```\n\n### MCP Server Configuration\n\nThe MCP (Model Context Protocol) server configuration modal provides a UI for managing external server integrations.\n\n**Supported Connection Types:**\n\n| Type | Description | Configuration Method |\n|------|-------------|----------------------|\n| `SSE` | Server-Sent Events | Form-based input |\n| `Stdio` | Standard I/O | Form-based input |\n| `JSON` | Raw JSON Config | Direct JSON editing |\n\n**Server Configuration Validation:**\n\n| Field | Validation Rule |\n|-------|----------------|\n| `serverName` | Required, alphanumeric characters only, max 50 characters |\n| `serverName` | Must be unique across all servers |\n\n```typescript\n// Example validation logic from McpConfigModal.tsx\nconst serverNameError = !serverName || !/^[a-zA-Z0-9]+$/.test(serverName);\nconst serverNameDuplicateError = existingServers.some(\n (s) => s.name === serverName && s.id !== server?.id\n);\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## UI Theme Configuration\n\n### Theme Application\n\nThe main layout component applies theme settings based on user preferences and system defaults:\n\n```typescript\n\n```\n\n**Theme Options:**\n\n| Mode | Algorithm | CSS Classes |\n|------|-----------|-------------|\n| Light | `defaultAlgorithm` | `bg-white`, `text-gray-900` |\n| Dark | `darkAlgorithm` | `bg-gray-900`, `text-gray-100` |\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n### Component-Level Styling\n\nComponents use Tailwind CSS utility classes for configuration of:\n\n- Color schemes (`bg-magenta-800`, `text-blue-400`)\n- Spacing (`p-3`, `mt-4`, `mb-2`)\n- Typography (`text-sm`, `font-medium`)\n- Transitions (`transition-colors`, `transition-all duration-300`)\n\n## Configuration Workflow\n\n```mermaid\ngraph LR\n A[Start Application] --> B{Backend or Frontend?}\n \n B -->|Backend| C[Load CLI Args]\n B -->|Backend| D[Parse YAML Config]\n B -->|Backend| E[Initialize Server]\n \n B -->|Frontend| F[Load .env.development]\n B -->|Frontend| G[Build API URL]\n B -->|Frontend| H[Render UI Components]\n \n C --> E\n D --> E\n F --> G\n G --> H\n E --> I[Server Ready]\n H --> J[User Interface Ready]\n```\n\n## Configuration Files Summary\n\n| File Path | Purpose | Format |\n|-----------|---------|--------|\n| `src/magentic_ui/_cli.py` | Main CLI entry point | Python |\n| `src/magentic_ui/backend/cli.py` | Backend CLI logic | Python |\n| `src/magentic_ui/backend/web/config.py` | Web server settings | Python (dataclass) |\n| `src/magentic_ui/backend/teammanager/teammanager.py` | Agent orchestration | Python |\n| `fara_config.yaml` | Web surfer agent settings | YAML |\n| `.env.development` | Frontend environment | Environment Variables |\n\n## Best Practices\n\n1. **Environment Isolation**: Keep development and production environment files separate\n2. **Validation**: Always validate MCP server names against alphanumeric patterns\n3. **CORS Settings**: Ensure backend CORS configuration matches frontend origin\n4. **Port Consistency**: The frontend expects the backend at `http://localhost:8081/api`\n5. **Theme Persistence**: User theme preferences should be stored in local storage or user profile\n\n---\n\n---\n\n## Doramagic 踩坑日志\n\n项目：microsoft/magentic-ui\n\n摘要：发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：安装坑 - 来源证据：Create tutorials and documentation for the codebase。\n\n## 1. 安装坑 · 来源证据：Create tutorials and documentation for the codebase\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Create tutorials and documentation for the codebase\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_c0979f7ebb064422a6a8095561f6a9bd | https://github.com/microsoft/magentic-ui/issues/154 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 2. 安装坑 · 来源证据：Support Podman in place of Docker\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Support Podman in place of Docker\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_f88231cc4cad442ca53d92ea3a40a655 | https://github.com/microsoft/magentic-ui/issues/312 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 3. 安装坑 · 来源证据：magentic-ui can't display all the html element\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：magentic-ui can't display all the html element\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_21f19953edd74379ab2d25cedc37ca1b | https://github.com/microsoft/magentic-ui/issues/362 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 4. 配置坑 · 来源证据：Refreshing or restart the web app will make the current Session unavailable\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个配置相关的待验证问题：Refreshing or restart the web app will make the current Session unavailable\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_83a9cafd59254028853ef84cd1ccc756 | https://github.com/microsoft/magentic-ui/issues/336 | 来源讨论提到 node 相关条件，需在安装/试用前复核。\n\n## 5. 能力坑 · 能力判断依赖假设\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：README/documentation is current enough for a first validation pass.\n- 对用户的影响：假设不成立时，用户拿不到承诺的能力。\n- 建议检查：将假设转成下游验证清单。\n- 防护动作：假设必须转成验证项；没有验证结果前不能写成事实。\n- 证据：capability.assumptions | github_repo:978331188 | https://github.com/microsoft/magentic-ui | README/documentation is current enough for a first validation pass.\n\n## 6. 运行坑 · 来源证据：Why not conduct a requirement analysis before the plan?\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个运行相关的待验证问题：Why not conduct a requirement analysis before the plan?\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6003a9c2194f40c0865145385cf98c32 | https://github.com/microsoft/magentic-ui/issues/321 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 7. 维护坑 · 来源证据：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_7e754869326e42e1a7c57f3a1962ef9e | https://github.com/microsoft/magentic-ui/issues/360 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 8. 维护坑 · 维护活跃度未知\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：未记录 last_activity_observed。\n- 对用户的影响：新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。\n- 建议检查：补 GitHub 最近 commit、release、issue/PR 响应信号。\n- 防护动作：维护活跃度未知时，推荐强度不能标为高信任。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | last_activity_observed missing\n\n## 9. 安全/权限坑 · 下游验证发现风险项\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：下游已经要求复核，不能在页面中弱化。\n- 建议检查：进入安全/权限治理复核队列。\n- 防护动作：下游风险存在时必须保持 review/recommendation 降级。\n- 证据：downstream_validation.risk_items | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 10. 安全/权限坑 · 存在评分风险\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：风险会影响是否适合普通用户安装。\n- 建议检查：把风险写入边界卡，并确认是否需要人工复核。\n- 防护动作：评分风险必须进入边界卡，不能只作为内部分数。\n- 证据：risks.scoring_risks | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 11. 安全/权限坑 · 来源证据：Settings redesign\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：Settings redesign\n- 对用户的影响：可能影响授权、密钥配置或安全边界。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6a2eeae98b6d4fdab476464d57e64e1d | https://github.com/microsoft/magentic-ui/issues/227 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 12. 维护坑 · issue/PR 响应质量未知\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：issue_or_pr_quality=unknown。\n- 对用户的影响：用户无法判断遇到问题后是否有人维护。\n- 建议检查：抽样最近 issue/PR，判断是否长期无人处理。\n- 防护动作：issue/PR 响应未知时，必须提示维护风险。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | issue_or_pr_quality=unknown\n\n## 13. 维护坑 · 发布节奏不明确\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：release_recency=unknown。\n- 对用户的影响：安装命令和文档可能落后于代码，用户踩坑概率升高。\n- 建议检查：确认最近 release/tag 和 README 安装命令是否一致。\n- 防护动作：发布节奏未知或过期时，安装说明必须标注可能漂移。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | release_recency=unknown\n\n\n",
"markdown_key": "magentic-ui",
"pages": "draft",
"source_refs": [
{
"evidence_id": "github_repo:978331188",
"kind": "repo",
"supports_claim_ids": [
"claim_identity",
"claim_distribution",
"claim_capability"
],
"url": "https://github.com/microsoft/magentic-ui"
},
{
"evidence_id": "art_79bc1e1e45bc472d8be53924dd351d48",
"kind": "docs",
"supports_claim_ids": [
"claim_identity",
"claim_distribution",
"claim_capability"
],
"url": "https://github.com/microsoft/magentic-ui#readme"
}
],
"summary": "DeepWiki/Human Wiki 完整输出，末尾追加 Discovery Agent 踩坑日志。",
"title": "magentic-ui 说明书",
"toc": [
"https://github.com/microsoft/magentic-ui 项目说明书",
"目录",
"Getting Started with Magentic-UI",
"Prerequisites",
"Installation",
"Quick Start with Docker",
"Local Development Setup",
"Using Fara-7B Locally",
"Doramagic 踩坑日志"
]
}
},
"quality_gate": {
"blocking_gaps": [],
"category_confidence": "medium",
"compile_status": "ready_for_review",
"five_assets_present": true,
"install_sandbox_verified": true,
"missing_evidence": [],
"next_action": "publish to Doramagic.ai project surfaces",
"prompt_preview_boundary_ok": true,
"publish_status": "publishable",
"quick_start_verified": true,
"repo_clone_verified": true,
"repo_commit": "f705b0ee29af5ddc03953e6d3be4c0a880a78b09",
"repo_inspection_error": null,
"repo_inspection_files": [
"pyproject.toml",
"README.md",
"uv.lock",
"src/magentic_ui/version.py",
"src/magentic_ui/_docker.py",
"src/magentic_ui/guarded_action.py",
"src/magentic_ui/_cli.py",
"src/magentic_ui/magentic_ui_config.py",
"src/magentic_ui/input_func.py",
"src/magentic_ui/approval_guard.py",
"src/magentic_ui/task_team.py",
"src/magentic_ui/types.py",
"src/magentic_ui/utils.py",
"src/magentic_ui/__init__.py",
"src/magentic_ui/backend/cli.py",
"src/magentic_ui/backend/README.md",
"src/magentic_ui/backend/__init__.py",
"src/magentic_ui/cli/pretty_console.py",
"src/magentic_ui/cli/__init__.py",
"src/magentic_ui/teams/roundrobin_orchestrator.py",
"src/magentic_ui/teams/__init__.py",
"src/magentic_ui/learning/memory_provider.py",
"src/magentic_ui/learning/learner.py",
"src/magentic_ui/learning/__init__.py",
"src/magentic_ui/tools/bing_search.py",
"src/magentic_ui/tools/tool_metadata.py",
"src/magentic_ui/tools/url_status_manager.py",
"src/magentic_ui/tools/__init__.py",
"src/magentic_ui/eval/evaluators.py",
"src/magentic_ui/eval/core.py",
"src/magentic_ui/eval/benchmark.py",
"src/magentic_ui/eval/basesystem.py",
"src/magentic_ui/eval/models.py",
"src/magentic_ui/eval/README.md",
"src/magentic_ui/eval/utils.py",
"src/magentic_ui/eval/__init__.py",
"src/magentic_ui/agents/_coder.py",
"src/magentic_ui/agents/_utils.py",
"src/magentic_ui/agents/_user_proxy.py",
"src/magentic_ui/agents/__init__.py"
],
"repo_inspection_verified": true,
"review_reasons": [],
"tag_count_ok": true,
"unsupported_claims": []
},
"schema_version": "0.1",
"user_assets": {
"ai_context_pack": {
"asset_id": "ai_context_pack",
"filename": "AI_CONTEXT_PACK.md",
"markdown": "# playwright-remote - Doramagic AI Context Pack\n\n> 定位：安装前体验与判断资产。它帮助宿主 AI 有一个好的开始，但不代表已经安装、执行或验证目标项目。\n\n## 充分原则\n\n- **充分原则，不是压缩原则**：AI Context Pack 应该充分到让宿主 AI 在开工前理解项目价值、能力边界、使用入口、风险和证据来源；它可以分层组织，但不以最短摘要为目标。\n- **压缩策略**：只压缩噪声和重复内容，不压缩会影响判断和开工质量的上下文。\n\n## 给宿主 AI 的使用方式\n\n你正在读取 Doramagic 为 playwright-remote 编译的 AI Context Pack。请把它当作开工前上下文：帮助用户理解适合谁、能做什么、如何开始、哪些必须安装后验证、风险在哪里。不要声称你已经安装、运行或执行了目标项目。\n\n## Claim 消费规则\n\n- **事实来源**：Repo Evidence + Claim/Evidence Graph；Human Wiki 只提供显著性、术语和叙事结构。\n- **事实最低状态**：`supported`\n- `supported`：可以作为项目事实使用，但回答中必须引用 claim_id 和证据路径。\n- `weak`：只能作为低置信度线索，必须要求用户继续核实。\n- `inferred`：只能用于风险提示或待确认问题，不能包装成项目事实。\n- `unverified`：不得作为事实使用，应明确说证据不足。\n- `contradicted`：必须展示冲突来源，不得替用户强行选择一个版本。\n\n## 它最适合谁\n\n- **AI 研究者或研究型 Agent 构建者**：README 明确围绕研究、实验或论文工作流展开。 证据：`README.md` Claim：`clm_0002` supported 0.86\n- **正在使用 Claude/Codex/Cursor/Gemini 等宿主 AI 的开发者**：README 或插件配置提到多个宿主 AI。 证据：`README.md` Claim：`clm_0003` supported 0.86\n\n## 它能做什么\n\n- **命令行启动或安装流程**（需要安装后验证）：项目文档中存在可执行命令，真实使用需要在本地或宿主环境中运行这些命令。 证据：`README.md` Claim：`clm_0001` supported 0.86\n\n## 怎么开始\n\n- `pip install magentic-ui --upgrade` 证据：`README.md` Claim：`clm_0004` supported 0.86\n- `pip install magentic-ui[azure]` 证据：`README.md` Claim：`clm_0005` supported 0.86\n- `pip install magentic-ui[ollama]` 证据：`README.md` Claim：`clm_0006` supported 0.86\n- `pip install magentic-ui` 证据：`README.md` Claim：`clm_0004` supported 0.86, `clm_0005` supported 0.86, `clm_0006` supported 0.86, `clm_0007` supported 0.86 等\n- `pip install magentic-ui[fara]` 证据：`README.md` Claim：`clm_0008` supported 0.86\n- `git clone https://github.com/microsoft/magentic-ui.git` 证据：`README.md` Claim：`clm_0009` supported 0.86\n- `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash` 证据：`README.md` Claim：`clm_0010` supported 0.86\n- `npm install -g gatsby-cli` 证据：`README.md` Claim：`clm_0011` supported 0.86\n- `npm install --global yarn` 证据：`README.md` Claim：`clm_0012` supported 0.86\n\n## 继续前判断卡\n\n- **当前建议**：需要管理员/安全审批\n- **为什么**：继续前可能涉及密钥、账号、外部服务或敏感上下文，建议先经过管理员或安全审批。\n\n### 30 秒判断\n\n- **现在怎么做**：需要管理员/安全审批\n- **最小安全下一步**：先跑 Prompt Preview；若涉及凭证或企业环境，先审批再试装\n- **先别相信**：角色质量和任务匹配不能直接相信。\n- **继续会触碰**：角色选择偏差、命令执行、本地环境或项目文件\n\n### 现在可以相信\n\n- **适合人群线索：AI 研究者或研究型 Agent 构建者**（supported）：有 supported claim 或项目证据支撑，但仍不等于真实安装效果。 证据：`README.md` Claim：`clm_0002` supported 0.86\n- **适合人群线索：正在使用 Claude/Codex/Cursor/Gemini 等宿主 AI 的开发者**（supported）：有 supported claim 或项目证据支撑，但仍不等于真实安装效果。 证据：`README.md` Claim：`clm_0003` supported 0.86\n- **能力存在：命令行启动或安装流程**（supported）：可以相信项目包含这类能力线索；是否适合你的具体任务仍要试用或安装后验证。 证据：`README.md` Claim：`clm_0001` supported 0.86\n- **存在 Quick Start / 安装命令线索**（supported）：可以相信项目文档出现过启动或安装入口；不要因此直接在主力环境运行。 证据：`README.md` Claim：`clm_0004` supported 0.86\n\n### 现在还不能相信\n\n- **角色质量和任务匹配不能直接相信。**（unverified）：角色库证明有很多角色，不证明每个角色都适合你的具体任务，也不证明角色能产生高质量结果。\n- **不能把角色文案当成真实执行能力。**（unverified）：安装前只能判断角色描述和任务画像是否匹配，不能证明它能在宿主 AI 里完成任务。\n- **真实输出质量不能在安装前相信。**（unverified）：Prompt Preview 只能展示引导方式，不能证明真实项目中的结果质量。\n- **宿主 AI 版本兼容性不能在安装前相信。**（unverified）：Claude、Cursor、Codex、Gemini 等宿主加载规则和版本差异必须在真实环境验证。\n- **不会污染现有宿主 AI 行为，不能直接相信。**（inferred）：Skill、plugin、AGENTS/CLAUDE/GEMINI 指令可能改变宿主 AI 的默认行为。\n- **可安全回滚不能默认相信。**（unverified）：除非项目明确提供卸载和恢复说明，否则必须先在隔离环境验证。\n- **真实安装后是否与用户当前宿主 AI 版本兼容？**（unverified）：兼容性只能通过实际宿主环境验证。\n- **项目输出质量是否满足用户具体任务？**（unverified）：安装前预览只能展示流程和边界，不能替代真实评测。\n\n### 继续会触碰什么\n\n- **角色选择偏差**：用户对任务应该由哪个专家角色处理的判断。 原因：选错角色会让 AI 从错误专业视角回答，浪费时间或误导决策。\n- **命令执行**：包管理器、网络下载、本地插件目录、项目配置或用户主目录。 原因：运行第一条命令就可能产生环境改动；必须先判断是否值得跑。 证据：`README.md`\n- **本地环境或项目文件**：安装结果、插件缓存、项目配置或本地依赖目录。 原因：安装前无法证明写入范围和回滚方式，需要隔离验证。 证据：`README.md`\n- **环境变量 / API Key**：项目入口文档明确出现 API key、token、secret 或账号凭证配置。 原因：如果真实安装需要凭证，应先使用测试凭证并经过权限/合规判断。 证据：`README.md`, `src/magentic_ui/backend/web/routes/plans.py`\n- **宿主 AI 上下文**：AI Context Pack、Prompt Preview、Skill 路由、风险规则和项目事实。 原因：导入上下文会影响宿主 AI 后续判断，必须避免把未验证项包装成事实。\n\n### 最小安全下一步\n\n- **先跑 Prompt Preview**：先用交互式试用验证任务画像和角色匹配，不要先导入整套角色库。（适用：任何项目都适用，尤其是输出质量未知时。）\n- **只在隔离目录或测试账号试装**：避免安装命令污染主力宿主 AI、真实项目或用户主目录。（适用：存在命令执行、插件配置或本地写入线索时。）\n- **不要使用真实生产凭证**：环境变量/API key 一旦进入宿主或工具链，可能产生账号和合规风险。（适用：出现 API、TOKEN、KEY、SECRET 等环境线索时。）\n- **安装后只验证一个最小任务**：先验证加载、兼容、输出质量和回滚，再决定是否深用。（适用：准备从试用进入真实工作流时。）\n\n### 退出方式\n\n- **保留安装前状态**：记录原始宿主配置和项目状态，后续才能判断是否可恢复。\n- **保留原始角色选择记录**：如果输出偏题，可以回到任务画像阶段重新选择角色，而不是继续沿着错误角色推进。\n- **记录安装命令和写入路径**：没有明确卸载说明时，至少要知道哪些目录或配置需要手动清理。\n- **准备撤销测试 API key 或 token**：测试凭证泄露或误用时，可以快速止损。\n- **如果没有回滚路径，不进入主力环境**：不可回滚是继续前阻断项，不应靠信任或运气继续。\n\n## 哪些只能预览\n\n- 解释项目适合谁和能做什么\n- 基于项目文档演示典型对话流程\n- 帮助用户判断是否值得安装或继续研究\n\n## 哪些必须安装后验证\n\n- 真实安装 Skill、插件或 CLI\n- 执行脚本、修改本地文件或访问外部服务\n- 验证真实输出质量、性能和兼容性\n\n## 边界与风险判断卡\n\n- **把安装前预览误认为真实运行**：用户可能高估项目已经完成的配置、权限和兼容性验证。 处理方式：明确区分 prompt_preview_can_do 与 runtime_required。 Claim：`clm_0013` inferred 0.45\n- **命令执行会修改本地环境**：安装命令可能写入用户主目录、宿主插件目录或项目配置。 处理方式：先在隔离环境或测试账号中运行。 证据：`README.md` Claim：`clm_0014` supported 0.86\n- **待确认**：真实安装后是否与用户当前宿主 AI 版本兼容？。原因：兼容性只能通过实际宿主环境验证。\n- **待确认**：项目输出质量是否满足用户具体任务？。原因：安装前预览只能展示流程和边界，不能替代真实评测。\n- **待确认**：安装命令是否需要网络、权限或全局写入？。原因：这影响企业环境和个人环境的安装风险。\n\n## 开工前工作上下文\n\n### 加载顺序\n\n- 先读取 how_to_use.host_ai_instruction，建立安装前判断资产的边界。\n- 读取 claim_graph_summary，确认事实来自 Claim/Evidence Graph，而不是 Human Wiki 叙事。\n- 再读取 intended_users、capabilities 和 quick_start_candidates，判断用户是否匹配。\n- 需要执行具体任务时，优先查 role_skill_index，再查 evidence_index。\n- 遇到真实安装、文件修改、网络访问、性能或兼容性问题时，转入 risk_card 和 boundaries.runtime_required。\n\n### 任务路由\n\n- **命令行启动或安装流程**：先说明这是安装后验证能力，再给出安装前检查清单。 边界：必须真实安装或运行后验证。 证据：`README.md` Claim：`clm_0001` supported 0.86\n\n### 上下文规模\n\n- 文件总数：302\n- 重要文件覆盖：40/302\n- 证据索引条目：43\n- 角色 / Skill 条目：22\n\n### 证据不足时的处理\n\n- **missing_evidence**：说明证据不足，要求用户提供目标文件、README 段落或安装后验证记录；不要补全事实。\n- **out_of_scope_request**：说明该任务超出当前 AI Context Pack 证据范围，并建议用户先查看 Human Manual 或真实安装后验证。\n- **runtime_request**：给出安装前检查清单和命令来源，但不要替用户执行命令或声称已执行。\n- **source_conflict**：同时展示冲突来源，标记为待核实，不要强行选择一个版本。\n\n## Prompt Recipes\n\n### 适配判断\n\n- 目标：判断这个项目是否适合用户当前任务。\n- 预期输出：适配结论、关键理由、证据引用、安装前可预览内容、必须安装后验证内容、下一步建议。\n\n```text\n请基于 playwright-remote 的 AI Context Pack，先问我 3 个必要问题，然后判断它是否适合我的任务。回答必须包含：适合谁、能做什么、不能做什么、是否值得安装、证据来自哪里。所有项目事实必须引用 evidence_refs、source_paths 或 claim_id。\n```\n\n### 安装前体验\n\n- 目标：让用户在安装前感受核心工作流，同时避免把预览包装成真实能力或营销承诺。\n- 预期输出：一段带边界标签的体验剧本、安装后验证清单和谨慎建议；不含真实运行承诺或强营销表述。\n\n```text\n请把 playwright-remote 当作安装前体验资产，而不是已安装工具或真实运行环境。\n\n请严格输出四段：\n1. 先问我 3 个必要问题。\n2. 给出一段“体验剧本”：用 [安装前可预览]、[必须安装后验证]、[证据不足] 三种标签展示它可能如何引导工作流。\n3. 给出安装后验证清单：列出哪些能力只有真实安装、真实宿主加载、真实项目运行后才能确认。\n4. 给出谨慎建议：只能说“值得继续研究/试装”“先补充信息后再判断”或“不建议继续”，不得替项目背书。\n\n硬性边界：\n- 不要声称已经安装、运行、执行测试、修改文件或产生真实结果。\n- 不要写“自动适配”“确保通过”“完美适配”“强烈建议安装”等承诺性表达。\n- 如果描述安装后的工作方式，必须使用“如果安装成功且宿主正确加载 Skill，它可能会……”这种条件句。\n- 体验剧本只能写成“示例台词/假设流程”：使用“可能会询问/可能会建议/可能会展示”，不要写“已写入、已生成、已通过、正在运行、正在生成”。\n- Prompt Preview 不负责给安装命令；如用户准备试装，只能提示先阅读 Quick Start 和 Risk Card，并在隔离环境验证。\n- 所有项目事实必须来自 supported claim、evidence_refs 或 source_paths；inferred/unverified 只能作风险或待确认项。\n\n```\n\n### 角色 / Skill 选择\n\n- 目标：从项目里的角色或 Skill 中挑选最匹配的资产。\n- 预期输出：候选角色或 Skill 列表，每项包含适用场景、证据路径、风险边界和是否需要安装后验证。\n\n```text\n请读取 role_skill_index，根据我的目标任务推荐 3-5 个最相关的角色或 Skill。每个推荐都要说明适用场景、可能输出、风险边界和 evidence_refs。\n```\n\n### 风险预检\n\n- 目标：安装或引入前识别环境、权限、规则冲突和质量风险。\n- 预期输出：环境、权限、依赖、许可、宿主冲突、质量风险和未知项的检查清单。\n\n```text\n请基于 risk_card、boundaries 和 quick_start_candidates，给我一份安装前风险预检清单。不要替我执行命令，只说明我应该检查什么、为什么检查、失败会有什么影响。\n```\n\n### 宿主 AI 开工指令\n\n- 目标：把项目上下文转成一次对话开始前的宿主 AI 指令。\n- 预期输出：一段边界明确、证据引用明确、适合复制给宿主 AI 的开工前指令。\n\n```text\n请基于 playwright-remote 的 AI Context Pack，生成一段我可以粘贴给宿主 AI 的开工前指令。这段指令必须遵守 not_runtime=true，不能声称项目已经安装、运行或产生真实结果。\n```\n\n\n## 角色 / Skill 索引\n\n- 共索引 22 个角色 / Skill / 项目文档条目。\n\n- **✨ What's New**（project_doc）：Automate your web tasks while you stay in control 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`README.md`\n- **🚀 Running UI in Dev Mode**（project_doc）：Run the UI in dev mode make changes and see them reflected in the browser with hotreloading : 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`frontend/README.md`\n- **Magentic-UI Sample Scripts**（project_doc）：This directory contains sample scripts to help explore Magentic-UI. Each script showcases a different agent or team configuration. See below for a brief explanation of each sample: 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`samples/README.md`\n- **Reproducing Experimental Results**（project_doc）：Make sure to clone the repo and install Magentic-UI with the following command: 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`experiments/eval/README.md`\n- **Readme**（project_doc）：This code has been adapted from AutoGen Studio https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/backend/README.md`\n- **Readme**（project_doc）： 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/README.md`\n- **Readme**（project_doc）： 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/README.md`\n- **Readme**（project_doc）：AssistantBench benchmark https://huggingface.co/AssistantBench 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/assistantbench/README.md`\n- **Readme**（project_doc）：These files were obtained from the creators of the AssistantBench benchmark and modified slightly. You can find the latest version at https://huggingface.co/spaces/AssistantBench/leaderboard/tree/main/evaluation https://huggingface.co/spaces/AssistantBench/leaderboard/tree/main/evaluation 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/assistantbench/evaluate_utils/readme.md`\n- **Readme**（project_doc）： 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/bearcubs/README.md`\n- **Readme**（project_doc）： 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/custom/README.md`\n- **Readme**（project_doc）：GAIA Benchmark https://huggingface.co/datasets/gaia-benchmark/GAIA 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/gaia/README.md`\n- **GPQA Eval**（project_doc）： 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/gpqa/README.md`\n- **SimpleQA Eval**（project_doc）：SimpleQA: Measuring short-form factuality in large language models Authors: Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, William Fedus https://cdn.openai.com/papers/simpleqa.pdf 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/simpleqa/README.md`\n- **Readme**（project_doc）：WebGames benchmark https://webgames.convergence.ai/ https://huggingface.co/datasets/convergence-ai/webgames 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/webgames/README.md`\n- **Readme**（project_doc）：WebVoyager benchmark https://github.com/MinorJerry/WebVoyager 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`src/magentic_ui/eval/benchmarks/webvoyager/README.md`\n- **Contributing to Magentic-UI**（project_doc）：Thank you for your interest in contributing to Magentic-UI! 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`CONTRIBUTING.md`\n- **Microsoft Open Source Code of Conduct**（project_doc）：Microsoft Open Source Code of Conduct 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`CODE_OF_CONDUCT.md`\n- **Security**（project_doc）：Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include Microsoft https://github.com/Microsoft , Azure https://github.com/Azure , DotNet https://github.com/dotnet , AspNet https://github.com/aspnet and Xamarin https://github.com/xamarin . 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`SECURITY.md`\n- **Support**（project_doc）：This project uses GitHub Issues to track bugs and feature requests. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new Issue. 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`SUPPORT.md`\n- **Magentic-UI**（project_doc）：Magentic-UI is a human-centered computer use agent CUA designed for collaboration with people on web-based tasks. Magentic-UI operates a web browser and other tools, like code execution and file navigation, in real-time while optimizing for human-in-the-loop HIL orchestration. 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`TRANSPARENCY_NOTE.md`\n- **⚠️ TROUBLESHOOTING**（project_doc）：This document lists common issues users have encountered with Magentic-UI and how to resolve them. 激活提示：当用户需要理解项目结构、安装方式或边界时参考。 证据：`TROUBLESHOOTING.md`\n\n## 证据索引\n\n- 共索引 43 条证据。\n\n- **✨ What's New**（documentation）：Automate your web tasks while you stay in control 证据：`README.md`\n- **🚀 Running UI in Dev Mode**（documentation）：Run the UI in dev mode make changes and see them reflected in the browser with hotreloading : 证据：`frontend/README.md`\n- **Magentic-UI Sample Scripts**（documentation）：This directory contains sample scripts to help explore Magentic-UI. Each script showcases a different agent or team configuration. See below for a brief explanation of each sample: 证据：`samples/README.md`\n- **Reproducing Experimental Results**（documentation）：Make sure to clone the repo and install Magentic-UI with the following command: 证据：`experiments/eval/README.md`\n- **Readme**（documentation）：This code has been adapted from AutoGen Studio https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio 证据：`src/magentic_ui/backend/README.md`\n- **Readme**（documentation）：AssistantBench benchmark https://huggingface.co/AssistantBench 证据：`src/magentic_ui/eval/benchmarks/assistantbench/README.md`\n- **Readme**（documentation）：These files were obtained from the creators of the AssistantBench benchmark and modified slightly. You can find the latest version at https://huggingface.co/spaces/AssistantBench/leaderboard/tree/main/evaluation https://huggingface.co/spaces/AssistantBench/leaderboard/tree/main/evaluation 证据：`src/magentic_ui/eval/benchmarks/assistantbench/evaluate_utils/readme.md`\n- **Readme**（documentation）：Bearcubs benchmark 证据：`src/magentic_ui/eval/benchmarks/bearcubs/README.md`\n- **Readme**（documentation）：GAIA Benchmark https://huggingface.co/datasets/gaia-benchmark/GAIA 证据：`src/magentic_ui/eval/benchmarks/gaia/README.md`\n- **GPQA Eval**（documentation）：GPQA Eval Run instructions: 证据：`src/magentic_ui/eval/benchmarks/gpqa/README.md`\n- **SimpleQA Eval**（documentation）：SimpleQA: Measuring short-form factuality in large language models Authors: Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, William Fedus https://cdn.openai.com/papers/simpleqa.pdf 证据：`src/magentic_ui/eval/benchmarks/simpleqa/README.md`\n- **Readme**（documentation）：WebGames benchmark https://webgames.convergence.ai/ https://huggingface.co/datasets/convergence-ai/webgames 证据：`src/magentic_ui/eval/benchmarks/webgames/README.md`\n- **Readme**（documentation）：WebVoyager benchmark https://github.com/MinorJerry/WebVoyager 证据：`src/magentic_ui/eval/benchmarks/webvoyager/README.md`\n- **Package**（package_manifest）：{ \"name\": \"Magentic-UI\", \"version\": \"1.0.0\", \"private\": true, \"description\": \"Magentic-UI\", \"author\": \"Microsoft\", \"keywords\": \"gatsby\" , \"scripts\": { \"develop\": \"gatsby clean && gatsby develop\", \"dev\": \"npm run develop\", \"start\": \"gatsby clean && gatsby develop\", \"build\": \"gatsby clean && rm -rf ../src/magentic ui/backend/web/ui && PREFIX PATH VALUE='' gatsby build --prefix-paths && rsync -a --delete public/ ../src/magentic ui/backend/web/ui/\", \"serve\": \"gatsby serve\", \"clean\": \"gatsby clean\", \"typecheck\": \"tsc --noEmit\" }, \"dependencies\": { \"@dagrejs/dagre\": \"^1.1.4\", \"@dnd-kit/core\": \"^6.2.0\", \"@headlessui/react\": \"^2.2.0\", \"@hello-pangea/dnd\": \"^17.0.0\", \"@heroicons/react\": \"^2.0.18\", \"… 证据：`frontend/package.json`\n- **Contributing to Magentic-UI**（documentation）：Thank you for your interest in contributing to Magentic-UI! 证据：`CONTRIBUTING.md`\n- **Package**（package_manifest）：{ \"name\": \"playwright-remote\", \"version\": \"1.0.0\", \"dependencies\": { \"playwright\": \"1.51.1\" } } 证据：`docker/magentic-ui-browser-docker/package.json`\n- **License**（source_file）：Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files the \"Software\" , to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 证据：`LICENSE`\n- **License**（source_file）：Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ 证据：`src/magentic_ui/eval/benchmarks/assistantbench/evaluate_utils/LICENSE`\n- **Microsoft Open Source Code of Conduct**（documentation）：Microsoft Open Source Code of Conduct 证据：`CODE_OF_CONDUCT.md`\n- **Security**（documentation）：Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include Microsoft https://github.com/Microsoft , Azure https://github.com/Azure , DotNet https://github.com/dotnet , AspNet https://github.com/aspnet and Xamarin https://github.com/xamarin . 证据：`SECURITY.md`\n- **Support**（documentation）：This project uses GitHub Issues to track bugs and feature requests. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new Issue. 证据：`SUPPORT.md`\n- **Magentic-UI**（documentation）：Magentic-UI is a human-centered computer use agent CUA designed for collaboration with people on web-based tasks. Magentic-UI operates a web browser and other tools, like code execution and file navigation, in real-time while optimizing for human-in-the-loop HIL orchestration. 证据：`TRANSPARENCY_NOTE.md`\n- **⚠️ TROUBLESHOOTING**（documentation）：This document lists common issues users have encountered with Magentic-UI and how to resolve them. 证据：`TROUBLESHOOTING.md`\n- **Tsconfig**（structured_config）：{ \"compilerOptions\": { / Visit https://aka.ms/tsconfig.json to read more about this file / 证据：`frontend/tsconfig.json`\n- **Dependabot**（source_file）：version: 2 updates: - package-ecosystem: \"pip\" directory: \"/\" project root where pyproject.toml and uv.lock live schedule: interval: \"weekly\" 证据：`.github/dependabot.yml`\n- **Byte-compiled / optimized / DLL files**（source_file）：Byte-compiled / optimized / DLL files pycache / .py cod $py.class 证据：`.gitignore`\n- **.python-version**（source_file）：3.12 证据：`.python-version`\n- **!/bin/bash**（source_file）：CWD=\"$ pwd \" if \"${CWD /}\" != \"docker\" ; then echo \"Error: Must run this script from the docker folder.\" exit 1 fi 证据：`docker/build-all.sh`\n- **Index**（source_file）：Magentic-UI body { margin: 0; padding: 0; background-color: ffffff; } .markdown-body { box-sizing: border-box; min-width: 200px; max-width: 980px; margin: 0 auto; padding: 45px; } @media max-width: 767px { .markdown-body { padding: 15px; } } .loading { text-align: center; padding: 50px; color: 666; } .error { text-align: center; padding: 50px; color: d73a49; } / GitHub-style header / .header { background-color: 24292f; color: white; padding: 16px 0; margin-bottom: 32px; } .header-content { max-width: 980px; margin: 0 auto; padding: 0 45px; display: flex; align-items: center; gap: 16px; } .header h1 { margin: 0; font-size: 20px; font-weight: 600; } .header a { color: 7d8590; text-decoration:… 证据：`docs/index.html`\n- **Fara Config**（source_file）：model config local surfer: &client surfer provider: OpenAIChatCompletionClient config: model: \"microsoft/Fara-7B\" base url: http://localhost:5000/v1 api key: not-needed model info: vision: true function calling: true json output: false family: \"unknown\" structured output: false multiple system messages: false 证据：`fara_config.yaml`\n- **.Env**（source_file）：GATSBY API URL=http://127.0.0.1:8081/api 证据：`frontend/.env.default`\n- **frontend/.gitignore**（source_file）：node modules/ .cache/ public src/gatsby-types.d.ts .env.development .env.production 证据：`frontend/.gitignore`\n- **Gatsby Browser**（source_file）：import \"antd/dist/reset.css\"; import \"./src/styles/global.css\"; 证据：`frontend/gatsby-browser.js`\n- **Gatsby Config**（source_file）：import type { GatsbyConfig } from \"gatsby\"; import fs from \"fs\"; 证据：`frontend/gatsby-config.ts`\n- **Gatsby Ssr**（source_file）：const codeToRunOnClient = function { try { var mode = localStorage.getItem 'darkmode' ; document.getElementsByTagName \"html\" 0 .className === 'dark' ? 'dark' : 'light'; } catch e {} } ; ; 证据：`frontend/gatsby-ssr.tsx`\n- **Postcss.Config**（source_file）：module.exports = { plugins: { tailwindcss: {}, autoprefixer: {}, }, } 证据：`frontend/postcss.config.js`\n- **Tailwind.Config**（source_file）：/ @type {import 'tailwindcss' .Config} / module.exports = { content: ./src/pages/ / .{js,jsx,ts,tsx} , ./src/components/ / .{js,jsx,ts,tsx} , , theme: { extend: { keyframes: { 'fade-in': { '0%': { opacity: '0' }, '100%': { opacity: '1' }, }, 'scale-in': { '0%': { transform: 'scale 0.95 ', opacity: '0' }, '100%': { transform: 'scale 1 ', opacity: '1' }, }, }, animation: { 'fade-in': 'fade-in 0.1s ease-out', 'scale-in': 'scale-in 0.1s ease-out', }, typography: { DEFAULT: { css: { maxWidth: \"100ch\", }, }, }, transitionProperty: { height: \"height\", spacing: \"margin, padding\", }, colors: { primary: \"var --color-bg-primary \", secondary: \"var --color-bg-secondary \", accent: \"var --color-bg-accent… 证据：`frontend/tailwind.config.js`\n- **from https://blog.wolt.com/engineering/2021/09/30/professional-grade-mypy-configuration/**（source_file）：build-system requires = \"hatchling\" build-backend = \"hatchling.build\" 证据：`pyproject.toml`\n- **Pytest**（source_file）：pytest asyncio default fixture loop scope = function markers = npx: Tests that require npx in the host PATH 证据：`pytest.ini`\n- **Simple approval function that always returns yes**（source_file）：import asyncio from autogen agentchat.messages import TextMessage from autogen core import CancellationToken from autogen ext.agents.azure. azure ai agent import AzureAIAgent from azure.ai.projects.aio import AIProjectClient from azure.identity.aio import DefaultAzureCredential import dotenv import os from typing import Union, Any, Dict, Optional from autogen core.models import ChatCompletionClient from autogen core import ComponentModel from autogen agentchat.agents import UserProxyAgent 证据：`samples/sample_azure_agent.py`\n- **Sample Coder**（source_file）：import asyncio import argparse from autogen agentchat.ui import Console from autogen ext.models.openai import OpenAIChatCompletionClient from autogen agentchat.conditions import TextMentionTermination from magentic ui.agents import CoderAgent from magentic ui.teams import RoundRobinGroupChat from autogen agentchat.agents import UserProxyAgent 证据：`samples/sample_coder.py`\n- **Configure logging to print to console**（source_file）：import asyncio import argparse from autogen agentchat.ui import Console from autogen ext.models.openai import OpenAIChatCompletionClient from autogen agentchat.conditions import TextMentionTermination from magentic ui.teams import RoundRobinGroupChat from magentic ui.agents import FileSurfer from autogen agentchat.agents import UserProxyAgent 证据：`samples/sample_file_surfer.py`\n- **Configure logging**（source_file）：import argparse import asyncio from pathlib import Path from autogen agentchat.ui import Console from autogen ext.models.openai import OpenAIChatCompletionClient from autogen agentchat.conditions import TextMentionTermination from autogen agentchat.teams import RoundRobinGroupChat from magentic ui.agents import WebSurfer from autogen agentchat.agents import UserProxyAgent import logging 证据：`samples/sample_web_surfer.py`\n\n## 宿主 AI 必须遵守的规则\n\n- **把本资产当作开工前上下文，而不是运行环境。**：AI Context Pack 只包含证据化项目理解，不包含目标项目的可执行状态。 证据：`README.md`, `frontend/README.md`, `samples/README.md`\n- **回答用户时区分可预览内容与必须安装后才能验证的内容。**：安装前体验的消费者价值来自降低误装和误判，而不是伪装成真实运行。 证据：`README.md`, `frontend/README.md`, `samples/README.md`\n\n## 用户开工前应该回答的问题\n\n- 你准备在哪个宿主 AI 或本地环境中使用它？\n- 你只是想先体验工作流，还是准备真实安装？\n- 你最在意的是安装成本、输出质量、还是和现有规则的冲突？\n\n## 验收标准\n\n- 所有能力声明都能回指到 evidence_refs 中的文件路径。\n- AI_CONTEXT_PACK.md 没有把预览包装成真实运行。\n- 用户能在 3 分钟内看懂适合谁、能做什么、如何开始和风险边界。\n\n---\n\n## Doramagic Context Augmentation\n\n下面内容用于强化 Repomix/AI Context Pack 主体。Human Manual 只提供阅读骨架；踩坑日志会被转成宿主 AI 必须遵守的工作约束。\n\n## Human Manual 骨架\n\n使用规则：这里只是项目阅读路线和显著性信号，不是事实权威。具体事实仍必须回到 repo evidence / Claim Graph。\n\n宿主 AI 硬性规则：\n- 不得把页标题、章节顺序、摘要或 importance 当作项目事实证据。\n- 解释 Human Manual 骨架时，必须明确说它只是阅读路线/显著性信号。\n- 能力、安装、兼容性、运行状态和风险判断必须引用 repo evidence、source path 或 Claim Graph。\n\n- **Getting Started with Magentic-UI**：importance `high`\n - source_paths: README.md, pyproject.toml, TROUBLESHOOTING.md\n- **Key Concepts**：importance `high`\n - source_paths: src/magentic_ui/approval_guard.py, src/magentic_ui/guarded_action.py, src/magentic_ui/learning/learner.py\n- **High-Level Architecture**：importance `high`\n - source_paths: src/magentic_ui/backend/web/app.py, src/magentic_ui/backend/teammanager/teammanager.py, src/magentic_ui/task_team.py\n- **Agent System**：importance `high`\n - source_paths: src/magentic_ui/agents/web_surfer/_web_surfer.py, src/magentic_ui/agents/file_surfer/_file_surfer.py, src/magentic_ui/agents/_coder.py, src/magentic_ui/agents/_user_proxy.py\n- **Team Orchestration**：importance `high`\n - source_paths: src/magentic_ui/teams/orchestrator/_orchestrator.py, src/magentic_ui/teams/orchestrator/_group_chat.py, src/magentic_ui/teams/orchestrator/_prompts.py, src/magentic_ui/teams/orchestrator/_sentinel_prompts.py\n- **Backend API**：importance `high`\n - source_paths: src/magentic_ui/backend/web/app.py, src/magentic_ui/backend/web/routes/ws.py, src/magentic_ui/backend/web/routes/runs.py, src/magentic_ui/backend/web/routes/sessions.py, src/magentic_ui/backend/web/managers/connection.py\n- **Frontend UI**：importance `high`\n - source_paths: frontend/src/components/views/chat/chat.tsx, frontend/src/components/views/manager.tsx, frontend/src/components/settings/SettingsModal.tsx, frontend/src/components/features/McpServerSelector/McpServerSelector.tsx, frontend/src/hooks/provider.tsx\n- **Browser Automation**：importance `high`\n - source_paths: src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py, src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py, src/magentic_ui/tools/playwright/browser/local_playwright_browser.py, src/magentic_ui/tools/playwright/playwright_controller.py, src/magentic_ui/tools/playwright/playwright_state.py\n\n## Repo Inspection Evidence / 源码检查证据\n\n- repo_clone_verified: true\n- repo_inspection_verified: true\n- repo_commit: `f705b0ee29af5ddc03953e6d3be4c0a880a78b09`\n- inspected_files: `pyproject.toml`, `README.md`, `uv.lock`, `src/magentic_ui/version.py`, `src/magentic_ui/_docker.py`, `src/magentic_ui/guarded_action.py`, `src/magentic_ui/_cli.py`, `src/magentic_ui/magentic_ui_config.py`, `src/magentic_ui/input_func.py`, `src/magentic_ui/approval_guard.py`, `src/magentic_ui/task_team.py`, `src/magentic_ui/types.py`, `src/magentic_ui/utils.py`, `src/magentic_ui/__init__.py`, `src/magentic_ui/backend/cli.py`, `src/magentic_ui/backend/README.md`, `src/magentic_ui/backend/__init__.py`, `src/magentic_ui/cli/pretty_console.py`, `src/magentic_ui/cli/__init__.py`, `src/magentic_ui/teams/roundrobin_orchestrator.py`\n\n宿主 AI 硬性规则：\n- 没有 repo_clone_verified=true 时，不得声称已经读过源码。\n- 没有 repo_inspection_verified=true 时，不得把 README/docs/package 文件判断写成事实。\n- 没有 quick_start_verified=true 时，不得声称 Quick Start 已跑通。\n\n## Doramagic Pitfall Constraints / 踩坑约束\n\n这些规则来自 Doramagic 发现、验证或编译过程中的项目专属坑点。宿主 AI 必须把它们当作工作约束，而不是普通说明文字。\n\n### Constraint 1: 来源证据：Create tutorials and documentation for the codebase\n\n- Trigger: GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Create tutorials and documentation for the codebase\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_c0979f7ebb064422a6a8095561f6a9bd | https://github.com/microsoft/magentic-ui/issues/154 | 来源类型 github_issue 暴露的待验证使用条件。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 2: 来源证据：Support Podman in place of Docker\n\n- Trigger: GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Support Podman in place of Docker\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_f88231cc4cad442ca53d92ea3a40a655 | https://github.com/microsoft/magentic-ui/issues/312 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 3: 来源证据：magentic-ui can't display all the html element\n\n- Trigger: GitHub 社区证据显示该项目存在一个安装相关的待验证问题：magentic-ui can't display all the html element\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_21f19953edd74379ab2d25cedc37ca1b | https://github.com/microsoft/magentic-ui/issues/362 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 4: 来源证据：Refreshing or restart the web app will make the current Session unavailable\n\n- Trigger: GitHub 社区证据显示该项目存在一个配置相关的待验证问题：Refreshing or restart the web app will make the current Session unavailable\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_83a9cafd59254028853ef84cd1ccc756 | https://github.com/microsoft/magentic-ui/issues/336 | 来源讨论提到 node 相关条件，需在安装/试用前复核。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 5: 能力判断依赖假设\n\n- Trigger: README/documentation is current enough for a first validation pass.\n- Host AI rule: 将假设转成下游验证清单。\n- Why it matters: 假设不成立时，用户拿不到承诺的能力。\n- Evidence: capability.assumptions | github_repo:978331188 | https://github.com/microsoft/magentic-ui | README/documentation is current enough for a first validation pass.\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 6: 来源证据：Why not conduct a requirement analysis before the plan?\n\n- Trigger: GitHub 社区证据显示该项目存在一个运行相关的待验证问题：Why not conduct a requirement analysis before the plan?\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_6003a9c2194f40c0865145385cf98c32 | https://github.com/microsoft/magentic-ui/issues/321 | 来源类型 github_issue 暴露的待验证使用条件。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 7: 来源证据：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n\n- Trigger: GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n- Host AI rule: 来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- Why it matters: 可能增加新用户试用和生产接入成本。\n- Evidence: community_evidence:github | cevd_7e754869326e42e1a7c57f3a1962ef9e | https://github.com/microsoft/magentic-ui/issues/360 | 来源类型 github_issue 暴露的待验证使用条件。\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 8: 维护活跃度未知\n\n- Trigger: 未记录 last_activity_observed。\n- Host AI rule: 补 GitHub 最近 commit、release、issue/PR 响应信号。\n- Why it matters: 新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。\n- Evidence: evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | last_activity_observed missing\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 9: 下游验证发现风险项\n\n- Trigger: no_demo\n- Host AI rule: 进入安全/权限治理复核队列。\n- Why it matters: 下游已经要求复核，不能在页面中弱化。\n- Evidence: downstream_validation.risk_items | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n\n### Constraint 10: 存在评分风险\n\n- Trigger: no_demo\n- Host AI rule: 把风险写入边界卡，并确认是否需要人工复核。\n- Why it matters: 风险会影响是否适合普通用户安装。\n- Evidence: risks.scoring_risks | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n- Hard boundary: 不要把这个坑点包装成已解决、已验证或可忽略，除非后续验证证据明确证明它已经关闭。\n",
"summary": "给宿主 AI 的上下文和工作边界。",
"title": "AI Context Pack / 带给我的 AI"
},
"boundary_risk_card": {
"asset_id": "boundary_risk_card",
"filename": "BOUNDARY_RISK_CARD.md",
"markdown": "# Boundary & Risk Card / 安装前决策卡\n\n项目：microsoft/magentic-ui\n\n## Doramagic 试用结论\n\n当前结论：可以进入发布前推荐检查；首次使用仍应从最小权限、临时目录和可回滚配置开始。\n\n## 用户现在可以做\n\n- 可以先阅读 Human Manual，理解项目目的和主要工作流。\n- 可以复制 Prompt Preview 做安装前体验；这只验证交互感，不代表真实运行。\n- 可以把官方 Quick Start 命令放到隔离环境中验证，不要直接进主力环境。\n\n## 现在不要做\n\n- 不要把 Prompt Preview 当成项目实际运行结果。\n- 不要把 metadata-only validation 当成沙箱安装验证。\n- 不要把未验证能力写成“已支持、已跑通、可放心安装”。\n- 不要在首次试用时交出生产数据、私人文件、真实密钥或主力配置目录。\n\n## 安装前检查\n\n- 宿主 AI 是否匹配：local_cli\n- 官方安装入口状态：已发现官方入口\n- 是否在临时目录、临时宿主或容器中验证：必须是\n- 是否能回滚配置改动：必须能\n- 是否需要 API Key、网络访问、读写文件或修改宿主配置：未确认前按高风险处理\n- 是否记录了安装命令、实际输出和失败日志：必须记录\n\n## 当前阻塞项\n\n- 无阻塞项。\n\n## 项目专属踩坑\n\n- 来源证据：Create tutorials and documentation for the codebase（medium）：可能增加新用户试用和生产接入成本。 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 来源证据：Support Podman in place of Docker（medium）：可能增加新用户试用和生产接入成本。 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 来源证据：magentic-ui can't display all the html element（medium）：可能增加新用户试用和生产接入成本。 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 来源证据：Refreshing or restart the web app will make the current Session unavailable（medium）：可能增加新用户试用和生产接入成本。 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 能力判断依赖假设（medium）：假设不成立时，用户拿不到承诺的能力。 建议检查：将假设转成下游验证清单。\n\n## 风险与权限提示\n\n- no_demo: medium\n\n## 证据缺口\n\n- 暂未发现结构化证据缺口。\n",
"summary": "安装、权限、验证和推荐前风险。",
"title": "Boundary & Risk Card / 边界与风险卡"
},
"human_manual": {
"asset_id": "human_manual",
"filename": "HUMAN_MANUAL.md",
"markdown": "# https://github.com/microsoft/magentic-ui 项目说明书\n\n生成时间：2026-05-18 02:19:23 UTC\n\n## 目录\n\n- [Getting Started with Magentic-UI](#getting-started)\n- [Key Concepts](#key-concepts)\n- [High-Level Architecture](#high-level-architecture)\n- [Agent System](#agent-system)\n- [Team Orchestration](#team-orchestration)\n- [Backend API](#backend-api)\n- [Frontend UI](#frontend-ui)\n- [Browser Automation](#browser-automation)\n- [Docker Containers](#docker-containers)\n- [Configuration](#configuration)\n\n\n\n## Getting Started with Magentic-UI\n\n### 相关页面\n\n相关主题：[Configuration](#configuration), [Docker Containers](#docker-containers)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [README.md](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n- [TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n- [CONTRIBUTING.md](https://github.com/microsoft/magentic-ui/blob/main/CONTRIBUTING.md)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [pyproject.toml](https://github.com/microsoft/magentic-ui/blob/main/pyproject.toml)\n \n\n# Getting Started with Magentic-UI\n\nMagentic-UI is a Microsoft open-source project that provides a multi-agent framework for building AI-powered user interfaces. It enables developers to create intelligent agents that can browse the web, execute plans, handle file uploads, and interact with users through a web-based chat interface.\n\n## Prerequisites\n\nBefore getting started with Magentic-UI, ensure your system meets the following requirements:\n\n| Requirement | Version/Details |\n|-------------|-----------------|\n| Python | 3.10 or higher |\n| Docker | Latest stable version |\n| Node.js | For frontend development |\n| pip | Latest version |\n\n资料来源：[README.md:1-20](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n### System Dependencies\n\n- **Docker**: Required for running agent containers that provide browser automation capabilities\n- **Node.js**: Needed only if you plan to modify the frontend code\n\n## Installation\n\n### Standard Installation\n\nThe simplest way to install Magentic-UI is via pip:\n\n```bash\npip install magentic-ui\n```\n\n资料来源：[README.md:25-30](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n### Installation with Fara-7B Support\n\nTo use the Fara-7B model locally, install with the fara extras:\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\npip install magentic-ui[fara]\n```\n\n资料来源：[README.md:55-60](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Quick Start with Docker\n\nAfter installation, start Magentic-UI using Docker:\n\n```bash\nmagentic-ui --port 8081\n```\n\n> **Note**: Running this command for the first time will pull two Docker images required for the Magentic-UI agents. If you encounter problems, you can build them directly with:\n\n```bash\ncd docker\nsh build-all.sh\n```\n\nOnce the server is running, access the UI at [http://localhost:8081](http://localhost:8081).\n\n资料来源：[README.md:30-45](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Local Development Setup\n\n### Backend Development\n\nFor local backend development, clone the repository and set up the environment:\n\n```bash\ngit clone https://github.com/microsoft/magentic-ui.git\ncd magentic-ui\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -e .\n```\n\nRun the development server:\n\n```bash\nmagentic ui --port 8081\n```\n\n资料来源：[TROUBLESHOOTING.md:1-20](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n### Frontend Development\n\nThe frontend is located in the `frontend/` directory of the repository. To set up for development:\n\n1. Navigate to the frontend directory:\n\n```bash\ncd frontend\n```\n\n2. Create environment configuration:\n\n```bash\ncp .env.default .env.development\n```\n\n3. Configure the API URL:\n\nEdit `.env.development` and set:\n\n```\nGATSBY_API_URL=http://localhost:8081/api\n```\n\n资料来源：[frontend/README.md:1-15](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Connecting Frontend to Backend\n\nThe frontend makes requests to the backend API expecting responses at `http://localhost:8081/api`. Ensure `GATSBY_API_URL` is correctly set in your environment configuration.\n\n资料来源：[frontend/README.md:10-12](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## Using Fara-7B Locally\n\nTo run Magentic-UI with a local Fara-7B model:\n\n### Step 1: Serve the Model\n\nIn a separate process, serve the Fara-7B model using vLLM:\n\n```bash\nvllm serve \"microsoft/Fara-7B\" --port 5000 --dtype auto \n```\n\n### Step 2: Create Configuration\n\nCreate a `fara_config.yaml` file with the following content:\n\n```yaml\nmodel_config_local_surfer: &client_surfer\n provider: OpenAIChatCompletionClient\n config:\n model: \"microsoft/Fara-7B\"\n base_url: http://localhost:5000/v1\n api_key: not-needed\n model_info:\n vision: true\n function_calling: true\n json_output: false\n family: \"unknown\" \n structured_output: false\n multiple_system_messages: false\n\norchestrator_client: *client\n```\n\n资料来源：[README.md:60-80](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Core Features\n\n### Web Surfer Agent\n\nThe web surfer agent enables browsing and interacting with web pages. Available actions include:\n\n| Action | Description |\n|--------|-------------|\n| `key` | Performs key presses (Enter, Alt, Shift, Tab, etc.) |\n| `type` | Types text into input fields |\n| `mouse_move` | Moves cursor to specified pixel coordinates |\n| `left_click` | Clicks the left mouse button |\n| `scroll` | Scrolls the mouse wheel |\n| `visit_url` | Navigates to a specified URL |\n| `web_search` | Performs a web search |\n| `history_back` | Goes back in browser history |\n| `pause_and_memorize_fact` | Stores information for future reference |\n| `wait` | Waits for specified seconds |\n| `terminate` | Ends the current task |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-60](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### Plans System\n\nMagentic-UI supports creating and managing reusable plans:\n\n- **Create Plans**: Build step-by-step task plans through the UI\n- **Attach Plans**: Attach saved plans to queries for execution\n- **Learn Plans**: Save successful conversation patterns as reusable plans\n- **Import/Export**: Import existing plans or export your library\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-50](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### File Handling\n\nThe system supports:\n\n- Arbitrary file uploads\n- File preview and download\n- Multiple file type support (images, documents, etc.)\n\n资料来源：[frontend/src/components/common/filerenderer.tsx:1-80](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n### MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers with multiple connection types:\n\n| Connection Type | Description |\n|-----------------|-------------|\n| SSE | Server-Sent Events connection |\n| Stdio | Standard input/output connection |\n| JSON Config | JSON-based configuration file |\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-60](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## Project Structure\n\n```\nmagentic-ui/\n├── frontend/ # React frontend application\n│ ├── src/\n│ │ ├── components/ # Reusable UI components\n│ │ │ ├── features/ # Feature-specific components\n│ │ │ ├── views/ # View components\n│ │ │ └── common/ # Common utilities\n│ │ └── pages/ # Page components\n│ └── README.md # Frontend development guide\n├── src/\n│ └── magentic_ui/ # Main Python package\n│ └── agents/ # Agent implementations\n├── docker/ # Docker configuration\n├── README.md # Main documentation\n├── CONTRIBUTING.md # Contribution guidelines\n└── TROUBLESHOOTING.md # Issue resolution guide\n```\n\n## Troubleshooting\n\n### Common Issues\n\n#### Port Already in Use\n\nIf port 8081 is occupied, either stop the existing service or use a different port:\n\n```bash\nmagentic ui --port 8082\n```\n\n#### Virtual Environment Activation\n\nIf you installed in a virtual environment but it didn't activate:\n\n```bash\ndeactivate\nsource .venv/bin/activate\nmagentic ui --port 8081\n```\n\n#### Wrong Package Installed\n\nEnsure you installed `magentic-ui` (not the unrelated `magentic` package):\n\n```bash\npip install magentic-ui\n```\n\n### Getting Help\n\nIf issues persist:\n\n1. Double-check all prerequisites in the README\n2. Search [GitHub Issues](https://github.com/microsoft/magentic-ui/issues) for similar problems\n3. Open a new issue with:\n - Detailed problem description\n - System information (OS, Docker version)\n - Steps to replicate\n\n资料来源：[TROUBLESHOOTING.md:1-50](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Contributing\n\nWe welcome community contributions:\n\n1. **Find an Issue**: Browse [All Issues](https://github.com/microsoft/magentic-ui/issues) and look for `help-wanted` labeled items\n2. **Fork and Clone**: Fork the repository and clone locally\n3. **Create a Branch**: Use descriptive names (e.g., `fix/session-bug` or `feature/file-upload`)\n4. **Write Code and Tests**: Include tests for new features\n5. **Run Checks**: Before submitting PR, run:\n\n```bash\npoe check\n```\n\n6. **Submit PR**: Open against `main` branch and reference the issue number\n\n### Top \"Help Wanted\" Issues\n\n| Issue | Description |\n|-------|-------------|\n| [#132](https://github.com/microsoft/magentic-ui/issues/132) | Allow MAGUI to understand video and audio |\n| [#128](https://github.com/microsoft/magentic-ui/issues/128) | Enable arbitrary file upload in UI |\n| [#126](https://github.com/microsoft/magentic-ui/issues/126) | Add streaming of final answer and coder messages |\n| [#123](https://github.com/microsoft/magentic-ui/issues/123) | Add unit tests |\n| [#124](https://github.com/microsoft/magentic-ui/issues/124) | Allow websurfer to scroll inside containers |\n\n资料来源：[CONTRIBUTING.md:1-40](https://github.com/microsoft/magentic-ui/blob/main/CONTRIBUTING.md)\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n A[User Interface] --> B[Frontend React App]\n B --> C[Backend API]\n C --> D[Magentic-UI Agents]\n D --> E[Web Surfer Agent]\n D --> F[Plans Executor]\n D --> G[MCP Servers]\n E --> H[Docker Containers]\n H --> I[Browser Automation]\n F --> J[Task Execution]\n G --> K[External Tools]\n```\n\n## Configuration Reference\n\n### Environment Variables (Frontend)\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `GATSBY_API_URL` | Backend API endpoint | `http://localhost:8081/api` |\n\n### Docker Ports\n\n| Port | Service |\n|------|---------|\n| 8081 | Main application (configurable) |\n| 5000 | vLLM model server (Fara-7B) |\n\n---\n\n\n\n## Key Concepts\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Agent System](#agent-system)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/approval_guard.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/approval_guard.py)\n- [src/magentic_ui/guarded_action.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/guarded_action.py)\n- [src/magentic_ui/learning/learner.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/learning/learner.py)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/hooks/store.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/store.tsx)\n \n\n# Key Concepts\n\nMagentic-UI is an interactive UI framework that enables AI agents to execute tasks with human oversight. This document explains the fundamental concepts that power the system's architecture, including approval workflows, guarded actions, plan management, and the learning system.\n\n---\n\n## 1. Approval Guard System\n\nThe Approval Guard System is a security mechanism that ensures AI agents require explicit human authorization before executing sensitive or irreversible actions. It acts as a gatekeeper between agent decision-making and action execution.\n\n### 1.1 Purpose and Scope\n\nThe approval guard intercepts actions proposed by agents and pauses execution until a human user provides approval or denial. This allows users to:\n\n- Review actions before they are executed\n- Modify parameters before approval\n- Deny dangerous or unintended operations\n- Maintain control over agent behavior in automated workflows\n\n### 1.2 Architecture Overview\n\n```mermaid\ngraph TD\n A[Agent Decision] --> B{Approval Required?}\n B -->|Yes| C[Approval Guard]\n B -->|No| D[Execute Action]\n C --> E[Pause Execution]\n E --> F[User Notification]\n F --> G{User Decision}\n G -->|Approve| H[Execute Action]\n G -->|Deny| I[Cancel Action]\n G -->|Modify| J[Update Parameters] --> H\n```\n\n### 1.3 Run Status States\n\nThe system uses a status-based workflow to track the state of agent runs. The following table documents the primary run states:\n\n| Status | Description |\n|--------|-------------|\n| `created` | Run has been initialized but not started |\n| `active` | Run is currently executing |\n| `awaiting_input` | Run is paused waiting for user input |\n| `paused` | Run has been paused by user or system |\n| `completed` | Run finished successfully |\n| `failed` | Run encountered an error |\n\n资料来源：[frontend/src/components/views/chat/runview.tsx:1-50]()\n\n### 1.4 Approval Guard Configuration\n\nThe approval guard supports different policy configurations:\n\n| Policy | Behavior |\n|--------|----------|\n| `AutoApprove` | All actions are automatically approved |\n| `RequireApproval` | All actions require explicit user approval |\n| `HybridPolicy` | Some actions auto-approved, others require approval |\n\n---\n\n## 2. Guarded Actions\n\nGuarded Actions represent the atomic operations that agents can perform. Each action has a type, parameters, and metadata about whether approval is required.\n\n### 2.1 Action Structure\n\nEach guarded action contains:\n\n```python\n{\n \"action\": str, # Action type identifier\n \"parameters\": dict, # Action-specific parameters\n \"requires_approval\": bool, # Whether approval is needed\n \"timestamp\": datetime, # When action was created\n \"status\": str # pending, approved, denied, executed\n}\n```\n\n### 2.2 Action Types\n\nThe system supports multiple action categories:\n\n| Category | Examples | Description |\n|----------|----------|-------------|\n| Navigation | `visit_url`, `history_back` | Browser navigation actions |\n| Input | `type`, `key`, `mouse_move` | User input simulation |\n| Control | `scroll`, `left_click` | Mouse and scroll interactions |\n| Search | `web_search` | Information retrieval |\n| System | `wait`, `terminate`, `pause_and_memorize_fact` | System-level operations |\n\n资料来源：[src/magentic_ui/guarded_action.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/guarded_action.py)\n\n### 2.3 Action Execution Flow\n\n```mermaid\nsequenceDiagram\n participant Agent\n participant Guard as Approval Guard\n participant User\n participant Executor\n \n Agent->>Guard: Propose Action\n Guard->>Guard: Check Approval Requirement\n alt Requires Approval\n Guard->>User: Display Action for Review\n User->>Guard: Approve/Deny/Modify\n alt Denied\n Guard->>Agent: Action Cancelled\n else Approved\n Guard->>Executor: Execute Action\n end\n else No Approval Required\n Guard->>Executor: Execute Action\n end\n Executor-->>Agent: Action Result\n```\n\n---\n\n## 3. Plans System\n\nThe Plans system enables users to create, save, and reuse task workflows. Plans consist of structured steps that guide agent behavior through complex multi-step tasks.\n\n### 3.1 Plan Structure\n\nA Plan consists of:\n\n| Component | Type | Description |\n|-----------|------|-------------|\n| `id` | string | Unique identifier |\n| `task` | string | High-level task description |\n| `steps` | array | Ordered list of execution steps |\n| `created_at` | datetime | Creation timestamp |\n| `metadata` | object | Additional configuration |\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### 3.2 Plan Execution\n\nWhen a plan is attached to a query:\n\n1. The plan is retrieved from the plan library\n2. Steps are displayed to the user for confirmation\n3. The agent follows the plan's structured workflow\n4. Progress is tracked through the progress bar component\n\n```mermaid\ngraph LR\n A[Create/Load Plan] --> B[Attach to Query]\n B --> C[Display Plan Steps]\n C --> D{User Approval?}\n D -->|Yes| E[Execute Plan]\n D -->|No| F[Cancel]\n E --> G[Track Progress]\n G --> H[Complete Task]\n```\n\n### 3.3 Plan Management\n\nPlans can be managed through the PlanCard component:\n\n- **Creation**: Users can create new plans from scratch\n- **Editing**: Existing plans can be modified through a modal interface\n- **Deletion**: Plans can be removed from the library\n- **Attachment**: Plans can be attached to queries via the chat input\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n---\n\n## 4. Learning System\n\nThe Learning System allows Magentic-UI to learn reusable plans from completed conversations. This enables knowledge transfer between sessions and automation of recurring tasks.\n\n### 4.1 Learn Plan Workflow\n\n```mermaid\ngraph TD\n A[Completed Conversation] --> B[Learn Plan Button]\n B --> C[Extract Task Structure]\n C --> D[Generate Plan Steps]\n D --> E[User Review]\n E --> F[Save to Library]\n F --> G[Available for Reuse]\n```\n\n### 4.2 Learning Process\n\n| Step | Description |\n|------|-------------|\n| 1 | Conversation completes with final answer |\n| 2 | User clicks \"Learn Plan\" button |\n| 3 | System analyzes conversation history |\n| 4 | Extracts reusable workflow patterns |\n| 5 | Presents plan for user confirmation |\n| 6 | Saves plan to user's plan library |\n\n资料来源：[src/magentic_ui/learning/learner.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/learning/learner.py)\n\n### 4.3 Learn Plan Button States\n\nThe LearnPlanButton component has multiple states:\n\n| State | Visual Indicator | Behavior |\n|-------|------------------|----------|\n| Loading | Spinner + \"Learning Plan...\" | Analyzing conversation |\n| Disabled | Opacity 50% | No session or user ID |\n| Ready | Blue button + \"Learn Plan\" | Ready to learn |\n| Dark Mode | Blue-400 text | Light theme variant |\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n---\n\n## 5. Chat Interface Components\n\nThe chat interface serves as the primary interaction point between users and the agent system.\n\n### 5.1 Chat Input Component\n\nThe ChatInput component handles user input and supports multiple input types:\n\n| Feature | Description |\n|---------|-------------|\n| Text Input | Free-form text queries |\n| File Attachment | Upload files via dropdown menu |\n| Plan Attachment | Attach predefined plans |\n| MCP Server Selection | Configure Model Context Protocol servers |\n| Pause Control | Pause active runs |\n\n资料来源：[frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n\n### 5.2 Run View Component\n\nThe RunView component displays the current state of an agent run:\n\n- Status indicators (icons for each run state)\n- Approval buttons for pending actions\n- Detail viewer for visual feedback\n- Input response handling\n\n```mermaid\ngraph TD\n A[User Input] --> B{ChatInput Component}\n B --> C{Run Status}\n C -->|awaiting_input| D[Show Input Request]\n C -->|paused| E[Show Pause Controls]\n C -->|active| F[Show Progress]\n D --> G[Approval Buttons]\n E --> H[Resume/Cancel]\n F --> I[Progress Bar]\n```\n\n资料来源：[frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n\n### 5.3 Progress Bar\n\nThe progress bar visualizes task completion:\n\n| Segment | Color | Description |\n|---------|-------|-------------|\n| Completed | Green (`#22c55e`) | Finished steps |\n| Current | Magenta (`#861657`) | Active step |\n| Remaining | Gray | Pending steps |\n\nWhen a final answer is available, the progress bar displays at 100% with a \"Task Completed\" status indicator.\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/progressbar.tsx)\n\n---\n\n## 6. State Management\n\nThe frontend uses a Zustand-based store for centralized state management.\n\n### 6.1 Store Structure\n\n| State Category | Key Properties |\n|----------------|----------------|\n| Session | `session`, `sessions`, `connectionId` |\n| Messages | `messages`, `setMessages` |\n| Configuration | `version`, `setVersion` |\n| Header | `title`, `breadcrumbs` |\n| Agent Flow | `direction`, `showLabels`, `showGrid`, `showTokens` |\n\n资料来源：[frontend/src/hooks/store.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/store.tsx)\n\n### 6.2 Agent Flow Settings\n\nAgent flow visualization settings:\n\n| Setting | Default | Description |\n|---------|---------|-------------|\n| `direction` | `\"TB\"` | Flow layout direction (TB=Top-Bottom) |\n| `showLabels` | `true` | Display node labels |\n| `showGrid` | `true` | Show background grid |\n| `showTokens` | `true` | Display token counts |\n| `showMessages` | `true` | Show message nodes |\n| `showMiniMap` | `false` | Enable minimap navigation |\n\n---\n\n## 7. MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers for extended agent capabilities.\n\n### 7.1 Server Configuration Types\n\n| Type | Use Case |\n|------|----------|\n| SSE | Server-Sent Events communication |\n| Stdio | Standard I/O subprocess communication |\n| JSON | Raw JSON configuration |\n\n### 7.2 Server Management\n\nServers can be added, updated, and removed through the MCPConfigModal component. Each server configuration includes:\n\n- Server name (alphanumeric, max 50 characters)\n- Connection type selection\n- Type-specific configuration parameters\n- Validation for duplicates and required fields\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n---\n\n## 8. File Handling\n\nThe system supports file uploads and rendering through the file renderer component.\n\n### 8.1 Supported File Operations\n\n| Operation | Description |\n|-----------|-------------|\n| Upload | Attach files to messages |\n| Preview | Display file thumbnails |\n| Download | Retrieve uploaded files |\n| Metadata | Extract and display file information |\n\n### 8.2 File Card Component\n\nFile cards display uploaded files with:\n\n- Icon representation based on file type\n- Truncated filename with full name tooltip\n- Download button for file retrieval\n- Click handler for file preview\n\n资料来源：[frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n---\n\n## 9. Browser Automation\n\nThe web surfer module provides browser automation capabilities through the FARA (Firefox Automation with Remote Access) system.\n\n### 9.1 Supported Browser Actions\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `key` | `keys[]` | Perform key press sequences |\n| `type` | `text`, `press_enter`, `delete_existing_text` | Type text input |\n| `mouse_move` | `coordinate[x,y]` | Move cursor to position |\n| `left_click` | `coordinate[x,y]` | Click at position |\n| `scroll` | `pixels` | Scroll wheel (positive=up, negative=down) |\n| `visit_url` | `url` | Navigate to URL |\n| `web_search` | `query` | Execute web search |\n| `history_back` | - | Navigate browser back |\n| `wait` | `time` (seconds) | Wait for page changes |\n| `terminate` | `status` | End task with status |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### 9.2 Display Configuration\n\n| Parameter | Description |\n|-----------|-------------|\n| `display_width_px` | Browser viewport width |\n| `display_height_px` | Browser viewport height |\n| `include_input_text_key_args` | Enable text input key arguments |\n\n---\n\n## 10. User Interface Layout\n\nThe application layout follows a structured component hierarchy.\n\n### 10.1 Layout Structure\n\n```mermaid\ngraph TD\n A[MagenticUILayout] --> B[ConfigProvider]\n A --> C[SessionManager]\n A --> D[Warning Banner]\n B --> C\n C --> E[Chat/RunView]\n C --> F[DetailViewer]\n E --> G[ChatInput]\n F --> H[Browser Modal]\n F --> I[Fullscreen Overlay]\n```\n\n### 10.2 Theme Support\n\nThe layout supports both light and dark themes through Ant Design's theme configuration system.\n\n### 10.3 Warning Banner\n\nA persistent disclaimer informs users about the system's limitations:\n\n> Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n---\n\n\n\n## High-Level Architecture\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [Team Orchestration](#team-orchestration), [Backend API](#backend-api)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/views/chat/detail_viewer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/detail_viewer.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n \n\n# High-Level Architecture\n\n## Overview\n\nMagentic-UI is a web-based interface that enables users to create, manage, and execute AI-driven task automation workflows. The system combines a React-based frontend with a Python backend to provide an interactive chat interface where users can execute multi-step plans, browse the web autonomously, and manage reusable workflow templates.\n\nThe architecture follows a **client-server model** where:\n- The **frontend** handles UI rendering, user interaction, and real-time display of task execution\n- The **backend** processes AI requests, manages agent execution, and coordinates with external tools\n- Communication occurs via RESTful API endpoints\n\n## System Components\n\n### Frontend Architecture\n\nThe frontend is a React application using TypeScript, organized into a modular component structure. It communicates with the backend API at `http://localhost:8081/api` as specified in the environment configuration.\n\n资料来源：[frontend/README.md:1-7]()\n\n```mermaid\ngraph TD\n A[User Browser] --> B[React Frontend]\n B --> C[Components]\n C --> D[Views/Chat]\n C --> E[Features]\n E --> F[Plans]\n E --> G[McpServersConfig]\n C --> H[Layout]\n B --> I[Backend API
localhost:8081/api]\n I --> J[Python Backend]\n J --> K[AI Agents]\n J --> L[Team Manager]\n```\n\n### Core Frontend Components\n\n| Component | File Path | Purpose |\n|-----------|-----------|---------|\n| `Chat` | `frontend/src/components/views/chat/chat.tsx` | Main chat interface container |\n| `RunView` | `frontend/src/components/views/chat/runview.tsx` | Manages run status and detail viewer |\n| `RenderMessage` | `frontend/src/components/views/chat/rendermessage.tsx` | Renders different message types |\n| `ChatInput` | `frontend/src/components/views/chat/chatinput.tsx` | User input with file/plan attachment |\n| `DetailViewer` | `frontend/src/components/views/chat/detail_viewer.tsx` | Browser view, screenshots, live tabs |\n| `ProgressBar` | `frontend/src/components/views/chat/progressbar.tsx` | Task progress visualization |\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:1-50]()\n\n## Frontend View Architecture\n\n### Chat System\n\nThe chat system is the primary user interaction point. It manages:\n- Display of conversation messages\n- Real-time run status updates\n- Progress tracking for multi-step tasks\n- Detail viewer integration for visual feedback\n\n```mermaid\ngraph LR\n A[ChatInput] -->|User Input| B[Chat Container]\n B --> C[RenderMessage]\n C -->|Multi-modal| D[PlanView]\n C -->|File| E[FileRenderer]\n B -->|Active Run| F[RunView]\n F --> G[DetailViewer]\n G --> H[Screenshots]\n G --> I[Live View]\n G --> J[BrowserModal]\n```\n\nThe chat component handles multiple run states:\n- `active` - Task is currently executing\n- `awaiting_input` - Waiting for user response\n- `paused` - Task temporarily paused\n- `pausing` - Pause operation in progress\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:30-40]()\n\n### Plan Management System\n\nPlans are reusable workflow templates that define multi-step task sequences. The plan system consists of:\n\n| Component | Function |\n|-----------|----------|\n| `PlanList` | Displays all saved plans with search/filter |\n| `PlanCard` | Individual plan summary with quick actions |\n| `PlanView` | Detailed plan editing and viewing |\n| `LearnPlanButton` | Creates new plans from conversation history |\n\n```mermaid\ngraph TD\n A[User Conversation] --> B[LearnPlanButton]\n B --> C[Plan Created]\n C --> D[PlanList]\n D --> E[PlanCard]\n E --> F[PlanView
Edit/View]\n F --> G[Save Plan]\n G --> D\n A --> H[Attach Plan to Chat]\n H --> I[Execute Plan]\n```\n\nPlans can be attached to chat queries using the dropdown menu in `ChatInput`, allowing users to:\n1. Create new empty plans\n2. Import plans from JSON files\n3. Search through existing plans\n4. Execute attached plans with the current query\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-80]()\n资料来源：[frontend/src/components/features/Plans/PlanList.tsx:1-50]()\n\n### Detail Viewer System\n\nThe detail viewer provides visual feedback during task execution through multiple tabs:\n\n| Tab | Purpose |\n|-----|---------|\n| Screenshots | Static screenshots captured during execution |\n| Live | Real-time browser view via noVNC |\n| Browser Modal | Full browser view in modal overlay |\n\nThe system supports control handover, allowing users to take control during autonomous browsing:\n\n```mermaid\ngraph TD\n A[Agent Execution] --> B[DetailViewer]\n B --> C[ScreenshotsTab]\n B --> D[LiveTab]\n D --> E[noVNC Connection]\n E --> F[User Control
Handover]\n F --> G[FullscreenOverlay]\n G --> H[User Input Response]\n H --> A\n```\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx:1-100]()\n资料来源：[frontend/src/components/views/chat/runview.tsx:1-50]()\n\n## MCP Server Configuration\n\nMagentic-UI supports the Model Context Protocol (MCP) for extending functionality through external servers. The configuration modal supports three connection types:\n\n| Connection Type | Description |\n|-----------------|-------------|\n| SSE | Server-Sent Events for streaming responses |\n| Stdio | Standard input/output process communication |\n| JSON Config | Direct JSON configuration import |\n\n```typescript\ninterface MCPConfig {\n serverName: string; // Unique identifier\n connectionType: 'sse' | 'stdio' | 'json';\n // Additional config based on type...\n}\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-100]()\n\n## Progress Tracking System\n\nThe progress bar component provides real-time feedback on task execution:\n\n```mermaid\ngraph TD\n A[Progress Update] --> B{Has Final Answer?}\n B -->|Yes| C[100% Complete
Green Bar]\n B -->|No| D[Calculate Percentage]\n D --> E[Current Step Highlight
Magenta]\n E --> F[Remaining Steps
Gray]\n C --> G[Status: Task Completed]\n F --> H[Status: Step X of Y]\n```\n\nThe progress system tracks:\n- `currentStep` - Current execution step index\n- `totalSteps` - Total number of steps in the plan\n- `plan.steps` - Array of step definitions with titles\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx:1-80]()\n\n## Message Rendering System\n\nMessages are rendered based on their content type:\n\n```mermaid\ngraph TD\n A[Raw Message] --> B{Parse Content}\n B --> C{Multi-Modal?}\n C -->|Yes| D[Map Each Item]\n C -->|No| E[Render as Text]\n D --> F{Is String?}\n F -->|Yes| G[Parse & Display]\n F -->|No| H[Skip/Empty]\n G --> I{Has Plan?}\n I -->|Yes| J[PlanView Component]\n J --> K[Final Output]\n H --> K\n E --> K\n```\n\nSupported content types:\n- Plain text with markdown rendering\n- Multi-step plans\n- File attachments with download capability\n- Code blocks\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx:1-100]()\n\n## Web Surfer Agent\n\nThe web surfer agent enables autonomous web browsing. It uses a structured action system:\n\n| Action | Purpose | Required Parameters |\n|--------|---------|---------------------|\n| `visit_url` | Navigate to URL | `url` |\n| `web_search` | Search the web | `query` |\n| `scroll` | Scroll page | `pixels` |\n| `click` | Click element | `coordinate` |\n| `type` | Input text | `text`, `coordinate` |\n| `pause_and_memorize_fact` | Store information | `fact` |\n| `wait` | Wait for page | `time` |\n| `terminate` | End task | `status` |\n\n```python\nparameters = {\n \"action\": {\n \"type\": \"string\",\n \"enum\": [\"visit_url\", \"web_search\", \"scroll\", \"click\", ...]\n },\n \"coordinate\": {\n \"description\": \"(x, y): The x and y coordinates for mouse actions\",\n \"type\": \"array\"\n }\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-80]()\n\n## Data Flow\n\n### User Query Flow\n\n```mermaid\nsequenceDiagram\n participant User\n participant ChatInput\n participant Backend\n participant Agent\n participant DetailViewer\n \n User->>ChatInput: Submit query\n ChatInput->>Backend: POST /api/run\n Backend->>Agent: Execute task\n Agent->>DetailViewer: Stream screenshots\n Agent->>Backend: Progress updates\n Backend->>ChatInput: Status updates\n Agent->>Backend: Final response\n Backend->>User: Display result\n```\n\n### Plan Attachment Flow\n\n```mermaid\ngraph TD\n A[User clicks attach] --> B[Dropdown shows plans]\n B --> C[Select plan]\n C --> D[PlanView Modal opens]\n D --> E[Confirm attachment]\n E --> F[Plan attached to input]\n F --> G[Submit with plan context]\n```\n\n## State Management\n\nThe frontend manages several key state objects:\n\n```typescript\ninterface ChatState {\n currentRun: Run | null;\n runStatus: 'idle' | 'active' | 'paused' | 'awaiting_input';\n progress: {\n currentStep: number;\n totalSteps: number;\n plan?: Plan;\n };\n hasFinalAnswer: boolean;\n}\n\ninterface PlanState {\n task: string;\n steps: Step[];\n created_at?: string;\n}\n```\n\n## Security Considerations\n\nThe layout includes a disclaimer for user awareness:\n\n> Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\n\nControl handover features allow users to take control from autonomous agents during execution, ensuring human oversight of automated tasks.\n\n资料来源：[frontend/src/components/layout.tsx:1-50]()\n\n## Configuration\n\n### Environment Variables\n\n| Variable | Default | Purpose |\n|----------|---------|---------|\n| `GATSBY_API_URL` | `http://localhost:8081/api` | Backend API endpoint |\n| API requests target | `http://localhost:8081/api` | Frontend-backend communication |\n\n### Theme Support\n\nThe application supports light and dark themes using Ant Design's theme algorithm system, dynamically switching between `darkAlgorithm` and `defaultAlgorithm` based on user preference.\n\n资料来源：[frontend/README.md:1-10]()\n资料来源：[frontend/src/components/layout.tsx:1-30]()\n\n---\n\n\n\n## Agent System\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Team Orchestration](#team-orchestration), [Browser Automation](#browser-automation)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n \n\n# Agent System\n\n## Overview\n\nThe Agent System in Magentic-UI is a multi-agent orchestration framework that enables autonomous task execution through specialized agents. The system coordinates various agent types including web surfers, file surfers, coders, and user proxies to accomplish complex tasks requested by users.\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n User[User Request] --> Orchestrator[Orchestrator Agent]\n Orchestrator --> WebSurfer[Web Surfer Agent]\n Orchestrator --> FileSurfer[File Surfer Agent]\n Orchestrator --> Coder[Coder Agent]\n Orchestrator --> UserProxy[User Proxy Agent]\n \n WebSurfer --> Browser[Browser Control]\n FileSurfer --> FileSystem[File System]\n Coder --> CodeExecution[Code Executor]\n UserProxy --> UserApproval[User Approval]\n \n Browser --> StateUpdate[State Update]\n FileSystem --> StateUpdate\n CodeExecution --> StateUpdate\n UserApproval --> StateUpdate\n \n StateUpdate --> Orchestrator\n```\n\n## Agent Types\n\n### Web Surfer Agent\n\nThe Web Surfer Agent enables autonomous web browsing by controlling a browser instance. It handles various browsing actions including navigation, interaction, and information retrieval.\n\n#### Supported Actions\n\n| Action | Description | Required Parameters |\n|--------|-------------|---------------------|\n| `visit_url` | Navigate to a specific URL | `url` |\n| `web_search` | Perform a web search | `query` |\n| `left_click` | Click at coordinates | `coordinate` |\n| `right_click` | Right-click at coordinates | `coordinate` |\n| `mouse_move` | Move mouse to coordinates | `coordinate` |\n| `type` | Type text into an element | `text`, `coordinate` |\n| `scroll` | Scroll the page | `pixels` |\n| `key` | Press keyboard keys | `keys` |\n| `pause_and_memorize_fact` | Store information for later use | `fact` |\n| `wait` | Wait for specified duration | `time` |\n| `history_back` | Navigate back in browser history | - |\n| `terminate` | End the browsing session | `status` |\n\n#### Configuration Parameters\n\nThe Web Surfer Agent supports the following configuration options:\n\n```python\n{\n \"display_width_px\": int, # Browser viewport width\n \"display_height_px\": int, # Browser viewport height\n \"include_input_text_key_args\": bool # Include type-specific arguments\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-80]()\n\n### File Surfer Agent\n\nThe File Surfer Agent provides file system navigation and file content interaction capabilities. It allows agents to read, write, and manage files within the project workspace.\n\n### Coder Agent\n\nThe Coder Agent handles code generation, analysis, and execution tasks. It works in conjunction with the orchestrator to implement requested functionality.\n\n### User Proxy Agent\n\nThe User Proxy Agent acts as an intermediary between the autonomous agent system and human users. It handles:\n\n- Requesting user confirmation for sensitive operations\n- Presenting information that requires human judgment\n- Managing user input during interactive sessions\n\n## Agent Communication Flow\n\n```mermaid\nsequenceDiagram\n participant User\n participant Frontend\n participant Orchestrator\n participant Agent\n \n User->>Frontend: Submit Task\n Frontend->>Orchestrator: Send Request\n Orchestrator->>Agent: Delegate Subtask\n Agent->>Agent: Execute Action\n Agent-->>Orchestrator: Return Result\n \n alt Requires Approval\n Orchestrator->>User: Request Approval\n User-->>Orchestrator: Approval/Denial\n end\n \n Orchestrator-->>Frontend: Final Response\n Frontend-->>User: Display Result\n```\n\n## Task Execution Workflow\n\nWhen a user submits a task through the chat interface, the system follows this execution model:\n\n1. **Task Submission**: User enters a query via `ChatInput` component\n2. **Agent Selection**: The orchestrator determines which agent(s) to invoke\n3. **Execution**: Selected agents perform their designated actions\n4. **Progress Tracking**: The system displays execution progress via `ProgressBar`\n5. **State Updates**: Real-time updates are rendered via `RenderMessage`\n6. **Completion**: Final results are presented with option to save plans\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:50-120]()\n\n## Plan System Integration\n\nThe Agent System integrates with a Plan System that breaks down complex tasks into executable steps:\n\n```mermaid\ngraph LR\n Task[User Task] --> Plan[Generated Plan]\n Plan --> Step1[Step 1]\n Plan --> Step2[Step 2]\n Plan --> Step3[Step 3]\n \n Step1 --> Execute1[Execute]\n Step2 --> Execute2[Execute]\n Step3 --> Execute3[Execute]\n \n Execute1 --> Result1[Result]\n Execute2 --> Result2[Result]\n Execute3 --> Result3[Result]\n \n Result1 --> Aggregate[Aggregate Results]\n Result2 --> Aggregate\n Result3 --> Aggregate\n```\n\n### Plan Components\n\n| Component | File | Purpose |\n|-----------|------|---------|\n| `PlanCard` | `PlanCard.tsx` | Displays individual plan summary |\n| `PlanList` | `PlanList.tsx` | Lists all available plans |\n| `PlanView` | `PlanView.tsx` | Interactive plan editor/viewer |\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx:1-100]()\n\n## Message Rendering System\n\nThe Agent System communicates results through a structured message rendering system:\n\n```mermaid\ngraph TD\n Message[Agent Message] --> Parse[Parse Content]\n Parse --> Type{Message Type}\n \n Type -->|Text| TextRender[Text Renderer]\n Type -->|Plan| PlanRender[Plan View]\n Type -->|File| FileRender[File Renderer]\n Type -->|Image| ImageRender[Image Renderer]\n \n TextRender --> Display[UI Display]\n PlanRender --> Display\n FileRender --> Display\n ImageRender --> Display\n```\n\nThe `RenderMessage` component handles the display of agent outputs, supporting:\n\n- Multi-modal content rendering\n- Plan visualization\n- File previews and downloads\n- Image galleries\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx:1-100]()\n\n## Browser Control Details\n\nThe Web Surfer Agent uses coordinate-based browser control:\n\n### Coordinate System\n\n- **X-axis**: Pixels from the left edge of the viewport\n- **Y-axis**: Pixels from the top edge of the viewport\n- **Scroll**: Positive values scroll up, negative values scroll down\n\n### Type Action Parameters\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `text` | string | Text to type |\n| `coordinate` | [x, y] | Target element position |\n| `press_enter` | boolean | Submit after typing |\n| `delete_existing_text` | boolean | Clear before typing |\n\n## Run Status States\n\nThe agent execution maintains the following status states:\n\n| Status | Description |\n|--------|-------------|\n| `active` | Agent is currently executing |\n| `paused` | Execution paused, awaiting resume |\n| `pausing` | Pause is in progress |\n| `awaiting_input` | Waiting for user input or approval |\n| `completed` | Task finished successfully |\n| `failed` | Task execution failed |\n\n资料来源：[frontend/src/components/views/chat/chat.tsx:30-60]()\n\n## Configuration Management\n\nThe Agent System configuration is managed through the frontend store:\n\n```typescript\ninterface IAgentFlowSettings {\n direction: \"TB\" | \"LR\"; // Flow chart orientation\n showLabels: boolean; // Display edge labels\n showGrid: boolean; // Show background grid\n showTokens: boolean; // Display token counts\n showMessages: boolean; // Show message nodes\n showMiniMap: boolean; // Show navigation minimap\n}\n```\n\n资料来源：[frontend/src/hooks/store.tsx:1-80]()\n\n## MCP Server Integration\n\nThe system supports Model Context Protocol (MCP) servers for extended agent capabilities:\n\n- SSE-based server connections\n- Stdio-based server connections\n- JSON configuration import\n\nMCP servers are configured via the `McpConfigModal` component and integrated into the agent selection process during task execution.\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-100]()\n\n## Detail Viewer\n\nThe `DetailViewer` component provides real-time visualization of agent activities:\n\n- **Screenshots Tab**: Periodic screenshots of browser state\n- **Live Tab**: Real-time browser view via noVNC\n- **Control Mode**: Fullscreen overlay for manual intervention\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx:1-100]()\n[frontend/src/components/views/chat/runview.tsx:1-80]()\n\n## Summary\n\nThe Agent System in Magentic-UI provides a comprehensive framework for autonomous task execution through:\n\n- **Specialized Agents**: Web Surfer, File Surfer, Coder, User Proxy\n- **Orchestration Layer**: Coordinates multi-agent collaboration\n- **Plan System**: Breaks tasks into executable steps\n- **Real-time Visualization**: Browser screenshots and live view\n- **Human-in-the-Loop**: User proxy for approval and input\n- **MCP Integration**: Extensible server architecture\n\nThis architecture enables complex, multi-step task automation while maintaining user oversight and control throughout the execution process.\n\n---\n\n\n\n## Team Orchestration\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [High-Level Architecture](#high-level-architecture)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n- [src/magentic_ui/agents/_coder.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/_coder.py)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n \n\n# Team Orchestration\n\n## Overview\n\nTeam Orchestration is a core system in Magentic-UI that coordinates multiple AI agents to work together on complex tasks. It provides the infrastructure for orchestrating agent teams, managing communication between agents, and handling task distribution and execution flow.\n\nThe orchestration system enables Magentic-UI to:\n\n- Coordinate multiple specialized agents (coders, web surfers, planners)\n- Manage agent collaboration through structured message passing\n- Handle approval policies for sensitive actions\n- Support dynamic task execution with planning and reflection capabilities\n\n## Architecture\n\n### Core Components\n\n| Component | File Path | Purpose |\n|-----------|-----------|---------|\n| Orchestrator | `src/magentic_ui/teams/orchestrator/_orchestrator.py` | Central coordinator for agent teams |\n| Group Chat | `src/magentic_ui/teams/orchestrator/_group_chat.py` | Manages multi-agent message passing |\n| Prompts | `src/magentic_ui/teams/orchestrator/_prompts.py` | Prompt templates for orchestration |\n| Sentinel Prompts | `src/magentic_ui/teams/orchestrator/_sentinel_prompts.py` | Safety and monitoring prompts |\n\n### Agent Types\n\nMagentic-UI supports several specialized agent types that can be orchestrated:\n\n| Agent Type | Purpose | Key Actions |\n|------------|---------|-------------|\n| Coder Agent | Execute Python/code tasks | Write, debug, execute code |\n| Web Surfer Agent | Browse and interact with web content | scroll, visit_url, web_search, wait |\n| Planner Agent | Create and manage execution plans | Task decomposition, step planning |\n| MCP Server Agents | External tool integrations | Configurable via SSE/stdio protocols |\n\n## Web Surfer Agent Actions\n\nThe web surfer agent supports a comprehensive set of actions for web interaction:\n\n```python\nparameters = {\n \"properties\": {\n \"scroll\": {\n \"description\": \"The number of pixels to scroll. Positive scrolls down, negative scrolls up.\",\n \"type\": \"number\",\n },\n \"url\": {\n \"description\": \"The URL to visit. Required only by `action=visit_url`.\",\n \"type\": \"string\",\n },\n \"query\": {\n \"description\": \"The query to search for. Required only by `action=web_search`.\",\n \"type\": \"string\",\n },\n \"fact\": {\n \"description\": \"The fact to remember for the future. Required only by `action=pause_and_memorize_fact`.\",\n \"type\": \"string\",\n },\n \"time\": {\n \"description\": \"The seconds to wait. Required only by `action=wait`.\",\n \"type\": \"number\",\n },\n \"status\": {\n \"description\": \"The status of the task. Required only by `action=terminate`.\",\n \"type\": \"string\",\n \"enum\": [\"success\", \"failure\"],\n },\n },\n \"required\": [\"action\"],\n \"type\": \"object\",\n}\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py:1-50](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n### Action Types\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `scroll` | `scroll` (pixels) | Scrolls the viewport |\n| `visit_url` | `url` | Navigate to a URL |\n| `web_search` | `query` | Search the web |\n| `pause_and_memorize_fact` | `fact` | Store information for context |\n| `wait` | `time` (seconds) | Wait before continuing |\n| `terminate` | `status` (success/failure) | End the task |\n\n## Function Call Handling\n\nThe system uses a specialized function call prompt system for agent communication:\n\n```python\ntool_descs = [{\"type\": \"function\", \"function\": f} for f in functions]\ntool_names = [\n function.get(\"name_for_model\", function.get(\"name\", \"\"))\n for function in functions\n]\ntool_descs = \"\\n\".join([json.dumps(f, ensure_ascii=False) for f in tool_descs])\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py:1-60](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/qwen_helpers/fncall_prompt.py)\n\n### Message Processing Flow\n\n```\ngraph TD\n A[User Input] --> B[Orchestrator]\n B --> C{Agent Selection}\n C -->|Planning| D[Planner Agent]\n C -->|Execution| E[Coder Agent]\n C -->|Web Tasks| F[Web Surfer Agent]\n D --> G[Execution Plan]\n E --> H[Code Execution]\n F --> I[Web Actions]\n G --> B\n H --> B\n I --> B\n B --> J[User Response]\n```\n\n### Role-Based Message Handling\n\n| Role | Processing | Content Format |\n|------|------------|----------------|\n| `ASSISTANT` | Appends tool calls to last message | `` XML blocks |\n| `FUNCTION` | Processes tool responses | `` XML blocks |\n| `USER` | Standard user messages | Plain text or structured |\n\n## MCP Server Integration\n\nMagentic-UI supports Model Context Protocol (MCP) servers for extended functionality:\n\n```typescript\ninterface McpServerConfig {\n serverName: string;\n agentName: string;\n agentDescription: string;\n connectionType: 'sse' | 'stdio' | 'json';\n}\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n### MCP Configuration Modes\n\n| Mode | Description | Use Case |\n|------|-------------|----------|\n| SSE | Server-Sent Events | Remote server connections |\n| Stdio | Standard I/O | Local process communication |\n| JSON Config | Raw JSON configuration | Advanced users |\n\n### Agent Description Requirements\n\nEach MCP server requires a description that helps the orchestrator decide when to invoke it:\n\n> \"Describe how and when this server should be used. This helps the orchestrator decide when to call this agent.\"\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx:1-120](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## Code Execution Flow\n\nThe coder agent provides secure code execution with output capture:\n\n```python\nasync def _summarize_coding(\n agent_name: str,\n model_client: ChatCompletionClient,\n thread: Sequence[BaseChatMessage | BaseAgentEvent],\n cancellation_token: CancellationToken,\n model_context: ChatCompletionContext,\n) -> TextMessage:\n```\n\n资料来源：[src/magentic_ui/agents/_coder.py:1-100](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/_coder.py)\n\n### Code Execution States\n\n| State | Description | Exit Code |\n|-------|-------------|-----------|\n| Success | Code executed without errors | `0` |\n| Timeout | Execution exceeded time limit | N/A |\n| Error | Runtime exception occurred | Non-zero |\n\n```python\n# Break if all code executions were successful\nif all([code_output == 0 for code_output in exit_code_list]):\n break\n```\n\n## CLI Configuration\n\nThe orchestration system is configured through the CLI entry point:\n\n```python\ndef main() -> None:\n \"\"\"\n Entry point for the magentic-cli command.\n Called from pyproject.toml's [project.scripts] section.\n \"\"\"\n app()\n```\n\n资料来源：[src/magentic_ui/_cli.py:1-50](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n\n### CLI Parameters\n\n| Parameter | Type | Purpose |\n|-----------|------|---------|\n| `mcp_agents` | List | External MCP server agents |\n| `run_without_docker` | bool | Run without container isolation |\n| `browser_headless` | bool | Run browser in headless mode |\n| `browser_local` | bool | Use local browser instead of remote |\n| `sentinel_tasks` | List | Background monitoring tasks |\n| `dynamic_sentinel_sleep` | int | Sleep interval for sentinel checks |\n\n## Approval Policies\n\nThe orchestration system supports configurable approval policies for controlling agent actions:\n\n| Policy | Behavior |\n|--------|----------|\n| `Auto Approve` | All actions execute automatically |\n| `Manual Approval` | User must approve each action |\n| `Policy Based` | Rules determine approval based on action type |\n\n## Error Handling\n\n### Timeout Handling\n\n```python\nexcept asyncio.TimeoutError:\n executor_msg = TextMessage(\n source=agent_name + \"-executor\",\n metadata={\"internal\": \"yes\"},\n content=\"Code execution timed out.\",\n )\n delta.append(executor_msg)\n yield executor_msg\n```\n\n## Summary\n\nTeam Orchestration in Magentic-UI provides a flexible framework for coordinating multiple AI agents. Key features include:\n\n1. **Multi-Agent Coordination**: Specialized agents work together through the orchestrator\n2. **Flexible Communication**: Role-based message passing with XML-formatted tool calls\n3. **MCP Integration**: Extensible architecture through Model Context Protocol servers\n4. **Safe Execution**: Code execution with timeout handling and error capture\n5. **Approval Workflows**: Configurable policies for sensitive operations\n\nThe system is designed to be modular, allowing new agent types and capabilities to be added through well-defined interfaces.\n\n---\n\n\n\n## Backend API\n\n### 相关页面\n\n相关主题：[High-Level Architecture](#high-level-architecture), [Frontend UI](#frontend-ui)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n \n\n# Backend API\n\n## Overview\n\nThe Magentic-UI Backend API is a FastAPI-based REST/WebSocket service that orchestrates multi-agent workflows, manages conversation sessions, and provides real-time execution capabilities for the frontend interface. It serves as the central hub for all backend operations including session management, run execution, agent coordination, and MCP (Model Context Protocol) integration.\n\nThe API is accessible at `http://localhost:8081/api` for local development and expects all frontend requests to be directed to this endpoint. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## Architecture\n\n### High-Level Architecture\n\n```mermaid\ngraph TB\n subgraph \"Frontend Client\"\n UI[UI Components]\n end\n \n subgraph \"Backend API - FastAPI\"\n App[Main Application]\n Routers[Route Handlers]\n Managers[Connection Managers]\n end\n \n subgraph \"Data Layer\"\n DB[(Database)]\n StaticFiles[Static Files]\n end\n \n UI --> |HTTP/WS| App\n App --> Routers\n Routers --> Managers\n Managers --> DB\n App --> StaticFiles\n```\n\n### API Router Structure\n\nThe backend organizes its functionality into modular routers, each handling a specific domain:\n\n| Router | Prefix | Purpose |\n|--------|--------|---------|\n| Teams Router | `/teams` | Multi-agent team coordination |\n| WebSocket Router | `/ws` | Real-time bidirectional communication |\n| Validation Router | `/validate` | Input validation endpoints |\n| Settings Router | `/settings` | User and system configuration |\n| MCP Router | `/mcp` | Model Context Protocol integration |\n| Sessions Router | `/sessions` | Conversation session management |\n| Runs Router | `/runs` | Execution run tracking and control |\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## Core Endpoints\n\n### Health Check\n\n```\nGET /api/health\n```\n\nReturns the health status of the API service.\n\n**Response:**\n```json\n{\n \"status\": true,\n \"message\": \"Service is healthy\"\n}\n```\n\n### Version Information\n\n```\nGET /api/version\n```\n\nRetrieves the current API version.\n\n**Response:**\n```json\n{\n \"status\": true,\n \"message\": \"Version retrieved successfully\",\n \"data\": {\n \"version\": \"\"\n }\n}\n```\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## API Configuration\n\n### Environment Variables\n\nThe frontend must be configured with the correct API URL. Create a `.env.development` file based on `.env.default`:\n\n```bash\ncp .env.default .env.development\n```\n\nThe primary configuration variable is `GATSBY_API_URL` which should be set to `http://localhost:8081/api` for local development. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Static File Serving\n\nThe backend mounts two static file directories:\n\n| Mount Path | Directory | Purpose |\n|------------|-----------|---------|\n| `/files` | `static_root` | File downloads with HTML fallback |\n| `/` | `ui_root` | Frontend UI assets |\n\n```python\napp.mount(\n \"/files\",\n StaticFiles(directory=initializer.static_root, html=True),\n name=\"files\",\n)\napp.mount(\"/\", StaticFiles(directory=initializer.ui_root, html=True), name=\"ui\")\n```\n\n资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## Error Handling\n\n### Internal Server Error Handler\n\nThe API includes a global exception handler for 500 errors:\n\n```python\n@app.exception_handler(500)\nasync def internal_error_handler(request: Request, exc: Exception):\n logger.error(f\"Internal error: {str(exc)}\")\n return {\n \"status\": False,\n \"message\": \"Internal server error\",\n \"detail\": str(exc) if settings.debug else None\n }\n```\n\nThis handler:\n- Logs the full error details server-side\n- Returns sanitized error messages to clients (hiding details in production)\n- Uses `settings.debug` to control error visibility 资料来源：[src/magentic_ui/backend/web/app.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/app.py)\n\n## WebSocket Communication\n\nThe WebSocket router (`/ws`) enables real-time bidirectional communication between the frontend and backend. This is essential for:\n\n- Live agent execution progress updates\n- Streaming intermediate results\n- Real-time user input responses during agent runs\n- Session state synchronization\n\n## Agent Actions and Tool Integration\n\nThe backend exposes agent capabilities through structured action parameters. Agents support the following action types:\n\n| Action | Description | Required Parameters |\n|--------|-------------|---------------------|\n| `key` | Perform keyboard key presses | `keys` (array) |\n| `type` | Type text into input fields | `text`, `press_enter`, `delete_existing_text` |\n| `mouse_move` | Move cursor to coordinates | `coordinate` [x, y] |\n| `left_click` | Click at coordinates | `coordinate` [x, y] |\n| `scroll` | Scroll mouse wheel | `pixels` |\n| `visit_url` | Navigate to URL | `url` |\n| `web_search` | Execute web search | `query` |\n| `history_back` | Go to previous page | - |\n| `pause_and_memorize_fact` | Store information | `fact` |\n| `wait` | Pause execution | `time` (seconds) |\n| `terminate` | End task | `status` (success/failure) |\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n## Running the Backend\n\n### Prerequisites\n\nEnsure all prerequisites are installed before running the backend. The system requires:\n\n- Python environment with dependencies installed\n- Node.js and npm for frontend development (if building from source)\n- nvm for Node version management\n\n### Starting the Server\n\n```bash\nmagentic-ui --port 8081\n```\n\nThe server will:\n1. Initialize the FastAPI application\n2. Mount all routers under `/api` prefix\n3. Establish database connections\n4. Start listening on the specified port\n\n### Development Mode\n\nFor frontend development with hot-reloading:\n\n1. Start frontend in development mode:\n```bash\ncd frontend\nnpm run start\n```\n\n2. Run the backend:\n```bash\nmagentic-ui --port 8081\n```\n\nThe frontend development server runs at `http://localhost:8000`, while the compiled frontend is available at `http://localhost:8081`. 资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n## API Response Format\n\nAll API responses follow a consistent format:\n\n```json\n{\n \"status\": true | false,\n \"message\": \"Human-readable status message\",\n \"data\": { ... } | null,\n \"detail\": \"Error details (optional, debug mode only)\"\n}\n```\n\nThis standardization allows the frontend to handle all responses uniformly regardless of which router handled the request.\n\n## Related Documentation\n\nFor troubleshooting and setup issues, refer to the [TROUBLESHOOTING.md](TROUBLESHOOTING.md) file in the repository root.\n\n---\n\n\n\n## Frontend UI\n\n### 相关页面\n\n相关主题：[Backend API](#backend-api), [High-Level Architecture](#high-level-architecture)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n- [frontend/src/components/views/manager.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/manager.tsx)\n- [frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n- [frontend/src/components/features/McpServersConfig/McpServerCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpServerCard.tsx)\n- [frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n- [frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n- [frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n- [frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n- [frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n- [frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n- [frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n \n\n# Frontend UI\n\n## Overview\n\nThe Frontend UI of magentic-ui is a React-based web application that provides an interactive interface for users to interact with AI agents. The frontend communicates with the backend API at `http://localhost:8081/api` and enables features such as chat conversations, plan management, MCP server configuration, and real-time task execution visualization.\n\nThe UI is built using:\n- **React** with TypeScript for component architecture\n- **Ant Design** as the primary UI component library\n- **Tailwind CSS** for custom styling\n- **React Markdown** for rendering markdown content\n\n资料来源：[frontend/package.json](https://github.com/microsoft/magentic-ui/blob/main/frontend/package.json)\n\n## Architecture Overview\n\nThe frontend application follows a component-based architecture with clear separation between layout, views, features, and common components.\n\n```mermaid\ngraph TD\n A[App Entry] --> B[MagenticUILayout]\n B --> C[SessionManager]\n C --> D[Views]\n C --> E[SubMenus]\n D --> F[ChatView]\n D --> G[RunView]\n D --> H[PlanList]\n E --> I[SessionList]\n E --> J[PlanLibrary]\n F --> K[ChatInput]\n F --> L[RenderMessage]\n F --> M[ProgressBar]\n F --> N[DetailViewer]\n G --> K\n G --> L\n G --> M\n G --> N\n```\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n## Application Layout\n\n### MagenticUILayout\n\nThe main layout wrapper component that provides theme configuration and session management context to all child components.\n\n| Prop | Type | Description |\n|------|------|-------------|\n| restricted | boolean | Whether to restrict access to authenticated users only |\n| children | ReactNode | Child components to render within the layout |\n\nKey responsibilities:\n- Applies theme algorithms (dark/light mode) via Ant Design's `ConfigProvider`\n- Wraps content in `AppContext` for global state access\n- Displays a disclaimer footer: \"Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.\"\n\n资料来源：[frontend/src/components/layout.tsx:1-100](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n### SessionManager\n\nThe central orchestrator component that manages the overall application state including sessions, plans, and navigation between different views.\n\n```mermaid\ngraph LR\n A[SessionManager] --> B[PlanList]\n A --> C[ChatView]\n A --> D[SessionEditor]\n B --> E[PlanCard]\n C --> F[ChatInput]\n C --> G[MessageList]\n C --> H[RunView]\n```\n\nState management includes:\n- `activeSubMenuItem`: Current navigation state\n- `sessions`: List of user sessions\n- `currentRun`: Active task execution state\n- `selectedMcpServers`: Selected MCP server configurations\n- `editingSession`: Session being edited\n\n资料来源：[frontend/src/components/views/manager.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/manager.tsx)\n\n## Chat System\n\n### ChatView\n\nThe primary chat interface where users interact with AI agents through messages and file uploads.\n\n```mermaid\nsequenceDiagram\n User->>ChatInput: Enter message\n ChatInput->>ChatView: handleSubmit(query, files, plan)\n ChatView->>Backend: runTask() via API\n Backend-->>ChatView: CurrentRun status\n ChatView->>RunView: Pass run data\n RunView->>MessageList: Display messages\n MessageList->>RenderMessage: Render each message\n```\n\n**Key Features:**\n- Message submission with text, files, and attached plans\n- Real-time run status display (running, paused, awaiting_input, completed)\n- MCP server selection for task execution\n- Plan execution control (approve, deny, pause, cancel)\n- Sample tasks for quick start\n\n资料来源：[frontend/src/components/views/chat/chat.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chat.tsx)\n\n### ChatInput\n\nA rich text input component supporting multi-line input, file attachments, and plan attachments.\n\n**Props:**\n| Prop | Type | Description |\n|------|------|-------------|\n| onSubmit | Function | Callback when message is submitted |\n| onCancel | Function | Callback to cancel current operation |\n| runStatus | string | Current run status |\n| inputRequest | object | Request for user input |\n| isPlanMessage | boolean | Whether input is for plan response |\n| onPause | Function | Callback to pause execution |\n| onExecutePlan | Function | Callback to execute a plan |\n| enable_upload | boolean | Enable file uploads |\n| selectedMcpServers | array | Selected MCP servers |\n\n**Features:**\n- File drag-and-drop and paste support\n- File list display with remove capability\n- Plan attachment modal for viewing attached plans\n- Auto-resizing textarea\n- Submit button with loading state\n\n资料来源：[frontend/src/components/views/chat/chatinput.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/chatinput.tsx)\n\n### RenderMessage\n\nComponent responsible for rendering different types of messages including user messages, AI responses, plans, and file previews.\n\n**Rendering Logic:**\n1. Checks if message is from user or assistant\n2. Parses content using `parseContent` utility\n3. Handles multi-modal content (text arrays)\n4. Renders plans via `PlanView` component\n5. Applies appropriate styling based on message type\n\n```typescript\n// Message type detection based on metadata\nif (message?.metadata?.type === \"file\" && message?.metadata?.files) {\n // File message handling\n const parsedFiles = JSON.parse(message.metadata.files);\n}\n```\n\n资料来源：[frontend/src/components/views/chat/rendermessage.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/rendermessage.tsx)\n\n### ProgressBar\n\nVisual indicator for task execution progress with step-by-step status display.\n\n**Display States:**\n- **Task Completed**: Green progress bar at 100% with \"Task Completed\" text\n- **In Progress**: Shows current step (e.g., \"Step 2 of 5\") with progress bar\n- **Planning**: Shows \"Planning...\" text when plan is being generated\n\n**Progress Calculation:**\n```javascript\n// Completed section width\nwidth: hasFinalAnswer ? \"100%\" : (currentStep / totalSteps) * 100 + \"%\"\n\n// Current step indicator position\nleft: (currentStep / totalSteps) * 100 + \"%\"\nwidth: (1 / totalSteps) * 100 + \"%\"\n```\n\n资料来源：[frontend/src/components/views/chat/progressbar.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/progressbar.tsx)\n\n### RunView\n\nContainer component that manages the detail viewer and message display during task execution.\n\n**Layout:**\n- Split view with message list on the left\n- Detail viewer on the right (collapsible/expandable)\n- Manages image gallery, VNC preview, and input responses\n\n资料来源：[frontend/src/components/views/chat/runview.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/runview.tsx)\n\n## Plan Management\n\n### PlanList\n\nDisplays the user's saved plans library with search, create, import, and management capabilities.\n\n**Features:**\n| Feature | Description |\n|---------|-------------|\n| Create | Create a new empty plan |\n| Import | Import plan from JSON file |\n| Search | Filter plans by name |\n| Export | Download plan as JSON |\n| Delete | Remove plan from library |\n\n**Plan Card Actions:**\n- **Run Plan**: Create new session with plan loaded\n- **Edit Plan**: Modify plan title and steps in modal\n\n资料来源：[frontend/src/components/features/Plans/PlanList.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanList.tsx)\n\n### PlanCard\n\nIndividual plan display card showing plan metadata and quick actions.\n\n**Displayed Information:**\n- Plan title (truncated to 40 characters)\n- Step count summary (showing first 3 steps)\n- Creation timestamp with relative time formatting\n- Hover actions for export and delete\n\n**Modal Editing:**\n- Editable plan title\n- Full plan step editor via `PlanView` component\n- Save and cancel functionality\n\n资料来源：[frontend/src/components/features/Plans/PlanCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/PlanCard.tsx)\n\n### LearnPlanButton\n\nButton component that allows users to extract and save a reusable plan from the current conversation.\n\n**States:**\n| State | Appearance |\n|-------|------------|\n| Disabled | Opacity 50%, cursor not-allowed |\n| Learning | Spinner with \"Learning Plan...\" text |\n| Ready | Normal button with \"Learn Plan\" label |\n\n**Behavior:**\n- Disabled when no `sessionId` or `effectiveUserId`\n- Triggers plan extraction from conversation history\n- Saves extracted plan to user's plan library\n\n资料来源：[frontend/src/components/features/Plans/LearnPlanButton.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/Plans/LearnPlanButton.tsx)\n\n## File Handling\n\n### RenderFile\n\nComponent for displaying and managing file attachments in messages.\n\n**Features:**\n- Detects file type from metadata\n- Renders appropriate preview based on file type\n- Provides download functionality\n- Supports modal view for detailed file inspection\n\n**File Type Detection:**\n```typescript\nif (message?.metadata?.type === \"file\" && message?.metadata?.files) {\n const parsedFiles = JSON.parse(message.metadata.files);\n // Process files to ensure correct type detection\n}\n```\n\n### FileCard\n\nDisplays individual file with icon, name, and download button.\n\n**Interactions:**\n- Click to open file in modal\n- Hover to show download button\n- Drag-and-drop zone support\n\n资料来源：[frontend/src/components/common/filerenderer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/filerenderer.tsx)\n\n## Markdown Rendering\n\n### MarkdownRender\n\nComponent for rendering markdown content with syntax highlighting and GitHub Flavored Markdown support.\n\n**Features:**\n- Syntax highlighting via language detection from file extensions\n- Configurable text truncation\n- Indentation indicator support\n- Dark/light mode compatible styling\n\n**Configuration:**\n| Option | Type | Description |\n|--------|------|-------------|\n| truncate | boolean | Enable content truncation |\n| maxLength | number | Maximum character length |\n| indented | boolean | Show indentation indicator |\n| isFilePreview | boolean | Wrap in code block |\n\n资料来源：[frontend/src/components/common/markdownrender.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/common/markdownrender.tsx)\n\n## MCP Server Configuration\n\n### McpServerCard\n\nCard component for displaying MCP (Model Context Protocol) server configurations.\n\n**Displayed Information:**\n- Server name\n- Agent description (truncated to 2 lines)\n- Availability status\n\n**Actions:**\n| Action | Description |\n|--------|-------------|\n| Edit | Modify server configuration |\n| Remove | Delete server from configuration |\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpServerCard.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpServerCard.tsx)\n\n## Relevant Plans\n\n### RelevantPlans\n\nComponent for displaying plans relevant to the current conversation context.\n\n**Features:**\n- Shows top 3 most relevant plans based on current query\n- Plan attachment to query\n- Play action to attach and run plan\n\n资料来源：[frontend/src/components/views/chat/relevant_plans.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/relevant_plans.tsx)\n\n## State Management\n\nThe frontend uses React Context for global state management:\n\n```mermaid\ngraph TD\n A[AppContext] --> B[User State]\n A --> C[Session State]\n A --> D[Theme State]\n A --> E[Provider State]\n \n B --> F[userId]\n B --> G[username]\n \n C --> H[sessions]\n C --> I[currentRun]\n C --> J[plans]\n \n E --> K[mcpServers]\n E --> L[selectedMcpServers]\n```\n\n### Provider Hook\n\nCustom hooks for accessing and manipulating application state:\n\n| Hook | Purpose |\n|------|---------|\n| useAppContext | Access global app context |\n| useSessions | Manage session list and operations |\n| usePlans | Manage saved plans |\n| useMcpServers | Manage MCP server configurations |\n\n资料来源：[frontend/src/hooks/provider.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/hooks/provider.tsx)\n\n## API Integration\n\nThe frontend communicates with the backend API at `http://localhost:8081/api`.\n\n**Environment Configuration:**\n```bash\n# In .env.development\nGATSBY_API_URL=http://localhost:8081/api\n```\n\n资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### Key API Operations\n\n| Operation | Description |\n|-----------|-------------|\n| runTask | Start a new task execution |\n| handleInputResponse | Respond to input requests |\n| handlePause | Pause current execution |\n| handleCancel | Cancel running task |\n| handleApprove | Approve pending action |\n| handleDeny | Deny pending action |\n| handleAcceptPlan | Accept proposed plan |\n| handleRegeneratePlan | Regenerate plan suggestions |\n\n## Component Hierarchy Summary\n\n```mermaid\ngraph TD\n Root[MagenticUILayout] --> SessionManager\n SessionManager --> Header\n SessionManager --> Sidebar\n SessionManager --> Content\n \n Sidebar --> PlanList\n Sidebar --> SessionList\n \n Content --> ChatView\n Content --> RunView\n \n ChatView --> ChatInput\n ChatView --> MessageList\n ChatView --> RelevantPlans\n \n MessageList --> RenderMessage\n RenderMessage --> PlanView\n RenderMessage --> RenderFile\n RenderMessage --> MarkdownRender\n \n ChatInput --> FileList\n ChatInput --> PlanModal\n \n RunView --> DetailViewer\n RunView --> ProgressBar\n```\n\n## Development Guidelines\n\n### Adding New Routes\n\nTo add a new route (e.g., `/about`):\n1. Create folder `src/pages/about`\n2. Add `index.tsx` file\n3. Follow content style from `src/pages/index.tsx`\n4. Place core logic in `src/components` folder\n\n### Key Dependencies\n\n| Package | Version | Purpose |\n|---------|---------|---------|\n| react | ^18.x | UI framework |\n| antd | ^5.x | Component library |\n| @ant-design/icons | ^5.x | Icon library |\n| react-markdown | ^9.x | Markdown rendering |\n| remark-gfm | ^4.x | GitHub Flavored Markdown |\n\n资料来源：[frontend/package.json](https://github.com/microsoft/magentic-ui/blob/main/frontend/package.json)\n\n---\n\n\n\n## Browser Automation\n\n### 相关页面\n\n相关主题：[Agent System](#agent-system), [Docker Containers](#docker-containers)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/browser/local_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/local_playwright_browser.py)\n- [src/magentic_ui/tools/playwright/playwright_controller.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_controller.py)\n- [src/magentic_ui/tools/playwright/playwright_state.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_state.py)\n- [src/magentic_ui/tools/playwright/_set_of_mark.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/_set_of_mark.py)\n- [src/magentic_ui/agents/web_surfer/_web_surfer.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/_web_surfer.py)\n \n\n# Browser Automation\n\n## Overview\n\nThe Browser Automation system in Magentic-UI provides a comprehensive framework for controlling web browsers through programmatic interactions. Built on top of Playwright, this system enables AI agents to navigate websites, interact with UI elements, extract content, and perform complex browsing tasks autonomously.\n\nThe system supports multiple browser deployment modes including local execution, headless Docker containers, and VNC-enabled Docker containers for visual debugging. This flexibility allows the system to operate in various environments from development machines to cloud deployments.\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n A[WebSurfer Agent] --> B[PlaywrightController]\n B --> C[Browser Implementations]\n C --> D[LocalPlaywrightBrowser]\n C --> E[HeadlessDockerPlaywrightBrowser]\n C --> F[VncDockerPlaywrightBrowser]\n G[PlaywrightState] --> B\n H[SetOfMark] --> B\n I[Playwright API] --> C\n```\n\n## Browser Implementations\n\nThe system implements a base `PlaywrightBrowser` class with three specialized implementations:\n\n### Base Architecture\n\nAll browser implementations inherit from `PlaywrightBrowser` which provides the core interface for browser operations. This design pattern allows for consistent API usage across different deployment scenarios.\n\n### LocalPlaywrightBrowser\n\nThe `LocalPlaywrightBrowser` provides direct browser control on the local machine. This implementation offers:\n\n- Full browser lifecycle management (launch, close, context management)\n- Synchronous and asynchronous operation support\n- Download folder configuration\n- Viewport customization\n- Screenshot capture capabilities\n\n**Key Features:**\n- Direct Playwright API access without containerization overhead\n- Ideal for development and testing environments\n- Supports all Playwright browser types (Chromium, Firefox, WebKit)\n\n资料来源：[src/magentic_ui/tools/playwright/browser/local_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/local_playwright_browser.py)\n\n### HeadlessDockerPlaywrightBrowser\n\nThe `HeadlessDockerPlaywrightBrowser` runs browsers inside headless Docker containers. This approach provides:\n\n- Isolated browser execution environment\n- Consistent behavior across different host systems\n- No visual rendering overhead\n- Enhanced security through containerization\n\n**Docker Integration:**\n- Automatic container image pulling on first use\n- Graceful container lifecycle management\n- Resource-efficient headless operation\n\n资料来源：[src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/headless_docker_playwright_browser.py)\n\n### VncDockerPlaywrightBrowser\n\nThe `VncDockerPlaywrightBrowser` extends headless Docker support with VNC connectivity, enabling:\n\n- Real-time visual browser observation\n- Interactive debugging capabilities\n- NoVNC support for browser-based VNC access\n- Remote control handover to human operators\n\n**Port Configuration:**\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `port` | Main VNC port for container communication | 5900 |\n| `novnc_port` | WebSocket port for noVNC browser access | 6080 |\n\n资料来源：[src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/browser/vnc_docker_playwright_browser.py)\n\n## PlaywrightController\n\nThe `PlaywrightController` serves as the central orchestrator for browser interactions. It abstracts the complexities of browser automation into a clean, agent-friendly interface.\n\n### Core Responsibilities\n\n**Page Navigation and Content Extraction:**\n- Visit URLs with configurable timeouts\n- Extract visible text content from pages\n- Capture full-page or viewport screenshots\n- Analyze DOM structure for interactive elements\n\n**User Interaction Simulation:**\n- Mouse movements to specific coordinates\n- Left-click and hover actions\n- Text input via keyboard typing\n- Keyboard shortcuts and key presses\n- Scroll operations with configurable pixels\n\n**Tab Management:**\n- Create new browser tabs\n- Switch between existing tabs\n- Close tabs\n- Refresh page content\n\n### Action Schema\n\nThe controller defines a structured JSON schema for all available actions:\n\n| Action | Parameters | Description |\n|--------|------------|-------------|\n| `visit_url` | `url` | Navigate to specified URL |\n| `web_search` | `query` | Execute web search |\n| `type` | `text`, `coordinate`, `press_enter`, `delete_existing_text` | Type text or interact with elements |\n| `key` | `keys` | Press keyboard keys |\n| `mouse_move` | `coordinate` | Move mouse cursor |\n| `left_click` | `coordinate` | Click at coordinate |\n| `hover` | `coordinate` | Hover over element |\n| `scroll` | `pixels` | Scroll page (positive=up, negative=down) |\n| `select_option` | `element`, `value` | Select dropdown option |\n| `create_tab` | `url` | Open new tab |\n| `switch_tab` | `tab_id` | Switch to specific tab |\n| `refresh_page` | - | Reload current page |\n| `history_back` | - | Navigate browser history back |\n| `sleep` | `time` | Wait specified seconds |\n| `stop_action` | - | Stop current action sequence |\n\n资料来源：[src/magentic_ui/tools/playwright/playwright_controller.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_controller.py)\n\n## State Management\n\n### PlaywrightState\n\nThe `PlaywrightState` module handles serialization and persistence of browser session state.\n\n**State Components:**\n\n| Component | Description |\n|-----------|-------------|\n| `BrowserState` | Complete snapshot of browser context |\n| `save_browser_state()` | Serialize current state to storage |\n| `load_browser_state()` | Restore state from storage |\n\n**State Data Structure:**\n- Current page URL and title\n- Tab information and active tab ID\n- Scroll position\n- Cookie and local storage data\n- Form input values\n- Screenshot history\n\nThis enables:\n- Session recovery after interruptions\n- Parallel agent execution with shared state\n- Checkpoint creation for long-running tasks\n\n资料来源：[src/magentic_ui/tools/playwright/playwright_state.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/playwright_state.py)\n\n## Interactive Element Marking\n\n### Set of Mark (_set_of_mark)\n\nThe `_set_of_mark` module enhances web pages with visual markers that identify interactive elements. This is crucial for LLM-based agents to accurately identify and target UI elements.\n\n**Marking Strategy:**\n- Assigns unique numeric identifiers to interactive elements\n- Overlays clickable numbers on buttons, links, inputs\n- Uses sequential numbering for easy reference\n- Provides coordinate mappings for action targeting\n\n**Benefits:**\n- Enables precise element targeting by AI agents\n- Reduces ambiguity in element selection\n- Supports visual debugging and verification\n- Works across different page layouts and frameworks\n\n资料来源：[src/magentic_ui/tools/playwright/_set_of_mark.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/tools/playwright/_set_of_mark.py)\n\n## WebSurfer Agent\n\nThe `WebSurfer` agent is the primary consumer of the browser automation system. It combines the browser implementations with an LLM to make intelligent browsing decisions.\n\n### Agent Capabilities\n\n**Autonomous Navigation:**\n- Follow links and navigate between pages\n- Complete multi-step web forms\n- Search the web and process results\n- Extract structured information from pages\n\n**Content Processing:**\n- Optical Character Recognition (OCR) for image content\n- Visual question answering on screenshots\n- Markdown rendering of page content\n- File download handling\n\n**Interaction Modes:**\n- Automatic execution with configurable action limits\n- Step-by-step mode with human approval\n- Control handover for human intervention\n- Pause and resume capabilities\n\n### Configuration Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `start_page` | str | Google | Initial page on browser launch |\n| `animate_actions` | bool | False | Enable action animation |\n| `save_screenshots` | bool | False | Persist screenshots to disk |\n| `max_actions_per_step` | int | 5 | Maximum actions per reasoning step |\n| `resize_viewport` | bool | True | Auto-resize viewport |\n| `url_statuses` | dict | None | URL allow/reject rules |\n| `single_tab_mode` | bool | False | Restrict to single tab |\n\n资料来源：[src/magentic_ui/agents/web_surfer/_web_surfer.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/_web_surfer.py)\n\n## Usage Patterns\n\n### Local Browser Usage\n\n```python\nfrom magentic_ui.tools.playwright.browser import LocalPlaywrightBrowser\n\nbrowser = LocalPlaywrightBrowser(\n headless=False,\n downloads_folder=\"./downloads\"\n)\nawait browser.start()\n```\n\n### Docker-based Browser with VNC\n\n```python\nfrom magentic_ui.tools.playwright.browser import VncDockerPlaywrightBrowser\n\nbrowser = VncDockerPlaywrightBrowser(\n port=5900,\n novnc_port=8080\n)\nawait browser.start()\n# Access via browser at http://localhost:8080/vnc.html\n```\n\n### Controller Integration\n\n```python\nfrom magentic_ui.tools.playwright.playwright_controller import PlaywrightController\n\ncontroller = PlaywrightController(browser)\nawait controller.async_setup()\n\n# Execute actions\nresult = await controller(\n {\"action\": \"visit_url\", \"url\": \"https://example.com\"}\n)\n```\n\n## Workflow Diagram\n\n```mermaid\nsequenceDiagram\n participant Agent\n participant Controller\n participant Browser\n participant Page\n \n Agent->>Controller: Execute action\n Controller->>Controller: Validate parameters\n Controller->>Browser: Get page instance\n Browser->>Page: Perform action\n Page-->>Browser: Action result\n Browser-->>Controller: Browser response\n Controller->>Controller: Process result\n Controller->>Agent: Action result\n \n Note over Agent,Page: Repeat until task complete\n```\n\n## Frontend Integration\n\nThe browser automation system integrates with the Magentic-UI frontend through:\n\n**DetailViewer Component:**\n- Real-time browser view in the UI\n- Screenshot gallery display\n- Live action feed\n- Control mode overlay for human takeover\n\n**Modal Components:**\n- BrowserModal for full-screen viewing\n- VNC connection handling\n- Pause and resume controls\n\n资料来源：[frontend/src/components/views/chat/detail_viewer.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/views/chat/detail_viewer.tsx)\n\n## Security Considerations\n\n**URL Filtering:**\n- `UrlStatusManager` validates navigation targets\n- Configurable allow/reject lists\n- Prevents navigation to untrusted domains\n\n**Sandbox Isolation:**\n- Docker containers provide process isolation\n- Restricted network access when needed\n- Resource limits prevent runaway processes\n\n**Approval Workflows:**\n- Human approval for sensitive actions\n- Configurable approval thresholds\n- Audit logging of all actions\n\n## Conclusion\n\nThe Browser Automation system provides a robust, flexible foundation for web interaction in Magentic-UI. By combining Playwright's powerful browser control with thoughtful abstractions and multiple deployment options, it enables AI agents to perform complex web-based tasks reliably and safely.\n\n---\n\n\n\n## Docker Containers\n\n### 相关页面\n\n相关主题：[Getting Started with Magentic-UI](#getting-started), [Browser Automation](#browser-automation)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [docker/magentic-ui-browser-docker/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/Dockerfile)\n- [docker/magentic-ui-browser-docker/supervisord.conf](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/supervisord.conf)\n- [docker/magentic-ui-browser-docker/playwright-server.js](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/playwright-server.js)\n- [docker/magentic-ui-python-env/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-python-env/Dockerfile)\n- [docker/build-all.sh](https://github.com/microsoft/magentic-ui/blob/main/docker/build-all.sh)\n- [src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n \n\n# Docker Containers\n\nMagentic-UI leverages Docker containers to provide isolated, reproducible environments for running browser automation and code execution tasks. This architecture enables the application to execute complex multi-agent workflows while maintaining system-level isolation and consistent runtime dependencies.\n\n## Architecture Overview\n\nMagentic-UI uses two primary Docker images working in tandem to deliver its functionality:\n\n```mermaid\ngraph TB\n subgraph \"Magentic-UI Architecture\"\n A[\"Frontend UI
(localhost:8081)\"] --> B[\"Backend API
(Python/FastAPI)\"]\n B --> C[\"Browser Container
(VNC + Playwright)\"]\n B --> D[\"Python Environment Container
(Code Execution)\"]\n end\n \n subgraph \"Container Communication\"\n C <-->|\"WebSocket/REST\"| B\n D <-->|\"STDIO/REST\"| B\n end\n```\n\n## Docker Image Types\n\n| Image Type | Purpose | Key Components |\n|------------|---------|----------------|\n| `magentic-ui-browser` | Browser automation and web interaction | VNC Server, noVNC, Playwright, Chromium |\n| `magentic-ui-python-env` | Safe Python code execution | Python runtime, isolated environment |\n\n资料来源：[docker/build-all.sh:1-20]()\n\n### Browser Docker Container\n\nThe browser container provides a full graphical environment for web surfing agents. It includes:\n\n- **VNC Server**: Provides virtual display access\n- **noVNC**: Web-based VNC client for browser access\n- **Playwright**: Browser automation framework for programmatic control\n- **Chromium**: Headless-capable web browser\n\n资料来源：[docker/magentic-ui-browser-docker/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/Dockerfile)\n\n### Python Environment Docker Container\n\nThe Python environment container provides a sandboxed environment for executing user-generated Python code safely:\n\n- Isolated Python runtime\n- Restricted file system access\n- Controlled network access\n- Independent package management\n\n资料来源：[docker/magentic-ui-python-env/Dockerfile](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-python-env/Dockerfile)\n\n## Docker Initialization Workflow\n\n```mermaid\nsequenceDiagram\n participant User\n participant CLI\n participant Docker Daemon\n participant Registry\n \n User->>CLI: magentic-ui --port 8081\n CLI->>Docker Daemon: Check if Docker is running\n Docker Daemon-->>CLI: Docker Status\n \n alt Docker not running\n CLI->>User: Error: Please start Docker\n else Docker running\n CLI->>Docker Daemon: Check browser image exists\n Docker Daemon-->>CLI: Image Status\n \n alt Image missing\n CLI->>Registry: Pull browser image\n Registry-->>Docker Daemon: Image layers\n Docker Daemon->>Docker Daemon: Build image\n end\n \n CLI->>Docker Daemon: Check Python image exists\n Docker Daemon-->>CLI: Image Status\n \n alt Image missing\n CLI->>Registry: Pull Python image\n Registry-->>Docker Daemon: Image layers\n Docker Daemon->>Docker Daemon: Build image\n end\n \n CLI->>User: Magentic-UI ready\n end\n```\n\n资料来源：[src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n\n## Container Management Functions\n\nThe `src/magentic_ui/_docker.py` module provides core Docker management functionality:\n\n| Function | Purpose |\n|----------|---------|\n| `check_docker_running()` | Verifies Docker daemon is accessible |\n| `check_browser_image()` | Checks if browser Docker image exists locally |\n| `check_python_image()` | Checks if Python environment Docker image exists locally |\n| `pull_browser_image()` | Pulls/updates the browser Docker image |\n| `pull_python_image()` | Pulls/updates the Python environment Docker image |\n\n资料来源：[src/magentic_ui/_docker.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_docker.py)\n\n## Build Process\n\nThe build script `docker/build-all.sh` constructs both Docker images:\n\n```bash\n# Build browser Docker image\ndocker build -t magentic-ui-browser ./magentic-ui-browser-docker\n\n# Build Python environment Docker image\ndocker build -t magentic-ui-python-env ./magentic-ui-python-env\n```\n\n资料来源：[docker/build-all.sh](https://github.com/microsoft/magentic-ui/blob/main/docker/build-all.sh)\n\n## Browser Container Services\n\nThe browser container runs multiple services managed by supervisord:\n\n```mermaid\ngraph LR\n subgraph \"Browser Container Services\"\n A[\"supervisord
(Process Manager)\"]\n A --> B[\"Xvfb
(Virtual Display)\"]\n A --> C[\"x11vnc
(VNC Server)\"]\n A --> D[\"noVNC
(Web VNC)\"]\n A --> E[\"Playwright
(Browser Control)\"]\n end\n```\n\n### Service Configuration\n\nThe browser container uses `supervisord.conf` for service orchestration:\n\n- **Process Management**: Supervisord manages all background services\n- **Auto-restart**: Services automatically restart on failure\n- **Log Management**: Centralized logging configuration\n\n资料来源：[docker/magentic-ui-browser-docker/supervisord.conf](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/supervisord.conf)\n\n### Playwright Server\n\nThe Playwright server (`playwright-server.js`) provides HTTP API access to browser automation:\n\n```javascript\n// Server initialization with browser configuration\n// Handles browser launching, page creation, and element interaction\n```\n\n资料来源：[docker/magentic-ui-browser-docker/playwright-server.js](https://github.com/microsoft/magentic-ui/blob/main/docker/magentic-ui-browser-docker/playwright-server.js)\n\n## Running Without Docker\n\nFor environments where Docker is unavailable, Magentic-UI supports a limited mode:\n\n```bash\nmagentic-ui --run-without-docker --port 8081\n```\n\n**Limitations in No-Docker Mode**:\n\n| Feature | With Docker | Without Docker |\n|---------|-------------|----------------|\n| Web Surfing | Full browser automation | Not available |\n| Code Execution | Isolated sandbox | Not available |\n| File Handling | Enhanced isolation | Basic support |\n| Agent Capabilities | Complete | Reduced |\n\n资料来源：[README.md](https://github.com/microsoft/magentic-ui/blob/main/README.md)\n\n## Prerequisites\n\n### System Requirements\n\n| Requirement | Minimum | Recommended |\n|-------------|---------|-------------|\n| Docker Version | Latest stable | Latest stable |\n| Python | 3.10+ | 3.11+ |\n| RAM | 4GB | 8GB+ |\n| Disk Space | 2GB | 5GB+ |\n\n### Platform Support\n\n- **Linux**: Full support with native Docker\n- **macOS**: Full support with Docker Desktop\n- **Windows**: WSL2 required for Docker support\n\n资料来源：[TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Troubleshooting\n\n### Common Docker Issues\n\n| Issue | Symptom | Solution |\n|-------|---------|----------|\n| Docker not running | \"Docker is not running\" error | Start Docker Desktop/daemon |\n| Image pull failure | Timeout during first run | Run `docker/build-all.sh` manually |\n| Port conflict | Container fails to start | Change port with `--port` flag |\n\n### Verification Commands\n\n```bash\n# Check Docker is running\ndocker info\n\n# Verify images exist\ndocker images | grep magentic-ui\n\n# Manually build images\ncd docker && sh build-all.sh\n```\n\n资料来源：[TROUBLESHOOTING.md](https://github.com/microsoft/magentic-ui/blob/main/TROUBLESHOOTING.md)\n\n## Configuration\n\n### Environment Variables\n\n| Variable | Purpose | Default |\n|----------|---------|---------|\n| `NOVNC_PORT` | noVNC web interface port | 6080 |\n| `PLAYWRIGHT_PORT` | Playwright API port | 8080 |\n| `PYTHON_ENV_PORT` | Python execution port | 8082 |\n\n### Workspace Configuration\n\nThe CLI manages workspace paths passed to containers:\n\n```python\nworkspace_config = {\n \"internal_workspace_root\": \"/path/to/internal\",\n \"external_workspace_root\": \"/path/to/external\",\n \"inside_docker\": True,\n \"config\": {...},\n \"run_without_docker\": False\n}\n```\n\n资料来源：[src/magentic_ui/backend/cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/cli.py)\n\n## Security Considerations\n\n### Container Isolation\n\n- **Network Isolation**: Containers communicate via internal bridge network\n- **File System Isolation**: Read-only base images with volume mounts for data\n- **Process Isolation**: Separate PID namespaces\n\n### Best Practices\n\n1. Always run Docker with non-root user when possible\n2. Keep Docker images updated with latest security patches\n3. Use the provided workspace paths for file operations\n4. Monitor container resource usage\n\n---\n\n\n\n## Configuration\n\n### 相关页面\n\n相关主题：[Getting Started with Magentic-UI](#getting-started)\n\n\nRelated Source Files
\n\nThe following source files were used to generate this documentation:\n\n- [src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n- [src/magentic_ui/backend/cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/cli.py)\n- [src/magentic_ui/backend/web/config.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/config.py)\n- [src/magentic_ui/backend/teammanager/teammanager.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/teammanager/teammanager.py)\n- [fara_config.yaml](https://github.com/microsoft/magentic-ui/blob/main/fara_config.yaml)\n- [frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n- [src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n- [frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n \n\n# Configuration\n\nMagentic-UI provides a multi-layered configuration system that spans both the backend (Python) and frontend (React/TypeScript) layers. The system handles environment-based settings, agent parameters, UI theming, and server configurations through YAML files, environment variables, and component-level props.\n\n## Overview\n\nThe configuration architecture in Magentic-UI can be visualized as follows:\n\n```mermaid\ngraph TD\n A[Configuration Sources] --> B[Backend CLI]\n A --> C[Environment Variables]\n A --> D[YAML Config Files]\n A --> E[Frontend React Components]\n \n B --> F[Server Initialization]\n C --> G[API URL Configuration]\n D --> H[Agent Parameters]\n E --> I[UI Theme & Modal Settings]\n \n F --> J[Backend Server Running on Port 8081]\n G --> K[Frontend Dev Server]\n H --> L[Web Surfer Agent]\n I --> M[User Interface]\n```\n\n## Backend Configuration\n\n### CLI Entry Point\n\nThe main CLI entry point in `src/magentic_ui/_cli.py` serves as the primary configuration bootstrap for the backend server. It handles argument parsing and delegates to the backend CLI module.\n\n**Key configuration parameters supported:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `--host` | string | Server host address |\n| `--port` | integer | Server port number (default: 8081) |\n| `--config` | string | Path to YAML configuration file |\n| `--debug` | boolean | Enable debug mode |\n\n资料来源：[src/magentic_ui/_cli.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/_cli.py)\n\n### Web Server Configuration\n\nThe web server configuration module (`src/magentic_ui/backend/web/config.py`) defines the core server settings used by the FastAPI-based backend.\n\n```python\nclass ServerConfig:\n host: str = \"0.0.0.0\"\n port: int = 8081\n cors_origins: list[str] = [\"http://localhost:8000\"]\n debug: bool = False\n```\n\n**Configuration Options:**\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `host` | `\"0.0.0.0\"` | Bind address for the server |\n| `port` | `8081` | HTTP port for the backend API |\n| `cors_origins` | `[\"http://localhost:8000\"]` | Allowed CORS origins |\n| `debug` | `false` | Enable verbose logging and hot reload |\n\n资料来源：[src/magentic_ui/backend/web/config.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/web/config.py)\n\n### Team Manager Configuration\n\nThe `teammanager.py` module handles multi-agent orchestration configuration. It manages agent teams and their coordination settings.\n\n**Key configuration aspects:**\n\n- Agent pool sizing\n- Maximum concurrent agents\n- Communication protocols between agents\n- Timeout settings for agent operations\n\n资料来源：[src/magentic_ui/backend/teammanager/teammanager.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/backend/teammanager/teammanager.py)\n\n## YAML Configuration Files\n\n### fara_config.yaml\n\nThe `fara_config.yaml` file contains configuration for the web surfer agent, including display settings and browser automation parameters.\n\n```yaml\ndisplay_width_px: 1280\ndisplay_height_px: 720\ninclude_input_text_key_args: false\n```\n\n**Web Surfer Agent Parameters:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `display_width_px` | integer | Browser viewport width in pixels |\n| `display_height_px` | integer | Browser viewport height in pixels |\n| `include_input_text_key_args` | boolean | Include text input keyboard shortcuts |\n\n资料来源：[fara_config.yaml](https://github.com/microsoft/magentic-ui/blob/main/fara_config.yaml)\n\n### Agent Parameter Handling\n\nThe `_prompts.py` module in the web surfer agent demonstrates how configuration is consumed:\n\n```python\ndef __init__(self, cfg=None):\n self.display_width_px = cfg[\"display_width_px\"]\n self.display_height_px = cfg[\"display_height_px\"]\n include_input_text_key_args = cfg.pop(\"include_input_text_key_args\", False)\n if not include_input_text_key_args:\n self.parameters[\"properties\"].pop(\"press_enter\", None)\n self.parameters[\"properties\"].pop(\"delete_existing_text\", None)\n super().__init__(cfg)\n```\n\n资料来源：[src/magentic_ui/agents/web_surfer/fara/_prompts.py](https://github.com/microsoft/magentic-ui/blob/main/src/magentic_ui/agents/web_surfer/fara/_prompts.py)\n\n## Frontend Configuration\n\n### Environment Variables\n\nThe frontend uses environment variables configured through a `.env` file structure. The development environment requires specific settings to connect to the backend API.\n\n**Setup Instructions:**\n\n1. Copy `.env.default` to `.env.development`\n2. Set the required variables in the new file\n\n| Variable | Required Value | Description |\n|----------|---------------|-------------|\n| `GATSBY_API_URL` | `http://localhost:8081/api` | Backend API endpoint |\n| `GATSBY_WS_URL` | `ws://localhost:8081/ws` | WebSocket endpoint (if applicable) |\n\n资料来源：[frontend/README.md](https://github.com/microsoft/magentic-ui/blob/main/frontend/README.md)\n\n### API Configuration Flow\n\n```mermaid\nsequenceDiagram\n participant FE as Frontend (React)\n participant API as Backend API\n participant WS as WebSocket\n \n FE->>FE: Load .env.development\n FE->>API: HTTP requests to GATSBY_API_URL\n FE->>WS: WebSocket connections\n API-->>FE: JSON responses\n WS-->>FE: Real-time updates\n```\n\n### MCP Server Configuration\n\nThe MCP (Model Context Protocol) server configuration modal provides a UI for managing external server integrations.\n\n**Supported Connection Types:**\n\n| Type | Description | Configuration Method |\n|------|-------------|----------------------|\n| `SSE` | Server-Sent Events | Form-based input |\n| `Stdio` | Standard I/O | Form-based input |\n| `JSON` | Raw JSON Config | Direct JSON editing |\n\n**Server Configuration Validation:**\n\n| Field | Validation Rule |\n|-------|----------------|\n| `serverName` | Required, alphanumeric characters only, max 50 characters |\n| `serverName` | Must be unique across all servers |\n\n```typescript\n// Example validation logic from McpConfigModal.tsx\nconst serverNameError = !serverName || !/^[a-zA-Z0-9]+$/.test(serverName);\nconst serverNameDuplicateError = existingServers.some(\n (s) => s.name === serverName && s.id !== server?.id\n);\n```\n\n资料来源：[frontend/src/components/features/McpServersConfig/McpConfigModal.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/features/McpServersConfig/McpConfigModal.tsx)\n\n## UI Theme Configuration\n\n### Theme Application\n\nThe main layout component applies theme settings based on user preferences and system defaults:\n\n```typescript\n\n```\n\n**Theme Options:**\n\n| Mode | Algorithm | CSS Classes |\n|------|-----------|-------------|\n| Light | `defaultAlgorithm` | `bg-white`, `text-gray-900` |\n| Dark | `darkAlgorithm` | `bg-gray-900`, `text-gray-100` |\n\n资料来源：[frontend/src/components/layout.tsx](https://github.com/microsoft/magentic-ui/blob/main/frontend/src/components/layout.tsx)\n\n### Component-Level Styling\n\nComponents use Tailwind CSS utility classes for configuration of:\n\n- Color schemes (`bg-magenta-800`, `text-blue-400`)\n- Spacing (`p-3`, `mt-4`, `mb-2`)\n- Typography (`text-sm`, `font-medium`)\n- Transitions (`transition-colors`, `transition-all duration-300`)\n\n## Configuration Workflow\n\n```mermaid\ngraph LR\n A[Start Application] --> B{Backend or Frontend?}\n \n B -->|Backend| C[Load CLI Args]\n B -->|Backend| D[Parse YAML Config]\n B -->|Backend| E[Initialize Server]\n \n B -->|Frontend| F[Load .env.development]\n B -->|Frontend| G[Build API URL]\n B -->|Frontend| H[Render UI Components]\n \n C --> E\n D --> E\n F --> G\n G --> H\n E --> I[Server Ready]\n H --> J[User Interface Ready]\n```\n\n## Configuration Files Summary\n\n| File Path | Purpose | Format |\n|-----------|---------|--------|\n| `src/magentic_ui/_cli.py` | Main CLI entry point | Python |\n| `src/magentic_ui/backend/cli.py` | Backend CLI logic | Python |\n| `src/magentic_ui/backend/web/config.py` | Web server settings | Python (dataclass) |\n| `src/magentic_ui/backend/teammanager/teammanager.py` | Agent orchestration | Python |\n| `fara_config.yaml` | Web surfer agent settings | YAML |\n| `.env.development` | Frontend environment | Environment Variables |\n\n## Best Practices\n\n1. **Environment Isolation**: Keep development and production environment files separate\n2. **Validation**: Always validate MCP server names against alphanumeric patterns\n3. **CORS Settings**: Ensure backend CORS configuration matches frontend origin\n4. **Port Consistency**: The frontend expects the backend at `http://localhost:8081/api`\n5. **Theme Persistence**: User theme preferences should be stored in local storage or user profile\n\n---\n\n---\n\n## Doramagic 踩坑日志\n\n项目：microsoft/magentic-ui\n\n摘要：发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：安装坑 - 来源证据：Create tutorials and documentation for the codebase。\n\n## 1. 安装坑 · 来源证据：Create tutorials and documentation for the codebase\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Create tutorials and documentation for the codebase\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_c0979f7ebb064422a6a8095561f6a9bd | https://github.com/microsoft/magentic-ui/issues/154 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 2. 安装坑 · 来源证据：Support Podman in place of Docker\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Support Podman in place of Docker\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_f88231cc4cad442ca53d92ea3a40a655 | https://github.com/microsoft/magentic-ui/issues/312 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 3. 安装坑 · 来源证据：magentic-ui can't display all the html element\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：magentic-ui can't display all the html element\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_21f19953edd74379ab2d25cedc37ca1b | https://github.com/microsoft/magentic-ui/issues/362 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 4. 配置坑 · 来源证据：Refreshing or restart the web app will make the current Session unavailable\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个配置相关的待验证问题：Refreshing or restart the web app will make the current Session unavailable\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_83a9cafd59254028853ef84cd1ccc756 | https://github.com/microsoft/magentic-ui/issues/336 | 来源讨论提到 node 相关条件，需在安装/试用前复核。\n\n## 5. 能力坑 · 能力判断依赖假设\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：README/documentation is current enough for a first validation pass.\n- 对用户的影响：假设不成立时，用户拿不到承诺的能力。\n- 建议检查：将假设转成下游验证清单。\n- 防护动作：假设必须转成验证项；没有验证结果前不能写成事实。\n- 证据：capability.assumptions | github_repo:978331188 | https://github.com/microsoft/magentic-ui | README/documentation is current enough for a first validation pass.\n\n## 6. 运行坑 · 来源证据：Why not conduct a requirement analysis before the plan?\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个运行相关的待验证问题：Why not conduct a requirement analysis before the plan?\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6003a9c2194f40c0865145385cf98c32 | https://github.com/microsoft/magentic-ui/issues/321 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 7. 维护坑 · 来源证据：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_7e754869326e42e1a7c57f3a1962ef9e | https://github.com/microsoft/magentic-ui/issues/360 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 8. 维护坑 · 维护活跃度未知\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：未记录 last_activity_observed。\n- 对用户的影响：新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。\n- 建议检查：补 GitHub 最近 commit、release、issue/PR 响应信号。\n- 防护动作：维护活跃度未知时，推荐强度不能标为高信任。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | last_activity_observed missing\n\n## 9. 安全/权限坑 · 下游验证发现风险项\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：下游已经要求复核，不能在页面中弱化。\n- 建议检查：进入安全/权限治理复核队列。\n- 防护动作：下游风险存在时必须保持 review/recommendation 降级。\n- 证据：downstream_validation.risk_items | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 10. 安全/权限坑 · 存在评分风险\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：风险会影响是否适合普通用户安装。\n- 建议检查：把风险写入边界卡，并确认是否需要人工复核。\n- 防护动作：评分风险必须进入边界卡，不能只作为内部分数。\n- 证据：risks.scoring_risks | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 11. 安全/权限坑 · 来源证据：Settings redesign\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：Settings redesign\n- 对用户的影响：可能影响授权、密钥配置或安全边界。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6a2eeae98b6d4fdab476464d57e64e1d | https://github.com/microsoft/magentic-ui/issues/227 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 12. 维护坑 · issue/PR 响应质量未知\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：issue_or_pr_quality=unknown。\n- 对用户的影响：用户无法判断遇到问题后是否有人维护。\n- 建议检查：抽样最近 issue/PR，判断是否长期无人处理。\n- 防护动作：issue/PR 响应未知时，必须提示维护风险。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | issue_or_pr_quality=unknown\n\n## 13. 维护坑 · 发布节奏不明确\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：release_recency=unknown。\n- 对用户的影响：安装命令和文档可能落后于代码，用户踩坑概率升高。\n- 建议检查：确认最近 release/tag 和 README 安装命令是否一致。\n- 防护动作：发布节奏未知或过期时，安装说明必须标注可能漂移。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | release_recency=unknown\n\n\n",
"summary": "DeepWiki/Human Wiki 完整输出，末尾追加 Discovery Agent 踩坑日志。",
"title": "Human Manual / 人类版说明书"
},
"pitfall_log": {
"asset_id": "pitfall_log",
"filename": "PITFALL_LOG.md",
"markdown": "# Pitfall Log / 踩坑日志\n\n项目：microsoft/magentic-ui\n\n摘要：发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：安装坑 - 来源证据：Create tutorials and documentation for the codebase。\n\n## 1. 安装坑 · 来源证据：Create tutorials and documentation for the codebase\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Create tutorials and documentation for the codebase\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_c0979f7ebb064422a6a8095561f6a9bd | https://github.com/microsoft/magentic-ui/issues/154 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 2. 安装坑 · 来源证据：Support Podman in place of Docker\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：Support Podman in place of Docker\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_f88231cc4cad442ca53d92ea3a40a655 | https://github.com/microsoft/magentic-ui/issues/312 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 3. 安装坑 · 来源证据：magentic-ui can't display all the html element\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安装相关的待验证问题：magentic-ui can't display all the html element\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_21f19953edd74379ab2d25cedc37ca1b | https://github.com/microsoft/magentic-ui/issues/362 | 来源讨论提到 docker 相关条件，需在安装/试用前复核。\n\n## 4. 配置坑 · 来源证据：Refreshing or restart the web app will make the current Session unavailable\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个配置相关的待验证问题：Refreshing or restart the web app will make the current Session unavailable\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_83a9cafd59254028853ef84cd1ccc756 | https://github.com/microsoft/magentic-ui/issues/336 | 来源讨论提到 node 相关条件，需在安装/试用前复核。\n\n## 5. 能力坑 · 能力判断依赖假设\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：README/documentation is current enough for a first validation pass.\n- 对用户的影响：假设不成立时，用户拿不到承诺的能力。\n- 建议检查：将假设转成下游验证清单。\n- 防护动作：假设必须转成验证项；没有验证结果前不能写成事实。\n- 证据：capability.assumptions | github_repo:978331188 | https://github.com/microsoft/magentic-ui | README/documentation is current enough for a first validation pass.\n\n## 6. 运行坑 · 来源证据：Why not conduct a requirement analysis before the plan?\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个运行相关的待验证问题：Why not conduct a requirement analysis before the plan?\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6003a9c2194f40c0865145385cf98c32 | https://github.com/microsoft/magentic-ui/issues/321 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 7. 维护坑 · 来源证据：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：Sticked at click the “Shopping Cart” icon and cannot goto check out page\n- 对用户的影响：可能增加新用户试用和生产接入成本。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_7e754869326e42e1a7c57f3a1962ef9e | https://github.com/microsoft/magentic-ui/issues/360 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 8. 维护坑 · 维护活跃度未知\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：未记录 last_activity_observed。\n- 对用户的影响：新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。\n- 建议检查：补 GitHub 最近 commit、release、issue/PR 响应信号。\n- 防护动作：维护活跃度未知时，推荐强度不能标为高信任。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | last_activity_observed missing\n\n## 9. 安全/权限坑 · 下游验证发现风险项\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：下游已经要求复核，不能在页面中弱化。\n- 建议检查：进入安全/权限治理复核队列。\n- 防护动作：下游风险存在时必须保持 review/recommendation 降级。\n- 证据：downstream_validation.risk_items | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 10. 安全/权限坑 · 存在评分风险\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：no_demo\n- 对用户的影响：风险会影响是否适合普通用户安装。\n- 建议检查：把风险写入边界卡，并确认是否需要人工复核。\n- 防护动作：评分风险必须进入边界卡，不能只作为内部分数。\n- 证据：risks.scoring_risks | github_repo:978331188 | https://github.com/microsoft/magentic-ui | no_demo; severity=medium\n\n## 11. 安全/权限坑 · 来源证据：Settings redesign\n\n- 严重度：medium\n- 证据强度：source_linked\n- 发现：GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：Settings redesign\n- 对用户的影响：可能影响授权、密钥配置或安全边界。\n- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。\n- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。\n- 证据：community_evidence:github | cevd_6a2eeae98b6d4fdab476464d57e64e1d | https://github.com/microsoft/magentic-ui/issues/227 | 来源类型 github_issue 暴露的待验证使用条件。\n\n## 12. 维护坑 · issue/PR 响应质量未知\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：issue_or_pr_quality=unknown。\n- 对用户的影响：用户无法判断遇到问题后是否有人维护。\n- 建议检查：抽样最近 issue/PR，判断是否长期无人处理。\n- 防护动作：issue/PR 响应未知时，必须提示维护风险。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | issue_or_pr_quality=unknown\n\n## 13. 维护坑 · 发布节奏不明确\n\n- 严重度：low\n- 证据强度：source_linked\n- 发现：release_recency=unknown。\n- 对用户的影响：安装命令和文档可能落后于代码，用户踩坑概率升高。\n- 建议检查：确认最近 release/tag 和 README 安装命令是否一致。\n- 防护动作：发布节奏未知或过期时，安装说明必须标注可能漂移。\n- 证据：evidence.maintainer_signals | github_repo:978331188 | https://github.com/microsoft/magentic-ui | release_recency=unknown\n",
"summary": "用户实践前最可能遇到的身份、安装、配置、运行和安全坑。",
"title": "Pitfall Log / 踩坑日志"
},
"prompt_preview": {
"asset_id": "prompt_preview",
"filename": "PROMPT_PREVIEW.md",
"markdown": "# magentic-ui - Prompt Preview\n\n> Copy the prompt below into your AI host before installing anything.\n> Its purpose is to let you safely feel the project's workflow, not to claim the project has already run.\n\n## Copy this prompt\n\n```text\nYou are using an independent Doramagic capability pack for microsoft/magentic-ui.\n\nProject:\n- Name: magentic-ui\n- Repository: https://github.com/microsoft/magentic-ui\n- Summary: A research prototype of a human-centered web agent\n- Host target: local_cli\n\nGoal:\nHelp me evaluate this project for the following task without installing it yet: A research prototype of a human-centered web agent\n\nBefore taking action:\n1. Restate my task, success standard, and boundary.\n2. Identify whether the next step requires tools, browser access, network access, filesystem access, credentials, package installation, or host configuration.\n3. Use only the Doramagic Project Pack, the upstream repository, and the source-linked evidence listed below.\n4. If a real command, install step, API call, file write, or host integration is required, mark it as \"requires post-install verification\" and ask for approval first.\n5. If evidence is missing, say \"evidence is missing\" instead of filling the gap.\n\nPreviewable capabilities:\n- Capability 1: A research prototype of a human-centered web agent\n\nCapabilities that require post-install verification:\n- Capability 1: Use the source-backed project context to guide one small, checkable workflow step.\n\nCore service flow:\n1. getting-started: Getting Started with Magentic-UI. Produce one small intermediate artifact and wait for confirmation.\n2. key-concepts: Key Concepts. Produce one small intermediate artifact and wait for confirmation.\n3. high-level-architecture: High-Level Architecture. Produce one small intermediate artifact and wait for confirmation.\n4. agent-system: Agent System. Produce one small intermediate artifact and wait for confirmation.\n5. team-orchestration: Team Orchestration. Produce one small intermediate artifact and wait for confirmation.\n\nSource-backed evidence to keep in mind:\n- https://github.com/microsoft/magentic-ui\n- https://github.com/microsoft/magentic-ui#readme\n- README.md\n- pyproject.toml\n- TROUBLESHOOTING.md\n- src/magentic_ui/approval_guard.py\n- src/magentic_ui/guarded_action.py\n- src/magentic_ui/learning/learner.py\n- src/magentic_ui/backend/web/app.py\n- src/magentic_ui/backend/teammanager/teammanager.py\n\nFirst response rules:\n1. Start Step 1 only.\n2. Explain the one service action you will perform first.\n3. Ask exactly three questions about my target workflow, success standard, and sandbox boundary.\n4. Stop and wait for my answers.\n\nStep 1 follow-up protocol:\n- After I answer the first three questions, stay in Step 1.\n- Produce six parts only: clarified task, success standard, boundary conditions, two or three options, tradeoffs for each option, and one recommendation.\n- End by asking whether I confirm the recommendation.\n- Do not move to Step 2 until I explicitly confirm.\n\nConversation rules:\n- Advance one step at a time and wait for confirmation after each small artifact.\n- Write outputs as recommendations or planned checks, not as completed execution.\n- Do not claim tests passed, files changed, commands ran, APIs were called, or the project was installed.\n- If the user asks for execution, first provide the sandbox setup, expected output, rollback, and approval checkpoint.\n```\n",
"summary": "不安装项目也能感受能力节奏的安全试用 Prompt。",
"title": "Prompt Preview / 安装前试用 Prompt"
},
"quick_start": {
"asset_id": "quick_start",
"filename": "QUICK_START.md",
"markdown": "# Quick Start / 官方入口\n\n项目：microsoft/magentic-ui\n\n## 官方安装入口\n\n### Python / pip · 官方安装入口\n\n```bash\npip install magentic-ui\n```\n\n来源：https://github.com/microsoft/magentic-ui#readme\n\n## 来源\n\n- repo: https://github.com/microsoft/magentic-ui\n- docs: https://github.com/microsoft/magentic-ui#readme\n",
"summary": "从项目官方 README 或安装文档提取的开工入口。",
"title": "Quick Start / 官方入口"
}
},
"validation_id": "dval_36857acfbba94767984ed9af003c2ae8"
}