{
"canonical_name": "nottelabs/notte",
"compilation_id": "pack_255e808812774f75bcff9aa5193d1278",
"created_at": "2026-05-19T06:39:15.861475+00:00",
"created_by": "project-pack-compiler",
"feedback": {
"carrier_selection_notes": [
"viable_asset_types=mcp_config, recipe, host_instruction, eval, preflight",
"recommended_asset_types=mcp_config, recipe, host_instruction, eval, preflight"
],
"evidence_delta": {
"confirmed_claims": [
"identity_anchor_present",
"capability_and_host_targets_present",
"install_path_declared_or_better"
],
"missing_required_fields": [],
"must_verify_forwarded": [
"Run or inspect `pip install notte` in an isolated environment.",
"Confirm the project exposes the claimed capability to at least one target host."
],
"quickstart_execution_scope": "allowlisted_sandbox_smoke",
"sandbox_command": "pip install notte",
"sandbox_container_image": "python:3.12-slim",
"sandbox_execution_backend": "docker",
"sandbox_planner_decision": "llm_execute_isolated_install",
"sandbox_validation_id": "sbx_7c72aa1d52be4f4f890b7156152c1ae7"
},
"feedback_event_type": "project_pack_compilation_feedback",
"learning_candidate_reasons": [],
"template_gaps": []
},
"identity": {
"canonical_id": "project_810c7b266241f3f0371a8fe76b94e566",
"canonical_name": "nottelabs/notte",
"homepage_url": null,
"license": "unknown",
"repo_url": "https://github.com/nottelabs/notte",
"slug": "notte",
"source_packet_id": "phit_360abee9881e44c1aa85b0faa3ddff1f",
"source_validation_id": "dval_ee6e6f447d8946f9a39f3a30f16f1c16"
},
"merchandising": {
"best_for": "需要流程自动化能力，并使用 chatgpt的用户",
"github_forks": 180,
"github_stars": 1953,
"one_liner_en": "🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.",
"one_liner_zh": "🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.",
"primary_category": {
"category_id": "workflow-automation",
"confidence": "high",
"name_en": "Workflow Automation",
"name_zh": "流程自动化",
"reason": "matched_keywords:automation, agent, browser"
},
"target_user": "使用 chatgpt 等宿主 AI 的用户",
"title_en": "notte",
"title_zh": "notte 能力包",
"visible_tags": [
{
"label_en": "Browser Agents",
"label_zh": "浏览器 Agent",
"source": "repo_evidence_project_characteristics",
"tag_id": "product_domain-browser-agents",
"type": "product_domain"
},
{
"label_en": "Web Task Automation",
"label_zh": "网页任务自动化",
"source": "repo_evidence_project_characteristics",
"tag_id": "user_job-web-task-automation",
"type": "user_job"
},
{
"label_en": "Natural-language Web Actions",
"label_zh": "自然语言网页操作",
"source": "repo_evidence_project_characteristics",
"tag_id": "core_capability-natural-language-web-actions",
"type": "core_capability"
},
{
"label_en": "Page Observation and Action Planning",
"label_zh": "页面观察与动作规划",
"source": "repo_evidence_project_characteristics",
"tag_id": "workflow_pattern-page-observation-and-action-planning",
"type": "workflow_pattern"
},
{
"label_en": "Structured Data Extraction",
"label_zh": "结构化数据提取",
"source": "repo_evidence_project_characteristics",
"tag_id": "selection_signal-structured-data-extraction",
"type": "selection_signal"
}
]
},
"packet_id": "phit_360abee9881e44c1aa85b0faa3ddff1f",
"page_model": {
"artifacts": {
"artifact_slug": "notte",
"files": [
"PROJECT_PACK.json",
"QUICK_START.md",
"PROMPT_PREVIEW.md",
"HUMAN_MANUAL.md",
"AI_CONTEXT_PACK.md",
"BOUNDARY_RISK_CARD.md",
"PITFALL_LOG.md",
"REPO_INSPECTION.json",
"REPO_INSPECTION.md",
"CAPABILITY_CONTRACT.json",
"EVIDENCE_INDEX.json",
"CLAIM_GRAPH.json"
],
"required_files": [
"PROJECT_PACK.json",
"QUICK_START.md",
"PROMPT_PREVIEW.md",
"HUMAN_MANUAL.md",
"AI_CONTEXT_PACK.md",
"BOUNDARY_RISK_CARD.md",
"PITFALL_LOG.md",
"REPO_INSPECTION.json"
]
},
"detail": {
"capability_source": "Project Hit Packet + DownstreamValidationResult",
"commands": [
{
"command": "pip install notte",
"label": "Python / pip · 官方安装入口",
"source": "https://github.com/nottelabs/notte#readme",
"verified": true
}
],
"display_tags": [
"浏览器 Agent",
"网页任务自动化",
"自然语言网页操作",
"页面观察与动作规划",
"结构化数据提取"
],
"eyebrow": "流程自动化",
"glance": [
{
"body": "判断自己是不是目标用户。",
"label": "最适合谁",
"value": "需要流程自动化能力，并使用 chatgpt的用户"
},
{
"body": "先理解能力边界，再决定是否继续。",
"label": "核心价值",
"value": "🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra."
},
{
"body": "未完成验证前保持审慎。",
"label": "继续前",
"value": "publish to Doramagic.ai project surfaces"
}
],
"guardrail_source": "Boundary & Risk Card",
"guardrails": [
{
"body": "Prompt Preview 只展示流程，不证明项目已安装或运行。",
"label": "Check 1",
"value": "不要把试用当真实运行"
},
{
"body": "chatgpt",
"label": "Check 2",
"value": "确认宿主兼容"
},
{
"body": "publish to Doramagic.ai project surfaces",
"label": "Check 3",
"value": "先隔离验证"
}
],
"mode": "mcp_config, recipe, host_instruction, eval, preflight",
"pitfall_log": {
"items": [
{
"body": "GitHub 社区证据显示该项目存在一个安装相关的待验证问题：v1.8.8",
"category": "安装坑",
"evidence": [
"community_evidence:github | cevd_5224b46e2312471c976888e8a2f8f4a6 | https://github.com/nottelabs/notte/releases/tag/v1.8.8 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.8",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：v1.8.13",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_56505da48cd74758ab7a9a1d82092e18 | https://github.com/nottelabs/notte/releases/tag/v1.8.13 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.13",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：v1.8.14",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_8f78a2591cea4f5aad8bfa0cb0ea15a0 | https://github.com/nottelabs/notte/releases/tag/v1.8.14 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.14",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：v1.8.15",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_f611f1d0dd2440af8e84e9544ab1bdb6 | https://github.com/nottelabs/notte/releases/tag/v1.8.15 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.15",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：v1.8.6",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_0ed3c86ff7a34e5896a4daeb75552d4d | https://github.com/nottelabs/notte/releases/tag/v1.8.6 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.6",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "GitHub 社区证据显示该项目存在一个配置相关的待验证问题：v1.8.9",
"category": "配置坑",
"evidence": [
"community_evidence:github | cevd_6195971390e8494f88083c862aef7eb6 | https://github.com/nottelabs/notte/releases/tag/v1.8.9 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.9",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "README/documentation is current enough for a first validation pass.",
"category": "能力坑",
"evidence": [
"capability.assumptions | github_repo:900152988 | https://github.com/nottelabs/notte | README/documentation is current enough for a first validation pass."
],
"severity": "medium",
"suggested_check": "将假设转成下游验证清单。",
"title": "能力判断依赖假设",
"user_impact": "假设不成立时，用户拿不到承诺的能力。"
},
{
"body": "GitHub 社区证据显示该项目存在一个运行相关的待验证问题：v1.8.7",
"category": "运行坑",
"evidence": [
"community_evidence:github | cevd_ab102ac99c2441118475601c34ab0ee9 | https://github.com/nottelabs/notte/releases/tag/v1.8.7 | 来源类型 github_release 暴露的待验证使用条件。"
],
"severity": "medium",
"suggested_check": "来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。",
"title": "来源证据：v1.8.7",
"user_impact": "可能增加新用户试用和生产接入成本。"
},
{
"body": "未记录 last_activity_observed。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:900152988 | https://github.com/nottelabs/notte | last_activity_observed missing"
],
"severity": "medium",
"suggested_check": "补 GitHub 最近 commit、release、issue/PR 响应信号。",
"title": "维护活跃度未知",
"user_impact": "新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。"
},
{
"body": "no_demo",
"category": "安全/权限坑",
"evidence": [
"downstream_validation.risk_items | github_repo:900152988 | https://github.com/nottelabs/notte | no_demo; severity=medium"
],
"severity": "medium",
"suggested_check": "进入安全/权限治理复核队列。",
"title": "下游验证发现风险项",
"user_impact": "下游已经要求复核，不能在页面中弱化。"
},
{
"body": "no_demo",
"category": "安全/权限坑",
"evidence": [
"risks.scoring_risks | github_repo:900152988 | https://github.com/nottelabs/notte | no_demo; severity=medium"
],
"severity": "medium",
"suggested_check": "把风险写入边界卡，并确认是否需要人工复核。",
"title": "存在评分风险",
"user_impact": "风险会影响是否适合普通用户安装。"
},
{
"body": "issue_or_pr_quality=unknown。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:900152988 | https://github.com/nottelabs/notte | issue_or_pr_quality=unknown"
],
"severity": "low",
"suggested_check": "抽样最近 issue/PR，判断是否长期无人处理。",
"title": "issue/PR 响应质量未知",
"user_impact": "用户无法判断遇到问题后是否有人维护。"
},
{
"body": "release_recency=unknown。",
"category": "维护坑",
"evidence": [
"evidence.maintainer_signals | github_repo:900152988 | https://github.com/nottelabs/notte | release_recency=unknown"
],
"severity": "low",
"suggested_check": "确认最近 release/tag 和 README 安装命令是否一致。",
"title": "发布节奏不明确",
"user_impact": "安装命令和文档可能落后于代码，用户踩坑概率升高。"
}
],
"source": "ProjectPitfallLog + ProjectHitPacket + validation + community signals",
"summary": "发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：安装坑 - 来源证据：v1.8.8。",
"title": "踩坑日志"
},
"snapshot": {
"contributors": 13,
"forks": 180,
"license": "unknown",
"note": "站点快照，非实时质量证明；用于开工前背景判断。",
"stars": 1953
},
"source_url": "https://github.com/nottelabs/notte",
"steps": [
{
"body": "不安装项目，先体验能力节奏。",
"code": "preview",
"title": "先试 Prompt"
},
{
"body": "理解输入、输出、失败模式和边界。",
"code": "manual",
"title": "读说明书"
},
{
"body": "把上下文交给宿主 AI 继续工作。",
"code": "context",
"title": "带给 AI"
},
{
"body": "进入主力环境前先完成安装入口与风险边界验证。",
"code": "verify",
"title": "沙箱验证"
}
],
"subtitle": "🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.",
"title": "notte 能力包",
"trial_prompt": "# notte - Prompt Preview\n\n> Copy the prompt below into your AI host before installing anything.\n> Its purpose is to let you safely feel the project's workflow, not to claim the project has already run.\n\n## Copy this prompt\n\n```text\nYou are using an independent Doramagic capability pack for nottelabs/notte.\n\nProject:\n- Name: notte\n- Repository: https://github.com/nottelabs/notte\n- Summary: 🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.\n- Host target: chatgpt\n\nGoal:\nHelp me evaluate this project for the following task without installing it yet: 🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.\n\nBefore taking action:\n1. Restate my task, success standard, and boundary.\n2. Identify whether the next step requires tools, browser access, network access, filesystem access, credentials, package installation, or host configuration.\n3. Use only the Doramagic Project Pack, the upstream repository, and the source-linked evidence listed below.\n4. If a real command, install step, API call, file write, or host integration is required, mark it as \"requires post-install verification\" and ask for approval first.\n5. If evidence is missing, say \"evidence is missing\" instead of filling the gap.\n\nPreviewable capabilities:\n- Capability 1: 🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.\n\nCapabilities that require post-install verification:\n- Capability 1: Use the source-backed project context to guide one small, checkable workflow step.\n\nCore service flow:\n1. introduction: Introduction to Notte. Produce one small intermediate artifact and wait for confirmation.\n2. quickstart: Quickstart Guide. Produce one small intermediate artifact and wait for confirmation.\n3. architecture: System Architecture. Produce one small intermediate artifact and wait for confirmation.\n4. agent-core: Agent Core System. Produce one small intermediate artifact and wait for confirmation.\n5. structured-output: Structured Output. Produce one small intermediate artifact and wait for confirmation.\n\nSource-backed evidence to keep in mind:\n- https://github.com/nottelabs/notte\n- https://github.com/nottelabs/notte#readme\n- README.md\n- src/notte/__init__.py\n- pyproject.toml\n- .env.example\n- docs/setup.md\n- examples/quickstart.py\n- packages/notte-agent/src/notte_agent/__init__.py\n- packages/notte-browser/src/notte_browser/__init__.py\n\nFirst response rules:\n1. Start Step 1 only.\n2. Explain the one service action you will perform first.\n3. Ask exactly three questions about my target workflow, success standard, and sandbox boundary.\n4. Stop and wait for my answers.\n\nStep 1 follow-up protocol:\n- After I answer the first three questions, stay in Step 1.\n- Produce six parts only: clarified task, success standard, boundary conditions, two or three options, tradeoffs for each option, and one recommendation.\n- End by asking whether I confirm the recommendation.\n- Do not move to Step 2 until I explicitly confirm.\n\nConversation rules:\n- Advance one step at a time and wait for confirmation after each small artifact.\n- Write outputs as recommendations or planned checks, not as completed execution.\n- Do not claim tests passed, files changed, commands ran, APIs were called, or the project was installed.\n- If the user asks for execution, first provide the sandbox setup, expected output, rollback, and approval checkpoint.\n```\n",
"voices": [
{
"body": "来源平台：github。github/github_release: v1.8.16（https://github.com/nottelabs/notte/releases/tag/v1.8.16）；github/github_release: v1.8.15（https://github.com/nottelabs/notte/releases/tag/v1.8.15）；github/github_release: v1.8.14（https://github.com/nottelabs/notte/releases/tag/v1.8.14）；github/github_release: v1.8.13（https://github.com/nottelabs/notte/releases/tag/v1.8.13）；github/github_release: v1.8.11（https://github.com/nottelabs/notte/releases/tag/v1.8.11）；github/github_release: v1.8.10（https://github.com/nottelabs/notte/releases/tag/v1.8.10）；github/github_release: v1.8.9（https://github.com/nottelabs/notte/releases/tag/v1.8.9）；github/github_release: v1.8.8（https://github.com/nottelabs/notte/releases/tag/v1.8.8）；github/github_release: v1.8.7（https://github.com/nottelabs/notte/releases/tag/v1.8.7）；github/github_release: v1.8.6（https://github.com/nottelabs/notte/releases/tag/v1.8.6）。这些是项目级外部声音，不作为单独质量证明。",
"items": [
{
"kind": "github_release",
"source": "github",
"title": "v1.8.16",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.16"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.15",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.15"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.14",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.14"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.13",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.13"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.11",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.11"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.10",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.10"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.9",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.9"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.8",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.8"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.7",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.7"
},
{
"kind": "github_release",
"source": "github",
"title": "v1.8.6",
"url": "https://github.com/nottelabs/notte/releases/tag/v1.8.6"
}
],
"status": "已收录 10 条来源",
"title": "社区讨论"
}
]
},
"homepage_card": {
"category": "流程自动化",
"desc": "🌸 Best framework to build web agents, and deploy serverless web automation functions on reliable browser infra.",
"effort": "安装已验证",
"forks": 180,
"icon": "bolt",
"name": "notte 能力包",
"risk": "可发布",
"slug": "notte",
"stars": 1953,
"tags": [
"浏览器 Agent",
"网页任务自动化",
"自然语言网页操作",
"页面观察与动作规划",
"结构化数据提取"
],
"thumb": "gray",
"type": "MCP 配置"
},
"manual": {
"markdown": "# https://github.com/nottelabs/notte 项目说明书\n\n生成时间：2026-05-16 12:22:12 UTC\n\n## 目录\n\n- [Introduction to Notte](#introduction)\n- [Quickstart Guide](#quickstart)\n- [System Architecture](#architecture)\n- [Agent Core System](#agent-core)\n- [Structured Output](#structured-output)\n- [Agent Fallback System](#agent-fallback)\n- [Browser Sessions](#sessions)\n- [Actions and Browser Controls](#actions-controls)\n- [Vaults and Credential Management](#vaults-credentials)\n- [Agent Personas](#personas)\n\n\n\n## Introduction to Notte\n\n### 相关页面\n\n相关主题：[Quickstart Guide](#quickstart), [System Architecture](#architecture)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [README.md](https://github.com/nottelabs/notte/blob/main/README.md)\n- [packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n- [packages/notte-core/src/notte_core/actions/actions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/actions/actions.py)\n- [packages/notte-core/src/notte_core/errors/processing.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/errors/processing.py)\n- [packages/notte-agent/src/notte_agent/gufo/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/system.md)\n- [packages/notte-agent/src/notte_agent/falco/prompt.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/falco/prompt.py)\n- [packages/notte-sdk/src/notte_sdk/endpoints/personas.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/personas.py)\n \n\n# Introduction to Notte\n\nNotte is a comprehensive software suite designed for internet-native agentic systems. It provides a robust framework for building, deploying, and managing AI agents capable of interacting with web content programmatically. The project is developed by Notte Labs, Inc. and licensed under the Server Side Public License v1.0.\n\n资料来源：[README.md](https://github.com/nottelabs/notte/blob/main/README.md)\n\n## Overview\n\nNotte transforms the internet into a structured, navigable space where each website becomes an accessible map for intelligent agents. The technology enables AI systems to interpret and interact with web content with precision, creating a programmatic layer over the web.\n\nThe suite offers multiple capabilities:\n\n- **Web Scraping**: Extract structured data from any website\n- **Browser Automation**: Navigate and interact with web pages programmatically\n- **Agent Framework**: Build AI agents that can perform complex web tasks\n- **Session Management**: Maintain stateful browsing sessions with cookie handling\n\n资料来源：[README.md](https://github.com/nottelabs/notte/blob/main/README.md), [packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n## Architecture\n\nThe Notte architecture consists of several interconnected packages:\n\n```mermaid\ngraph TD\n A[Client SDK] --> B[notte-core]\n A --> C[notte-llm]\n A --> D[notte-agent]\n B --> E[Actions Module]\n B --> F[Error Handling]\n C --> G[Prompts]\n C --> H[Data Extraction]\n D --> I[Falco Agent]\n D --> J[Gufo Agent]\n```\n\n### Core Packages\n\n| Package | Purpose |\n|---------|---------|\n| `notte-core` | Core actions, error handling, and browser interaction primitives |\n| `notte-llm` | LLM integration, prompts for document analysis, data extraction, and action generation |\n| `notte-agent` | Agent implementations (Falco, Gufo) with validation and execution logic |\n| `notte-sdk` | Python SDK for easy integration and API consumption |\n\n资料来源：[packages/notte-core/src/notte_core/actions/actions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/actions/actions.py), [packages/notte-core/src/notte_core/errors/processing.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/errors/processing.py)\n\n## Browser Actions\n\nNotte provides a comprehensive set of browser actions that agents can execute. Actions are defined as typed classes with execution messages and parameter validation.\n\n### Navigation Actions\n\n| Action | Description | Parameters |\n|--------|-------------|------------|\n| `goto` | Navigate to a URL | `url: str` |\n| `goto_new_tab` | Open URL in a new tab | `url: str` |\n| `close_tab` | Close the current tab | None |\n\n**Example Usage:**\n\n```python\nsession.execute(type=\"goto\", url=\"https://console.notte.cc\")\nsession.execute(type=\"goto_new_tab\", url=\"https://example.com\")\nsession.execute(type=\"close_tab\")\n```\n\n资料来源：[packages/notte-core/src/notte_core/actions/actions.py:1-100](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/actions/actions.py)\n\n## Session Management\n\nThe SDK provides a session-based interface for browser automation. Sessions maintain state across multiple interactions and support cookie persistence.\n\n### Session Lifecycle\n\n```mermaid\ngraph LR\n A[Start Session] --> B[Execute Actions]\n B --> C[Observe State]\n C --> B\n B --> D[Stop Session]\n D --> E[Save Cookies]\n```\n\n### Basic Session Usage\n\n```python\nfrom notte_sdk import NotteClient\n\nclient = NotteClient()\nwith client.Session() as session:\n session.execute(type=\"goto\", url=\"https://www.notte.cc\")\n obs = session.observe()\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n### Observation Types\n\nSessions support two perception modes for observing page state:\n\n| Mode | Description | Use Case |\n|------|-------------|----------|\n| `fast` | Simple page perception for quick queries | Basic element detection |\n| `deep` | LLM-powered formatting for rich action spaces | Complex interactions |\n\n```python\n# Fast observation\nobs = session.observe(perception_type='fast')\n\n# Deep observation for LLM-ready action space\nobs = session.observe(perception_type='deep')\nprint(obs.space.description)\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n### Cookie Management\n\nSessions automatically handle cookie persistence:\n\n```python\nclient = NotteClient(cookie_file=\"cookies.json\")\nwith client.Session() as session:\n # Cookies are loaded on start and saved on stop\n session.execute(type=\"goto\", url=\"https://example.com\")\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n## Agent System\n\nNotte includes sophisticated agent implementations for autonomous web navigation.\n\n### Action Identification System\n\nAgents identify interactive elements using a structured ID system:\n\n| Prefix | Element Type | Examples |\n|--------|--------------|----------|\n| `I` | Input fields | Textboxes, selects, checkboxes |\n| `B` | Buttons | Clickable buttons |\n| `L` | Links | Hypertext links |\n| `F` | Figures/Images | Visual elements |\n| `O` | Select options | Dropdown options |\n| `M` | Miscellaneous | Modals, dialogs |\n\n**ID Format:** `[:]` (e.g., `B1`, `I2`, `L3:button`)\n\n> **Note:** IDs can change at each step. Agents must not assume IDs persist across observations.\n\n资料来源：[packages/notte-agent/src/notte_agent/gufo/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/system.md)\n\n### CAPTCHA Handling\n\nAgents have built-in CAPTCHA detection and handling:\n\n- Never interact directly with CAPTCHA elements\n- Use the `captcha_solve` action when detection occurs\n- Supported types: reCAPTCHA, hCaptcha, image verification, checkbox verification\n\n```json\n{\n \"action\": \"captcha_solve\",\n \"captcha_type\": \"recaptcha\"\n}\n```\n\n资料来源：[packages/notte-agent/src/notte_agent/gufo/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/system.md)\n\n### Validation System\n\nThe agent framework includes a validation pipeline:\n\n```mermaid\ngraph TD\n A[Execute Action] --> B[Validate Output]\n B --> C{Has Observations?}\n C -->|No| D[Return Error]\n C -->|Yes| E[LLM Validation]\n E --> F{Is Valid?}\n F -->|Yes| G[Return Success]\n F -->|No| H[Return Failure]\n```\n\nThe validator uses vision models when available to verify action outcomes against expected results.\n\n资料来源：[packages/notte-agent/src/notte_agent/common/validator.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/common/validator.py)\n\n## Data Extraction\n\nNotte provides structured data extraction capabilities through LLM-powered document analysis.\n\n### Document Analysis Pipeline\n\n| Stage | Description |\n|-------|-------------|\n| Analysis | Identify sections, content types, and structured data |\n| Category | Classify document type (search-results, item, other) |\n| Extraction | Transform content into structured format |\n\n### Output Format\n\nExtracted data is organized into two sections:\n\n1. **``**: Logical breakdown of the document structure\n2. **``**: Structured Markdown output with tables and lists\n\n**Example Categories:**\n\n| Category | Use Case |\n|----------|----------|\n| `search-results` | Google Flights, search engine results |\n| `item` | Recipe pages, product details |\n| `other` | General content (Allrecipes homepage) |\n\n资料来源：[packages/notte-llm/src/notte_llm/prompts/document-category/base/user.md](https://github.com/nottelabs/notte/blob/main/packages/notte-llm/src/notte_llm/prompts/document-category/base/user.md)\n\n## API Integration\n\n### REST API Endpoint\n\n```bash\ncurl -X POST 'https://api.notte.cc/scrape' \\\n -H 'Authorization: Bearer ' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"url\": \"https://notte.cc\",\n \"only_main_content\": false\n }'\n```\n\n### SDK Client Usage\n\n```python\nfrom notte_sdk import NotteClient\nfrom pydantic import BaseModel\n\n# Basic scraping\nresponse = client.scrape(\n url=\"https://notte.cc\",\n scrape_links=True,\n only_main_content=True\n)\n\n# Structured scraping\nclass Article(BaseModel):\n title: str\n content: str\n date: str\n\nresponse = client.scrape(\n url=\"https://example.com/blog\",\n response_format=Article,\n instructions=\"Extract only the title, date and content of the articles\"\n)\n```\n\n资料来源：[README.md](https://github.com/nottelabs/notte/blob/main/README.md)\n\n## Personas\n\nNotte supports persona-based operations for enhanced privacy and automation:\n\n```python\nimport notte\n\npersona = notte.Persona(\"\")\nsms = persona.sms(only_unread=True)\n```\n\n### Available Operations\n\n| Method | Description |\n|--------|-------------|\n| `sms()` | Retrieve SMS messages for the persona |\n| `create_number()` | Create a phone number |\n| `delete_number()` | Delete the persona's phone number |\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/personas.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/personas.py)\n\n## Error Handling\n\nNotte defines a comprehensive error hierarchy for different failure scenarios:\n\n### Core Error Classes\n\n| Error | Description |\n|-------|-------------|\n| `InvalidA11yTreeType` | Invalid accessibility tree type |\n| `InvalidA11yChildrenError` | Invalid child element count |\n| `InvalidPlaceholderError` | Unhandled placeholder in vault |\n| `ScrapeFailedError` | Structured data extraction failure |\n\nAll errors provide developer advice and user-facing messages for appropriate handling.\n\n资料来源：[packages/notte-core/src/notte_core/errors/processing.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/errors/processing.py)\n\n## Search Demo\n\nNotte provides a live search demonstration using MCP server integration:\n\n- **Demo URL**: [https://search.notte.cc/](https://search.notte.cc/)\n- **Features**: Real-time search in LLM chatbots leveraging the scraping endpoint\n\n资料来源：[README.md](https://github.com/nottelabs/notte/blob/main/README.md)\n\n## License and Citation\n\nThis project is licensed under the **Server Side Public License v1.0 (SSPL-1.0)**.\n\nFor academic or commercial use, cite as:\n\n```bibtex\n@software{notte2025,\n author = {Pinto, Andrea and Giordano, Lucas and {nottelabs-team}},\n title = {Notte: Software suite for internet-native agentic systems},\n url = {https://github.com/nottelabs/notte},\n year = {2025},\n publisher = {GitHub},\n license = {SSPL-1.0},\n version = {1.4.4}\n}\n```\n\n资料来源：[README.md](https://github.com/nottelabs/notte/blob/main/README.md)\n\n---\n\n\n\n## Quickstart Guide\n\n### 相关页面\n\n相关主题：[Introduction to Notte](#introduction)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [pyproject.toml](https://github.com/nottelabs/notte/blob/main/pyproject.toml)\n- [.env.example](https://github.com/nottelabs/notte/blob/main/.env.example)\n- [docs/setup.md](https://github.com/nottelabs/notte/blob/main/docs/setup.md)\n- [examples/quickstart.py](https://github.com/nottelabs/notte/blob/main/examples/quickstart.py)\n \n\n# Quickstart Guide\n\nThe Quickstart Guide provides a streamlined path for developers to begin using the Notte SDK within minutes. It covers environment configuration, SDK initialization, and the fundamental workflows for web scraping and browser automation.\n\n## Prerequisites\n\nBefore starting, ensure your development environment meets the following requirements:\n\n| Requirement | Minimum Version | Notes |\n|-------------|-----------------|-------|\n| Python | 3.10+ | Required for type annotations and modern async features |\n| pip | 21.0+ | For package installation |\n\n## Environment Setup\n\n### 1. Obtain API Credentials\n\nRegister at [notte.cc](https://notte.cc) to obtain your API key. The service requires authentication via Bearer token for all API requests.\n\n### 2. Configure Environment Variables\n\nCreate a `.env` file in your project root with the following variables:\n\n```bash\nNOTTE_API_KEY=your_api_key_here\nNOTTE_API_URL=https://api.notte.cc # Optional, defaults to this value\n```\n\n资料来源：[.env.example:1-2]()\n\n### 3. Install the SDK\n\nInstall the Notte SDK using pip:\n\n```bash\npip install notte\n```\n\nFor additional providers or extras, install from the project root:\n\n```bash\npip install -e \".[providers]\"\n```\n\n资料来源：[pyproject.toml:1-50]()\n\n## Basic Usage\n\n### SDK Client Initialization\n\nInitialize the Notte client using environment variables or direct configuration:\n\n```python\nfrom notte import Notte\n\n# Using environment variables (recommended)\nclient = Notte()\n\n# Or with explicit parameters\nclient = Notte(\n api_key=\"your_api_key\",\n base_url=\"https://api.notte.cc\"\n)\n```\n\n### Simple Web Scraping\n\nPerform basic webpage scraping with minimal configuration:\n\n```python\nresponse = client.scrape(\n url=\"https://notte.cc\",\n scrape_links=True,\n only_main_content=True\n)\nprint(response.content)\n```\n\n资料来源：[examples/quickstart.py:1-20]()\n\n### Structured Data Extraction\n\nExtract structured data using Pydantic models for type-safe responses:\n\n```python\nfrom notte import BaseModel\n\nclass Article(BaseModel):\n title: str\n content: str\n date: str\n\nresponse = client.scrape(\n url=\"https://example.com/blog\",\n response_format=Article,\n instructions=\"Extract only the title, date and content of the articles\"\n)\n```\n\n资料来源：[README.md:1-30]()\n\n## Session-Based Automation\n\nFor complex interactions requiring multiple steps, use the Session API:\n\n```python\nwith client.session() as session:\n # Navigate to a page\n session.execute(type=\"goto\", url=\"https://example.com\")\n \n # Observe available actions\n obs = session.observe(perception_type=\"deep\")\n \n # Execute form filling or clicking actions\n session.execute(type=\"click\", id=\"B1\")\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py:1-50]()\n\n## API Reference\n\n### Client Configuration Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `api_key` | `str` | Yes | - | Your Notte API key |\n| `base_url` | `str` | No | `https://api.notte.cc` | Base URL for API requests |\n| `timeout` | `int` | No | `60` | Request timeout in seconds |\n\n### Scrape Parameters\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| `url` | `str` | Yes | - | Target URL to scrape |\n| `only_main_content` | `bool` | No | `True` | Exclude navbars and footers |\n| `scrape_links` | `bool` | No | `True` | Include hyperlinks in response |\n| `response_format` | `BaseModel` | No | `None` | Pydantic model for structured output |\n| `instructions` | `str` | No | `None` | Natural language extraction instructions |\n\n## Workflow Diagram\n\n```mermaid\ngraph TD\n A[Start] --> B[Install SDK]\n B --> C[Configure API Key]\n C --> D{Use Case}\n D -->|Simple Scrape| E[client.scrape]\n D -->|Structured Data| F[Define BaseModel]\n F --> G[client.scrape with response_format]\n D -->|Complex Automation| H[Create Session]\n H --> I[Observe Actions]\n I --> J[Execute Actions]\n J --> K[Return Results]\n E --> L[End]\n G --> L\n K --> L\n```\n\n## cURL Alternative\n\nFor environments without Python, use the REST API directly:\n\n```bash\ncurl -X POST 'https://api.notte.cc/scrape' \\\n -H 'Authorization: Bearer ' \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"url\": \"https://notte.cc\",\n \"only_main_content\": false\n }'\n```\n\n资料来源：[README.md:40-50]()\n\n## Next Steps\n\n- Review the [Setup Documentation](https://github.com/nottelabs/notte/blob/main/docs/setup.md) for advanced configuration\n- Explore the [Examples Directory](https://github.com/nottelabs/notte/tree/main/examples) for complete use cases\n- Check the [Agent Documentation](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/README.md) for browser automation with AI agents\n\n---\n\n\n\n## System Architecture\n\n### 相关页面\n\n相关主题：[Introduction to Notte](#introduction), [Agent Core System](#agent-core), [Browser Sessions](#sessions)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/notte-agent/src/notte_agent/__init__.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/__init__.py)\n- [packages/notte-browser/src/notte_browser/__init__.py](https://github.com/nottelabs/notte/blob/main/packages/notte-browser/src/notte_browser/__init__.py)\n- [packages/notte-core/src/notte_core/__init__.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/__init__.py)\n- [packages/notte-sdk/src/notte_sdk/__init__.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/__init__.py)\n- [packages/notte-llm/src/notte_llm/__init__.py](https://github.com/nottelabs/notte/blob/main/packages/notte-llm/src/notte_llm/__init__.py)\n \n\n# System Architecture\n\nNotte is a software suite designed for internet-native agentic systems, providing a comprehensive infrastructure for browser automation, web interaction, and AI-driven document processing. The architecture follows a modular design pattern with distinct layers for browser control, agent orchestration, LLM integration, and SDK accessibility.\n\n## Overview\n\nThe Notte system is composed of five primary packages that work together to enable autonomous web interaction:\n\n| Package | Purpose |\n|---------|---------|\n| `notte-core` | Core utilities, error handling, and shared data structures |\n| `notte-browser` | Browser interaction and CDP integration layer |\n| `notte-agent` | AI agent orchestration (Gufo and Falco subsystems) |\n| `notte-llm` | LLM prompt management and document processing |\n| `notte-sdk` | Public SDK for client applications |\n\n## High-Level Architecture\n\n```mermaid\ngraph TD\n Client[Client Application]\n SDK[notte-sdk]\n Agent[notte-agent]\n Browser[notte-browser]\n LLM[notte-llm]\n Core[notte-core]\n CDP[Chrome DevTools Protocol]\n Remote[Remote Browser]\n\n Client --> SDK\n SDK --> Agent\n SDK --> LLM\n Agent --> Browser\n Browser --> CDP\n CDP --> Remote\n LLM --> Core\n Core --> Browser\n```\n\n## Core Package (`notte-core`)\n\nThe `notte-core` package provides foundational components used across all other packages, including error handling, accessibility tree processing, and placeholder management.\n\n### Error Handling Architecture\n\nThe error system is built on a hierarchical class structure rooted in `NotteBaseError`:\n\n```python\n# Source: packages/notte-core/src/notte_core/errors/processing.py\nclass NotteBaseError(Exception):\n def __init__(self, agent_message, user_message, dev_message)\n```\n\n**Error Categories:**\n\n| Error Class | Purpose |\n|-------------|---------|\n| `InvalidInternalCheckError` | Internal validation failures with developer guidance |\n| `InvalidA11yTreeType` | Unsupported accessibility tree format |\n| `InvalidA11yChildrenError` | Accessibility tree structure violations |\n| `InvalidPlaceholderError` | Vault placeholder resolution failures |\n| `ScrapeFailedError` | Structured data extraction failures |\n\n### Placeholder System\n\nThe placeholder system enables secure credential management through a vault mechanism. When an action requires sensitive data, the system substitutes placeholders that are resolved at runtime.\n\n```python\n# Source: packages/notte-core/src/notte_core/errors/processing.py\nclass InvalidPlaceholderError(NotteBaseError):\n def __init__(self, placeholder: str) -> None:\n dev_message = f\"The placeholder {placeholder} is not handled by your current vault.\"\n agent_message = f\"Could not perform action with value {placeholder}. Try picking a different value\"\n```\n\n## SDK Package (`notte-sdk`)\n\nThe SDK provides the primary interface for client applications to interact with the Notte system. It exposes session management, observation capabilities, and action execution through a Pythonic API.\n\n### Session Management\n\nSessions are the fundamental unit of work in Notte, representing a single browser session with associated state and context.\n\n```python\n# Source: packages/notte-sdk/src/notte_sdk/endpoints/sessions.py\nclass Session:\n def __init__(self, ..., timeout_minutes: int = ...):\n self.response = None\n self._cookie_file = None\n```\n\n**Session Lifecycle:**\n\n| State | Description |\n|-------|-------------|\n| `created` | Session object instantiated |\n| `started` | `client.start()` called, `session_id` available |\n| `active` | Browser operations in progress |\n| `stopped` | `client.stop()` called, session terminated |\n\n### Observation System\n\nThe observation system retrieves the current state of the webpage and available interactive elements:\n\n```python\n# Source: packages/notte-sdk/src/notte_sdk/endpoints/sessions.py\ndef observe(self, *, perception_type: str = None, instructions: str = None, **data):\n if data.get(\"perception_type\") is None:\n data[\"perception_type\"] = self.default_perception_type\n return self.client.page.observe(session_id=self.session_id, **data)\n```\n\n**Perception Types:**\n\n| Type | Description | Use Case |\n|------|-------------|----------|\n| `fast` | Simple page perception for quick queries | Default, rapid action space generation |\n| `deep` | LLM-powered element formatting | Complex pages requiring structured analysis |\n\n### Action Execution\n\nActions are executed through the unified `execute()` method with type-based dispatch:\n\n```python\n# Source: packages/notte-sdk/src/notte_sdk/endpoints/sessions.py\ndef execute(self, *, raise_on_failure: bool = None, **kwargs: Unpack[FormFillActionDict]) -> ExecutionResult\n```\n\n### Cookie Management\n\nSessions automatically persist cookies to file for session continuity:\n\n```python\n# Source: packages/notte-sdk/src/notte_sdk/endpoints/sessions.py\nif self._cookie_file is not None:\n cookies = self.get_cookies()\n create_or_append_cookies_to_file(self._cookie_file, cookies)\n```\n\n## Agent Package (`notte-agent`)\n\nThe agent package contains two distinct agent subsystems: **Gufo** and **Falco**. Both subsystems handle browser automation but use different prompt strategies and action registries.\n\n### Agent Subsystem Comparison\n\n| Aspect | Gufo | Falco |\n|--------|------|-------|\n| System Prompt | `gufo/system.md` | `falco/system.md` |\n| Element Format | Markdown with backticks | `id[:]text` |\n| Tools | Configurable via `BaseTool` | Configurable via `BaseTool` |\n| Action Registry | Custom implementation | `ActionRegistry` class |\n\n### Element Identification System\n\nBoth agents use a consistent element identification scheme for interactive elements:\n\n| Prefix | Element Type | Examples |\n|--------|--------------|----------|\n| `I` | Input fields | Textbox, select, checkbox, radio |\n| `B` | Buttons | Submit, clickable elements |\n| `L` | Links | Hypertext navigation |\n| `F` | Figures/Images | Visual content |\n| `O` | Options | Select dropdown items |\n| `M` | Miscellaneous | Modals, dialogs, overlays |\n\n```json\n{\n \"id\": \"I1\",\n \"type\": \"input\",\n \"label\": \"email\",\n \"value\": \"user@example.com\"\n}\n```\n\n### Gufo Agent System\n\nThe Gufo agent (`packages/notte-agent/src/notte_agent/gufo/system.md`) operates through structured JSON commands:\n\n```json\n{\n \"actions\": [{\"type\": \"click\", \"id\": \"B1\"}],\n \"reasoning\": \"User wants to submit the form\"\n}\n```\n\n### Falco Agent System\n\nThe Falco agent (`packages/notte-agent/src/notte_agent/falco/prompt.py`) uses a prompt-based approach with configurable tools:\n\n```python\n# Source: packages/notte-agent/src/notte_agent/falco/prompt.py\nclass FalcoPrompt(BasePrompt):\n def __init__(\n self,\n prompt_file: Path | None = None,\n tools: list[BaseTool] | None = None,\n ) -> None:\n self.action_registry: ActionRegistry = ActionRegistry(tools)\n```\n\n### CAPTCHA Handling\n\nBoth agents implement strict CAPTCHA detection and handling:\n\n```python\n# Source: packages/notte-agent/src/notte_agent/gufo/system.md\n# CAPTCHA HANDLING - CRITICAL RULES:\n# - NEVER click on captcha elements directly\n# - NEVER use \"click\", \"type\", or any other action on captcha elements\n# - If detected, use ONLY the \"captcha_solve\" action\n```\n\n### Action Examples\n\n**Form Filling:**\n```json\n// Source: packages/notte-agent/src/notte_agent/falco/prompt.py\n{\n \"type\": \"form_fill\",\n \"value\": {\n \"address1\": \"\",\n \"city\": \"\",\n \"state\": \"\"\n }\n}\n```\n\n**Navigation and Extraction:**\n```json\n{\n \"type\": \"scrape\",\n \"instructions\": \"Extract the search results from the page\"\n}\n```\n\n## LLM Package (`notte-llm`)\n\nThe LLM package manages prompts and processing for document analysis, categorization, and structured data extraction.\n\n### Prompt Categories\n\n| Category | Purpose | Output Format |\n|----------|---------|---------------|\n| `document-category` | Classify web documents | `type` |\n| `data-extraction` | Extract structured data | Markdown with sections |\n| `action-listing` | List available actions | JSON action array |\n| `extract-without-json-schema` | LLM-native extraction | Structured JSON |\n\n### Document Categorization\n\nDocuments are classified into categories for downstream processing:\n\n| Category | Description | Example |\n|----------|-------------|---------|\n| `search-results` | Search engine results page | Google search |\n| `item` | Individual item/product page | Recipe, product listing |\n| `other` | Uncategorized content | General pages |\n\n```markdown\n\nother\n```\n\n### Data Extraction Templates\n\nThe system supports multiple extraction formats:\n\n| Template | Sections | Use Case |\n|----------|----------|----------|\n| `two_sections` | ``, `` | Standard extraction |\n| `all_data` | Analysis + detailed extraction | Comprehensive data |\n| `user.md` (base) | Custom format | Flexible extraction |\n\n### Structured Output Generation\n\n```markdown\n\n\nFound X menus, Y text elements, Z interactive elements\n[Analysis content...]\n\n\n[Extracted data in Markdown format...]\n\n```\n\n## Browser Package (`notte-browser`)\n\nThe browser package provides the low-level interface to the Chrome DevTools Protocol (CDP) for controlling headless browsers.\n\n### Key Responsibilities\n\n- Page navigation and loading\n- Element interaction (click, type, scroll)\n- Screenshot capture\n- Accessibility tree generation\n- Cookie management\n\n### CDP Integration\n\n```python\n# Source: packages/notte-sdk/README.md\nfrom patchright.sync_api import sync_playwright\nfrom notte_sdk import NotteClient\n\nwith notte.Session() as session:\n # Browser operations via CDP\n _ = session.execute(type=\"goto\", url=\"https://example.com\")\n```\n\n## Data Flow Architecture\n\n```mermaid\ngraph LR\n User[User Request]\n SDK[SDK Session]\n Agent[Agent Processor]\n LLM[LLM Processing]\n Browser[Browser Engine]\n CDP[CDP Commands]\n \n User --> SDK\n SDK --> Agent\n Agent --> LLM\n LLM --> Agent\n Agent --> Browser\n Browser --> CDP\n CDP --> Browser\n Browser --> Agent\n Agent --> SDK\n SDK --> User\n```\n\n## Session State Machine\n\n```mermaid\ngraph TD\n Init[Session Created] --> Start[client.start]\n Start --> Active[Session Active]\n Active --> Observe[session.observe]\n Active --> Execute[session.execute]\n Observe --> Active\n Execute --> Active\n Active --> Stop[client.stop]\n Stop --> End[Session Ended]\n \n Start -->|Error| Error[Error State]\n Error -->|Retry| Start\n```\n\n## Configuration Options\n\n### SDK Initialization\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `timeout_minutes` | `int` | - | Session timeout |\n| `open_viewer` | `bool` | `False` | Open browser viewer |\n| `proxies` | `dict` | `None` | Proxy configuration |\n| `_cookie_file` | `str` | `None` | Cookie persistence file |\n\n### Perception Configuration\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `perception_type` | `str` | `fast` or `deep` |\n| `instructions` | `str` | Natural language filtering |\n\n## Error Handling Flow\n\n```mermaid\ngraph TD\n Action[Action Request]\n Validate{Validation}\n Validate -->|Pass| Execute\n Validate -->|Fail| InvalidError[InvalidInternalCheckError]\n \n Execute --> CDP[CDP Call]\n CDP -->|Success| Result\n CDP -->|Failure| ScrapeError[ScrapeFailedError]\n \n Placeholder{Placeholder Check}\n Result --> Placeholder\n Placeholder -->|Found| PlaceholderError[InvalidPlaceholderError]\n Placeholder -->|None| Complete\n```\n\n## Integration Examples\n\n### Basic Session Usage\n\n```python\n# Source: packages/notte-sdk/README.md\nfrom notte_sdk import NotteClient\n\nclient = NotteClient()\nwith client.Session() as session:\n session.execute(type=\"goto\", url=\"https://example.com\")\n obs = session.observe()\n action = obs.space.sample(type='click')\n result = session.execute(action)\n```\n\n### Agent Deployment\n\n```python\n# Source: packages/notte-sdk/README.md\nwith notte.Session(open_viewer=True) as session:\n agent = notte.Agent(session=session)\n agent.start(\n task=\"Summarize the content of the page\",\n url=\"https://www.google.com\"\n )\n```\n\n## Summary\n\nThe Notte architecture provides a robust, layered approach to browser automation:\n\n1. **Core Layer** (`notte-core`): Provides shared utilities, error handling, and base data structures\n2. **Browser Layer** (`notte-browser`): Abstracts Chrome DevTools Protocol for browser control\n3. **Agent Layer** (`notte-agent`): Implements AI-driven automation with Gufo and Falco subsystems\n4. **LLM Layer** (`notte-llm`): Manages document processing and prompt engineering\n5. **SDK Layer** (`notte-sdk`): Exposes the complete API to client applications\n\nThe modular design allows each layer to be used independently or in combination, enabling flexible deployment scenarios from simple web scraping to complex autonomous agent workflows.\n\n---\n\n\n\n## Agent Core System\n\n### 相关页面\n\n相关主题：[Structured Output](#structured-output), [Agent Fallback System](#agent-fallback), [Browser Sessions](#sessions)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/notte-agent/src/notte_agent/agent.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/agent.py)\n- [packages/notte-agent/src/notte_agent/common/conversation.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/common/conversation.py)\n- [packages/notte-agent/src/notte_agent/common/perception.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/common/perception.py)\n- [packages/notte-agent/src/notte_agent/falco/agent.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/falco/agent.py)\n- [packages/notte-agent/src/notte_agent/gufo/agent.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/agent.py)\n- [packages/notte-core/src/notte_core/agent_types.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/agent_types.py)\n \n\n# Agent Core System\n\nThe Agent Core System is the central orchestration layer in Notte that enables AI agents to autonomously navigate and interact with web pages. It provides a unified interface for browser automation tasks, handling perception of web elements, action execution, and conversation management between the agent and web content.\n\n## Architecture Overview\n\nThe Agent Core System consists of multiple layered components that work together to enable autonomous web interaction.\n\n```mermaid\ngraph TD\n A[Agent Client] --> B[Agent Core]\n B --> C[Browser Session]\n C --> D[Web Page]\n B --> E[Perception Module]\n B --> F[Conversation Module]\n E --> G[Action Registry]\n G --> H[Falco Actions]\n G --> I[Gufo Actions]\n```\n\n## Agent Types\n\nNotte supports multiple agent implementations, each designed for specific automation scenarios. The agent type determines the underlying action execution engine and available capabilities.\n\n| Agent Type | Description | Use Case |\n|------------|-------------|----------|\n| `falco` | Standard browser automation agent | General web interaction, form filling, navigation |\n| `gufo` | Advanced automation with stealth features | CAPTCHA handling, proxy rotation, anti-detection |\n\n资料来源：[packages/notte-core/src/notte_core/agent_types.py:1-50]()\n\n## Core Components\n\n### Agent Base Class\n\nThe base `Agent` class provides the primary interface for task execution and state management.\n\n```python\nclass Agent:\n def __init__(\n self,\n session: Session,\n vault: Vault | None = None,\n max_steps: int = 10,\n agent_type: AgentType = AgentType.FALCO,\n )\n \n def run(self, task: str) -> AgentResponse\n def start(self, task: str, url: str | None = None) -> None\n def status() -> AgentStatus\n def stop() -> None\n```\n\n**Key Parameters:**\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `session` | `Session` | Required | Browser session for interaction |\n| `vault` | `Vault \\| None` | `None` | Secure credential storage |\n| `max_steps` | `int` | `10` | Maximum action steps before termination |\n| `agent_type` | `AgentType` | `FALCO` | Backend engine selection |\n\n资料来源：[packages/notte-agent/src/notte_agent/agent.py:1-100]()\n\n### Perception Module\n\nThe perception module analyzes web page structure and generates interactive action spaces for the agent.\n\n```python\nclass Perception:\n def observe(\n self,\n perception_type: str = 'fast',\n instructions: str | None = None\n ) -> ObservationResponse\n```\n\n**Perception Types:**\n\n| Type | Description | Performance |\n|------|-------------|-------------|\n| `fast` | Simple page element parsing | Low latency |\n| `deep` | LLM-powered element formatting | Higher accuracy, slower |\n\nThe perception system generates element IDs using a role-based prefix system:\n\n- `I` - Input fields (textbox, select, checkbox)\n- `B` - Buttons\n- `L` - Links\n- `F` - Figures/Images\n- `O` - Select options\n- `M` - Miscellaneous elements (modals, dialogs)\n\n资料来源：[packages/notte-agent/src/notte_agent/common/perception.py:1-80]()\n\n### Conversation Module\n\nManages the dialogue history between the agent and web content, maintaining context across multiple interaction steps.\n\n```python\nclass Conversation:\n def __init__(self, system_prompt: str)\n def add_user_message(self, content: str) -> None\n def add_agent_message(self, content: str) -> None\n def get_messages() -> list[Message]\n```\n\nThe conversation system tracks:\n- User task requests\n- Agent reasoning and decisions\n- Action execution results\n- Page observations\n\n资料来源：[packages/notte-agent/src/notte_agent/common/conversation.py:1-60]()\n\n## Action System\n\n### Action Types\n\nThe agent supports a comprehensive set of browser automation actions through a registry pattern.\n\n```mermaid\ngraph LR\n A[Agent Decision] --> B[Action Registry]\n B --> C[FormFillAction]\n B --> D[ClickAction]\n B --> E[ScrapeAction]\n B --> F[CaptchaSolveAction]\n B --> G[GotoAction]\n```\n\n| Action | Description | Parameters |\n|--------|-------------|------------|\n| `goto` | Navigate to URL | `url: str` |\n| `click` | Click element | `id: str` |\n| `fill` | Fill form fields | `value: dict[str, str]` |\n| `scrape` | Extract structured data | `instructions: str` |\n| `captcha_solve` | Solve CAPTCHA | `captcha_type: str` |\n\n资料来源：[packages/notte-agent/src/notte_agent/falco/prompt.py:1-150]()\n\n### Action Registry\n\nThe `ActionRegistry` maintains available actions and their schemas, enabling dynamic action discovery.\n\n```python\nclass ActionRegistry:\n def __init__(self, tools: list[BaseTool])\n def get_action_schemas(self) -> list[ActionSchema]\n def register(self, action_cls: type[BaseTool]) -> None\n```\n\n### Supported Action Formats\n\nActions are serialized using JSON schema format for agent consumption:\n\n```json\n{\n \"type\": \"object\",\n \"properties\": {\n \"id\": {\"type\": \"string\", \"description\": \"Element identifier\"},\n \"value\": {\"type\": \"string\", \"description\": \"Action value\"}\n }\n}\n```\n\n资料来源：[packages/notte-agent/src/notte_agent/falco/prompt.py:30-80]()\n\n## Agent Implementations\n\n### Falco Agent\n\nThe default agent implementation using standard Playwright-based browser automation.\n\n```python\nclass FalcoAgent(BaseAgent):\n def __init__(self, tools: list[BaseTool] | None = None)\n def execute(self, action: dict) -> ExecutionResult\n```\n\n**Features:**\n- Standard form filling\n- Click-based navigation\n- Basic scraping operations\n- Simple CAPTCHA detection\n\n资料来源：[packages/notte-agent/src/notte_agent/falco/agent.py:1-100]()\n\n### Gufo Agent\n\nAdvanced agent with stealth capabilities for bypassing detection systems.\n\n```python\nclass GufoAgent(BaseAgent):\n def __init__(self, tools: list[BaseTool] | None = None)\n def execute(self, action: dict) -> ExecutionResult\n```\n\n**Features:**\n- Automatic CAPTCHA solving\n- Proxy rotation support\n- User agent spoofing\n- Cookie management\n\n资料来源：[packages/notte-agent/src/notte_agent/gufo/agent.py:1-100]()\n\n## Session Integration\n\nAgents operate within browser sessions that provide the execution environment.\n\n```python\nwith client.Session(browser_type=\"chrome\", open_viewer=True) as session:\n agent = client.Agent(session=session, max_steps=15)\n response = agent.run(\n task=\"Navigate to the form and submit with sample data\",\n url=\"https://example.com/form\"\n )\n```\n\n### Session Configuration\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `browser_type` | `str` | `\"chrome\"` | Browser engine |\n| `open_viewer` | `bool` | `False` | Display browser window |\n| `timeout_minutes` | `int` | `5` | Session timeout |\n| `proxies` | `bool \\| str` | `False` | Proxy configuration |\n| `solve_captchas` | `bool` | `False` | Auto-CAPTCHA solving |\n\n## Workflow Execution\n\n```mermaid\nsequenceDiagram\n participant User\n participant Agent\n participant Session\n participant Browser\n User->>Agent: run(task)\n Agent->>Session: observe()\n Session->>Browser: Get page state\n Browser-->>Session: Page elements\n Session-->>Agent: Observation\n Agent->>Agent: Plan action\n Agent->>Session: execute(action)\n Session->>Browser: Perform action\n Browser-->>Session: Result\n Session-->>Agent: ExecutionResult\n Agent->>Agent: Check completion\n Note over Agent,Browser: Loop until task complete or max_steps\n```\n\n## Error Handling\n\nThe system provides structured error handling for various failure scenarios.\n\n| Error Class | Description | Resolution |\n|-------------|-------------|------------|\n| `InvalidPlaceholderError` | Vault credential unavailable | Select alternative value |\n| `ScrapeFailedError` | Data extraction failed | Retry with different instructions |\n| `InvalidA11yTreeType` | Unknown accessibility tree type | Check code implementation |\n| `InvalidA11yChildrenError` | Element hierarchy mismatch | Verify page structure |\n\n资料来源：[packages/notte-core/src/notte_core/errors/processing.py:1-100]()\n\n## SDK Usage Example\n\n```python\nfrom notte_sdk import NotteClient\n\nclient = NotteClient()\n\n# Basic agent usage\nwith client.Session(open_viewer=True) as session:\n agent = client.Agent(session=session, max_steps=10)\n response = agent.run(\n task=\"Find the search box and search for 'python tutorials'\",\n url=\"https://www.google.com\"\n )\n print(response.answer)\n\n# With persona (digital identity)\nwith client.Persona(create_phone_number=True) as persona:\n with client.Session(browser_type=\"chrome\") as session:\n agent = client.Agent(session=session, persona=persona, max_steps=15)\n response = agent.run(\n task=\"Complete the registration form\",\n url=\"https://example.com/register\"\n )\n```\n\n## Best Practices\n\n1. **Set appropriate max_steps** - Balance between task completion and resource usage\n2. **Use fast perception** for simple tasks, deep perception for complex page analysis\n3. **Implement vault storage** for reusable credentials across sessions\n4. **Handle errors gracefully** - Check execution results before proceeding\n5. **Use stealth features** when bypassing detection is required\n\n---\n\n\n\n## Structured Output\n\n### 相关页面\n\n相关主题：[Agent Core System](#agent-core)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/notte-core/src/notte_core/utils/pydantic_schema.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/utils/pydantic_schema.py)\n- [packages/notte-browser/src/notte_browser/scraping/schema.py](https://github.com/nottelabs/notte/blob/main/packages/notte-browser/src/notte_browser/scraping/schema.py)\n- [packages/notte-llm/src/notte_llm/prompts/generate-json-schema/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-llm/src/notte_llm/prompts/generate-json-schema/system.md)\n- [packages/notte-llm/src/notte_llm/prompts/extract-without-json-schema/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-llm/src/notte_llm/prompts/extract-without-json-schema/system.md)\n \n\n# Structured Output\n\nStructured Output enables Notte to extract web page content and return it as typed, structured data using Pydantic models. This feature bridges the gap between unstructured web content and programmatic data processing, allowing developers to define expected output schemas and receive validated, type-safe data.\n\n## Overview\n\nNotte's Structured Output system consists of two complementary approaches:\n\n1. **Schema-Based Extraction**: Uses dynamically generated JSON schemas from Pydantic models\n2. **Natural Language Extraction**: Uses instructions-based extraction without strict schema enforcement\n\nThe system leverages Pydantic for schema definition, ensuring type safety and validation at the application layer. When a `response_format` is provided, Notte generates a corresponding JSON Schema that guides the LLM in producing correctly structured output.\n\n资料来源：[packages/notte-core/src/notte_core/utils/pydantic_schema.py](packages/notte-core/src/notte_core/utils/pydantic_schema.py)\n\n## Architecture\n\n```mermaid\ngraph TD\n A[User Request] --> B[Define Pydantic Model]\n B --> C[response_format Parameter]\n C --> D{Schema Type}\n D -->|With Schema| E[Generate JSON Schema]\n D -->|Without Schema| F[Instructions-Only Extraction]\n E --> G[Prompt Engineering]\n F --> G\n G --> H[LLM Processing]\n H --> I[Output Validation]\n I --> J[Typed Response]\n```\n\n## Core Components\n\n### Pydantic Schema Generation\n\nThe `pydantic_schema.py` module provides utilities for converting Python Pydantic models into JSON schemas that can be consumed by LLMs.\n\n| Function | Purpose |\n|----------|---------|\n| `model_to_json_schema()` | Converts Pydantic model class to JSON schema |\n| `validate_response()` | Validates LLM output against expected schema |\n| `extract_structured_data()` | Extracts and parses structured data from response |\n\n资料来源：[packages/notte-core/src/notte_core/utils/pydantic_schema.py](packages/notte-core/src/notte_core/utils/pydantic_schema.py)\n\n### Scraping Schema\n\nThe `schema.py` module defines the scraping configuration and response handling for structured output.\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `response_format` | `type[BaseModel]` | `None` | Pydantic model for structured output |\n| `instructions` | `str` | `None` | Natural language extraction instructions |\n| `raise_on_failure` | `bool` | `True` | Raise exception on extraction failure |\n\n资料来源：[packages/notte-browser/src/notte_browser/scraping/schema.py](packages/notte-browser/src/notte_browser/scraping/schema.py)\n\n## Usage Patterns\n\n### Basic Structured Extraction\n\n```python\nfrom notte_sdk import NotteClient\nfrom pydantic import BaseModel\n\nclass Product(BaseModel):\n title: str\n price: str\n description: str | None = None\n\nclient = NotteClient()\ndata = client.scrape(\n url=\"https://example.com/product\",\n response_format=Product,\n instructions=\"Extract the product title, price, and description\"\n)\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/client.py](packages/notte-sdk/src/notte_sdk/client.py)\n\n### Session-Based Extraction\n\n```python\nfrom notte_sdk import NotteClient\nfrom pydantic import BaseModel\n\nclass Article(BaseModel):\n title: str\n content: str\n date: str\n\nclient = NotteClient()\nwith client.Session() as session:\n session.execute(type=\"goto\", url=\"https://example.com/blog\")\n data = session.scrape(\n response_format=Article,\n instructions=\"Extract the title, date and content of the articles\"\n )\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n### Error Handling\n\n```python\nfrom notte_sdk import NotteClient\nfrom notte_core.errors import ScrapeFailedError\n\nclient = NotteClient()\n\n# With raise_on_failure=False, returns StructuredData wrapper\nresult = client.scrape(\n url=\"https://example.com\",\n response_format=Product,\n raise_on_failure=False\n)\n\nif not result.success:\n print(f\"Extraction failed: {result.error}\")\nelse:\n data = result.data\n```\n\n资料来源：[packages/notte-core/src/notte_core/errors/processing.py](packages/notte-core/src/notte_core/errors/processing.py)\n\n## Prompt Engineering\n\n### JSON Schema Generation Prompt\n\nThe `generate-json-schema/system.md` template guides the LLM in producing valid JSON output conforming to a specified schema. This prompt includes:\n\n- Success examples showing correct JSON output\n- Failure examples demonstrating invalid output handling\n- Timestamp context for time-sensitive extraction\n\n```markdown\nToday is: {{timestamp}}\n\nTransform the following document into structured JSON output based on the provided user request:\n\n```markdown\n{{& document}}\n```\n```\n\n资料来源：[packages/notte-llm/src/notte_llm/prompts/generate-json-schema/system.md](packages/notte-llm/src/notte_llm/prompts/generate-json-schema/system.md)\n\n### Extraction Without Schema\n\nThe `extract-without-json-schema/system.md` template provides an alternative approach for natural language extraction:\n\n```markdown\n```json\n{{& success_example}}\n```\n\nExample of a valid output if you cannot answer the user request:\n```json\n{{& failure_example}}\n```\n```\n\nThis approach allows flexible extraction when strict schema conformance is not required.\n\n资料来源：[packages/notte-llm/src/notte_llm/prompts/extract-without-json-schema/system.md](packages/notte-llm/src/notte_llm/prompts/extract-without-json-schema/system.md)\n\n## Scrape Action Configuration\n\nThe underlying scrape action provides granular control over extraction:\n\n```python\nsession.execute(type=\"scrape\", only_images=True) # Scrape only images\nsession.execute(type=\"scrape\", response_format={\"type\": \"object\", \"properties\": {...}}) # With JSON schema\n```\n\n| Action Parameter | Description |\n|-----------------|-------------|\n| `instructions` | Natural language instructions for extraction |\n| `only_main_content` | Exclude navbars, footers (default: `True`) |\n| `selector` | Playwright selector to scope extraction |\n| `only_images` | Extract images only |\n| `scrape_links` | Include links in output (default: `True`) |\n| `scrape_images` | Include image data |\n\n资料来源：[packages/notte-core/src/notte_core/actions/actions.py](packages/notte-core/src/notte_core/actions/actions.py)\n\n## Workflow\n\n```mermaid\nsequenceDiagram\n participant User\n participant SDK as NotteClient\n participant LLM as LLM Engine\n participant API as Notte API\n\n User->>SDK: scrape(url, response_format=Model)\n SDK->>LLM: Generate JSON Schema from Model\n SDK->>API: POST /scrape with schema + instructions\n API->>LLM: Process page with schema\n LLM-->>API: Structured JSON response\n API-->>SDK: Validated response\n SDK-->>User: Typed Model instance\n```\n\n## Best Practices\n\n1. **Define Clear Schemas**: Use descriptive field names and include type annotations\n2. **Provide Contextual Instructions**: Give the LLM context about what to extract\n3. **Handle Optional Fields**: Use `| None` for fields that may not always be present\n4. **Validate Output**: Enable `raise_on_failure=True` for production use\n5. **Scope Extraction**: Use selectors when extracting from specific page regions\n\n## Error Handling\n\n| Error Type | Cause | Resolution |\n|------------|-------|------------|\n| `ScrapeFailedError` | Extraction validation failed | Check instructions and schema compatibility |\n| `LLMParsingError` | Malformed JSON in response | Ensure schema is properly generated |\n| `InvalidPlaceholderError` | Missing credential reference | Configure vault for required credentials |\n\n资料来源：[packages/notte-core/src/notte_core/errors/processing.py](packages/notte-core/src/notte_core/errors/processing.py)\n\n---\n\n\n\n## Agent Fallback System\n\n### 相关页面\n\n相关主题：[Agent Core System](#agent-core), [Actions and Browser Controls](#actions-controls)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/notte-agent/src/notte_agent/agent_fallback.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/agent_fallback.py)\n- [packages/notte-sdk/src/notte_sdk/agent_fallback.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/agent_fallback.py)\n- [packages/notte-agent/src/notte_agent/falco/prompt.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/falco/prompt.py)\n- [packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n- [packages/notte-agent/src/notte_agent/gufo/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/system.md)\n \n\n# Agent Fallback System\n\n## Overview\n\nThe Agent Fallback System in Notte provides a robust mechanism for handling agent execution failures and gracefully recovering from errors during browser automation tasks. When an agent encounters execution issues—whether due to captcha detection, action failures, or other runtime errors—the fallback system intercepts these failures, analyzes the error context, and determines the appropriate recovery strategy.\n\nThe system operates across two primary layers: the SDK layer (`notte_sdk`) and the agent layer (`notte_agent`), ensuring consistent error handling and recovery whether agents are executed locally or through the Notte API.\n\n## Architecture\n\n```mermaid\ngraph TD\n A[Agent Execution] --> B{Action Execution Success?}\n B -->|Yes| C[Continue Normal Flow]\n B -->|No| D[Fallback System Triggered]\n D --> E{Error Type Classification}\n E -->|Captcha| F[Captcha Handler]\n E -->|Action Failure| G[Retry Strategy]\n E -->|Fatal Error| H[Graceful Degradation]\n F --> I[Attempt Resolution]\n G --> J[Apply Retry Policy]\n H --> K[Report to User]\n I --> C\n J --> C\n K --> L[Session Cleanup]\n```\n\n## Core Components\n\n### FallbackAction Class\n\nThe `FallbackAction` class represents the fundamental unit of fallback handling. It encapsulates the error context and provides structured data for downstream processing.\n\n| Property | Type | Description |\n|----------|------|-------------|\n| `error_type` | `str` | Classification of the error (e.g., \"captcha\", \"action_failure\") |\n| `error_message` | `str` | Detailed description of the failure |\n| `metadata` | `dict` | Additional context including HTML element data, playwright code, and execution state |\n| `retry_count` | `int` | Number of retry attempts performed |\n| `timestamp` | `datetime` | When the error occurred |\n\n资料来源：[packages/notte-agent/src/notte_agent/agent_fallback.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/agent_fallback.py)\n\n### AgentFallbackManager\n\nThe `AgentFallbackManager` serves as the central coordinator for fallback operations, managing the lifecycle of error handling and recovery strategies.\n\n```mermaid\nsequenceDiagram\n participant Agent as Agent\n participant FallbackManager as FallbackManager\n participant RecoveryStrategy as RecoveryStrategy\n participant Session as Session\n \n Agent->>FallbackManager: Register failure context\n FallbackManager->>FallbackManager: Classify error type\n FallbackManager->>RecoveryStrategy: Select appropriate strategy\n RecoveryStrategy->>Session: Apply recovery action\n Session-->>Agent: Resume or terminate\n```\n\n#### Key Methods\n\n| Method | Purpose |\n|--------|---------|\n| `register_failure()` | Records a new failure event in the fallback system |\n| `classify_error()` | Determines the error category based on error characteristics |\n| `select_recovery_strategy()` | Chooses the optimal recovery approach |\n| `apply_recovery()` | Executes the chosen recovery mechanism |\n| `should_retry()` | Evaluates whether additional attempts are warranted |\n\n资料来源：[packages/notte-sdk/src/notte_sdk/agent_fallback.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/agent_fallback.py)\n\n## Error Classification\n\nThe fallback system categorizes failures into distinct types, each with tailored handling strategies:\n\n| Error Type | Description | Default Recovery |\n|------------|-------------|------------------|\n| `captcha` | CAPTCHA or verification challenges detected | Initiate captcha solving flow |\n| `action_failure` | Interactive element action failed | Retry with modified selectors |\n| `navigation_error` | Page navigation or URL resolution failure | Retry with exponential backoff |\n| `timeout_error` | Operation exceeded time limits | Extend timeout and retry |\n| `invalid_state` | Agent reached an inconsistent state | Reset to known good state |\n| `fatal_error` | Unrecoverable error requiring termination | Graceful session cleanup |\n\n## Recovery Strategies\n\n### 1. Captcha Resolution Strategy\n\nWhen the system detects CAPTCHA challenges, it automatically triggers the captcha solving mechanism.\n\n```python\n# Captcha detection triggers automatic resolution\nif error_type == \"captcha\":\n captcha_type = detect_captcha_type(metadata)\n captcha_action = CaptchaSolveAction(captcha_type=captcha_type)\n result = session.execute(captcha_action)\n```\n\n资料来源：[packages/notte-agent/src/notte_agent/gufo/system.md](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/gufo/system.md)\n\n### 2. Retry with Backoff Strategy\n\nFor transient failures, the system implements configurable retry logic with exponential backoff:\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `max_retries` | `3` | Maximum retry attempts |\n| `base_delay` | `1000` | Initial delay in milliseconds |\n| `backoff_factor` | `2.0` | Multiplier for each retry |\n| `jitter` | `True` | Randomization to prevent thundering herd |\n\n### 3. Graceful Degradation Strategy\n\nWhen recovery is not possible, the system performs controlled cleanup:\n\n```python\n# Graceful termination sequence\ntry:\n session.stop()\nexcept Exception as e:\n logger.warning(f\"Session cleanup warning: {e}\")\nfinally:\n fallback_manager.record_final_state(error_context)\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n\n## Integration with Session Management\n\nThe fallback system integrates tightly with the Notte session lifecycle:\n\n```mermaid\ngraph LR\n A[Session Start] --> B[Agent Initialization]\n B --> C[Task Execution]\n C --> D{Success?}\n D -->|Yes| E[Complete Task]\n D -->|No| F[Fallback Check]\n F --> G{Retryable?}\n G -->|Yes| H[Apply Recovery]\n G -->|No| I[Log Failure]\n H --> C\n I --> J[Session Cleanup]\n E --> J\n```\n\n### Session Callback Integration\n\nThe SDK exposes callback hooks for fallback integration:\n\n```python\nsession.on_failure(callback=fallback_manager.handle_failure)\nsession.on_retry(callback=fallback_manager.prepare_retry)\nsession.on_success(callback=fallback_manager.record_success)\n```\n\n## Configuration Options\n\n| Configuration | Type | Default | Description |\n|---------------|------|---------|-------------|\n| `fallback_enabled` | `bool` | `True` | Enable/disable fallback system |\n| `max_retry_attempts` | `int` | `3` | Global retry limit |\n| `fallback_timeout_ms` | `int` | `30000` | Timeout for fallback operations |\n| `capture_screenshots` | `bool` | `True` | Screenshot on failure for debugging |\n| `verbose_logging` | `bool` | `False` | Detailed fallback logging |\n\n## Usage Patterns\n\n### Basic Usage with SDK\n\n```python\nfrom notte_sdk import NotteClient\n\nclient = NotteClient()\n\nwith client.Session() as session:\n agent = client.Agent(session=session)\n agent.start(task=\"Navigate and extract data\", url=\"https://example.com\")\n \n # Fallback system handles failures automatically\n status = agent.status()\n```\n\n资料来源：[packages/notte-sdk/README.md](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/README.md)\n\n### Custom Fallback Handler\n\n```python\nclass CustomFallbackHandler:\n def handle_failure(self, fallback_action: FallbackAction) -> RecoveryResult:\n # Custom logic for specific error types\n if fallback_action.error_type == \"captcha\":\n return RecoveryResult(strategy=\"custom_captcha_solver\")\n return RecoveryResult(strategy=\"default\")\n```\n\n## Error Reporting and Monitoring\n\nThe fallback system generates structured error reports containing:\n\n| Field | Description |\n|-------|-------------|\n| `session_id` | Unique identifier for the session |\n| `agent_id` | Identifier of the agent that failed |\n| `error_type` | Classification of the failure |\n| `error_message` | Human-readable error description |\n| `playwright_code` | Relevant browser automation code |\n| `html_element` | DOM element context when available |\n| `timestamp` | ISO 8601 timestamp of the failure |\n| `retry_history` | Array of previous retry attempts |\n\n## Best Practices\n\n1. **Enable Verbose Logging During Development**: Set `verbose_logging=True` to capture detailed fallback behavior\n2. **Configure Appropriate Timeouts**: Match `fallback_timeout_ms` to your expected operation durations\n3. **Monitor Retry Counts**: Track `retry_count` to identify persistent issues\n4. **Preserve Error Context**: Always include `metadata` for effective debugging\n5. **Test Fallback Paths**: Regularly validate fallback behavior under failure conditions\n\n## Related Components\n\n| Component | Purpose |\n|-----------|---------|\n| `FalcoPrompt` | Agent instruction and action generation |\n| `SessionManager` | Browser session lifecycle |\n| `CaptchaSolveAction` | Specialized captcha resolution |\n| `ScrapeAction` | Data extraction with fallback support |\n| `ErrorProcessing` | Low-level error handling |\n\n资料来源：[packages/notte-agent/src/notte_agent/falco/prompt.py](https://github.com/nottelabs/notte/blob/main/packages/notte-agent/src/notte_agent/falco/prompt.py)\n\n---\n\n\n\n## Browser Sessions\n\n### 相关页面\n\n相关主题：[Actions and Browser Controls](#actions-controls), [Vaults and Credential Management](#vaults-credentials)\n\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/notte-browser/src/notte_browser/session.py](https://github.com/nottelabs/notte/blob/main/packages/notte-browser/src/notte_browser/session.py)\n- [packages/notte-browser/src/notte_browser/captcha.py](https://github.com/nottelabs/notte/blob/main/packages/notte-browser/src/notte_browser/captcha.py)\n- [packages/notte-browser/src/notte_browser/controller.py](https://github.com/nottelabs/notte/blob/main/packages/notte-browser/src/notte_browser/controller.py)\n- [packages/notte-sdk/src/notte_sdk/endpoints/sessions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/src/notte_sdk/endpoints/sessions.py)\n- [packages/notte-core/src/notte_core/actions/actions.py](https://github.com/nottelabs/notte/blob/main/packages/notte-core/src/notte_core/actions/actions.py)\n- [packages/notte-mcp/README.md](https://github.com/nottelabs/notte/blob/main/packages/notte-mcp/README.md)\n- [packages/notte-sdk/README.md](https://github.com/nottelabs/notte/blob/main/packages/notte-sdk/README.md)\n \n\n# Browser Sessions\n\n## Overview\n\nBrowser Sessions in Notte provide a managed environment for automating web interactions through a cloud-based browser infrastructure. Sessions encapsulate the state of a browser instance, allowing developers to navigate websites, interact with elements, extract data, and handle complex web automation tasks programmatically.\n\nThe session system abstracts away the complexities of browser automation, providing a high-level API for:\n- Navigating to URLs and managing browser tabs\n- Observing page elements and available actions\n- Executing automated interactions (clicks, form fills, scrolling)\n- Scraping structured and unstructured data from web pages\n- Solving CAPTCHA challenges automatically\n\n资料来源：[packages/notte-sdk/README.md]()\n\n## Architecture\n\n```mermaid\ngraph TD\n A[NotteClient] --> B[Session]\n B --> C[Browser Controller]\n C --> D[Cloud Browser Instance]\n E[Actions] --> B\n B --> F[Observations]\n B --> G[Scrape Results]\n H[Captcha Handler] --> C\n```\n\n### Core Components\n\n| Component | Package | Responsibility |\n|-----------|---------|----------------|\n| `Session` | `notte-sdk` | High-level API for session management |\n| `BrowserController` | `notte-browser` | Low-level browser control and state |\n| `CaptchaSolver` | `notte-browser` | Automatic CAPTCHA resolution |\n| `ActionExecutor` | `notte-core` | Action execution and validation |\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py]()\n资料来源：[packages/notte-browser/src/notte_browser/controller.py]()\n\n## Session Lifecycle\n\n### Starting a Session\n\nSessions can be initialized using the context manager pattern for automatic cleanup:\n\n```python\nfrom notte_sdk import NotteClient\n\nclient = NotteClient()\nwith client.Session() as session:\n session.execute(type=\"goto\", url=\"https://www.example.com\")\n # Perform actions...\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py:30-40]()\n\n### Session States\n\n```mermaid\nstateDiagram-v2\n [*] --> Idle: Client initialized\n Idle --> Active: Session started\n Active --> Active: Actions executed\n Active --> Stopping: stop() called\n Stopping --> Stopped: Cleanup complete\n Stopped --> [*]: Context exited\n```\n\n### Stopping a Session\n\nWhen a session stops, cookies are automatically saved if configured:\n\n```python\ndef stop(self) -> None:\n if self._cookie_file is not None:\n try:\n cookies = self.get_cookies()\n create_or_append_cookies_to_file(self._cookie_file, cookies)\n except Exception as e:\n logger.error(f\"🍪 Error saving cookies to {self._cookie_file}: {e}\")\n self.response = self.client.stop(session_id=self.session_id)\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py:50-65]()\n\n## Session Configuration\n\n### Configuration Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `timeout_minutes` | `int` | `5` | Session timeout in minutes |\n| `proxies` | `bool` | `True` | Enable proxy support |\n| `cookie_file` | `str` | `None` | Path to cookie persistence file |\n| `open_viewer` | `bool` | `False` | Open visual browser viewer |\n\n### Creating a Configured Session\n\n```python\nwith client.Session(timeout_minutes=10, open_viewer=True) as session:\n status = session.status()\n session.viewer()\n```\n\n资料来源：[packages/notte-sdk/README.md]()\n\n## Page Interaction\n\n### Observation\n\nThe `observe()` method retrieves interactive elements on the current page. Notte supports two perception modes:\n\n#### Fast Perception\nA quick, simple page scan for basic element identification:\n\n```python\nobs = session.observe(perception_type='fast')\n```\n\n#### Deep Perception\nLLM-powered analysis for richer action space generation:\n\n```python\nobs = session.observe(perception_type='deep')\nprint(obs.space.description)\n```\n\n#### Focused Observation\nUse `instructions` to narrow the action space to a specific intent:\n\n```python\nactions = session.observe(instructions=\"Fill the email input\")\nprint(actions[0].model_dump())\n```\n\n资料来源：[packages/notte-sdk/src/notte_sdk/endpoints/sessions.py:90-110]()\n\n### Element Identification\n\nElements are identified using structured IDs with the format `[:]`:\n\n| Role Letter | Element Type |\n|-------------|--------------|\n| `I` | Input fields (textbox, select, checkbox) |\n| `B` | Buttons |\n| `L` | Links |\n| `F` | Figures and images |\n| `O` | Options in select elements |\n| `M` | Miscellaneous (modals, dialogs) |\n\nExample element ID: `I2[:]