# https://github.com/test-zeus-ai/testzeus-hercules 项目说明书
生成时间：2026-05-17 01:20:40 UTC
## 目录
- [Getting Started with TestZeus Hercules](#getting-started)
- [System Architecture](#system-architecture)
- [Agent System](#agent-system)
- [Browser Automation](#browser-automation)
- [Tool System](#tool-system)
- [LLM Configuration](#llm-configuration)
- [Memory Management](#memory-management)
- [API Testing](#api-testing)
- [Security Testing](#security-testing)
- [MCP Integration](#mcp-integration)
## Getting Started with TestZeus Hercules
相关源码文件
以下源码文件用于生成本页说明：
- [CONTRIBUTING.md](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/CONTRIBUTING.md)
- [testzeus_hercules/__main__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/__main__.py)
- [testzeus_hercules/config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
- [testzeus_hercules/telemetry.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/telemetry.py)
- [testzeus_hercules/utils/response_parser.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/response_parser.py)
- [frontend/non-interactive/index.html](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/frontend/non-interactive/index.html)
- [frontend/interactive/index.html](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/frontend/interactive/index.html)
# Getting Started with TestZeus Hercules
TestZeus Hercules is an open-source AI-powered end-to-end testing framework that leverages large language models (LLMs) to automate browser testing. It provides both interactive and non-interactive modes for executing automated tests against web applications.
## Overview
TestZeus Hercules serves as an intelligent testing agent that can:
- Navigate web pages and interact with UI elements
- Generate and execute Gherkin-style test scenarios
- Perform API security scanning using Nuclei
- Parse accessibility trees for element identification
- Execute Python scripts in a sandboxed environment
资料来源：[CONTRIBUTING.md]()
## Architecture
```mermaid
graph TD
A[User Input] --> B[CLI / Main Entry]
B --> C[Global Configuration]
C --> D[Navigation Agent]
D --> E[Browser Controller]
E --> F[CDP Stream Renderer]
F --> G[Accessibility Tree]
G --> D
D --> H[Python Sandbox Executor]
H --> I[Test Results]
D --> J[API Security Scanner]
J --> K[Nuclei Integration]
```
### Core Components
| Component | Purpose |
|-----------|---------|
| `testzeus_hercules/__main__.py` | Entry point handling bulk test execution |
| `testzeus_hercules/config.py` | Command-line argument parsing and configuration |
| `testzeus_hercules/telemetry.py` | Installation tracking and error reporting |
| `testzeus_hercules/core/agents/executor_nav_agent.py` | Navigation agent for browser automation |
| `frontend/*/index.html` | CDP stream rendering interfaces |
资料来源：[testzeus_hercules/__main__.py:25-45]()
资料来源：[testzeus_hercules/config.py]()
## Installation
### Prerequisites
- Python 3.x
- Git
- Make
### Setup Steps
1. **Fork the repository**
```bash
git clone git@github.com:YOUR_GIT_USERNAME/testzeus-hercules.git
cd testzeus-hercules
git remote add upstream https://github.com/test-zeus-ai/testzeus-hercules
```
2. **Create virtual environment**
```bash
make virtualenv
source .venv/bin/activate
```
3. **Install in development mode**
```bash
make install
```
资料来源：[CONTRIBUTING.md]()
## Command-Line Interface
TestZeus Hercules provides extensive CLI options for configuration.
### Basic Options
| Parameter | Type | Description |
|-----------|------|-------------|
| `--input-file` | str | Path to the input file |
| `--output-path` | str | Path to the output directory |
| `--test-data-path` | str | Path to the test data directory |
| `--project-base` | str | Path to the project base directory |
资料来源：[testzeus_hercules/config.py]()
### LLM Configuration
| Parameter | Type | Description |
|-----------|------|-------------|
| `--llm-model` | str | Name of the LLM model |
| `--llm-model-api-key` | str | API key for the LLM model |
| `--llm-model-base-url` | str | Base URL for the LLM API |
| `--llm-model-api-type` | str | Type of API (openai, anthropic, azure) |
| `--llm-temperature` | float | Temperature for LLM sampling (0.0-1.0) |
| `--agents-llm-config-file` | str | Path to agents LLM configuration file |
资料来源：[testzeus_hercules/config.py]()
### Browser Options
| Parameter | Description |
|-----------|-------------|
| `--browser-channel` | Browser channel (e.g., chrome-beta, firefox-nightly) |
| `--browser-path` | Custom path to browser executable |
| `--browser-version` | Specific browser version (e.g., '114', '115.0.1', 'latest') |
| `--enable-ublock` | Enable uBlock Origin extension |
| `--disable-ublock` | Disable uBlock Origin extension |
| `--auto-accept-screen-sharing` | Automatically accept screen sharing prompts |
资料来源：[testzeus_hercules/config.py]()
### Test Execution Options
| Parameter | Description |
|-----------|-------------|
| `--bulk` | Execute tests in bulk from tests directory |
| `--reuse-vector-db` | Reuse existing vector DB instead of creating fresh one |
资料来源：[testzeus_hercules/config.py]()
资料来源：[testzeus_hercules/__main__.py:45-60]()
### Portkey Integration
| Parameter | Description |
|-----------|-------------|
| `--enable-portkey` | Enable Portkey integration for LLM routing |
| `--portkey-api-key` | API key for Portkey |
| `--portkey-strategy` | Routing strategy (fallback or loadbalance) |
资料来源：[testzeus_hercules/config.py]()
### Sandbox Configuration
| Parameter | Description |
|-----------|-------------|
| `--sandbox-tenant-id` | Tenant ID for sandbox isolation |
资料来源：[testzeus_hercules/config.py]()
## Running TestZeus Hercules
### Interactive Mode
Run the interactive CDP stream renderer with user input capabilities:
```bash
make run-interactive
```
This launches the frontend at `frontend/interactive/index.html` which provides:
- Real-time screencast display
- Crosshair cursor for element selection
- Input capture for typing into the remote page
资料来源：[frontend/interactive/index.html]()
### Non-Interactive Mode
Run tests without user interaction:
```bash
make run
```
This uses the non-interactive frontend at `frontend/non-interactive/index.html` which displays:
- Connection status
- Screencast output only
资料来源：[frontend/non-interactive/index.html]()
### Bulk Execution
Execute multiple tests from a tests directory:
```bash
python -m testzeus_hercules --bulk
```
The system checks for a `tests` directory in the project source root and processes each test folder:
```python
if get_global_conf().should_execute_bulk():
project_base = get_global_conf().get_project_source_root()
tests_dir = os.path.join(project_base, "tests")
```
资料来源：[testzeus_hercules/__main__.py:45-55]()
## Response Parsing
TestZeus Hercules includes a robust response parser for handling LLM outputs:
```mermaid
graph LR
A[LLM Response] --> B{Is JSON?}
B -->|Yes| C[Parse JSON]
B -->|No| D[Extract Plan/Next Step]
C --> E[Return Dict]
D --> E
```
The parser handles:
- JSON wrapped in ```json code blocks
- Plain JSON responses
- Fallback extraction for `plan` and `next_step` fields
资料来源：[testzeus_hercules/utils/response_parser.py]()
## Telemetry and Installation Tracking
On first run, TestZeus Hercules generates a unique installation ID:
```python
def get_installation_id(file_path: str = "installation_id.txt", is_manual_run: bool = True):
if os.path.exists(file_path):
# Load existing installation data
else:
# Generate new installation ID
installation_id = str(uuid.uuid4())
```
资料来源：[testzeus_hercules/telemetry.py]()
## Development Workflow
### Code Quality
| Command | Purpose |
|---------|---------|
| `make fmt` | Format code using black & isort |
| `make lint` | Run pep8, black, mypy linters |
| `make test` | Run tests and generate coverage report |
| `make watch` | Run tests on every change |
资料来源：[CONTRIBUTING.md]()
### Testing Requirements
- Code coverage must show `100%` coverage
- Add tests for all changes in your PR
```bash
make test
```
资料来源：[CONTRIBUTING.md]()
### Release Process
1. Make changes following the contribution guidelines
2. Commit using conventional commit messages
3. Run tests to ensure everything works
4. Execute `make release` to create a new tag and push
> **CAUTION**: The make release will change local changelog files and commit all unstaged changes.
资料来源：[CONTRIBUTING.md]()
## Navigation Agent Execution
The executor navigation agent follows specific guidelines:
### Execution Principles
1. **Error Review**: Review previous step outcomes before proceeding
2. **Script Execution**: Use `execute_python_sandbox` tool with access to `page`, `browser`, `context`, `playwright_manager`, `logger`, `config`
3. **Sequential Execution**: Execute one script at a time and await results
4. **Validation**: Check for successful execution status before proceeding
资料来源：[testzeus_hercules/core/agents/executor_nav_agent.py]()
## API Security Scanning
TestZeus Hercules integrates with Nuclei for API security testing:
```python
async def run_nuclei_command(
is_open_api_spec: bool,
open_api_spec_path: Optional[str],
target_url: Optional[str],
tag: str,
output_file: Path,
headers: Optional[List[Tuple[str, str]]] = None,
):
```
资料来源：[testzeus_hercules/core/tools/api_sec_calls.py]()
## Helper Scripts
### CDP Journey Script
Generate test cases from journey data:
```bash
python helper_scripts/cdp_journey_script.py --number_of_testcase 5
```
This produces Gherkin specifications and test data files from JSON journey definitions.
资料来源：[helper_scripts/cdp_journey_script.py]()
### API Functional Gherkin Test Generator
Generate Gherkin test cases from OpenAPI specifications:
```bash
python helper_scripts/generate_api_functional_gherkin_test.py spec.yaml --output ./features --number_of_testcase 100
```
资料来源：[helper_scripts/generate_api_functional_gherkin_test.py]()
## Accessibility Tree Processing
TestZeus Hercules processes DOM elements to generate accessibility trees:
- Identifies interactive elements (buttons, links, inputs)
- Detects draggable elements
- Filters out non-interactive elements
- Provides detailed element metadata for the AI agent
资料来源：[testzeus_hercules/utils/get_detailed_accessibility_tree.py]()
## Summary
TestZeus Hercules provides a comprehensive end-to-end testing solution with:
- AI-powered browser automation via LLM integration
- Flexible deployment (interactive and non-interactive modes)
- Extensive CLI configuration options
- Built-in support for bulk test execution
- API security scanning capabilities
- Gherkin test generation from various sources
资料来源：[CONTRIBUTING.md]()
---
## System Architecture
### 相关页面
相关主题：[Agent System](#agent-system), [Tool System](#tool-system)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/runner.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/runner.py)
- [testzeus_hercules/core/simple_hercules.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
- [testzeus_hercules/core/__init__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/__init__.py)
- [testzeus_hercules/interactive.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/interactive.py)
- [testzeus_hercules/config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
- [testzeus_hercules/core/extra_tools/__init__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/extra_tools/__init__.py)
- [testzeus_hercules/core/tools/execute_python_sandbox.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/execute_python_sandbox.py)
- [testzeus_hercules/telemetry.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/telemetry.py)
# System Architecture
## Overview
TestZeus Hercules is an open-source AI agent framework designed for **end-to-end testing** of web applications. The system leverages Large Language Models (LLMs) to orchestrate browser automation through Playwright, enabling natural language-driven test execution without requiring users to write traditional test scripts.
The architecture follows a modular design pattern with clear separation between:
- Core execution engine
- Agent-based navigation and task handling
- Python sandbox environment for script execution
- Frontend visualization components
- Configuration and telemetry systems
**资料来源：** [testzeus_hercules/__main__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/__main__.py)
---
## High-Level Architecture
```mermaid
graph TD
User[User / CI Pipeline] --> CLI[CLI Entry Point __main__.py]
CLI --> Config[Global Configuration config.py]
CLI --> Runner[CommandPromptRunner runner.py]
Runner --> SimpleHercules[SimpleHercules simple_hercules.py]
Runner --> Interactive[Interactive Runner interactive.py]
SimpleHercules --> AgentSystem[Agent System]
AgentSystem --> ExecutorNavAgent[ExecutorNavAgent]
AgentSystem --> ExtraTools[Extra Tools Loader extra_tools/__init__.py]
ExecutorNavAgent --> Sandbox[Python Sandbox execute_python_sandbox.py]
Sandbox --> Playwright[Playwright Browser Automation]
SimpleHercules --> Telemetry[Telemetry Module telemetry.py]
SimpleHercules --> ResponseParser[Response Parser response_parser.py]
Interactive --> CDPStream[CDP Stream Frontend frontend/interactive]
Config --> LLMConfig[LLM Configuration openai, anthropic, azure]
Config --> Portkey[Portkey Integration]
```
---
## Core Components
### 1. Entry Points
The framework provides multiple entry points for different use cases:
| Entry Point | File | Purpose |
|-------------|------|---------|
| CLI | `testzeus_hercules/__main__.py` | Main command-line interface with ASCII banner |
| Interactive | `testzeus_hercules/interactive.py` | Interactive test session runner |
| Module Import | `testzeus_hercules/core/__init__.py` | Programmatic API for embedding |
**资料来源：** [testzeus_hercules/interactive.py:1-9](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/interactive.py)
#### Interactive Entry Point
```python
import asyncio
from testzeus_hercules.config import get_global_conf
from testzeus_hercules.core.runner import CommandPromptRunner
if __name__ == "__main__":
conf = get_global_conf()
conf.set_default_test_id("interactive_runner")
runner = CommandPromptRunner()
asyncio.run(runner.start())
```
### 2. Configuration System
The configuration system (`config.py`) manages all runtime parameters:
**资料来源：** [testzeus_hercules/config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
#### Configuration Categories
| Category | Parameters |
|----------|------------|
| **Path Configuration** | `--input-file`, `--output-path`, `--test-data-path`, `--project-base` |
| **LLM Model** | `--llm-model`, `--llm-model-api-key`, `--llm-model-base-url`, `--llm-model-api-type` |
| **LLM Sampling** | `--llm-temperature` (0.0-1.0) |
| **LLM Config Files** | `--agents-llm-config-file`, `--agents-llm-config-file-ref-key` |
| **Portkey Integration** | `--enable-portkey`, `--portkey-api-key`, `--portkey-strategy` (fallback/loadbalance) |
| **Browser Options** | `--browser-channel`, `--browser-path`, `--browser-version` |
| **Browser Extensions** | `--enable-ublock`, `--disable-ublock` |
| **Screen Sharing** | `--auto-accept-screen-sharing`, `--disable-auto-accept-screen-sharing` |
| **Execution** | `--bulk`, `--reuse-vector-db` |
| **Sandbox** | `--sandbox-tenant-id`, `--sandbox-custom-injections` |
---
## Agent System Architecture
### SimpleHercules Core
The `SimpleHercules` class is the central orchestration engine:
```mermaid
graph TD
subgraph "SimpleHercules Core"
Planner[Planner Agent]
Executor[ExecutorNavAgent]
Notifier[Message Notification System]
State[State Management]
end
Planner --> LLM[LLM Provider]
Executor --> Sandbox[Python Sandbox]
Executor --> Playwright[Playwright Manager]
Notifier --> UI[Frontend / CLI Output]
State --> PlanSteps[Plan Steps]
State --> StepHistory[Step History]
State --> Assertions[Assertion Results]
```
**资料来源：** [testzeus_hercules/core/simple_hercules.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
#### Message Flow
The system processes responses through a structured message format:
| Field | Type | Description |
|-------|------|-------------|
| `plan` | `List[str]` | List of planned test steps |
| `next_step` | `str` | Current step instruction |
| `is_assert` | `bool` | Whether current step is an assertion |
| `is_passed` | `bool` | Assertion pass/fail status |
| `assert_summary` | `str` | Summary of assertion results |
| `is_terminated` | `bool` | Whether execution is terminated |
| `is_completed` | `bool` | Whether test suite is completed |
| `final_response` | `str` | Final summary response |
#### Message Notification Types
```python
class MessageType(Enum):
PLAN = "plan" # Plan step notifications
STEP = "step" # Current step notifications
INFO = "info" # Information messages
ERROR = "error" # Error messages
RESULT = "result" # Execution results
```
**资料来源：** [testzeus_hercules/core/simple_hercules.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
### Executor Navigation Agent
The `ExecutorNavAgent` handles the execution of browser automation scripts:
**资料来源：** [testzeus_hercules/core/agents/executor_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/executor_nav_agent.py)
#### Execution Guidelines
| Rule | Description |
|------|-------------|
| **Step Review** | Review previous step outcome before proceeding |
| **Sequential Execution** | Execute one script/function at a time |
| **Sandbox Usage** | Execute scripts using `execute_python_sandbox` tool |
| **Access Variables** | Scripts have access to: `page`, `browser`, `context`, `playwright_manager`, `logger`, `config` |
| **Validation** | Verify each script execution result before moving on |
| **Output Processing** | Parse JSON outputs, handle stdout/stderr |
---
## Python Sandbox Architecture
The sandbox system (`execute_python_sandbox.py`) provides secure script execution with configurable injections:
**资料来源：** [testzeus_hercules/core/tools/execute_python_sandbox.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/execute_python_sandbox.py)
```mermaid
graph TD
Request[Sandbox Request] --> TenantCheck{Tenant ID Check}
TenantCheck -->|executor_agent| HerculesUtils[hercules_utils from testzeus_hercules]
TenantCheck -->|tenant specific| TenantModules[Custom Modules from SANDBOX_TENANT_ID]
TenantCheck -->|none| DefaultModules[Default Modules requests, beautifulsoup, etc.]
ConfigDriven -->|SANDBOX_PACKAGES| ConfigPackages[Config-defined packages]
CustomInjections -->|JSON string| CustomPackages[Custom Package Injections]
DefaultModules --> Merge[Injection Merge]
TenantModules --> Merge
ConfigPackages --> Merge
CustomPackages --> Merge
Merge --> SandboxEnv[Sandbox Environment]
```
#### Injection Sources
| Source | Configuration Key | Example |
|--------|-------------------|---------|
| Default packages | Built-in | `requests`, `beautifulsoup4` |
| Tenant-specific | `SANDBOX_TENANT_ID` | Custom module per tenant |
| Config-driven | `SANDBOX_PACKAGES` | Package list from config |
| Per-call | `--sandbox-custom-injections` | JSON-formatted custom injections |
---
## Extra Tools System
The extra tools loader (`extra_tools/__init__.py`) dynamically imports all modules in the `extra_tools` directory:
**资料来源：** [testzeus_hercules/core/extra_tools/__init__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/extra_tools/__init__.py)
```python
# Dynamic module loading pattern
if get_global_conf().get_load_extra_tools().lower().strip() != "false":
for _, module_name, _ in pkgutil.iter_modules([str(package_path)]):
full_module_name = f"testzeus_hercules.core.extra_tools.{module_name}"
module = importlib.import_module(full_module_name)
for attribute_name in dir(module):
if not attribute_name.startswith("_"):
globals()[attribute_name] = getattr(module, attribute_name)
```
---
## Telemetry System
The telemetry module (`telemetry.py`) handles installation tracking and error reporting:
**资料来源：** [testzeus_hercules/telemetry.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/telemetry.py)
#### Installation ID Management
```python
def get_installation_id(file_path: str = "installation_id.txt", is_manual_run: bool = True) -> Dict[str, Any]
```
| Scenario | Behavior |
|----------|----------|
| File exists with dict | Return parsed dict with `installation_id` |
| File exists with string | Return dict with `installation_id` and default email |
| File not exists + manual run | Prompt for `user_email` |
| File not exists + automated | Use default values, generate new UUID |
#### Sentry Integration
| Configuration | Purpose |
|---------------|---------|
| `max_breadcrumbs=0` | Disable breadcrumb collection |
| `send_default_pii=False` | Exclude personally identifiable information |
| `send_client_reports=False` | Disable client reports |
| `sys.argv` extra | Set to `None` to prevent leakage |
| `denylist` | Scrub sensitive fields recursively |
---
## Frontend Architecture
### CDP Stream Renderer
The frontend provides real-time browser visualization:
**资料来源：** [frontend/interactive/index.html](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/frontend/interactive/index.html)
| Version | File | Features |
|---------|------|----------|
| Interactive | `frontend/interactive/index.html` | Mouse/keyboard input capture, crosshair cursor |
| Non-interactive | `frontend/non-interactive/index.html` | Display-only screencast |
#### Interactive Features
- Real-time screencast display via CDP
- Input capture via hidden dummy input field
- Crosshair cursor for element selection
- Pre-formatted output log display
---
## Response Parser
The response parser (`response_parser.py`) handles LLM output normalization:
**资料来源：** [testzeus_hercules/utils/response_parser.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/response_parser.py)
```python
def parse_response(message: str) -> dict[str, Any]
```
#### Parsing Flow
1. **Strip markdown code blocks**: Handle ` ```json ` and ` ``` ` wrappers
2. **Normalize whitespace**: Replace `\n` with spaces
3. **JSON parsing**: Attempt `json.loads()`
4. **Fallback parsing**: If JSON fails, extract `plan` and `next_step` via string matching
5. **Error logging**: Warn and continue if parsing fails
---
## Execution Workflow
```mermaid
sequenceDiagram
participant User
participant CLI
participant Config
participant Runner
participant SimpleHercules
participant Executor
participant Sandbox
participant Playwright
User->>CLI: Run command
CLI->>Config: Parse arguments
Config-->>CLI: Global config
CLI->>Runner: Create CommandPromptRunner
Runner->>SimpleHercules: Initialize
SimpleHercules->>Executor: Create ExecutorNavAgent
loop Test Execution
SimpleHercules->>SimpleHercules: Generate plan via LLM
SimpleHercules->>Executor: Execute next_step
Executor->>Sandbox: Run Python script
Sandbox->>Playwright: Browser automation
Playwright-->>Sandbox: Result
Sandbox-->>Executor: Execution result
Executor-->>SimpleHercules: Response with plan/next_step
SimpleHercules->>SimpleHercules: Parse and notify
end
SimpleHercules-->>Runner: Completion / Termination
Runner-->>User: Final response
```
---
## LLM Integration
### Supported Providers
| Provider | Configuration | Reference |
|----------|---------------|-----------|
| OpenAI | `--llm-model-api-type=openai` | Default |
| Anthropic | `--llm-model-api-type=anthropic` | Claude models |
| Azure OpenAI | `--llm-model-api-type=azure` | Azure-hosted |
| Custom | `--llm-model-base-url` | Self-hosted endpoints |
### Portkey Integration
For production routing and observability:
```bash
--enable-portkey
--portkey-api-key=
--portkey-strategy=[fallback|loadbalance]
```
---
## Configuration File Structure
The system supports YAML/JSON configuration files via `--agents-llm-config-file`:
| Parameter | Description |
|-----------|-------------|
| `ref_key` | Reference key for loading specific config sections |
| `temperature` | Sampling temperature (0.0-1.0) |
| `max_tokens` | Maximum response length |
| `model` | Model identifier |
---
## Summary
The TestZeus Hercules architecture demonstrates a clean separation of concerns:
1. **Entry Layer**: CLI and interactive interfaces for user access
2. **Configuration Layer**: Centralized parameter management with environment and file support
3. **Orchestration Layer**: `SimpleHercules` coordinates agent communication
4. **Execution Layer**: `ExecutorNavAgent` handles browser automation
5. **Sandbox Layer**: Secure Python script execution with dynamic injections
6. **Telemetry Layer**: Installation tracking and error reporting via Sentry
7. **Presentation Layer**: CDP-based frontend for real-time visualization
This design enables the system to serve as both a standalone testing tool and an embeddable library for larger test frameworks.
---
## Agent System
### 相关页面
相关主题：[System Architecture](#system-architecture), [Memory Management](#memory-management), [LLM Configuration](#llm-configuration)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/agents/high_level_planner_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/high_level_planner_agent.py)
- [testzeus_hercules/core/agents/browser_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/browser_nav_agent.py)
- [testzeus_hercules/core/agents/api_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/api_nav_agent.py)
- [testzeus_hercules/core/agents/sql_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/sql_nav_agent.py)
- [testzeus_hercules/core/agents/mcp_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/mcp_nav_agent.py)
- [testzeus_hercules/core/agents/sec_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/sec_nav_agent.py)
- [testzeus_hercules/core/agents/executor_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/executor_nav_agent.py)
- [testzeus_hercules/core/agents/time_keeper_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/time_keeper_nav_agent.py)
- [testzeus_hercules/core/agent_registry.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agent_registry.py)
# Agent System
## Overview
The Agent System is the core orchestration layer of the Hercules testing framework. It implements a multi-agent architecture where specialized agents collaborate to execute end-to-end testing scenarios across web browsers, APIs, databases, and other system components.
**资料来源：** [testzeus_hercules/core/agent_registry.py:1-50](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agent_registry.py)
## Architecture Overview
The system follows a hierarchical agent design where a central planner coordinates specialized navigation agents, each responsible for a specific domain of interaction.
```mermaid
graph TD
A[HighLevelPlannerAgent] --> B[BrowserNavAgent]
A --> C[ApiNavAgent]
A --> D[SqlNavAgent]
A --> E[McpNavAgent]
A --> F[SecNavAgent]
A --> G[TimeKeeperNavAgent]
B --> H[ExecutorNavAgent]
C --> H
D --> H
E --> H
F --> H
G --> H
H --> I[Browser/API/SQL/MCP/Security]
```
**资料来源：** [testzeus_hercules/core/simple_hercules.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
## Agent Types
### High-Level Planner Agent
The `HighLevelPlannerAgent` serves as the central coordinator that receives high-level test instructions and decomposes them into executable steps for specialized agents.
**Key Responsibilities:**
- Parsing test instructions and generating execution plans
- Routing tasks to appropriate specialized agents
- Aggregating results and handling test completion
- Managing assertions and validating expected outcomes
**资料来源：** [testzeus_hercules/core/agents/high_level_planner_agent.py:1-80](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/high_level_planner_agent.py)
### Browser Navigation Agent
The `BrowserNavAgent` handles all browser-based interactions including page navigation, element interaction, and DOM manipulation.
**Capabilities:**
- Web page navigation and URL handling
- Element clicking and text input
- Screenshot capture and visual validation
- Cookie and session management
**资料来源：** [testzeus_hercules/core/agents/browser_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/browser_nav_agent.py)
### API Navigation Agent
The `ApiNavAgent` manages HTTP-based interactions for testing RESTful APIs and web services.
**Capabilities:**
- HTTP request construction and execution
- Response validation and assertion
- Authentication handling (OAuth, API keys, Bearer tokens)
- Multi-step API workflows
**资料来源：** [testzeus_hercules/core/agents/api_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/api_nav_agent.py)
### SQL Navigation Agent
The `SqlNavAgent` handles database interactions for data validation and setup during test execution.
**Capabilities:**
- SQL query execution
- Database connection management
- Result set validation
- Test data preparation and teardown
**资料来源：** [testzeus_hercules/core/agents/sql_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/sql_nav_agent.py)
### MCP Navigation Agent
The `McpNavAgent` provides Model Context Protocol integration for interacting with external AI models and tools.
**Capabilities:**
- MCP server connection management
- Tool invocation through MCP protocol
- Context propagation for AI-assisted testing
**资料来源：** [testzeus_hercules/core/agents/mcp_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/mcp_nav_agent.py)
### Security Navigation Agent
The `SecNavAgent` handles security-related testing scenarios including authentication flows, authorization checks, and vulnerability scanning.
**Capabilities:**
- Authentication flow testing
- Session security validation
- Authorization boundary testing
- Security header verification
**资料来源：** [testzeus_hercules/core/agents/sec_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/sec_nav_agent.py)
### Time Keeper Navigation Agent
The `TimeKeeperNavAgent` manages time-related test scenarios including scheduling, delays, and time-based assertions.
**Capabilities:**
- Time-based test scheduling
- Delay and timeout management
- Timestamp validation
- Scheduled task execution
**资料来源：** [testzeus_hercules/core/agents/time_keeper_nav_agent.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/time_keeper_nav_agent.py)
### Executor Navigation Agent
The `ExecutorNavAgent` serves as the execution engine that runs Python scripts and commands within a sandboxed environment.
**Key Features:**
- Python script execution in isolated sandbox
- Dynamic module injection based on tenant configuration
- Access to browser context, page objects, and configuration
- Custom injection support for tenant-specific utilities
**资料来源：** [testzeus_hercules/core/agents/executor_nav_agent.py:1-150](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/executor_nav_agent.py)
## Agent Registry
The `AgentRegistry` provides a centralized registration and lookup mechanism for all agents in the system.
### Registry Operations
| Operation | Description |
|-----------|-------------|
| `register_agent(name, agent)` | Register a new agent with a unique name |
| `get_agent(name)` | Retrieve an agent by name |
| `list_agents()` | List all registered agents |
| `remove_agent(name)` | Remove an agent from the registry |
**资料来源：** [testzeus_hercules/core/agent_registry.py:50-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agent_registry.py)
## Agent Creation Flow
The `SimpleHercules` class orchestrates agent creation with the following workflow:
```mermaid
sequenceDiagram
participant SH as SimpleHercules
participant Planner as HighLevelPlannerAgent
participant Nav as Navigation Agents
participant Exec as ExecutorNavAgent
SH->>SH: Initialize configuration
SH->>Planner: Create planner agent
SH->>Nav: Create navigation agents (Browser, API, SQL, etc.)
SH->>Exec: Create executor agent
SH->>SH: Register all agents in registry
Planner->>Nav: Route tasks based on type
Nav->>Exec: Execute concrete actions
```
**资料来源：** [testzeus_hercules/core/simple_hercules.py:100-200](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
## Message Flow
Agents communicate through a structured message passing system with the following message types:
| Message Type | Purpose |
|--------------|---------|
| `PLAN` | Initial test plan and steps |
| `STEP` | Individual test step execution |
| `INFO` | Informational messages |
| `ASSERT` | Assertion results |
| `COMPLETED` | Task completion notification |
| `TERMINATED` | Agent termination signal |
**资料来源：** [testzeus_hercules/core/simple_hercules.py:200-300](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
## Configuration
### LLM Model Configuration
Each agent supports individual LLM model configuration:
| Parameter | Type | Description |
|-----------|------|-------------|
| `model` | string | Model name (e.g., gpt-4, claude-3) |
| `temperature` | float | Sampling temperature (0.0-1.0) |
| `max_tokens` | int | Maximum response tokens |
| `api_key` | string | API authentication key |
| `base_url` | string | Custom API endpoint URL |
**资料来源：** [testzeus_hercules/config.py:1-80](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
### Agent-Specific Settings
```python
# Example agent configuration structure
agent_config = {
"model_config_params": {
"model": "gpt-4",
"temperature": 0.7,
"max_tokens": 2000
},
"llm_config_params": {
"timeout": 60,
"retry_attempts": 3
},
"other_settings": {
"system_prompt": "You are a testing agent...",
"max_chat_rounds": 10
}
}
```
## Response Parsing
The system uses `parse_response()` from the response parser module to extract structured data from agent outputs:
```python
def parse_response(message: str) -> dict[str, Any]:
# Handles JSON extraction from markdown code blocks
# Normalizes newlines and whitespace
# Extracts plan and next_step fields
```
**资料来源：** [testzeus_hercules/utils/response_parser.py:1-60](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/response_parser.py)
## Sandbox Execution
The ExecutorNavAgent provides a secure Python execution environment with configurable module injection:
### Available Injections
| Module | Description |
|--------|-------------|
| `playwright` | Browser automation library |
| `requests` | HTTP client library |
| `beautifulsoup4` | HTML parsing |
| `hercules_utils` | Project utility functions |
| Custom packages | Configured via SANDBOX_PACKAGES |
**资料来源：** [testzeus_hercules/core/tools/execute_python_sandbox.py:1-100](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/execute_python_sandbox.py)
### Sandbox Context Variables
Scripts executed in the sandbox have automatic access to:
| Variable | Type | Description |
|----------|------|-------------|
| `page` | Playwright Page | Current browser page |
| `browser` | Playwright Browser | Browser instance |
| `context` | Playwright Context | Browser context |
| `playwright_manager` | Manager | Playwright management |
| `logger` | Logger | Logging interface |
| `config` | Config | Global configuration |
## Command-Line Interface
The agent system supports configuration via command-line arguments:
| Argument | Description |
|----------|-------------|
| `--llm-model` | Specify LLM model name |
| `--llm-temperature` | Set sampling temperature |
| `--agents-llm-config-file` | Path to agent config file |
| `--enable-portkey` | Enable Portkey routing |
| `--browser-channel` | Browser channel selection |
| `--reuse-vector-db` | Reuse existing vector database |
**资料来源：** [testzeus_hercules/config.py:80-150](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
## Error Handling
Agents implement robust error handling with:
1. **Termination Message Check**: Each agent validates termination conditions via `is_xxx_termination_message()` functions
2. **Tool Call Monitoring**: Tracks pending tool calls to prevent premature termination
3. **Graceful Degradation**: Continues execution with alternative approaches on failure
4. **Logging**: Comprehensive logging for debugging and audit trails
**资料来源：** [testzeus_hercules/core/simple_hercules.py:300-400](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/simple_hercules.py)
---
## Browser Automation
### 相关页面
相关主题：[Tool System](#tool-system)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/playwright_manager.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/playwright_manager.py)
- [testzeus_hercules/core/browser_logger.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/browser_logger.py)
- [testzeus_hercules/utils/dom_helper.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/dom_helper.py)
- [testzeus_hercules/utils/get_detailed_accessibility_tree.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/get_detailed_accessibility_tree.py)
- [helper_scripts/browser_stack_generate_cdp_url.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/helper_scripts/browser_stack_generate_cdp_url.py)
- [helper_scripts/lambda_test_generate_cdp_url.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/helper_scripts/lambda_test_generate_cdp_url.py)
# Browser Automation
## Overview
Browser Automation in TestZeus Hercules provides a comprehensive framework for controlling web browsers through Playwright, enabling autonomous agents to perform complex web interactions, testing, and data extraction tasks. The system acts as a bridge between LLM-powered agents and real browser instances, translating natural language instructions into precise DOM manipulations.
The automation layer handles multiple browser types (Chromium, Firefox, WebKit), manages browser contexts with device emulation, supports cloud-based testing platforms via CDP (Chrome DevTools Protocol) tunneling, and provides sophisticated DOM interaction tools including accessibility-aware element selection and real-time mutation observation.
**资料来源：** [testzeus_hercules/core/playwright_manager.py:1-50]()
## Architecture
### System Components
```mermaid
graph TD
A[SimpleHercules Core] --> B[PlaywrightManager]
B --> C[Browser Context]
C --> D[Browser Instance Chromium|Firefox|WebKit]
B --> E[Tool Registry]
E --> F[Navigation Tools]
E --> G[Interaction Tools]
E --> H[Extraction Tools]
B --> I[DOM Mutation Observer]
B --> J[BrowserLogger]
J --> K[Interaction Logs]
K --> L[Accessibility Tree]
L --> M[AccessibilityInfo]
```
### Core Components
| Component | File | Purpose |
|-----------|------|---------|
| PlaywrightManager | `core/playwright_manager.py` | Central browser lifecycle management |
| BrowserLogger | `core/browser_logger.py` | Interaction logging and proof generation |
| DOMHelper | `utils/dom_helper.py` | DOM state management and waiting |
| AccessibilityTree | `utils/get_detailed_accessibility_tree.py` | Extract and format accessibility information |
| ToolRegistry | `core/tools/tool_registry.py` | Dynamic tool registration and routing |
**资料来源：** [testzeus_hercules/core/playwright_manager.py]()
**资料来源：** [testzeus_hercules/utils/dom_helper.py]()
---
## PlaywrightManager
The `PlaywrightManager` class is the central orchestrator for all browser operations. It handles browser launching, page management, element location, and interaction recording.
### Browser Launch Modes
```mermaid
graph LR
A[Browser Launch Request] --> B{Launch Type?}
B -->|Persistent| C[Launch Persistent Context]
B -->|Remote CDP| D[Connect via CDP URL]
B -->|Recording| E[Launch with Video Recording]
C --> F[Browser Instance]
D --> F
E --> F
```
**资料来源：** [testzeus_hercules/core/playwright_manager.py:100-200]()
### Browser Types
The system supports three browser engines:
| Browser | Type Identifier | Extensions Support | Certificate Handling |
|---------|-----------------|-------------------|---------------------|
| Chromium | `chromium` | Yes (uBlock Origin) | Default ignore |
| Firefox | `firefox` | Yes | Via args |
| WebKit | `webkit` | Limited | Via args |
**资料来源：** [testzeus_hercules/core/playwright_manager.py:180-220]()
### Context Options
The `_build_emulation_context_options` method configures device emulation:
```mermaid
graph TD
A[Build Context Options] --> B{Device Name?}
B -->|Yes| C[Get Device from Playwright]
C --> D[Merge Device Properties]
B -->|No| E[Skip Device Emulation]
D --> F[Return Context Options]
E --> F
```
Context options include viewport size, user agent, device scale factor, and touch capabilities based on the selected device profile.
**资料来源：** [testzeus_hercules/core/playwright_manager.py:200-230]()
### CDP Remote Connection
The system supports connecting to remote browser instances through BrowserStack and LambdaTest via CDP tunneling.
#### BrowserStack Integration
The `browser_stack_generate_cdp_url.py` helper generates CDP URLs for BrowserStack's remote browser grid:
```python
# Simplified flow from helper_scripts/browser_stack_generate_cdp_url.py
def generate_cdp_url(username, access_key, session_id):
ws_endpoint = f"wss://{username}:{access_key}@cdp.browserstack.com"
return f"{ws_endpoint}/playwright?sessionId={session_id}"
```
**资料来源：** [helper_scripts/browser_stack_generate_cdp_url.py]()
#### LambdaTest Integration
Similarly, LambdaTest provides CDP access through their Selenium Smart Proxy:
```python
# Simplified flow from helper_scripts/lambda_test_generate_cdp_url.py
def generate_cdp_url(username, access_key, session_id):
hub_endpoint = f"wss://{username}:{access_key}@hub.lambdatest.com/cdp"
return f"{hub_endpoint}/{session_id}"
```
**资料来源：** [helper_scripts/lambda_test_generate_cdp_url.py]()
---
## Browser Tools
Browser automation tools are registered in the Tool Registry and decorated with `@tool` to enable agent consumption.
### Tool Categories
| Category | Tools | Purpose |
|----------|-------|---------|
| Navigation | `open_url`, `go_back`, `go_forward` | URL navigation and history |
| Interaction | `click`, `hover`, `type`, `upload_file` | User interaction simulation |
| Extraction | `get_page_text`, `get_input_fields`, `get_interactive_elements` | DOM content retrieval |
| Accessibility | `get_accessibility_info`, `run_accessibility_check` | WCAG compliance testing |
| Special | `set_date_time_value`, `read_clipboard` | Specialized interactions |
**资料来源：** [testzeus_hercules/core/tools/tool_registry.py]()
**资料来源：** [testzeus_hercules/core/tools/open_url.py]()
### Selector Convention
All tools use the `md` attribute for element selection, enabling stable selectors independent of DOM structure changes:
```python
# Selector format
selector = "[md='element_id']"
# Automatic conversion if not present
if "md=" not in selector:
selector = f"[md='{selector}']"
```
**资料来源：** [testzeus_hercules/core/tools/hover.py:25-30]()
### Interactive Elements Extraction
The `get_interactive_elements` tool retrieves all clickable, focusable, and input elements with their accessibility properties:
```python
@tool(
agent_names=["browser_nav_agent"],
description="""DOM Type dict Retrieval Tool, giving all interactive elements on page.
Notes: [Elements ordered as displayed, Consider ordinal/numbered item positions, List ordinal represent z-index on page]""",
name="get_interactive_elements",
)
```
Elements are ordered by visual display position and include z-index representation.
**资料来源：** [testzeus_hercules/core/tools/get_interactive_elements.py]()
### Input Fields Retrieval
The `get_input_fields` tool specifically targets form input elements:
```python
@tool(
agent_names=["browser_nav_agent"],
description="""DOM Type dict Retrieval Tool, giving only html input types elements on page.
Notes: [Elements ordered as displayed, Consider ordinal/numbered item positions]""",
name="get_input_fields",
)
```
**资料来源：** [testzeus_hercules/core/tools/get_input_fields.py]()
### Page Text Extraction
The `get_page_text` tool fetches text content from the DOM, stripping HTML tags for readability:
```python
async def get_page_text() -> Annotated[str, "DOM content based on type to analyze and decide"]:
# Uses innerText or documentElement.innerText
text_content = await page.evaluate("() => document.body.innerText || document.documentElement.innerText")
```
**资料来源：** [testzeus_hercules/core/tools/get_page_text.py]()
### File Upload Handling
The `upload_file` tool supports two upload mechanisms:
1. Direct file input: Sets files directly on `` elements
2. File chooser API: Clicks element, intercepts file chooser, and sets files
```python
# Check element type
element_type = await element.evaluate("el => el.type")
if element_type != "file":
# Use FileChooser API
async with page.expect_file_chooser() as fc_info:
await element.click()
file_chooser = await fc_info.value
await file_chooser.set_files(file_path)
```
**资料来源：** [testzeus_hercules/core/tools/upload_file.py]()
### Clipboard Operations
The `read_clipboard` tool reads clipboard content with type specification:
```python
@tool(
name="read_clipboard",
description="""Reads the clipboard content from the current page.
Accepts a parameter to specify the clipboard type: 'text' for plain text
or 'binary' for a binary object representation."""
)
async def read_clipboard(clipboard_type: Annotated[str, "'text' or 'binary'"] = "text")
```
**资料来源：** [testzeus_hercules/core/extra_tools/clipboard_tools.py]()
---
## DOM Mutation Observation
The system monitors DOM changes in real-time using a mutation observer pattern, enabling agents to detect when actions cause new elements to appear.
```mermaid
sequenceDiagram
participant Agent
participant Tool
participant Observer
participant Page
Agent->>Tool: Execute action
Tool->>Observer: Subscribe to changes
Tool->>Page: Perform DOM operation
Page->>Observer: Mutation detected
Observer->>Tool: DOM changes detected
Tool->>Agent: Return result with new elements info
Tool->>Observer: Unsubscribe
```
**资料来源：** [testzeus_hercules/utils/dom_mutation_observer.py]()
**资料来源：** [testzeus_hercules/core/tools/enter_date_time.py]()
### Wait Strategies
The `wait_for_non_loading_dom_state` function waits for the DOM to reach a stable state before proceeding:
```python
await wait_for_non_loading_dom_state(page, max_wait_seconds=1)
```
This prevents race conditions where tools attempt to interact with elements that are still being loaded or rendered.
**资料来源：** [testzeus_hercules/utils/dom_helper.py]()
---
## Accessibility Integration
### Accessibility Tree Generation
The `get_detailed_accessibility_tree.py` module extracts and formats accessibility information using Playwright's accessibility API:
```python
from testzeus_hercules.utils.get_detailed_accessibility_tree import (
do_get_accessibility_info,
rename_children,
)
```
**资料来源：** [testzeus_hercules/utils/get_detailed_accessibility_tree.py]()
### Accessibility Testing
The `run_accessibility_check` tool integrates axe-core for automated accessibility audits:
```python
# Inject axe-core from CDN
response = await page.evaluate(f'fetch("{AXE_SCRIPT_URL}").then(res => res.text())')
await page.add_script_tag(content=response)
# Run accessibility checks
axe_results = await page.evaluate("""
async () => {
return await axe.run();
}
""")
```
Violations are logged with detailed failure summaries, enabling agents to identify and address accessibility issues.
**资料来源：** [testzeus_hercules/core/tools/accessibility_calls.py]()
---
## Configuration
### Command-Line Arguments
The `config.py` file defines extensive browser automation configuration options:
| Argument | Type | Default | Description |
|----------|------|---------|-------------|
| `--browser-channel` | str | None | Browser channel (e.g., chrome-beta, firefox-nightly) |
| `--browser-path` | str | None | Custom path to browser executable |
| `--browser-version` | str | None | Specific browser version (e.g., '114', '115.0.1') |
| `--enable-ublock` | flag | False | Enable uBlock Origin extension |
| `--disable-ublock` | flag | False | Disable uBlock Origin extension |
| `--auto-accept-screen-sharing` | flag | False | Auto-accept screen sharing prompts |
| `--disable-auto-accept-screen-sharing` | flag | False | Disable auto screen sharing acceptance |
**资料来源：** [testzeus_hercules/config.py]()
### Global Configuration
The `get_global_conf()` function provides runtime configuration access for:
- Source log folder paths
- Proof path for screenshots and logs
- Delay time between operations
- Load state configuration
**资料来源：** [testzeus_hercules/config.py]()
---
## Browser Logging
### Interaction Logging
The `BrowserLogger` class records all browser interactions for debugging and proof generation:
```python
await browser_logger.log_browser_interaction(
tool_name="openurl",
action="navigate",
interaction_type="navigation",
success=True,
additional_data={
"url": url,
"title": title,
"from_cache": True,
},
)
```
**资料来源：** [testzeus_hercules/core/browser_logger.py]()
**资料来源：** [testzeus_hercules/core/tools/open_url.py]()
### Screenshot Capture
Tools automatically capture screenshots before and after execution:
```python
await browser_manager.take_screenshots(f"{function_name}_start", page)
# ... perform action ...
await browser_manager.take_screenshots(f"{function_name}_end", page)
```
**资料来源：** [testzeus_hercules/core/tools/hover.py]()
### Selector Logging
Failed selector interactions are logged with detailed error information:
```python
await selector_logger.log_selector_interaction(
tool_name="upload_file",
selector=selector,
action="upload",
selector_type="css",
success=False,
error_message=f"Error: Selector '{selector}' not found.",
)
```
**资料来源：** [testzeus_hercules/core/tools/upload_file.py]()
---
## Agent Integration
### BrowserNavAgent
The `SimpleHercules` class orchestrates agent-based browser automation:
```python
browser_nav_agent = BrowserNavAgent(
self.browser_nav_agent_model_config,
self.nav_agent_config["llm_config_params"],
self.nav_agent_config.get("other_settings", {}).get("system_prompt"),
user_proxy_agent,
)
```
Agents receive tools through the Tool Registry and execute browser operations autonomously based on natural language instructions.
**资料来源：** [testzeus_hercules/core/simple_hercules.py]()
### System Prompts
Agent behavior is guided by system prompts that define DOM handling rules:
```python
# From prompts.py - key instructions for agents:
- Use md attribute selectors for element identification
- Never substring extracted HTML DOM
- Remove entire sections if not needed (scripts, links)
- Use innerText for text content retrieval
- Verify task completion after execution
- Use ##TERMINATE## signal when complete
```
**资料来源：** [testzeus_hercules/core/prompts.py]()
---
## Frontend Interface
### CDP Stream Renderer
The project includes frontend interfaces for viewing browser sessions in real-time:
| File | Purpose |
|------|---------|
| `frontend/non-interactive/index.html` | Basic screencast display |
| `frontend/interactive/index.html` | Interactive viewer with mouse/keyboard input forwarding |
The interactive interface uses a hidden input element to capture focus and prevent scrolling while forwarding input events to the remote browser session.
**资料来源：** [frontend/non-interactive/index.html]()
**资料来源：** [frontend/interactive/index.html]()
---
## Summary
Browser Automation in TestZeus Hercules provides a production-ready framework for autonomous web interaction. Key capabilities include:
1. **Multi-Browser Support**: Chromium, Firefox, and WebKit with extension support
2. **Remote Browser Access**: CDP tunneling for BrowserStack and LambdaTest
3. **Sophisticated Element Selection**: Accessibility-aware tools using the `md` attribute convention
4. **Real-Time DOM Monitoring**: Mutation observers for detecting dynamic content changes
5. **Comprehensive Logging**: Screenshots, interaction logs, and accessibility audits
6. **Agent Integration**: LLM-powered automation through BrowserNavAgent
---
## Tool System
### 相关页面
相关主题：[Browser Automation](#browser-automation)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/tools/tool_registry.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/tool_registry.py)
- [testzeus_hercules/core/tools/click_using_selector.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/click_using_selector.py)
- [testzeus_hercules/core/tools/enter_text_using_selector.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/enter_text_using_selector.py)
- [testzeus_hercules/core/tools/hover.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/hover.py)
- [testzeus_hercules/core/tools/drag_and_drop_tool.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/drag_and_drop_tool.py)
- [testzeus_hercules/core/tools/get_interactive_elements.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/get_interactive_elements.py)
- [testzeus_hercules/core/tools/accessibility_calls.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/accessibility_calls.py)
- [testzeus_hercules/core/tools/upload_file.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/upload_file.py)
- [testzeus_hercules/core/extra_tools/browser_assist_tools.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/extra_tools/browser_assist_tools.py)
# Tool System
## Overview
The Tool System is the core execution layer of TestZeus Hercules, providing AI agents with capabilities to interact with web pages through a unified, decorator-based interface. The system abstracts browser automation operations (clicking, typing, hovering, dragging, etc.) into discrete, callable tools that agents can invoke during test execution.
Tools serve as the fundamental building blocks that bridge natural language agent instructions with Playwright browser automation. Each tool encapsulates a specific browser interaction pattern, handles error cases gracefully, and returns structured results that agents can parse and respond to. 资料来源：[testzeus_hercules/core/tools/click_using_selector.py:1-50]()
## Architecture
### System Components
```mermaid
graph TD
subgraph "Agent Layer"
A["Browser Nav Agent"]
B["Executor Nav Agent"]
end
subgraph "Tool Registry"
C["tool_registry.py"]
D["Tool Decorator"]
end
subgraph "Core Browser Tools"
E["click_using_selector"]
F["enter_text_using_selector"]
G["hover"]
H["drag_and_drop_tool"]
I["get_interactive_elements"]
end
subgraph "Extra Tools"
J["browser_assist_tools"]
K["accessibility_calls"]
L["upload_file"]
end
subgraph "Browser Automation"
M["Playwright Manager"]
N["Page Object"]
end
A --> C
B --> C
C --> E
C --> F
C --> G
C --> H
C --> I
C --> J
J --> K
J --> L
E --> M
F --> M
G --> M
M --> N
```
### Tool Categories
| Category | Purpose | Location | Example Tools |
|----------|---------|----------|---------------|
| **Core Browser Tools** | Primary page interactions | `testzeus_hercules/core/tools/` | click, enter_text, hover, drag_drop |
| **Extra Tools** | Auxiliary functionality | `testzeus_hercules/core/extra_tools/` | accessibility, browser_assist |
| **Upload Tools** | File handling | `testzeus_hercules/core/tools/` | upload_file |
## Tool Definition Pattern
### The `@tool` Decorator
All tools in the system are defined using the `@tool` decorator, which registers the function with the tool registry and provides metadata for agent consumption.
```python
from functools import wraps
from typing import Annotated, Any, Dict, List, Optional
def tool(agent_names: List[str], description: str, name: str):
"""Decorator to register a function as a callable tool."""
def decorator(func):
@wraps(func)
async def wrapper(*args, **kwargs):
return await func(*args, **kwargs)
wrapper._tool_config = {
"agent_names": agent_names,
"description": description,
"name": name,
}
return wrapper
return decorator
```
### Tool Registration Metadata
| Parameter | Type | Description | Required |
|-----------|------|-------------|----------|
| `agent_names` | `List[str]` | List of agent identifiers that can call this tool | Yes |
| `description` | `str` | Human-readable description for the LLM | Yes |
| `name` | `str` | Unique identifier for the tool | Yes |
Example usage:
```python
@tool(
agent_names=["browser_nav_agent"],
description="Click on an element using selector",
name="click"
)
async def click_element(selector: Annotated[str, "CSS selector"]) -> dict:
# Implementation
pass
```
资料来源：[testzeus_hercules/core/tools/click_using_selector.py:1-30]()
## Core Browser Tools
### Click Tool (`click_using_selector`)
The primary interaction tool for clicking on page elements.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="used to click on an element in the DOM.",
name="click"
)
async def click_using_selector(
selector: Annotated[
str,
"md attribute value of the dom element to interact, md is an ID"
],
click_type: Annotated[
Optional[str],
"type of click - left, right, double, mouseover, mouseenter, mouseleave, mouseexit, mousedown, mouseup"
] = "left",
timeout: Annotated[int, "Timeout in milliseconds"] = 30000,
) -> Annotated[Dict[str, Any], "Result of the click operation"]
```
**Execution Flow:**
```mermaid
sequenceDiagram
participant Agent
participant ClickTool
participant PlaywrightManager
participant Page
participant SelectorLogger
Agent->>ClickTool: click_using_selector(selector, click_type)
ClickTool->>PlaywrightManager: find_element(selector)
PlaywrightManager->>Page: locator(selector).first
Page-->>PlaywrightManager: ElementHandle
PlaywrightManager-->>ClickTool: element
alt Element not found
ClickTool->>SelectorLogger: log_selector_interaction(success=False)
ClickTool-->>Agent: Error response
end
ClickTool->>Page: element.scroll_into_view_if_needed()
ClickTool->>Page: element.is_visible()
alt Element not visible
ClickTool-->>Agent: Try another element response
end
ClickTool->>SelectorLogger: get_alternative_selectors()
ClickTool->>SelectorLogger: get_element_attributes()
ClickTool->>Page: element.click(click_type)
alt Click success
ClickTool->>SelectorLogger: log_selector_interaction(success=True)
ClickTool-->>Agent: Success response
else Click fails
ClickTool->>SelectorLogger: log_selector_interaction(success=False)
ClickTool-->>Agent: Error response
end
```
**Error Handling:**
The click tool performs comprehensive error handling:
1. **Element Not Found**: Logs selector interaction with `success=False` and raises `ValueError`
2. **Element Not Visible**: Returns alternative suggestion response
3. **Scroll Failure**: Gracefully continues (non-blocking)
4. **Click Execution Failure**: Logs failure and returns error dictionary
```python
if element is None:
await selector_logger.log_selector_interaction(
tool_name="click",
selector=selector,
action=type_of_click,
selector_type="css" if "md=" in selector else "custom",
success=False,
error_message=f'Element with selector: "{selector}" not found',
)
raise ValueError(f'Element with selector: "{selector}" not found')
```
资料来源：[testzeus_hercules/core/tools/click_using_selector.py:40-80]()
### Enter Text Tool (`enter_text_using_selector`)
Handles text input into form fields and contenteditable elements.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="used to enter the given text into an input field or a contenteditable element.",
name="enter_text"
)
async def enter_text_using_selector(
selector: Annotated[str, "md attribute value of the dom element to interact"],
text_to_fill: Annotated[str, "text to enter into the element"],
submit: Annotated[Optional[bool], "whether to submit after entering text"] = False,
press_enter_after_input: Annotated[bool, "press Enter key after filling text"] = False,
) -> Annotated[Dict[str, str], "Result dictionary with summary and details"]
```
### Hover Tool (`hover`)
Moves mouse cursor over an element without clicking.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="used to hover over an element in the DOM.",
name="hover"
)
async def hover_using_selector(
selector: Annotated[str, "md attribute value of the dom element to interact"],
timeout: Annotated[int, "Timeout in milliseconds"] = 30000,
) -> Annotated[Dict[str, Any], "Result of the hover operation"]
```
### Drag and Drop Tool (`drag_and_drop_tool`)
Performs HTML5 drag-and-drop operations between elements.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="used to drag and drop an element.",
name="drag_and_drop"
)
async def drag_and_drop(
source: Annotated[str, "md attribute value of source element"],
target: Annotated[str, "md attribute value of target element"],
) -> Annotated[Dict[str, str], "Result of the drag and drop operation"]
```
### Get Interactive Elements (`get_interactive_elements`)
Retrieves all interactive elements from the current page DOM.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="Get interactive elements from the current page",
name="get_interactive_elements"
)
async def get_interactive_elements() -> Annotated[str, "JSON string of interactive elements"]
```
**DOM Processing Logic:**
The tool executes JavaScript in the browser context to identify interactive elements:
```javascript
const isInteractive = (element) => {
// Check input-related elements
const inputRelatedTags = new Set(['input', 'textarea', 'select', ...]);
const interactiveRoles = new Set(['button', 'link', 'checkbox', ...]);
// Check for ARIA attributes
const hasAriaProps = element.hasAttribute('aria-haspopup') ||
element.hasAttribute('aria-expanded') ||
element.hasAttribute('aria-checked');
// Check cursor style
const style = window.getComputedStyle(element);
const hasPointerCursor = style.cursor === 'pointer';
// Check draggable attribute
const isDraggable = element.draggable;
// Skip body and its direct children
if (element.tagName.toLowerCase() === 'body') return false;
return hasAriaProps || hasClickHandler || isDraggable;
};
```
资料来源：[testzeus_hercules/core/tools/get_interactive_elements.py:1-50]()
### Upload File Tool (`upload_file`)
Handles file upload dialogs by setting file paths on input elements.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="Upload a file to the page",
name="upload_file"
)
async def upload_file(
selector: Annotated[str, "md attribute value of file input element"],
file_path: Annotated[str, "Path to the file to upload"],
) -> Annotated[Dict[str, str], "Result of the upload operation"]
```
## Extra Tools System
### Dynamic Tool Loading
The extra tools are loaded dynamically at runtime using Python's `pkgutil` module. This allows the system to extend functionality without modifying core code.
**Loading Mechanism:**
```python
# testzeus_hercules/core/extra_tools/__init__.py
import importlib
import pkgutil
from pathlib import Path
from testzeus_hercules.config import get_global_conf
package_path = Path(__file__).parent
if get_global_conf().get_load_extra_tools().lower().strip() != "false":
for _, module_name, _ in pkgutil.iter_modules([str(package_path)]):
full_module_name = f"testzeus_hercules.core.extra_tools.{module_name}"
module = importlib.import_module(full_module_name)
# Export all public attributes to current namespace
for attribute_name in dir(module):
if not attribute_name.startswith("_"):
globals()[attribute_name] = getattr(module, attribute_name)
```
**Configuration:**
Extra tools can be disabled via configuration:
```bash
python -m testzeus_hercules --load-extra-tools=false
```
资料来源：[testzeus_hercules/core/extra_tools/__init__.py:1-25]()
### Accessibility Calls (`accessibility_calls`)
Provides accessibility testing using the axe-core library.
**Function Signature:**
```python
@tool(
agent_names=["browser_nav_agent"],
description="Check page accessibility using axe-core",
name="check_accessibility"
)
async def check_accessibility(page_path: Annotated[str, "Page path or URL"]) -> str
```
**Execution Flow:**
1. Fetches axe-core script from CDN
2. Injects script into page
3. Runs `axe.run()` in browser context
4. Collects violations and incomplete checks
5. Logs results in structured format
```javascript
// Inject axe-core
await page.evaluate(
`fetch("${AXE_SCRIPT_URL}").then(res => res.text())`
);
// Run accessibility checks
const results = await page.evaluate("async () => { return await axe.run(); }");
```
**Response Format:**
```json
{
"status": "success|failure",
"message": "Human readable summary",
"details": ["failure summary 1", "failure summary 2"]
}
```
资料来源：[testzeus_hercules/core/tools/accessibility_calls.py:1-80]()
## Tool Return Format
All tools return a standardized dictionary structure:
| Field | Type | Description |
|-------|------|-------------|
| `summary_message` | `str` | Brief status message for agent consumption |
| `detailed_message` | `str` | Extended information including errors if any |
| `status` | `str` | Operation status (success/failure) |
**Success Response Example:**
```python
return {
"summary_message": f'Successfully clicked element with selector: "{selector}"',
"detailed_message": f'Element with selector: "{selector}" clicked successfully. Tag: {element_tag_name}',
}
```
**Error Response Example:**
```python
return {
"summary_message": f'Element with selector: "{selector}" is not visible, Try another element',
"detailed_message": f'Element with selector: "{selector}" is not visible, Try another element',
}
```
## Selector Logging System
Every tool interaction is logged using the `SelectorLogger` for proof generation and debugging.
### Logging Interface
```python
from testzeus_hercules.utils.browser_logger import get_browser_logger
selector_logger = get_browser_logger(get_global_conf().get_proof_path())
# Log successful interaction
await selector_logger.log_selector_interaction(
tool_name="click",
selector=selector,
action="left",
selector_type="css",
success=True,
)
# Log failed interaction
await selector_logger.log_selector_interaction(
tool_name="click",
selector=selector,
action="left",
selector_type="css",
success=False,
error_message="Element not found",
)
```
### Captured Data
The logger captures:
- Tool name and action type
- Selector used
- Selector type (CSS vs custom)
- Success/failure status
- Alternative selectors for the element
- Element attributes
- Error messages on failure
## Agent Integration
### Browser Nav Agent
The primary agent that consumes browser tools. It receives natural language instructions and translates them into tool calls.
**Tool Invocation Pattern:**
```python
# From executor_nav_agent.py
async def execute_task(self, instruction: str):
# Agent decides which tool to call
# Tool receives selector from instruction
result = await click_using_selector(
selector="[md='submit-button']",
click_type="left"
)
# Agent processes result
if "error" in result.get("summary_message", "").lower():
# Handle error or try alternative
```
### Executor Nav Agent
Handles script execution and Python sandbox operations:
**Script Execution Context:**
| Variable | Type | Description |
|----------|------|-------------|
| `page` | `Page` | Playwright page object |
| `browser` | `Browser` | Playwright browser instance |
| `context` | `BrowserContext` | Browser context |
| `playwright_manager` | `PlaywrightManager` | Manager instance |
| `logger` | `Logger` | Logging utility |
| `config` | `GlobalConf` | Global configuration |
资料来源：[testzeus_hercules/core/agents/executor_nav_agent.py:1-50]()
## Bulk Operations
The tool system supports bulk execution for batch operations:
### Bulk Slider Tool
```python
@tool(
agent_names=["browser_nav_agent"],
description="used to set slider values in multiple sliders in single attempt.",
name="bulk_set_slider"
)
async def bulk_set_slider(
entries: Annotated[
List[List[str]],
"List of [selector, value] pairs",
],
) -> Annotated[List[Dict[str, str]], "List of results"]
```
## Configuration
### Command Line Arguments
The tool system behavior can be configured via CLI:
| Argument | Type | Description |
|----------|------|-------------|
| `--llm-model` | `str` | LLM model for agent decisions |
| `--llm-temperature` | `float` | LLM sampling temperature (0.0-1.0) |
| `--agents-llm-config-file` | `str` | Path to agents LLM config file |
| `--enable-portkey` | `flag` | Enable Portkey LLM routing |
| `--portkey-api-key` | `str` | Portkey API key |
| `--browser-channel` | `str` | Browser channel (chrome-beta, etc.) |
| `--browser-version` | `str` | Specific browser version |
| `--enable-ublock` | `flag` | Enable uBlock Origin extension |
| `--load-extra-tools` | `str` | Load extra tools (default: true) |
资料来源：[testzeus_hercules/config.py:1-100]()
## Summary
The Tool System provides:
1. **Unified Interface**: All browser interactions follow the `@tool` decorator pattern
2. **Agent Compatibility**: Tools specify which agents can invoke them
3. **Error Resilience**: Graceful handling of element not found, not visible, and execution failures
4. **Proof Generation**: Comprehensive logging of all selector interactions
5. **Extensibility**: Dynamic loading of extra tools without core modifications
6. **Standardized Results**: Consistent return format across all tools
This architecture enables AI agents to reliably control browser automation while maintaining clean separation of concerns and testable component boundaries.
---
## LLM Configuration
### 相关页面
相关主题：[Agent System](#agent-system), [Memory Management](#memory-management)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/agents_llm_config_manager.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents_llm_config_manager.py)
- [testzeus_hercules/core/agents_llm_config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents_llm_config.py)
- [agents_llm_config-example.json](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/agents_llm_config-example.json)
- [testzeus_hercules/utils/llm_helper.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/llm_helper.py)
- [testzeus_hercules/utils/model_utils.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/model_utils.py)
- [docs/GPT5_USAGE.md](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/docs/GPT5_USAGE.md)
# LLM Configuration
## Overview
The LLM Configuration system in testzeus-hercules provides a flexible, multi-provider framework for configuring Large Language Models across different agent types. This system enables the framework to support various LLM providers (OpenAI, Anthropic, Ollama, Azure) while maintaining provider-specific configurations through a centralized registry mechanism.
The configuration architecture supports:
- Multiple LLM providers simultaneously
- Per-agent model selection
- Dynamic parameter adaptation based on model family
- External configuration file loading
- Environment variable integration
资料来源：[testzeus_hercules/core/agents_llm_config.py:1-20]()
## Architecture
### High-Level Component Architecture
```mermaid
graph TD
A[CLI Arguments / Environment] --> B[Global Config]
C[agents-llm-config-file.json] --> D[AgentsLLMConfig]
D --> E[AgentRegistry]
E --> F[Provider Configs]
F --> G[planner_agent]
F --> H[nav_agent]
F --> I[mem_agent]
F --> J[helper_agent]
B --> K[Model Utils]
K --> L[adapt_llm_params_for_model]
L --> G
L --> H
L --> I
L --> J
```
### Configuration Flow
```mermaid
sequenceDiagram
participant CLI as CLI Arguments
participant Config as Global Config
participant JSON as LLM Config File
participant Processor as AgentsLLMConfig
participant Registry as AgentRegistry
participant Agent as SimpleHercules
CLI->>Config: Parse --llm-* arguments
JSON->>Processor: load_from_file()
Processor->>Registry: register_provider()
Registry->>Registry: Store configs per provider
Config->>Agent: Pass model configs
Agent->>ModelUtils: adapt_llm_params_for_model()
ModelUtils->>Agent: Adapted LLM params
```
资料来源：[testzeus_hercules/core/agents_llm_config.py:25-50]()
## Core Components
### AgentRegistry
The `AgentRegistry` manages configurations for multiple providers and supports switching between them.
```python
class AgentRegistry:
def __init__(self) -> None:
self._providers: Dict[str, Dict[str, AgentConfig]] = {}
self._active_provider: Optional[str] = None
```
| Method | Purpose |
|--------|---------|
| `register_provider(provider_key, configs)` | Register agent configs for a provider |
| `get_active_provider()` | Retrieve currently active provider configuration |
| `set_active_provider(provider_key)` | Switch active provider |
| `get_all_providers()` | List all registered providers |
资料来源：[testzeus_hercules/core/agents_llm_config.py:20-35]()
### AgentsLLMConfig
The main configuration processor that handles loading and normalization of agent configurations.
```python
class AgentsLLMConfig:
def __init__(self) -> None:
self.registry = AgentRegistry()
def load_from_file(self, file_path: str, provider_key: Optional[str] = None) -> None
def normalize_agent_config(self, agent_config: Dict[str, Any]) -> AgentConfig
```
资料来源：[testzeus_hercules/core/agents_llm_config.py:40-70]()
## Agent Types
The framework defines four specialized agent roles, each with configurable LLM parameters:
| Agent Type | Purpose | Typical Model Requirements |
|------------|---------|----------------------------|
| `planner_agent` | Strategic task planning and decomposition | High reasoning capability |
| `nav_agent` | Browser navigation and UI interaction | Vision-capable, fast response |
| `mem_agent` | Memory and context management | Balanced performance |
| `helper_agent` | Utility functions and data processing | Variable based on task |
资料来源：[testzeus_hercules/core/simple_hercules.py:1-30]()
## Configuration File Format
### JSON Schema Structure
```json
{
"provider_name": {
"agent_type": {
"model_name": "string",
"model_api_key": "string",
"model_api_type": "openai|anthropic|azure|ollama",
"model_client_host": "string (optional)",
"model_native_tool_calls": true|false,
"model_hide_tools": "if_any_run|user|never",
"llm_config_params": {
"cache_seed": null|integer,
"temperature": 0.0,
"seed": 12345,
"max_tokens": 4096,
"presence_penalty": 0.0,
"frequency_penalty": 0.0,
"stop": []
}
}
}
}
```
### Example Configuration
```json
{
"openai": {
"planner_agent": {
"model_name": "gpt-4o",
"model_api_key": "${OPENAI_API_KEY}",
"model_api_type": "openai",
"llm_config_params": {
"cache_seed": null,
"temperature": 0.0,
"seed": 12345
}
}
},
"anthropic": {
"nav_agent": {
"model_name": "claude-3-5-haiku-latest",
"model_api_key": "",
"model_api_type": "anthropic",
"llm_config_params": {
"cache_seed": null,
"temperature": 0.0
}
}
}
}
```
资料来源：[agents_llm_config-example.json:1-60]()
## CLI Configuration Options
The framework provides comprehensive command-line arguments for LLM configuration:
### Basic LLM Parameters
| Argument | Type | Description |
|----------|------|-------------|
| `--llm-model` | string | Name of the LLM model |
| `--llm-model-api-key` | string | API key for the LLM model |
| `--llm-model-base-url` | string | Base URL for the LLM API |
| `--llm-model-api-type` | string | API type (openai, anthropic, azure, etc.) |
| `--llm-temperature` | float | Temperature for LLM sampling (0.0-1.0) |
### LLM Configuration File Options
| Argument | Type | Description |
|----------|------|-------------|
| `--agents-llm-config-file` | string | Path to the agents LLM configuration file |
| `--agents-llm-config-file-ref-key` | string | Reference key for selecting provider |
### Parameter Configuration
| Argument | Type | Description |
|----------|------|-------------|
| `LLM_MODEL_PRICING` | env | Model pricing for cost tracking |
| `LLM_MODEL_TEMPERATURE` | env | Default temperature setting |
| `LLM_MODEL_CACHE_SEED` | env | Cache seed for reproducible results |
| `LLM_MODEL_SEED` | env | Random seed for generation |
| `LLM_MODEL_MAX_TOKENS` | env | Maximum tokens in response |
| `LLM_MODEL_PRESENCE_PENALTY` | env | Presence penalty (-2.0 to 2.0) |
| `LLM_MODEL_FREQUENCY_PENALTY` | env | Frequency penalty (-2.0 to 2.0) |
| `LLM_MODEL_STOP` | env | Stop sequences |
资料来源：[testzeus_hercules/config.py:1-100]()
## Portkey Integration
The framework supports Portkey for advanced LLM routing with fallback and load balancing capabilities.
### Portkey Configuration Options
| Argument | Type | Description |
|----------|------|-------------|
| `--enable-portkey` | flag | Enable Portkey integration |
| `--portkey-api-key` | string | API key for Portkey |
| `--portkey-strategy` | choice | Routing strategy: `fallback` or `loadbalance` |
### Environment Variables
| Variable | Description |
|----------|-------------|
| `ENABLE_PORTKEY` | Enable/disable Portkey |
| `PORTKEY_API_KEY` | Portkey API key |
| `PORTKEY_STRATEGY` | Routing strategy |
| `PORTKEY_CACHE_ENABLED` | Enable response caching |
| `PORTKEY_TARGETS` | Target models for routing |
| `PORTKEY_GUARDRAILS` | Enable safety guardrails |
| `PORTKEY_RETRY_COUNT` | Number of retries on failure |
资料来源：[testzeus_hercules/config.py:50-80]()
## Model Parameter Adaptation
The `model_utils` module provides intelligent parameter adaptation based on the model family being used.
### adapt_llm_params_for_model
```python
def adapt_llm_params_for_model(model_name: str, llm_config_params: Dict) -> Dict
```
This function automatically adjusts LLM parameters based on the detected model family:
| Model Family | Adaptation Behavior |
|--------------|---------------------|
| o1-series | Removes temperature, adjusts max_tokens handling |
| GPT-4o | Standard parameters |
| Claude | Adjusts for Anthropic API format |
| Ollama | Configures for local inference |
### Applied to All Agents
```python
# In SimpleHercules initialization
planner_model = self.planner_agent_config["model_config_params"].get("model") or \
self.planner_agent_config["model_config_params"].get("model_name")
self.planner_agent_config["llm_config_params"] = adapt_llm_params_for_model(
planner_model,
self.planner_agent_config["llm_config_params"]
)
```
资料来源：[testzeus_hercules/core/simple_hercules.py:100-130]()
## LLM Helper Utilities
The `llm_helper` module provides utility functions for LLM interactions:
| Function | Purpose |
|----------|---------|
| `convert_model_config_to_autogen_format()` | Convert config to AutoGen format |
| `create_multimodal_agent()` | Create agents with vision capabilities |
| `extract_target_helper()` | Extract target information from responses |
| `format_plan_steps()` | Format planning step outputs |
| `parse_agent_response()` | Parse agent response structure |
| `process_chat_message_content()` | Process chat message content |
| `parse_response()` | General response parsing |
资料来源：[testzeus_hercules/utils/llm_helper.py:1-30]()
## Environment Variable Configuration
### Full Configuration Matrix
| Environment Variable | Type | Default | Purpose |
|---------------------|------|---------|---------|
| `LLM_MODEL_PRICING` | dict | - | Model pricing information |
| `LLM_MODEL_TEMPERATURE` | float | 0.0 | Default sampling temperature |
| `LLM_MODEL_CACHE_SEED` | int | null | Caching seed |
| `LLM_MODEL_SEED` | int | - | Random seed |
| `LLM_MODEL_MAX_TOKENS` | int | 4096 | Max response tokens |
| `LLM_MODEL_PRESENCE_PENALTY` | float | 0.0 | Presence penalty |
| `LLM_MODEL_FREQUENCY_PENALTY` | float | 0.0 | Frequency penalty |
| `LLM_MODEL_STOP` | list | [] | Stop sequences |
| `TOKEN_VERBOSE` | bool | false | Enable token verbose logging |
| `HF_HOME` | path | - | HuggingFace cache location |
| `TOKENIZERS_PARALLELISM` | bool | false | Parallel tokenizer config |
资料来源：[testzeus_hercules/config.py:100-150]()
## Multi-Provider Configuration
### Provider Switching
The system supports runtime provider switching through the configuration file:
```mermaid
graph LR
A[Config File] --> B["provider: openai"]
A --> C["provider: anthropic"]
B --> D["planner_agent: GPT-4"]
B --> E["nav_agent: GPT-4o-mini"]
C --> F["planner_agent: Claude-3"]
C --> G["nav_agent: Claude-3-Haiku"]
```
### Selecting Active Provider
```bash
python -m testzeus_hercules \
--agents-llm-config-file ./config.json \
--agents-llm-config-file-ref-key "anthropic"
```
资料来源：[testzeus_hercules/core/agents_llm_config.py:60-80]()
## Integration with SimpleHercules
The `SimpleHercules` class integrates all LLM configurations:
```python
class SimpleHercules:
def __init__(
self,
planner_agent_config: Dict[str, Any],
nav_agent_config: Dict[str, Any],
mem_agent_config: Dict[str, Any],
helper_agent_config: Dict[str, Any],
planner_max_chat_round: int = 50,
browser_nav_max_chat_round: int = 100,
):
# Configuration processing
self.planner_agent_config = planner_agent_config
self.nav_agent_config = nav_agent_config
self.mem_agent_config = mem_agent_config
self.helper_agent_config = helper_agent_config
# Parameter adaptation per agent
from testzeus_hercules.utils.model_utils import adapt_llm_params_for_model
self.planner_agent_config["llm_config_params"] = adapt_llm_params_for_model(
planner_model,
self.planner_agent_config["llm_config_params"]
)
```
资料来源：[testzeus_hercules/core/simple_hercules.py:50-100]()
## Best Practices
### 1. Configuration File Organization
- Group configurations by provider (openai, anthropic, etc.)
- Use environment variables for sensitive API keys
- Maintain consistent agent naming across providers
### 2. Model Selection Guidelines
| Use Case | Recommended Models |
|----------|-------------------|
| Complex planning | GPT-4, Claude-3-Opus |
| Fast navigation | GPT-4o-mini, Claude-3-Haiku |
| Vision tasks | GPT-4o, Claude-3-Sonnet |
| Local inference | Ollama models |
### 3. Parameter Tuning
- Use `temperature: 0.0` for deterministic outputs
- Set appropriate `max_tokens` based on expected response length
- Enable `model_native_tool_calls` for better function calling
### 4. Cost Optimization
- Use `LLM_MODEL_PRICING` for tracking
- Enable Portkey caching with `PORTKEY_CACHE_ENABLED`
- Consider fallback strategies for reliability
## Troubleshooting
### Common Issues
| Issue | Solution |
|-------|----------|
| Model not recognized | Check `model_name` matches provider format |
| Temperature ignored | Some models (o1-series) ignore temperature parameter |
| API key errors | Ensure `${ENV_VAR}` syntax or actual key in config |
| Provider not found | Verify provider key matches config file structure |
### Debug Configuration
```bash
export TOKEN_VERBOSE=true
export ENABLE_BROWSER_LOGS=true
python -m testzeus_hercules --llm-model gpt-4o --llm-temperature 0.7
```
## See Also
- [GPT-5 Usage Guide](../docs/GPT5_USAGE.md)
- [Browser Configuration](./browser-configuration.md)
- [Agent Architecture](./agent-architecture.md)
---
## Memory Management
### 相关页面
相关主题：[Agent System](#agent-system), [LLM Configuration](#llm-configuration)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/memory/state_handler.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/memory/state_handler.py)
- [testzeus_hercules/core/memory/dynamic_ltm.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/memory/dynamic_ltm.py)
- [testzeus_hercules/core/memory/static_ltm.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/memory/static_ltm.py)
# Memory Management
## Overview
The Memory Management system in testzeus-hercules provides persistent and contextual memory capabilities for AI agents executing browser automation tests. The system consists of three primary components: **Static LTM** (Long Term Memory), **Dynamic LTM**, and **State Handler**, each serving distinct purposes in managing test execution context and data persistence.
The architecture follows a multi-layered approach where Static LTM loads pre-configured test data, Dynamic LTM manages vector-based retrieval augmented memory, and State Handler provides runtime state tracking for agent coordination.
```mermaid
graph TD
A[Testzeus-Hercules] --> B[Static LTM]
A --> C[Dynamic LTM]
A --> D[State Handler]
B --> E[Test Data Files]
B --> F[Stored Data]
B --> G[Run Data]
C --> H[Vector Store]
C --> I[RetrieveUserProxyAgent]
D --> J[_state_string]
D --> K[_state_dict]
L[Agents] --> M[Memory Access]
M --> B
M --> C
M --> D
```
## Static LTM (Long Term Memory)
### Purpose and Scope
Static LTM is responsible for loading and consolidating pre-configured test data at application initialization. It operates as a singleton pattern, ensuring that all test data is loaded once and made available throughout the test execution lifecycle.
资料来源：[testzeus_hercules/core/memory/static_ltm.py:1-47]()
### Implementation
The `StaticLTM` class extends the singleton pattern to ensure only one instance exists:
```python
class StaticLTM:
_instance = None
def __new__(cls) -> "StaticLTM":
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialize()
return cls._instance
```
资料来源：[testzeus_hercules/core/memory/static_ltm.py:17-22]()
### Data Consolidation
During initialization, Static LTM consolidates three types of data sources:
| Data Source | Description | Method |
|-------------|-------------|--------|
| Base Test Data | Loaded from `test_data.txt` via `load_data()` | `StaticDataLoader` |
| Stored Data | User-defined test artifacts | `get_stored_data()` |
| Run Data | Previous test execution context | `get_run_data()` |
资料来源：[testzeus_hercules/core/memory/static_ltm.py:26-34]()
The consolidated data is stored in `self.consolidated_data` and accessed via `get_user_ltm()`:
```python
def get_user_ltm(self) -> Optional[str]:
return self.consolidated_data
```
资料来源：[testzeus_hercules/core/memory/static_ltm.py:40-47]()
### Usage Pattern
Agents access Static LTM through a module-level function:
```python
def get_user_ltm() -> Optional[str]:
return StaticLTM().get_user_ltm()
```
资料来源：[testzeus_hercules/core/memory/static_ltm.py:50-54]()
## Dynamic LTM
### Purpose and Scope
Dynamic LTM provides runtime memory management with vector-based retrieval capabilities. It enables agents to store, retrieve, and utilize contextual information during test execution using a `RetrieveUserProxyAgent` backed by ChromaDB for vector storage.
资料来源：[testzeus_hercules/core/memory/dynamic_ltm.py:1-40]()
### Core Components
#### SilentRetrieveUserProxyAgent
A specialized agent that extends `RetrieveUserProxyAgent` with suppressed output to prevent console noise during agent conversations:
```python
class SilentRetrieveUserProxyAgent(RetrieveUserProxyAgent):
@suppress_prints
def initiate_chat(self, *args: Any, **kwargs: Any) -> Any:
return super().initiate_chat(*args, **kwargs)
@suppress_prints
async def a_initiate_chat(self, *args: Any, **kwargs: Any) -> Any:
return await super().a_initiate_chat(*args, **kwargs)
```
资料来源：[testzeus_hercules/core/memory/dynamic_ltm.py:37-46]()
#### Print Suppression Decorator
The `@suppress_prints` decorator redirects stdout to a `StringIO` buffer during function execution:
```python
def suppress_prints(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
silent_stdout = io.StringIO()
original_stdout, sys.stdout = sys.stdout, silent_stdout
try:
return func(*args, **kwargs)
finally:
sys.stdout = original_stdout
return wrapper
```
资料来源：[testzeus_hercules/core/memory/dynamic_ltm.py:17-29]()
### Integration with Configuration
Dynamic LTM respects global configuration to enable or disable its functionality:
```python
from testzeus_hercules.config import get_global_conf
def save_content(self, content: str) -> None:
config = get_global_conf()
if not config.should_use_dynamic_ltm():
return # Skip when disabled
```
资料来源：[testzeus_hercules/core/simple_hercules.py:89-94]()
### External Dependencies
Dynamic LTM utilizes the `unstructured` library for document parsing:
```python
from unstructured.documents.elements import NarrativeText, Text, Title
from unstructured.partition.auto import partition
```
资料来源：[testzeus_hercules/core/memory/dynamic_ltm.py:13-14]()
## State Handler
### Purpose and Scope
State Handler provides lightweight runtime state management for coordinating data between agents during test execution. It maintains module-level dictionaries for storing string-based state and structured data.
资料来源：[testzeus_hercules/core/memory/state_handler.py:1-70]()
### Module-Level State Storage
```python
# Module-level state string
_state_string: Dict[str, str] = defaultdict(str)
_state_dict: Dict[str, Any] = defaultdict(deque)
```
资料来源：[testzeus_hercules/core/memory/state_handler.py:13-14]()
### store_data Tool
The `store_data` function is registered as a tool for browser, API, and SQL navigation agents to persist information:
```python
@tool(
agent_names=["browser_nav_agent", "api_nav_agent", "sql_nav_agent"],
description="Tool to store information.",
name="store_data",
)
def store_data(
text: Annotated[str, "The confirmation of stored value."],
) -> Annotated[Dict[str, Union[str, None]], "A dictionary containing a 'message' key..."]:
global _state_string
try:
DynamicLTM().save_content(text)
_state_string[get_global_conf().get_default_test_id()] += text
return {"message": "Text appended successfully."}
except Exception as e:
return {"error": str(e)}
```
资料来源：[testzeus_hercules/core/memory/state_handler.py:23-47]()
### Key Behaviors
| Behavior | Description |
|----------|-------------|
| Test ID Isolation | State is keyed by `get_global_conf().get_default_test_id()` |
| Dual Storage | Data propagates to both `_state_string` and Dynamic LTM |
| Error Resilience | Returns error dictionary instead of raising exceptions |
资料来源：[testzeus_hercules/core/memory/state_handler.py:30-46]()
## Memory Architecture Diagram
```mermaid
graph LR
subgraph "Initialization Phase"
A[Load Config] --> B[StaticLTM Singleton]
B --> C[Load test_data.txt]
C --> D[Consolidate Data]
end
subgraph "Runtime Phase"
E[Agents Execute Tests]
E --> F[store_data Tool Call]
F --> G[State Handler]
G --> H[DynamicLTM.save_content]
H --> I[Vector Store Update]
J[Test Query] --> K[RetrieveUserProxyAgent]
K --> L[Vector Similarity Search]
L --> M[Context Injection]
M --> E
end
```
## Integration with SimpleHercules
The `SimpleHercules` class coordinates all memory components:
```python
class SimpleHercules:
def _save_to_memory(self, content: str) -> None:
"""Helper method to save content to memory."""
config = get_global_conf()
if not config.should_use_dynamic_ltm():
return
if self.memory:
self.memory.save_content(content)
else:
logger.warning("Memory system not initialized")
```
资料来源：[testzeus_hercules/core/simple_hercules.py:85-97]()
### Memory Initialization Flow
```mermaid
sequenceDiagram
participant SH as SimpleHercules
participant DLT as DynamicLTM
participant CFG as Config
participant LOG as Logger
SH->>CFG: should_use_dynamic_ltm()
CFG-->>SH: boolean
SH->>DLT: save_content(content)
alt LTM Enabled
DLT->>DLT: Vector store update
DLT-->>SH: success
else LTM Disabled
DLT-->>SH: skipped
end
```
## Configuration
### Global Configuration Methods
| Method | Purpose |
|--------|---------|
| `should_use_dynamic_ltm()` | Check if Dynamic LTM is enabled |
| `get_hf_home()` | Get HuggingFace cache directory for vector store |
| `get_default_test_id()` | Get current test execution identifier |
### Related Configuration File
The configuration is managed through `testzeus_hercules/config.py`, which provides command-line arguments for memory-related settings including:
- `--reuse-vector-db`: Reuse existing vector DB instead of creating fresh one
- `--sandbox-tenant-id`: Python sandbox tenant configuration
资料来源：[testzeus_hercules/config.py:45-58]()
## Data Flow Summary
| Layer | Storage Type | Access Pattern | Persistence |
|-------|--------------|----------------|-------------|
| Static LTM | In-memory string | Singleton `get_user_ltm()` | Session-scoped |
| Dynamic LTM | Vector (ChromaDB) | RetrieveUserProxyAgent | Persistent |
| State Handler | In-memory dict | Module-level `_state_string` | Test execution-scoped |
## Error Handling
All memory components implement robust error handling:
```python
try:
DynamicLTM().save_content(text)
_state_string[get_global_conf().get_default_test_id()] += text
return {"message": "Text appended successfully."}
except Exception as e:
traceback.print_exc()
logger.error(f"An error occurred while appending to state: {e}")
return {"error": str(e)}
```
资料来源：[testzeus_hercules/core/memory/state_handler.py:30-42]()
## Related Components
| Component | File Path | Role |
|-----------|-----------|------|
| PlannerAgent | `core/agents/high_level_planner_agent.py` | Consumes memory for test planning |
| ExecutorNavAgent | `core/agents/executor_nav_agent.py` | Executes test steps with memory context |
| BaseNavAgent | `core/agents/base_nav_agent.py` | Agent base class with memory integration |
## Summary
The Memory Management system in testzeus-hercules implements a comprehensive multi-tier approach:
1. **Static LTM** provides pre-loaded test data consolidation via singleton pattern
2. **Dynamic LTM** offers vector-based retrieval augmented memory for contextual queries
3. **State Handler** enables runtime state sharing between agents through the `store_data` tool
This architecture ensures agents have access to both static test fixtures and dynamic execution context, enabling sophisticated AI-driven browser automation testing.
---
## API Testing
### 相关页面
相关主题：[Security Testing](#security-testing), [Tool System](#tool-system)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/tools/api_calls.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/api_calls.py)
- [testzeus_hercules/core/tools/sql_calls.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/sql_calls.py)
- [testzeus_hercules/core/agents/api_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/api_nav_agent.py)
- [helper_scripts/generate_api_functional_gherkin_test.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/helper_scripts/generate_api_functional_gherkin_test.py)
- [testzeus_hercules/core/tools/execute_python_sandbox.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/execute_python_sandbox.py)
- [testzeus_hercules/config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
- [testzeus_hercules/utils/response_parser.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/response_parser.py)
# API Testing
## Overview
API Testing in testzeus-hercules enables automated end-to-end testing of REST APIs through AI-driven agents. The system leverages LLM-powered agents to parse OpenAPI specifications, generate Gherkin test scenarios, execute API calls, and validate responses against expected outcomes. This module integrates with the broader Hercules testing framework to provide comprehensive API validation capabilities alongside browser-based UI testing.
The API Testing feature accepts OpenAPI specification files (YAML or JSON format) and automatically generates executable Gherkin test cases that can be run against live API endpoints. The generated tests follow behavior-driven development (BDD) conventions, making them readable for both technical and non-technical stakeholders.
## Architecture
The API Testing module consists of several interconnected components that work together to provide end-to-end API testing capabilities.
### Component Overview
```mermaid
graph TD
A[OpenAPI Specification] --> B[generate_api_functional_gherkin_test.py]
B --> C[Gherkin Test Cases]
C --> D[API Navigation Agent]
D --> E[API Calls Tool]
E --> F[SQL Calls Tool]
E --> G[Python Sandbox Executor]
F --> H[Database Validation]
G --> I[Custom Logic Validation]
D --> J[Response Parser]
J --> K[Test Results]
```
### Core Components
| Component | File Path | Purpose |
|-----------|-----------|---------|
| API Navigation Agent | `testzeus_hercules/core/agents/api_nav_agent.py` | Orchestrates API test execution using LLM-driven decision making |
| API Calls Tool | `testzeus_hercules/core/tools/api_calls.py` | Executes HTTP requests to API endpoints |
| SQL Calls Tool | `testzeus_hercules/core/tools/sql_calls.py` | Validates API data against database state |
| Python Sandbox | `testzeus_hercules/core/tools/execute_python_sandbox.py` | Executes custom validation logic |
| Gherkin Generator | `helper_scripts/generate_api_functional_gherkin_test.py` | Generates test cases from OpenAPI specs |
| Response Parser | `testzeus_hercules/utils/response_parser.py` | Parses and validates API responses |
资料来源：[helper_scripts/generate_api_functional_gherkin_test.py:1-80]()
## Test Generation Workflow
### OpenAPI Specification Processing
The test generation process begins with parsing OpenAPI specification files. The system accepts both YAML and JSON formatted OpenAPI specs through the `generate_api_functional_gherkin_test.py` helper script.
```python
parser.add_argument(
"input_files",
metavar="input_files",
type=str,
nargs="+",
help="One or more OpenAPI spec files (YAML or JSON).",
)
```
资料来源：[helper_scripts/generate_api_functional_gherkin_test.py:15-22]()
### Gherkin Test Case Generation
The LLM generates test cases based on the OpenAPI specification content. The generation uses a specialized prompt that instructs the model to produce Gherkin-format scenarios covering functional test cases.
```python
def generate_test_cases(prompt: str, model: str) -> str:
"""Generates test cases using the OpenAI API."""
client = OpenAI()
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
temperature=0.7,
)
return completion.choices[0].message.content
```
资料来源：[helper_scripts/generate_api_functional_gherkin_test.py:82-90]()
### Generation Parameters
| Parameter | CLI Flag | Default | Description |
|-----------|----------|---------|-------------|
| Model | `--model` | o1-preview | LLM model for test generation |
| Output Folder | `--output` | (required) | Destination for generated feature files |
| Number of Test Cases | `--number_of_testcase` | 100 | Maximum test cases to generate per endpoint |
## API Navigation Agent
The API Navigation Agent (`api_nav_agent.py`) serves as the orchestrator for executing API tests. It receives parsed test scenarios and coordinates execution across multiple tools to validate API behavior.
```mermaid
sequenceDiagram
participant Test as Test Scenario
participant Agent as API Nav Agent
participant API as API Calls Tool
participant SQL as SQL Calls Tool
participant Sandbox as Python Sandbox
Test->>Agent: Execute scenario
Agent->>API: Send HTTP request
API-->>Agent: Response data
Agent->>SQL: Validate database state
SQL-->>Agent: Validation result
Agent->>Sandbox: Run custom assertions
Sandbox-->>Agent: Assertion results
Agent->>Test: Pass/Fail outcome
```
资料来源：[testzeus_hercules/core/agents/api_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/api_nav_agent.py)
## Execution Environment
### Python Sandbox
API tests execute within a secured Python sandbox environment that provides controlled access to necessary resources while maintaining isolation.
```python
def _get_config_driven_injections(config: Any) -> Dict[str, Any]:
"""
Get injections defined in configuration.
Allows dynamic configuration of available modules.
"""
injections = {}
# Read from config: SANDBOX_PACKAGES="requests,pandas,numpy"
sandbox_packages = config.get_config().get("SANDBOX_PACKAGES", "").split(",")
for package_name in sandbox_packages:
package_name = package_name.strip()
if package_name:
try:
injections[package_name] = __import__(package_name)
except ImportError:
logger.warning(f"Could not import configured package: {package_name}")
return injections
```
资料来源：[testzeus_hercules/core/tools/execute_python_sandbox.py:80-100]()
### Sandbox Access Variables
Scripts executing within the sandbox have automatic access to the following variables:
| Variable | Type | Description |
|----------|------|-------------|
| `page` | Playwright Page | Current browser page context |
| `browser` | Browser instance | Active browser session |
| `context` | Browser Context | Isolated browsing context |
| `playwright_manager` | PlaywrightManager | Manages Playwright lifecycle |
| `logger` | Logger | Logging utility |
| `config` | Configuration | Global configuration object |
Additional tenant-specific modules can be injected based on the `SANDBOX_TENANT_ID` environment variable, and custom injections are available via the `SANDBOX_CUSTOM_INJECTIONS` environment variable.
资料来源：[testzeus_hercules/core/tools/execute_python_sandbox.py:40-55]()
## Response Handling
### JSON Response Parsing
The response parser handles API responses with multiple fallback strategies for extracting structured data:
```python
def parse_response(message: str) -> dict[str, Any]:
# Check if message is wrapped in ```json ``` blocks
if "```json" in message:
start_idx = message.find("```json") + 7
end_idx = message.find("```", start_idx + 7)
message = message[start_idx:end_idx]
else:
if message.startswith("```"):
message = message[3:]
if message.endswith("```):
message = message[:-3]
if message.startswith("json"):
message = message[4:]
message = message.strip()
message = message.replace("\\n", "\n")
json_response: dict[str, Any] = json.loads(message)
```
资料来源：[testzeus_hercules/utils/response_parser.py:9-35]()
### Error Recovery
When JSON parsing fails, the response parser attempts to extract plan and next_step fields from unstructured responses, ensuring graceful degradation when APIs return non-standard response formats.
## Configuration
### LLM Configuration
API Testing relies on LLM configuration for test generation and agent decision-making. Configuration can be provided via command-line arguments or through a dedicated configuration file.
```python
parser.add_argument(
"--llm-model",
type=str,
help="Name of the LLM model.",
required=False,
)
parser.add_argument(
"--llm-temperature",
type=float,
help="Temperature for LLM sampling (0.0-1.0).",
required=False,
)
```
资料来源：[testzeus_hercules/config.py:35-45]()
### Agents LLM Config File
For multi-agent configurations, specify the configuration file path:
```bash
--agents-llm-config-file /path/to/agents_llm_config.json
--agents-llm-config-file-ref-key
```
### Portkey Integration
Enable Portkey for LLM routing with fallback or load balancing strategies:
```bash
--enable-portkey
--portkey-api-key
--portkey-strategy fallback|loadbalance
```
资料来源：[testzeus_hercules/config.py:60-75]()
## Usage Examples
### Generate Gherkin Tests from OpenAPI Spec
```bash
python helper_scripts/generate_api_functional_gherkin_test.py \
spec/openapi.yaml \
--output tests/api/ \
--model gpt-4 \
--number_of_testcase 50
```
### Run API Tests
Tests can be executed through the main Hercules CLI or integrated into CI/CD pipelines. The agent configuration file supports specifying different models for different agents:
```json
{
"openai": {
"planner_agent": {
"model_name": "gpt-4",
"model_api_type": "openai"
}
}
}
```
资料来源：[agents_llm_config-example.json](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/agents_llm_config-example.json)
## Integration with Browser Testing
The API Testing module integrates seamlessly with browser-based testing capabilities. The API Navigation Agent can coordinate with the Browser Navigation Agent to perform scenarios that span both API validation and UI verification.
When executing multi-step workflows, the system can:
1. Call API endpoints to set up test data
2. Launch browsers to verify UI state reflects API changes
3. Execute SQL queries to validate data persistence
4. Run custom Python assertions for complex business logic
## Security Considerations
### API Key Management
API keys should be provided through environment variables or secure configuration management, never hardcoded in test files:
```bash
export OPENAI_API_KEY=
export PORTKEY_API_KEY=
```
### Sandbox Isolation
The Python sandbox provides execution isolation for custom test logic. Configure allowed packages through the `SANDBOX_PACKAGES` configuration parameter to limit access to only required libraries.
## Best Practices
1. **Organize OpenAPI specs by version** - Maintain separate specification files for different API versions
2. **Use meaningful test case names** - Generated tests should clearly describe the scenario being validated
3. **Combine with database validation** - Use SQL Calls Tool to verify data consistency
4. **Leverage response parsing** - Use the response parser for handling complex API response formats
5. **Configure appropriate LLM models** - Use faster models for generation and more capable models for complex validation logic
## Related Documentation
- [CONTRIBUTING.md](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/CONTRIBUTING.md) - Contribution guidelines for the project
- [Makefile](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/Makefile) - Build and test automation targets
- [Browser Testing](https://github.com/test-zeus-ai/testzeus-hercules) - Companion documentation for UI testing
---
## Security Testing
### 相关页面
相关主题：[API Testing](#api-testing), [Tool System](#tool-system)
相关源码文件
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/tools/api_sec_calls.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/api_sec_calls.py)
- [testzeus_hercules/core/agents/sec_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/sec_nav_agent.py)
- [helper_scripts/generate_api_security_gherkin_test.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/helper_scripts/generate_api_security_gherkin_test.py)
# Security Testing
Security Testing in TestZeus Hercules is an automated framework designed to validate API security by generating and executing Gherkin-based test scenarios. The system leverages LLM-powered agents to analyze OpenAPI specifications and produce comprehensive security validation tests that check for vulnerabilities, configuration weaknesses, and proper handling of sensitive data.
## Overview
The Security Testing module provides an end-to-end solution for validating API security without requiring manual test case authoring. It integrates with the broader Hercules testing framework to execute security validation scenarios alongside functional and navigation tests.
### Core Components
| Component | File Path | Purpose |
|-----------|-----------|---------|
| Security Navigation Agent | `testzeus_hercules/core/agents/sec_nav_agent.py` | Orchestrates security test execution using LLM-driven agents |
| API Security Calls | `testzeus_hercules/core/tools/api_sec_calls.py` | Provides low-level HTTP client operations for security validation |
| Gherkin Test Generator | `helper_scripts/generate_api_security_gherkin_test.py` | Generates security-focused Gherkin test cases from OpenAPI specs |
## Architecture
```mermaid
graph TD
A[OpenAPI Spec Files] --> B[generate_api_security_gherkin_test.py]
B --> C[LLM API - OpenAI]
C --> D[Security Gherkin Test Cases]
D --> E[Hercules Test Executor]
E --> F[sec_nav_agent.py]
F --> G[api_sec_calls.py]
G --> H[Target API Endpoints]
H --> I[Security Validation Results]
J[Configuration] --> F
K[LLM Config] --> B
```
## Security Navigation Agent
The `sec_nav_agent.py` module implements the `BrowserNavAgent` pattern specialized for security testing scenarios. It follows the same agent architecture used by the browser navigation agent but focuses on API security validation.
### Agent Configuration
The security agent inherits the core agent configuration structure from the Hercules framework, utilizing the same LLM integration patterns as the main navigation agent defined in `simple_hercules.py`. 资料来源：[testzeus_hercules/core/agents/sec_nav_agent.py:1-50]()
### Execution Flow
```mermaid
sequenceDiagram
participant TestRunner
participant SecNavAgent
participant APISecCalls
participant TargetAPI
participant ResponseParser
TestRunner->>SecNavAgent: Execute security test scenario
SecNavAgent->>APISecCalls: Send HTTP request with security payload
APISecCalls->>TargetAPI: Validated HTTP request
TargetAPI->>APISecCalls: Response with headers/body
APISecCalls->>ResponseParser: Parse response data
ResponseParser->>SecNavAgent: Structured security results
SecNavAgent->>TestRunner: Security validation report
```
## API Security Calls Module
The `api_sec_calls.py` module provides the foundational HTTP client capabilities for executing security tests. It supports various HTTP methods and authentication schemes required for comprehensive API security testing.
### Supported Security Test Operations
| Operation | Description | Authentication Support |
|-----------|-------------|----------------------|
| GET Security Headers | Validate presence and correctness of security headers | Bearer, API Key, Basic |
| POST Injection Tests | Execute payload injection for XSS, SQLi validation | Bearer, API Key |
| Authentication Bypass | Test unauthorized access to protected endpoints | Token validation |
| Rate Limiting | Verify rate limiting mechanisms | None required |
| CORS Validation | Check cross-origin resource sharing policies | None required |
资料来源：[testzeus_hercules/core/tools/api_sec_calls.py:1-100]()
### Request Configuration
```python
# Security test request structure
{
"method": "GET|POST|PUT|DELETE|PATCH",
"url": "https://api.target.com/endpoint",
"headers": {
"Authorization": "Bearer {token}",
"Content-Type": "application/json"
},
"params": {}, # Query parameters
"data": {}, # Request body for POST/PUT/PATCH
"timeout": 30,
"verify_ssl": true
}
```
## Gherkin Test Generation
The `generate_api_security_gherkin_test.py` helper script uses LLM to automatically generate security-focused Gherkin test cases from OpenAPI specification files.
### Input Processing
The generator accepts OpenAPI specifications in both YAML and JSON formats, parsing the specification to identify:
- Endpoints and their HTTP methods
- Security schemes defined in the spec
- Request/response schemas
- Authentication requirements
资料来源：[helper_scripts/generate_api_security_gherkin_test.py:1-60]()
### Generation Prompt Strategy
The LLM prompt instructs the model to focus on generating tests that validate:
1. **Vulnerability Detection**: Tests that check for common vulnerabilities
2. **Configuration Weaknesses**: Validation of security configurations
3. **Sensitive Data Handling**: Verification of proper data protection
4. **Authentication/Authorization**: Access control testing
5. **Input Validation**: Sanitization and validation checks
### Output Format
Generated test cases follow this structure:
```gherkin
Feature: API Security Validation - {Endpoint_Name}
Scenario: Validate security headers on {method} {path}
Given the API endpoint "{path}" requires authentication
When I send a {method} request without authorization
Then the response should have status code 401
And the response should include "WWW-Authenticate" header
Scenario: Test for SQL injection vulnerability on {path}
Given the API endpoint "{path}" accepts query parameters
When I send a GET request with malicious payload in parameter "id"
Then the response should have status code 400 or 422
And no SQL error should be present in response body
```
资料来源：[helper_scripts/generate_api_security_gherkin_test.py:80-120]()
## Command Line Interface
### Running Security Tests
```bash
# Generate security tests from OpenAPI spec
python -m helper_scripts.generate_api_security_gherkin_test \
--input spec.yaml \
--output ./tests/security \
--model gpt-4o \
--number_of_testcase 50
# Execute security tests with Hercules
testzeus-hercules --input-file ./tests/security/api_security.feature
```
### Generation Script Arguments
| Argument | Type | Default | Description |
|----------|------|---------|-------------|
| `input_files` | list[str] | required | One or more OpenAPI spec files (YAML or JSON) |
| `--output` | str | required | Output folder for generated feature files |
| `--model` | str | o1-preview | OpenAI model for test generation |
| `--number_of_testcase` | int | 100 | Number of test cases to generate |
资料来源：[helper_scripts/generate_api_security_gherkin_test.py:30-55]()
## Integration with Hercules Framework
### Agent Initialization
The security agent is initialized through the same mechanism as other Hercules agents, using configuration from the LLM configuration file specified via CLI:
```bash
testzeus-hercules \
--input-file security_tests.feature \
--agents-llm-config-file ./config/security_agent.yaml \
--llm-model gpt-4o
```
### Execution Context
Security tests execute within the same context as functional tests, providing:
- Shared browser/playwright session management
- Consistent logging and telemetry
- Unified reporting and result aggregation
- Access to shared utilities and helpers
## Security Test Scenarios
### Common Test Categories
| Category | Test Focus | Example Validation |
|----------|------------|-------------------|
| Authentication | Token validation, session management | Invalid token returns 401 |
| Authorization | Access control, privilege escalation | User cannot access admin endpoints |
| Input Validation | Payload sanitization, type checking | Malformed input returns 400 |
| Headers | Security header presence | X-Frame-Options, CSP headers present |
| Rate Limiting | Request throttling | Excessive requests return 429 |
| CORS | Cross-origin policy | Invalid origins rejected |
## Best Practices
### Test Data Management
- Use dedicated security test environments
- Isolate security tests from production data
- Implement proper cleanup for test artifacts
- Rotate API keys/tokens used in tests
### Test Coverage
- Aim for comprehensive endpoint coverage
- Include both positive and negative test cases
- Validate all security headers defined in your policy
- Test authentication bypass scenarios
## Configuration
### Environment Variables
| Variable | Purpose |
|----------|---------|
| `OPENAI_API_KEY` | Required for LLM-powered test generation |
| `SECURITY_TEST_API_KEY` | API key for testing authenticated endpoints |
| `SECURITY_TEST_BASE_URL` | Override target API base URL |
### LLM Configuration
The security agent uses the same LLM configuration structure as other agents, specified through:
```yaml
# security_agent_config.yaml
llm_config:
model: gpt-4o
temperature: 0.7
max_tokens: 4096
other_settings:
system_prompt: "You are a security testing expert..."
max_consecutive_auto_reply: 10
```
## Reporting
Security test results are integrated into the standard Hercules reporting format:
- Test pass/fail status per scenario
- Detailed assertion results
- HTTP request/response logging
- Security-specific metrics (headers present, vulnerabilities detected)
## Related Documentation
- [Browser Navigation Agent](testzeus_hercules/core/agents/executor_nav_agent.py) - Core navigation patterns
- [Simple Hercules](testzeus_hercules/core/simple_hercules.py) - Main framework architecture
- [API Functional Testing](helper_scripts/generate_api_functional_gherkin_test.py) - Functional test generation
---
## MCP Integration
### 相关页面
相关主题：[Agent System](#agent-system), [Tool System](#tool-system)
Related Source Files
以下源码文件用于生成本页说明：
- [testzeus_hercules/core/agents/mcp_nav_agent.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/mcp_nav_agent.py)
- [testzeus_hercules/core/tools/mcp_tools.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/tools/mcp_tools.py)
- [testzeus_hercules/utils/mcp_helper.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/mcp_helper.py)
- [mcp_servers.example.json](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/mcp_servers.example.json)
- [testzeus_hercules/config.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/config.py)
# MCP Integration
## Overview
The MCP (Model Context Protocol) Integration in TestZeus Hercules enables the testing agent to discover, catalog, and execute tools exposed by external MCP servers. This integration allows Hercules to extend its capabilities by leveraging tools from multiple Model Context Protocol-compliant servers during end-to-end testing workflows.
MCP serves as a standardized communication layer that allows the testing framework to:
- Enumerate and connect to configured MCP servers
- List available tools and resource namespaces from each connected server
- Execute remote tool calls with correct parameters
- Retrieve resources by URI when required for test execution
资料来源：[testzeus_hercules/core/agents/mcp_nav_agent.py:1-10]()
## Architecture
### System Components
The MCP integration is built on three primary components:
| Component | File | Purpose |
|-----------|------|---------|
| McpNavAgent | `core/agents/mcp_nav_agent.py` | Main navigation agent that orchestrates MCP server interactions |
| MCPHelper | `utils/mcp_helper.py` | Utility class providing MCP client functionality |
| MCP Tools | `core/tools/mcp_tools.py` | Tool implementations for MCP operations |
### Component Relationship
```mermaid
graph TD
A[TestZeus Hercules Core] --> B[McpNavAgent]
B --> C[MCPHelper]
C --> D[MCP Servers]
B --> E[get_configured_mcp_servers]
B --> F[check_mcp_server_connection]
B --> G[execute_mcp_tool]
B --> H[read_mcp_resource]
E --> I[Server Discovery]
F --> J[Connection Status]
G --> K[Tool Execution]
H --> L[Resource Retrieval]
```
## McpNavAgent
The `McpNavAgent` is the central agent responsible for all MCP-related operations. It inherits from `BaseNavAgent` and implements the Model Context Protocol interaction patterns.
资料来源：[testzeus_hercules/core/agents/mcp_nav_agent.py:6-9]()
### Agent Configuration
| Property | Value | Description |
|----------|-------|-------------|
| `agent_name` | `mcp_nav_agent` | Unique identifier for the agent |
| Inherits | `BaseNavAgent` | Base navigation agent functionality |
### Core Functions
The MCP Navigation Agent implements the following core functions:
1. **Server Discovery** - Enumerate configured MCP servers and their connection status
2. **Capability Cataloging** - List tools and resource namespaces for each connected server
3. **Tool Execution** - Call tools with correct parameters and handle responses
4. **Resource Retrieval** - Read resources by URI when required
5. **Result Summarization** - Capture server, tool, arguments, outputs; include timings and status
资料来源：[testzeus_hercules/core/agents/mcp_nav_agent.py:14-28]()
### Operational Rules
#### Rule 1: Previous Step Validation
Before any new action, explicitly review the previous step and its outcome. Do not proceed if the prior critical step failed; address it first.
```mermaid
graph TD
A[Execute Action] --> B{Previous Step Succeeded?}
B -->|No| C[Address Failure First]
B -->|Yes| D[Continue to Next Action]
C --> D
```
#### Rule 2: Server Scan First
The agent must call `get_configured_mcp_servers` and for each server, call `check_mcp_server_connection` before taking any other action.
资料来源：[testzeus_hercules/core/agents/mcp_nav_agent.py:31-35]()
### Agent Prompt
The agent uses a specialized system prompt that defines its role and behavioral guidelines:
```
### MCP Navigation Agent
You are an MCP (Model Context Protocol) Navigation Agent that assists the Testing Agent by discovering MCP servers, cataloging their exposed tools/resources, and executing the right tool calls to complete the task. Always begin by scanning all configured servers before taking any action.
```
资料来源：[testzeus_hercules/core/agents/mcp_nav_agent.py:11-21]()
## MCPHelper Utility
The `MCPHelper` class provides the underlying functionality for MCP server interactions. It is exported through the `mcp_helper.py` module and integrates with the agent system through `set_mcp_agents`.
资料来源：[testzeus_hercules/utils/mcp_helper.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/utils/mcp_helper.py)
### Key Functions
| Function | Purpose |
|----------|---------|
| `MCPHelper` | Main helper class for MCP operations |
| `set_mcp_agents` | Configures MCP agents within the testing framework |
## Configuration
### Configuration File Format
MCP servers are configured using a JSON file. The example file `mcp_servers.example.json` demonstrates the expected format:
```json
{
"mcpServers": {
"server_name": {
"command": "command_to_run",
"args": ["arg1", "arg2"],
"env": {
"KEY": "value"
}
}
}
}
```
### Command Line Arguments
The MCP integration can be configured through command line arguments:
| Argument | Type | Description |
|----------|------|-------------|
| `--agents-llm-config-file` | string | Path to the agents LLM configuration file |
| `--agents-llm-config-file-ref-key` | string | Reference key for the agents LLM configuration file |
资料来源：[testzeus_hercules/config.py:27-39]()
## Workflow
### Standard MCP Interaction Flow
```mermaid
graph TD
A[Start Test Execution] --> B[Initialize McpNavAgent]
B --> C[Call get_configured_mcp_servers]
C --> D[For Each Server]
D --> E[Call check_mcp_server_connection]
E --> F{Server Connected?}
F -->|No| G[Log Error / Skip Server]
F -->|Yes| H[Catalog Tools & Resources]
H --> I[Task Requires MCP Tool?]
I -->|Yes| J[Call execute_mcp_tool]
I -->|No| K[Continue with Other Tasks]
J --> L[Process Tool Response]
L --> M[Return Results to Testing Agent]
G --> D
K --> N[Complete Test]
M --> N
```
### Tool Execution Workflow
When executing MCP tools, the agent follows this sequence:
1. Identify the target MCP server
2. Verify server connection status
3. Determine the correct tool and parameters
4. Execute the tool call via MCP protocol
5. Capture response including timing and status
6. Return formatted results to the testing agent
## Integration with Testing Framework
### Agent Hierarchy
```mermaid
graph BT
A[BrowserNavAgent] --> B[BaseNavAgent]
C[ApiNavAgent] --> B
D[SqlNavAgent] --> B
E[McpNavAgent] --> B
F[SecNavAgent] --> B
B --> G[TestZeus Hercules Core]
```
All navigation agents, including `McpNavAgent`, inherit from `BaseNavAgent`, ensuring consistent behavior and integration with the core testing framework.
资料来源：[testzeus_hercules/core/agents/__init__.py](https://github.com/test-zeus-ai/testzeus-hercules/blob/main/testzeus_hercules/core/agents/__init__.py)
### Available Navigation Agents
| Agent | Purpose |
|-------|---------|
| `BrowserNavAgent` | Web browser interaction and navigation |
| `ApiNavAgent` | API testing and validation |
| `SqlNavAgent` | Database query execution |
| `McpNavAgent` | MCP server tool execution |
| `SecNavAgent` | Security testing operations |
## Best Practices
### Initialization
1. Always ensure MCP servers are properly configured before test execution
2. Verify server connectivity before attempting tool calls
3. Use the configured servers list as the authoritative source of available MCP servers
### Error Handling
1. Check previous step outcomes before proceeding
2. Log connection failures with server identification
3. Handle tool execution errors with proper parameter validation
4. Provide clear error messages when MCP operations fail
### Task Focus
- Execute only actions required by the primary testing task
- Use extra information from MCP responses cautiously
- Avoid unnecessary server scans after initial discovery
## Security Considerations
The MCP integration supports sensitive operations requiring careful configuration:
- API keys should be provided through secure environment variables
- Server configurations should be validated before use
- Tool execution permissions should be properly scoped
- Resource access should follow least-privilege principles
## Summary
The MCP Integration module provides TestZeus Hercules with the ability to extend its testing capabilities through external MCP servers. By implementing a dedicated `McpNavAgent` that follows standardized MCP protocols, the framework can seamlessly discover servers, catalog their capabilities, and execute tools as needed during end-to-end testing scenarios.
Key benefits include:
- **Extensibility**: Add new testing capabilities without modifying core framework code
- **Standardization**: Uses the Model Context Protocol for consistent server communication
- **Resource Management**: Access remote resources via standardized URI-based retrieval
- **Comprehensive Logging**: Captures server status, tool execution times, and results
---
---
## Doramagic 踩坑日志
项目：test-zeus-ai/testzeus-hercules
摘要：发现 13 个潜在踩坑项，其中 0 个为 high/blocking；最高优先级：配置坑 - 来源证据：0.1.1。
## 1. 配置坑 · 来源证据：0.1.1
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个配置相关的待验证问题：0.1.1
- 对用户的影响：可能增加新用户试用和生产接入成本。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_87e563b6a81f4a529799ac528ee21610 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.1.1 | 来源类型 github_release 暴露的待验证使用条件。
## 2. 能力坑 · 能力判断依赖假设
- 严重度：medium
- 证据强度：source_linked
- 发现：README/documentation is current enough for a first validation pass.
- 对用户的影响：假设不成立时，用户拿不到承诺的能力。
- 建议检查：将假设转成下游验证清单。
- 防护动作：假设必须转成验证项；没有验证结果前不能写成事实。
- 证据：capability.assumptions | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | README/documentation is current enough for a first validation pass.
## 3. 维护坑 · 来源证据：0.0.40
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：0.0.40
- 对用户的影响：可能增加新用户试用和生产接入成本。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_02c1945d063b4151a3df1526b85e9d50 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.0.40 | 来源类型 github_release 暴露的待验证使用条件。
## 4. 维护坑 · 来源证据：0.1.0
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：0.1.0
- 对用户的影响：可能影响升级、迁移或版本选择。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_1d1eb9ae89f04ff9b339c73cc932f2f7 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.1.0 | 来源类型 github_release 暴露的待验证使用条件。
## 5. 维护坑 · 来源证据：0.1.2
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：0.1.2
- 对用户的影响：可能增加新用户试用和生产接入成本。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_e0e02b07ea6845eaac8c4f18871aaa32 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.1.2 | 来源类型 github_release 暴露的待验证使用条件。
## 6. 维护坑 · 来源证据：0.1.6
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个维护/版本相关的待验证问题：0.1.6
- 对用户的影响：可能增加新用户试用和生产接入成本。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_41cf6e34c47b4f929a67426b258b8fd8 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.1.6 | 来源类型 github_release 暴露的待验证使用条件。
## 7. 维护坑 · 维护活跃度未知
- 严重度：medium
- 证据强度：source_linked
- 发现：未记录 last_activity_observed。
- 对用户的影响：新项目、停更项目和活跃项目会被混在一起，推荐信任度下降。
- 建议检查：补 GitHub 最近 commit、release、issue/PR 响应信号。
- 防护动作：维护活跃度未知时，推荐强度不能标为高信任。
- 证据：evidence.maintainer_signals | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | last_activity_observed missing
## 8. 安全/权限坑 · 下游验证发现风险项
- 严重度：medium
- 证据强度：source_linked
- 发现：no_demo
- 对用户的影响：下游已经要求复核，不能在页面中弱化。
- 建议检查：进入安全/权限治理复核队列。
- 防护动作：下游风险存在时必须保持 review/recommendation 降级。
- 证据：downstream_validation.risk_items | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | no_demo; severity=medium
## 9. 安全/权限坑 · 存在评分风险
- 严重度：medium
- 证据强度：source_linked
- 发现：no_demo
- 对用户的影响：风险会影响是否适合普通用户安装。
- 建议检查：把风险写入边界卡，并确认是否需要人工复核。
- 防护动作：评分风险必须进入边界卡，不能只作为内部分数。
- 证据：risks.scoring_risks | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | no_demo; severity=medium
## 10. 安全/权限坑 · 来源证据：0.1.4
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：0.1.4
- 对用户的影响：可能影响授权、密钥配置或安全边界。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_d6aeb053b3a541778b07c0bca68c5c82 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.1.4 | 来源类型 github_release 暴露的待验证使用条件。
## 11. 安全/权限坑 · 来源证据：0.2.2
- 严重度：medium
- 证据强度：source_linked
- 发现：GitHub 社区证据显示该项目存在一个安全/权限相关的待验证问题：0.2.2
- 对用户的影响：可能增加新用户试用和生产接入成本。
- 建议检查：来源显示可能已有修复、规避或版本变化，说明书中必须标注适用版本。
- 防护动作：不得脱离来源链接放大为确定性结论；需要标注适用版本和复核状态。
- 证据：community_evidence:github | cevd_963ec16ebc32452a8ca13e8ebc1aed96 | https://github.com/test-zeus-ai/testzeus-hercules/releases/tag/0.2.2 | 来源讨论提到 python 相关条件，需在安装/试用前复核。
## 12. 维护坑 · issue/PR 响应质量未知
- 严重度：low
- 证据强度：source_linked
- 发现：issue_or_pr_quality=unknown。
- 对用户的影响：用户无法判断遇到问题后是否有人维护。
- 建议检查：抽样最近 issue/PR，判断是否长期无人处理。
- 防护动作：issue/PR 响应未知时，必须提示维护风险。
- 证据：evidence.maintainer_signals | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | issue_or_pr_quality=unknown
## 13. 维护坑 · 发布节奏不明确
- 严重度：low
- 证据强度：source_linked
- 发现：release_recency=unknown。
- 对用户的影响：安装命令和文档可能落后于代码，用户踩坑概率升高。
- 建议检查：确认最近 release/tag 和 README 安装命令是否一致。
- 防护动作：发布节奏未知或过期时，安装说明必须标注可能漂移。
- 证据：evidence.maintainer_signals | github_repo:888701643 | https://github.com/test-zeus-ai/testzeus-hercules | release_recency=unknown