Related Pages

Related topics: System Architecture: CLI + Daemon, Protocol, and Session Registry, Tool Reference: Commands, Aria Snapshots, and Code Generation

Overview, Installation, and Quick Start

se-cli (also referred to as selenium-cli) is a Node.js-based CLI and long-running daemon that exposes Selenium WebDriver and Selenium 4 BiDi capabilities to AI coding agents in a token-efficient, shell-friendly form. It targets the same use cases as Playwright or MCP-based browser tools, but favors short, plain-text commands over JSON-schema-heavy tool calls to reduce the token cost of agent-driven browser automation. Source: README.md:1-12

Project Goals and Scope

The project's stated focus is giving agents a way to drive a real browser without paying the schema overhead of full MCP servers or the verbosity of raw WebDriver JSON. It exposes a small command surface — session start, navigate, click, type, screenshot, evaluate JavaScript, and tear-down — so an agent can run an end-to-end browser scenario with a handful of shell invocations. Source: README.md:14-38

Key scope decisions reflected in the repository:

• The core runtime is a persistent local daemon that holds a single browser session, so session reuse is cheap across commands. Source: src/daemon.ts:1-40
• Browser I/O is delegated to Selenium (WebDriver / BiDi) rather than a custom CDP client, which lets the tool reuse whichever browser the host machine already has installed. Source: README.md:40-55
• A skill/SKILL.md is shipped alongside the CLI so agents can self-document the available verbs and argument shapes. Source: skill/SKILL.md:1-24

Community discussion also flags longer-term scope items that this page deliberately does not over-claim: cloud browser providers (Browserbase, Browserless, etc.), Python/Java SDKs, and MCP server mode are tracked as future enhancements rather than shipped capabilities. Source: community issue #26, #25, #27

Installation

se-cli is published as an npm package and can be installed either globally or invoked on demand via npx.

Prerequisites:

• Node.js 18 or later (the daemon and CLI rely on native fetch and Web Streams). Source: package.json:18-22
• A local Selenium-compatible browser: Chrome/Chromium, Firefox, or Microsoft Edge, plus a matching WebDriver (e.g. chromedriver, geckodriver, or Selenium Manager 4+). Source: docs/installation.md:6-19
• On Linux, the usual X/Wayland or headless dependencies if not running purely headless. Source: docs/installation.md:21-27

Install steps (global):

npm install -g se-cli
se --version

Install steps (project-local, recommended for CI):

npm install --save-dev se-cli
npx se --version

After install, the se (or selenium-cli) binary on PATH is the entry point; it bootstraps the daemon on first use and persists it across commands. Source: bin/se.js:1-30

Quick Start

A minimal agent-driven session typically follows four phases: start the daemon/session, drive the browser, capture evidence, and clean up.

sequenceDiagram
    participant Agent
    participant CLI as se CLI (bin/se.js)
    participant Daemon as se-daemon (src/daemon.ts)
    participant Browser as Selenium / BiDi
    Agent->>CLI: se session start
    CLI->>Daemon: spawn + handshake
    Daemon->>Browser: new WebDriver / BiDi session
    Agent->>CLI: se navigate / click / type
    CLI->>Daemon: RPC over local socket
    Daemon->>Browser: WebDriver command
    Browser-->>Agent: screenshot / text result

Because the daemon holds the session, every subsequent se invocation skips the browser-launch cost. Source: src/daemon.ts:42-70

How Agents Use It

Agents consume se-cli in two complementary ways:

1. Direct shell calls. The agent emits se <verb> <args> from a Bash-style tool; the output is short, human-readable text (e.g. an element handle or a trimmed DOM snippet), which keeps prompt tokens low. Source: skill/SKILL.md:6-40
2. Skill-file lookup. The shipped skill/SKILL.md is consumed by agent runtimes that auto-load skill files (Claude Code, Cursor, GitHub Copilot CLI), giving the model an authoritative reference for command grammar instead of relying on guesses. Source: skill/SKILL.md:1-24

A community request (#28) tracks tailoring skill/SKILL.md per agent so each runtime gets idiomatic command shapes; until that lands, the current single skill file is the canonical reference for every supported agent. Source: community issue #28

Verification After Install

A few sanity checks help confirm the install is wired correctly before depending on it in an agent workflow:

• se doctor prints the detected Node version, browser binary path, and WebDriver version. Source: src/cli/commands.ts:120-138
• se session start --browser chrome --headless followed by se navigate https://example.com and se session stop exercises the daemon bootstrap, BiDi/WebDriver round-trip, and clean shutdown path in one shot. Source: src/daemon.ts:72-104
• If a command hangs, se daemon status and se daemon stop --force give a clean recovery without orphaning the browser process. Source: src/cli/commands.ts:140-162

With these checks passing, se-cli is ready to be invoked by an agent in place of a heavier MCP browser server, while leaving the door open to the planned cloud-provider and SDK integrations tracked in issues #25 and #26.

Related Pages

Related topics: Tool Reference: Commands, Aria Snapshots, and Code Generation, AI Agent Integration, Extensibility, and Roadmap

System Architecture: CLI + Daemon, Protocol, and Session Registry

selenium-cli is structured as two cooperating processes: a short-lived CLI frontend and a long-running Node.js daemon. The CLI parses user commands, the daemon owns browser sessions, and a JSON-over-stdio protocol glues them together. This page documents the runtime topology, the responsibility split, the session registry that backs it, and the message envelope that travels across the boundary.

Runtime Topology

The CLI binary and the daemon never share memory. Every invocation of the CLI (selenium …) either re-uses an already-running daemon or spawns one fresh and then talks to it through newline-delimited JSON written to its stdin. Output is read from stdout; diagnostics and lifecycle events go to stderr, keeping them out of the machine-readable channel.

┌────────┐   stdin (JSONL)    ┌─────────────┐
│  CLI   │ ─────────────────▶ │   Daemon    │
│process │ ◀───────────────── │  (Node.js)  │
└────────┘   stdout (JSONL)   └─────────────┘
                                     │
                          ┌──────────┴──────────┐
                          │  Session Registry   │
                          │   (in-process map)  │
                          └─────────────┬───────┘
                               ┌────────┴────────┐
                            session-A        session-B
                          (WebDriver/BiDi)  (WebDriver/BiDi)

This split exists because spawning a real browser per command is expensive, and because keeping the browser connection alive across many small CLI invocations is what makes the tool usable from agents that call it hundreds of times in a row.

CLI Process Responsibilities

src/cli.ts is the entry point invoked by the bin script. Its job is intentionally narrow: resolve the user's subcommand, parse flags, dispatch a single protocol request, await the matching response, and print it. It does not import WebDriver, manage sessions, or touch the registry — those concerns live in the daemon.

Command surface is defined in src/program.ts, where each subcommand is registered with its name, argument shape, and the handler that builds the protocol envelope. Handlers in src/program.ts only know how to translate *user intent* into a request frame; they do not know how the daemon will execute it. This keeps the CLI thin and the daemon the single source of truth for browser state.

Source: src/cli.ts() — entry point and dispatch loop. Source: src/program.ts() — subcommand and argument definitions.

Daemon, Sessions, and the Registry

The daemon is a long-lived Node.js process that holds a SessionRegistry instance, defined in src/registry.ts. The registry is an in-process map keyed by a session id, with each entry wrapping a live Session (defined in src/session.ts). A Session owns the WebDriver/BiDi connection, the target browser handle, and any per-session state needed to translate protocol commands into driver calls.

When the CLI sends a request that references a session, the daemon looks the id up in the registry, dispatches into the matching Session, and serializes the result back through src/response.ts. Sessions that go idle can be evicted; sessions explicitly closed are removed from the registry and their driver connection is shut down. Because the registry lives entirely inside the daemon, the CLI never holds driver objects directly and never has to reason about driver lifecycle.

This design is what makes future client SDKs possible: issue #25 tracks Python and Java bindings, and both can be implemented as thin protocol clients because all browser state is already centralized behind the registry.

Source: src/registry.ts() — registry data structure and lookup semantics. Source: src/session.ts() — per-session WebDriver/BiDi state.

Protocol and Response Envelopes

The wire format is defined in src/protocol.ts. Each request is a single JSON object containing the command name, the target session id (when relevant), and the command-specific payload. Requests are written one-per-line to the daemon's stdin, and each request carries a correlation id so that concurrent CLI invocations can share one daemon without their replies being misrouted.

Responses are shaped in src/response.ts. A response carries the same correlation id, a status (ok or error), the command-specific result payload on success, and a structured error code and message on failure. Errors are data, not exceptions across the boundary: a failed command produces a well-formed response the CLI can print or branch on, rather than a crash that takes the daemon down.

Source: src/protocol.ts() — request envelope and framing rules. Source: src/response.ts() — success and error envelope shape.

Design Trade-offs and Forward Compatibility

Keeping the protocol small and JSON-only is deliberate: it avoids the schema overhead that MCP clients must carry (see issue #27), which is exactly why selenium-cli is token-efficient for agents. The same property is what would let a future MCP compatibility layer wrap the daemon without changing the registry or the session code — the boundary is already clean.

The registry-in-process model also dovetails with issue #26's cloud-browser-provider direction: a provider backend can be slotted behind the Session abstraction while the registry, protocol, and CLI stay unchanged. As long as new session types honor the same protocol contract, they integrate without touching the CLI surface in src/program.ts.

In summary: the CLI is a stateless translator from flags to protocol frames; the daemon is a stateful host whose only state of consequence is the SessionRegistry; the protocol is the contract that makes those two halves independently replaceable.

Related Pages

Related topics: Overview, Installation, and Quick Start, AI Agent Integration, Extensibility, and Roadmap

Tool Reference: Commands, Aria Snapshots, and Code Generation

The se-cli (selenium-cli) daemon exposes a curated set of browser-automation "tools" that an AI agent can invoke by name. Each tool is registered in a central registry, validates its inputs against a Zod schema, dispatches into Selenium WebDriver, and returns a normalized result. This page is a technical reference to the tool surface, the ARIA snapshot mechanism used for element resolution, and the code-generation tool that converts a recorded session into a runnable Selenium script.

Tool Registry and Dispatch Model

All tools live under src/daemon/tools/ and are wired together in src/daemon/tools/index.ts. The registry exports a map of name → handler plus a parallel map of name → Zod schema, so the daemon can both validate incoming requests and route them to the right module without an if/else chain.

src/daemon/tools/shared.ts provides the shared infrastructure every tool depends on:

• defineTool({ name, schema, run }) — wraps a handler so its schema is automatically registered and its result is normalized through ToolResult.fromError and ToolResult.success. Source: src/daemon/tools/shared.ts.
• ToolResult — a discriminated result type carrying either a string payload or a structured { screenshot?, refs?, diff?, console? } envelope.
• LocatorRef — the canonical { kind: 'aria' | 'selector' | 'xpath', value } reference used by every interaction tool.

Because every tool is wrapped in defineTool, the registry's schemas map doubles as a self-describing schema dump that the daemon serves to clients for introspection. Source: src/daemon/tools/index.ts.

Core Interaction Tools

The interaction tools (click, fill, check, eval, navigate, screenshot) all accept a LocatorRef and translate it into a Selenium By locator before delegating to the WebDriver session.

Locators created from a LocatorRef go through toBy(ref) in shared.ts, which switches on kind to produce a By.css, By.xpath, or By.id selector. This is the single point at which the agent's abstract reference becomes a Selenium primitive. Source: src/daemon/tools/shared.ts.

Aria Snapshots

The snapshot tool is what lets an agent see the page in a form it can reason about. Instead of returning raw HTML, it walks the accessibility tree and produces an indented YAML-style outline where each node carries its ARIA role, accessible name, and a stable ref (e.g. [ref=e12]) that can be fed straight back into click, fill, or check. Source: src/daemon/tools/snapshot.ts.

Key properties of the snapshot format:

• Role + name — every visible node is emitted as - role "name", mimicking Playwright's aria-snapshot so prior agent prompts transfer cleanly.
• Refs are monotonic per session — the counter resets only when a new snapshot is taken, so a ref captured from snapshot N is valid until snapshot N+1.
• Filter modes — snapshot({ interactive: true }) keeps only roles an agent would act on (button, link, textbox, combobox, etc.); snapshot({ selector }) returns the subtree rooted at a CSS selector. Source: src/daemon/tools/snapshot.ts.

A typical round-trip looks like:

1. Agent calls snapshot({ interactive: true }).
2. Daemon walks the accessibility tree, assigns refs, returns the YAML text plus a refs map.
3. Agent replies with click({ ref: { kind: 'aria', value: 'e12' } }).
4. toBy in src/daemon/tools/shared.ts looks the ref up in the live map, resolves it to the current XPath/CSS, and dispatches the click.

Code Generation

After a successful session, an agent (or a human replaying one) can call code to materialize the recorded steps as a runnable Selenium script. The handler lives in src/daemon/tools/code.ts.

Inputs and behavior:

• language: 'javascript' | 'python' | 'java' — selects the emitter.
• framework: 'selenium' | 'pytest' | 'junit' — wraps the script in the right imports, fixtures, and setUp/tearDown boilerplate.
• includeScreenshots: boolean — when true, the emitter inserts a screenshot({ ref }) call after every interaction step and writes the PNGs next to the generated file.
• The tool reads from the daemon's in-memory step log (each defineTool call appends to it via shared.ts), so no separate recording step is required.

The output is a single self-contained file: a Java/Python file with the right entry-point signature, or a *.spec.js that can be run with node after npm i selenium-webdriver. Source: src/daemon/tools/code.ts.

Putting It Together

flowchart LR
  A[Agent request] --> B[index.ts registry]
  B --> C{Validate via Zod}
  C -- invalid --> X[ToolResult.fromError]
  C -- valid --> D[defineTool handler]
  D --> E[toBy / toRef resolver]
  E --> F[Selenium WebDriver]
  F --> G[ToolResult.success]
  G --> H{Generate code?}
  H -- yes --> I[code.ts emitter]
  H -- no --> J[Return to agent]

The three primitives — a closed-set tool surface, a ref-bearing ARIA snapshot, and a deterministic code emitter — are what let selenium-cli stay compact while still giving agents enough structure to drive a browser and then hand the result back to a human-readable test file. Source: src/daemon/tools/index.ts, src/daemon/tools/shared.ts, src/daemon/tools/snapshot.ts, src/daemon/tools/code.ts.

Related Community Threads

• Issue #25 proposes Python/Java SDK bindings alongside the code emitter; today, multi-language output is already supported via code's language parameter, but the SDK side is still pending. Source: src/daemon/tools/code.ts.
• Issue #27 discusses an MCP compatibility layer; the tool registry in index.ts is already structured so an MCP adapter could iterate over schemas and handlers without touching individual tools. Source: src/daemon/tools/index.ts.

Related Pages

Related topics: Overview, Installation, and Quick Start, Tool Reference: Commands, Aria Snapshots, and Code Generation

AI Agent Integration, Extensibility, and Roadmap

Overview

selenium-cli (se-cli) is designed as an AI-agent-friendly command-line wrapper around Selenium. It exposes a compact shell surface that coding agents (Claude Code, Cursor, GitHub Copilot CLI, and others) can invoke directly, and it runs a long-lived Node.js daemon that maintains a real browser session between calls. The integration story, the extensibility story, and the public roadmap converge on a single principle: keep the agent-facing surface stable and token-efficient while progressively widening the ecosystems (Python/Java SDKs, cloud browsers, MCP) that can drive the same daemon. Source: site/index.html:1-1

AI Agent Integration via `skill/SKILL.md`

The on-ramp for every coding agent is skill/SKILL.md, a single Markdown file that the agent ingests as system context. It describes the available CLI commands, expected flags, output conventions, and the daemon lifecycle (start, reuse, stop). Because the document is generic — covering command semantics without targeting any one agent's quirks — the same skill file can be dropped into Claude Code, Cursor, or GitHub Copilot CLI projects. Source: skill/SKILL.md:1-1

The intent is documented in the spec: the CLI must be self-describing for the agent, output concise human-readable text rather than verbose JSON, and never require the agent to parse a schema up front. This is the core token-saving mechanism versus full MCP-style tool calls. Source: docs/spec.md:1-1

Known limitation and next step

The community has flagged that the generic SKILL.md is not yet tailored to per-agent conventions (different skill-file formats, tool invocation patterns, and naming). A long-term proposal is to ship per-agent variants of SKILL.md (one each for Claude Code, Cursor, Copilot CLI) so each assistant receives guidance in its native shape. This is tracked as issue #28. Source: community issue #28

Extensibility Surfaces

The codebase exposes three extension axes, all centered on the Node.js daemon.

The CLI surface is the canonical one today: agents shell out, the daemon persists the WebDriver session, and tokens stay low because the agent never has to consume a tool schema per call. Source: docs/spec.md:1-1

MCP compatibility layer

Some agents cannot execute shell commands at all and only understand the Model Context Protocol. The proposal is to wrap the daemon as an MCP server so MCP-only clients can still reach existing selenium-cli sessions, while CLI users keep the token-efficient path. This is the planned bridge in issue #27. Source: community issue #27

Python and Java SDK bindings

The daemon is Node.js, but pytest-selenium, Robot Framework, and Selenium-Java users want first-class libraries. The roadmap calls for a Python SDK first (issue #25), talking to the daemon over its existing transport, followed by a Java equivalent. Source: community issue #25

Cloud browser providers

Running a local Chrome/Firefox is not always possible — CI runners and serverless hosts often lack a display. Integrating providers such as Browserbase, Browserless, Sauce Labs, and LambdaTest would let the daemon attach to remote browser endpoints through the standard WebDriver URL, removing the local-browser prerequisite. Source: community issue #26

Roadmap

The planning document groups work into near-term (current minor), mid-term, and long-term buckets. Near-term work focuses on tightening the agent-facing CLI and reducing per-call cost; mid-term work adds observability features; long-term work is the extensibility matrix above. Source: docs/plan.md:1-1

flowchart LR
    A[Agent] -->|shell| B[se CLI]
    B --> C[Node Daemon]
    C --> D[Local Browser]
    C -.planned.-> E[Cloud Browser]
    C -.planned.-> F[MCP Server]
    F -.planned.-> G[MCP-only Agent]
    C -.planned.-> H[Python SDK]
    C -.planned.-> I[Java SDK]

A concrete near-to-mid term deliverable is a simplified trace viewer (issue #24) that consumes Selenium 4 BiDi events to produce a Playwright-style timeline of DOM snapshots, screenshots, network, and console output per step — useful when an agent's run fails and needs post-mortem. Source: community issue #24

The manifest exposes version and entry points that the wiki and site use to stay in sync with the plan, so roadmap claims here track the same source-of-truth files. Source: site/manifest.json:1-1 Source: docs/index.html:1-1

Doramagic Pitfall Log

Found 23 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/25

2. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/28

3. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/17

4. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/15

5. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/18

6. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://github.com/se-cli/se-cli

7. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/11

8. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/13

9. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/12

10. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/24

11. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: Project evidence flags a capability evidence risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/se-cli/se-cli/issues/23

12. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/se-cli/se-cli

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using se-cli with real data or production workflows.

• feat(long-term): per-agent SKILL.md optimization - github / github_issue
• feat(long-term): MCP compatibility layer - github / github_issue
• feat(long-term): cloud browser provider integration - github / github_issue
• feat(long-term): Python/Java client SDK bindings - github / github_issue
• feat(v0.7): simplified trace viewer - github / github_issue
• feat(v0.7): plan/generate/heal test workflow - github / github_issue
• feat(v0.7): attach to pytest-selenium pause point - github / github_issue
• feat(v0.6): attach to Selenium Grid 4 (--endpoint) - github / github_issue
• feat(v0.6): attach --extension (browser extension control) - github / github_issue
• feat(v0.6): show --annotate (page annotation) - github / github_issue
• feat(v0.6): show dashboard (live session previews) - github / github_issue
• feat(v0.5): recording mode for test generation - github / github_issue