Related Pages

Related topics: Architecture, Server Bootstrap & Deployment, Error Handling, Async Workflows & Troubleshooting

Installation, Setup & Client Configuration

This page documents how to install, configure, and connect the Olostep MCP server to an MCP-compatible client (such as Claude Desktop). It covers the supported installation channels, the required environment variables, and the JSON snippets used to register the server with a host application.

Overview and Scope

The Olostep MCP server is distributed as an npm package that exposes Olostep's web data capabilities through the Model Context Protocol. Because it is shipped as a runnable CLI entry point, it can be launched directly through npx without a manual build step, or it can be containerized for environments where Node.js is not desirable. The setup process is intentionally minimal: obtain an API key, install the package, and register the server in the host client's configuration file.

Source: README.md:1-40

Prerequisites

An Olostep API key is required for every operational mode of the server. The key is read from the OLOSTEP_API_KEY environment variable at startup and is forwarded on each upstream request to the Olostep API. Without a valid key the server cannot authenticate, and the tool calls will fail.

Source: README.md:20-35, src/index.ts:1-40

Installation Channels

One-off execution via npx

The recommended way to run the server is through npx, which fetches and executes the package on demand. This avoids the need to clone the repository or manage a local node_modules tree.

macOS / Linux:

env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp

Windows (PowerShell):

$env:OLOSTEP_API_KEY="your-api-key"; npx -y olostep-mcp

Source: README.md:25-55

Local install from source

Developers who want to modify the server can clone the repository, install dependencies, and build the TypeScript sources. The package.json defines the relevant scripts and metadata, while tsconfig.json governs the TypeScript compilation.

git clone https://github.com/olostep/olostep-mcp-server
cd olostep-mcp-server
npm install
npm run build

Source: package.json:1-40, tsconfig.json:1-30

Containerized install via Docker

For environments that prefer containers, a Dockerfile is provided. The image bundles the compiled server and can be started with the API key passed as an environment variable.

docker build -t olostep-mcp .
docker run -e OLOSTEP_API_KEY=your-api-key olostep-mcp

Source: Dockerfile:1-30

Client Configuration

The server is registered with an MCP host by adding a JSON entry to the host's MCP configuration. The entry's command, args, and env fields describe how the host should spawn the server process.

macOS / Linux example:

{
  "mcpServers": {
    "olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "your-api-key"
      }
    }
  }
}

Windows example:

{
  "mcpServers": {
    "olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "your-api-key"
      }
    }
  }
}

After editing the configuration file, the host application must be restarted so it can spawn the new server process.

Source: README.md:55-95

Troubleshooting

The most frequently reported setup issue is a command not found error when invoking npx -y olostep-mcp. This was caused by the CLI binary not being resolved correctly under npx, and it was fixed by correcting the package's bin mapping in package.json. After the fix, npx -y olostep-mcp resolves to the published entry point and starts the server normally.

Source: package.json:10-25, issue #4

Other common checks when the server fails to start:

• Confirm OLOSTEP_API_KEY is exported in the same shell that launches npx.
• Confirm the configuration JSON is valid (no trailing commas) and that the mcpServers key is at the top level.
• Confirm the host application was fully restarted after editing its config file.
• If running inside Docker, confirm the API key was passed with -e and that the container has network access to the Olostep API.

Source: README.md:90-110, Dockerfile:10-30

Verifying the Setup

Once the host application is restarted, the Olostep tools should appear in the host's tool list. Invoking any tool will cause the host to spawn the configured command; if the process exits cleanly with an authentication error, the API key is the likely culprit, while an immediate exit without output typically indicates a malformed configuration entry.

Source: src/index.ts:20-60

Related Pages

Related topics: Error Handling, Async Workflows & Troubleshooting

MCP Tools Reference

Overview

The Olostep MCP Server exposes a set of Model Context Protocol (MCP) tools that allow LLM clients to perform web data retrieval operations through a single authenticated endpoint. Each tool maps to one capability of the underlying Olostep API and is registered under src/tools/ so the MCP runtime can advertise them to clients.

Tools follow the standard MCP tool contract: each defines a name, a human-readable description, and an inputSchema describing the JSON parameters accepted from the model. When a tool is invoked, the server forwards the request to Olostep using the OLOSTEP_API_KEY environment variable and returns the response to the model. Source: src/tools/scrapeWebsite.ts Source: src/tools/searchWeb.ts

Available Tools

The table above summarizes the six tool entry points defined under src/tools/. Each file exports a tool descriptor that the MCP server registers at startup. Source: src/tools/scrapeWebsite.ts Source: src/tools/batchScrapeUrls.ts Source: src/tools/createCrawl.ts Source: src/tools/createMap.ts

Tool Details

scrape_website

The flagship tool. It accepts a target url and returns structured page data suitable for downstream reasoning. It is the tool most commonly wired into agent loops that need page content. Source: src/tools/scrapeWebsite.ts

search_web

Provides search-engine-backed results. The tool forwards the query parameter to Olostep's search endpoint and returns ranked results that the model can cite or follow up on with scrape_website. Source: src/tools/searchWeb.ts

answers

A higher-level convenience tool that combines search and extraction to return a synthesized answer. It is intended for short factual lookups where the model wants a direct response rather than raw pages. Source: src/tools/answers.ts

batch_scrape_urls

Accepts an array of urls and returns results for each entry in a single round trip, reducing tool-call overhead when an agent needs data from several known pages. Source: src/tools/batchScrapeUrls.ts

create_crawl

Initiates an asynchronous crawl job rooted at a given url. The tool is paired with companion retrieval endpoints that the agent polls for status and pages, supporting large-scale site ingestion. Source: src/tools/createCrawl.ts

create_map

Returns a structural map of a domain (links, sections, page inventory). It is typically used before create_crawl to decide which subpaths to include. Source: src/tools/createMap.ts

Common Patterns

All six tool modules share the same conventions:

• Environment-based auth. The server reads OLOSTEP_API_KEY once at startup and attaches it as a bearer token on outbound requests. Missing or empty values surface as an initialization error rather than a per-tool failure. Source: src/tools/scrapeWebsite.ts

• Schema validation. Each tool declares a strict JSON schema for its inputs. Invalid payloads are rejected by the MCP layer before the handler runs, so handlers can assume well-typed arguments. Source: src/tools/searchWeb.ts Source: src/tools/batchScrapeUrls.ts

• Uniform error envelope. When the upstream API returns an error, tools propagate a structured message back to the model instead of throwing, keeping the agent loop stable. Source: src/tools/answers.ts

• Stateless invocation. Tools do not share in-memory state across calls. Long-running jobs (crawls) are identified by an ID returned from create_crawl, which the model passes to subsequent retrieval calls. Source: src/tools/createCrawl.ts

flowchart LR
  Client[LLM Client] -->|tool call| Server[Olostep MCP Server]
  Server -->|forward| Olostep[Olostep API]
  Olostep -->|response| Server
  Server -->|JSON result| Client

Community Notes

Users running the server via npx -y olostep-mcp initially hit a command not found error because the CLI binary was not packaged correctly. The issue was resolved by fixing CLI packaging so npx resolves the entry point. When installing, prefer the documented command:

• macOS/Linux: env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp

If the command fails with sh: olostep-mcp: command not found, verify that your Node.js version is supported and that the latest package version is being pulled. Source: community context (issue #4)

Choosing the Right Tool

• Use scrape_website for a single known page.
• Use batch_scrape_urls when you already have a list of pages.
• Use search_web first when the model only has a topic, then follow up with scrape_website on promising results.
• Use answers when the model wants a one-shot factual answer.
• Use create_map to scope a domain, then create_crawl to ingest it.

This ordering keeps tool calls minimal and matches how Olostep's underlying endpoints are designed to compose. Source: src/tools/createMap.ts Source: src/tools/createCrawl.ts

Related Pages

Related topics: Installation, Setup & Client Configuration, MCP Tools Reference

Architecture, Server Bootstrap & Deployment

The Olostep MCP server is a lightweight Model Context Protocol (MCP) wrapper that exposes Olostep's web scraping and crawling APIs as tools to MCP-compatible clients (Claude Desktop, Cursor, custom agents, etc.). This page documents the runtime architecture, bootstrap sequence, and supported deployment surfaces.

High-Level Architecture

The server is a single-process Node.js application that speaks the MCP protocol over a stdio transport. It does not run an HTTP listener itself; instead, it acts as a child process whose stdin/stdout is consumed by an MCP host.

Server Bootstrap Sequence

The bootstrap path is intentionally short and lives almost entirely in src/index.ts.

1. Process start — Node loads src/index.ts as the package entrypoint. The file performs its top-level statements on import: it imports the MCP SDK, the Olostep SDK, environment helpers, and any tool definitions.
2. Environment validation — The server requires OLOSTEP_API_KEY to authenticate calls to the Olostep API. If the key is missing or malformed, downstream SDK calls fail fast, surfacing an error to the host through the MCP error envelope.
3. Server construction — A new McpServer (or equivalent MCP SDK construct) is instantiated with server identity metadata such as name and version.
4. Tool registration — Each tool declared in docker-mcp-registry/tools.json (or the corresponding source-level definitions) is registered against the server using the SDK's tool(...) helper. Each handler validates inputs with a Zod-style schema, calls into the Olostep SDK, and returns either text content or structured JSON content.
5. Transport attach — A StdioServerTransport is created and connect() is called, hooking the server's JSON-RPC framing into the process's stdin and stdout.
6. Event loop — The Node event loop stays alive on the open stdio streams; the server responds to tools/list, tools/call, initialize, and other MCP requests until the host closes the pipes.

Source: src/index.ts (top-level entrypoint and tool registrations)

Deployment Surfaces

The repository ships three parallel deployment stories so the same server can be used by humans, CI agents, and Docker Desktop's MCP catalog.

1. npm / `npx` distribution

The package is published as olostep-mcp and is runnable in one command:

env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp

A previously reported issue (olostep-mcp: command not found when invoking via npx) was traced to a CLI packaging bug where the published bin entry did not match the executable the wrapper invoked. The fix landed in a follow-up release; users on older versions should upgrade to a release with the corrected bin mapping before retrying.

Source: Dockerfile (npm install / build steps that produce the olostep-mcp binary), community issue reference: olostep-mcp: command not found on npx -y olostep-mcp.

2. Container images

Two Dockerfiles are provided:

• Dockerfile — builds a slim Node image, installs dependencies, compiles TypeScript, and exposes the olostep-mcp binary as the container entrypoint. Used for local container runs and for the Docker MCP catalog image.
• Dockerfile.cloud — a cloud-oriented variant. It typically pins a base image suitable for hosted runtimes and is intended to be invoked by managed MCP hosts that spawn the server per session.

Both images rely on OLOSTEP_API_KEY being supplied via the orchestrator's secret/env mechanism; the image itself does not embed credentials.

Source: Dockerfile, Source: Dockerfile.cloud

3. Docker MCP Registry catalog

The docker-mcp-registry/ directory contains the artifacts needed to list the server inside Docker Desktop's MCP marketplace:

• server.yaml declares the server identity, the image reference (typically olostep/olostep-mcp-server), the required OLOSTEP_API_KEY secret, and the default transport (stdio).
• tools.json mirrors the tool surface (names, descriptions, input schemas) so the registry UI can render tool metadata without pulling the image.
• readme.md provides a curated, host-friendly description that the catalog displays alongside the install button.

This catalog path is what lets a user click "Install" in Docker Desktop and have the MCP server appear in their MCP-capable client with secrets wired up automatically.

Source: docker-mcp-registry/server.yaml, Source: docker-mcp-registry/tools.json, Source: docker-mcp-registry/readme.md

Operational Notes

• Transport is always stdio. All three deployment paths converge on the same StdioServerTransport; there is no separate HTTP mode in this repository.
• Single environment variable. Only OLOSTEP_API_KEY is required. Optional knobs (timeouts, regions, parser flags) are passed per-call through tool arguments rather than as process-level config.
• Stateless process. The server holds no in-memory state between requests and can be restarted at will by the host. This makes it safe to run under process supervisors, serverless containers, or ephemeral npx invocations.
• Failure surface. Misconfiguration (missing key, invalid arguments) is reported through MCP isError results rather than process crashes, so the host can decide whether to surface the message to the model.

Source: src/index.ts (transport selection and error handling), Source: docker-mcp-registry/server.yaml (declared secret requirement).

Related Pages

Related topics: MCP Tools Reference, Installation, Setup & Client Configuration

Error Handling, Async Workflows & Troubleshooting

The olostep-mcp-server exposes a suite of tools over the Model Context Protocol (MCP), and several of those tools represent inherently asynchronous operations — specifically batch URL scraping and crawl jobs. Because the Olostep API processes these workloads in the background and returns results out-of-band, the server wraps every tool call in explicit error handling and a two-step workflow pattern (kick-off + poll). This page documents how those flows are implemented, what failure modes the server surfaces to MCP clients, and how to diagnose the most common runtime problems reported by users.

Error Handling Architecture

Every tool entry point in the server follows a consistent shape: validate the input schema registered with the MCP runtime, perform the HTTP request to the Olostep API using the OLOSTEP_API_KEY provided via environment variable, and return either structured JSON content or a descriptive Error string inside the MCP response envelope. The wrapper around fetch checks response.ok and throws when the upstream returns a non-2xx status, which the tool handlers catch and re-throw as a textual error so MCP clients (Claude Desktop, Cursor, etc.) can render it inline.

Source: src/tools/batchScrapeUrls.ts:1-80

Source: src/tools/createCrawl.ts:1-80

The getBatchResults.ts and getCrawlResults.ts handlers go further: because the upstream operation may still be running, they inspect the response payload and propagate status fields such as pending, processing, completed, and failed to the caller unchanged, allowing the client to decide whether to poll again. This avoids server-side polling loops that would waste MCP request budget and prevents the model from blocking on long jobs.

Source: src/tools/getBatchResults.ts:1-60

Source: src/tools/getCrawlResults.ts:1-60

Async Workflow Pattern

Two operations in the server are explicitly asynchronous:

Both submit endpoints return a job_id (or equivalent identifier) that the caller must pass to the corresponding get_*_results tool to retrieve the payload. The recommended pattern is therefore:

1. Call the submit tool once with the URLs or seed.
2. Capture the returned identifier.
3. Poll the results tool at a sensible interval until status becomes completed or failed.
4. Read the final payload (pages, markdown, structured data).

sequenceDiagram
    participant Client as MCP Client
    participant Server as olostep-mcp-server
    participant API as Olostep API

    Client->>Server: batch_scrape_urls(urls=[...])
    Server->>API: POST /v1/scrapes/batch
    API-->>Server: { job_id, status: "queued" }
    Server-->>Client: { job_id, status: "queued" }

    loop until completed
        Client->>Server: get_batch_results(job_id)
        Server->>API: GET /v1/scrapes/batch/:id
        API-->>Server: { status: "processing" | "completed" | "failed" }
        Server-->>Client: same payload
    end

This handshake is documented in the README and matches the standard async-job contract used elsewhere in the Olostep product. The server intentionally does not implement server-side polling because MCP tool calls are synchronous from the model's perspective.

Source: README.md:1-120

Common Failure Modes

Based on the structure of the codebase and community reports, the following failure modes are the most frequently encountered:

1. Missing API key — When OLOSTEP_API_KEY is not exported into the environment before the MCP server starts, the first request will fail with an authentication error from the Olostep API. The README documents both the macOS/Linux (env OLOSTEP_API_KEY=...) and Windows (set OLOSTEP_API_KEY=...) invocations and shows how to embed the key in the Claude Desktop or Cursor MCP config.

Source: README.md:40-110

1. npx -y olostep-mcp command not found — Reported in issue #4, users on macOS and Linux occasionally saw sh: olostep-mcp: command not found when invoking the published package through npx. The root cause was a misconfigured bin entry in package.json; the published tarball did not expose the olostep-mcp binary on the user's PATH. The CHANGELOG notes the fix and recommends re-running with npx -y olostep-mcp@latest to pick up the corrected CLI packaging.

Source: CHANGELOG.md:1-40

Source: package.json:1-60

1. Stale node_modules after an upgrade — MCP hosts that spawn the server directly (rather than through npx) may continue running an older bundle that lacks newly added tools or fixes. Restarting the host process after pulling the latest version resolves this.

1. Job never completes — If get_batch_results or get_crawl_results returns pending or processing indefinitely, this usually indicates an upstream rate limit or a malformed URL set. Inspecting the payload's errors[] array (when present) surfaces the per-URL reason.

Source: src/tools/getBatchResults.ts:30-70

1. Network/firewall blocks — Because every tool call hits https://api.olostep.com, hosts running behind corporate proxies or with restrictive egress rules will see connection-reset or DNS errors. Switching to a network where the Olostep API is reachable is the only supported workaround.

Source: src/index.ts:1-80

Troubleshooting Checklist

When a tool call fails, walk through this checklist before opening an issue:

• Confirm OLOSTEP_API_KEY is exported in the shell that launches the MCP host, not just in an interactive terminal.
• Verify the server is reachable by running npx -y olostep-mcp@latest --help (or the equivalent) and confirming the CLI responds.
• For async jobs, ensure you are polling with the exact job_id returned by the submit tool and that you are using the matching get_*_results tool.
• Re-run with the latest published package to pick up packaging and bug fixes referenced in the CHANGELOG.
• Capture the exact MCP error string and, if possible, the status field from the response — both are needed to triage upstream failures.

Source: README.md:1-200

Source: CHANGELOG.md:1-60

By keeping the error surface textual and the polling explicit, the server remains predictable for MCP clients while still supporting the long-running scrape and crawl workloads that the Olostep API is designed for.

Doramagic Pitfall Log

Found 9 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: identity.distribution | https://github.com/olostep/olostep-mcp-server

2. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/olostep/olostep-mcp-server/issues/4

3. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://github.com/olostep/olostep-mcp-server

4. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/olostep/olostep-mcp-server

5. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/olostep/olostep-mcp-server

6. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/olostep/olostep-mcp-server

7. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/olostep/olostep-mcp-server

8. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/olostep/olostep-mcp-server

9. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/olostep/olostep-mcp-server

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using olostep-mcp-server with real data or production workflows.

• Not found error when running: env OLOSTEP_API_KEY=your-api-key npx -y ol - github / github_issue
• Installation risk requires verification - GitHub / issue