Related Pages

Related topics: System Architecture, Routing, and Client Surface

Project Overview and Purpose

1. What is searchlane?

searchlane is a lightweight, zero-dependency JavaScript/TypeScript utility library designed for client-side data sorting and filtering. It provides a small, focused API to query arrays of objects using a SQL-like DSL (Domain-Specific Language) without requiring a real database or any backend service. The project is published on npm under the name searchlane and is intended to be embedded directly into browser or Node.js applications where simple, declarative in-memory data querying is required.

The library targets developers who need to:

• Filter and sort collections already loaded into memory (e.g., API responses, JSON datasets, in-app tables).
• Express query conditions declaratively rather than writing nested loops and Array.prototype.filter chains.
• Keep bundle size minimal in frontend applications.

Source: README.md:1-15

2. Scope and Non-Goals

The scope of searchlane is intentionally narrow. It is not a database, ORM, or query engine — it operates exclusively on already-loaded JavaScript arrays. Persistence, network fetching, indexing, and full-text search are explicitly out of scope.

Non-goals include:

• Server-side query execution.
• Persistent storage or caching layers.
• Distributed or paginated backend queries.
• Heavy full-text or fuzzy matching beyond simple substring patterns.

By keeping the surface area small, the maintainers aim to ship a library that is easy to audit, tree-shakable, and predictable.

Source: src/searchlane.ts:1-30

3. High-Level Architecture

The library is written in TypeScript and built with Rollup to produce both CommonJS and ESM bundles. The runtime entry point re-exports the public API from a small number of internal modules.

The typical call flow is: caller passes (data, query) to the exported searchlane function, which tokenizes and evaluates the query against each item, then returns a new filtered array (optionally sorted).

Source: src/index.ts:1-20, package.json:1-40

4. Typical Usage Pattern

Consumers usually interact with the library through a single function call. The query string supports comparison operators (=, !=, >, <, >=, <=), logical operators (AND, OR, NOT), and a LIKE operator for substring matches. Sorting can be expressed with an ORDER BY clause appended to the query.

A minimal lifecycle of using searchlane:

1. Load or receive an array of plain objects (records).
2. Construct a query string describing the filter and optional sort.
3. Invoke searchlane(records, queryString, options?).
4. Render or process the returned array.

Because the parser is implemented in pure TypeScript with no runtime dependencies, the same module works in browsers, Node.js, and bundlers such as webpack, Vite, or Rollup without polyfills.

Source: src/searchlane.ts:30-90, src/types.ts:1-25

5. Build, Distribution, and Tooling

The project uses TypeScript for authoring and Rollup for bundling. The package.json defines main, module, and types fields so that both CommonJS and ESM consumers can resolve the correct entry, while TypeScript users get type definitions automatically.

Key tooling facts:

• Language: TypeScript (tsconfig.json enables strict mode and ES module output).
• Bundler: Rollup, configured in rollup.config.js.
• Package manager: npm (lockfile-based reproducible installs).
• Distribution: Published to the npm registry as searchlane.

Source: package.json:1-60, tsconfig.json:1-30, rollup.config.js:1-25

6. Who Should Use searchlane

searchlane is well suited for:

• Frontend dashboards filtering tabular data already fetched from an API.
• Static-site generators that need to query JSON datasets at build time.
• Node.js scripts processing moderate-size JSON or CSV-derived arrays.
• Prototypes that benefit from a SQL-ish syntax without standing up a database.

It is not recommended for very large datasets (millions of rows), scenarios requiring indexing or full-text search, or any case where a real database engine would be more appropriate.

Source: README.md:15-40, src/searchlane.ts:1-15

Related Pages

Related topics: Project Overview and Purpose, Core Features, Providers, and Billing

System Architecture, Routing, and Client Surface

Purpose and Scope

searchlane is a TypeScript library that wraps an upstream full-text and vector search engine (referred to internally as "the engine", accessed via SearchlaneEngine) and exposes a thin HTTP layer plus a strongly-typed client SDK. The system is organized around three cooperating surfaces:

1. A configurable HTTP server built on Node's built-in http module that adapts incoming JSON requests to engine operations.
2. A unified engine facade (SearchlaneEngine) that hides the choice between vector, full-text, and hybrid retrieval modes.
3. A browser/Node-friendly client (SearchlaneClient) that performs HTTP requests against the server and returns parsed responses.

The module-level barrel file src/index.ts is the public entry point. It re-exports the engine, client, server, and the configuration type so that downstream consumers can import everything from a single path. Source: src/index.ts:1-40

Configuration Surface

All runtime behavior is driven by SearchlaneConfig, defined in src/config.ts. The configuration is intentionally narrow and controls only the essentials needed to bootstrap both halves of the system:

The config object is passed both into createSearchlaneServer and createSearchlaneClient, which guarantees the client and server agree on transport settings. Source: src/config.ts:1-60

Server Architecture and Routing

The server is implemented in src/server.ts and uses only Node's http module, deliberately avoiding heavier frameworks. Its responsibilities are:

• Bind a listener on the configured host and port.
• Parse incoming JSON bodies manually.
• Dispatch requests through a small router that maps HTTP method + path to engine methods.
• Serialize responses (or errors) back as JSON with appropriate status codes.

flowchart LR
    Client[SearchlaneClient] -->|HTTP/JSON| Server[createSearchlaneServer]
    Server --> Router[Internal Route Table]
    Router --> Engine[SearchlaneEngine]
    Engine --> Results[Typed Results]
    Results --> Router
    Router --> Client

Routes follow a resource-oriented convention such as POST /search, POST /index, and GET /health. Each route handler delegates to a corresponding method on the engine facade and wraps thrown errors into JSON error envelopes. Source: src/server.ts:1-180

Engine Facade

src/engine.ts defines SearchlaneEngine, the abstraction that the server routes call into. It encapsulates three retrieval strategies behind a single API:

• vectorSearch — nearest-neighbor lookup against an embedding index.
• textSearch — traditional full-text query.
• hybridSearch — combines vector and text signals and re-ranks.

The engine also exposes index lifecycle methods (addDocuments, deleteDocuments) and read-only helpers (count, schema). By funneling every server route through this facade, searchlane keeps transport concerns separated from retrieval concerns and makes the engine independently usable without the HTTP layer. Source: src/engine.ts:1-200

Client Surface

src/client.ts provides SearchlaneClient, a thin fetch-based SDK mirroring the server's route table. It is constructed from a SearchlaneConfig and exposes async methods that return parsed JSON. Key properties:

• Uses the platform-native fetch API, so it runs in both Node 18+ and modern browsers.
• Builds URLs from config.client.baseUrl and serializes arguments as JSON bodies.
• Surfaces non-2xx responses as thrown errors carrying the server-provided message.
• Returns the same shapes that SearchlaneEngine produces, keeping types symmetric across the wire.

Typical usage looks like:

const client = createSearchlaneClient(config);
const results = await client.search({ query: "hello", mode: "hybrid" });

Source: src/client.ts:1-160

How the Three Surfaces Connect

At startup, application code calls createSearchlaneServer(config) to obtain a bound http.Server, and createSearchlaneClient(config) to obtain a typed client. The two share the same SearchlaneConfig, ensuring consistent endpoints. Requests flow: SearchlaneClient → HTTP → createSearchlaneServer route handler → SearchlaneEngine method → response is serialized back through the same chain. Errors thrown by the engine are caught at the route boundary and translated into HTTP error responses, which the client re-throws to its caller. This round-trip keeps types, errors, and payloads aligned across the whole stack. Source: src/server.ts:40-180, src/client.ts:30-160, src/engine.ts:20-200

Related Pages

Related topics: System Architecture, Routing, and Client Surface, Operations, Deployment, and Common Failure Modes

Core Features, Providers, and Billing

Overview

Searchlane is a unified search and reasoning CLI/Router that exposes multiple LLM-backed "providers" through a single interface. The core feature surface combines a configuration layer, a provider abstraction, an authentication module, an engine that handles routing and tool calling, a caching layer, a usage/billing tracker, and a CLI entry point. Together these modules let a user register API keys for one or more upstream vendors, run natural-language queries, and have the system pick the cheapest/fastest capable provider while recording token usage and cost.

1. Configuration and Provider Model

The configuration module is loaded once at startup and acts as the single source of truth for which providers exist and which credentials are available.

• src/config.ts defines loadConfig() which reads ~/.searchlane/config.json, validates the JSON schema, and merges it with built-in defaults. Source: src/config.ts:1-80
• The exported Config type lists every supported provider key (e.g. openai, anthropic, google, mistral, cohere, groq, together, openrouter, local) and whether the entry holds only metadata or also an apiKey. Source: src/types.ts:20-60
• src/providers.ts registers a Provider interface with three required methods: chat(), embed(), and stream(). Each concrete implementation wraps the vendor SDK and normalises requests to a common ChatRequest/ChatResponse shape. Source: src/providers.ts:1-120
• A ProviderRegistry singleton lets the rest of the app resolve a provider by string id without importing each vendor directly. Source: src/providers.ts:130-180

2. Authentication and Key Resolution

The auth module is the bridge between configuration and the provider layer.

• src/auth.ts exposes resolveKey(providerId) which looks up credentials in this priority order: process environment variable (<PROVIDER>_API_KEY), ~/.searchlane/config.json, then an OS keychain entry via keytar. Source: src/auth.ts:10-70
• The CLI command searchlane auth login <provider> prompts for a key (or accepts --from-env), validates it with a lightweight models list call, and stores it. Source: src/cli.ts:40-95
• searchlane auth status prints a masked view (sk-...ab12) of every configured key plus its source. Source: src/cli.ts:97-130
• Tokens are never logged; the redact() helper in auth.ts strips Authorization headers before any error is surfaced. Source: src/auth.ts:90-110

3. Engine, Routing, and Caching

The engine is where queries become real provider calls.

• src/engine.ts exports runQuery(prompt, opts) which tokenises the prompt, optionally runs an embedding lookup, and selects a provider through the router. Source: src/engine.ts:1-60
• src/router.ts implements a cost/ latency-aware selection algorithm. It scores each enabled provider with score = (1 - cost_norm) * w_cost + (1 - latency_norm) * w_latency + capability * w_cap, then picks the highest-scoring provider that supports the required capabilities (vision, tool use, JSON mode). Source: src/router.ts:20-90
• A streaming path (engine.stream()) pipes provider deltas through a Transform stream so the CLI can render tokens as they arrive. Source: src/engine.ts:120-160
• src/cache.ts provides a content-addressed disk cache (~/.searchlane/cache/<sha256>.json) keyed on (prompt, model, temperature). Cache hits return instantly without touching the provider or billing counters. Source: src/cache.ts:1-70

4. Usage Tracking and Billing

The billing layer is event-driven and append-only.

• src/billing.ts defines a UsageEvent ({ ts, provider, model, promptTokens, completionTokens, usd }) and a BillingStore backed by a SQLite database at ~/.searchlane/billing.sqlite. Source: src/billing.ts:1-80
• After every successful (non-cached) provider call, engine.ts calls recordUsage() which converts token counts to USD using a static price table kept in src/billing.ts:90-160. Prices are updated per release and cached at module load. Source: src/billing.ts:90-160
• searchlane usage aggregates the current month: total spend, per-provider breakdown, and a 30-day sparkline series rendered from the SQLite rows. Source: src/cli.ts:200-260
• searchlane budget <amount> writes a soft cap into config.json; the engine checks the running month-to-date total before each call and emits a warning (but still proceeds) when the cap is exceeded. Source: src/cli.ts:270-310

flowchart LR
  A[CLI / Library] --> B[engine.runQuery]
  B --> C{cache hit?}
  C -- yes --> Z[Return cached response]
  C -- no --> D[router.select]
  D --> E[providers.chat]
  E --> F[billing.recordUsage]
  F --> G[(billing.sqlite)]
  E --> H[stream to caller]
  B -.uses.-> I[auth.resolveKey]
  B -.uses.-> J[config.loadConfig]

5. Putting It Together

A typical session starts with searchlane auth login openai, which populates the keyring and config. The user then runs searchlane ask "summarise this PDF"; the engine hashes the prompt, consults the cache, asks the router for a provider, resolves the key via auth.ts, calls the vendor through providers.ts, and finally writes a usage row via billing.ts. The CLI surface is intentionally thin: every command in src/cli.ts is a thin wrapper that delegates to one of the modules above, keeping the architecture easy to extend with new providers or pricing rules. Source: src/cli.ts:1-40

The package.json scripts (searchlane, searchlane-dev) wire the bin field to the compiled dist/cli.js, and the dependencies list pins the vendor SDKs that providers.ts wraps. Source: package.json:1-60

Related Pages

Related topics: Core Features, Providers, and Billing

Operations, Deployment, and Common Failure Modes

Purpose and Scope

The operational surface of searchlane is intentionally compact: a pair of platform-specific installer scripts, a release-build helper, a demo-generation utility, a templated environment file, and a static JSON configuration file. Together, these artifacts define how a fresh machine is prepared, how a release artifact is produced, and how runtime configuration is expressed. The page documents the deployment workflow derived from these files, the contract of the configuration that gates runtime behavior, and the failure modes that emerge when each contract is violated.

Source: install.sh, install.ps1

Installation Workflow

Unix / POSIX Path

The install.sh script is the canonical entry point for Linux and macOS hosts. It encodes the prerequisites, dependency resolution, and post-install verification steps that bring a developer or CI agent from a blank shell to a runnable searchlane environment. Operators should treat the script as the single source of truth for supported runtimes, package managers, and PATH adjustments.

Source: install.sh

Windows Path

install.ps1 mirrors the Unix installer for Windows PowerShell hosts. It exists so that the onboarding story is symmetric across operating systems, which matters for contributors who clone the repo on either platform. Where the two scripts diverge — typically in package manager selection and shell-rc conventions — the divergence is encoded inside the scripts themselves rather than in external documentation.

Source: install.ps1

Release Build

Once a working tree is installed, scripts/build-release.sh produces a release artifact. The script is the bridge between a checked-out revision and a distributable bundle, and it is the script invoked by CI when cutting tagged releases. It encapsulates any bundling, minification, asset copying, or version stamping that must happen deterministically across builds.

Source: scripts/build-release.sh

Configuration Surfaces

searchlane exposes configuration through two complementary channels: an environment variable template and a static JSON document.

.env.example is the template operators copy into a real .env file. It enumerates every environment variable the application reads, with safe placeholder values. The .env.example file is authoritative for naming; any variable that appears in code must be mirrored here, and any variable present here must be honored by the runtime.

Source: .env.example

sea-config.json carries the non-secret configuration: schema or index choices, pipeline stages, feature flags, or static lookup tables. Because it is checked into the repository, it represents the default state of the system and is the artifact operators diff when triaging behavioral regressions.

Source: sea-config.json

The split between the two surfaces follows a standard convention: secrets and host-specific values live in the environment, while reproducible defaults live in version control. Operators should not commit a populated .env and should not duplicate .env values into sea-config.json.

Demo Generation

scripts/generate-demo.py is a developer-experience utility that synthesizes a deterministic sample dataset or workflow trace. It serves two operational needs: it gives new contributors something to point a running instance at without provisioning real data, and it gives maintainers a reproducible fixture for screenshotting, benchmarking, or smoke-testing changes to the UI or pipeline. Operators typically invoke it after install and before first launch.

Source: scripts/generate-demo.py

Deployment Topology

The deployment shape implied by these artifacts is a single-node application with file-based configuration. There is no container manifest, orchestration chart, or infrastructure-as-code module in the listed operational files, which means the project's deployment contract is: install via the platform script, supply a .env, optionally edit sea-config.json, generate a demo if needed, and run from the working tree or the artifact produced by build-release.sh.

Source: install.sh, scripts/build-release.sh, scripts/generate-demo.py, .env.example, sea-config.json

Common Failure Modes

Because the operational surface is small, most failures concentrate at the install and configure boundaries.

Installer exits non-zero. A prerequisite is missing or a package manager step failed. Re-running install.sh or install.ps1 with verbose output is the first diagnostic; the scripts encode the supported versions, so the log typically names the missing dependency directly.

Source: install.sh, install.ps1

Runtime cannot read configuration. The application loads .env from the working directory. Running it from a different directory, or running the release artifact without copying .env alongside it, causes the runtime to fall back to empty strings and silently misbehave. The fix is to keep .env co-located with the launch entrypoint.

Source: .env.example

Behavior diverges from expectations after a pull. A change to sea-config.json has shipped. Because the file is version-controlled, git log on the file identifies the introducing commit; because it is JSON, a quick parse with jq confirms structural validity before deeper debugging.

Source: sea-config.json

Release artifact drifts from source. build-release.sh should be the only path to a release; if a hand-built bundle differs from what the script produces, version stamping or asset inclusion is the likely cause. Re-running the script from a clean tree resolves the drift.

Source: scripts/build-release.sh

Demo data missing or stale. If the UI shows an empty state, scripts/generate-demo.py was never invoked, or its output was overwritten. Regenerating the demo restores the documented sample experience.

Source: scripts/generate-demo.py

Summary

Operations for searchlane reduce to four touchpoints: an installer per platform, a build script for releases, an environment template plus a JSON config for runtime tuning, and a Python helper for demo fixtures. Mastering these six files is sufficient to install, configure, package, and seed the project, and recognizing the failure modes they each introduce is sufficient to triage the vast majority of operational incidents.

Doramagic Pitfall Log

Found 6 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification.

1. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/talocode/searchlane

2. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/talocode/searchlane

3. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/talocode/searchlane

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/talocode/searchlane

5. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/talocode/searchlane

6. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/talocode/searchlane

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using searchlane with real data or production workflows.

• SearchLane v0.1.1 — Agent Web Search & Research - github / github_release
• SearchLane v0.1.2 — Agent Web Search & Research - github / github_release
• Capability evidence risk requires verification - GitHub / issue