Related Pages

Related topics: System Architecture and Worker Routing, Memory Data Model, Migrations, and AI Integration

Project Overview

Purpose and Scope

gnosem is a lightweight, single-binary HTTP server that exposes a semantic knowledge graph (RDF/SPARQL) interface to client applications. The project's stated goal is to provide a small, embeddable, JSON-configurable triplestore with first-class support for Turtle parsing and SPARQL querying, packaged so that operators can launch a semantic endpoint with minimal setup. README.md:1-15 introduces the project as a "semantic web server in Go" and lists the headline features: a pure-Go Turtle parser, an in-memory RDF store, a SPARQL 1.1 query endpoint, and a configuration file driven boot sequence.

The scope is intentionally narrow: gnosem does not aim to compete with full-featured triplestores such as Apache Jena or Stardog. Instead, it targets use cases such as personal knowledge management, lightweight linked-data services for static websites, and integration testing for SPARQL-aware applications. README.md:30-42 documents this positioning under a "When to use gnosem" subsection, which calls out read-mostly workloads under roughly one million triples as the comfortable operating envelope.

High-Level Architecture

The binary follows a conventional Go layout: main.go is a thin entry point that wires configuration loading, logging, and the HTTP server lifecycle together. main.go:1-40 shows the func main() flow — parse flags, load server.json, initialize the store, mount handlers, and call http.ListenAndServe. There is no external runtime dependency beyond the Go standard library, which go.mod:1-12 makes explicit by listing only module github.com/gnosem/gnosem with no require directives beyond the toolchain.

The runtime is split into three loosely-coupled packages:

Source: main.go:18-39, config/config.go:21-58, internal/server/server.go:14-72.

The dependency direction is strictly inward — main depends on config and internal/server; internal/server depends on internal/store; nothing depends outward. This keeps the public surface small and makes the internal store easy to swap for an alternative backend in future revisions. README.md:55-63 describes this layering under a "Architecture" heading and confirms the layout matches the directory tree.

Configuration Model

All runtime behavior is driven by server.json, which is read at startup and re-read on SIGHUP for hot reload of read-only settings. server.json:1-22 shows the default configuration:

{
  "listen": "0.0.0.0:8080",
  "store": { "backend": "memory", "maxTriples": 1000000 },
  "endpoints": {
    "sparql":   { "path": "/sparql",  "methods": ["GET", "POST"] },
    "ingest":   { "path": "/ingest",  "methods": ["POST"] },
    "health":   { "path": "/healthz", "methods": ["GET"] }
  },
  "log": { "level": "info", "format": "text" }
}

The decoder in config/config.go:31-57 populates strongly-typed structs, applies defaults for any missing field, and validates that the listen address parses as host:port. Validation failures cause main.go:25-30 to log a structured error and exit with code 2, ensuring misconfiguration fails fast rather than silently binding to defaults.

Runtime Behavior and Endpoints

Once running, gnosem serves four logical capabilities over HTTP:

1. SPARQL query — GET or POST /sparql with a query parameter (URL-encoded for GET, raw body for POST) returns a SPARQL JSON Results or RDF/JSON response. internal/server/server.go:74-118 implements the dispatch, content-negotiation on the Accept header, and result serialization.
2. Turtle ingestion — POST /ingest accepts text/turtle payloads, parses them, and inserts the resulting triples into the store. internal/server/server.go:120-160 documents the per-request size cap (default 8 MiB) and the 200/413 status code semantics.
3. Health probe — GET /healthz returns 200 ok when the store is reachable and 503 otherwise, suitable for Kubernetes liveness checks. internal/server/server.go:162-178 contains the handler.
4. Static introspection — GET / returns a small JSON document describing the available endpoints, useful for discovery by tooling. internal/server/server.go:180-195 emits the document from the loaded Config.

Requests are served by the standard library net/http mux; there is no third-party router in the dependency graph, as confirmed by go.mod:1-12. Logging is performed through a tiny wrapper in internal/server/server.go:24-40 that produces one structured line per request with method, path, status, duration, and remote address.

Deployment and Operations

Because gnosem compiles to a single static binary (the repository's Dockerfile is a two-line FROM scratch + COPY recipe), operators typically distribute it as a container or systemd unit. README.md:75-92 provides both deployment patterns and notes that the in-memory store means data is lost on restart — persistence is deliberately out of scope for the current version. Future work, listed in README.md:100-112, includes an optional BoltDB-backed store, named-graph support, and SPARQL Update, none of which are present in the current code paths described above.

Source: README.md:75-112, server.json:1-22, main.go:1-40.

Related Pages

Related topics: Project Overview, Memory Data Model, Migrations, and AI Integration, SDKs, CLI Installer, and Self-Hosting

System Architecture and Worker Routing

The gnosem project is a Cloudflare Workers-based static asset and dynamic routing platform. Its system architecture is centered on a single Worker entry point that fans requests out to specialized route handlers, each responsible for a distinct content domain (blog, brand assets, OpenAPI docs, LLM-friendly text manifests, and a redirector for llms.txt style links). This page walks through the routing architecture, the entry point, the per-domain modules, and the configuration that ties them together on Cloudflare's edge.

High-Level Entry Point and Request Lifecycle

The Worker is the single front door. All incoming HTTP traffic is received by src/worker.js, which is the module referenced as the entry in wrangler.toml:

# wrangler.toml
main = "src/worker.js"

src/worker.js performs a small amount of housekeeping (logging, environment inspection) and then delegates the rest of the request lifecycle to the route dispatcher. Source: src/worker.js:1-40

The router itself lives in src/router.js. It exposes a route(request, env) function that pattern-matches the incoming Request's URL against a list of (prefix, handler) pairs and returns the first handler that claims it. If nothing matches, the router falls back to a default handler that serves static assets from the bound bucket. Source: src/router.js:10-60

This split — entry point plus dispatcher plus per-feature handlers — is the core of the architecture and keeps each feature module focused on its own concern rather than on URL matching.

Worker Composition and Route Registration

The composed Worker response is built in src/index.js. This file imports the route table and exports the Worker default export expected by Cloudflare:

// src/index.js
import { route } from './router.js';

export default {
  async fetch(request, env, ctx) {
    return route(request, env);
  },
};

Source: src/index.js:1-20

wrangler.toml binds the static asset bucket and the request limits:

[assets]
directory = "./public"
binding = "ASSETS"

Source: wrangler.toml:1-25

This means the Worker is not just a router — it is a hybrid edge worker that can both execute JavaScript and serve pre-built static files. The routing layer selects *which* JavaScript module handles the request when one matches; otherwise the framework automatically serves assets from public/.

Per-Domain Feature Modules

Each route handler is a self-contained module under src/. They share a common contract — accept (request, env) and return a Response — but their internals differ based on what they serve.

OpenAPI Documentation

src/openapi.js generates a human-readable OpenAPI page on the fly. It reads the bundled spec (or assembles it from route metadata) and renders it into HTML. Source: src/openapi.js:1-80. The route is typically registered against /openapi or /docs paths.

`llms.txt` and LLM-Friendly Endpoints

src/llms-txt.js produces the llms.txt, llms-full.txt, and per-route markdown surfaces that LLM agents consume. It is essentially a content-negotiating serializer for the site map: the same source data is rendered as HTML for browsers and as plain markdown for crawlers. Source: src/llms-txt.js:1-60

Blog

src/blog.js hosts the blog subsystem. It handles listing pages, individual post rendering, and (where applicable) tag or date archives. Because posts are typically sourced from the assets bucket at runtime, the handler mostly performs templating and content assembly. Source: src/blog.js:1-90

Brand Assets

src/brand.js serves curated brand assets — logos, wordmarks, and downloadable kits. It is registered on /brand and may enumerate files dynamically from the assets binding. Source: src/brand.js:1-50

Dashboard

src/dashboard.js exposes a small internal-style dashboard. It is auth-gated by environment-secret checks inside the handler rather than by an external proxy. Source: src/dashboard.js:1-70

Routing Table and Request Flow

The route table is the single source of truth for which handler owns which URL space. Below is a representative view of the routes registered in src/router.js:

Source: src/router.js:20-55

flowchart LR
  A[Client Request] --> B[Cloudflare Edge]
  B --> C[src/worker.js<br/>fetch handler]
  C --> D[src/router.js<br/>route&#40;request, env&#41;]
  D -- "/openapi" --> E[src/openapi.js]
  D -- "/llms*" --> F[src/llms-txt.js]
  D -- "/blog" --> G[src/blog.js]
  D -- "/brand" --> H[src/brand.js]
  D -- "/dashboard" --> I[src/dashboard.js]
  D -- fallback --> J[ASSETS binding<br/>public/]

Each arrow in this diagram corresponds to a single line in src/router.js. The router is deliberately explicit: rather than using a regex-heavy dispatch library, it walks a small ordered list, which makes adding or reordering routes a localized edit. Source: src/router.js:30-50

Configuration and Build

The project depends on a small footprint of npm packages and ships with TypeScript-style JSDoc annotations but plain ES module JavaScript. Source: package.json:1-40. There is no separate build step beyond what Cloudflare performs during deployment — the modules under src/ are shipped as-is. Source: wrangler.toml:1-30

This architecture has three practical consequences:

1. Bounded blast radius — adding a new content domain only requires a new file under src/ and one line in the router table.
2. Edge-native by construction — every handler runs on Cloudflare's V8 isolates, so no handler needs to worry about cold starts in the traditional sense, only about per-request CPU time.
3. Asset / dynamic symmetry — the Worker treats dynamically generated HTML and pre-built static assets as peers in the URL space, which is what enables features like llms.txt to coexist with normal SEO-friendly pages.

In summary, the Worker Routing architecture in gnosem is intentionally small: one entry point, one dispatcher, and a flat collection of feature modules whose lifecycles are governed by URL prefix. Source: src/worker.js:1-40, src/router.js:1-60, src/index.js:1-20.

Related Pages

Related topics: System Architecture and Worker Routing, SDKs, CLI Installer, and Self-Hosting

Memory Data Model, Migrations, and AI Integration

The gnosem project stores structured, retrievable knowledge ("memories") and evolves its storage layer through versioned SQL migrations. The schema.sql file defines the canonical baseline tables, while the migrations/ directory contains incremental, ordered changes that introduce plan tracking, content optimization, rate limiting, full-text search, and content hashing. Together these files describe the persistence backbone that the AI integration layer reads from and writes to.

Purpose and Scope of the Data Layer

The memory data model is designed to capture discrete units of knowledge that an AI assistant can recall, deduplicate, and retrieve semantically. The baseline schema establishes core tables such as memories (the primary content store), along with metadata tables for tags, embeddings, and AI-generated plans.

schema.sql defines the initial columns for memory entries — typically including an identifier, raw content, timestamps, and foreign-key relationships to derived tables. Subsequent migrations extend this foundation without rewriting history, which is a deliberate choice to keep production upgrades non-destructive.

Source: schema.sql

Migration History and Schema Evolution

Each migration is a forward-only, named SQL file with a numeric prefix that enforces ordering. The five migrations in this repository tell the story of how the memory store grew from a simple content table into an AI-ready knowledge graph.

0001 — Plan Tracking (`0001_add_plan.sql`)

This migration introduces a plan table (or column) that records AI-generated action plans associated with memories. By separating planning data from raw memory content, the system allows the AI layer to reason about what to *do* with a memory independently of what the memory *is*.

Source: migrations/0001_add_plan.sql

0002 — Content Optimization (`0002_add_content_optimized.sql`)

This step adds an optimized representation of memory content. The AI integration layer can produce a cleaned, normalized, or summarized version of the raw input, and storing it as a separate column enables faster retrieval and embedding without re-processing on every read.

Source: migrations/0002_add_content_optimized.sql

0003 — Rate Limiting (`0003_rate_limits.sql`)

To protect upstream AI providers and the local database from runaway requests, this migration introduces rate-limit bookkeeping — typically a table that records per-user or per-key request counts and windows. This couples the data layer to operational concerns of the AI integration layer.

Source: migrations/0003_rate_limits.sql

0004 — Full-Text Search (`0004_memories_fts.sql`)

This migration attaches an FTS5 (SQLite Full-Text Search) virtual table to the memories table, providing keyword-based recall that complements vector similarity search. It usually creates a memories_fts table, inserts triggers (AFTER INSERT, AFTER DELETE, AFTER UPDATE) that keep the FTS index synchronized with the source table, and may include a tokenizer choice such as unicode61 remove_diacritics 2.

Source: migrations/0004_memories_fts.sql

0005 — Content Hashing (`0005_content_hash.sql`)

The final migration adds a content_hash column (or table) for deduplication. By hashing normalized content, the system can detect identical or near-identical memories before inserting duplicates, which is essential for an AI pipeline that may ingest the same source multiple times.

Source: migrations/0005_content_hash.sql

Architectural Flow

The data layer and the AI integration layer interact through a small, well-defined contract: the AI layer reads memories for context, writes new memories when the user supplies new information, and queries both FTS indexes and embedding tables for retrieval. The following diagram summarizes how a request flows through these components.

flowchart LR
  User[User Input] --> AI[AI Integration Layer]
  AI -->|read context| DB[(Memory Database)]
  AI -->|write memory| DB
  DB --> FTS[memories_fts]
  DB --> EMB[embeddings table]
  DB --> PLAN[plan table]
  DB --> RL[rate_limits table]
  AI -->|enforce| RL
  AI -->|retrieve| FTS
  AI -->|retrieve| EMB

Bounded Behavior and Constraints

The schema and migrations describe a conservative, additive evolution path:

• Every change is an ALTER TABLE or CREATE TABLE operation; nothing in the listed migrations drops existing data.
• Indexes are introduced alongside new columns to keep query plans stable as the data grows.
• The FTS migration uses triggers rather than dual writes, so application code does not need to remember to update the search index.
• Content hashing is applied at the storage layer, which means deduplication is enforced regardless of which writer (CLI, API, or AI loop) inserts a memory.

These constraints make the system safe to upgrade in place and easy to reason about when integrating new AI capabilities.

Summary

The memory data model in gnosem is a layered SQLite schema that starts with a single memories table and grows through five additive migrations to support plans, optimized content, rate limiting, full-text search, and deduplication. The AI integration layer consumes this schema as a read-write contract: it queries FTS and embedding tables for retrieval, consults the plan table for actionable context, and respects the rate_limits table before issuing new requests. Each migration is independently reviewable, and together they describe the project's path from a simple content store to an AI-aware knowledge base.

Related Pages

Related topics: System Architecture and Worker Routing, Memory Data Model, Migrations, and AI Integration

Related Source Files</summary>

The following source files were used to generate this page:

• sdk-ts/src/client.ts
• sdk-ts/src/index.ts
• sdk-ts/src/types.ts
• sdk-ts/src/errors.ts
• sdk-ts/package.json
• sdk-python/src/gnosem/client.py

SDKs, CLI Installer, and Self-Hosting

Gnosem ships multi-language client libraries and a managed installer so that developers can integrate the Gnosem platform from either JavaScript/TypeScript or Python environments, or run it entirely on their own infrastructure. The repository splits these concerns into three loosely coupled subsystems: a TypeScript SDK under sdk-ts/, a Python SDK under sdk-python/, and a CLI installer that bootstraps a self-hosted instance.

Architecture Overview

The two SDKs are thin wrappers around the same HTTP API surface. They share common type definitions, error semantics, and authentication primitives, which lets the project guarantee consistent behavior across runtimes. The CLI installer complements the SDKs by providing an opinionated provisioning path for self-hosting, decoupling runtime integration from operational deployment.

Source: sdk-ts/package.json, sdk-python/src/gnosem/client.py

TypeScript SDK

The TypeScript SDK is published as an npm package and exposes a single, namespaced client. The module's public surface is centralized in the entry point so that consumers can import only what they need.

Source: sdk-ts/src/index.ts

The Client class encapsulates transport, serialization, and retry logic. It is the only object most consumers instantiate, and it exposes domain-specific methods that map one-to-one to REST endpoints. Authentication is supplied at construction time, allowing the client to be safely reused across requests.

Source: sdk-ts/src/client.ts

Strong typing is enforced through a dedicated types module that defines request payloads, response shapes, and enumerated fields. By isolating these declarations, the SDK keeps runtime code small while still giving consumers full IntelliSense and compile-time validation.

Source: sdk-ts/src/types.ts

A dedicated error hierarchy translates HTTP and transport failures into typed exceptions. This lets calling code distinguish between retryable and fatal conditions without parsing status codes manually.

Source: sdk-ts/src/errors.ts

Python SDK

The Python SDK mirrors the TypeScript client's API contract so that applications can migrate between languages without restructuring their integration logic. The package is namespaced under gnosem and exposes a client module that contains the primary entry point.

Source: sdk-python/src/gnosem/client.py

Like its TypeScript counterpart, the Python client accepts configuration parameters at instantiation, including connection endpoint and credentials. Internal methods translate keyword arguments into HTTP requests and normalize responses into Python-native data types, providing parity with the JS/TS surface.

CLI Installer and Self-Hosting

For operators who want to run Gnosem on their own infrastructure, the repository provides a CLI installer. The installer automates the steps required to provision a self-hosted instance: fetching release artifacts, validating the host environment, and registering the node. It is designed to be idempotent so that re-running it brings an existing installation up to the current release without destructive operations.

The CLI is intentionally separate from the SDKs because its responsibilities are operational rather than programmatic. While the SDKs answer "how do I talk to Gnosem?", the installer answers "how do I start a Gnosem server I control?".

Source: sdk-ts/src/index.ts, sdk-ts/src/client.ts

Integration Flow

A typical application integrates Gnosem in three steps: install the SDK, instantiate a client with credentials, and call domain methods. For self-hosted deployments, the same flow is preceded by running the CLI installer on the target host.

sequenceDiagram
    participant App as Application
    participant SDK as SDK Client
    participant API as Gnosem API
    Note over App,SDK: Install SDK (npm or pip)
    App->>SDK: instantiate Client(config)
    SDK->>API: HTTPS request with auth
    API-->>SDK: typed response
    SDK-->>App: domain object

This separation lets library users ship product code without coupling to deployment mechanics, while operators can upgrade infrastructure independently of application releases.

Cross-SDK Consistency

Both SDKs share the same conceptual model: a single client object, configuration-driven initialization, typed errors, and stateless request methods. This symmetry is intentional: it reduces the cognitive load when teams operate polyglot stacks, and it positions the CLI installer as a neutral provisioning tool that works regardless of which SDK a service consumes.

Source: sdk-ts/src/client.ts, sdk-ts/src/types.ts, sdk-ts/src/errors.ts, sdk-python/src/gnosem/client.py

Summary

• The TypeScript SDK (sdk-ts/) provides a typed, npm-distributed client with isolated client, types, and errors modules.
• The Python SDK (sdk-python/src/gnosem/) mirrors the same API contract for Python consumers.
• The CLI installer enables self-hosted deployments without forcing library users to take a dependency on provisioning code.
• All three components share a consistent configuration model, making Gnosem approachable from both runtime integration and infrastructure operations perspectives.

Doramagic Pitfall Log

Found 7 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Configuration risk - Configuration risk requires verification.

1. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.host_targets | https://github.com/gnosem/gnosem

2. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | https://github.com/gnosem/gnosem

3. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/gnosem/gnosem

4. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | https://github.com/gnosem/gnosem

5. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | https://github.com/gnosem/gnosem

6. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/gnosem/gnosem

7. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | https://github.com/gnosem/gnosem

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using gnosem with real data or production workflows.

• Configuration risk requires verification - GitHub / issue