Doramagic Project Pack · Human Manual
lantern
PostgreSQL vector database extension for building AI applications
Lantern Overview and Component Architecture
Related topics: lanternhnsw C Extension Internals, lanterncli and lanternextras Tooling, Build, Installation, Release, and Known Failure Modes
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Related Pages
Related topics: lantern_hnsw C Extension Internals, lantern_cli and lantern_extras Tooling, Build, Installation, Release, and Known Failure Modes
Lantern Overview and Component Architecture
Lantern is an open-source PostgreSQL extension that brings production-grade vector similarity search into the database engine. It combines an in-process C/C++ HNSW implementation with an external Rust daemon that can offload heavy index construction, autotuning, and embedding generation out of the database backend. The project is designed so that users get the ergonomics of standard SQL plus the performance and flexibility of an HNSW (Hierarchical Navigable Small World) ANN index.
Purpose and Scope
Lantern's primary goal is to enable efficient approximate nearest neighbor (ANN) search on vector columns stored inside PostgreSQL. The project exposes:
- A native HNSW index access method usable with standard
CREATE INDEX ... USING hnswsyntax. - Operator classes supporting common distance functions (
l2sq_dist,cos_dist,hamming_dist, etc.). - An external indexing path that streams index construction over a TCP socket so memory-heavy builds do not exhaust the PostgreSQL backend.
- Optional jobs (autotune, embedding generation, external index builds) that are dispatched to a companion daemon.
The repository is intentionally modular: SQL glue and C entry points live under lantern_hnsw/, while the long-running background workers and external index server live in the Rust workspace under lantern_cli/. Source: README.md:1-40.
Core Components
PostgreSQL Extension (`lantern_hnsw/`)
This directory is the heart of the database integration. It is built with CMake and registered as a PostgreSQL extension.
lantern_hnsw/CMakeLists.txtdefines the build target, source set, and PostgreSQL include/link flags.lantern_hnsw/sql/lantern.sqlregisters the data types, operators, operator classes, and thelantern_hnswaccess method in the catalog.lantern_hnsw/src/hnsw.hdeclares the HNSW entry points (hnsw_build,hnsw_insert,hnsw_search,hnsw_handle_delete) that the index access method calls.lantern_hnsw/src/hnsw/delete.cimplements HNSW-aware deletion handling and integrates with PostgreSQL's vacuum infrastructure. Source: lantern_hnsw/src/hnsw/delete.c:42.
The extension uses the usearch library as its underlying SIMD-optimized distance/search engine, wrapped through lantern_hnsw/src/hnsw/usearch_storage.hpp. Source: lantern_hnsw/src/hnsw/usearch_storage.hpp:1-80.
Rust Workspace and External Index Daemon (`lantern_cli/`)
The Rust workspace provides:
- An external index server that receives index build requests over TCP (with optional SSL) and streams back the completed index file. Source: lantern_cli/src/main.rs:1-120.
- A job runner for autotune and embedding tasks that are feature-gated so deployments can disable them.
- A companion CLI that the SQL extension invokes when the
external_indexingGUC is enabled.
Feature gating, introduced in v0.5.0, lets operators turn off embedding generation, autotune, and external indexing independently. Source: README.md:1-60.
SQL Surface and Operator Classes
The SQL declarations define what users actually type:
- Vector type and
l2sq_dist,cos_dist,hamming_distoperators for<->distance queries. - The
lantern_hnswaccess method with configurable parameters (dim,ef_construction,ef_search,M). - Helper functions to inspect index metadata and to coordinate with the external daemon. Source: lantern_hnsw/sql/lantern.sql:1-120.
High-Level Architecture
The diagram below shows how a query or build request flows through the system when external indexing is enabled.
flowchart LR
A[SQL Client] --> B[PostgreSQL Backend]
B --> C[lantern_hnsw Access Method]
C -- in-process build --> D[HNSW + usearch]
C -- external build --> E[lantern_cli TCP Server]
E --> D
D --> F[Index File on Disk]
C --> G[Catalog & WAL]
B --> H[Background Worker / Job]
H -- autotune/embed --> ETwo important architectural properties follow from this layout:
- Single WAL pass for builds: Recent versions optimized index construction so that node writes go through PostgreSQL's WAL in a single pass rather than buffering the whole graph in memory.
- Streaming index selection: Result sets can be streamed as the HNSW graph is walked, reducing latency for large
LIMITqueries.
Known Limitations and Build Considerations
From community-reported issues, several practical boundaries are worth recording:
- The current extension caps vector dimensions at 2000, which blocks use of larger embedding models such as OpenAI's
text-embedding-3-large(3072 dims). Source: lantern_hnsw/sql/lantern.sql:1-60. - Builds against PostgreSQL 18 beta currently fail because
vacuum_delay_point()requires a newis_analyzeargument that older Lantern sources do not pass. Source: lantern_hnsw/src/hnsw/delete.c:42. - macOS users have reported intermittent failures (
os error 22) when the external index path is enabled on Homebrew-built PostgreSQL 16; the recommended workaround is to rebuild withoutexternal_indexingor against a matching PostgreSQL build. - Documentation historically lived at
lantern.dev/docs/develop/index; the site has had outages (404s), so users should fall back to the in-repolantern_hnsw/sql/lantern.sqland theREADME.mdas the canonical reference until the hosted docs recover.
Versioning and Compatibility
Releases are tracked as semantic versions of the PostgreSQL extension. As of the latest tag (v0.5.0), Lantern ships:
- Official PostgreSQL 17 support (added in #348).
- Feature-gated embedding, autotune, and external indexing jobs (added in #349).
- An updated dependency tree to keep pace with
usearchandpgvectorsubmodules used in tests.
Operators upgrading across major PostgreSQL versions should rebuild both lantern_hnsw/ and lantern_cli/ and confirm that the GUCs controlling external indexing still match the daemon's listening address and SSL configuration.
Source: https://github.com/lanterndata/lantern / Human Manual
lantern_hnsw C Extension Internals
Related topics: Lantern Overview and Component Architecture, lanterncli and lanternextras Tooling, Build, Installation, Release, and Known Failure Modes
Continue reading this section for the full explanation and source context.
Related Pages
Related topics: Lantern Overview and Component Architecture, lantern_cli and lantern_extras Tooling, Build, Installation, Release, and Known Failure Modes
lantern_hnsw C Extension Internals
The lantern_hnsw directory hosts the compiled PostgreSQL extension that implements the lantern vector index access method. It is the low-level layer behind the user-facing lantern extension: SQL operators, index AM hooks, and type-casts all flow into this C code. The build is a CMake project configured by lantern_hnsw/src/CMakeLists.txt, and the resulting shared library is loaded by Postgres as a standard PG_MODULE_MAGIC extension. Source: lantern_hnsw/src/hnsw.c:1-1 and lantern_hnsw/src/CMakeLists.txt:1-1
Role in the Repository
lantern_hnsw is one half of the project's split architecture. The lantern SQL extension wires up operators (<->, distance functions, type I/O) and supports background jobs; lantern_hnsw provides the actual index access method, the HNSW graph data structure, and persistence to disk. Everything in lantern_hnsw/src/hnsw/ is paired with lantern_hnsw/src/hnsw.h, which holds the shared in-memory layout for the index, the per-tuple HnswTuple structure, and the PostgreSQL index-handle wrapper (HnswHandle). Source: lantern_hnsw/src/hnsw.h:1-1
The module's responsibilities are tight:
- Parse and validate the
USING lanternoperator-class options (dim,dist_fn,ef_construction,ef,M, etc.). - Manage a single in-memory HNSW graph per index, built lazily during the first
INSERTand updated incrementally. - Serve kNN queries using the standard Postgres index AM callbacks (
ambuild,aminsert,ambeginscan,amgettuple,amrescan,amendscan,amvacuumcleanup). - Marshal vectors between Postgres
Datumrepresentations and internalfloat/lvectorstorage. - Stream the finalized graph to a file so the index survives a server restart. Source: lantern_hnsw/src/hnsw.c:1-1
Build, Insert, and Maintenance Pipeline
Index construction is split between an external build path (recommended for large datasets) and the incremental build path driven by INSERTs.
- Access-method registration happens in
hnsw.c, where the index AM is wired into Postgres'IndexAmRoutine. Source: lantern_hnsw/src/hnsw.c:1-1 - Index creation parses operator-class options, allocates the on-disk metadata page (storing
dim, element type, and a magic/version field), and registers the index for incremental population. Source: lantern_hnsw/src/hnsw.h:1-1 - Incremental inserts are handled in
insert.c. Each new tuple is converted to an internal vector, a graph search finds its neighbors, and edges are bidirectionally linked. Writes hit Postgres' buffer manager and WAL so recovery is correct after a crash. Source: lantern_hnsw/src/hnsw/insert.c:1-1 - Bulk builds use
build.c. Whenlantern_create_indexis called inexternalmode the work is offloaded to a separatelantern-cliprocess over a TCP socket, then the resulting file is read back in chunks and persisted throughStoreExternalIndexNodes. Source: lantern_hnsw/src/hnsw/build.c:1-1 - Maintenance and deletes are handled in
delete.c. The routine calls into Postgres'vacuum_delay_point()to remain cooperative during long GC passes — this is the same point that breaks the build against PostgreSQL 18 beta 1, where the helper gained a mandatoryis_analyzeargument. Source: lantern_hnsw/src/hnsw/delete.c:42-42
flowchart LR
SQL[SQL: CREATE INDEX / INSERT / SELECT] --> AM[Index AM in hnsw.c]
AM --> INSERT[insert.c<br/>incremental]
AM --> BUILD[build.c<br/>bulk via lantern-cli]
AM --> SCAN[scan.c<br/>kNN search]
AM --> DEL[delete.c<br/>vacuum cleanup]
BUILD -- TCP socket --> CLI[lantern-cli external]
CLI -- file chunks --> BUILDA reusable 2000-dimensional limit is enforced at vector-parse time, which is why embeddings larger than 2000 elements (for example OpenAI text-embedding-3-large at 3072) are rejected up-front rather than failing inside the HNSW graph. Source: lantern_hnsw/src/hnsw.h:1-1
Query Path and Streaming Selects
For SELECT ... ORDER BY vec <-> $1 LIMIT k, Postgres calls ambeginscan, optionally amrescan for parameterized rescans, and then amgettuple per row. The streaming variant introduced in 0.3.1 (PR #322) pushes a custom executor node so results can flow without materializing the full candidate list. Source: lantern_hnsw/src/hnsw/scan.c:1-1
Internally the search:
- Loads the index from the shared buffer if it is not already resident, attaching it to the backend's
HnswHandle. - Walks from the graph's entry point down through the upper layers using the configured
ef_constructionbeam width, then refines the candidate set at layer 0 withef. - Streams neighbors into a binary heap of size
k, applying the<->distance function specified by the operator class. Source: lantern_hnsw/src/hnsw/scan.c:1-1
When the extension is built in external mode, the index server reports connection lines such as [Lantern External Index] New connection: 127.0.0.1:63501 and may surface lower-level errno failures (for example os error 22 on macOS when the index file descriptor is closed before the socket stream is fully drained). Source: lantern_hnsw/src/hnsw/build.c:1-1
Version Compatibility and Operational Notes
The extension is tested against the PostgreSQL versions enumerated in the CMake configuration and against the release matrix published in the changelog. The 0.5.0 release notes confirm explicit pg17 support and a feature gate that hides embedding, autotune, and external-indexing jobs from the daemon when not needed. Source: lantern_hnsw/src/CMakeLists.txt:1-1
Two outstanding community-reported failure modes are rooted in lantern_hnsw internals rather than the SQL wrapper:
- A PostgreSQL 18 beta 1 build failure caused by a changed
vacuum_delay_point()signature insidedelete.c(issue #375). - A macOS
EINVAL(os error 22) when an external index's file descriptor is closed before the streamed buffer is fully consumed (issue #370).
Knowing which C file owns each operation makes it much easier to triage these reports, because reproducer logs almost always point to either the access-method dispatcher in hnsw.c, the incremental path in insert.c, the bulk/external path in build.c, or the maintenance path in delete.c.
Source: https://github.com/lanterndata/lantern / Human Manual
lantern_cli and lantern_extras Tooling
Related topics: Lantern Overview and Component Architecture, lanternhnsw C Extension Internals, Build, Installation, Release, and Known Failure Modes
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Related Pages
Related topics: Lantern Overview and Component Architecture, lantern_hnsw C Extension Internals, Build, Installation, Release, and Known Failure Modes
lantern_cli and lantern_extras Tooling
Overview and Scope
The lantern_cli is a standalone Rust binary that complements the core lantern PostgreSQL extension. While the C-based extension provides the HNSW index implementation and SQL types, lantern_cli operates as an out-of-process companion daemon and client that manages long-running background tasks. The companion library lantern_extras (a PostgreSQL extension written in C) bridges SQL-level operations to the daemon via background workers and helper functions.
lantern_cli exposes a binary entry point through main.rs, which dispatches between direct CLI commands and a long-lived daemon mode. Source: lantern_cli/src/main.rs:1-50.
The release history shows the tooling has matured quickly:
- v0.3.2 introduced external indexing via TCP socket, plus SSL support (v0.3.2 release notes).
- v0.3.4 added an external index router and refactored
StoreExternalIndexNodes(v0.3.4 release notes). - v0.4.0 merged
lantern_extrasinto the main repository and shipped macOS binaries forlantern_cli(v0.4.0 release notes). - v0.4.1 bumped
lantern_clito 0.4.0 (v0.4.1 release notes). - v0.5.0 added PostgreSQL 17 support and feature-gated embedding, autotune, and external-indexing daemon jobs (v0.5.0 release notes).
Architecture and Components
The tooling is split across three logical layers.
1. lantern_cli CLI binary (Rust)
main.rs parses arguments and routes to either an immediate command or the daemon subsystem. Source: lantern_cli/src/main.rs:1-50. The CLI argument definitions live in cli.rs, while shared structures such as connection settings, index parameters, and job descriptors are centralized in types.rs to keep both modes consistent. Source: lantern_cli/src/cli.rs:1-40, lantern_cli/src/types.rs:1-40.
The Cargo.toml declares the package metadata and feature flags that gate daemon subsystems. Source: lantern_cli/Cargo.toml:1-40.
2. lantern_cli daemon subsystem
The daemon lives under lantern_cli/src/daemon/. It is composed of:
mod.rs— entry point that initializes logging, connects to PostgreSQL, and starts the worker loops. Source: lantern_cli/src/daemon/mod.rs:1-60.cli.rs— daemon-specific subcommands (start, stop, status). Source: lantern_cli/src/daemon/cli.rs:1-50.db.rs— PostgreSQL connection pool and schema introspection helpers. Source: lantern_cli/src/daemon/db.rs:1-50.http_client.rs— outbound HTTP client used to call embedding providers. Source: lantern_cli/src/daemon/http_client.rs:1-40.autotune_jobs.rs— worker that picks parameter-tuning jobs from the queue and runs HNSW parameter experiments against the target table. Source: lantern_cli/src/daemon/autotune_jobs.rs:1-50.embeddings_jobs.rs— worker that streams rows to an embedding service and writes vectors back. Source: lantern_cli/src/daemon/embeddings_jobs.rs:1-50.external_indexing_jobs.rs— worker that builds an HNSW index out-of-process and streams the binary index file to the PostgreSQL backend over a TCP socket. Source: lantern_cli/src/daemon/external_indexing_jobs.rs:1-50.
3. lantern_extras PostgreSQL extension
lantern_extras is a SQL/C extension that registers background worker types, helper SQL functions, and the lantern_extras schema used to queue jobs. It is built via its own CMakeLists.txt and exposes daemon_tools.c which contains C glue code that PostgreSQL background workers call to hand work off to the daemon. Source: lantern_extras/CMakeLists.txt:1-30, lantern_extras/src/daemon_tools/daemon_tools.c:1-50.
Data Flow and Job Lifecycle
The diagram below traces a typical external-indexing request from SQL to the Rust daemon and back.
sequenceDiagram
participant SQL as PostgreSQL (lantern_extras)
participant BG as Background Worker
participant Daemon as lantern_cli daemon
participant Net as TCP/SSL Socket
SQL->>BG: CREATE INDEX ... USING lantern_extras_external
BG->>Daemon: enqueue job row
Daemon->>Daemon: build HNSW in Rust
Daemon->>Net: stream index bytes (+ header)
Net->>BG: receive chunked file
BG->>SQL: StoreExternalIndexNodes()
SQL-->>SQL: index visible to plannerKey control points referenced by the sources:
- The job queue schema and
StoreExternalIndexNodesplumbing live inlantern_extras/src/daemon_tools/daemon_tools.c. Source: lantern_extras/src/daemon_tools/daemon_tools.c:1-80. - The external indexing worker negotiates the connection and reads the header (vector element bits, file size) before streaming the index. Source: lantern_cli/src/daemon/external_indexing_jobs.rs:1-60.
- Feature flags in
Cargo.tomlallow builds to disable the embedding, autotune, or external-indexing workers when not needed. Source: lantern_cli/Cargo.toml:1-40.
Feature Gating, Platforms, and Known Limitations
As of v0.5.0, the embedding, autotune, and external-indexing workers are gated behind Cargo features so that minimal builds only include the daemon core. Source: lantern_cli/Cargo.toml:1-40, v0.5.0 release notes.
Platform support:
- macOS binaries are published for
lantern_clistarting in v0.4.0. Source: v0.4.0 release notes. - PostgreSQL 17 is officially supported in v0.5.0. Source: v0.5.0 release notes.
- A PostgreSQL 18 beta 1 build failure was reported in issue #375, caused by
vacuum_delay_point()requiring the newis_analyzeargument. Source: issue #375. - A macOS build issue was tracked in #352, where local compilation from source produced errors after
make. Source: issue #352. - An
os error 22was raised on macOS 15.2 with PostgreSQL 16 from Homebrew when an external index was built againstlantern_hnswmaster (issue #370). Source: issue #370. - The project currently caps vector dimensions at 2000 (issue #374), which limits external-indexing payloads the daemon can stream. Source: issue #374.
- Earlier community-reported documentation 404s on lantern.dev (issue #371) have since been marked closed. Source: issue #371.
Operational Notes
When deploying lantern_cli:
- Build the Rust binary with the desired feature set:
cargo build --release --features "embeddings,autotune,external-indexing". Source: lantern_cli/Cargo.toml:1-40. - Install
lantern_extrasinto the target PostgreSQL instance so the background workers and helper functions are registered. Source: lantern_extras/CMakeLists.txt:1-30. - Start the daemon with
lantern_cli daemon start; it reads connection settings from CLI flags or environment variables defined intypes.rs. Source: lantern_cli/src/daemon/cli.rs:1-50, lantern_cli/src/types.rs:1-40. - SQL-level operations triggered inside PostgreSQL enqueue rows that the daemon polls via the helpers in
daemon_tools.c. Source: lantern_extras/src/daemon_tools/daemon_tools.c:1-80.
This split — extension for in-database primitives, daemon for heavy/long-running work, and lantern_extras for the glue — is the recurring pattern across recent releases and is what allows Lantern to build HNSW indexes externally without blocking the database process.
Source: https://github.com/lanterndata/lantern / Human Manual
Build, Installation, Release, and Known Failure Modes
Related topics: Lantern Overview and Component Architecture, lanternhnsw C Extension Internals, lanterncli and lanternextras Tooling
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Continue reading this section for the full explanation and source context.
Related Pages
Related topics: Lantern Overview and Component Architecture, lantern_hnsw C Extension Internals, lantern_cli and lantern_extras Tooling
Build, Installation, Release, and Known Failure Modes
Overview
Lantern is a PostgreSQL extension that implements the HNSW (Hierarchical Navigable Small World) algorithm for vector similarity search. Building, packaging, and releasing the project requires coordinating three components: the PostgreSQL extension C code under lantern_hnsw/, the companion lantern_extras and lantern_cli packages, and the optional standalone external index daemon. The project supports local source builds, Docker-based builds, and packaged releases distributed via GitHub. Source: README.md:1-40
This page documents the build pipeline, installation entry points, release cadence, and the failure modes that the community has reported (such as PostgreSQL 18 beta incompatibilities, macOS build errors, and runtime dimension limits).
Build Process
Local Source Build
The repository is a CMake project rooted at lantern_hnsw/CMakeLists.txt. The canonical local build sequence is:
git clone --recursive https://github.com/lanterndata/lantern.git
cd lantern
cmake -DMARCH_NATIVE=ON -S lantern_hnsw -B build
make -C build install -j
MARCH_NATIVE=ON enables CPU-specific vector instruction optimizations. The --recursive flag is required because several dependencies (notably pgvector) live in git submodules. Source: lantern_hnsw/CMakeLists.txt:1-40
Docker Build
Two Dockerfiles are provided:
docker/Dockerfilebuilds the PostgreSQL extension and ships a runnable image that wires the extension into a Postgres data directory.docker/Dockerfile.clibuilds thelantern_clistandalone binary, mirroring the macOS and Linux CLI distribution targets added in v0.4.0.
The .dockerignore file excludes transient build artifacts (e.g., the build/ directory) and local git metadata so the build context stays small. Source: .dockerignore:1-30, docker/Dockerfile:1-60, docker/Dockerfile.cli:1-40
Contributing and CI
The CONTRIBUTING.md file documents the developer workflow, coding style for C and SQL, the test harness, and the matrix of PostgreSQL versions exercised in CI. As of the v0.5.0 release, pg17 support was added officially, broadening the supported matrix. Source: CONTRIBUTING.md:1-80
Installation
Once the extension is compiled, installation follows the standard PostgreSQL extension pattern:
- Place the shared object into the PostgreSQL
libdirectory (handled automatically bymake install). - Add
lanterntoshared_preload_librariesif using background workers or the external index daemon. - In the target database, run
CREATE EXTENSION lantern;to load the SQL bindings.
The lantern_cli binary is distributed as a separate artifact (Linux and macOS binaries are produced from docker/Dockerfile.cli). It is used to build HNSW indexes out-of-process and stream them back to PostgreSQL over a TCP socket, a feature introduced in v0.3.2 and stabilized across v0.3.3 and v0.3.4. Source: CHANGELOG.md:1-80
Release History
Lantern follows an iterative release model. The most recent releases are summarized in the table below.
| Version | Headline Changes |
|---|---|
| v0.5.0 | pg17 support, dependency updates, feature-gated daemon jobs, lantern_extras coverage via grcov |
| v0.4.1 | lantern_cli bumped to 0.4.0, public SELECT grant on job tables |
| v0.4.0 | Fix pgvector installation in CI via submodules, move lantern_extras into the lantern repo, ship macOS lantern_cli binaries |
| v0.3.4 | Refactored StoreExternalIndexNodes to receive index file in chunks, external index router, element-bits in header |
| v0.3.3 | Close index FD only when not built via socket |
| v0.3.2 | External indexing via TCP socket, SSL support, extension version check refactor |
| v0.3.1 | Streaming HNSW index selects |
| v0.3.0 | Eliminate blockmap, single WAL pass for HNSW build |
| v0.2.7 | HNSW delete handling |
| v0.2.6 | Async task job-id fix |
Source: CHANGELOG.md:1-200
The pattern is consistent: each minor bump corresponds to a single architectural change (delete handling → blockmap removal → single WAL pass → socket-based external indexing → version check refactor → pg17 support), with patch releases (v0.4.1, v0.3.3) hardening the prior minor.
Known Failure Modes
The community has reported several concrete failure patterns that anyone building or running Lantern should be aware of.
PostgreSQL 18 beta 1 build failure
On PostgreSQL 18 beta 1, the source fails to compile because the upstream signature of vacuum_delay_point() changed to require an is_analyze argument. The error originates in lantern_hnsw/src/hnsw/delete.c:42 and breaks the build on the new server header. This is tracked in issue #375 and is a forward-compatibility gap, not a regression in Lantern's own logic. Source: issue #375
macOS local build failure
Users building from master on macOS 15.2 (with Homebrew PostgreSQL 16) have reported build errors during the make install step. The failure is associated with the same MARCH_NATIVE=ON and external index code paths, suggesting that the combination of Apple Clang and the new socket-based external indexer (introduced in v0.3.2) is not yet fully stable on macOS. Track via issue #352. Source: issue #352
Vector dimension ceiling (2000)
Lantern currently rejects vector dimensions greater than 2000 with the message vector dimension 3072 is too large. LanternDB currently supports up to 2000dim vectors. This blocks users of newer OpenAI embedding models such as text-embedding-3-large (3072 dims) without a manual recompile against a raised LANTERN_MAX_DIM. Tracked in issue #374. Source: issue #374
External index "os error 22"
When building an external index on macOS 15.2, the daemon logs [Lantern External Index] New connection: 127.0.0.1:63501 and then fails with an os error 22 (EINVAL). This typically indicates a malformed or truncated header frame being sent to the socket; v0.3.4 added explicit element-bit headers to mitigate this class of error. Tracked in issue #370. Source: issue #370
Documentation 404s (transient)
The lantern.dev documentation site has intermittently returned 404 for the /docs/develop/* paths. The issue was closed once the site was restored, but it is a reminder that release notes and the GitHub README are the canonical source of truth when the docs site is unavailable. Tracked in issue #371. Source: issue #371
Workflow Summary
The end-to-end path from source to running extension is:
- Clone with
--recursiveto fetch pgvector and other submodules. - Configure with CMake (optionally
MARCH_NATIVE=ON). - Build and install with
make -C build install -j. CREATE EXTENSION lantern;in the target database.- Optionally install
lantern_cliand start the external index daemon for out-of-process index construction.
When hitting a build error, first check the PostgreSQL major version against the supported matrix (pg13–pg17 in v0.5.0; pg18 is not yet supported), then confirm the submodule state, and finally reproduce inside the Docker images as a clean baseline. Source: README.md:20-80, docker/Dockerfile:1-60
Source: https://github.com/lanterndata/lantern / Human Manual
Doramagic Pitfall Log
Source-linked risks stay visible on the manual page so the preview does not read like a recommendation.
May increase setup, validation, or first-run risk for the user.
May increase setup, validation, or first-run risk for the user.
May increase setup, validation, or first-run risk for the user.
May increase setup, validation, or first-run risk for the user.
Doramagic Pitfall Log
Found 13 structured pitfall item(s), including 1 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.
1. Installation risk: Installation risk requires verification
- Severity: high
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: community_evidence:github | https://github.com/lanterndata/lantern/issues/370
2. Installation risk: Installation risk requires verification
- Severity: medium
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: identity.distribution | https://github.com/lanterndata/lantern
3. Installation risk: Installation risk requires verification
- Severity: medium
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: community_evidence:github | https://github.com/lanterndata/lantern/issues/375
4. Installation risk: Installation risk requires verification
- Severity: medium
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: community_evidence:github | https://github.com/lanterndata/lantern/issues/352
5. Capability evidence risk: Capability evidence risk requires verification
- Severity: medium
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: capability.assumptions | https://github.com/lanterndata/lantern
6. Runtime risk: Runtime risk requires verification
- Severity: medium
- Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: community_evidence:github | https://github.com/lanterndata/lantern/issues/371
7. Runtime risk: Runtime risk requires verification
- Severity: medium
- Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: community_evidence:github | https://github.com/lanterndata/lantern/issues/374
8. Runtime risk: Runtime risk requires verification
- Severity: medium
- Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: packet_text.keyword_scan | https://github.com/lanterndata/lantern
9. Maintenance risk: Maintenance risk requires verification
- Severity: medium
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: evidence.maintainer_signals | https://github.com/lanterndata/lantern
10. Security or permission risk: Security or permission risk requires verification
- Severity: medium
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: downstream_validation.risk_items | https://github.com/lanterndata/lantern
11. Security or permission risk: Security or permission risk requires verification
- Severity: medium
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: risks.scoring_risks | https://github.com/lanterndata/lantern
12. Maintenance risk: Maintenance risk requires verification
- Severity: low
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Recommended check: Reproduce the official install and quickstart path in an isolated environment.
- Evidence: evidence.maintainer_signals | https://github.com/lanterndata/lantern
Source: Doramagic discovery, validation, and Project Pack records
Community Discussion Evidence
These external discussion links are review inputs, not standalone proof that the project is production-ready.
Count of project-level external discussion links exposed on this manual page.
Open the linked issues or discussions before treating the pack as ready for your environment.
Community Discussion Evidence
Doramagic exposes project-level community discussion separately from official documentation. Review these links before using lantern with real data or production workflows.
- Build failure with PostgreSQL 18 beta 1 - github / github_issue
- Vector dimension 3072 is too large. LanternDB currently supports up to 2 - github / github_issue
- os error 22 when indexing - github / github_issue
- Documentation on lantern.dev is giving 404 - github / github_issue
- Build is failing in local in MacOS - github / github_issue
- Lantern v0.5.0 - github / github_release
- Lantern v0.4.1 - github / github_release
- Lantern v0.4.0 - github / github_release
- Lantern v0.3.4 - github / github_release
- Lantern v0.3.3 - github / github_release
- Lantern v0.3.2 - github / github_release
- Lantern v0.3.1 - github / github_release
Source: Project Pack community evidence and pitfall evidence