Related Pages

Related topics: Storage Layer, Vector Indexes, and Search, Cluster, Replication, Sharding, and Raft, APIs, Schema, Modules, and Operations

Weaviate Overview and System Architecture

Purpose and Scope

Weaviate is an open-source, cloud-native vector database that stores both objects and vector embeddings side-by-side, enabling semantic search at scale. According to the project README.md, the database combines vector similarity search with keyword (BM25) filtering, retrieval-augmented generation (RAG), and reranking inside a single query interface. The project exposes three wire protocols to clients: REST, gRPC, and GraphQL (README.md).

The system is implemented primarily in Go, which the README.md cites as the reason for its "speed and reliability". It targets AI-driven workloads including RAG systems, semantic and image search, recommendation engines, chatbots, and content classification. Production deployments leverage first-class multi-tenancy, replication, RBAC, vector quantization, and time-to-live (TTL) support, all surfaced through the same APIs.

Two vectorization paths exist (README.md):

• Automatic vectorization at import time using integrated providers (OpenAI, Cohere, HuggingFace, Google, etc.).
• Pre-computed vectors supplied by the client.

High-Level System Architecture

The repository is organized around several cooperating subsystems:

1. Cluster / replication layer — message types in cluster/proto/api/message.pb.go enumerate the internal RPC vocabulary, including TYPE_GET_SHARDING_STATE, TYPE_GET_CLASS_VERSIONS, TYPE_GET_COLLECTIONS_COUNT, TYPE_HAS_PERMISSION, TYPE_GET_ROLES, TYPE_GET_USERS_FOR_ROLE, TYPE_USER_IDENTIFIER_EXISTS, TYPE_GET_NAMESPACES, TYPE_RESOLVE_ALIAS, TYPE_GET_ALIASES, and the four replication-detail message variants (TYPE_GET_REPLICATION_DETAILS, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION_AND_SHARD, TYPE_GET_REPLICATION_DETAILS_BY_TARGET_NODE). These 30 message types drive schema, RBAC, namespaces, alias, and replication coordination across nodes.
2. Vectorization modules — modules such as text2vec-contextionary interact with the rest of Weaviate through gRPC. The modules/text2vec-contextionary/client/contextionary.go file shows the translation between internal traverser.SearchParams / traverser.SearchType and the protobuf representation, indicating that modules are pluggable workers that can be enabled or disabled per deployment.
3. Runtime configuration system — a hot-reloadable configuration layer ensures operators can tune parameters without restart, exercised by the tests in usecases/config/runtime/manager_test.go.
4. Developer tooling — auxiliary utilities, including a local telemetry dashboard, a release-notes generator, and benchmarking/UI helpers.

The following Mermaid diagram summarizes the high-level request flow for a typical query.

flowchart LR
    Client["Client (Python / TS / Java / Go / .NET)"] -->|REST / gRPC / GraphQL| API[Adapters / Handlers]
    API --> UC[Use Cases]
    UC --> DB[(Vector + Object Store)]
    UC --> Mod[Vectorization Module]
    UC --> Cluster[Cluster State / Replication]
    Cluster -->|protobuf RPC| Nodes[Peer Nodes]
    Mod -->|gRPC| Ext[External Model Provider]

Key Subsystems in Detail

Cluster, Replication, and Schema Coordination

Replication, namespaces, aliases, RBAC, and collection metadata are coordinated through the typed messages generated under cluster/proto/api/message.pb.go. The descriptor at the bottom of the file registers 30 messages and 4 enums in a single FileDescriptor, which is the single source of truth for the cross-node protocol. Community reports (for example, issue #2985) show that a common operational pain point is multi-host replication where a node advertises a private IP that other nodes cannot route to. This is a direct consequence of the cluster subsystem needing an externally reachable address for each node.

Pluggable Vectorization and Retrieval

Vectorization is decoupled from the storage layer. The modules/text2vec-contextionary/client/contextionary.go file shows the gRPC boundary: requests are marshalled to pb.SchemaSearchParams with a Certainty, a Name, and a SearchType that is converted from internal traverser.SearchType to pb.SearchType_CLASS or pb.SearchType_PROPERTY. Results are converted back via schemaSearchResultsFromProto. This indirection lets Weaviate swap providers and indexing back-ends without changing the public API.

Runtime Configuration

Production deployments require configuration that can be updated without restarting database traffic. The tests in usecases/config/runtime/manager_test.go demonstrate the contract: the ConfigManager watches a YAML file, calls a registered parser, applies the result to a registered updater, and exposes Prometheus metrics such as weaviate_runtime_config_last_load_success. A failing write keeps the previously valid configuration active ("injecting new invalid config file should keep using old valid config"), so operators can roll out changes without risking downtime. The 100-goroutine fan-out assertion in the file also confirms that the manager is safe for high-concurrency reads.

Developer and Operator Tooling

• Local telemetry dashboard (tools/telemetry-dashboard/README.md) listens on :8080 and visualizes payloads from Weaviate instances, including machine statistics, client library usage (Python, Java, TypeScript, Go, C#), and module usage.
• Release-notes generator (tools/dev/generate_release_notes/README.md) uses GITHUB_TOKEN, CURRENT_VERSION, and PREVIOUS_VERSION to publish changelogs between tagged releases.
• Indexing-mistakes demo UI (tools/dev/bench/demo_indexing_mistakes_ui/README.md) imports one million synthetic product objects to demonstrate how poor index configuration (for example, missing vectorization, wrong distance metric) affects latency and result quality.

Notable Operational Concerns from the Community

Several community issues map directly onto architectural components:

• Deterministic ordering for equal-distance results (issue #11609) notes that the HNSW search returns by distance only; when distances are bit-identical, ordering becomes non-deterministic. This is a search-layer concern sitting above the storage engine.
• Batch partial update (issue #2124) highlights that updating subsets of properties across hundreds of thousands of objects one-by-one is slow, reflecting current batch limitations.
• Multiple named vectors (issue #2465) and nested-object filtering (issue #3694) indicate ongoing expansion of the schema and indexing model.
• "Not" operator in filters (issue #3683) is a long-standing request for richer filter expressions.

Recent releases (for example, v1.35.23) ship stability fixes — including a rate limiter in batch simple logic — that show the operational hardening path the project is following.

See Also

• README.md — full feature list and quickstart.
• cluster/proto/api/message.pb.go — cluster / replication protocol definitions.
• usecases/config/runtime/manager_test.go — runtime configuration contract.
• modules/text2vec-contextionary/client/contextionary.go — vectorization module boundary.
• tools/telemetry-dashboard/README.md — local observability helper.

Related Pages

Related topics: Weaviate Overview and System Architecture, Cluster, Replication, Sharding, and Raft, APIs, Schema, Modules, and Operations

Storage Layer, Vector Indexes, and Search

Weaviate is an open-source, cloud-native vector database that stores both objects and vectors, combining vector similarity search with keyword filtering, retrieval-augmented generation (RAG), and reranking in a single query interface. Source: README.md:1-12. The storage layer and the vector index subsystem are the two components that make this combination possible: the storage layer persists objects and their inverted indexes, while vector indexes accelerate approximate nearest neighbor (ANN) search over embeddings.

High-Level Architecture

At a high level, a Weaviate deployment processes queries through a layered stack: a query layer (REST, gRPC, GraphQL) parses the request, the vector index executes the ANN search, the storage layer resolves the resulting object IDs to full records, and a filtering and reranking pipeline merges the result with metadata filters.

flowchart TD
    A[Client Query: REST / gRPC / GraphQL] --> B[Query Planner]
    B --> C[Vector Index: HNSW / Flat / Dynamic / HFresh]
    B --> D[Storage Layer: LSMKV bucket per property]
    C --> E[Candidate Object IDs]
    D --> F[Object Properties + Filters]
    E --> G[Merge + Rerank + Filter]
    F --> G
    G --> H[Response]

The README frames the database around a few production-critical features that interact with the storage and search layers: multi-tenancy, replication, RBAC, vector compression, and object TTL. Source: README.md:1-40. These features are not orthogonal to the storage layer: for example, replication and sharding require the cluster protocol to coordinate per-shard state.

Vector Indexes

Weaviate ships several vector index types, exposed under adapters/repos/db/vector/ in the codebase. The README's "Fast Search Performance" feature explicitly references sub-millisecond ANN search and points to the ANN benchmarks. Source: README.md:34-38.

The community has surfaced concrete pain points around vector indexes. Issue #11609 reports non-deterministic ordering when two results share a float32-equivalent distance, which is directly relevant to HNSW search tie-breaking behavior. Source: README.md:34-40. Older issues such as #2465 request support for multiple named vector indexes per object, which interacts with the per-vector-index configuration surfaced in the schema.

Vectorization is decoupled from indexing. The text2vec-contextionary module exposes a client (Client) that performs remote calls for operations such as SchemaSearch, VectorForWord, and MultiVectorForWord. Source: modules/text2vec-contextionary/client/contextionary.go:1-15. This shows the typical contract between Weaviate's storage/index layer and a vectorizer module: the module returns a vector, the index then stores it and becomes queryable.

Storage Layer

Weaviate uses an LSM-based key/value store (LSMKV) for persisting objects and inverted indexes. Each shard typically owns multiple LSMKV buckets — one per inverted-index column and one for the object store — and the bucket abstraction supports concurrent reads, batched writes, and background compaction. A notable recent fix in multiple release lines (v1.35.22, v1.36.17, v1.37.7) is "make compaction abort immediately on cancel," indicating that compaction is a critical-path operation that can be cancelled by a higher-priority task.

The storage layer is also where multi-tenancy and replication are implemented at the lowest level. The cluster protocol exposes query types such as TYPE_GET_SHARDING_STATE, TYPE_GET_TENANTS_SHARDS, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION, and TYPE_GET_REPLICATION_OPERATION_STATE in its Protobuf API. Source: cluster/proto/api/message.pb.go:1-40(TYPE_GET_SHARDING_STATE, TYPE_GET_TENANTS_SHARDS, TYPE_GET_REPLICATION_DETAILS_*). These types are how a node asks another node for the authoritative state of a shard or replication operation, which is essential for cross-node consistency.

A common operational concern highlighted in the community is replication nodes advertising private IP addresses to remote nodes (issue #2985). Because replication operates at the storage layer, the wrong advertised address can prevent a replica from ever joining a shard — a failure mode worth keeping in mind when configuring a multi-node cluster.

Search Pipeline

Search in Weaviate is expressed as a combination of vector similarity, keyword (BM25) search, filtering, and optional reranking. Source: README.md:40-48. The pipeline:

1. Vector search — the configured vector index (HNSW by default) returns the top-k candidates by distance.
2. Filter resolution — the LSMKV-backed inverted indexes are queried to apply property and tenant filters, narrowing or excluding candidates.
3. Hybrid merge — vector and BM25 scores are combined when hybrid search is used.
4. Reranking — an optional reranker module rescores the top candidates.
5. Object hydration — surviving IDs are looked up in the object store to return full records.

The tools/dev/bench/demo_indexing_mistakes_ui/ tool demonstrates that small configuration choices (vectorizer, compression setting, HNSW parameters) have dramatic effects on search latency. Source: tools/dev/bench/demo_indexing_mistakes_ui/README.md:1-10.

Configuration and Operations

Runtime configuration is loaded and hot-reloaded through a ConfigManager that watches a YAML file and re-parses it on change. Source: usecases/config/runtime/manager_test.go:1-30. The same pattern applies to storage- and index-related settings such as backup_interval, memlimit thresholds, and HNSW parameters; changes propagate to all goroutines reading the registered config. The release history shows that the SSB memlimit threshold was raised from 80% to 90% (v1.36.16 / v1.37.6) — a configuration change that directly affects how aggressively LSMKV flushes and compacts.

For observability, the tools/telemetry-dashboard collects POSTed telemetry from Weaviate instances and visualizes payload-level, machine, and client-usage statistics. Source: tools/telemetry-dashboard/README.md:1-15. The tools/dev/generate_release_notes tool automates release-note generation between two Git tags. Source: tools/dev/generate_release_notes/README.md:1-5.

Common Failure Modes

From the community context:

• Non-deterministic tie-breaking (#11609) — equal-distance results have no guaranteed order, which can break tests and downstream logic.
• Cross-node networking (#2985) — replication requires that nodes advertise reachable addresses; private Docker network IPs will break replica joins.
• Filter and reranking gaps (#3683, #3694) — the Not operator and nested-object filtering/vectorization are recurring feature requests that would extend the search pipeline.
• Compaction cancellation (v1.35.22 / v1.36.17 / v1.37.7) — long-running compactions can be cancelled, which is now expected to abort immediately rather than finish the current segment.

See Also

• Schema and multi-tenancy model
• Replication and sharding protocol
• Vector quantization and compression
• HNSW configuration and tuning

Related Pages

Related topics: Weaviate Overview and System Architecture, Storage Layer, Vector Indexes, and Search, APIs, Schema, Modules, and Operations

Cluster, Replication, Sharding, and Raft

Overview

Weaviate's cluster subsystem is the control plane that keeps every node in agreement on schema, tenants, shard ownership, and ongoing replication work. All node-local data-plane stores (HNSW indexes, LSM buckets) are eventually reconciled to match the decisions recorded here. The subsystem is exposed through a Raft-driven log: every mutating request is encoded as an ApplyRequest, every read-only request is encoded as a QueryRequest, and both are defined in cluster/proto/api/message.pb.go.

The high-level separation of responsibilities is:

• Raft consensus (defined under cluster/raft.go and surfaced by cluster/store.go) decides *what* the cluster looks like and in *what order* changes happen.
• Apply (cluster/store_apply.go) executes those decisions against in-memory state machines for schema, tenants, aliases, RBAC, and replication.
• Query (cluster/store_query.go) serves the latest committed snapshot to clients and operators.
• Replication engine (cluster/replication/) translates replication operations into long-running, per-shard tasks that the data plane carries out.

Weaviate exposes a REST API, gRPC API, and GraphQL API to the database server, but all cluster-level coordination flows through this Raft-based store. Source: README.md.

Raft-Backed State Machine

The cluster uses a strongly-consistent log to serialize every change. The protobuf contract for that log lives in cluster/proto/api/message.pb.go, which defines two top-level messages: QueryRequest for reads and ApplyRequest for writes.

ApplyRequest_Type enumerates the mutating operations. The most relevant categories are:

Source: cluster/proto/api/message.pb.go. The ApplyResponse carries the log version (index/term) and the leader that accepted it, which nodes use to learn commit order. Source: cluster/proto/api/message.pb.go.

QueryRequest_Type mirrors this on the read path and exposes things like TYPE_GET_SHARDING_STATE, TYPE_GET_TENANTS_SHARDS, TYPE_GET_SHARD_OWNER, TYPE_GET_CLASSES, TYPE_GET_SCHEMA, and a family of replication inspectors (TYPE_GET_REPLICATION_DETAILS, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION, TYPE_GET_REPLICATION_OPERATION_STATE, TYPE_GET_REPLICATION_SCALE_PLAN, etc.). Source: cluster/proto/api/message.pb.go.

The Mermaid diagram below shows how a write request flows from a client through Raft to the in-memory state and the data plane:

flowchart LR
    Client[Client SDK] -->|gRPC Apply| Leader[(Raft Leader)]
    Leader --> Log[(Raft Log Entry)]
    Log -->|commit| SM[Schema/ Tenant/ RBAC State Machine]
    SM -->|notify| DB[(Per-node DB / Shard)]
    DB -->|HSM/HFresh| Engine[Replication / HFresh Engine]
    Engine -->|Ack| SM
    Client -->|Query| Follower[Read on Follower] --> SM

Sharding and Replication

Sharding partitions a collection's objects across nodes, while replication duplicates those partitions for availability. Both are coordinated by Raft:

• A new replica is added via TYPE_ADD_REPLICA_FROM_SHARD and removed via TYPE_DELETE_REPLICA_FROM_SHARD. The shard status transitions are tracked with TYPE_UPDATE_SHARD_STATUS. Source: cluster/proto/api/message.pb.go.
• Per-tenant shard lifecycles (TYPE_ADD_TENANT, TYPE_UPDATE_TENANT, TYPE_DELETE_TENANT) are also committed through Raft, which is why multi-tenant operations are linearizable across the cluster. Source: cluster/proto/api/message.pb.go.

Replication of *data* (copying the contents of one shard to another) is layered on top of this control plane. The manager registers a TYPE_REPLICATION_REPLICATE operation, and the per-shard engine (cluster/replication/shard_replication_engine.go) progresses it as a distributed task. Cancellation and forced cleanup are first-class: the family of TYPE_REPLICATION_REPLICATE_FORCE_DELETE_* requests plus TYPE_DISTRIBUTED_TASK_CANCEL and TYPE_DISTRIBUTED_TASK_CLEAN_UP allow operators to abort a stuck copy without leaving partial state. Source: cluster/proto/api/message.pb.go.

The engine reports progress and finalization back to Raft through TYPE_DISTRIBUTED_TASK_UPDATE_UNIT_PROGRESS, TYPE_DISTRIBUTED_TASK_RECORD_UNIT_COMPLETED, and TYPE_DISTRIBUTED_TASK_MARK_FINALIZED. After a node finishes its unit of work it sends TYPE_DISTRIBUTED_TASK_RECORD_NODE_COMPLETED, and once the leader observes the work is durable it closes the loop with RecordDistributedTaskPostCompletionAckRequest { Namespace, Id, Version, NodeId, Success, Error, AckedAtUnixMillis }. The comment in source notes this "closes the crash-safety gap where a node whose RunSwapOnShard silently failed could otherwise let the cluster-wide schema flip commit while one replica's bucket pointer never moved." Source: cluster/proto/api/message.pb.go.

Common Failure Modes and Operational Notes

Several recurring community-reported issues are rooted in this subsystem:

• Wrong advertised address between nodes. A replication node may publish its private Docker network IP to a peer, so cross-host replication hangs. This is purely a control-plane bootstrap issue, not a data-corruption one, and is fixed by setting the right advertise address on each node. Reference: weaviate#2985.
• Deflaked replication tests. Recent releases (v1.36.16, v1.37.6) fix TestReplicationAbort/Error and /DecodeResponse flakiness, indicating that abort and error-handling paths in the replication engine are non-trivial and continue to harden. Source: v1.36.16 release notes.
• Compaction race with cancel. Three backport releases (v1.35.22, v1.36.17, v1.37.7) carry the same LSM compaction abort fix, showing that the abort path between the data plane and the replication engine needs to be observed carefully to avoid torn deletes. Source: v1.37.7 release notes.

Runtime configuration that affects cluster behavior (timeouts, replication factor, sharding defaults) is loaded through usecases/config/runtime, where a ConfigManager watches the on-disk YAML and re-applies changes to registered components. The hot-reload contract is exercised in usecases/config/runtime/manager_test.go, which asserts that updates to backup_interval propagate to a registered config struct under concurrent reads and that invalid YAML does not silently clear the previously loaded configuration.

See Also

• Weaviate README
• Cluster Proto API (message.pb.go)
• Issue #2985 — Replication node advertises a private IP
• v1.37.6 — Core Stability, Backup, and Compression Fixes
• v1.37.7 — Speed up collection and shard delete

Related Pages

Related topics: Weaviate Overview and System Architecture, Storage Layer, Vector Indexes, and Search, Cluster, Replication, Sharding, and Raft

APIs, Schema, Modules, and Operations

Overview

Weaviate is an open-source, cloud-native vector database that exposes a layered surface to clients, operators, and integrators. The project ships three wire protocols (REST, gRPC, and GraphQL), a pluggable module system for vectorization and transformation, a runtime configuration manager for live operational tuning, and a cluster messaging layer for distributed operations such as replication and sharding. Source: README.md:1-30.

This page describes the four interrelated subsystems that sit between end users and the storage engine: the public APIs, the in-process Schema model, the Modules extension points, and the runtime Operations primitives (cluster proto, runtime config, telemetry, release tooling).

Public APIs

Weaviate communicates with client libraries over three independent interfaces, each serving a distinct audience. Source: README.md.

The gRPC and GraphQL services share the same Go-level request parsing path, so changes to filter syntax or vector index parameters are typically mirrored across both protocols. Client libraries are officially maintained for Python, JavaScript/TypeScript, Java, Go, and C#/.NET, and additional community-maintained libraries are listed in the docs. Source: README.md.

flowchart LR
    Client[Client Library] -->|REST| REST[REST Handlers]
    Client -->|gRPC| GRPC[gRPC Service]
    Client -->|GraphQL| GQL[GraphQL Resolver]
    REST --> UC[Use Cases]
    GRPC --> UC
    GQL --> UC
    UC --> DB[(Vector Index + Object Store)]

Schema, Modules, and Integrations

The Schema defines collections (formerly "classes"), their properties, vector index configuration, multi-tenancy settings, replication factors, and module bindings. A schema declaration can attach a vectorizer module that is invoked at import time, so that raw text or images are turned into vectors before persistence. Source: README.md:1-40.

Modules are Go plugins that plug into specific extension points such as text2vec-*, text2vec-contextionary, sum-transformers, reranker-*, and generative-*. Each module provides a small client wrapper that talks to either an external HTTP/gRPC service or an in-process provider. For example, the contextionary client exposes a SchemaSearch operation that converts a SearchParams value into a protobuf request and maps the response back into the domain SearchResults type. Source: modules/text2vec-contextionary/client/contextionary.go:1-60.

pbParams := &pb.SchemaSearchParams{
    Certainty:  params.Certainty,
    Name:       params.Name,
    SearchType: searchTypeToProto(params.SearchType),
}
res, err := c.grpcClient.SchemaSearch(ctx, pbParams)

Likewise, the sum-transformers module performs HTTP POST calls against an external summarization service and returns a list of SummaryResult entries keyed by property name. Source: modules/sum-transformers/client/client.go:1-30. The README cross-references dozens of integrations spanning hyperscalers, LLM/agent frameworks (LangChain, LlamaIndex, Haystack, DSPy, Semantic Kernel), and observability tooling (Arize, Cleanlab, Comet, Ragas, Weights & Biases). Source: README.md.

Cluster Messaging and Replication Operations

Distributed operations are coordinated through a strongly-typed protobuf message catalog defined in cluster/proto/api. The QueryRequest_Type enum identifies every supported cluster-level operation, including:

• Schema and sharding queries: TYPE_GET_CLASSES, TYPE_GET_SCHEMA, TYPE_GET_SHARDING_STATE, TYPE_GET_SHARDING_STATE_BY_COLLECTION, TYPE_GET_SHARDING_STATE_BY_COLLECTION_AND_SHARD
• Tenancy queries: TYPE_GET_TENANTS, TYPE_GET_TENANTS_SHARDS
• Authorization queries: TYPE_HAS_PERMISSION, TYPE_GET_ROLES, TYPE_GET_ROLES_FOR_USER, TYPE_GET_USERS_FOR_ROLE
• Replication details: TYPE_GET_REPLICATION_DETAILS, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION, TYPE_GET_REPLICATION_DETAILS_BY_COLLECTION_AND_SHARD, TYPE_GET_REPLICATION_DETAILS_BY_TARGET_NODE, TYPE_GET_ALL_REPLICATION_DETAILS, TYPE_GET_REPLICATION_OPERATION_STATE, TYPE_GET_REPLICATION_SCALE_PLAN
• Distributed task listing: TYPE_DISTRIBUTED_TASK_LIST

Source: cluster/proto/api/message.pb.go:1-120.

These message types are serialized with the standard google.golang.org/protobuf runtime and are dispatched by the cluster service. Community issue #2985 highlights a common operational failure mode: when two Dockerized nodes advertise their private network addresses, remote replication handshakes fail. The proper remedy is to publish an explicit, routable address in the node's join configuration rather than letting the OS pick one.

Runtime Configuration and Operations Tooling

Operational tunables such as backup cadence, compaction thresholds, and rate limits are exposed through a generic, type-safe runtime configuration manager. The ConfigManager periodically reloads a YAML/JSON file, parses it, and calls a registered Updater that diffs the new config against the current one. Source: usecases/config/runtime/manager.go:1-80.

Key invariants enforced by the manager include:

• An empty filepath argument is rejected with a descriptive error so that the manager is never constructed in a degraded state.
• Only registered keys are accepted; unknown keys surface as the Found sentinel error.
• The previously loaded config remains in effect if a reload fails, ensuring operational continuity.
• Prometheus gauges (lastLoadSuccess, configHash) are exported to give operators visibility into reload health.

The unit tests in manager_test.go validate that 100 concurrent readers observe the latest configuration after a reload, demonstrating that the manager is safe to consume from many goroutines. Source: usecases/config/runtime/manager_test.go:1-90.

Release and Telemetry Tooling

Two tools/-level helpers round out the operations surface. The release-notes generator reads merged pull requests between two tag references and produces GitHub-flavoured Markdown:

GITHUB_TOKEN="<token>" CURRENT_VERSION=v1.26.12 PREVIOUS_VERSION=v1.26.11 go run .

Source: tools/dev/generate_release_notes/README.md:1-10.

The local telemetry dashboard is a Go program that listens for anonymous usage payloads from running Weaviate instances, aggregates them by machine ID, and renders client/module/object statistics in a browser at http://localhost:8080. Operators can redirect the base64-encoded defaultConsumer URL in usecases/telemetry/telemetry.go to point at the local dashboard for development. Source: tools/telemetry-dashboard/README.md:1-50.

Common Failure Modes

Based on community-reported issues, three failure modes recur in this subsystem:

1. Private-address replication — nodes expose Docker-bridge IPs to peers; configure a routable join address (#2985).
2. Non-deterministic equal-distance ordering — when ANN distances tie at float32 precision, result order is unstable; tracked in #11609.
3. Long batch update latency — users report 20h updates for 400k records when patching one-by-one; a first-class batch patch is tracked in #2124.

See Also

• README.md — Project introduction, feature list, integrations matrix
• tools/dev/generate_release_notes/README.md — Release-notes generator
• tools/telemetry-dashboard/README.md — Local telemetry dashboard
• usecases/config/runtime/manager.go — Runtime configuration manager
• cluster/proto/api/message.pb.go — Cluster proto catalog

Doramagic Pitfall Log

Found 30 structured pitfall item(s), including 7 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

1. Installation risk: Installation risk requires verification

• Severity: high
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11690

2. Installation risk: Installation risk requires verification

• Severity: high
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11687

3. Installation risk: Installation risk requires verification

• Severity: high
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11693

4. Runtime risk: Runtime risk requires verification

• Severity: high
• Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11685

5. Maintenance risk: Maintenance risk requires verification

• Severity: high
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11686

6. Maintenance risk: Maintenance risk requires verification

• Severity: high
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11692

7. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11688

8. Installation risk: Installation risk requires verification

• Severity: medium
• Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/2985

9. Configuration risk: Configuration risk requires verification

• Severity: medium
• Finding: Developers should check this configuration risk before relying on the project: Replication node advertises a private IP address to a remote node
• User impact: Developers may misconfigure credentials, environment, or host setup: Replication node advertises a private IP address to a remote node
• Recommended check: Before packaging this project, run the relevant install/config/quickstart check for: Replication node advertises a private IP address to a remote node. Context: Observed when using node, docker
• Evidence: failure_mode_cluster:github_issue | https://github.com/weaviate/weaviate/issues/2985

10. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | github_repo:55072677 | https://github.com/weaviate/weaviate

11. Runtime risk: Runtime risk requires verification

• Severity: medium
• Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/weaviate/weaviate/issues/11691

12. Runtime risk: Runtime risk requires verification

• Severity: medium
• Finding: Project evidence flags a runtime risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: packet_text.keyword_scan | github_repo:55072677 | https://github.com/weaviate/weaviate

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using weaviate with real data or production workflows.

• PR vulnerability check red on every PR: remote build cache pins pre-CVE- - github / github_issue
• enable-rangeable migration drops ~13% of concurrent writes when the prop - github / github_issue
• Runtime reindex: multi-node clock skew can reopen the migration double-w - github / github_issue
• Date values outside ~1678–2262 misbehave across all epoch-nano paths (fa - github / github_issue
• Collection: pre-existing defects & flaky tests surfaced during the colum - github / github_issue
• deepcopy.Prop drops BucketGeneration — RAFT-replicated reindex counter l - github / github_issue
• Multi-node filtered aggregation crashes on mean/median/mode: interface c - github / github_issue
• Flaky test: TestReplicaMovementShardScaleOutParallelWrites — parallel wr - github / github_issue
• Flaky test: TestProcessingQueue/multiple_keys,_cancelled_context — cance - github / github_issue
• Handle all processes in actions.js - github / github_issue
• [[POC] grpc-web to enable fully-client side marimo notebook with weaviate](https://github.com/weaviate/weaviate/issues/11672) - github / github_issue
• Replication node advertises a private IP address to a remote node - github / github_issue