\n
\n\n# System Architecture\n\nLangfuse is a comprehensive observability and analytics platform designed for Large Language Model (LLM) applications. The system architecture is built on a modern, modular design that separates concerns across frontend, backend worker services, and shared infrastructure layers.\n\n## Overview\n\nLangfuse follows a distributed architecture pattern with the following primary components:\n\n| Layer | Technology Stack | Purpose |\n|-------|-----------------|---------|\n| Frontend | Next.js, React, TypeScript | User interface and visualization |\n| Backend Worker | Node.js, BullMQ, TypeScript | Event processing and queue management |\n| Shared Packages | TypeScript | Common utilities, types, and infrastructure clients |\n| Database | PostgreSQL | Primary data storage |\n| Cache/Queue | Redis | Queue management and caching |\n| Analytics | ClickHouse | High-performance analytics queries |\n| Observability | OpenTelemetry | Distributed tracing |\n\n## High-Level Architecture\n\n```mermaid\ngraph TD\n subgraph Client[\"Frontend (Next.js)\"]\n UI[User Interface]\n Pages[Page Components]\n DesignSystem[Design System]\n end\n \n subgraph Shared[\"Shared Packages\"]\n DB[(PostgreSQL)]\n Redis[(Redis)]\n ClickHouse[(ClickHouse)]\n Otel[OpenTelemetry]\n end\n \n subgraph Backend[\"Worker Service\"]\n Queues[Queue Workers]\n Scripts[Utility Scripts]\n end\n \n Client <-->|tRPC API| Shared\n Backend <-->|Event Processing| Shared\n Client -->|Ingestion| Backend\n```\n\n## Infrastructure Layer\n\n### Database Connection\n\nThe PostgreSQL database is the central data store for Langfuse, managed through a shared database client module. The connection is centralized in the `packages/shared/src/db.ts` module, which provides a unified interface for all database operations across the application.\n\n资料来源：[packages/shared/src/db.ts:1-50]()\n\n### Redis Client\n\nRedis serves dual purposes in the Langfuse architecture:\n\n1. **Queue Management**: BullMQ queues for asynchronous event processing\n2. **Caching**: Session and temporary data caching\n\nThe Redis client is configured in `packages/shared/src/server/redis/redis.ts` and is shared across worker services.\n\n资料来源：[packages/shared/src/server/redis/redis.ts:1-30]()\n\n### ClickHouse Integration\n\nClickHouse provides the analytical query engine for high-performance aggregations. The client is initialized in `packages/shared/src/server/clickhouse/client.ts` and is primarily used for:\n\n- Score analytics aggregations\n- Time-series data analysis\n- Large dataset transformations\n\n资料来源：[packages/shared/src/server/clickhouse/client.ts:1-40]()\n\n### OpenTelemetry\n\nThe OpenTelemetry integration (`packages/shared/src/server/otel/index.ts`) provides distributed tracing across all services. This enables:\n\n- Request tracing across frontend and backend\n- Event processing workflow visibility\n- Performance monitoring\n\n资料来源：[packages/shared/src/server/otel/index.ts:1-60]()\n\n## Frontend Architecture\n\n### Page Structure\n\nAll frontend pages use a standardized layout system defined in `web/src/components/layouts/`. The `Page` component is the required wrapper for all application pages, ensuring consistent layout behavior.\n\nKey layout patterns:\n\n| Pattern | Component | Use Case |\n|---------|-----------|----------|\n| Standard Pages | `Page` | Most application pages |\n| Wide Content | `ContainerPage` | Settings, setup pages with wide content |\n\nThe page wrapper provides:\n- Sticky header management\n- Scroll behavior control (`content-scroll` or `page-scroll`)\n- Breadcrumb navigation\n- Custom header actions\n\n资料来源：[web/src/components/layouts/README.md:1-60]()\n\n### Design System\n\nThe design system (`web/src/components/design-system/`) provides primitive, reusable UI components following strict architectural principles:\n\n**Principles:**\n- Presentational only (no business logic)\n- Explicit, strictly typed APIs\n- Props over context (no React Context)\n\n**Component Structure:**\n```\ndesign-system/\n Button/\n Button.tsx\n Button.stories.tsx\n```\n\n**Styling Rules:**\n- No arbitrary CSS values\n- Explicit enums for variants (`size: \"sm\" | \"md\" | \"lg\"`)\n- CVA (Class Variance Authority) for variant management\n- Boolean props use positive naming (`isLoading`, `shouldTruncate`)\n\n资料来源：[web/src/components/design-system/README.md:1-80]()\n\n### State Management Patterns\n\nLangfuse uses a sophisticated state management approach with the Peek Table State system:\n\n```mermaid\ngraph TD\n A[Page Load] --> B{Peek Context?}\n B -->|Yes| C[PeekTableStateProvider]\n B -->|No| D[URL/Session State]\n C --> E[Table State Preserved]\n D --> F[Standard State]\n \n E --> G[K/J Navigation]\n G --> H[State Retained ✓]\n```\n\nThe `PeekTableStateProvider` maintains table state (filters, sorting, pagination) across K/J keyboard navigation between items of the same type. State resets only when the peek view closes.\n\n资料来源：[web/src/components/table/peek/README.md:1-100]()\n\n## Feature Modules\n\n### Entitlements System\n\nThe entitlements feature controls feature availability at the organization level:\n\n| Concept | Definition |\n|---------|------------|\n| Plan | Feature tier (OSS, cloud:pro, self-hosted:enterprise) |\n| Entitlement | Available feature (e.g., playground, score analytics) |\n| EntitlementLimit | Resource limits (e.g., annotation-queue-count) |\n\n**Plan Resolution:**\n- Cloud: Added to organization via JWT from NextAuth\n- Self-hosted: From license key or environment configuration\n\n资料来源：[web/src/features/entitlements/README.md:1-50]()\n\n### Score Analytics\n\nThe score analytics feature provides comprehensive statistical analysis of evaluation scores:\n\n**Architecture Components:**\n\n| Component | Location | Responsibility |\n|-----------|----------|----------------|\n| Provider | `ScoreAnalyticsProvider.tsx` | Context management |\n| Hook | `useScoreAnalyticsQuery` | Data fetching and transformation |\n| Transformers | `scoreAnalyticsTransformers.ts` | Data transformation pipeline |\n| Router | `scoreAnalyticsRouter.ts` | tRPC API endpoint |\n\n**Data Flow:**\n```mermaid\ngraph LR\n A[API Request] --> B[tRPC Router]\n B --> C[ClickHouse Query]\n C --> D[Transformers]\n D --> E[ScoreAnalyticsProvider]\n E --> F[Chart Components]\n```\n\n资料来源：[packages/shared/src/features/scores/interfaces/README.md:1-80]()\n资料来源：[web/src/features/score-analytics/README.md:1-100]()\n\n### Score Interfaces Architecture\n\nLangfuse maintains a versioned API structure for scores:\n\n```\ninterfaces/\n├── api/\n│ ├── v1/ # Legacy API (trace-focused)\n│ ├── v2/ # Current API (supports traces, sessions)\n│ └── shared.ts\n├── application/\n├── ingestion/\n└── ui/\n```\n\n**API Versioning Strategy:**\n- POST/DELETE APIs: Support all score types across v1 and v2\n- GET APIs:\n - V1: Requires `traceId`, trace-level only\n - V2: `traceId` optional, adds `sessionId` support\n\n资料来源：[packages/shared/src/features/scores/interfaces/README.md:1-50]()\n\n## Backend Worker Architecture\n\n### Event Processing\n\nThe worker service processes ingestion events asynchronously using BullMQ queues:\n\n**Queue Types:**\n| Queue | Purpose | Consumer |\n|-------|---------|----------|\n| `IngestionSecondaryQueue` | Standard event processing | Worker |\n| `OtelIngestionQueue` | OpenTelemetry events | Worker |\n\n**Event Flow:**\n```mermaid\ngraph LR\n A[Ingestion API] --> B{Event Type?}\n B -->|Standard| C[S3 Key Parse]\n B -->|OTEL| D[OTEL Key Parse]\n C --> E[Queue Payload]\n D --> E\n E --> F[BullMQ]\n F --> G[Worker Processing]\n G --> H[(ClickHouse)]\n G --> I[(PostgreSQL)]\n```\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md:1-60]()\n\n### Replay Ingestion Events V2\n\nThe `replayIngestionEventsV2` script enables replaying historical events from S3 storage:\n\n**Key Features:**\n- Batch processing with configurable size\n- Checkpoint/resume capability\n- Rate limiting with exponential backoff\n- Error handling with detailed logging\n\n**Differences from V1:**\n\n| Aspect | V1 | V2 |\n|--------|----|----|\n| Infrastructure | Redis, ClickHouse, PostgreSQL, S3 | Langfuse host URL only |\n| Setup | Full repo clone + `.env` | `npx tsx` + env vars |\n| Event Delivery | BullMQ `addBulk` to Redis | HTTP POST to admin API |\n| Resume | Manual | Built-in checkpoint |\n\n**Event Transformation:**\n- Standard keys: `{projectId}/{type}/{eventBodyId}/{eventId}.json`\n- OTEL keys: `otel/{projectId}/{yyyy}/{mm}/{dd}/{hh}/{mm}/{eventId}.json`\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md:1-120]()\n\n### Refill Queue Event\n\nThe `refillQueueEvent` utility script backfills queues with events from local files:\n\n**Usage Pattern:**\n```bash\npnpm run --filter=worker refill-queue-event\n```\n\n**Requirements:**\n- `./worker/events.jsonl` file with JSON events\n- Redis connection via `REDIS_CONNECTION_STRING`\n- Supporting services: S3, ClickHouse\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md:1-60]()\n\n## MCP Server Architecture\n\nLangfuse includes an MCP (Model Context Protocol) server for programmatic prompt management:\n\n**Stateless Design:**\n1. Fresh server instance per request\n2. Authentication context captured in handler closures\n3. Server discarded after request completes\n4. No state between requests\n\n**Available Tools:**\n| Tool | Purpose |\n|------|---------|\n| `getPrompt` | Fetch resolved prompt with dependencies |\n| `getPromptUnresolved` | Fetch raw prompt without resolution |\n| `listPrompts` | List prompts with filtering |\n| `createTextPrompt` | Create text prompt version |\n| `createChatPrompt` | Create chat prompt version |\n| `updatePromptLabels` | Manage prompt labels |\n\n**Prompt Resolution:**\n- Resolved: Recursively replaces `@@@langfusePrompt:...@@@` tags\n- Unresolved: Returns raw content with tags intact\n\n资料来源：[web/src/features/mcp/README.md:1-100]()\n\n## Component Communication Flow\n\n```mermaid\ngraph TD\n subgraph Pages[\"Page Layer\"]\n P[Page Component]\n PC[PeekTableStateProvider]\n end\n \n subgraph Features[\"Feature Layer\"]\n FP[Feature Provider]\n FH[Feature Hook]\n FT[Feature Transformers]\n end\n \n subgraph Services[\"Service Layer\"]\n TRPC[tRPC Router]\n API[API Routes]\n end\n \n subgraph Data[\"Data Layer\"]\n Repo[Repositories]\n DB[(PostgreSQL)]\n CH[(ClickHouse)]\n Redis[(Redis)]\n end\n \n P --> PC\n PC --> FP\n FP --> FH\n FH --> FT\n FT --> TRPC\n TRPC --> Repo\n Repo --> DB\n Repo --> CH\n Repo --> Redis\n```\n\n## Configuration Management\n\nLangfuse uses environment-based configuration across layers:\n\n| Environment | Scope | Examples |\n|-------------|-------|----------|\n| `REDIS_CONNECTION_STRING` | Worker, Queue | Redis URL |\n| `CLICKHOUSE_URL` | Analytics | ClickHouse connection |\n| `LANGFUSE_S3_EVENT_UPLOAD_BUCKET` | Storage | S3 bucket name |\n| `ADMIN_API_KEY` | Admin API | Authentication |\n\n## Security Architecture\n\n**Key Security Components:**\n\n1. **JWT Authentication**: Organization and user context embedded in JWT tokens\n2. **API Key Validation**: Admin API uses dedicated key authentication\n3. **Scope-based Authorization**: Project-level access control\n4. **Plan Entitlements**: Feature availability based on subscription tier\n\n**Self-hosted Considerations:**\n- License key validation for enterprise features\n- Environment-based plan override capability\n\n## Performance Optimizations\n\n**Frontend Optimizations:**\n- Memoized transformations in hooks\n- Virtualized table rendering (TanStack Virtual)\n- Iterative algorithms (no recursion, preventing stack overflow)\n\n**Backend Optimizations:**\n- Batch processing for event replay\n- Checkpoint/resume for long-running operations\n- Client-side rate limiting with exponential backoff\n\n## Summary\n\nThe Langfuse architecture demonstrates a well-structured approach to observability platforms:\n\n- **Separation of Concerns**: Clear boundaries between UI, business logic, and data layers\n- **Scalability**: Asynchronous processing via queues enables horizontal scaling\n- **Extensibility**: Feature modules and MCP server support programmatic access\n- **Observability**: Built-in OpenTelemetry integration for distributed tracing\n- **Performance**: ClickHouse for analytics, iterative algorithms, and batch processing\n\n---\n\n\n\n## Monorepo Configuration\n\n### 相关页面\n\n相关主题：[Project Structure](#project-structure), [System Architecture](#system-architecture)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/src/server/clickhouse/client.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/clickhouse/client.ts)\n- [packages/shared/src/server/redis/redis.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/redis.ts)\n- [packages/shared/src/db.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/db.ts)\n- [packages/shared/src/server/otel/index.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/otel/index.ts)\n- [web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n- [web/src/components/design-system/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/design-system/README.md)\n- [web/src/components/table/peek/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/table/peek/README.md)\n- [web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n- [packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n- [worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n
\n\n# Monorepo Configuration\n\nLangfuse uses a **monorepo architecture** managed with [Turborepo](https://turbo.build/), [pnpm](https://pnpm.io/) workspaces, and shared configuration packages. This setup enables efficient builds, consistent code quality standards, and streamlined dependency management across the project's multiple packages.\n\n## Architecture Overview\n\nThe Langfuse repository is organized as a pnpm workspace monorepo with the following core structure:\n\n```yaml\nlangfuse/\n├── web/ # Next.js frontend application\n├── worker/ # Background job processing\n├── packages/\n│ ├── shared/ # Shared utilities and types\n│ ├── config-eslint/ # Shared ESLint configuration\n│ └── config-typescript/ # Shared TypeScript configurations\n├── turbo.json # Turborepo pipeline definition\n└── pnpm-workspace.yaml # Workspace package definitions\n```\n\n## Turborepo Pipeline Configuration\n\nThe `turbo.json` file defines the build pipeline and task orchestration across packages.\n\n### Core Pipeline Tasks\n\n| Task | Description | Cache Strategy |\n|------|-------------|----------------|\n| `build` | Compiles TypeScript and bundles assets | Enabled |\n| `dev` | Starts development servers | Local only |\n| `test` | Runs unit and integration tests | Enabled |\n| `lint` | ESLint code quality checks | Enabled |\n| `typecheck` | TypeScript type validation | Enabled |\n\n### Task Dependencies\n\nTurborepo automatically resolves dependencies between packages. For example:\n\n```mermaid\ngraph TD\n A[packages/shared] -->|build| B[Type definitions]\n B --> C[packages/config-*]\n C --> D[web]\n C --> E[worker]\n D --> F[Build output]\n E --> F\n```\n\n## Shared ESLint Configuration\n\nThe `packages/config-eslint/index.js` provides standardized ESLint rules across all packages.\n\n### Configuration Features\n\n- **React/Next.js support** for the web application\n- **TypeScript-aware linting** via `@typescript-eslint`\n- **Import ordering** rules for consistent module organization\n- **JSX accessibility** checks\n\n### Usage\n\nPackages extend the shared configuration:\n\n```javascript\n// In package's .eslintrc.js\nmodule.exports = {\n extends: ['@langfuse/config-eslint'],\n // Package-specific overrides\n rules: {\n // Custom rules\n }\n};\n```\n\n资料来源：[packages/config-eslint/index.js]()\n\n## Shared TypeScript Configuration\n\nThe `packages/config-typescript/` directory contains base TypeScript configurations.\n\n### Base Configuration (`base.json`)\n\n```json\n{\n \"compilerOptions\": {\n \"target\": \"ES2020\",\n \"module\": \"ESNext\",\n \"moduleResolution\": \"bundler\",\n \"strict\": true,\n \"esModuleInterop\": true,\n \"skipLibCheck\": true,\n \"forceConsistentCasingInFileNames\": true\n }\n}\n```\n\n### Package-Specific Configurations\n\nIndividual packages extend the base configuration:\n\n```json\n// packages/shared/tsconfig.json\n{\n \"extends\": \"@langfuse/config-typescript/base.json\",\n \"compilerOptions\": {\n \"outDir\": \"./dist\",\n \"rootDir\": \"./src\"\n },\n \"include\": [\"src/**/*\"],\n \"exclude\": [\"node_modules\", \"dist\"]\n}\n```\n\n资料来源：[packages/config-typescript/base.json](), [packages/shared/tsconfig.json]()\n\n## Package Scripts and Commands\n\nEach package defines its own scripts in `package.json`. The web package demonstrates the typical pattern:\n\n### Build Commands\n\n| Command | Purpose |\n|---------|---------|\n| `pnpm build` | Production build with `INLINE_RUNTIME_CHUNK=false` |\n| `pnpm build:check` | Build without emitting (type-checking) |\n| `pnpm dev` | Development server on localhost:3000 |\n| `pnpm dev:http` | HTTPS development server for local testing |\n\n### Quality Assurance Commands\n\n| Command | Purpose |\n|---------|---------|\n| `pnpm lint` | ESLint with caching enabled |\n| `pnpm lint:fix` | Auto-fix linting issues |\n| `pnpm typecheck` | TypeScript validation with incremental compilation |\n| `pnpm test` | Vitest with server and in-source test projects |\n\n资料来源：[web/package.json]()\n\n## Development Workflow\n\n### Starting Development\n\n```bash\n# Install dependencies\npnpm install\n\n# Start all dev servers based on turbo.json\npnpm dev\n\n# Or start a specific package\ncd web && pnpm dev\n```\n\n### Running Tests\n\n```bash\n# All tests\npnpm test\n\n# Client-side tests only (e.g., AdvancedJsonViewer)\npnpm --filter=web run test-client --testPathPattern=\"AdvancedJsonViewer\"\n\n# Server-side tests\npnpm --filter=web run test --project server\n```\n\n### Build Pipeline\n\n```mermaid\ngraph LR\n A[pnpm build] --> B[turbo build]\n B --> C{Cache hit?}\n C -->|Yes| D[Use cached output]\n C -->|No| E[Build dependencies]\n E --> F[packages/shared]\n F --> G[packages/config-*]\n G --> H[web/worker]\n H --> I[Save to cache]\n I --> J[Build artifacts]\n```\n\n## Environment Configuration\n\nThe project uses `.env` files for environment-specific configuration:\n\n| File | Purpose |\n|------|---------|\n| `.env` | Default environment variables |\n| `.env.local` | Local overrides (git-ignored) |\n| `.env.test` | Test environment variables |\n\nScripts use `dotenv` to load these files:\n\n```bash\ndotenv -e ../.env -- next build\ndotenv -e ../.env.test -e ../.env -- vitest run\n```\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n## Best Practices\n\n### Adding a New Package\n\n1. Create the package under `packages/` or `web/` directories\n2. Extend the shared TypeScript and ESLint configurations\n3. Add the package to `pnpm-workspace.yaml` if needed\n4. Define tasks in `turbo.json` if custom pipeline is required\n5. Add appropriate scripts to `package.json`\n\n### Caching Strategy\n\n- **Build caching**: Enabled by default via Turborepo\n- **Lint caching**: ESLint caches to `.next/cache/eslint/`\n- **TypeScript incremental**: Uses `.tsbuildinfo` files\n\n### CI/CD Integration\n\nIn CI environments, clear caches to ensure fresh builds:\n\n```bash\n# Clear turbo cache\nrm -rf .turbo node_modules/.cache\n\n# Clear ESLint cache\nrm -rf .next/cache/eslint/\n\n# Fresh build\npnpm install --frozen-lockfile\npnpm build\n```\n\n## Key Configuration Files Summary\n\n| File | Purpose |\n|------|---------|\n| `turbo.json` | Task pipeline and dependency graph |\n| `pnpm-workspace.yaml` | Workspace package definitions |\n| `packages/config-eslint/index.js` | Shared ESLint rules |\n| `packages/config-typescript/base.json` | Shared TypeScript base config |\n| `.eslintrc.js` (per package) | Package-specific lint overrides |\n| `tsconfig.json` (per package) | Package-specific TypeScript config |\n\n---\n\n\n\n## Database Schema (Prisma)\n\n### 相关页面\n\n相关主题：[System Architecture](#system-architecture), [ClickHouse Analytics Layer](#clickhouse-analytics)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [turbo.json](https://github.com/langfuse/langfuse/blob/main/turbo.json)\n- [packages/config-eslint/index.js](https://github.com/langfuse/langfuse/blob/main/packages/config-eslint/index.js)\n- [packages/config-typescript/base.json](https://github.com/langfuse/langfuse/blob/main/packages/config-typescript/base.json)\n- [packages/shared/tsconfig.json](https://github.com/langfuse/langfuse/blob/main/packages/shared/tsconfig.json)\n- [web/package.json](https://github.com/langfuse/langfuse/blob/main/web/package.json)\n- [worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n
\n\n# Database Schema (Prisma)\n\n## Overview\n\nLangfuse uses Prisma ORM to manage its PostgreSQL database schema. The schema defines the core data models for the application, including organizations, projects, users, traces, observations, and scores. Prisma serves as the primary interface between the application's business logic and the relational database.\n\nThe Prisma schema is located at `packages/shared/prisma/schema.prisma` and is shared across multiple packages in the monorepo structure. This centralized schema approach ensures consistency in data modeling across the web application, worker services, and shared libraries.\n\n### Design Philosophy\n\nThe schema follows several key principles:\n\n- **Normalized relationships**: Related entities are linked through foreign keys with proper cascading behaviors\n- **Soft deletes**: Key entities support soft deletion for data recovery and audit purposes\n- **Audit fields**: Most tables include `createdAt`, `updatedAt`, and `createdBy` fields\n- **Multi-tenancy**: The schema supports multi-tenant architecture with organization and project isolation\n- **Extensible metadata**: JSON fields allow flexible storage of custom attributes\n\n## Core Data Models\n\n### Organization and User Models\n\nThe foundation of Langfuse's multi-tenant architecture begins with the `Organization` model, which represents the top-level tenant entity. Each organization can have multiple users with different roles and permission levels.\n\nThe `User` model stores authentication and profile information, linked to organizations through the `Membership` junction table. This many-to-many relationship enables users to belong to multiple organizations with potentially different roles in each.\n\n```mermaid\nerDiagram\n Organization ||--o{ User : \"contains\"\n Organization ||--o{ Project : \"contains\"\n User ||--o{ Membership : \"has\"\n Membership }o--|| Organization : \"belongs to\"\n Membership }o--|| User : \"belongs to\"\n```\n\n### Project Model\n\nProjects serve as the primary container for observability data. Each project belongs to exactly one organization and contains all traces, observations, and scores related to a specific application or use case.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | `String` | UUID primary key |\n| `name` | `String` | Project display name |\n| `organizationId` | `String` | Foreign key to organization |\n| `createdAt` | `DateTime` | Creation timestamp |\n| `updatedAt` | `DateTime` | Last modification timestamp |\n| `deletedAt` | `DateTime?` | Soft delete timestamp |\n| `settings` | `Json` | Project-specific configuration |\n\n资料来源：[packages/shared/prisma/schema.prisma]()\n\n## Traces\n\nTraces represent the top-level unit of observability in Langfuse. A trace encapsulates a complete interaction or request, typically corresponding to a single LLM call or a multi-step workflow.\n\n### Trace Model Schema\n\n```prisma\nmodel Trace {\n id String @id @default(cuid())\n name String?\n project Project @relation(fields: [projectId], references: [id])\n projectId String\n user String?\n metadata Json?\n sessionId String?\n release String?\n version String?\n tags String[]\n \n // Timestamps\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n \n // Soft delete\n deletedAt DateTime?\n \n // Relations\n observations Observation[]\n scores Score[]\n \n @@index([projectId])\n @@index([sessionId])\n @@index([createdAt])\n}\n```\n\n资料来源：[packages/shared/prisma/schema.prisma]()\n\n### Key Fields\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | `String` | Unique identifier using CUID algorithm |\n| `name` | `String?` | Optional human-readable trace name |\n| `projectId` | `String` | Reference to parent project |\n| `user` | `String?` | Identifier for the end user |\n| `sessionId` | `String?` | Groups related traces into sessions |\n| `release` | `String?` | Application release version |\n| `version` | `String?` | Trace format version |\n| `tags` | `String[]` | Array of string tags for categorization |\n\n### Repository Pattern\n\nTraces are accessed through the repository pattern defined in `packages/shared/src/server/repositories/traces.ts`. This abstraction provides a clean interface for CRUD operations while encapsulating query logic.\n\n资料来源：[packages/shared/src/server/repositories/traces.ts]()\n\n```typescript\n// Repository interface pattern (simplified)\ninterface ITraceRepository {\n create(data: CreateTraceInput): PromiseRelevant Source Files
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/prisma/schema.prisma](https://github.com/langfuse/langfuse/blob/main/packages/shared/prisma/schema.prisma)\n- [packages/shared/src/server/repositories/traces.ts](https://packages/shared/src/server/repositories/traces.ts)\n- [packages/shared/src/server/repositories/observations.ts](https://packages/shared/src/server/repositories/observations.ts)\n- [packages/shared/src/server/repositories/scores.ts](https:///packages/shared/src/server/repositories/scores.ts)\n- [packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n\n
\n\n# ClickHouse Analytics Layer\n\n## Overview\n\nThe ClickHouse Analytics Layer is a core infrastructure component in Langfuse that provides high-performance analytical capabilities for processing and querying large-scale observability data. ClickHouse serves as the primary OLAP (Online Analytical Processing) database for storing traces, observations, and score analytics with optimized columnar storage and efficient aggregation queries.\n\nLangfuse leverages ClickHouse for:\n\n- High-throughput event ingestion during trace collection\n- Complex analytical queries for score comparisons and distributions\n- Time-series analysis with efficient aggregation\n- Large dataset sampling and optimization strategies\n 资料来源：[packages/shared/scripts/seeder/utils/README.md]()\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n subgraph Ingestion[\"Ingestion Layer\"]\n W[Worker Service] --> CW[ClickhouseWriter]\n CW --> CH[ClickHouse Cluster]\n end\n \n subgraph Storage[\"Storage Layer\"]\n CH --> TS[Traces Table]\n CH --> OS[Observations Table]\n CH --> SS[Scores Table]\n end\n \n subgraph Query[\"Query Layer\"]\n CR[ClickHouse Repository] --> CH\n SA[Score Analytics] --> CR\n WEB[Web Frontend] --> SA\n end\n \n subgraph Optimization[\"Optimization Layer\"]\n CR --> HASH[cityHash64 Sampling]\n CR --> FINAL[Adaptive FINAL]\n CR --> INTERVAL[Time Interval Alignment]\n end\n```\n\n### Components Overview\n\n| Component | Location | Purpose |\n|-----------|----------|---------|\n| ClickhouseWriter | `worker/src/services/ClickhouseWriter/index.ts` | Writes ingestion events to ClickHouse |\n| ClickHouse Repository | `packages/shared/src/server/repositories/clickhouse.ts` | Provides query interface and optimization |\n| Score Analytics | `packages/shared/src/server/repositories/score-analytics.ts` | Specialized analytics queries |\n| Schema Definitions | `packages/shared/src/server/clickhouse/schema.ts` | TypeScript types for ClickHouse data |\n| Migrations | `packages/shared/clickhouse/migrations/clustered/` | Database schema migrations |\n\n资料来源：[worker/src/services/ClickhouseWriter/index.ts]()\n资料来源：[packages/shared/src/server/repositories/clickhouse.ts]()\n\n## Data Schema\n\n### Traces Table\n\nThe traces table stores the fundamental trace records with hierarchical observation data. The clustered migration defines the primary schema with optimized column types for analytical queries.\n\nKey columns include:\n\n| Column | Type | Description |\n|--------|------|-------------|\n| id | UUID | Unique trace identifier |\n| project_id | String | Project association |\n| timestamp | DateTime64 | Event timestamp with millisecond precision |\n| name | String | Trace name |\n| user_id | String | User identifier |\n| metadata | JSON | Flexible metadata storage |\n| tags | Array(String) | Tag-based categorization |\n| input | Text | Input data |\n| output | Text | Output data |\n\n资料来源：[packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql]()\n\n### Observations Table\n\nObservations represent individual events within a trace, storing:\n\n- Model inputs/outputs\n- Function calls\n- Embeddings\n- Generation events\n\nEach observation is linked to its parent trace via `trace_id` and supports nested hierarchies through `parent_observation_id`.\n\n资料来源：[packages/shared/src/server/clickhouse/schema.ts]()\n\n### Scores Table\n\nScores store evaluation metrics associated with traces and observations:\n\n| Column | Type | Purpose |\n|--------|------|---------|\n| trace_id | UUID | Associated trace |\n| observation_id | UUID | Optional observation link |\n| name | String | Score identifier |\n| value | Float64 | Numeric score value |\n| data_type | Enum | NUMERIC, BOOLEAN, or CATEGORICAL |\n| source | String | Score origin (e.g., \"framework-trace\") |\n\n资料来源：[packages/shared/src/server/repositories/score-analytics.ts]()\n\n## Ingestion Pipeline\n\n### Event Flow\n\n```mermaid\nsequenceDiagram\n participant API as Ingestion API\n participant Queue as Redis Queue\n participant Worker as Worker Service\n participant Writer as ClickhouseWriter\n participant CH as ClickHouse\n \n API->>Queue: Enqueue OtelIngestionEvent\n Worker->>Queue: Dequeue Event\n Worker->>Writer: Process Event\n Writer->>CH: Insert Batch (ClickHouseQueryBuilder)\n CH-->>Writer: Confirmation\n Writer->>Worker: Acknowledge\n```\n\n### ClickhouseWriter Service\n\nThe `ClickhouseWriter` handles the actual data insertion into ClickHouse:\n\n```typescript\n// Simplified flow from worker/src/services/ClickhouseWriter/index.ts\nclass ClickhouseWriter {\n async writeBatch(events: IngestionEvent[]): Promise相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql](https://github.com/langfuse/langfuse/blob/main/packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql)\n- [packages/shared/src/server/clickhouse/schema.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/clickhouse/schema.ts)\n- [packages/shared/src/server/repositories/clickhouse.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/repositories/clickhouse.ts)\n- [worker/src/services/ClickhouseWriter/index.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/services/ClickhouseWriter/index.ts)\n- [packages/shared/src/server/repositories/score-analytics.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/repositories/score-analytics.ts)\n- [web/src/features/score-analytics/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/score-analytics/README.md)\n- [packages/shared/scripts/seeder/utils/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/scripts/seeder/utils/README.md)\n\n
\n\n# Queue System (Redis/BullMQ)\n\nLangfuse employs a distributed queue system built on **Redis** for storage and **BullMQ** for job orchestration. This architecture enables asynchronous processing of high-volume operations including event ingestion, evaluation execution, batch actions, and webhook delivery.\n\n## Architecture Overview\n\nThe queue system follows a producer-consumer pattern where the web application enqueues jobs and worker processes consume them asynchronously.\n\n```mermaid\ngraph TD\n subgraph \"Langfuse Web Application\"\n A[API Request] --> B[Queue Client]\n B --> C[Redis Queue]\n end\n \n subgraph \"Langfuse Worker\"\n C --> D[Worker Manager]\n D --> E1[Ingestion Worker]\n D --> E2[Eval Worker]\n D --> E3[Batch Worker]\n D --> E4[Webhook Worker]\n end\n \n subgraph \"Redis\"\n C --> F[(Redis Cluster)]\n end\n \n subgraph \"External Services\"\n E1 --> G[(ClickHouse)]\n E1 --> H[(PostgreSQL)]\n E2 --> H\n E3 --> H\n E4 --> I[(External APIs)]\n end\n```\n\n## Queue Types\n\nLangfuse defines multiple specialized queues for different workloads:\n\n| Queue Name | Purpose | Processing Type | Priority |\n|------------|---------|-----------------|----------|\n| `ingestion` | Event ingestion and processing | Async batch | Medium |\n| `evalExecution` | LLM evaluation execution | Async | Medium |\n| `batchAction` | Bulk operations on data | Async batch | Low |\n| `webhook` | Outbound webhook delivery | Async | High |\n| `OtelIngestion` | OpenTelemetry event ingestion | Async | Medium |\n| `IngestionSecondary` | Secondary ingestion processing | Async | Medium |\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n## Queue Configuration\n\n### Redis Connection\n\nAll queues rely on a Redis connection string configured via environment variables:\n\n```bash\nREDIS_CONNECTION_STRING=redis://:myredissecret@127.0.0.1:6379\n```\n\n### Queue Initialization\n\nEach queue is initialized with specific BullMQ configuration:\n\n```typescript\nconst myQueue = new QueueRelevant Source Files
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/src/server/redis/ingestionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/ingestionQueue.ts)\n- [packages/shared/src/server/redis/evalExecutionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/evalExecutionQueue.ts)\n- [packages/shared/src/server/redis/batchActionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/batchActionQueue.ts)\n- [packages/shared/src/server/redis/webhookQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/webhookQueue.ts)\n- [worker/src/queues/workerManager.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/queues/workerManager.ts)\n\n
\n\n# Worker Service\n\n## Overview\n\nThe Worker Service is a core backend component in Langfuse responsible for asynchronous processing of long-running tasks. It operates as a separate Node.js process that communicates with the main Langfuse server through message queues, primarily using BullMQ backed by Redis.\n\n```mermaid\ngraph TB\n subgraph \"Langfuse Server\"\n A[API Endpoints]\n B[tRPC Routers]\n end\n \n subgraph \"Redis\"\n C[(Ingestion Queues)]\n D[(Evaluation Queues)]\n E[(Batch Action Queues)]\n end\n \n subgraph \"Worker Service\"\n F[Worker Manager]\n G[Evaluation Service]\n H[Batch Action Handler]\n I[Queue Processors]\n end\n \n A -->|\"Enqueue Jobs\"| C\n B -->|\"Dispatch Tasks\"| C\n F -->|\"Process\"| C\n G -->|\"Execute\"| D\n H -->|\"Execute\"| E\n C --> F\n D --> G\n E --> H\n```\n\n### Purpose and Scope\n\nThe Worker Service handles the following categories of work:\n\n- **Event Ingestion Processing**: Processing and persisting trace events, observations, and spans from the ingestion queues\n- **Evaluation Execution**: Running LLM-based evaluations on traces and observations\n- **Batch Actions**: Executing bulk operations on datasets, traces, and other resources\n- **Queue Replay**: Replaying historical ingestion events for data recovery or reprocessing\n\n资料来源：[worker/src/app.ts]()\n\n---\n\n## Architecture\n\n### Entry Point\n\nThe worker service is bootstrapped through `worker/src/index.ts`, which initializes the application context and starts the queue processors. The main application logic resides in `worker/src/app.ts`, which sets up the BullMQ workers and registers job handlers.\n\n```typescript\n// Simplified worker initialization flow\nconst app = new WorkerApp();\nawait app.initialize();\nawait app.start();\n```\n\n资料来源：[worker/src/index.ts]()\n\n### Worker Manager\n\nThe `WorkerManager` is the central orchestrator that manages all queue workers. It is responsible for:\n\n- Registering queue processors for different job types\n- Configuring concurrency settings per queue\n- Handling job failures and retries\n- Graceful shutdown coordination\n\n```mermaid\ngraph LR\n A[Job Enqueued] --> B[Worker Manager]\n B --> C{Job Type?}\n C -->|Ingestion| D[Ingestion Processor]\n C -->|Evaluation| E[Eval Service]\n C -->|Batch Action| F[Batch Handler]\n \n D --> G[Success]\n E --> G\n F --> G\n \n D --> H[Retry/Fail]\n E --> H\n F --> H\n```\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n### Queue Architecture\n\nLangfuse uses multiple queues for different purposes:\n\n| Queue Name | Purpose | Priority | Concurrency |\n|------------|---------|----------|-------------|\n| `IngestionSecondaryQueue` | Standard event ingestion | Medium | Configurable |\n| `OtelIngestionQueue` | OpenTelemetry-format events | High | Configurable |\n| `EvalQueue` | LLM evaluation jobs | Low | Configurable |\n| `BatchActionQueue` | Bulk operations | Medium | Configurable |\n\n资料来源：[worker/src/app.ts]()\n资料来源：[worker/src/queues/workerManager.ts]()\n\n---\n\n## Core Components\n\n### Evaluation Service\n\nThe Evaluation Service (`evalService.ts`) handles asynchronous evaluation of traces and observations using LLM-based judges. It supports:\n\n- **Trace-level evaluations**: Running multiple evaluation criteria against a complete trace\n- **Observation-level evaluations**: Evaluating individual spans or generations\n- **Dataset-based evaluations**: Running evaluations against dataset items\n- **Configurable retry logic**: Handling transient failures gracefully\n\n```mermaid\ngraph TD\n A[Evaluation Job Received] --> B[Load Trace/Observation]\n B --> C[Fetch Eval Config]\n C --> D[Prepare Prompt]\n D --> E[Call LLM Provider]\n E --> F{Success?}\n F -->|Yes| G[Score Result]\n F -->|No| H{Retry < 3?}\n H -->|Yes| E\n H -->|No| I[Mark Failed]\n G --> J[Persist Score]\n J --> K[Job Complete]\n I --> K\n```\n\n资料来源：[worker/src/features/evaluation/evalService.ts]()\n\n### Batch Action Handler\n\nThe Batch Action Handler processes bulk operations requested through the admin API or scheduled jobs. Common batch actions include:\n\n- Bulk dataset operations (import, export, deletion)\n- Mass updates to traces or observations\n- Batch score calculations\n- Export operations\n\n```typescript\ninterface BatchActionJob {\n id: string;\n type: BatchActionType;\n payload: BatchActionPayload;\n priority?: number;\n createdAt: Date;\n}\n```\n\n资料来源：[worker/src/features/batchAction/handleBatchActionJob.ts]()\n\n---\n\n## Queue Processing\n\n### Job Flow\n\n```mermaid\nsequenceDiagram\n participant API as API Server\n participant Redis as Redis Queue\n participant Worker as Worker Service\n \n API->>Redis: Enqueue Job\n Worker->>Redis: Poll for Jobs\n Redis-->>Worker: Available Jobs\n Worker->>Worker: Process Job\n Worker->>Redis: Mark Complete\n Worker->>API: Update Status (optional)\n```\n\n### Error Handling and Retries\n\nThe worker implements exponential backoff for transient failures:\n\n1. **Transient failures** (network timeouts, rate limits): Retry up to 3 times with exponential backoff\n2. **Permanent failures** (validation errors, missing data): Mark as failed, log error details\n3. **Rate limiting**: Respects server-side rate limits and backs off accordingly\n\n| Error Type | HTTP Code | Retry Behavior |\n|------------|-----------|----------------|\n| Rate Limited | 429 | Exponential backoff |\n| Server Error | 5xx | Retry up to 3 times |\n| Client Error | 4xx (not 429) | No retry, log and skip |\n| Validation | N/A | No retry, mark failed |\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n### Concurrency Configuration\n\nWorkers can be configured with different concurrency levels per queue:\n\n```typescript\nconst workerConfig = {\n ingestion: {\n concurrency: 10,\n maxRetries: 3,\n },\n evaluation: {\n concurrency: 5,\n maxRetries: 2,\n },\n batchAction: {\n concurrency: 3,\n maxRetries: 1,\n },\n};\n```\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n---\n\n## Utility Scripts\n\n### Replay Ingestion Events V2\n\nThe replay script allows reprocessing of historical ingestion events from S3 storage.\n\n**Usage:**\n```bash\npnpm --filter=worker run replay-ingestion-events-v2 \\\n --input ./events.csv \\\n --batch-size 500 \\\n --concurrency 4\n```\n\n**Key Features:**\n- Checkpoint/resume support for interrupted runs\n- Configurable batch size and concurrency\n- Rate limiting with exponential backoff\n- Progress tracking with percentage and ETA\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `--input` | Required | Path to CSV file with S3 keys |\n| `--batch-size` | 500 | Keys per API request |\n| `--concurrency` | 4 | Parallel API requests |\n| `--rate-limit` | 50 | Max requests per second |\n| `--dry-run` | false | Validate without sending |\n| `--resume` | false | Continue from checkpoint |\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n### Refill Queue Event\n\nThe refill script backfills queues with events from local files for testing and development.\n\n**Requirements:**\n1. Create `./worker/events.jsonl` with JSON events\n2. Configure `.env` with Redis and supporting service credentials\n3. Run: `pnpm run --filter=worker refill-queue-event`\n\n**Event Format:**\n```jsonl\n{\"projectId\": \"project-123\", \"orgId\": \"org-456\"}\n{\"projectId\": \"project-789\", \"orgId\": \"org-101\"}\n```\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md]()\n\n---\n\n## Environment Configuration\n\n### Required Environment Variables\n\n| Variable | Description | Required |\n|----------|-------------|----------|\n| `REDIS_CONNECTION_STRING` | Redis connection URL | Yes |\n| `LANGFUSE_S3_EVENT_UPLOAD_BUCKET` | S3 bucket for event uploads | Yes |\n| `CLICKHOUSE_URL` | ClickHouse connection URL | Yes |\n| `CLICKHOUSE_USER` | ClickHouse username | Yes |\n| `CLICKHOUSE_PASSWORD` | ClickHouse password | Yes |\n| `ADMIN_API_KEY` | Admin API key for replay scripts | For replay only |\n| `LANGFUSE_HOST` | Target Langfuse instance URL | For replay only |\n\n### Optional Configuration\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `WORKER_CONCURRENCY` | 10 | Default queue concurrency |\n| `EVALUATION_CONCURRENCY` | 5 | Evaluation queue concurrency |\n| `BATCH_ACTION_CONCURRENCY` | 3 | Batch action concurrency |\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md]()\n资料来源：[worker/src/app.ts]()\n\n---\n\n## Health and Monitoring\n\n### Debug Mode\n\nEnable detailed logging for the AdvancedJsonViewer (useful for debugging UI components):\n\n```javascript\nlocalStorage.setItem(\"debug:AdvancedJsonViewer\", \"true\");\n```\n\n### Logging Categories\n\nThe worker service emits structured logs for:\n\n- Queue processing events\n- Job duration and throughput\n- Error rates and failure reasons\n- Resource utilization metrics\n\n### Graceful Shutdown\n\nThe worker service supports graceful shutdown to prevent job loss:\n\n1. Stop accepting new jobs\n2. Wait for in-progress jobs to complete (with timeout)\n3. Release Redis connections\n4. Exit cleanly\n\n---\n\n## Performance Considerations\n\n### Scalability\n\n- Multiple worker instances can run in parallel\n- Each queue can have independent concurrency settings\n- Redis acts as a shared job queue, enabling horizontal scaling\n\n### Memory Management\n\n| Strategy | Implementation |\n|----------|----------------|\n| Iterative Processing | Uses explicit stack-based iteration instead of recursion to prevent stack overflow |\n| Batch Processing | Processes events in configurable batch sizes |\n| Lazy Loading | Loads trace/observation data on-demand within job handlers |\n\n### Known Limitations\n\n1. **No horizontal virtualization in UI**: Wide JSON rows render fully\n2. **Client-side search only**: All matches computed in memory\n3. **Memory constraints**: 1M+ nodes may cause issues despite virtualization\n4. **Wrap mode performance**: Long strings require height measurement\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md]()\n\n---\n\n## Summary\n\nThe Worker Service is essential to Langfuse's architecture, providing asynchronous processing capabilities that keep the main API server responsive. Key takeaways:\n\n- **BullMQ + Redis**: Provides reliable job queuing with built-in retry logic\n- **Specialized processors**: Dedicated handlers for ingestion, evaluation, and batch actions\n- **Configurable concurrency**: Fine-tune throughput per queue type\n- **Replay capabilities**: Built-in utilities for reprocessing historical events\n- **Horizontal scaling**: Multiple worker instances share work through Redis\n\n---\n\n\n\n## API Layer\n\n### 相关页面\n\n相关主题：[System Architecture](#system-architecture)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [worker/src/app.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/app.ts)\n- [worker/src/index.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/index.ts)\n- [worker/src/queues/workerManager.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/queues/workerManager.ts)\n- [worker/src/features/evaluation/evalService.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/features/evaluation/evalService.ts)\n- [worker/src/features/batchAction/handleBatchActionJob.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/features/batchAction/handleBatchActionJob.ts)\n\n
\n\n# API Layer\n\nThe API Layer is the central communication bridge between the Langfuse frontend and backend services. Built on **tRPC** (TypeScript RPC), it provides end-to-end type safety, enabling the web application to interact with server-side logic through strongly-typed procedure calls.\n\n## Overview\n\nThe API Layer serves multiple critical functions:\n\n- **Type-Safe Communication**: All API calls are fully typed from server to client\n- **Authentication & Authorization**: Every procedure is wrapped with auth middleware\n- **Business Logic Isolation**: Procedures delegate to repository layer for data access\n- **Input Validation**: Zod schemas validate all incoming requests\n- **Feature Organization**: Procedures are grouped by domain (traces, observations, scores)\n\n```mermaid\ngraph TD\n subgraph Frontend\n UI[React Components]\n end\n \n subgraph API Layer\n TRPC[tRPC Client]\n Procedures[tRPC Procedures]\n Middleware[Auth Middleware]\n end\n \n subgraph Backend\n Repositories[Repositories]\n Database[(Database)]\n end\n \n UI --> TRPC\n TRPC --> Procedures\n Procedures --> Middleware\n Middleware --> Repositories\n Repositories --> Database\n```\n\n## Architecture\n\n### Core Components\n\n| Component | File | Purpose |\n|-----------|------|---------|\n| tRPC Instance | `trpc.ts` | Initialize tRPC with middleware and context |\n| Root Router | `root.ts` | Register all feature routers |\n| Trace Router | `routers/traces.ts` | Trace CRUD and query operations |\n| Observation Router | `routers/observations.ts` | Span/generation/event operations |\n| Score Router | `routers/scores.ts` | Score management and analytics |\n| Score Analytics | `features/score-analytics/server/scoreAnalyticsRouter.ts` | Score aggregation and statistics |\n\n### Router Registration Flow\n\nThe root router aggregates all feature routers under a namespace:\n\n```typescript\n// Simplified from web/src/server/api/root.ts\nexport const rootRouter = createTRPCRouter({\n trace: traceRouter,\n observation: observationRouter,\n score: scoreRouter,\n scoreAnalytics: scoreAnalyticsRouter,\n // ... other routers\n});\n```\n\n资料来源：[web/src/server/api/root.ts:1-50]()\n\n## tRPC Configuration\n\n### Initialization\n\nThe tRPC instance is initialized in `trpc.ts` with:\n\n1. **Context Creation**: Builds request-scoped context with authentication\n2. **Middleware Chain**: Applies auth, rate limiting, and logging\n3. **Error Handling**: Transforms errors into HTTP-compatible responses\n\n```typescript\n// From web/src/server/api/trpc.ts\nexport const createTRPCContext = async (opts: CreateNextContextOptions) => {\n return {\n session: await getServerSession(authOptions),\n // ... additional context\n };\n};\n\nconst t = initTRPC.context相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [web/src/server/api/root.ts](https://github.com/langfuse/langfuse/blob/main/web/src/server/api/root.ts)\n- [web/src/server/api/routers/traces.ts](https://github.com/langfuse/langfuse/blob/main/web/src/server/api/routers/traces.ts)\n- [web/src/server/api/routers/scores.ts](https://github.com/langfuse/langfuse/blob/main/web/src/server/api/routers/scores.ts)\n- [web/src/server/api/routers/observations.ts](https://github.com/langfuse/langfuse/blob/main/web/src/server/api/routers/observations.ts)\n- [web/src/server/api/trpc.ts](https://github.com/langfuse/langfuse/blob/main/web/src/server/api/trpc.ts)\n- [packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n- [web/src/features/score-analytics/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/score-analytics/README.md)\n- [web/src/features/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/README.md)\n\n
\n\n# Project Introduction\n\nLangfuse is an open-source **observability and analytics platform** designed for Large Language Model (LLM) applications. It provides comprehensive tracing, evaluation, and prompt management capabilities that enable developers to monitor, debug, and optimize their AI-powered applications.\n\n## Overview\n\nLangfuse serves as a centralized platform for capturing and analyzing interactions between AI models and end users. The project is MIT licensed and supports both cloud-hosted and self-hosted deployment models, making it accessible for teams of various sizes and requirements.\n\n### Core Purpose\n\nThe platform addresses several critical needs in LLM application development:\n\n- **Observability**: Track and visualize traces, observations, and model interactions in real-time\n- **Evaluation**: Measure and analyze AI application performance through configurable scoring systems\n- **Prompt Management**: Create, version, and manage prompts with support for complex dependency resolution\n- **Collaboration**: Enable team collaboration through commenting and sharing features\n- **Analytics**: Provide insights into AI application behavior through comprehensive analytics dashboards\n\n### High-Level Architecture\n\nLangfuse follows a modern microservices-inspired architecture with clear separation between frontend, backend processing, and data storage components.\n\n```mermaid\ngraph TD\n subgraph Frontend[\"Frontend (Next.js/React)\"]\n UI[User Interface]\n DesignSystem[Design System Components]\n FeatureFlags[Feature Flags]\n end\n \n subgraph Backend[\"Backend Services\"]\n API[API Server]\n MCP[MCP Server]\n Worker[Worker/Queue Processing]\n end\n \n subgraph Storage[\"Data Layer\"]\n Postgres[(PostgreSQL)]\n ClickHouse[(ClickHouse)]\n Redis[(Redis)]\n S3[(S3 Storage)]\n end\n \n UI --> API\n MCP --> API\n Worker --> API\n API --> Postgres\n API --> ClickHouse\n API --> Redis\n API --> S3\n```\n\n## Technology Stack\n\nLangfuse is built using a modern technology stack optimized for performance and developer experience.\n\n### Frontend Stack\n\n| Component | Technology | Purpose |\n|-----------|------------|---------|\n| Framework | Next.js | Server-side rendering and routing |\n| UI Library | React | Component-based UI development |\n| State Management | React Context + Hooks | Local and global state |\n| Virtualization | TanStack Virtual | Efficient rendering of large lists |\n| Styling | Tailwind CSS + cva | Utility-first styling with variant handling |\n| Forms | Zod | Schema validation |\n| Data Fetching | tRPC | Type-safe API communication |\n\n资料来源：[web/src/components/design-system/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/design-system/README.md)\n\n### Backend Stack\n\n| Component | Technology | Purpose |\n|-----------|------------|---------|\n| Runtime | Node.js/TypeScript | Server-side logic |\n| Database | PostgreSQL | Primary data storage |\n| Analytics | ClickHouse | High-performance analytics queries |\n| Cache | Redis | Caching and queue management |\n| Queue | BullMQ | Background job processing |\n| Storage | S3-compatible | File and event storage |\n\n### Key Frontend Components\n\nThe frontend architecture is organized around several key systems:\n\n#### Design System\n\nThe design system (`web/src/components/design-system/`) provides reusable, primitive UI components following strict principles:\n\n- **Presentational only**: No business logic in components\n- **Explicit, typed APIs**: Strict TypeScript definitions\n- **No className/style props**: Prevents style leakage\n- **cva for variants**: Consistent variant handling\n\n```mermaid\ngraph LR\n A[Design System] --> B[Button]\n A --> C[Input]\n A --> D[Modal]\n B --> E[Consistent Styling]\n C --> E\n D --> E\n```\n\n资料来源：[web/src/components/design-system/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/design-system/README.md)\n\n#### Layout System\n\nAll pages use a standardized `Page` wrapper component that ensures:\n\n- Consistent layout structure\n- Sticky header behavior\n- Proper scroll management (`\"content-scroll\"` or `\"page-scroll\"`)\n- Breadcrumb navigation support\n- Custom header actions\n\n```mermaid\ngraph TD\n Page[Page Component] --> Header[Sticky Header]\n Page --> Content[Scrollable Content]\n Header --> Breadcrumb[Breadcrumb Navigation]\n Header --> Actions[Action Buttons]\n```\n\n资料来源：[web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n\n#### JSON Viewer Component\n\nThe `AdvancedJsonViewer` component provides efficient rendering of large JSON datasets:\n\n- **Virtualization**: Uses TanStack Virtual for row-based rendering\n- **Iterative algorithms**: Explicit stack-based iteration to prevent stack overflow\n- **Client-side search**: In-memory matching with binary search navigation\n- **Theme support**: Customizable JSON syntax highlighting\n\n```mermaid\ngraph TD\n Input[Large JSON Data] --> Parser[JSON Parser]\n Parser --> TreeBuilder[Tree Builder]\n TreeBuilder --> Virtualizer[TanStack Virtual]\n Virtualizer --> Renderer[Row Renderer]\n \n Search[Search Query] --> Matcher[In-Memory Matcher]\n Matcher --> Navigator[Binary Search Navigator]\n```\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/README.md)\n\n## Core Features\n\n### Tracing and Observability\n\nLangfuse provides comprehensive tracing capabilities that capture the full lifecycle of AI interactions:\n\n- **Traces**: Complete request/response cycles\n- **Observations**: Individual components within a trace (spans, events, generations)\n- **Metadata**: Custom metadata attachment for context\n- **Tree Structure**: Hierarchical representation of nested observations\n\nThe tree-building system uses iterative algorithms to handle millions of observations without stack overflow:\n\n```typescript\n// Iterative traversal pattern\nfunction traverse(rootNode: TreeNode) {\n const stack = [rootNode];\n while (stack.length > 0) {\n const node = stack.pop()!;\n process(node);\n node.children.forEach((child) => stack.push(child));\n }\n}\n```\n\n资料来源：[web/src/components/trace/lib/tree-building.clienttest.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace/lib/tree-building.clienttest.ts)\n\n### Prompt Management\n\nLangfuse supports sophisticated prompt management with dependency resolution:\n\n- **Prompt Stacking**: Compose prompts from multiple sources\n- **Dependency Tags**: Reference other prompts using `@@@langfusePrompt:...@@@` syntax\n- **Resolution Modes**:\n - `getPromptResolved`: Returns fully resolved prompt with dependencies inlined\n - `getPromptUnresolved`: Returns raw prompt with tags preserved for analysis\n\n```mermaid\ngraph LR\n A[Prompt A] -->|references| B[Prompt B]\n A -->|references| C[Prompt C]\n B -->|references| D[Prompt D]\n A -->|resolved| E[Final Prompt]\n```\n\n资料来源：[web/src/features/mcp/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/mcp/README.md)\n\n### Score Analytics\n\nThe scoring system enables quantitative evaluation of AI application performance:\n\n- **Multiple Score Types**: Supports numeric, categorical, and boolean scores\n- **Time Series Analysis**: Track score changes over configurable intervals\n- **Distribution Analysis**: Visualize score distributions with bins and categories\n- **Comparison Mode**: Compare two scores side-by-side\n\nThe analytics layer provides interpretive functions for common metrics:\n\n| Metric | Interpretation | Threshold |\n|--------|----------------|-----------|\n| Agreement (Cohen's Kappa) | Excellent | ≥ 0.9 |\n| Agreement (Cohen's Kappa) | Good | ≥ 0.8 |\n| Agreement (Cohen's Kappa) | Fair | ≥ 0.6 |\n| Agreement (Cohen's Kappa) | Poor | ≥ 0.4 |\n\n资料来源：[web/src/features/score-analytics/lib/statistics-utils.ts](https://github.com/langfuse/langfuse/blob/main/web/src/features/score-analytics/lib/statistics-utils.ts)\n\n### Entitlements System\n\nAccess control is managed through a hierarchical entitlements system:\n\n```mermaid\ngraph TD\n Plan[Plan] -->|contains| Entitlements[Entitlements]\n Plan -->|contains| Limits[Entitlement Limits]\n \n Entitlements -->|grants| Features[Feature Access]\n Limits -->|restricts| Resources[Resource Quotas]\n \n PlanTypes[Plan Types] --> OSS[OSS]\n PlanTypes --> CloudPro[Cloud Pro]\n PlanTypes --> SelfHostedEnterprise[Self-Hosted Enterprise]\n```\n\nAvailable entitlements include:\n\n- **Feature Flags**: Enable/disable features via `useIsFeatureEnabled` hook\n- **Entitlement Limits**: Quotas on resources (e.g., annotation queue count)\n- **Plan-based Access**: Cloud and self-hosted enterprise plans\n\n资料来源：[web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n\n### Feature Flags\n\nFeature flags control feature availability dynamically:\n\n```typescript\nconst isFeatureEnabled = useIsFeatureEnabled(\"feature-flag-name\");\n```\n\nA feature flag is enabled when:\n\n1. Flag is in user's `feature_flags` list\n2. `LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES` environment variable is set\n3. User has admin privileges\n\n资料来源：[web/src/features/feature-flags/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/feature-flags/README.md)\n\n### Collaboration Features\n\nLangfuse includes team collaboration capabilities:\n\n- **Mention Parser**: Extract and resolve user mentions in comments\n- **User References**: Syntax `@[Display Name](user:userId)` for linking users\n- **Sanitization**: Clean user-generated content for safe display\n\n```typescript\n// Mention format: @[Alice](user:alice123)\n// Parser extracts: alice123\n```\n\n资料来源：[web/src/features/comments/lib/mentionParser.clienttest.ts](https://github.com/langfuse/langfuse/blob/main/web/src/features/comments/lib/mentionParser.clienttest.ts)\n\n## Data Flow Architecture\n\n### Ingestion Pipeline\n\nEvents flow through the system in a structured pipeline:\n\n```mermaid\ngraph LR\n S3[S3 Event Storage] --> Worker[Worker Processing]\n Worker -->|Standard| IngestionQueue[IngestionSecondaryQueue]\n Worker -->|OTEL| OtelQueue[OtelIngestionQueue]\n IngestionQueue --> Postgres[(PostgreSQL)]\n OtelQueue --> ClickHouse[(ClickHouse)]\n```\n\nEvent processing includes:\n\n- **Checkpointing**: Resume from failures using `.checkpoint` files\n- **Rate Limiting**: Client-side and server-side throttling\n- **Retry Logic**: Exponential backoff with jitter for transient failures\n- **Error Logging**: Failed events appended to `errors.csv`\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n### API Architecture\n\nLangfuse uses tRPC for type-safe API communication:\n\n- **Server-side validation**: Zod schemas for input validation\n- **tRPC routers**: Organized endpoint handlers\n- **API versioning**: V1 (legacy) and V2 (current) API support\n\n| API Version | GET Support | Notes |\n|-------------|-------------|-------|\n| V1 | `traceId` required | Legacy, trace-focused |\n| V2 | `traceId` optional | Adds `sessionId` support |\n\n资料来源：[packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n\n### MCP Server Architecture\n\nThe Model Context Protocol (MCP) server provides external access to Langfuse:\n\n- **Stateless per-request**: Fresh server instance for each request\n- **Context via closures**: Authentication captured in handler closures\n- **No session storage**: Request-disposable architecture\n\n```mermaid\ngraph TD\n Request[MCP Request] --> Instance[New Server Instance]\n Instance --> Auth[Auth Context Closure]\n Auth --> Handler[Request Handler]\n Handler --> Response[Response]\n Response --> Discard[Instance Discarded]\n```\n\n资料来源：[web/src/features/mcp/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/mcp/README.md)\n\n## Filtering System\n\nLangfuse implements a sophisticated filtering system with type-safe encoding:\n\n### Filter Types\n\n| Type | Description | Example |\n|------|-------------|---------|\n| `string` | Simple string matching | Trace name |\n| `number` | Numeric comparison | Latency values |\n| `datetime` | Date/time filtering | Time ranges |\n| `boolean` | True/false matching | Flag states |\n| `arrayOptions` | Multi-value selection | Tags |\n| `categoryOptions` | Categorical filtering | Status values |\n| `positionInTrace` | Nested location | Span hierarchy |\n\n### State Management\n\nFilters support multiple storage locations:\n\n- **URL**: Persisted in query parameters\n- **Session Storage**: In-memory per session\n- **Peek Context**: Temporary state for preview panels\n\n```typescript\nconst filterOptions: UseSidebarFilterStateOptions = {\n stateLocation: \"urlAndSessionStorage\",\n sessionFilterContextId: projectId,\n implicitDefaultConfig: DEFAULT_SIDEBAR_IMPLICIT_ENVIRONMENT_CONFIG,\n};\n```\n\n资料来源：[web/src/components/table/peek/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/table/peek/README.md)\n\n## Deployment Models\n\nLangfuse supports multiple deployment configurations:\n\n| Model | Plan Options | Authentication | License |\n|-------|--------------|----------------|---------|\n| Cloud | `cloud:pro` | JWT via NextAuth | Proprietary |\n| Self-Hosted | `self-hosted:enterprise` | JWT + License Key | Proprietary |\n| Open Source | `oss` | Basic auth | MIT |\n\n### Self-Hosted Configuration\n\nSelf-hosted deployments require:\n\n- PostgreSQL database\n- ClickHouse for analytics\n- Redis for caching/queues\n- S3-compatible storage for events\n- License key for enterprise features\n\n资料来源：[web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n\n## Performance Considerations\n\n### Large Dataset Handling\n\nLangfuse handles large datasets through several mechanisms:\n\n| Scale | Mechanism | Threshold |\n|-------|-----------|-----------|\n| 10k+ rows | TanStack Virtual | Row-based rendering |\n| 1M+ nodes | Iterative algorithms | No stack overflow |\n| 10k+ search matches | Binary search | Efficient navigation |\n\n### Known Limitations\n\n- No horizontal virtualization for wide rows\n- Client-side search only (can be slow with many matches)\n- Memory constraints at 1M+ nodes\n- Read-only JSON viewer (no inline editing)\n- Wrap mode may cause layout thrashing\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/README.md)\n\n## Development Guidelines\n\n### Testing\n\nThe project uses Jest with custom extensions:\n\n| File Pattern | Purpose | Location |\n|--------------|---------|----------|\n| `.clienttest.ts` | Client-side tests | Colocated with components |\n| `*.test.ts` | Standard unit tests | `__tests__` or inline |\n\n```bash\n# Run client tests\npnpm --filter=web run test-client --testPathPattern=\"ComponentName\"\n```\n\n### Debugging\n\nEnable debug logging via localStorage:\n\n```typescript\nlocalStorage.setItem(\"debug:ComponentName\", \"true\");\n```\n\n## Summary\n\nLangfuse is a comprehensive observability platform that bridges the gap between AI application development and operational monitoring. Its modular architecture, built on proven technologies like PostgreSQL, ClickHouse, and React, provides a scalable foundation for teams to understand, evaluate, and optimize their LLM applications.\n\nThe platform's emphasis on type safety, performance optimization, and developer experience makes it suitable for both small development teams and large-scale enterprise deployments.\n\n---\n\n\n\n## Project Structure\n\n### 相关页面\n\n相关主题：[Monorepo Configuration](#monorepo-structure), [System Architecture](#system-architecture)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [README.md](https://github.com/langfuse/langfuse/blob/main/README.md)\n- [packages/shared/src/constants/VERSION.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/constants/VERSION.ts)\n- [web/src/components/ui/AdvancedJsonViewer/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/README.md)\n- [web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n- [web/src/features/mcp/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/mcp/README.md)\n- [web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n- [web/src/features/feature-flags/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/feature-flags/README.md)\n- [web/src/components/design-system/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/design-system/README.md)\n\n
\n\n# Project Structure\n\n## Overview\n\nLangfuse is a comprehensive observability and analytics platform for LLM applications, structured as a **monorepo** using pnpm workspaces and Turborepo for build orchestration. The repository contains multiple packages including the frontend web application, backend worker services, and shared libraries.\n\n## Monorepo Architecture\n\n### Workspace Configuration\n\nLangfuse uses pnpm workspaces defined in `pnpm-workspace.yaml` to manage multiple packages within a single repository.\n\n```yaml\npackages:\n - \"packages/*\"\n - \"web\"\n - \"worker\"\n```\n\n资料来源：[pnpm-workspace.yaml](https://github.com/langfuse/langfuse/blob/main/pnpm-workspace.yaml)\n\n### Build System (Turborepo)\n\nThe project uses Turborepo for efficient incremental builds and task caching. The turbo configuration defines the build pipeline and dependencies between packages.\n\n资料来源：[turbo.json](https://github.com/langfuse/langfuse/blob/main/turbo.json)\n\n## Package Structure\n\n### 1. Web Application (`/web`)\n\nThe frontend application built with Next.js and React, containing the user interface and client-side logic.\n\n| Directory | Purpose |\n|-----------|---------|\n| `src/components/` | Reusable UI components including layouts, tables, and specialized viewers |\n| `src/features/` | Feature-specific modules with their own logic and components |\n| `src/lib/` | Utility functions and helpers |\n| `src/hooks/` | Custom React hooks |\n\n资料来源：[web/package.json](https://github.com/langfuse/langfuse/blob/main/web/package.json)\n\n### 2. Worker Service (`/worker`)\n\nBackend service handling background processing, event ingestion, and queue management.\n\n| Directory | Purpose |\n|-----------|---------|\n| `src/scripts/` | Utility scripts for data operations and migrations |\n| `src/queues/` | Queue handlers for async processing |\n\n资料来源：[worker/package.json](https://github.com/langfuse/langfuse/blob/main/worker/package.json)\n\n### 3. Shared Packages (`/packages/shared`)\n\nCommon utilities, types, and validation schemas shared across web and worker packages.\n\n| Module | Purpose |\n|--------|---------|\n| Validation | Zod schemas for type-safe data validation |\n| Types | Shared TypeScript type definitions |\n| Utilities | Common helper functions |\n\n资料来源：[packages/shared/package.json](https://github.com/langfuse/langfuse/blob/main/packages/shared/package.json)\n\n## Web Application Structure\n\n### Component Architecture\n\n```mermaid\ngraph TD\n A[Web Application] --> B[Layout Components]\n A --> C[UI Components]\n A --> D[Feature Modules]\n \n B --> B1[Page Wrapper]\n B --> B2[ContainerPage]\n B --> B3[Breadcrumb]\n \n C --> C1[AdvancedJsonViewer]\n C --> C2[Table Components]\n C --> C3[Peek Components]\n \n D --> D1[MCP]\n D --> D2[Score Analytics]\n D --> D3[Comments]\n D --> D4[Filters]\n D --> D5[Entitlements]\n```\n\n### Layout System\n\nThe `Page` component is the standard wrapper for all pages, providing:\n\n- **Sticky Header**: Consistent header across pages\n- **Scroll Management**: Supports `\"content-scroll\"` and `\"page-scroll\"` modes\n- **Breadcrumb Navigation**: Easy navigation path display\n- **Custom Header Actions**: Flexible button/link placement\n\nFor content that doesn't scale well with page width (e.g., settings pages), use `ContainerPage` instead.\n\n资料来源：[web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n\n### Key Feature Modules\n\n#### AdvancedJsonViewer\n\nA virtualized JSON tree viewer with the following characteristics:\n\n- **Performance**: Uses TanStack Virtual for rendering large JSON structures\n- **Search**: Client-side search with regex support\n- **Theme Support**: Multiple color themes (GitHub, Monokai, Solarized)\n- **Tree Navigation**: Binary search for efficient node access\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/README.md)\n\n#### Score Analytics\n\nProvides analytics dashboard capabilities for score data:\n\n- **Score Comparison**: Compare two scores over time\n- **Distribution Analysis**: Histogram and heatmap visualizations\n- **Time Series**: Temporal trend analysis with configurable intervals\n- **Data Transformation**: Pure functions for data processing\n\n资料来源：[web/src/features/score-analytics/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/score-analytics/README.md)\n\n#### MCP (Model Context Protocol)\n\nEnables integration with external systems through MCP:\n\n- **Stateless Architecture**: Fresh server instance per request\n- **Prompt Management**: Support for `getPrompt` and `getPromptUnresolved`\n- **Resource Handling**: MCP resources and tool support\n\n资料来源：[web/src/features/mcp/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/mcp/README.md)\n\n#### Entitlements\n\nFeature availability control system:\n\n- **Plan-based Access**: `oss`, `cloud:pro`, `self-hosted:enterprise`\n- **Entitlement Limits**: Resource quotas per plan\n- **Server/Client Support**: Available in both frontend hooks and backend\n\n资料来源：[web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n\n### Table and Peek System\n\n#### PeekTableStateProvider\n\nManages table state for peek views (slide-over panels showing item details):\n\n```mermaid\ngraph LR\n A[PeekTableStateProvider] --> B[Filters]\n A --> C[Sorting]\n A --> D[Pagination]\n A --> E[Search]\n```\n\n**State Persistence**: Filter, sort, and pagination state persists across K/J navigation between items of the same type.\n\n**State Reset**: State clears when the peek view closes (via X button, Escape, or click outside).\n\n资料来源：[web/src/components/table/peek/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/table/peek/README.md)\n\n## Worker Service Structure\n\n### Scripts\n\nThe worker contains utility scripts for data operations:\n\n| Script | Purpose |\n|--------|---------|\n| `replayIngestionEventsV2` | Replay events from CSV to ingestion queues |\n| `refillQueueEvent` | Backfill queue with events from local files |\n\n#### Replay Ingestion Events V2\n\nReplays S3-stored events back to Langfuse:\n\n- **Batch Processing**: Processes events in configurable batches\n- **Checkpoint Support**: Resume capability via checkpoint files\n- **Rate Limiting**: Respects server-side rate limits with exponential backoff\n- **Error Handling**: Retries transient failures, logs permanent failures\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n#### Refill Queue Event\n\nBackfills queues with events from local JSONL files:\n\n1. Create `./worker/events.jsonl` with one JSON event per line\n2. Configure Redis connection and supporting services\n3. Run via `pnpm run --filter=worker refill-queue-event`\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/refillQueueEvent/README.md)\n\n## Public API Architecture\n\n### Adding New API Routes\n\nThe project follows a structured pattern for public API development:\n\n1. **Implementation**: Wrap routes with `withMiddleware` and `createAuthedAPIRoute`\n2. **Type Definition**: Add Zod types to `/features/public-api/types` using `coerce` for primitives\n3. **Validation**: Use `validateZodSchema` for response validation\n4. **Documentation**: Add to Fern with `docs` attributes\n5. **SDK Updates**: Copy types to Python and JS SDKs\n\n```typescript\n// Response type example\nconst responseSchema = z.object({\n data: z.string(),\n timestamp: z.coerce.date(),\n}).strict();\n```\n\n资料来源：[web/src/features/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/README.md)\n\n## Testing Infrastructure\n\n### Client-Side Testing\n\nTests use `.clienttest.ts` extension and are colocated with components:\n\n```bash\npnpm --filter=web run test-client --testPathPattern=\"ComponentName\"\n```\n\nExample: `AdvancedJsonViewer` tests cover tree building, navigation, expansion, and search operations.\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/ui/AdvancedJsonViewer/README.md)\n\n### Trace Tree Building Tests\n\nPerformance tests for large observation sets:\n\n| Scale | Threshold | Structure Types |\n|-------|-----------|-----------------|\n| 10k | 500ms | flat, deep, balanced, realistic |\n| 25k | 2s | flat, realistic |\n| 50k | 5s | flat, realistic |\n| 500k | 60s | realistic |\n\n资料来源：[web/src/components/trace/lib/tree-building.clienttest.ts](https://github.com/langfuse/langfuse/blob/main/web/src/components/trace/lib/tree-building.clienttest.ts)\n\n## Development Workflow\n\n### Component Development Guidelines\n\n1. **Use Page Wrapper**: Always wrap pages with `相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [pnpm-workspace.yaml](https://github.com/langfuse/langfuse/blob/main/pnpm-workspace.yaml)\n- [turbo.json](https://github.com/langfuse/langfuse/blob/main/turbo.json)\n- [packages/shared/package.json](https://github.com/langfuse/langfuse/blob/main/packages/shared/package.json)\n- [web/package.json](https://github.com/langfuse/langfuse/blob/main/web/package.json)\n- [worker/package.json](https://github.com/langfuse/langfuse/blob/main/worker/package.json)\n\n
\n\n# Quickstart Guide\n\nLangfuse is an open-source LLM engineering platform that provides observability, analytics, and prompt management for LLM applications. This guide walks you through setting up a local development environment, understanding the core architecture, and deploying Langfuse for production use.\n\n## Overview\n\nLangfuse supports multiple deployment scenarios:\n\n| Deployment Type | Use Case | Key Components |\n|----------------|----------|----------------|\n| Local Development | Testing and development | Docker Compose with all services |\n| Self-Hosted | Production deployment | Docker/Kubernetes with externalized services |\n| Cloud | Managed SaaS offering | Langfuse-hosted infrastructure |\n\n资料来源：[README.md:1-20]()\n\n## Architecture Overview\n\nLangfuse consists of three main components:\n\n```mermaid\ngraph TD\n A[Web UI] --> B[Server API]\n B --> C[(PostgreSQL)]\n B --> D[(ClickHouse)]\n B --> E[(Redis)]\n F[Workers] --> C\n F --> D\n F --> E\n G[S3 Storage] --> F\n```\n\n### Core Components\n\n| Component | Purpose | Technology |\n|-----------|---------|------------|\n| `web/` | Frontend UI application | Next.js, React, TanStack |\n| `server/` | API server and business logic | Node.js, tRPC, Prisma |\n| `worker/` | Background job processing | BullMQ, Redis |\n| `clickhouse/` | Analytics storage | ClickHouse |\n\n资料来源：[docker-compose.yml:1-50]()\n\n## Local Development Setup\n\n### Prerequisites\n\n- Node.js 20+ (using pnpm as package manager)\n- Docker and Docker Compose\n- Git\n\n### 1. Clone the Repository\n\n```bash\ngit clone https://github.com/langfuse/langfuse.git\ncd langfuse\n```\n\n### 2. Environment Configuration\n\nCreate a `.env` file in the root directory with the following required variables:\n\n```bash\n# Database\nDATABASE_URL=postgresql://langfuse:langfuse@localhost:5432/langfuse\n\n# ClickHouse\nCLICKHOUSE_URL=http://localhost:8123\nCLICKHOUSE_USER=langfuse\nCLICKHOUSE_PASSWORD=langfuse\n\n# Redis\nREDIS_URL=redis://localhost:6379\n\n# Auth (NextAuth.js)\nNEXTAUTH_SECRET=your-secret-key\nNEXTAUTH_URL=http://localhost:3000\n\n# S3 Storage (MinIO for local dev)\nS3_ACCESS_KEY=langfuse\nS3_SECRET_KEY=langfuse\nS3_REGION=us-east-1\nS3_ENDPOINT_URL=http://localhost:9000\nS3_EVENT_UPLOAD_BUCKET=langfuse\n```\n\n资料来源：[docker-compose.yml:50-120]()\n\n### 3. Start Infrastructure Services\n\nLaunch all supporting services using Docker Compose:\n\n```bash\ndocker compose up -d\n```\n\nThis starts the following services:\n\n| Service | Port | Purpose |\n|---------|------|---------|\n| postgres | 5432 | Primary database |\n| clickhouse | 8123 | Analytics storage |\n| redis | 6379 | Job queue broker |\n| minio | 9000/9001 | S3-compatible storage |\n\n资料来源：[docker-compose.yml:120-180]()\n\n### 4. Install Dependencies\n\n```bash\npnpm install\n```\n\n### 5. Run Database Migrations\n\n```bash\npnpm db:migrate\n```\n\n### 6. Start Development Servers\n\nLangfuse uses a monorepo structure with multiple development servers:\n\n```bash\n# Start all services in development mode\npnpm run dev\n\n# Or start individual services\npnpm --filter=server run dev # API server\npnpm --filter=web run dev # Web UI\npnpm --filter=worker run dev # Background workers\n```\n\n资料来源：[CONTRIBUTING.md:50-100]()\n\n## Project Structure\n\n```\nlangfuse/\n├── web/ # Next.js frontend\n│ ├── src/\n│ │ ├── components/ # Reusable UI components\n│ │ ├── features/ # Feature modules\n│ │ ├── pages/ # Next.js pages\n│ │ └── lib/ # Utilities\n│ └── public/ # Static assets\n├── server/ # API server\n│ ├── src/\n│ │ ├── api/ # tRPC routers\n│ │ ├── services/ # Business logic\n│ │ └── lib/ # Utilities\n├── worker/ # Background workers\n│ └── src/\n│ ├── workers/ # Queue processors\n│ └── scripts/ # Utility scripts\n├── clickhouse/ # ClickHouse migrations\n└── docker-compose.yml # Local infrastructure\n```\n\n### Page Component Pattern\n\nThe `Page` component is the standard wrapper for all pages in the application:\n\n```tsx\nimport Page from \"@/src/components/layouts/Page\";\n\nexport default function MyPage() {\n return (\n 相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [README.md](https://github.com/langfuse/langfuse/blob/main/README.md)\n- [docker-compose.yml](https://github.com/langfuse/langfuse/blob/main/docker-compose.yml)\n- [CONTRIBUTING.md](https://github.com/langfuse/langfuse/blob/main/CONTRIBUTING.md)\n- [web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n- [worker/src/scripts/refillQueueEvent/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/refillQueueEvent/README.md)\n- [web/src/features/slack/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/slack/README.md)\nContent here...
\n \n
\n\n# System Architecture\n\nLangfuse is a comprehensive observability and analytics platform designed for Large Language Model (LLM) applications. The system architecture is built on a modern, modular design that separates concerns across frontend, backend worker services, and shared infrastructure layers.\n\n## Overview\n\nLangfuse follows a distributed architecture pattern with the following primary components:\n\n| Layer | Technology Stack | Purpose |\n|-------|-----------------|---------|\n| Frontend | Next.js, React, TypeScript | User interface and visualization |\n| Backend Worker | Node.js, BullMQ, TypeScript | Event processing and queue management |\n| Shared Packages | TypeScript | Common utilities, types, and infrastructure clients |\n| Database | PostgreSQL | Primary data storage |\n| Cache/Queue | Redis | Queue management and caching |\n| Analytics | ClickHouse | High-performance analytics queries |\n| Observability | OpenTelemetry | Distributed tracing |\n\n## High-Level Architecture\n\n```mermaid\ngraph TD\n subgraph Client[\"Frontend (Next.js)\"]\n UI[User Interface]\n Pages[Page Components]\n DesignSystem[Design System]\n end\n \n subgraph Shared[\"Shared Packages\"]\n DB[(PostgreSQL)]\n Redis[(Redis)]\n ClickHouse[(ClickHouse)]\n Otel[OpenTelemetry]\n end\n \n subgraph Backend[\"Worker Service\"]\n Queues[Queue Workers]\n Scripts[Utility Scripts]\n end\n \n Client <-->|tRPC API| Shared\n Backend <-->|Event Processing| Shared\n Client -->|Ingestion| Backend\n```\n\n## Infrastructure Layer\n\n### Database Connection\n\nThe PostgreSQL database is the central data store for Langfuse, managed through a shared database client module. The connection is centralized in the `packages/shared/src/db.ts` module, which provides a unified interface for all database operations across the application.\n\n资料来源：[packages/shared/src/db.ts:1-50]()\n\n### Redis Client\n\nRedis serves dual purposes in the Langfuse architecture:\n\n1. **Queue Management**: BullMQ queues for asynchronous event processing\n2. **Caching**: Session and temporary data caching\n\nThe Redis client is configured in `packages/shared/src/server/redis/redis.ts` and is shared across worker services.\n\n资料来源：[packages/shared/src/server/redis/redis.ts:1-30]()\n\n### ClickHouse Integration\n\nClickHouse provides the analytical query engine for high-performance aggregations. The client is initialized in `packages/shared/src/server/clickhouse/client.ts` and is primarily used for:\n\n- Score analytics aggregations\n- Time-series data analysis\n- Large dataset transformations\n\n资料来源：[packages/shared/src/server/clickhouse/client.ts:1-40]()\n\n### OpenTelemetry\n\nThe OpenTelemetry integration (`packages/shared/src/server/otel/index.ts`) provides distributed tracing across all services. This enables:\n\n- Request tracing across frontend and backend\n- Event processing workflow visibility\n- Performance monitoring\n\n资料来源：[packages/shared/src/server/otel/index.ts:1-60]()\n\n## Frontend Architecture\n\n### Page Structure\n\nAll frontend pages use a standardized layout system defined in `web/src/components/layouts/`. The `Page` component is the required wrapper for all application pages, ensuring consistent layout behavior.\n\nKey layout patterns:\n\n| Pattern | Component | Use Case |\n|---------|-----------|----------|\n| Standard Pages | `Page` | Most application pages |\n| Wide Content | `ContainerPage` | Settings, setup pages with wide content |\n\nThe page wrapper provides:\n- Sticky header management\n- Scroll behavior control (`content-scroll` or `page-scroll`)\n- Breadcrumb navigation\n- Custom header actions\n\n资料来源：[web/src/components/layouts/README.md:1-60]()\n\n### Design System\n\nThe design system (`web/src/components/design-system/`) provides primitive, reusable UI components following strict architectural principles:\n\n**Principles:**\n- Presentational only (no business logic)\n- Explicit, strictly typed APIs\n- Props over context (no React Context)\n\n**Component Structure:**\n```\ndesign-system/\n Button/\n Button.tsx\n Button.stories.tsx\n```\n\n**Styling Rules:**\n- No arbitrary CSS values\n- Explicit enums for variants (`size: \"sm\" | \"md\" | \"lg\"`)\n- CVA (Class Variance Authority) for variant management\n- Boolean props use positive naming (`isLoading`, `shouldTruncate`)\n\n资料来源：[web/src/components/design-system/README.md:1-80]()\n\n### State Management Patterns\n\nLangfuse uses a sophisticated state management approach with the Peek Table State system:\n\n```mermaid\ngraph TD\n A[Page Load] --> B{Peek Context?}\n B -->|Yes| C[PeekTableStateProvider]\n B -->|No| D[URL/Session State]\n C --> E[Table State Preserved]\n D --> F[Standard State]\n \n E --> G[K/J Navigation]\n G --> H[State Retained ✓]\n```\n\nThe `PeekTableStateProvider` maintains table state (filters, sorting, pagination) across K/J keyboard navigation between items of the same type. State resets only when the peek view closes.\n\n资料来源：[web/src/components/table/peek/README.md:1-100]()\n\n## Feature Modules\n\n### Entitlements System\n\nThe entitlements feature controls feature availability at the organization level:\n\n| Concept | Definition |\n|---------|------------|\n| Plan | Feature tier (OSS, cloud:pro, self-hosted:enterprise) |\n| Entitlement | Available feature (e.g., playground, score analytics) |\n| EntitlementLimit | Resource limits (e.g., annotation-queue-count) |\n\n**Plan Resolution:**\n- Cloud: Added to organization via JWT from NextAuth\n- Self-hosted: From license key or environment configuration\n\n资料来源：[web/src/features/entitlements/README.md:1-50]()\n\n### Score Analytics\n\nThe score analytics feature provides comprehensive statistical analysis of evaluation scores:\n\n**Architecture Components:**\n\n| Component | Location | Responsibility |\n|-----------|----------|----------------|\n| Provider | `ScoreAnalyticsProvider.tsx` | Context management |\n| Hook | `useScoreAnalyticsQuery` | Data fetching and transformation |\n| Transformers | `scoreAnalyticsTransformers.ts` | Data transformation pipeline |\n| Router | `scoreAnalyticsRouter.ts` | tRPC API endpoint |\n\n**Data Flow:**\n```mermaid\ngraph LR\n A[API Request] --> B[tRPC Router]\n B --> C[ClickHouse Query]\n C --> D[Transformers]\n D --> E[ScoreAnalyticsProvider]\n E --> F[Chart Components]\n```\n\n资料来源：[packages/shared/src/features/scores/interfaces/README.md:1-80]()\n资料来源：[web/src/features/score-analytics/README.md:1-100]()\n\n### Score Interfaces Architecture\n\nLangfuse maintains a versioned API structure for scores:\n\n```\ninterfaces/\n├── api/\n│ ├── v1/ # Legacy API (trace-focused)\n│ ├── v2/ # Current API (supports traces, sessions)\n│ └── shared.ts\n├── application/\n├── ingestion/\n└── ui/\n```\n\n**API Versioning Strategy:**\n- POST/DELETE APIs: Support all score types across v1 and v2\n- GET APIs:\n - V1: Requires `traceId`, trace-level only\n - V2: `traceId` optional, adds `sessionId` support\n\n资料来源：[packages/shared/src/features/scores/interfaces/README.md:1-50]()\n\n## Backend Worker Architecture\n\n### Event Processing\n\nThe worker service processes ingestion events asynchronously using BullMQ queues:\n\n**Queue Types:**\n| Queue | Purpose | Consumer |\n|-------|---------|----------|\n| `IngestionSecondaryQueue` | Standard event processing | Worker |\n| `OtelIngestionQueue` | OpenTelemetry events | Worker |\n\n**Event Flow:**\n```mermaid\ngraph LR\n A[Ingestion API] --> B{Event Type?}\n B -->|Standard| C[S3 Key Parse]\n B -->|OTEL| D[OTEL Key Parse]\n C --> E[Queue Payload]\n D --> E\n E --> F[BullMQ]\n F --> G[Worker Processing]\n G --> H[(ClickHouse)]\n G --> I[(PostgreSQL)]\n```\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md:1-60]()\n\n### Replay Ingestion Events V2\n\nThe `replayIngestionEventsV2` script enables replaying historical events from S3 storage:\n\n**Key Features:**\n- Batch processing with configurable size\n- Checkpoint/resume capability\n- Rate limiting with exponential backoff\n- Error handling with detailed logging\n\n**Differences from V1:**\n\n| Aspect | V1 | V2 |\n|--------|----|----|\n| Infrastructure | Redis, ClickHouse, PostgreSQL, S3 | Langfuse host URL only |\n| Setup | Full repo clone + `.env` | `npx tsx` + env vars |\n| Event Delivery | BullMQ `addBulk` to Redis | HTTP POST to admin API |\n| Resume | Manual | Built-in checkpoint |\n\n**Event Transformation:**\n- Standard keys: `{projectId}/{type}/{eventBodyId}/{eventId}.json`\n- OTEL keys: `otel/{projectId}/{yyyy}/{mm}/{dd}/{hh}/{mm}/{eventId}.json`\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md:1-120]()\n\n### Refill Queue Event\n\nThe `refillQueueEvent` utility script backfills queues with events from local files:\n\n**Usage Pattern:**\n```bash\npnpm run --filter=worker refill-queue-event\n```\n\n**Requirements:**\n- `./worker/events.jsonl` file with JSON events\n- Redis connection via `REDIS_CONNECTION_STRING`\n- Supporting services: S3, ClickHouse\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md:1-60]()\n\n## MCP Server Architecture\n\nLangfuse includes an MCP (Model Context Protocol) server for programmatic prompt management:\n\n**Stateless Design:**\n1. Fresh server instance per request\n2. Authentication context captured in handler closures\n3. Server discarded after request completes\n4. No state between requests\n\n**Available Tools:**\n| Tool | Purpose |\n|------|---------|\n| `getPrompt` | Fetch resolved prompt with dependencies |\n| `getPromptUnresolved` | Fetch raw prompt without resolution |\n| `listPrompts` | List prompts with filtering |\n| `createTextPrompt` | Create text prompt version |\n| `createChatPrompt` | Create chat prompt version |\n| `updatePromptLabels` | Manage prompt labels |\n\n**Prompt Resolution:**\n- Resolved: Recursively replaces `@@@langfusePrompt:...@@@` tags\n- Unresolved: Returns raw content with tags intact\n\n资料来源：[web/src/features/mcp/README.md:1-100]()\n\n## Component Communication Flow\n\n```mermaid\ngraph TD\n subgraph Pages[\"Page Layer\"]\n P[Page Component]\n PC[PeekTableStateProvider]\n end\n \n subgraph Features[\"Feature Layer\"]\n FP[Feature Provider]\n FH[Feature Hook]\n FT[Feature Transformers]\n end\n \n subgraph Services[\"Service Layer\"]\n TRPC[tRPC Router]\n API[API Routes]\n end\n \n subgraph Data[\"Data Layer\"]\n Repo[Repositories]\n DB[(PostgreSQL)]\n CH[(ClickHouse)]\n Redis[(Redis)]\n end\n \n P --> PC\n PC --> FP\n FP --> FH\n FH --> FT\n FT --> TRPC\n TRPC --> Repo\n Repo --> DB\n Repo --> CH\n Repo --> Redis\n```\n\n## Configuration Management\n\nLangfuse uses environment-based configuration across layers:\n\n| Environment | Scope | Examples |\n|-------------|-------|----------|\n| `REDIS_CONNECTION_STRING` | Worker, Queue | Redis URL |\n| `CLICKHOUSE_URL` | Analytics | ClickHouse connection |\n| `LANGFUSE_S3_EVENT_UPLOAD_BUCKET` | Storage | S3 bucket name |\n| `ADMIN_API_KEY` | Admin API | Authentication |\n\n## Security Architecture\n\n**Key Security Components:**\n\n1. **JWT Authentication**: Organization and user context embedded in JWT tokens\n2. **API Key Validation**: Admin API uses dedicated key authentication\n3. **Scope-based Authorization**: Project-level access control\n4. **Plan Entitlements**: Feature availability based on subscription tier\n\n**Self-hosted Considerations:**\n- License key validation for enterprise features\n- Environment-based plan override capability\n\n## Performance Optimizations\n\n**Frontend Optimizations:**\n- Memoized transformations in hooks\n- Virtualized table rendering (TanStack Virtual)\n- Iterative algorithms (no recursion, preventing stack overflow)\n\n**Backend Optimizations:**\n- Batch processing for event replay\n- Checkpoint/resume for long-running operations\n- Client-side rate limiting with exponential backoff\n\n## Summary\n\nThe Langfuse architecture demonstrates a well-structured approach to observability platforms:\n\n- **Separation of Concerns**: Clear boundaries between UI, business logic, and data layers\n- **Scalability**: Asynchronous processing via queues enables horizontal scaling\n- **Extensibility**: Feature modules and MCP server support programmatic access\n- **Observability**: Built-in OpenTelemetry integration for distributed tracing\n- **Performance**: ClickHouse for analytics, iterative algorithms, and batch processing\n\n---\n\n\n\n## Monorepo Configuration\n\n### 相关页面\n\n相关主题：[Project Structure](#project-structure), [System Architecture](#system-architecture)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/src/server/clickhouse/client.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/clickhouse/client.ts)\n- [packages/shared/src/server/redis/redis.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/redis.ts)\n- [packages/shared/src/db.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/db.ts)\n- [packages/shared/src/server/otel/index.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/otel/index.ts)\n- [web/src/components/layouts/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/layouts/README.md)\n- [web/src/components/design-system/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/design-system/README.md)\n- [web/src/components/table/peek/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/components/table/peek/README.md)\n- [web/src/features/entitlements/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/entitlements/README.md)\n- [packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n- [worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n
\n\n# Monorepo Configuration\n\nLangfuse uses a **monorepo architecture** managed with [Turborepo](https://turbo.build/), [pnpm](https://pnpm.io/) workspaces, and shared configuration packages. This setup enables efficient builds, consistent code quality standards, and streamlined dependency management across the project's multiple packages.\n\n## Architecture Overview\n\nThe Langfuse repository is organized as a pnpm workspace monorepo with the following core structure:\n\n```yaml\nlangfuse/\n├── web/ # Next.js frontend application\n├── worker/ # Background job processing\n├── packages/\n│ ├── shared/ # Shared utilities and types\n│ ├── config-eslint/ # Shared ESLint configuration\n│ └── config-typescript/ # Shared TypeScript configurations\n├── turbo.json # Turborepo pipeline definition\n└── pnpm-workspace.yaml # Workspace package definitions\n```\n\n## Turborepo Pipeline Configuration\n\nThe `turbo.json` file defines the build pipeline and task orchestration across packages.\n\n### Core Pipeline Tasks\n\n| Task | Description | Cache Strategy |\n|------|-------------|----------------|\n| `build` | Compiles TypeScript and bundles assets | Enabled |\n| `dev` | Starts development servers | Local only |\n| `test` | Runs unit and integration tests | Enabled |\n| `lint` | ESLint code quality checks | Enabled |\n| `typecheck` | TypeScript type validation | Enabled |\n\n### Task Dependencies\n\nTurborepo automatically resolves dependencies between packages. For example:\n\n```mermaid\ngraph TD\n A[packages/shared] -->|build| B[Type definitions]\n B --> C[packages/config-*]\n C --> D[web]\n C --> E[worker]\n D --> F[Build output]\n E --> F\n```\n\n## Shared ESLint Configuration\n\nThe `packages/config-eslint/index.js` provides standardized ESLint rules across all packages.\n\n### Configuration Features\n\n- **React/Next.js support** for the web application\n- **TypeScript-aware linting** via `@typescript-eslint`\n- **Import ordering** rules for consistent module organization\n- **JSX accessibility** checks\n\n### Usage\n\nPackages extend the shared configuration:\n\n```javascript\n// In package's .eslintrc.js\nmodule.exports = {\n extends: ['@langfuse/config-eslint'],\n // Package-specific overrides\n rules: {\n // Custom rules\n }\n};\n```\n\n资料来源：[packages/config-eslint/index.js]()\n\n## Shared TypeScript Configuration\n\nThe `packages/config-typescript/` directory contains base TypeScript configurations.\n\n### Base Configuration (`base.json`)\n\n```json\n{\n \"compilerOptions\": {\n \"target\": \"ES2020\",\n \"module\": \"ESNext\",\n \"moduleResolution\": \"bundler\",\n \"strict\": true,\n \"esModuleInterop\": true,\n \"skipLibCheck\": true,\n \"forceConsistentCasingInFileNames\": true\n }\n}\n```\n\n### Package-Specific Configurations\n\nIndividual packages extend the base configuration:\n\n```json\n// packages/shared/tsconfig.json\n{\n \"extends\": \"@langfuse/config-typescript/base.json\",\n \"compilerOptions\": {\n \"outDir\": \"./dist\",\n \"rootDir\": \"./src\"\n },\n \"include\": [\"src/**/*\"],\n \"exclude\": [\"node_modules\", \"dist\"]\n}\n```\n\n资料来源：[packages/config-typescript/base.json](), [packages/shared/tsconfig.json]()\n\n## Package Scripts and Commands\n\nEach package defines its own scripts in `package.json`. The web package demonstrates the typical pattern:\n\n### Build Commands\n\n| Command | Purpose |\n|---------|---------|\n| `pnpm build` | Production build with `INLINE_RUNTIME_CHUNK=false` |\n| `pnpm build:check` | Build without emitting (type-checking) |\n| `pnpm dev` | Development server on localhost:3000 |\n| `pnpm dev:http` | HTTPS development server for local testing |\n\n### Quality Assurance Commands\n\n| Command | Purpose |\n|---------|---------|\n| `pnpm lint` | ESLint with caching enabled |\n| `pnpm lint:fix` | Auto-fix linting issues |\n| `pnpm typecheck` | TypeScript validation with incremental compilation |\n| `pnpm test` | Vitest with server and in-source test projects |\n\n资料来源：[web/package.json]()\n\n## Development Workflow\n\n### Starting Development\n\n```bash\n# Install dependencies\npnpm install\n\n# Start all dev servers based on turbo.json\npnpm dev\n\n# Or start a specific package\ncd web && pnpm dev\n```\n\n### Running Tests\n\n```bash\n# All tests\npnpm test\n\n# Client-side tests only (e.g., AdvancedJsonViewer)\npnpm --filter=web run test-client --testPathPattern=\"AdvancedJsonViewer\"\n\n# Server-side tests\npnpm --filter=web run test --project server\n```\n\n### Build Pipeline\n\n```mermaid\ngraph LR\n A[pnpm build] --> B[turbo build]\n B --> C{Cache hit?}\n C -->|Yes| D[Use cached output]\n C -->|No| E[Build dependencies]\n E --> F[packages/shared]\n F --> G[packages/config-*]\n G --> H[web/worker]\n H --> I[Save to cache]\n I --> J[Build artifacts]\n```\n\n## Environment Configuration\n\nThe project uses `.env` files for environment-specific configuration:\n\n| File | Purpose |\n|------|---------|\n| `.env` | Default environment variables |\n| `.env.local` | Local overrides (git-ignored) |\n| `.env.test` | Test environment variables |\n\nScripts use `dotenv` to load these files:\n\n```bash\ndotenv -e ../.env -- next build\ndotenv -e ../.env.test -e ../.env -- vitest run\n```\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n## Best Practices\n\n### Adding a New Package\n\n1. Create the package under `packages/` or `web/` directories\n2. Extend the shared TypeScript and ESLint configurations\n3. Add the package to `pnpm-workspace.yaml` if needed\n4. Define tasks in `turbo.json` if custom pipeline is required\n5. Add appropriate scripts to `package.json`\n\n### Caching Strategy\n\n- **Build caching**: Enabled by default via Turborepo\n- **Lint caching**: ESLint caches to `.next/cache/eslint/`\n- **TypeScript incremental**: Uses `.tsbuildinfo` files\n\n### CI/CD Integration\n\nIn CI environments, clear caches to ensure fresh builds:\n\n```bash\n# Clear turbo cache\nrm -rf .turbo node_modules/.cache\n\n# Clear ESLint cache\nrm -rf .next/cache/eslint/\n\n# Fresh build\npnpm install --frozen-lockfile\npnpm build\n```\n\n## Key Configuration Files Summary\n\n| File | Purpose |\n|------|---------|\n| `turbo.json` | Task pipeline and dependency graph |\n| `pnpm-workspace.yaml` | Workspace package definitions |\n| `packages/config-eslint/index.js` | Shared ESLint rules |\n| `packages/config-typescript/base.json` | Shared TypeScript base config |\n| `.eslintrc.js` (per package) | Package-specific lint overrides |\n| `tsconfig.json` (per package) | Package-specific TypeScript config |\n\n---\n\n\n\n## Database Schema (Prisma)\n\n### 相关页面\n\n相关主题：[System Architecture](#system-architecture), [ClickHouse Analytics Layer](#clickhouse-analytics)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [turbo.json](https://github.com/langfuse/langfuse/blob/main/turbo.json)\n- [packages/config-eslint/index.js](https://github.com/langfuse/langfuse/blob/main/packages/config-eslint/index.js)\n- [packages/config-typescript/base.json](https://github.com/langfuse/langfuse/blob/main/packages/config-typescript/base.json)\n- [packages/shared/tsconfig.json](https://github.com/langfuse/langfuse/blob/main/packages/shared/tsconfig.json)\n- [web/package.json](https://github.com/langfuse/langfuse/blob/main/web/package.json)\n- [worker/src/scripts/replayIngestionEventsV2/README.md](https://github.com/langfuse/langfuse/blob/main/worker/src/scripts/replayIngestionEventsV2/README.md)\n\n
\n\n# Database Schema (Prisma)\n\n## Overview\n\nLangfuse uses Prisma ORM to manage its PostgreSQL database schema. The schema defines the core data models for the application, including organizations, projects, users, traces, observations, and scores. Prisma serves as the primary interface between the application's business logic and the relational database.\n\nThe Prisma schema is located at `packages/shared/prisma/schema.prisma` and is shared across multiple packages in the monorepo structure. This centralized schema approach ensures consistency in data modeling across the web application, worker services, and shared libraries.\n\n### Design Philosophy\n\nThe schema follows several key principles:\n\n- **Normalized relationships**: Related entities are linked through foreign keys with proper cascading behaviors\n- **Soft deletes**: Key entities support soft deletion for data recovery and audit purposes\n- **Audit fields**: Most tables include `createdAt`, `updatedAt`, and `createdBy` fields\n- **Multi-tenancy**: The schema supports multi-tenant architecture with organization and project isolation\n- **Extensible metadata**: JSON fields allow flexible storage of custom attributes\n\n## Core Data Models\n\n### Organization and User Models\n\nThe foundation of Langfuse's multi-tenant architecture begins with the `Organization` model, which represents the top-level tenant entity. Each organization can have multiple users with different roles and permission levels.\n\nThe `User` model stores authentication and profile information, linked to organizations through the `Membership` junction table. This many-to-many relationship enables users to belong to multiple organizations with potentially different roles in each.\n\n```mermaid\nerDiagram\n Organization ||--o{ User : \"contains\"\n Organization ||--o{ Project : \"contains\"\n User ||--o{ Membership : \"has\"\n Membership }o--|| Organization : \"belongs to\"\n Membership }o--|| User : \"belongs to\"\n```\n\n### Project Model\n\nProjects serve as the primary container for observability data. Each project belongs to exactly one organization and contains all traces, observations, and scores related to a specific application or use case.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | `String` | UUID primary key |\n| `name` | `String` | Project display name |\n| `organizationId` | `String` | Foreign key to organization |\n| `createdAt` | `DateTime` | Creation timestamp |\n| `updatedAt` | `DateTime` | Last modification timestamp |\n| `deletedAt` | `DateTime?` | Soft delete timestamp |\n| `settings` | `Json` | Project-specific configuration |\n\n资料来源：[packages/shared/prisma/schema.prisma]()\n\n## Traces\n\nTraces represent the top-level unit of observability in Langfuse. A trace encapsulates a complete interaction or request, typically corresponding to a single LLM call or a multi-step workflow.\n\n### Trace Model Schema\n\n```prisma\nmodel Trace {\n id String @id @default(cuid())\n name String?\n project Project @relation(fields: [projectId], references: [id])\n projectId String\n user String?\n metadata Json?\n sessionId String?\n release String?\n version String?\n tags String[]\n \n // Timestamps\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n \n // Soft delete\n deletedAt DateTime?\n \n // Relations\n observations Observation[]\n scores Score[]\n \n @@index([projectId])\n @@index([sessionId])\n @@index([createdAt])\n}\n```\n\n资料来源：[packages/shared/prisma/schema.prisma]()\n\n### Key Fields\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | `String` | Unique identifier using CUID algorithm |\n| `name` | `String?` | Optional human-readable trace name |\n| `projectId` | `String` | Reference to parent project |\n| `user` | `String?` | Identifier for the end user |\n| `sessionId` | `String?` | Groups related traces into sessions |\n| `release` | `String?` | Application release version |\n| `version` | `String?` | Trace format version |\n| `tags` | `String[]` | Array of string tags for categorization |\n\n### Repository Pattern\n\nTraces are accessed through the repository pattern defined in `packages/shared/src/server/repositories/traces.ts`. This abstraction provides a clean interface for CRUD operations while encapsulating query logic.\n\n资料来源：[packages/shared/src/server/repositories/traces.ts]()\n\n```typescript\n// Repository interface pattern (simplified)\ninterface ITraceRepository {\n create(data: CreateTraceInput): PromiseRelevant Source Files
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/prisma/schema.prisma](https://github.com/langfuse/langfuse/blob/main/packages/shared/prisma/schema.prisma)\n- [packages/shared/src/server/repositories/traces.ts](https://packages/shared/src/server/repositories/traces.ts)\n- [packages/shared/src/server/repositories/observations.ts](https://packages/shared/src/server/repositories/observations.ts)\n- [packages/shared/src/server/repositories/scores.ts](https:///packages/shared/src/server/repositories/scores.ts)\n- [packages/shared/src/features/scores/interfaces/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/features/scores/interfaces/README.md)\n\n
\n\n# ClickHouse Analytics Layer\n\n## Overview\n\nThe ClickHouse Analytics Layer is a core infrastructure component in Langfuse that provides high-performance analytical capabilities for processing and querying large-scale observability data. ClickHouse serves as the primary OLAP (Online Analytical Processing) database for storing traces, observations, and score analytics with optimized columnar storage and efficient aggregation queries.\n\nLangfuse leverages ClickHouse for:\n\n- High-throughput event ingestion during trace collection\n- Complex analytical queries for score comparisons and distributions\n- Time-series analysis with efficient aggregation\n- Large dataset sampling and optimization strategies\n 资料来源：[packages/shared/scripts/seeder/utils/README.md]()\n\n## Architecture Overview\n\n```mermaid\ngraph TD\n subgraph Ingestion[\"Ingestion Layer\"]\n W[Worker Service] --> CW[ClickhouseWriter]\n CW --> CH[ClickHouse Cluster]\n end\n \n subgraph Storage[\"Storage Layer\"]\n CH --> TS[Traces Table]\n CH --> OS[Observations Table]\n CH --> SS[Scores Table]\n end\n \n subgraph Query[\"Query Layer\"]\n CR[ClickHouse Repository] --> CH\n SA[Score Analytics] --> CR\n WEB[Web Frontend] --> SA\n end\n \n subgraph Optimization[\"Optimization Layer\"]\n CR --> HASH[cityHash64 Sampling]\n CR --> FINAL[Adaptive FINAL]\n CR --> INTERVAL[Time Interval Alignment]\n end\n```\n\n### Components Overview\n\n| Component | Location | Purpose |\n|-----------|----------|---------|\n| ClickhouseWriter | `worker/src/services/ClickhouseWriter/index.ts` | Writes ingestion events to ClickHouse |\n| ClickHouse Repository | `packages/shared/src/server/repositories/clickhouse.ts` | Provides query interface and optimization |\n| Score Analytics | `packages/shared/src/server/repositories/score-analytics.ts` | Specialized analytics queries |\n| Schema Definitions | `packages/shared/src/server/clickhouse/schema.ts` | TypeScript types for ClickHouse data |\n| Migrations | `packages/shared/clickhouse/migrations/clustered/` | Database schema migrations |\n\n资料来源：[worker/src/services/ClickhouseWriter/index.ts]()\n资料来源：[packages/shared/src/server/repositories/clickhouse.ts]()\n\n## Data Schema\n\n### Traces Table\n\nThe traces table stores the fundamental trace records with hierarchical observation data. The clustered migration defines the primary schema with optimized column types for analytical queries.\n\nKey columns include:\n\n| Column | Type | Description |\n|--------|------|-------------|\n| id | UUID | Unique trace identifier |\n| project_id | String | Project association |\n| timestamp | DateTime64 | Event timestamp with millisecond precision |\n| name | String | Trace name |\n| user_id | String | User identifier |\n| metadata | JSON | Flexible metadata storage |\n| tags | Array(String) | Tag-based categorization |\n| input | Text | Input data |\n| output | Text | Output data |\n\n资料来源：[packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql]()\n\n### Observations Table\n\nObservations represent individual events within a trace, storing:\n\n- Model inputs/outputs\n- Function calls\n- Embeddings\n- Generation events\n\nEach observation is linked to its parent trace via `trace_id` and supports nested hierarchies through `parent_observation_id`.\n\n资料来源：[packages/shared/src/server/clickhouse/schema.ts]()\n\n### Scores Table\n\nScores store evaluation metrics associated with traces and observations:\n\n| Column | Type | Purpose |\n|--------|------|---------|\n| trace_id | UUID | Associated trace |\n| observation_id | UUID | Optional observation link |\n| name | String | Score identifier |\n| value | Float64 | Numeric score value |\n| data_type | Enum | NUMERIC, BOOLEAN, or CATEGORICAL |\n| source | String | Score origin (e.g., \"framework-trace\") |\n\n资料来源：[packages/shared/src/server/repositories/score-analytics.ts]()\n\n## Ingestion Pipeline\n\n### Event Flow\n\n```mermaid\nsequenceDiagram\n participant API as Ingestion API\n participant Queue as Redis Queue\n participant Worker as Worker Service\n participant Writer as ClickhouseWriter\n participant CH as ClickHouse\n \n API->>Queue: Enqueue OtelIngestionEvent\n Worker->>Queue: Dequeue Event\n Worker->>Writer: Process Event\n Writer->>CH: Insert Batch (ClickHouseQueryBuilder)\n CH-->>Writer: Confirmation\n Writer->>Worker: Acknowledge\n```\n\n### ClickhouseWriter Service\n\nThe `ClickhouseWriter` handles the actual data insertion into ClickHouse:\n\n```typescript\n// Simplified flow from worker/src/services/ClickhouseWriter/index.ts\nclass ClickhouseWriter {\n async writeBatch(events: IngestionEvent[]): Promise相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql](https://github.com/langfuse/langfuse/blob/main/packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql)\n- [packages/shared/src/server/clickhouse/schema.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/clickhouse/schema.ts)\n- [packages/shared/src/server/repositories/clickhouse.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/repositories/clickhouse.ts)\n- [worker/src/services/ClickhouseWriter/index.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/services/ClickhouseWriter/index.ts)\n- [packages/shared/src/server/repositories/score-analytics.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/repositories/score-analytics.ts)\n- [web/src/features/score-analytics/README.md](https://github.com/langfuse/langfuse/blob/main/web/src/features/score-analytics/README.md)\n- [packages/shared/scripts/seeder/utils/README.md](https://github.com/langfuse/langfuse/blob/main/packages/shared/scripts/seeder/utils/README.md)\n\n
\n\n# Queue System (Redis/BullMQ)\n\nLangfuse employs a distributed queue system built on **Redis** for storage and **BullMQ** for job orchestration. This architecture enables asynchronous processing of high-volume operations including event ingestion, evaluation execution, batch actions, and webhook delivery.\n\n## Architecture Overview\n\nThe queue system follows a producer-consumer pattern where the web application enqueues jobs and worker processes consume them asynchronously.\n\n```mermaid\ngraph TD\n subgraph \"Langfuse Web Application\"\n A[API Request] --> B[Queue Client]\n B --> C[Redis Queue]\n end\n \n subgraph \"Langfuse Worker\"\n C --> D[Worker Manager]\n D --> E1[Ingestion Worker]\n D --> E2[Eval Worker]\n D --> E3[Batch Worker]\n D --> E4[Webhook Worker]\n end\n \n subgraph \"Redis\"\n C --> F[(Redis Cluster)]\n end\n \n subgraph \"External Services\"\n E1 --> G[(ClickHouse)]\n E1 --> H[(PostgreSQL)]\n E2 --> H\n E3 --> H\n E4 --> I[(External APIs)]\n end\n```\n\n## Queue Types\n\nLangfuse defines multiple specialized queues for different workloads:\n\n| Queue Name | Purpose | Processing Type | Priority |\n|------------|---------|-----------------|----------|\n| `ingestion` | Event ingestion and processing | Async batch | Medium |\n| `evalExecution` | LLM evaluation execution | Async | Medium |\n| `batchAction` | Bulk operations on data | Async batch | Low |\n| `webhook` | Outbound webhook delivery | Async | High |\n| `OtelIngestion` | OpenTelemetry event ingestion | Async | Medium |\n| `IngestionSecondary` | Secondary ingestion processing | Async | Medium |\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n## Queue Configuration\n\n### Redis Connection\n\nAll queues rely on a Redis connection string configured via environment variables:\n\n```bash\nREDIS_CONNECTION_STRING=redis://:myredissecret@127.0.0.1:6379\n```\n\n### Queue Initialization\n\nEach queue is initialized with specific BullMQ configuration:\n\n```typescript\nconst myQueue = new QueueRelevant Source Files
\n\n以下源码文件用于生成本页说明：\n\n- [packages/shared/src/server/redis/ingestionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/ingestionQueue.ts)\n- [packages/shared/src/server/redis/evalExecutionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/evalExecutionQueue.ts)\n- [packages/shared/src/server/redis/batchActionQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/batchActionQueue.ts)\n- [packages/shared/src/server/redis/webhookQueue.ts](https://github.com/langfuse/langfuse/blob/main/packages/shared/src/server/redis/webhookQueue.ts)\n- [worker/src/queues/workerManager.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/queues/workerManager.ts)\n\n
\n\n# Worker Service\n\n## Overview\n\nThe Worker Service is a core backend component in Langfuse responsible for asynchronous processing of long-running tasks. It operates as a separate Node.js process that communicates with the main Langfuse server through message queues, primarily using BullMQ backed by Redis.\n\n```mermaid\ngraph TB\n subgraph \"Langfuse Server\"\n A[API Endpoints]\n B[tRPC Routers]\n end\n \n subgraph \"Redis\"\n C[(Ingestion Queues)]\n D[(Evaluation Queues)]\n E[(Batch Action Queues)]\n end\n \n subgraph \"Worker Service\"\n F[Worker Manager]\n G[Evaluation Service]\n H[Batch Action Handler]\n I[Queue Processors]\n end\n \n A -->|\"Enqueue Jobs\"| C\n B -->|\"Dispatch Tasks\"| C\n F -->|\"Process\"| C\n G -->|\"Execute\"| D\n H -->|\"Execute\"| E\n C --> F\n D --> G\n E --> H\n```\n\n### Purpose and Scope\n\nThe Worker Service handles the following categories of work:\n\n- **Event Ingestion Processing**: Processing and persisting trace events, observations, and spans from the ingestion queues\n- **Evaluation Execution**: Running LLM-based evaluations on traces and observations\n- **Batch Actions**: Executing bulk operations on datasets, traces, and other resources\n- **Queue Replay**: Replaying historical ingestion events for data recovery or reprocessing\n\n资料来源：[worker/src/app.ts]()\n\n---\n\n## Architecture\n\n### Entry Point\n\nThe worker service is bootstrapped through `worker/src/index.ts`, which initializes the application context and starts the queue processors. The main application logic resides in `worker/src/app.ts`, which sets up the BullMQ workers and registers job handlers.\n\n```typescript\n// Simplified worker initialization flow\nconst app = new WorkerApp();\nawait app.initialize();\nawait app.start();\n```\n\n资料来源：[worker/src/index.ts]()\n\n### Worker Manager\n\nThe `WorkerManager` is the central orchestrator that manages all queue workers. It is responsible for:\n\n- Registering queue processors for different job types\n- Configuring concurrency settings per queue\n- Handling job failures and retries\n- Graceful shutdown coordination\n\n```mermaid\ngraph LR\n A[Job Enqueued] --> B[Worker Manager]\n B --> C{Job Type?}\n C -->|Ingestion| D[Ingestion Processor]\n C -->|Evaluation| E[Eval Service]\n C -->|Batch Action| F[Batch Handler]\n \n D --> G[Success]\n E --> G\n F --> G\n \n D --> H[Retry/Fail]\n E --> H\n F --> H\n```\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n### Queue Architecture\n\nLangfuse uses multiple queues for different purposes:\n\n| Queue Name | Purpose | Priority | Concurrency |\n|------------|---------|----------|-------------|\n| `IngestionSecondaryQueue` | Standard event ingestion | Medium | Configurable |\n| `OtelIngestionQueue` | OpenTelemetry-format events | High | Configurable |\n| `EvalQueue` | LLM evaluation jobs | Low | Configurable |\n| `BatchActionQueue` | Bulk operations | Medium | Configurable |\n\n资料来源：[worker/src/app.ts]()\n资料来源：[worker/src/queues/workerManager.ts]()\n\n---\n\n## Core Components\n\n### Evaluation Service\n\nThe Evaluation Service (`evalService.ts`) handles asynchronous evaluation of traces and observations using LLM-based judges. It supports:\n\n- **Trace-level evaluations**: Running multiple evaluation criteria against a complete trace\n- **Observation-level evaluations**: Evaluating individual spans or generations\n- **Dataset-based evaluations**: Running evaluations against dataset items\n- **Configurable retry logic**: Handling transient failures gracefully\n\n```mermaid\ngraph TD\n A[Evaluation Job Received] --> B[Load Trace/Observation]\n B --> C[Fetch Eval Config]\n C --> D[Prepare Prompt]\n D --> E[Call LLM Provider]\n E --> F{Success?}\n F -->|Yes| G[Score Result]\n F -->|No| H{Retry < 3?}\n H -->|Yes| E\n H -->|No| I[Mark Failed]\n G --> J[Persist Score]\n J --> K[Job Complete]\n I --> K\n```\n\n资料来源：[worker/src/features/evaluation/evalService.ts]()\n\n### Batch Action Handler\n\nThe Batch Action Handler processes bulk operations requested through the admin API or scheduled jobs. Common batch actions include:\n\n- Bulk dataset operations (import, export, deletion)\n- Mass updates to traces or observations\n- Batch score calculations\n- Export operations\n\n```typescript\ninterface BatchActionJob {\n id: string;\n type: BatchActionType;\n payload: BatchActionPayload;\n priority?: number;\n createdAt: Date;\n}\n```\n\n资料来源：[worker/src/features/batchAction/handleBatchActionJob.ts]()\n\n---\n\n## Queue Processing\n\n### Job Flow\n\n```mermaid\nsequenceDiagram\n participant API as API Server\n participant Redis as Redis Queue\n participant Worker as Worker Service\n \n API->>Redis: Enqueue Job\n Worker->>Redis: Poll for Jobs\n Redis-->>Worker: Available Jobs\n Worker->>Worker: Process Job\n Worker->>Redis: Mark Complete\n Worker->>API: Update Status (optional)\n```\n\n### Error Handling and Retries\n\nThe worker implements exponential backoff for transient failures:\n\n1. **Transient failures** (network timeouts, rate limits): Retry up to 3 times with exponential backoff\n2. **Permanent failures** (validation errors, missing data): Mark as failed, log error details\n3. **Rate limiting**: Respects server-side rate limits and backs off accordingly\n\n| Error Type | HTTP Code | Retry Behavior |\n|------------|-----------|----------------|\n| Rate Limited | 429 | Exponential backoff |\n| Server Error | 5xx | Retry up to 3 times |\n| Client Error | 4xx (not 429) | No retry, log and skip |\n| Validation | N/A | No retry, mark failed |\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n### Concurrency Configuration\n\nWorkers can be configured with different concurrency levels per queue:\n\n```typescript\nconst workerConfig = {\n ingestion: {\n concurrency: 10,\n maxRetries: 3,\n },\n evaluation: {\n concurrency: 5,\n maxRetries: 2,\n },\n batchAction: {\n concurrency: 3,\n maxRetries: 1,\n },\n};\n```\n\n资料来源：[worker/src/queues/workerManager.ts]()\n\n---\n\n## Utility Scripts\n\n### Replay Ingestion Events V2\n\nThe replay script allows reprocessing of historical ingestion events from S3 storage.\n\n**Usage:**\n```bash\npnpm --filter=worker run replay-ingestion-events-v2 \\\n --input ./events.csv \\\n --batch-size 500 \\\n --concurrency 4\n```\n\n**Key Features:**\n- Checkpoint/resume support for interrupted runs\n- Configurable batch size and concurrency\n- Rate limiting with exponential backoff\n- Progress tracking with percentage and ETA\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `--input` | Required | Path to CSV file with S3 keys |\n| `--batch-size` | 500 | Keys per API request |\n| `--concurrency` | 4 | Parallel API requests |\n| `--rate-limit` | 50 | Max requests per second |\n| `--dry-run` | false | Validate without sending |\n| `--resume` | false | Continue from checkpoint |\n\n资料来源：[worker/src/scripts/replayIngestionEventsV2/README.md]()\n\n### Refill Queue Event\n\nThe refill script backfills queues with events from local files for testing and development.\n\n**Requirements:**\n1. Create `./worker/events.jsonl` with JSON events\n2. Configure `.env` with Redis and supporting service credentials\n3. Run: `pnpm run --filter=worker refill-queue-event`\n\n**Event Format:**\n```jsonl\n{\"projectId\": \"project-123\", \"orgId\": \"org-456\"}\n{\"projectId\": \"project-789\", \"orgId\": \"org-101\"}\n```\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md]()\n\n---\n\n## Environment Configuration\n\n### Required Environment Variables\n\n| Variable | Description | Required |\n|----------|-------------|----------|\n| `REDIS_CONNECTION_STRING` | Redis connection URL | Yes |\n| `LANGFUSE_S3_EVENT_UPLOAD_BUCKET` | S3 bucket for event uploads | Yes |\n| `CLICKHOUSE_URL` | ClickHouse connection URL | Yes |\n| `CLICKHOUSE_USER` | ClickHouse username | Yes |\n| `CLICKHOUSE_PASSWORD` | ClickHouse password | Yes |\n| `ADMIN_API_KEY` | Admin API key for replay scripts | For replay only |\n| `LANGFUSE_HOST` | Target Langfuse instance URL | For replay only |\n\n### Optional Configuration\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `WORKER_CONCURRENCY` | 10 | Default queue concurrency |\n| `EVALUATION_CONCURRENCY` | 5 | Evaluation queue concurrency |\n| `BATCH_ACTION_CONCURRENCY` | 3 | Batch action concurrency |\n\n资料来源：[worker/src/scripts/refillQueueEvent/README.md]()\n资料来源：[worker/src/app.ts]()\n\n---\n\n## Health and Monitoring\n\n### Debug Mode\n\nEnable detailed logging for the AdvancedJsonViewer (useful for debugging UI components):\n\n```javascript\nlocalStorage.setItem(\"debug:AdvancedJsonViewer\", \"true\");\n```\n\n### Logging Categories\n\nThe worker service emits structured logs for:\n\n- Queue processing events\n- Job duration and throughput\n- Error rates and failure reasons\n- Resource utilization metrics\n\n### Graceful Shutdown\n\nThe worker service supports graceful shutdown to prevent job loss:\n\n1. Stop accepting new jobs\n2. Wait for in-progress jobs to complete (with timeout)\n3. Release Redis connections\n4. Exit cleanly\n\n---\n\n## Performance Considerations\n\n### Scalability\n\n- Multiple worker instances can run in parallel\n- Each queue can have independent concurrency settings\n- Redis acts as a shared job queue, enabling horizontal scaling\n\n### Memory Management\n\n| Strategy | Implementation |\n|----------|----------------|\n| Iterative Processing | Uses explicit stack-based iteration instead of recursion to prevent stack overflow |\n| Batch Processing | Processes events in configurable batch sizes |\n| Lazy Loading | Loads trace/observation data on-demand within job handlers |\n\n### Known Limitations\n\n1. **No horizontal virtualization in UI**: Wide JSON rows render fully\n2. **Client-side search only**: All matches computed in memory\n3. **Memory constraints**: 1M+ nodes may cause issues despite virtualization\n4. **Wrap mode performance**: Long strings require height measurement\n\n资料来源：[web/src/components/ui/AdvancedJsonViewer/README.md]()\n\n---\n\n## Summary\n\nThe Worker Service is essential to Langfuse's architecture, providing asynchronous processing capabilities that keep the main API server responsive. Key takeaways:\n\n- **BullMQ + Redis**: Provides reliable job queuing with built-in retry logic\n- **Specialized processors**: Dedicated handlers for ingestion, evaluation, and batch actions\n- **Configurable concurrency**: Fine-tune throughput per queue type\n- **Replay capabilities**: Built-in utilities for reprocessing historical events\n- **Horizontal scaling**: Multiple worker instances share work through Redis\n\n---\n\n\n\n## API Layer\n\n### 相关页面\n\n相关主题：[System Architecture](#system-architecture)\n\n相关源码文件
\n\n以下源码文件用于生成本页说明：\n\n- [worker/src/app.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/app.ts)\n- [worker/src/index.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/index.ts)\n- [worker/src/queues/workerManager.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/queues/workerManager.ts)\n- [worker/src/features/evaluation/evalService.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/features/evaluation/evalService.ts)\n- [worker/src/features/batchAction/handleBatchActionJob.ts](https://github.com/langfuse/langfuse/blob/main/worker/src/features/batchAction/handleBatchActionJob.ts)\n\n
\n\n# API Layer\n\nThe API Layer is the central communication bridge between the Langfuse frontend and backend services. Built on **tRPC** (TypeScript RPC), it provides end-to-end type safety, enabling the web application to interact with server-side logic through strongly-typed procedure calls.\n\n## Overview\n\nThe API Layer serves multiple critical functions:\n\n- **Type-Safe Communication**: All API calls are fully typed from server to client\n- **Authentication & Authorization**: Every procedure is wrapped with auth middleware\n- **Business Logic Isolation**: Procedures delegate to repository layer for data access\n- **Input Validation**: Zod schemas validate all incoming requests\n- **Feature Organization**: Procedures are grouped by domain (traces, observations, scores)\n\n```mermaid\ngraph TD\n subgraph Frontend\n UI[React Components]\n end\n \n subgraph API Layer\n TRPC[tRPC Client]\n Procedures[tRPC Procedures]\n Middleware[Auth Middleware]\n end\n \n subgraph Backend\n Repositories[Repositories]\n Database[(Database)]\n end\n \n UI --> TRPC\n TRPC --> Procedures\n Procedures --> Middleware\n Middleware --> Repositories\n Repositories --> Database\n```\n\n## Architecture\n\n### Core Components\n\n| Component | File | Purpose |\n|-----------|------|---------|\n| tRPC Instance | `trpc.ts` | Initialize tRPC with middleware and context |\n| Root Router | `root.ts` | Register all feature routers |\n| Trace Router | `routers/traces.ts` | Trace CRUD and query operations |\n| Observation Router | `routers/observations.ts` | Span/generation/event operations |\n| Score Router | `routers/scores.ts` | Score management and analytics |\n| Score Analytics | `features/score-analytics/server/scoreAnalyticsRouter.ts` | Score aggregation and statistics |\n\n### Router Registration Flow\n\nThe root router aggregates all feature routers under a namespace:\n\n```typescript\n// Simplified from web/src/server/api/root.ts\nexport const rootRouter = createTRPCRouter({\n trace: traceRouter,\n observation: observationRouter,\n score: scoreRouter,\n scoreAnalytics: scoreAnalyticsRouter,\n // ... other routers\n});\n```\n\n资料来源：[web/src/server/api/root.ts:1-50]()\n\n## tRPC Configuration\n\n### Initialization\n\nThe tRPC instance is initialized in `trpc.ts` with:\n\n1. **Context Creation**: Builds request-scoped context with authentication\n2. **Middleware Chain**: Applies auth, rate limiting, and logging\n3. **Error Handling**: Transforms errors into HTTP-compatible responses\n\n```typescript\n// From web/src/server/api/trpc.ts\nexport const createTRPCContext = async (opts: CreateNextContextOptions) => {\n return {\n session: await getServerSession(authOptions),\n // ... additional context\n };\n};\n\nconst t = initTRPC.context