Related Pages

Related topics: Serving: Workers, Web UI, and APIs, Training Pipeline and Model Support

Overview and Distributed Serving Architecture

Purpose and Scope

FastChat is an open platform for training, serving, and evaluating large language model based chatbots. The codebase powers Chatbot Arena (lmarena.ai), which has served over 10 million chat requests for 70+ LLMs and collected over 1.5M human votes from side-by-side LLM battles to compile the LLM Elo leaderboard. Source: README.md:1-15

The platform provides two principal capabilities:

Source: README.md:16-22

The "distributed serving architecture" specifically refers to the runtime layer that allows multiple models to be hosted concurrently across multiple processes and machines, while presenting a unified interface to end users and downstream clients.

The Three-Tier Architecture

FastChat's serving layer is composed of three decoupled components, each running as its own process. The README explicitly groups them as "web servers that interface with users, model workers that host one or more models, and a controller to coordinate the webserver and model workers." Source: README.md:39-43

flowchart LR
    User[End User / SDK Client]
    WS[Web Servers<br/>Gradio Web UI<br/>OpenAI-Compatible API]
    Ctrl[Controller<br/>Service Registry]
    MW1[Model Worker A<br/>e.g. Vicuna-7B]
    MW2[Model Worker B<br/>e.g. vLLM / SGLang backend]
    MW3[Model Worker C<br/>e.g. LightLLM backend]

    User -->|HTTP / WebSocket| WS
    WS <-->|registry queries| Ctrl
    MW1 -->|register_worker| Ctrl
    MW2 -->|register_worker| Ctrl
    MW3 -->|register_worker| Ctrl
    WS -->|forward generation request| MW1
    WS -->|forward generation request| MW2
    WS -->|forward generation request| MW3

Key responsibilities of each tier:

• Web Server — Accepts user input from the Gradio UI or HTTP API clients. It is the only component exposed to end users.
• Model Worker — Loads one or more model checkpoints into memory and exposes generation endpoints. Each worker self-registers with the controller on startup.
• Controller — Acts as a lightweight service registry. It tracks which worker hosts which model so the web server can route requests without hardcoded addresses.

The architecture is intentionally decoupled so that a single web server can fan out across many heterogeneous backends. Recent releases reflect this trend: v0.2.36 added an SGLang worker for vision language models and a LightLLM worker for higher throughput (see community evidence, Release v0.2.36).

Component Lifecycle and Launch Sequence

The README documents a canonical launch order. Each component is started in its own terminal:

1. Controller — The registry must be available before any worker tries to register.

``bash python3 -m fastchat.serve.controller ``

Source: README.md:45-50

1. Model Worker(s) — One process per model (or per backend). The worker blocks until the model finishes loading and prints "Uvicorn running on ..."; only then is it ready to accept traffic.

``bash python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5 ``

Source: README.md:52-58

1. Verification — A lightweight probe confirms the wiring:

``bash python3 -m fastchat.serve.test_message --model-name vicuna-7b-v1.5 ``

Source: README.md:60-64

1. Web Server — The user-facing entry point:

``bash python3 -m fastchat.serve.gradio_web_server ``

Source: README.md:66-71

The README notes that "if the models do not show up, try to reboot the gradio web server," which is the canonical first troubleshooting step when the web server started before the workers finished registering. Source: README.md:72-74

Backend Flexibility and Operational Concerns

The serving architecture is backend-agnostic at the controller layer. A single web server can simultaneously route to workers running on different inference engines:

• HuggingFace Transformers (default) — Highest compatibility, used in fastchat.serve.model_worker.
• vLLM — For high-throughput batched serving; documented in docs/vllm_integration.md and recommended in the LLM Judge README, which states: "if you experience slow answer generation, please refer to Other Backends section to use inference engine to speed up by 20x." Source: fastchat/llm_judge/README.md:54-62
• ExLlama V2 — Documented in docs/exllama_v2.md.
• GPTQ / AWQ — 4-bit quantized inference documented in docs/gptq.md and docs/awq.md.
• SGLang / LightLLM — Added in v0.2.36 for vision-language and high-throughput workloads respectively.

Resource Constraints and Acceleration

The README documents runtime knobs that affect the serving tier directly:

• --load-8bit — Reduces VRAM by roughly half with slight quality loss; Vicuna-13B fits on a 16 GB card such as an RTX 3090. Source: README.md:240-258
• --cpu-offloading — Requires --load-8bit and bitsandbytes (Linux only).
• --device {cpu, xpu, npu} — CPU, Intel XPU, and Ascend NPU paths are documented as supported.
• --max-gpu-memory — Caps per-GPU weight memory to leave room for activations, useful for longer contexts or larger batches. Source: README.md:215-227

Known Security Consideration

Community evidence documents an unauthenticated SSRF / worker-spoofing issue: the controller's /register_worker endpoint accepts an attacker-controlled worker_name address and immediately fetches it. Operators exposing the controller to untrusted networks should front it with network controls (firewall, reverse proxy with allow-listing, mTLS) or upgrade to a patched release. Source: community issue #3886.

Common Failure Modes

• Models missing in the UI — Usually a startup ordering issue; restart the Gradio web server after workers have registered. Source: README.md:72-74
• Slow answer generation — Switch from the default Transformers worker to vLLM or another optimized backend. Source: fastchat/llm_judge/README.md:54-62
• stop parameter regressions — Community issue #1048 reports the stop parameter in the OpenAI-compatible API was overwritten by conv.stop_str starting at v0.2.5; users upgrading from older versions should verify stop-string behavior.
• Context-length bugs — Community issue #1711 documents that FastChat-T5 advertises a 4K context but only accepts 2K input tokens (encode + output, not input + output).

See Also

• OpenAI-Compatible REST API
• vLLM Integration
• Model Support and Adding New Models
• LangChain Integration
• Third-Party UI
• MT-Bench Evaluation
• ExLlama V2 Backend
• GPTQ 4-bit Inference
• AWQ 4-bit Inference

Related Pages

Related topics: Overview and Distributed Serving Architecture, Training Pipeline and Model Support

Serving: Workers, Web UI, and APIs

FastChat's serving subsystem is the production backbone of the project. It powers Chatbot Arena (lmarena.ai), serving over 10 million chat requests for 70+ LLMs, and provides a distributed, multi-model stack that can run locally, in containers, or across clouds. Source: README.md.

Purpose and Scope

The serving layer exposes three primary capabilities to end users and developers:

1. Model workers that host one or more LLM backends (Vicuna, Llama 2, Mistral, Qwen, etc.).
2. Web UIs (Gradio-based) for single-model chat, multi-model switching, and side-by-side Arena battles.
3. OpenAI-compatible RESTful APIs that make FastChat a drop-in replacement for the OpenAI Python SDK and curl.

The subsystem is intentionally decoupled: workers, the controller, and the web server run as independent processes, so they can scale and fail independently. Source: README.md.

High-Level Architecture

FastChat organizes serving into three roles — a Controller (registry and router), one or more Model Workers (inference hosts), and one or more Web/API Servers (front ends).

flowchart LR
    User[User / SDK Client] -->|HTTP| WebUI[Gradio Web Server]
    User -->|OpenAI-compatible API| APISrv[FastChat API Server]
    WebUI -->|dispatch| Ctrl[Controller]
    APISrv -->|dispatch| Ctrl
    Ctrl -->|heartbeat / register| W1[HF Worker<br/>Vicuna-7B]
    Ctrl -->|heartbeat / register| W2[vLLM Worker<br/>Llama-2-13B]
    Ctrl -->|heartbeat / register| W3[SGLang Worker<br/>Vision LLM]
    Ctrl -->|heartbeat / register| W4[LightLLM Worker]
    W1 -->|tokens| WebUI
    W2 -->|tokens| APISrv
    W3 -->|tokens| WebUI
    W4 -->|tokens| APISrv

The three-tier split is the key design choice that lets FastChat mix inference backends (Hugging Face transformers, vLLM, SGLang, LightLLM, MLX, ExLlama V2, GPTQ, AWQ) without changing the front end. Source: README.md.

Model Workers

Standard launch sequence

The README documents the canonical three-step bring-up. Source: README.md.

# 1. Start the controller (registry/router)
python3 -m fastchat.serve.controller

# 2. Start one or more model workers (self-register with the controller)
python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5

# 3. Start the Gradio web server
python3 -m fastchat.serve.gradio_web_server

Once the worker prints Uvicorn running on ..., it is live. The README recommends python3 -m fastchat.serve.test_message --model-name vicuna-7b-v1.5 to verify registration end-to-end. Source: README.md.

Backend options

The Hugging Face worker remains the default, but the README explicitly states the vLLM backend should be preferred for "high-throughput batched serving." Source: README.md.

Hardware and memory knobs

The README documents several deployment-relevant flags. Source: README.md.

• Single GPU — --model-path accepts a local folder or a HF repo; ~14 GB for Vicuna-7B, ~28 GB for Vicuna-13B.
• Multi-GPU — --num-gpus 2 plus optional --max-gpu-memory 8GiB to leave headroom for activations and longer contexts.
• CPU only — --device cpu (~30 GB RAM for 7B, ~60 GB for 13B).
• Intel / AMD / Ascend — --device xpu and --device npu are supported via the respective PyTorch adapters.
• Low VRAM — --load-8bit (bitsandbytes) halves memory; adding --cpu-offloading spills excess weights to CPU.

Web UI

The Gradio web server is the primary user-facing surface. There are two entry points. Source: README.md.

• python3 -m fastchat.serve.gradio_web_server — single-model chatbot UI.
• python3 -m fastchat.serve.gradio_web_server_multi — multi-tab variant that includes Chatbot Arena battle tabs.

If models do not appear in the UI, the README recommends rebooting the Gradio web server (workers may have registered before the web server queried the controller). Source: README.md.

Custom UIs are supported through docs/third_party_ui.md, and vision-language UIs were added in v0.2.36 alongside the SGLang vision worker. Source: README.md and release notes.

OpenAI-Compatible APIs

FastChat exposes an OpenAI-compatible REST surface so existing OpenAI clients work without modification. The README explicitly notes compatibility with both the openai-python SDK and raw cURL, and a Colab notebook (playground/FastChat_API_GoogleColab.ipynb) demonstrates the free-tier execution path. Source: README.md.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="vicuna-7b-v1.5",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Detailed reference lives in docs/openai_api.md, and the supported parameters mirror the upstream OpenAI schema (with image input added in v0.2.36). Source: README.md.

Common Failure Modes and Gotchas

Several recurring issues surface in the community evidence and the README itself:

• Workers not visible in the UI — the web server cached the worker list before registration finished. Reboot gradio_web_server after workers are up. Source: README.md.
• OpenAI stop parameter ignored — issue #1048 reports that since v0.2.5, the stop field is read from conv.stop_str rather than the request body, breaking clients that set per-request stop sequences.
• Out of memory — use --load-8bit, --cpu-offloading, or switch to a quantized backend (GPTQ, AWQ, ExLlama V2). Source: README.md.
• Multi-GPU imbalance — set --max-gpu-memory to leave KV-cache / activation room. Source: README.md.

See Also

• docs/server_arch.md — deeper architecture notes for the serving subsystem.
• docs/openai_api.md — full OpenAI-compatible API reference.
• docs/vllm_integration.md — vLLM worker setup.
• docs/model_support.md — how to add a new model and its conversation template.
• fastchat/llm_judge/README.md — using the served workers to generate MT-bench answers and judgments.
• playground/test_embedding/README.md — embedding-based ML examples that reuse served models.

Related Pages

Related topics: Overview and Distributed Serving Architecture, Serving: Workers, Web UI, and APIs, Evaluation, Datasets, and Chatbot Arena Monitoring

Training Pipeline and Model Support

FastChat is an open platform that covers the full lifecycle of large language model chatbots: training, serving, and evaluation. The training pipeline produces state-of-the-art models such as Vicuna, while the model support layer makes it possible to run dozens of base models — Llama 2, Alpaca, Falcon, GPT4ALL, T5 variants, and more — through a unified command-line interface. This page describes how those two pillars fit together and how to invoke them.

Source: README.md

Training Pipeline

Purpose and Data Conventions

The training pipeline is designed to fine-tune a Llama-family base model on user-shared conversations. According to the project README, Vicuna was created by fine-tuning a Llama base model on approximately 125K conversations sourced from ShareGPT.com. The pipeline expects HTML-cleaned, markdown-converted, and length-segmented conversations. Because ShareGPT itself is not redistributed, the repository ships a data/dummy_conversation.json file so users can validate the format before plugging in private data.

Source: README.md Source: data/dummy_conversation.json

Training Entry Points

The fastchat/train directory exposes multiple entry points so that the same supervised fine-tuning recipe can run on different backends without rewriting data loading or loss handling. The canonical Llama fine-tune lives in train.py, which is described in the README as "based on Stanford Alpaca" and dropped in as a drop-in replacement for that script.

Source: fastchat/train/train.py Source: README.md

The README also calls out the failure modes that motivate this split: out-of-memory during FSDP initialization (linked to a Hugging Face Transformers issue), out-of-memory during model checkpoint saving (linked to a PyTorch issue), and a recommendation to use the xFormers variant for the memory-efficient FlashAttention implementation.

Source: README.md

Experiment Tracking and Cloud Execution

Training scripts accept the standard --report_to argument so users can wire up Tensorboard, MLflow, or Weights & Biases. The README also documents SkyPilot integration for running the same training job on AWS, GCP, Azure, or Lambda with managed spot instances, so that large runs are not tied to a single on-prem box.

Source: README.md

flowchart LR
    A[ShareGPT conversations] --> B[HTML to markdown + filter]
    B --> C[data/dummy_conversation.json schema]
    C --> D[train.py / train_mem.py]
    D --> E{train_lora.py?}
    E -- yes --> F[LoRA adapter weights]
    E -- no --> G[Full Vicuna weights]
    F --> H[fastchat.serve.cli]
    G --> H
    H --> I[fastchat.serve.model_worker]
    I --> J[Controller + Gradio / OpenAI API]

The diagram shows the path from raw conversation data, through a fine-tune variant, into a model worker that the controller can route requests to.

Model Support

Supported Model Registry

FastChat advertises support for Llama 2, Vicuna, Alpaca, Baize, ChatGLM, Dolly, Falcon, FastChat-T5, GPT4ALL, Guanaco, MTP, OpenAssistant, OpenChat, RedPajama, StableLM, WizardLM, and xDAN-AI. Release notes confirm ongoing additions: Dolphin was added in v0.2.35, Yi support in v0.2.34, OpenChat 3.5 in v0.2.33, and stable-vicuna in v0.2.34. The full model list and instructions for adding a new model live in docs/model_support.md.

Source: README.md Source: docs/model_support.md

Vicuna Weight Bundles

The README publishes a ready-to-run table that pairs each Vicuna bundle with a one-line CLI command. Two sizes are highlighted as the canonical downloads:

The README warns that transformers>=4.31 is required for the 16K variants. A community issue also clarifies that FastChat-T5's advertised "4K context" actually splits into 2K input + 2K output, which matters when configuring --max-new-tokens for long-context inference.

Source: README.md

Inference Backends

Once a model weight bundle is available, fastchat.serve.cli is the unified entry point. The README documents the major runtime switches:

• Single GPU: defaults to roughly 14GB VRAM for Vicuna-7B and 28GB for Vicuna-13B.
• Multi-GPU: --num-gpus N activates model parallelism; --max-gpu-memory overrides the per-GPU cap so that more room is left for activations.
• CPU: --device cpu requires roughly 30GB for Vicuna-7B and 60GB for Vicuna-13B.
• Intel XPU: --device xpu after installing the Intel Extension for PyTorch.
• Ascend NPU: --device npu after sourcing the CANN environment.
• 8-bit compression: --load-8bit halves VRAM, allowing Vicuna-13B to fit on a single 16GB card such as the RTX 3090, T4, or AMD RX 6800 XT.

Source: README.md

For batched, high-throughput serving, the README points to a separate vLLM integration in docs/vllm_integration.md, and the MT-bench evaluation pipeline demonstrates an OpenAI-compatible endpoint that can be backed by vLLM for up to 20x faster answer generation.

Source: README.md Source: fastchat/llm_judge/README.md

From CLI to Serving

After validation via the CLI, the same weights are promoted into the multi-model serving system by starting a model worker with python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5. The worker self-registers with the controller, after which the Gradio web server (fastchat.serve.gradio_web_server) or the OpenAI-compatible API (fastchat.serve.api) can route traffic to it. If the model does not appear in the UI, the README recommends rebooting the Gradio web server so it re-fetches the worker registry.

Source: README.md

Common Failure Modes

Several issues surface repeatedly in the community context and the README's troubleshooting section:

1. Context length misconfiguration. FastChat-T5 does not accept a single 4K prompt; split the budget between input and output tokens.
2. Stop-string override. Between v0.2.3 and v0.2.5, the OpenAI-compatible API stopped honoring the per-request stop parameter and started using conv.stop_str directly, which breaks clients that rely on dynamic stop sequences.
3. Worker spoofing. The controller's /register_worker endpoint accepts an attacker-controlled address with no authentication, so production deployments must not expose the controller port to untrusted networks.
4. Memory pressure during training. The README explicitly points to two upstream issues for FSDP and saving OOMs and recommends the xFormers variant of the training script.

Source: README.md

See Also

• Architecture overview
• Model support registry
• Training guide
• vLLM integration
• MT-Bench evaluation
• OpenAI-compatible API

Related Pages

Related topics: Serving: Workers, Web UI, and APIs, Training Pipeline and Model Support

Evaluation, Datasets, and Chatbot Arena Monitoring

1. Overview

FastChat is an open platform for training, serving, and evaluating large language model (LLM) based chatbots. It powers Chatbot Arena (lmarena.ai), which has served over 10 million chat requests across 70+ LLMs and collected more than 1.5M human votes to compile an online LLM Elo leaderboard (README.md:1-9). The project bundles three tightly linked subsystems for quality assurance and benchmarking:

• MT-bench / LLM-as-a-judge evaluation pipeline under fastchat/llm_judge/ (Source: fastchat/llm_judge/README.md:1-15).
• Public conversation and judgment datasets released by the LMSYS team on Hugging Face.
• Chatbot Arena monitoring utilities under fastchat/serve/monitor/classify/ for runtime content classification.

MT-bench is the recommended benchmark for measuring chat quality and replaces the older 80-question Vicuna eval suite (README.md:148-156).

2. MT-Bench Evaluation Pipeline

The MT-bench workflow is a three-stage pipeline: answer generation, LLM-based judgment, and score reporting. Optional agreement computation quantifies how closely an LLM judge matches human annotators.

flowchart LR
    A[Model Weights<br/>local or HF repo] --> B[gen_model_answer.py<br/>or gen_api_answer.py]
    B --> C[data/mt_bench/<br/>model_answer/*.jsonl]
    C --> D[gen_judgment.py<br/>GPT-4 judge]
    D --> E[data/mt_bench/<br/>model_judgment/*.jsonl]
    E --> F[show_result.py<br/>MT-bench scores]
    G[mt_bench_human_judgments] --> H[compute_agreement.py]
    E --> H
    H --> I[Human vs GPT-4<br/>agreement %]

2.1 Generating Model Answers

gen_model_answer.py runs the candidate model locally over the 80 MT-bench questions and saves per-model JSONL output under data/mt_bench/model_answer/<MODEL-ID>.jsonl (Source: fastchat/llm_judge/README.md:40-55). Important flags include --num-gpus-per-model for model parallelism and --num-gpus-total to parallelize answer generation across multiple GPUs (Source: fastchat/llm_judge/README.md:55-58). To ensure FastChat applies the correct prompt template, consult docs/model_support.md.

For models hosted behind an OpenAI-compatible endpoint (e.g. vLLM, SGLang, LightLLM workers shipped in v0.2.36), gen_api_answer.py issues parallel HTTP calls:

vllm serve [MODEL-PATH] --dtype auto
python gen_api_answer.py --model [MODEL-NAME] \
    --openai-api-base http://localhost:8000/v1 --parallel 50

(Source: fastchat/llm_judge/README.md:82-95)

2.2 Generating LLM Judgments

gen_judgment.py invokes a strong judge model (typically gpt-4 or gpt-3.5-turbo) over the generated answers. Three grading modes are supported (Source: fastchat/llm_judge/README.md:106-130):

2.3 Reporting Scores

show_result.py aggregates judgments into a final scoreboard. The user can filter to a subset of model IDs or print all available entries (Source: fastchat/llm_judge/README.md:95-104).

2.4 Human–LLM Agreement

compute_agreement.py compares GPT-4 judgments against the lmsys/mt_bench_human_judgments dataset (3.3K human annotations spanning 6 models × 80 questions). The official Colab notebook reproduces the headline result that humans and GPT-4 agree in over 80% of cases, matching inter-human agreement (Source: fastchat/llm_judge/README.md:135-141).

3. Datasets Released by LMSYS

The platform publishes several datasets that pair naturally with the evaluation pipeline:

• Chatbot Arena Conversations – 33K real-world side-by-side battles with human preferences. Hosted at lmsys/chatbot_arena_conversations (Source: fastchat/llm_judge/README.md:144-148).
• MT-bench Human Annotation Dataset – 3.3K human judgments for agreement studies. Hosted at lmsys/mt_bench_human_judgments (Source: fastchat/llm_judge/README.md:144-148).
• LMSYS-Chat-1M – a million-scale corpus of real LLM conversations, released September 2023 (README.md:18-22).
• Dummy conversations – data/dummy_conversation.json lets users exercise the fine-tuning code without ShareGPT data, which is intentionally not redistributed (Source: README.md:165-170 and community discussion in issue #90).

For embedding-based experiments, the playground ships reference classifiers and similarity scripts that reuse the Vicuna tokenizer (playground/test_embedding/README.md:1-12).

4. Chatbot Arena Monitoring (Category Classifier)

Production Arena traffic is routed through a category classifier so conversations can be bucketed into leaderboard dimensions such as creative writing or coding. The subsystem lives under fastchat/serve/monitor/classify/. Workflow:

1. Pull pre-generated category benchmark data with git lfs from lmarena-ai/categories-benchmark-eval into a label_bench/ directory (Source: fastchat/serve/monitor/classify/README.md:3-22).
2. Subclass category.py for any new category.
3. Edit config.yaml and run python label.py --config config.yaml --testing to produce classification labels. Add --vision for multimodal categories (Source: fastchat/serve/monitor/classify/README.md:28-35).

API-based Arena models are defined in api_endpoint.json (e.g. gpt-4o-2024-05-13 with api_type: openai, api_type: anthropic_message, or api_type: gemini) and the gradio multi-tab server loads them at startup (Source: README.md:128-150 and fastchat/serve/api_provider.py).

5. Known Issues and Operational Pitfalls

• SSRF via /register_worker – The controller endpoint accepts an unauthenticated worker_name and immediately issues requests.post(worker_name + "/worker_get_status"), exposing the deployment to server-side request forgery and worker spoofing (community issue #3886).
• stop parameter regression – Since v0.2.5, fastchat/serve/api.py overrides conv.stop_str instead of honoring the per-request stop value, breaking OpenAI-API compatibility with clients that rely on runtime stop sequences (community issue #1048).
• FastChat-T5 4K context – The encoder/decoder caps at 2K tokens each; longer prompts require a different model class such as Llama variants (community issue #1711).
• Arena access – arena.ai may present a Cloudflare interstitial for legitimate users (community issue #3779).
• Missing models – If served models do not appear in the Gradio UI after launching workers, the recommended remediation is to restart gradio_web_server (Source: README.md:115-119).

See Also

• Server Architecture
• MT-bench paper
• Chatbot Arena paper
• OpenAI-compatible API
• vLLM Integration
• Third Party UI

Doramagic Pitfall Log

Found 10 structured pitfall item(s), including 2 high/blocking item(s). Top priority: Maintenance risk - Maintenance risk requires verification.

1. Maintenance risk: Maintenance risk requires verification

• Severity: high
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/lm-sys/FastChat/issues/1048

2. Security or permission risk: Security or permission risk requires verification

• Severity: high
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/lm-sys/FastChat/issues/1711

3. Identity risk: Identity risk requires verification

• Severity: medium
• Finding: Project evidence flags a identity risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: identity.distribution | github_repo:615882673 | https://github.com/lm-sys/FastChat

4. Capability evidence risk: Capability evidence risk requires verification

• Severity: medium
• Finding: README/documentation is current enough for a first validation pass.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: capability.assumptions | github_repo:615882673 | https://github.com/lm-sys/FastChat

5. Maintenance risk: Maintenance risk requires verification

• Severity: medium
• Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | github_repo:615882673 | https://github.com/lm-sys/FastChat

6. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: downstream_validation.risk_items | github_repo:615882673 | https://github.com/lm-sys/FastChat

7. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: no_demo
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: risks.scoring_risks | github_repo:615882673 | https://github.com/lm-sys/FastChat

8. Security or permission risk: Security or permission risk requires verification

• Severity: medium
• Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: community_evidence:github | https://github.com/lm-sys/FastChat/issues/3886

9. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: issue_or_pr_quality=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | github_repo:615882673 | https://github.com/lm-sys/FastChat

10. Maintenance risk: Maintenance risk requires verification

• Severity: low
• Finding: release_recency=unknown。
• User impact: May increase setup, validation, or first-run risk for the user.
• Recommended check: Reproduce the official install and quickstart path in an isolated environment.
• Evidence: evidence.maintainer_signals | github_repo:615882673 | https://github.com/lm-sys/FastChat

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using FastChat with real data or production workflows.

• Unauthenticated SSRF and worker/model spoofing via the controller /regis - github / github_issue
• Cloudflare Block Issue When Accessing arena.ai - github / github_issue
• FastChat-T5 4K context - github / github_issue
• The stop parameter in openai API doesn't work since v0.2.5 - github / github_issue
• Release v0.2.36 - github / github_release
• Release v0.2.35 - github / github_release
• Release v0.2.34 - github / github_release
• Release v0.2.33 - github / github_release
• Release v0.2.32 - github / github_release
• Release v0.2.30 - github / github_release
• Release v0.2.28 - github / github_release
• v0.2.25 - github / github_release