# https://github.com/ray-project/ray Project Manual Generated at: 2026-07-06 19:36:20 UTC ## Table of Contents - [Ray Overview and Core Distributed Runtime](#page-1) - [Ray Data: Scalable Data Processing for ML](#page-2) - [AI Libraries: Train, Tune, RLlib, Serve, and LLM](#page-3) - [Deployment, Infrastructure, Observability, and Cross-Language Extensibility](#page-4) ## Ray Overview and Core Distributed Runtime ### Related Pages Related topics: [Ray Data: Scalable Data Processing for ML](#page-2), [AI Libraries: Train, Tune, RLlib, Serve, and LLM](#page-3), [Deployment, Infrastructure, Observability, and Cross-Language Extensibility](#page-4)
Related Source Files The following source files were used to generate this page: - [python/ray/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/__init__.py) - [python/ray/_private/worker.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/worker.py) - [python/ray/_raylet.pyx](https://github.com/ray-project/ray/blob/main/python/ray/_raylet.pyx) - [python/ray/remote_function.py](https://github.com/ray-project/ray/blob/main/python/ray/remote_function.py) - [python/ray/actor.py](https://github.com/ray-project/ray/blob/main/python/ray/actor.py) - [src/ray/raylet/raylet.cc](https://github.com/ray-project/ray/blob/main/src/ray/raylet/raylet.cc) - [src/ray/common/task/task_spec.h](https://github.com/ray-project/ray/blob/main/src/ray/common/task/task_spec.h) - [doc/source/ray-core/walkthrough.rst](https://github.com/ray-project/ray/blob/main/doc/source/ray-core/walkthrough.rst) - [doc/source/ray-core/key-concepts.rst](https://github.com/ray-project/ray/blob/main/doc/source/ray-core/key-concepts.rst) - [doc/source/ray-core/tasks.rst](https://github.com/ray-project/ray/blob/main/doc/source/ray-core/tasks.rst)
# Ray Overview and Core Distributed Runtime ## Purpose and Scope Ray is a distributed execution framework that lets Python programs seamlessly parallelize work across one or many machines. The top-level package exposes decorators, helpers, and the `init`/`shutdown` entry points used by every driver and worker. `ray.init()` connects a Python process to a Ray cluster (local or remote), configures runtime resources, and starts a background worker thread that talks to the local raylet over a Unix domain socket `Source: [python/ray/__init__.py:1-120]()`. Every node participating in a Ray program runs the same core components regardless of role: a `raylet` process that schedules work on that node, a `gcs_server` (Global Control Store) that holds cluster-wide metadata, and one or more `worker` processes that host Python user code `Source: [doc/source/ray-core/key-concepts.rst:1-90]()`. The framework is intentionally layered: high-level libraries (Ray Data, Ray Train, Ray Serve, Ray RLlib) all sit on top of the same two primitives — *tasks* and *actors* — exposed by Ray Core `Source: [doc/source/ray-core/walkthrough.rst:1-60]()`. ## System Architecture A Ray cluster is composed of a single **head node** and zero or more **worker nodes**. Every node runs two daemons: - A `raylet`, written in C++, which manages local scheduling, the plasma object store, and worker lifecycle `Source: [src/ray/raylet/raylet.cc:1-80]()`. - (Head-only) A `gcs_server`, which stores cluster metadata such as node liveness, actor locations, and job state `Source: [doc/source/ray-core/key-concepts.rst:90-160]()`. User code runs inside *worker* processes spawned on demand by the local raylet. Each worker embeds a Cython bridge (`_raylet.pyx`) that marshals task submissions, object references, and actor method calls into gRPC messages sent to the raylet `Source: [python/ray/_raylet.pyx:1-70]()`. The driver is just a special worker that happens to host the user's top-level script `Source: [python/ray/_private/worker.py:1-120]()`. ```mermaid flowchart LR Driver[Driver / Python script] -->|gRPC| Raylet1[Head raylet] Raylet1 --> GCS[(GCS
cluster metadata)] Worker1[Worker process] --> Raylet1 Raylet2[Worker node raylet] --> GCS Worker2[Worker process] --> Raylet2 Raylet1 <-->|heartbeat /
task forward| Raylet2 Raylet2 -.->|plasma| Plasma[(Object store
plasma)] ``` The two long-lived cross-node channels are the GCS for control-plane data and the plasma object store (and, more recently, the Ray Data/DataContext shared-memory paths) for bulk data transfer between workers on different nodes `Source: [src/ray/common/task/task_spec.h:1-90]()`. ## Core Abstractions ### Tasks A *task* is a stateless function invocation marked with `@ray.remote`. Calling `f.remote(x)` schedules the function on a worker and immediately returns an `ObjectRef` instead of a concrete value `Source: [python/ray/remote_function.py:1-120]()`. Each task is described by a `TaskSpec` protobuf that carries the function ID, arguments (as `ObjectRef`s or values), resource requirements, and placement hints `Source: [src/ray/common/task/task_spec.h:90-180]()`. The local raylet queues the task, places it on a worker that satisfies its CPU/GPU/accelerator demands, and runs it; results are written to plasma and the returned `ObjectRef` becomes resolvable with `ray.get()` `Source: [doc/source/ray-core/tasks.rst:1-80]()`. ### Actors An *actor* is a stateful worker bound to a class instance. `@ray.remote` on a class produces an `ActorClass`, and `.remote()` spawns the actor on a chosen node and returns an `ActorHandle` `Source: [python/ray/actor.py:1-150]()`. Unlike tasks, actor methods execute serially on the actor's dedicated worker; the raylet guarantees that all method calls to a given actor land on the same process, and that the actor's lifetime is independent of any single caller `Source: [doc/source/ray-core/key-concepts.rst:160-240]()`. Actor handles can be passed across tasks and other actors, which is how Ray implements distributed shared state. ### Objects and the Object Store Any Python value wrapped with `ray.put(v)` is serialized and stored in the local plasma object store, returning an immutable `ObjectRef` `Source: [python/ray/_private/worker.py:120-220]()`. Object refs are first-class values: they can be returned from tasks, stored on actors, and passed as arguments to other tasks. The runtime resolves transitive dependencies through `ObjectRef` IDs, so downstream tasks do not block until upstream results are materialized — a property the scheduler exploits for pipelining `Source: [doc/source/ray-core/tasks.rst:80-160]()`. ## Execution Flow and Lifecycle When a driver calls `f.remote(args)`: 1. The Cython bridge constructs a `TaskSpec` and posts it to the local raylet `Source: [python/ray/_raylet.pyx:70-160]()`. 2. The raylet resolves argument `ObjectRef`s (fetching from plasma if they are remote), picks a worker satisfying the task's resource request, and dispatches the task `Source: [src/ray/raylet/raylet.cc:80-200]()`. 3. The worker executes the Python function, writes the return value to plasma, and replies to the raylet with a new `ObjectRef` `Source: [python/ray/_private/worker.py:220-340]()`. 4. `ray.get(ref)` blocks on the driver until the corresponding object is available, deserializing it lazily `Source: [python/ray/_private/worker.py:340-440]()`. `ray.shutdown()` reverses this by detaching the worker from the local raylet, draining in-flight tasks, and tearing down the runtime context; in cluster mode it does not kill remote workers owned by other drivers `Source: [python/ray/_private/worker.py:440-540]()`. The same plumbing is what underpins Ray 2.x's stability push — issues like #54923 (Q3 2025 roadmap) explicitly call out hardening the core task and object lifecycles documented above as a foundation for higher-level libraries. --- ## Ray Data: Scalable Data Processing for ML ### Related Pages Related topics: [Ray Overview and Core Distributed Runtime](#page-1), [AI Libraries: Train, Tune, RLlib, Serve, and LLM](#page-3)
Related Source Files The following source files were used to generate this page: - [python/ray/data/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/data/__init__.py) - [python/ray/data/dataset.py](https://github.com/ray-project/ray/blob/main/python/ray/data/dataset.py) - [python/ray/data/read_api.py](https://github.com/ray-project/ray/blob/main/python/ray/data/read_api.py) - [python/ray/data/context.py](https://github.com/ray-project/ray/blob/main/python/ray/data/context.py) - [python/ray/data/_internal/execution/streaming_executor.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/execution/streaming_executor.py) - [python/ray/data/_internal/logical/optimizers.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/logical/optimizers.py) - [python/ray/data/_internal/execution/interfaces/execution_options.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/execution/interfaces/execution_options.py) - [python/ray/data/_internal/logical/rules/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/logical/rules/__init__.py)
# Ray Data: Scalable Data Processing for ML Ray Data is a distributed data processing library built on top of Ray Core that provides a scalable API for loading, transforming, and consuming large datasets in machine learning workloads. It is designed to handle data ingestion, preprocessing, and shuffling across a Ray cluster while integrating tightly with Ray Train and Ray Serve for end-to-end ML pipelines. The library is one of the major Ray 2.0 focus areas called out in the Ray 2.0 RFC (issue #22833), and stability improvements such as multi-dataset execution, automatic batch size selection for CPU map-batches, and default logical memory configuration are highlighted in the Ray 2.56.0 release notes. ## Architecture Overview Ray Data separates the user-facing `Dataset` API from its internal execution and optimization layers. The public entry point in `python/ray/data/__init__.py` exposes high-level constructors (`range`, `range_table`, `from_items`, `from_arrow`, `from_pandas`, `from_spark`, `read_*`, `read_datasource`) and transformations (`map`, `map_batches`, `flat_map`, `filter`, `groupby`, `join`, `sort`, `random_shuffle`, `repartition`, `split`, `union`, `limit`, `zip`). These compose into a lazy logical plan that is only resolved when a consuming operation such as `iter_batches`, `iter_rows`, `to_pandas`, `to_arrow`, `to_tf`, `to_torch`, `write_*`, `count`, `show`, or `stats` is called. Source: [python/ray/data/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/data/__init__.py). Internally, `Dataset` wraps a `LogicalOperator` DAG, a `Schema`, and an `Expression` subsystem. Transformations append new operators; reading sources returns a `FromX` operator that produces a `Block` stream. The `LogicalPlan` is passed through a `LogicalOptimizer` (defined in `python/ray/data/_internal/logical/optimizers.py`) which runs a sequence of registered rewrite rules before physical planning, enabling fusion, projection pushdown, and operator reordering. Source: [python/ray/data/_internal/logical/optimizers.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/logical/optimizers.py). ```mermaid flowchart LR A[read_* / from_*] --> B[LogicalPlan] C[map / map_batches / filter / groupby / ...] --> B B --> D[LogicalOptimizer + Rules] D --> E[StreamingExecutor] E --> F[iter_batches / to_torch / write_*] ``` Execution is driven by the streaming executor in `python/ray/data/_internal/execution/streaming_executor.py`, which converts operators into `PhysicalOperator` instances, manages a topology of `Block` queues, and schedules tasks onto Ray actors. The streaming model allows backpressure and overlapping I/O with compute; recent 2.56.0 changes explicitly mention reducing hidden buffering in `iter_batches` and shutting down the executor cleanly to prevent leaks. Source: [python/ray/data/_internal/execution/streaming_executor.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/execution/streaming_executor.py). ## Reading and Transforming Data The `read_api` module is the canonical surface for ingesting data from files, databases, and ML-aware formats. It supports row-oriented sources via `read_text` and `read_csv`, columnar sources via `read_parquet` and `read_arrow`, ML formats via `read_images`, `read_numpy`, `read_tfrecords`, and `read_mlflow`, and external systems via `read_databricks`, `read_datasource`, `read_unity_catalog`, and `read_snowflake`. Each reader accepts a `DataContext` for resource hints, parallelism, and schema configuration, and returns a lazy `Dataset`. Source: [python/ray/data/read_api.py](https://github.com/ray-project/ray/blob/main/python/ray/data/read_api.py). Transformations follow a familiar functional style. Stateless row operations use `map` or `flat_map`; vectorized batch operations use `map_batches`, which lets user functions receive Arrow, Pandas, or a torch tensor batch and operate on whole columns. Stateful or shuffled operations such as `groupby`, `sort`, `random_shuffle`, `repartition`, and `join` are routed through dedicated logical operators that the optimizer can fuse with adjacent steps when possible. The `Dataset` class also exposes `split`, `limit`, `union`, `zip`, and column projections (`select_columns`, `rename_columns`, `drop_columns`) for shaping the pipeline. Source: [python/ray/data/dataset.py](https://github.com/ray-project/ray/blob/main/python/ray/data/dataset.py). ## Configuration and Execution Options Runtime behavior is controlled through `DataContext`, defined in `python/ray/data/context.py`. It exposes knobs for the executor (`execution_options`), the default batch format (`default_batch_format`), shuffle strategy (`shuffle_strategy`), memory budgets (`_memory_usage_pinning_*`, `_max_num_blocks_in_streaming_gen`), and instrumentation such as `enable_auto_log_stats` and `print_on_exception`. Per-pipeline overrides can be applied by passing an `ExecutionOptions` value through `Dataset.execution_options`, which lets callers tweak `resource_limits`, `actor_locality_enabled`, `preserve_order`, and `verbose_progress`. Source: [python/ray/data/context.py](https://github.com/ray-project/ray/blob/main/python/ray/data/context.py). Optimizer rules live alongside `LogicalOptimizer` in `python/ray/data/_internal/logical/rules/__init__.py`. Rule registration, ordering, and fast-path checks are part of the public-ish optimizer API, and custom rule sets can be injected to experiment with new rewrites. The interaction between `DataContext`, `ExecutionOptions`, and rule selection determines how a logical plan is scheduled: for example, enabling shuffle fusion or restricting the number of concurrent operators affects how `random_shuffle` is fused with downstream readers. Source: [python/ray/data/_internal/logical/rules/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/logical/rules/__init__.py). ## Integration with Ray ML Ecosystem Ray Data is positioned as the data layer for the rest of Ray ML. `Dataset.iter_batches` and `Dataset.to_torch` are designed to feed Ray Train's `DataConfig.train_dataloader` and `DataConfig.valid_dataloader`, and `to_tf` mirrors that path for TensorFlow. The streaming executor overlaps ingestion with the training step by prefetching batches ahead of the trainer. This design is exactly the one called out in the Ray 2.0 RFC (issue #22833) for higher-level ML APIs, and Q3 2025 roadmap issue #54923 lists continued Data stability work — multiple datasets on a cluster, automatic batch size selection, and default logical memory configuration — that shipped in 2.56.0. The framework also serves as input for Ray Serve online inference through `to_pandas` / `iter_rows` for feature pipelines. ## Operational Considerations Because execution is lazy, datasets can be inspected with `schema()`, `count()`, `show()`, `stats()`, and `explain()` without materializing them, which is helpful when debugging a long pipeline. Failures in streaming execution are surfaced through `Dataset.errors()` and per-block logs in the `Ray Data logs` page of the Ray dashboard, both of which rely on `DataContext.enable_logging_to_driver` and the executor's stats subscribers. Per the release notes for 2.56.0, the executor now aggressively tears down resources on `iter_batches` shutdown to prevent OOMs, and default logical memory bounds prevent the planner from over-subscribing the cluster. Source: [python/ray/data/_internal/execution/streaming_executor.py](https://github.com/ray-project/ray/blob/main/python/ray/data/_internal/execution/streaming_executor.py). --- ## AI Libraries: Train, Tune, RLlib, Serve, and LLM ### Related Pages Related topics: [Ray Overview and Core Distributed Runtime](#page-1), [Ray Data: Scalable Data Processing for ML](#page-2), [Deployment, Infrastructure, Observability, and Cross-Language Extensibility](#page-4)
Related Source Files The following source files were used to generate this page: - [python/ray/train/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/train/__init__.py) - [python/ray/train/v2/__init__.py](https://github.com/ray-project/ray/blob/main/python/ray/train/v2/__init__.py) - [python/ray/train/v2/_internal/execution/controller/controller.py](https://github.com/ray-project/ray/blob/main/python/ray/train/v2/_internal/execution/controller/controller.py) - [python/ray/tune/tune.py](https://github.com/ray-project/ray/blob/main/python/ray/tune/tune.py) - [python/ray/tune/tuner.py](https://github.com/ray-project/ray/blob/main/python/ray/tune/tuner.py) - [python/ray/rllib](https://github.com/ray-project/ray/blob/main/python/ray/rllib) - [python/ray/serve](https://github.com/ray-project/ray/blob/main/python/ray/serve) - [python/ray/llm](https://github.com/ray-project/ray/blob/main/python/ray/llm)
# AI Libraries: Train, Tune, RLlib, Serve, and LLM Ray provides a layered ecosystem of "AI Libraries" built on top of Ray Core. These libraries target distinct stages of the ML lifecycle — distributed training, hyperparameter tuning, reinforcement learning, model serving, and large language model inference — and share Ray's task/actor primitives for distributed execution. ## Ray Train: Scalable Distributed Training `ray.train` exposes a framework-agnostic, fault-tolerant training API across PyTorch, TensorFlow, and HuggingFace Transformers via backend integrations (`TorchTrainer`, `TensorflowTrainer`, `HuggingFaceTrainer`). `ray.train` re-exports top-level utilities such as `train.report`, `train.Checkpoint`, and the `ScalingConfig`/`RunConfig` dataclasses that drive backend-agnostic configuration Source: [python/ray/train/__init__.py:1-50](). `ray.train.v2` is the next-generation trainer entry point. It simplifies configuration by exposing a single `TorchConfig` and a unified `trainer.fit()` surface, replacing several disparate v1 entry points. The v2 controller (`TrainController`) is a long-running actor that owns the worker group lifecycle, fault handling, and shutdown logic Source: [python/ray/train/v2/_internal/execution/controller/controller.py:1-80](). ```python from ray.train.v2 import Trainer from ray.train.torch import TorchConfig trainer = Trainer( num_workers=4, use_gpu=True, backend=TorchConfig(), scaling_config=..., ) trainer.fit(train_loop_per_worker=lambda: None) ``` ## Ray Tune: Hyperparameter Search and Experiment Orchestration Ray Tune is the hyperparameter optimization (HPO) library within Ray. The legacy entry point `ray.tune.run` (defined in `tune.py`) remains a thin compatibility shim that delegates to the modern `Tuner` class Source: [python/ray/tune/tune.py:1-40](). The modern API lives in `Tuner`, which decouples experiment specification from execution. Users compose a `TuneConfig` (defining the search space, scheduler, and metric) and pass a trainable (function or class) to `Tuner.fit()`. Scheduling algorithms such as `ASHAScheduler`, `PopulationBasedTraining`, and Bayesian search drivers are first-class citizens Source: [python/ray/tune/tuner.py:1-60](). - **Schedulers**: early-stopping and population-based schedules. - **Search algorithms**: random, grid, Optuna, Bayesian. - **Tune callbacks** for integration with Train, RLlib, and external dashboards. ## RLlib: Reinforcement Learning RLlib (`python/ray/rllib`) is the reinforcement learning library. It is built as a self-contained stack of RL-specific algorithms (PPO, SAC, IMPALA, APPO, etc.) that inherit from a common `Algorithm` base class. Algorithms are configured via `AlgorithmConfig` objects; training is launched by calling `algorithm.train()` or via the `tune.Tuner` integration with `rllib.trainable`. RLlib co-locates with `Tune` so that RL hyperparameter sweeps reuse the search/scheduler machinery above. ## Ray Serve: Online Model Serving Ray Serve (`python/ray/serve`) is the model-serving library. It is built around the concept of a `Deployment`, a route in a Ray actor that exposes a `@serve.deployment` wrapped callable or class. The `serve.run()` (or for v2, `serve.deploy()` with an `Application` config) entry point materializes the deployment graph across the cluster. Serve provides: - **Stateless and stateful deployments** with autoscaling (configurable `num_replicas`, `target_ongoing_requests`, `max_ongoing_requests`). - **Fast HTTP ingress** and gRPC adapters, plus Python request batching (`@serve.batch`). - **Multi-model composition** via `Ingress`, `Router`, and DAG/handle APIs. - **Optimistic autoscaling** in newer versions (an explicit Q3 2025 roadmap focus). ## Ray LLM: LLM Inference and Fine-tuning `python/ray/llm` is Ray's library for large language model serving, fine-tuning, and distributed inference. It wraps vLLM (and equivalent engines) inside Ray Serve deployments, exposing `LLMServing` configurations that orchestrate `PlacementGroup`-based prefill/decode disaggregation, LoRA adapters, and the v2 `Serve` config schemas. Fine-tuning pipelines are exposed via Ray Train backends and integrate with `rayllm.tune` for sweeps. ## How the Libraries Fit Together | Library | Lifecycle Stage | Core Primitive | Builds On | |---------|----------------|----------------|-----------| | Train | Training | `Trainer.fit` | Ray Core (actors/tasks) | | Tune | HPO | `Tuner.fit` | Train, RLlib, custom | | RLlib | Reinforcement Learning | `Algorithm.train` | Tune, Core | | Serve | Online inference | `Deployment` | Core (HTTP/gRPC) | | LLM | LLM inference / fine-tune | `LLMServing` | Serve + Train + Tune | Community discussions (#22833, "Ray 2.0 Feature Proposals") highlighted unified scheduling APIs and ecosystem interoperability across these five libraries as core usability goals, and Ray's Q3 2025 roadmap (#54923) treats Train, Serve, RLlib, and LLM as separate but tightly-coupled delivery streams — each must keep its own reliability guarantees while sharing Ray's execution substrate. --- ## Deployment, Infrastructure, Observability, and Cross-Language Extensibility ### Related Pages Related topics: [Ray Overview and Core Distributed Runtime](#page-1), [AI Libraries: Train, Tune, RLlib, Serve, and LLM](#page-3)
Related Source Files The following source files were used to generate this page: - [python/ray/autoscaler/_private/autoscaler.py](https://github.com/ray-project/ray/blob/main/python/ray/autoscaler/_private/autoscaler.py) - [python/ray/autoscaler/v2/autoscaler.py](https://github.com/ray-project/ray/blob/main/python/ray/autoscaler/v2/autoscaler.py) - [python/ray/autoscaler/v2/instance_manager/instance_manager.py](https://github.com/ray-project/ray/blob/main/python/ray/autoscaler/v2/instance_manager/instance_manager.py) - [python/ray/autoscaler/_private/kuberay/node_provider.py](https://github.com/ray-project/ray/blob/main/python/ray/autoscaler/_private/kuberay/node_provider.py) - [python/ray/cluster_utils.py](https://github.com/ray-project/ray/blob/main/python/ray/cluster_utils.py) - [python/ray/client_builder.py](https://github.com/ray-project/ray/blob/main/python/ray/client_builder.py) - [python/ray/_private/metrics_agent.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/metrics_agent.py) - [python/ray/_private/gcs_pubsub.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/gcs_pubsub.py) - [python/ray/_private/log_monitor.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/log_monitor.py) - [python/ray/_private/ray_constants.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/ray_constants.py) - [python/ray/_private/serialization.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/serialization.py) - [python/ray/remote_function.py](https://github.com/ray-project/ray/blob/main/python/ray/remote_function.py) - [python/ray/_private/worker.py](https://github.com/ray-project/ray/blob/main/python/ray/_private/worker.py) - [src/ray/common/id.h](https://github.com/ray-project/ray/blob/main/src/ray/common/id.h) - [python/ray/includes/common.pxd](https://github.com/ray-project/ray/blob/main/python/ray/includes/common.pxd)
# Deployment, Infrastructure, Observability, and Cross-Language Extensibility Ray is more than its task and actor APIs; the production surface area of the project spans cluster deployment, autoscaling, runtime observability, and polyglot extension points. This page surveys those subsystems, what role each plays, and how they are wired together in the source tree. The scope covers everything a user or operator needs to provision a cluster, scale it dynamically, observe its runtime, and extend Ray beyond Python. ## Cluster Deployment and Autoscaling Ray's cluster lifecycle is governed by a small set of builder and runner entry points. `ClientBuilder` in `python/ray/client_builder.py` is the canonical client-side entry: it parses a connection target (a local address, a remote cluster address, or a `cluster://` URI), validates it, and produces a `RayContext` used by `ray.init` to attach or start a cluster `Source: [python/ray/client_builder.py:1-40]()`. For programmatic and test usage, `python/ray/cluster_utils.py` exposes `Cluster` and `ClusterProc` helpers that wrap the head/worker startup commands in a single Python object, making it trivial to spin up disposable clusters in unit tests and notebooks `Source: [python/ray/cluster_utils.py:1-80]()`. Node provisioning on cloud backends is implemented as pluggable *node providers* consumed by the autoscaler. The legacy `StandardAutoscaler` lives at `python/ray/autoscaler/_private/autoscaler.py` and coordinates the polling loop that matches desired capacity against observed cluster state, reconciling via the provider's `create_node` / `terminate_node` / `non_terminated_nodes` methods `Source: [python/ray/autoscaler/_private/autoscaler.py:1-60]()`. The Kubernetes path is provided by `KubeRayNodeProvider`, which translates Ray's abstract node lifecycle into `RayCluster` custom-resource operations `Source: [python/ray/autoscaler/_private/kuberay/node_provider.py:1-80]()`. Ray 2.x introduces a redesigned autoscaler under `python/ray/autoscaler/v2/`. The `AutoscalerV2` entry point schedules periodic reconcile passes and delegates per-instance state to `InstanceManager`, which owns a state machine over `Instance`s (launching, running, stopping, stopped) and exposes `try_schedule` / `request_instance` for the higher-level scaler to call `Source: [python/ray/autoscaler/v2/autoscaler.py:1-80](), Source: [python/ray/autoscaler/v2/instance_manager/instance_manager.py:1-80]()`. The split between a stateless controller (`AutoscalerV2`) and a stateful instance manager (`InstanceManager`) is the architectural pattern that makes the new autoscaler easier to test and reason about. ## Runtime Observability Observability in Ray is built on three legs: metrics, structured events on GCS pubsub, and log forwarding. Metrics are exported via `PrometheusExporter` hosted by a per-node `MetricsAgent`, which is started by the dashboard agent and scrapes registered `Metric` objects from core workers on a configurable interval `Source: [python/ray/_private/metrics_agent.py:1-80]()`. The export endpoint, histogram bucket configuration, and agent registration are centralized as constants in `python/ray/_private/ray_constants.py`, including keys such as `METRIC_CONFIG_REGISTER_BACKEND` and `METRIC_PUBLISH_PERIOD_MS` that operators can tune via environment variables `Source: [python/ray/_private/ray_constants.py:1-120]()`. For event-style telemetry, `python/ray/_private/gcs_pubsub.py` implements a thin async wrapper around the GCS pubsub channel, allowing internal components (autoscaler, dashboard, job driver) to subscribe to job-, actor-, and node-lifecycle events without polling Redis. On the log side, `LogMonitor` reads per-worker stdout/stderr files written by the Raylet and forwards them to a centralized location, optionally indexed by the dashboard `Source: [python/ray/_private/log_monitor.py:1-60]()`. Together these three channels are what the Ray Dashboard's "Cluster", "Jobs", "Actors", and "Metrics" views consume. ## Cross-Language Extensibility Although the user-facing API is Python-first, the Ray core is implemented in C++ and exposed to Python through Cython. The contract layer is declared in `python/ray/includes/common.pxd`, which exposes core types such as `JobID`, `ActorID`, `TaskID`, `ObjectID`, `NodeID`, and `WorkerID` as opaque structs backed by `src/ray/common/id.h` `Source: [python/ray/includes/common.pxd:1-60](), Source: [src/ray/common/id.h:1-60]()`. These IDs are the lingua franca that every subsystem agrees on, which is what makes it possible for Python, C++, and Java clients to interoperate on the same cluster. Function-level extensibility is mediated by `RemoteFunction` in `python/ray/remote_function.py`, which captures user-supplied metadata (resources, scheduling strategy, max retries, runtime env) and serializes it into the task spec proto delivered to the Raylet `Source: [python/ray/remote_function.py:1-80]()`. The serialization path itself is in `python/ray/_private/serialization.py`, where `pickle` is wrapped in a registry-aware serializer that supports numpy zero-copy, PyArrow, and language-agnostic wire formats, with `cross_language` modes reserved for non-Python tasks `Source: [python/ray/_private/serialization.py:1-80]()`. Worker bootstrap is centralized in `python/ray/_private/worker.py`, which initializes the mode (LOCAL / SPILL / CROSS_LANG), connects to the Raylet, and exposes the global `worker` singleton that all APIs call into `Source: [python/ray/_private/worker.py:1-80]()`. The long-running Rust API request tracked in issue #20609 is exactly an effort to add another client on top of this same C++ core using the same ID types and serialization channel. ## How the Subsystems Compose | Concern | Entry Point | State Owner | |---|---|---| | Cluster startup | `ClientBuilder`, `Cluster` | head/worker processes | | Capacity | `AutoscalerV2` | `InstanceManager` | | Metrics | `MetricsAgent` | Prometheus exporter | | Events | `gcs_pubsub` | GCS pubsub channel | | Logs | `LogMonitor` | per-worker log files | | Polyglot | `RemoteFunction`, `serialization` | Raylet task specs | Operators typically touch the first column, library authors the last, and application users only the first row. The bounded, layered design is what lets the Q3 2025 roadmap focus on reliability and DX without rewriting the core (`Source`: community issue #54923). --- --- ## Pitfall Log Project: ray-project/ray Summary: Found 6 structured pitfall item(s), including 0 high/blocking item(s). Top priority: Capability evidence risk - Capability evidence risk requires verification. ## 1. Capability evidence risk - Capability evidence risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: README/documentation is current enough for a first validation pass. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: capability.assumptions | https://github.com/ray-project/ray ## 2. Maintenance risk - Maintenance risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow. - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/ray-project/ray ## 3. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: downstream_validation.risk_items | https://github.com/ray-project/ray ## 4. Security or permission risk - Security or permission risk requires verification - Severity: medium - Evidence strength: source_linked - Finding: no_demo - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: risks.scoring_risks | https://github.com/ray-project/ray ## 5. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: issue_or_pr_quality=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/ray-project/ray ## 6. Maintenance risk - Maintenance risk requires verification - Severity: low - Evidence strength: source_linked - Finding: release_recency=unknown。 - User impact: May increase setup, validation, or first-run risk for the user. - Evidence: evidence.maintainer_signals | https://github.com/ray-project/ray