> ## Documentation Index
> Fetch the complete documentation index at: https://rllm-org-rllm-19-feat-renderer-parser-backend.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Ray runtime environment

> How rLLM builds the Ray runtime_env that controls environment variables on training workers, and how to override it

When rLLM launches verl training, it calls `ray.init()` with a `runtime_env` that controls which environment variables every Ray worker sees. This page describes what `get_ppo_ray_runtime_env()` puts in that dict, the precedence order between rLLM defaults, your shell, and `ray job submit --runtime-env-json`, and the knobs you have for overriding each layer.

## Precedence

`get_ppo_ray_runtime_env()` composes the runtime env from three layers. Higher layers win on key conflicts.

| Priority | Source                 | Where it comes from                                                    |
| -------- | ---------------------- | ---------------------------------------------------------------------- |
| 1 (low)  | rLLM defaults          | `PPO_RAY_RUNTIME_ENV` in `rllm/trainer/verl/ray_runtime_env.py`        |
| 2        | Forwarded host env     | Variables in your shell whose names match a `FORWARD_PREFIXES` entry   |
| 3 (high) | Job-submit runtime env | `runtime_env` field of `--runtime-env-json` passed to `ray job submit` |

The first two layers are merged inside `get_ppo_ray_runtime_env()` itself: the host-forwarded vars overwrite the rLLM defaults via a plain `dict.update`. The third layer is honored by Ray's own merge inside `ray.init()`. To make that merge behave as a proper override (instead of raising on a key conflict), rLLM pops every key the job config sets from the dict it returns, so the job config's value is the one that survives.

<Note>
  This is the contract as of [PR #521](https://github.com/rllm-org/rllm/pull/521). Earlier versions did not look at the job-submit runtime env at all, so a `ray job submit --runtime-env-json=...` invocation could collide with rLLM's defaults and fail.
</Note>

## Default environment variables

These are the values rLLM sets at the lowest priority. Anything you forward from your shell or specify in `--runtime-env-json` will override them.

```python theme={null}
PPO_RAY_RUNTIME_ENV = {
    "env_vars": {
        "TOKENIZERS_PARALLELISM": "true",
        "NCCL_DEBUG": "WARN",
        "VLLM_LOGGING_LEVEL": "WARN",
        "VLLM_ALLOW_RUNTIME_LORA_UPDATING": "true",
        "CUDA_DEVICE_MAX_CONNECTIONS": "1",
        "VLLM_DISABLE_COMPILE_CACHE": "1",
        "NCCL_CUMEM_ENABLE": "0",
    },
}
```

The last two are present to work around known issues — vLLM's compile cache corruption ([vllm-project/vllm#31199](https://github.com/vllm-project/vllm/issues/31199)) and a hang during weight sync in disaggregated mode. Override them with care.

## Forwarding from your shell

Any variable in your shell whose name starts with one of these prefixes is forwarded to workers automatically:

| Category             | Prefixes                                                    |
| -------------------- | ----------------------------------------------------------- |
| Inference engines    | `VLLM_`, `SGL_`, `SGLANG_`                                  |
| HuggingFace          | `HF_`, `TOKENIZERS_`, `DATASETS_`                           |
| Training frameworks  | `TORCH_`, `PYTORCH_`, `DEEPSPEED_`, `MEGATRON_`             |
| CUDA / NCCL          | `NCCL_`, `CUDA_`, `CUBLAS_`, `CUDNN_`, `NV_`, `NVIDIA_`     |
| rLLM extension hooks | `RLLM_` (used by `RLLM_EXTRA_WORKER_SETUP_HOOK`; see below) |

So, to bump vLLM's logging level for a single run, you can just export it before launching training:

```bash theme={null}
export VLLM_LOGGING_LEVEL=DEBUG
export NCCL_DEBUG=INFO
python train.py ...
```

Both values will overwrite rLLM's defaults (`WARN`) on every worker.

## Suppressing forwarding with `RLLM_EXCLUDE`

`RLLM_EXCLUDE` is a comma-separated list that opts variables out of the host-forwarding step. Two forms are supported:

<CodeGroup>
  ```bash Specific variables theme={null}
  export RLLM_EXCLUDE="CUDA_VISIBLE_DEVICES,HF_TOKEN"
  ```

  ```bash Whole prefixes theme={null}
  export RLLM_EXCLUDE="VLLM*,CUDA*,NCCL_IB_DISABLE"
  ```
</CodeGroup>

A value containing `*` removes the corresponding entry from the forward-prefix list (so `VLLM*` drops the `VLLM_` prefix entirely). Values without `*` are matched against full variable names and excluded individually. The two forms can be combined.

<Tip>
  Reach for `RLLM_EXCLUDE` when a host variable is leaking into workers and breaking them — a stray `CUDA_VISIBLE_DEVICES` from your launcher is the classic case. It only affects the host-forwarding layer; the rLLM defaults and any `--runtime-env-json` keys are untouched.
</Tip>

## Overriding with `ray job submit`

If you launch training via `ray job submit`, the `runtime_env` field of `--runtime-env-json` wins over both rLLM defaults and host-forwarded vars. This is the right escape hatch when you need a different value on the cluster than on the submitting node.

```bash theme={null}
ray job submit \
  --runtime-env-json='{
    "env_vars": {
      "NCCL_DEBUG": "INFO",
      "VLLM_LOGGING_LEVEL": "DEBUG"
    },
    "working_dir": "."
  }' \
  -- python train.py ...
```

`get_ppo_ray_runtime_env()` reads this dict from `RAY_JOB_CONFIG_JSON_ENV_VAR` (which the Ray job runtime sets for you), pops the listed keys from its own `env_vars` so they cannot collide, and only sets `working_dir=None` when the job config does not specify a `working_dir`. Without that handling Ray's `ray.init()` would refuse to merge and raise unless you also exported `RAY_OVERRIDE_JOB_RUNTIME_ENV=1`.

## Worker process setup hook

Beyond environment variables, rLLM also wires a `worker_process_setup_hook` into the runtime env. Ray invokes this once at the start of every worker interpreter, before any user code runs. rLLM points it at `rllm.experimental.verl.patch.apply_all_verl_patches`, which monkey-patches the verl modules that the FSDP / vLLM workers re-import on startup.

This matters because verl is pinned (currently 0.7.1) and rLLM ships small backports for known issues — e.g. the qwen3-VL inplace-add fix from [verl#5881](https://github.com/volcengine/verl/pull/5881), the NCCL-deadlock fix on dynamic-batching from [verl#5750](https://github.com/volcengine/verl/issues/5750), and the jagged-NestedTensor `_ragged_idx` preservation fix from [verl#6127](https://github.com/volcengine/verl/pull/6127). Ray spawns each worker as a fresh Python interpreter that re-imports verl from disk, so a driver-side `import verl; verl.X = patched_X` never propagates. The setup hook runs the patches *inside* every worker.

If your `--runtime-env-json` already specifies a `worker_process_setup_hook`, rLLM steps out of the way (same precedence rule as for `env_vars`): the job-submitted hook wins and rLLM does not register its own. In that case you are responsible for calling `apply_all_verl_patches()` yourself if you want the verl backports.

### Adding your own setup hook (`RLLM_EXTRA_WORKER_SETUP_HOOK`)

Sometimes you need to apply environment-specific tweaks on every worker — for example, disabling cuDNN on a host whose cuDNN install is broken, or pointing Triton at a per-PID cache to avoid races between concurrent FSDP workers. These do not belong in rLLM itself (they are properties of one machine), so we expose an extension point instead of asking you to fork the runtime env.

Set the `RLLM_EXTRA_WORKER_SETUP_HOOK` environment variable to `"<absolute-path-to-file.py>:<function-name>"`. `apply_all_verl_patches` reads it on every worker, loads the file via `importlib.util.spec_from_file_location` (so it works even when the file is not on `sys.path` — e.g. lives under a gitignored `tmp/` directory), and calls the named function. Failures are logged but never re-raised.

```python title="my_local_setup.py" theme={null}
def apply():
    import torch
    torch.backends.cudnn.enabled = False  # workaround broken cuDNN install
    import os
    os.environ.setdefault("TRITON_CACHE_DIR", f"/tmp/triton-cache-{os.getpid()}")
```

```python title="train.py" theme={null}
import os
from pathlib import Path

# Set BEFORE importing rllm — get_ppo_ray_runtime_env() reads it via
# the RLLM_ forwarded prefix and propagates it to every Ray worker.
os.environ["RLLM_EXTRA_WORKER_SETUP_HOOK"] = f"{Path(__file__).parent / 'my_local_setup.py'}:apply"
```

The `RLLM_` prefix is in `FORWARD_PREFIXES`, so the variable is forwarded to workers without any extra config. The hook runs *after* rLLM's verl patches.

## Programmatic access

`AgentTrainer` calls `get_ppo_ray_runtime_env()` for you. If you need the same dict for a custom Ray actor, import it directly:

```python theme={null}
import ray
from rllm.trainer.verl.ray_runtime_env import get_ppo_ray_runtime_env

runtime_env = get_ppo_ray_runtime_env()
ray.init(runtime_env=runtime_env)
```

The returned dict always contains an `env_vars` key. It contains a `working_dir` key only when no job-submit `working_dir` is in effect.