---
name: mujoco-rendering
description: >-
  MuJoCo offscreen image rendering using mujoco.Renderer in Python and the MuJoCo C/C++ rendering pipeline (mjrContext, mjvScene, mjr_render, mjr_readPixels): RGB, metric depth maps, segmentation masks (mjtObj), MJCF offwidth/offheight framebuffer sizing, MUJOCO_GL (osmesa, egl), MuJoCo cameras (fixed, tracking, free, MjvCamera), and mjvOption flags/geomgroup. Use only for headless camera rendering, dataset recording, and visual observations in MuJoCo simulations. Don't use for generic OpenGL/EGL/OSMesa offscreen rendering, glReadPixels depth/segmentation post-processing, or synthetic dataset pipelines that do not use the `mujoco` library, or for interactive viewers/Studio (mujoco-studio) or ImGui controls (mujoco-gui).
metadata:
  publisher: google
title: mujoco-rendering
canonical_url: https://skilld.dev/gh/google-deepmind/mujoco/rendering
last_updated: 2026-11-19T21:15:49.000Z
---

> Index of every page on this site: [llms.txt](https://skilld.dev/llms.txt)

> **Skill from skilld.dev.** Follow the user's requested action.
> If the user asked to fork this Skill, follow the fork workflow below. Do not execute the Skill instructions while copying it.
> Otherwise, follow the instructions below for this session. You do not need to install anything.
>
> If the user asked to install this Skill, run `npx skilld install google-deepmind/mujoco/rendering`. Install writes the Skill files into the project, so every session loads them.
>
> ## Fork workflow
>
> A fork creates an editable local Skill with its original author and licence. The request authorizes copying and local installation.
> 1. Check `./skills/rendering`, the project lockfile, and selected Agent targets together. If the local directory or installed Skill exists, stop. Never overwrite an existing directory or Agent target.
> 2. Read [source metadata](https://skilld.dev/api/v1/skills/google-deepmind/mujoco/rendering) once. Use sourceUrl, sourceCommit, skillPath, sourceGone, and license. If the source is gone or its path is missing, stop. If license is null, read licence files at the source commit.
> 3. Fetch only the source commit into a temporary Git repository. Do not clone full history. Derive repository_url from sourceUrl, including repository renames. If sourceCommit is absent, resolve the sourceUrl ref once. Set source_commit to that actual commit. Run these commands in one shell call:
>
> ```sh
> git init --quiet "$temporary_dir"
> git -C "$temporary_dir" fetch --quiet --depth=1 "$repository_url" "$source_commit"
> git -C "$temporary_dir" checkout --quiet --detach FETCH_HEAD
> ```
>
> Read applicable licence declarations and notices at that commit. If copying is not permitted, report the restriction and stop.
> 4. Inspect source entries together, then copy the directory containing skillPath into `./skills/rendering`. Use the user's path if selected. Keep the original SKILL.md, relative links, scripts, binary assets, and executable modes. Exclude .git metadata. Reject symlinks and paths outside the Skill directory. After checking entries, use cp -a where available. A regular source directory needs no custom copy script. Do not save this page wrapper as SKILL.md.
> Preserve author credit, notices, and applicable licence files from repository or parent directories. Add PROVENANCE.md with the Skill page, source URL, actual commit, original path, and licence. Retain any existing PROVENANCE.md and record new provenance separately. Batch source inspection, copying, and provenance work where practical.
> 5. In the project root, run `skilld install ./skills/rendering --mode copy --plain`. If skilld is unavailable, use `npx skilld install ./skills/rendering --mode copy --plain`. This known command needs no help lookup. Install does not support --json. Use detected Agent targets, or add --agent for the targets the user selected. Install the local path, never the upstream selector. If installation fails, preserve the local copy and report the exact failure.
> 6. Confirm the local lockfile source and installed Agent copies once. Report the local path, actual commit, and Agent targets. After edits, reinstall the same local path. Upstream updates must not replace it. Do not publish or push unless the user asks.

# MuJoCo Rendering: Cameras, Contexts & Offscreen Buffers

> [!TIP]
>
> **Official MuJoCo documentation (`doc/*.rst`):** Read
> [`../../programming/rendering.rst`](../../programming/rendering.rst)
> (`.. _Rendering:`, `.. _FilamentRendering:`, `.. _ClassicRendering:`) and
> [`../../programming/visualization.rst`](../../programming/visualization.rst)
> (`.. _Visualization:`) directly when in a MuJoCo checkout; for large
> reference files like [`../../python.rst`](../../python.rst) (`.. _PyRender:`),
> [`../../XMLreference.rst`](../../XMLreference.rst), and
> [`../../APIreference/functions.rst`](../../APIreference/functions.rst), grep
> the literal `.. _Label:` anchor. Outside a checkout, the same `doc/*.rst` tree
> ships offline in `pip install mujoco` at
> `pathlib.Path(mujoco.__file__).parent / 'doc'`; otherwise fetch
> `https://mujoco.readthedocs.io/en/latest/<path>.html` (or `/en/stable/` for
> released pages; `functions.rst` renders inside
> `APIreference/APIfunctions.html`, and Sphinx anchors lowercase the label,
> e.g. `.. _PyRender:` → `#pyrender`) and browse repo paths (`../../../`) under
> `https://github.com/google-deepmind/mujoco/tree/main/`.
>
> **Related skills:**
>
> -   [mujoco-python](../python/SKILL.md) — Core physics simulation, stepping,
>     and state management.
> -   [mujoco-studio](../studio/SKILL.md) — Interactive simulation, Studio
>     viewer apps, and plugin architecture.
> -   [mujoco-gui](../gui/SKILL.md) — Dear ImGui UI design and custom controls.

## 1. Overview: Rendering Backends & Workflows

MuJoCo provides two built-in 3D rendering backends (compared side-by-side in
[programming/rendering.rst](../../programming/rendering.rst), `.. _Rendering:`):

-   **Classic OpenGL (`mujoco.Renderer` / C `mjr`)**: Immediate-mode OpenGL 1.5
    renderer (`mjv_updateScene` -> `mjr_render` -> `mjr_readPixels`), now in
    maintenance mode. Bundled out of the box in `pip install mujoco`, making
    `mujoco.Renderer` the simplest single-class helper for headless RGB, depth,
    and segmentation capture. See
    [programming/rendering.rst](../../programming/rendering.rst)
    (`.. _ClassicRendering:`) and [python.rst](../../python.rst)
    (`.. _PyClassicRender:`).
-   **Filament PBR (`mujoco.mjrf` & `mujoco.rendering.filament` / C `mjrf`)**:
    Retained-mode, asynchronous physically-based renderer supporting OpenGL,
    Vulkan, Metal, and WebGL (owns its own headless or windowed GPU context and
    arbitrary-size `mjrfRenderTarget` buffers). Recommended for PBR materials,
    image-based/photometric lighting, and new C++/Studio applications; see §7
    and [programming/rendering.rst](../../programming/rendering.rst)
    (`.. _FilamentRendering:`).
-   **Interactive Viewers (`mujoco.studio` / `mujoco.viewer`)**: Desktop or
    web-streamed interactive viewers; use [`mujoco-studio`](../studio/SKILL.md)
    instead.

## 2. Headless Contexts on Linux: `MUJOCO_GL`

On headless Linux systems (servers, containers), configure the OpenGL context
provider via the `MUJOCO_GL` environment variable **before importing `mujoco`**:

```python
import os
os.environ['MUJOCO_GL'] = 'egl'      # Hardware-accelerated offscreen on NVIDIA GPUs
# os.environ['MUJOCO_GL'] = 'osmesa' # Portable CPU software rasterization

import mujoco  # Must import AFTER setting MUJOCO_GL
```

Or from the command line:
```bash
MUJOCO_GL=egl python3 my_script.py
```

## 3. The Python Renderer Class (RGB, Depth & Segmentation)

`mujoco.Renderer` manages GL context creation, scene geometry synchronization,
and buffer readback. **Always use it as a context manager**:

```python
import mujoco
import numpy as np

model = mujoco.MjModel.from_xml_path('scene.xml')
data = mujoco.MjData(model)

with mujoco.Renderer(model, height=480, width=640) as renderer:
  mujoco.mj_forward(model, data)

  # 1. RGB Rendering
  renderer.update_scene(data, camera='overhead_cam')
  rgb = renderer.render()  # Shape: (480, 640, 3), dtype: uint8

  # 2. Metric Depth Rendering (distance in meters)
  renderer.enable_depth_rendering()
  renderer.update_scene(data, camera='overhead_cam')
  depth = renderer.render()  # Shape: (480, 640), dtype: float32
  renderer.disable_depth_rendering()

  # 3. Semantic Segmentation Rendering
  renderer.enable_segmentation_rendering()
  renderer.update_scene(data, camera='overhead_cam')
  seg = renderer.render()  # Shape: (480, 640, 2), dtype: int32
  # seg[:, :, 0] = object ID (-1 for background)
  # seg[:, :, 1] = object type (mjtObj enum, e.g. mjOBJ_GEOM)
  renderer.disable_segmentation_rendering()
```

### Rendering Inside Simulation Loops

Creating a `Renderer` allocates GPU framebuffers. In RL or evaluation loops,
**create the renderer once outside the loop**:

```python
with mujoco.Renderer(model, height=84, width=84) as renderer:
  for step in range(1000):
    mujoco.mj_step(model, data)
    if step % render_interval == 0:
      renderer.update_scene(data, camera='agent_cam')
      # Call .copy() because the renderer's internal buffer is reused
      frame = renderer.render().copy()
```

## 4. C++ Offscreen Rendering Pipeline (mjr & mjv)

In C++, offscreen rendering is performed using the `mjvScene` and `mjrContext`
APIs:

```cpp
#include <mujoco/mujoco.h>

// 1. Initialize visualization structures
mjvCamera cam;
mjv_defaultCamera(&cam);
cam.type = mjCAMERA_FIXED;
cam.fixedcamid = mj_name2id(m, mjOBJ_CAMERA, "overhead_cam");

mjvOption opt;
mjv_defaultOption(&opt);

mjvScene scn;
mjv_makeScene(m, &scn, 2000);  // Buffer up to 2000 geoms

// 2. Initialize offscreen rendering context
mjrContext con;
mjr_defaultContext(&con);
mjr_makeContext(m, &con, mjFONTSCALE_100);

// 3. Render frame
mjrRect viewport = {0, 0, 640, 480};
mjv_updateScene(m, d, &opt, nullptr, &cam, mjCAT_ALL, &scn);
mjr_render(viewport, &scn, &con);

// 4. Read pixels into user buffers
unsigned char rgb[640 * 480 * 3];
float depth[640 * 480];
mjr_readPixels(rgb, depth, viewport, &con);

// 5. Clean up
mjr_freeContext(&con);
mjv_freeScene(&scn);
```

## 5. Camera Definition & Placement

MuJoCo cameras look down their **negative Z-axis**:
-   **-Z**: Viewing forward direction
-   **+X**: Right
-   **+Y**: Up

### Defining Cameras in MjSpec

```python
# Fixed camera pointing down at ground plane
cam = spec.worldbody.add_camera(
    name='overhead',
    pos=[0, 0, 3.0],
    quat=[0.707, 0.707, 0, 0],  # 90 deg rotation around X
    fovy=60,
)

# Camera attached to robot wrist
wrist_cam = wrist_body.add_camera(
    name='wrist_cam',
    pos=[0.05, 0, 0],
    xyaxes=[0, -1, 0, 0, 0, -1],
    fovy=75,
)
```

### Free Camera (`MjvCamera`)

For custom interactive or orbital viewpoints:

```python
cam = mujoco.MjvCamera()
cam.type = mujoco.mjtCamera.mjCAMERA_FREE
cam.lookat[:] = [0, 0, 0.5]
cam.distance = 2.5
cam.azimuth = 90.0
cam.elevation = -20.0

renderer.update_scene(data, camera=cam)
```

## 6. Framebuffers & Scene Options

### Offscreen Framebuffer Resolution

The offscreen rendering resolution cannot exceed the dimensions pre-allocated in
the model. Adjust the buffer limit in the spec before compilation:

```python
spec.visual.global_.offwidth = 1920
spec.visual.global_.offheight = 1080
model = spec.compile()
```

### The Single `<visual>` Block Rule

MuJoCo uses only the **last** `<visual>` block defined in an XML file. Multiple
visual blocks will silently overwrite earlier settings:

```xml
<!-- ❌ WRONG — second visual block silently resets offwidth/offheight to default -->
<visual><global offwidth="1920" offheight="1080"/></visual>
<visual><headlight ambient="0.3 0.3 0.3"/></visual>

<!-- ✅ RIGHT — consolidate into one visual block -->
<visual>
  <global offwidth="1920" offheight="1080"/>
  <headlight ambient="0.3 0.3 0.3"/>
</visual>
```

### Geom Group Visibility & Visualization Flags

Control rendering layers via `MjvOption`:

```python
opt = mujoco.MjvOption()
opt.geomgroup[:] = False   # Hide all groups
opt.geomgroup[0] = True    # Show primary geometry (default group)
opt.geomgroup[1] = True    # Show secondary visuals

# Toggle visual debugging flags
opt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTFORCE] = True
opt.flags[mujoco.mjtVisFlag.mjVIS_TRANSPARENT] = True

renderer.update_scene(data, camera='overhead', scene_option=opt)
```

## 7. Filament Physically-Based Rendering (`mjrf`)

MuJoCo's Filament backend (`mjrfilament.h` in C/C++, `from mujoco import mjrf`
and `mujoco.rendering.filament` in Python) provides retained-mode PBR with
OpenGL, Vulkan, Metal, and WebGL support, arbitrary `mjrfRenderTarget` sizes
(independent of `<global offwidth offheight/>`), and asynchronous
`mjrfReadRequest` pixel readback:

-   **Python (`ModelScene` + `Renderer`)**: Create an `mjrf.Context`, bind
    model lights and renderables once via
    `mujoco.rendering.filament.model_scene.ModelScene`, update in place with
    `ms.update(data)`, and render into a NumPy target buffer via
    `mujoco.rendering.filament.renderer.Renderer` — see the complete example in
    [python.rst](../../python.rst) (`.. _PyFilamentRender:`).
-   **C/C++ & CMake**: Build with `-DMUJOCO_USE_FILAMENT=ON` (or
    `-DMUJOCO_USE_FILAMENT_MJR_COMPAT=ON` for drop-in `mjr_` compatibility) and
    consult [programming/rendering.rst](../../programming/rendering.rst)
    (`.. _reProgramming:`, `.. _reSupport:`).
-   **Photometric light-intensity gotcha**: If *any* `<light>` has positive
    `intensity`, Filament switches the entire scene from classic headlight +
    default environment lighting to photometric mode (turning off the headlight
    and darkening zero-intensity lights). For PBR materials, textures, and
    lighting authoring, read
    [programming/rendering.rst](../../programming/rendering.rst)
    (`.. _reAuthoring:`, `.. _reMaterials:`, `.. _reLighting:`) and inspect the
    reference PBR/IBL test scenes in `../../../test/render/testdata/`
    (`pbr.xml`, `lighting.xml`, `emissive_bloom.xml`, `reflect_pbr.xml`,
    `ibl.ktx`) and `../../../model/mug/mug.xml`.

## 8. Common Gotchas & Best Practices

1.  **GL context leaks**: Always close `mujoco.Renderer` or use a `with` block.
    Failing to free renderers in loops exhausts GL contexts.
2.  **Destruction order**: In C++, `mjr_freeContext` must be called before the
    underlying OpenGL context is destroyed (see
    [programming/rendering.rst](../../programming/rendering.rst),
    `.. _reContext:`).
3.  **Buffer re-use**: `renderer.render()` returns an internal buffer view. Call
    `.copy()` if storing frames in a list.
4.  **Camera lookup by name**: Pass `camera='camera_name'` or the integer ID.
    Avoid passing raw string IDs that are not present in the model.

## 9. Key References

### Documentation

-   [Rendering Guide (Filament `mjrf` & Classic OpenGL `mjr`)](../../programming/rendering.rst)
    (`.. _Rendering:`, `.. _FilamentRendering:`, `.. _ClassicRendering:`)
-   [Abstract Visualization (`mjvCamera`, `mjvOption`, `mjvScene`)](../../programming/visualization.rst)
    (`.. _Visualization:`, `.. _viCamera:`, `.. _viOption:`)
-   [Python Rendering Bindings](../../python.rst) (`.. _PyRender:`,
    `.. _PyFilamentRender:`, `.. _PyClassicRender:`)
-   [MJCF `<visual>` & `<camera>` Reference](../../XMLreference.rst)
    (`.. _visual-global:`, `.. _body-camera:`),
    [Modeling Cameras](../../modeling.rst) (`.. _CCamera:`), and
    [MuJoCo Menagerie](../../models.rst) (`.. _Menagerie:`)
-   [C Rendering API Functions](../../APIreference/functions.rst)
    (`.. _Visualization-api:`, `.. _OpenGLrendering:`,
    `.. _FilamentRenderingApi:`) and
    [C++ Render Sample](../../programming/samples.rst) (`.. _saRender:`)

### Source Code & Model Examples

-   [C API Headers](../../../include/mujoco/mujoco.h) and
    [`mjrfilament.h`](../../../include/mujoco/mjrfilament.h)
-   [C++ Offscreen & Filament Sample](../../../sample/render.cc) and
    [Filament PBR Test Scenes](../../../test/render/testdata/pbr.xml)
