Index of every page on this site: llms.txt
All skills
jtydhr88 avatar

/comfyui-node-datatypes

@afb2d78
Run it yourself

ComfyUI data types - IMAGE, LATENT, MASK, CONDITIONING, MODEL, CLIP, VAE, AUDIO, VIDEO, 3D types, widget types, and custom types. Use when working with ComfyUI tensors, model types, or defining input/output data types.

Use this Skill: https://skilld.dev/gh/jtydhr88/comfyui-custom-node-skills/comfyui-node-datatypes

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈61 tokens for metadata: the name and description. ≈5k when used: this file.

Description uses 3.0% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈243 characters, or 3.0%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

ComfyUI Data Types

ComfyUI uses specific data types for node inputs and outputs. Understanding tensor shapes and data formats is essential.

Complete Type Reference

Tensor/Data Types

Type V3 Class Format Description
IMAGE io.Image torch.Tensor [B,H,W,C] float32 0-1 Batch of RGB images
MASK io.Mask torch.Tensor [H,W] or [B,H,W] float32 0-1 Grayscale masks
LATENT io.Latent {"samples": Tensor[B,C,H,W] or [B,C,T,H,W], "noise_mask"?: Tensor, "batch_index"?: list[int], "type"?: str} Latent space (4D image / 5D video)
CONDITIONING io.Conditioning list[tuple[Tensor, PooledDict]] Text conditioning with pooled outputs
AUDIO io.Audio {"waveform": Tensor[B,C,T], "sample_rate": int} Audio data
VIDEO io.Video VideoInput ABC Video data (abstract base class)
SIGMAS io.Sigmas torch.Tensor 1D, length steps+1 Noise schedule
NOISE io.Noise Object with generate_noise() Noise generator
LORA_MODEL io.LoraModel dict[str, torch.Tensor] LoRA weight deltas
LOSS_MAP io.LossMap {"loss": list[torch.Tensor]} Loss map
TRACKS io.Tracks {"track_path": Tensor, "track_visibility": Tensor} Motion tracking data
WAN_CAMERA_EMBEDDING io.WanCameraEmbedding torch.Tensor WAN camera embeddings
LATENT_OPERATION io.LatentOperation Callable[[Tensor], Tensor] Latent transform function
TIMESTEPS_RANGE io.TimestepsRange tuple[int, int] Range 0.0-1.0
DICT io.Dict dict Generic dictionary
ARRAY io.Array list Generic list/array
LAYERS io.Layers {"version": int, "canvas"?: (w, h), "layers": list[LayerItem]} Layer stack (v0.31+); LayerItem: image tensor, type: "raster", z_index, optional x, y, mask, name, opacity, blend_mode (io.Layers.BlendMode), visible, flip_h/v, rotation, w, h

Model Types (opaque, typically pass-through)

Type V3 Class Python Type
MODEL io.Model ModelPatcher
CLIP io.Clip CLIP
VAE io.Vae VAE
CONTROL_NET io.ControlNet ControlNet
CLIP_VISION io.ClipVision ClipVisionModel
CLIP_VISION_OUTPUT io.ClipVisionOutput ClipVisionOutput
STYLE_MODEL io.StyleModel StyleModel
GLIGEN io.Gligen ModelPatcher (wrapping Gligen)
UPSCALE_MODEL io.UpscaleModel ImageModelDescriptor
BACKGROUND_REMOVAL io.BackgroundRemoval BackgroundRemovalModel (e.g. BiRefNet)
LATENT_UPSCALE_MODEL io.LatentUpscaleModel Any
SAMPLER io.Sampler Sampler
GUIDER io.Guider CFGGuider
HOOKS io.Hooks HookGroup
HOOK_KEYFRAMES io.HookKeyframes HookKeyframeGroup
MODEL_PATCH io.ModelPatch Any
AUDIO_ENCODER io.AudioEncoder Any
AUDIO_ENCODER_OUTPUT io.AudioEncoderOutput Any
PHOTOMAKER io.Photomaker Any
POINT io.Point Any
FACE_ANALYSIS io.FaceAnalysis Any
BBOX io.BBOX Any
SEGS io.SEGS Any

3D Types

Type V3 Class Python Type Description
MESH io.Mesh MESH(vertices, faces) 3D mesh with vertices + faces tensors
VOXEL io.Voxel VOXEL(data) Voxel data tensor
SPLAT io.Splat SPLAT Gaussian splat data
FILE_3D io.File3DAny File3D Any supported 3D format
FILE_3D_GLB io.File3DGLB File3D Binary glTF
FILE_3D_GLTF io.File3DGLTF File3D JSON-based glTF
FILE_3D_FBX io.File3DFBX File3D FBX format
FILE_3D_OBJ io.File3DOBJ File3D OBJ format
FILE_3D_STL io.File3DSTL File3D STL format (3D printing)
FILE_3D_USDZ io.File3DUSDZ File3D Apple AR format
FILE_3D_PLY io.File3DPLY File3D PLY (point cloud / splat)
FILE_3D_SPLAT io.File3DSPLAT File3D .splat gaussian splat file
FILE_3D_SPZ io.File3DSPZ File3D Compressed splat (.spz)
FILE_3D_KSPLAT io.File3DKSPLAT File3D .ksplat format
FILE_3D_SPLAT_ANY io.File3DSplatAny File3D Any splat format
FILE_3D_POINT_CLOUD_ANY io.File3DPointCloudAny File3D Any point cloud format
SVG io.SVG SVG Scalable vector graphics
LOAD_3D io.Load3D Model3DDict (see below) 3D model with renders
LOAD_3D_ANIMATION io.Load3DAnimation Same as Load3D Animated 3D model
LOAD3D_CAMERA io.Load3DCamera CameraInfo (see below) 3D camera info
LOAD3D_MODEL_INFO io.Load3DModelInfo list[Model3DTransform] Per-model transforms (position/quaternion/scale)

Load3D.Model3DDict: {"image": str, "mask": str, "normal": str, "camera_info": CameraInfo, "recording"?: str, "model_3d_info"?: list[Model3DTransform]}

Load3DCamera.CameraInfo (right-handed, Y-up, camera looks down -Z): required keys position, target, zoom, cameraType ('perspective' | 'orthographic'); optional keys quaternion (camera world rotation), useCustomUp (bool, v0.40+: use the quaternion's up instead of world +Y), fov (vertical, degrees, perspective only), aspect, near, far, frustum (orthographic only: {left, right, top, bottom}).

Load3DModelInfo.Model3DTransform: {"position": dict, "quaternion": dict, "scale": dict} in world space.

Widget Types (create UI controls)

Type V3 Class Python Type Description
INT io.Int int Integer with min/max/step
FLOAT io.Float float Float with min/max/step/round
STRING io.String str Text (single/multi-line)
BOOLEAN io.Boolean bool Toggle with labels
COMBO io.Combo str Dropdown selection
COMBO (multi) io.MultiCombo list[str] Multi-select dropdown
COLOR io.Color str (hex) Color picker, default #ffffff
COLORS io.Colors list[str] (hex) Color palette (list of colors)
BOUNDING_BOX io.BoundingBox {"x": int, "y": int, "width": int, "height": int} Rectangle region
BOUNDING_BOXES io.BoundingBoxes list[{"x", "y", "width", "height", "metadata": dict}] Multiple labeled regions
CURVE io.Curve list[tuple[float, float]] Spline curve points
RANGE io.Range RangeInput (min/max + optional midpoint) Levels/range editor with gradient display
IMAGECOMPARE io.ImageCompare dict Image comparison widget
WEBCAM io.Webcam str Webcam capture widget
HISTOGRAM io.Histogram list[int] Histogram bin counts
VIDEO_EDIT io.VideoEdit {"trim"?: {start_time, duration}, "crop"?: {x, y, width, height}} Video trim/crop editor (features=["trim","crop"])
RESOLUTION_PREVIEW io.ResolutionPreview dict Live resolution preview bound to other widgets by name
COMPOSITOR io.Compositor dict (LayerState) Layer-editor state, default {}
CAMERA_INFO_STATE io.CameraInfoState dict Camera editor state (input only)

Special Types

Type V3 Class Description
* (ANY) io.AnyType Matches any type
COMFY_MULTITYPED_V3 io.MultiType Accept multiple specific types on one input
COMFY_MATCHTYPE_V3 io.MatchType Generic type matching across inputs/outputs
COMFY_AUTOGROW_V3 io.Autogrow Dynamic growing inputs
COMFY_DYNAMICCOMBO_V3 io.DynamicCombo Combo that reveals sub-inputs per option
COMFY_DYNAMICSLOT_V3 io.DynamicSlot Connection slot that reveals sub-inputs when connected (not yet used by core nodes)
COMFY_DYNAMICGROUP_V3 (internal _DynamicGroup) Repeated widget rows; not exported, unstable — do not use (frontend support since v1.57.0)
FLOW_CONTROL io.FlowControl Internal testing only
ACCUMULATION io.Accumulation Internal testing only

IMAGE Type

Images are torch.Tensor with shape [B, H, W, C]:

  • B = batch size (1 for single image)
  • H = height in pixels
  • W = width in pixels
  • C = channels (3 for RGB, values 0.0-1.0)
import torch
import numpy as np
from PIL import Image as PILImage

class ImageProcessor(io.ComfyNode):
    @classmethod
    def define_schema(cls):
        return io.Schema(
            node_id="ImageProcessor",
            display_name="Image Processor",
            category="image",
            inputs=[io.Image.Input("image")],
            outputs=[io.Image.Output("IMAGE")],
        )

    @classmethod
    def execute(cls, image):
        b, h, w, c = image.shape
        result = torch.clamp(image * 1.5, 0.0, 1.0)
        return io.NodeOutput(result)

Loading / Saving Images

from PIL import ImageOps

# Load from file → tensor
def load_image(path):
    img = PILImage.open(path)
    img = ImageOps.exif_transpose(img)   # fix rotation from camera EXIF
    if img.mode == "I":                  # handle 16-bit images
        img = img.point(lambda i: i * (1 / 255))
    img = img.convert("RGB")
    return torch.from_numpy(np.array(img).astype(np.float32) / 255.0).unsqueeze(0)

# Tensor → save to file
def save_image(tensor, path):
    if tensor.dim() == 4:
        tensor = tensor[0]
    PILImage.fromarray(np.clip(255.0 * tensor.cpu().numpy(), 0, 255).astype(np.uint8)).save(path)

# Batch operations
batch = torch.cat([img1, img2], dim=0)    # stack into batch
single = image[i]                          # extract from batch [H,W,C]
single_batch = image.unsqueeze(0)          # add batch dim [1,H,W,C]

MASK Type

torch.Tensor with shape [H, W] or [B, H, W], values 0.0-1.0.

# Invert mask
inverted = 1.0 - mask

# Mask ↔ Image conversion
alpha = mask.unsqueeze(0).unsqueeze(-1)                   # [1,H,W,1]
gray_mask = 0.299*img[:,:,:,0] + 0.587*img[:,:,:,1] + 0.114*img[:,:,:,2]
image_from_mask = mask.unsqueeze(-1).repeat(1, 1, 1, 3)  # [B,H,W,3]

# Ensure batch dim
if mask.dim() == 2:
    mask = mask.unsqueeze(0)  # [1, H, W]

LATENT Type

Dict with typed keys:

class LatentDict(TypedDict):
    samples: torch.Tensor       # [B, C, H, W] (image) or [B, C, T, H, W] (video) - required
    noise_mask: NotRequired[torch.Tensor]
    batch_index: NotRequired[list[int]]
    type: NotRequired[str]      # only for "audio", "hunyuan3dv2"

Image models (SD1.5, SDXL, SD3, Flux): 4D [B, C, H, W] — SD1.5/SDXL = 4 channels, SD3/Flux = 16 channels. Latent dimensions are 1/8 of pixel dims.

Video models (Hunyuan Video, Wan, Cosmos, LTX Video, Mochi): 5D [B, C, T, H, W] — T is the temporal (frame) dimension.

samples = latent["samples"]
# Check dimensionality:
if samples.ndim == 5:
    B, C, T, H, W = samples.shape   # video latent
else:
    B, C, H, W = samples.shape      # image latent

# Always preserve extra keys when modifying:
result = latent.copy()
result["samples"] = modified_samples

CONDITIONING Type

list[tuple[Tensor, PooledDict]] — a list of (cond_tensor, metadata_dict) pairs.

The PooledDict contains many optional keys for different models:

class PooledDict(TypedDict):
    pooled_output: torch.Tensor
    control: NotRequired[ControlNet]
    area: NotRequired[tuple[int, ...]]
    strength: NotRequired[float]           # default 1.0
    mask: NotRequired[torch.Tensor]
    start_percent: NotRequired[float]      # 0.0-1.0
    end_percent: NotRequired[float]        # 0.0-1.0
    guidance: NotRequired[float]           # Flux-like models
    hooks: NotRequired[HookGroup]
    # ... many more model-specific keys (SDXL, SVD, WAN, etc.)

Combine conditioning: result = cond_a + cond_b (list concatenation).

VIDEO Type

VideoInput is an abstract base class with methods:

class VideoInput(ABC):
    def get_components(self) -> VideoComponents    # images tensor + audio + frame_rate
    def save_to(self, path, format=VideoContainer.AUTO, codec=VideoCodec.AUTO, metadata=None,
                bit_depth=None,     # None keeps native depth
                crf=None,           # H.264/AV1 CRF; None -> 18 (H.264) / 24 (AV1)
                color_space=None,   # "sRGB" | "HDR" (BT.2020 HLG) | "HDR PQ"
                preset=None)        # H.264 encoder preset, e.g. "ultrafast"
    def as_trimmed(self, start_time=None, duration=None, strict_duration=False) -> VideoInput | None
    def as_cropped(self, x=0, y=0, width=0, height=0) -> VideoInput  # clamped, even-aligned; empty/full rect = self
    def get_stream_source(self) -> str | BytesIO
    def get_dimensions(self) -> tuple[int, int]     # (width, height)
    def get_duration(self) -> float                  # seconds
    def get_frame_count(self) -> int
    def get_frame_rate(self) -> Fraction
    def get_container_format(self) -> str
    def get_bit_depth(self) -> int                   # max component bits of the stream, e.g. 8/10/12 (default 8)
    def get_color_space(self) -> str                 # "sRGB" | "HDR" | "HDR PQ" | "auto" (unknown)

High bit depth and HDR (v0.34+) are supported end-to-end: loaders report get_bit_depth() / get_color_space(), save nodes preserve them. Codecs: Types.VideoCodec.AUTO/H264/AV1; containers: Types.VideoContainer.AUTO/MP4/MKV/WEBM.

Concrete implementations (via from comfy_api.latest import InputImpl):

  • VideoFromFile(file, *, start_time=0, duration=0, crop=None) — path or BytesIO; crop=(x, y, w, h)
  • VideoFromComponents(components, bit_depth=8, color_space="sRGB") — from tensors
  • VideoFromList(videos, complete_audio=None, codec=AUTO) — concatenation (v0.36+)

3D Types

File3D

from comfy_api.latest import Types

# File3D wraps a 3D file (disk path or BytesIO stream)
file_3d = Types.File3D(source="/path/to/model.glb", file_format="glb")
file_3d.format              # "glb"
file_3d.is_disk_backed      # True
file_3d.get_data()          # BytesIO
file_3d.get_bytes()         # raw bytes
file_3d.save_to("/output/model.glb")

MESH, VOXEL and SPLAT

from comfy_api.latest import Types

mesh = Types.MESH(vertices=torch.tensor(...), faces=torch.tensor(...))  # (B,N,3), (B,M,3)
voxel = Types.VOXEL(data=torch.tensor(...), voxel_colors=None, resolution=None)
splat = Types.SPLAT(...)   # gaussian splat data

MESH optional fields (pass by keyword — metallic_roughness was inserted before vertex_counts in v0.34, breaking positional calls): uvs (B,N,2), vertex_colors (B,N,3|4), texture (B,H,W,3 baseColor), metallic_roughness (B,H,W,3; G=roughness, B=metallic), vertex_counts/face_counts (B,) real lengths for padded batches (both or neither), unlit, normals (B,N,3), tangents (B,N,4), normal_map, occlusion_in_mr, material (dict), emissive.

Widget Types with Special Features

Color

io.Color.Input("color", default="#ff0000", socketless=True)
# Value is a hex string like "#ff0000"

Colors (palette)

io.Colors.Input("palette", default=["#ff0000", "#00ff00"], socketless=True)
# Value is list[str] of hex colors

BoundingBox

io.BoundingBox.Input("bbox",
    default={"x": 0, "y": 0, "width": 512, "height": 512},
    socketless=True,
    component="my_component",  # optional custom UI component
)
# Value is {"x": int, "y": int, "width": int, "height": int}

BoundingBoxes (multiple regions)

io.BoundingBoxes.Input("regions", default=[], socketless=True)
# Value is list of {"x": int, "y": int, "width": int, "height": int, "metadata": dict}

Curve

from comfy_api.input import CurveInput

io.Curve.Input("curve",
    default=[(0.0, 0.0), (1.0, 1.0)],  # linear
    socketless=True,
)
# In execute(), normalize the raw value first:
curve = CurveInput.from_raw(curve)

Range (levels editor)

from comfy_api.input import RangeInput

io.Range.Input("levels",
    default={"min": 0.0, "max": 1.0},
    display=None,                # widget visualization mode
    gradient_stops=None,         # gradient background for the slider
    show_midpoint=True,          # show gamma midpoint handle
    midpoint_scale=None,
    value_min=0.0, value_max=1.0,  # UI bounds
)
# In execute(), normalize with RangeInput.from_raw(value):
# .min_val, .max_val, .midpoint (gamma = -log2(midpoint), 0.5 = linear)
# .to_lut(size) generates a GIMP-style levels lookup table

MultiCombo

io.MultiCombo.Input("tags",
    options=["tag1", "tag2", "tag3"],
    default=["tag1"],
    placeholder="Select tags...",
    chip=True,  # show as chips
)
# Value is list[str]

Webcam

io.Webcam.Input("webcam_capture")
# Value is str (captured image data)

ImageCompare

io.ImageCompare.Input("comparison", socketless=True)
# Value is dict

Custom Types

# Simple: create inline custom type
MyData = io.Custom("MY_DATA_TYPE")

# Use in inputs/outputs
io.Schema(
    inputs=[MyData.Input("data")],
    outputs=[MyData.Output("MY_DATA")],
)

Advanced: @comfytype decorator

For custom types with type hints or custom Input/Output classes:

from comfy_api.latest._io import comfytype, ComfyTypeIO

@comfytype(io_type="MY_DATA_TYPE")
class MyData(ComfyTypeIO):
    Type = dict[str, Any]  # type hint for the data

RAM cache sizing: the default RAM-pressure cache measures CPU tensors in outputs (also inside lists/tuples/dicts); opaque objects count as a small default. For a custom class holding large tensors, add def _comfy_cache_tensors(self): return (self.a, self.b) (as core MESH/VOXEL/SPLAT do, v0.35+) so it is sized and evicted correctly.

AnyType / Wildcard

# Accept any single type (always a connection input, no widget)
io.AnyType.Input("anything")

# Accept specific multiple types
io.MultiType.Input("data", types=[io.Image, io.Mask, io.Latent])

# MultiType with widget override (shows widget for first type)
io.MultiType.Input(
    io.Float.Input("value", default=1.0),
    types=[io.Float, io.Int],
)

Imports from comfy_api.latest

from comfy_api.latest import (
    ComfyExtension,  # extension registration
    ComfyAPI,        # runtime API (progress, node replacement)
    io,              # all io types (io.Image, io.Schema, io.ComfyNode, etc.)
    ui,              # UI output helpers (ui.PreviewImage, ui.SavedImages, etc.)
    Input,           # Input.Image (ImageInput), Input.Audio, Input.Mask, Input.Latent, Input.Video
    InputImpl,       # InputImpl.VideoFromFile, .VideoFromComponents, .VideoFromList
    Types,           # Types.MESH, Types.VOXEL, Types.File3D, Types.VideoCodec, etc.
)

Tensor Safety

When checking if a tensor exists, always use is not None instead of truthiness:

# CORRECT
if image is not None:
    process(image)

# WRONG — multi-element tensors don't support bool()
if image:       # raises RuntimeError
    process(image)

# For boolean conditions on tensors, use .all() or .any()
if (mask > 0.5).all():
    ...

Type Conversion Patterns

# IMAGE [B,H,W,C] → MASK [B,H,W]
mask = 0.299 * image[:,:,:,0] + 0.587 * image[:,:,:,1] + 0.114 * image[:,:,:,2]

# MASK [B,H,W] → IMAGE [B,H,W,C]
image = mask.unsqueeze(-1).repeat(1, 1, 1, 3)

# Resize image tensor
import torch.nn.functional as F
resized = F.interpolate(
    image.permute(0, 3, 1, 2),  # [B,C,H,W] for interpolate
    size=(new_h, new_w), mode='bilinear', align_corners=False
).permute(0, 2, 3, 1)  # back to [B,H,W,C]

See Also

  • comfyui-node-basics - Node class structure and registration
  • comfyui-node-inputs - Input configuration details (widget options)
  • comfyui-node-outputs - Output types and UI outputs
  • comfyui-node-advanced - MatchType, MultiType, Autogrow, DynamicCombo

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts28d4 checks · Risk SAFE
  • Gen Agent Trust Hub28d

    The skill is a technical reference guide for ComfyUI data types, providing documentation and code examples for developers building custom nodes. No security issues were detected.

  • Socket1mo

    No alerts

  • Snyk1mo

    Risk: LOW · No issues

  • Runlayer8mo

    1 file scanned · No issues

Signed by skilld at afb2d78. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last month.

Steadyupdated last month

README badge

README badge for jtydhr88/comfyui-custom-node-skills/comfyui-node-datatypes