Index of every page on this site: llms.txt
All skills
openmontageinc avatar

/video-understand

@2e003e6
by openmontageincopenmontageinc/openmontage82k stars
11,135
Run it yourself

Understand video content locally using ffmpeg frame extraction and Whisper transcription. No API keys needed. Use when: (1) Understanding what a video contains, (2) Transcribing video audio locally, (3) Extracting key frames for visual analysis, (4) Getting video content without API keys.

Use this Skill: https://skilld.dev/gh/openmontageinc/openmontage/video-understand

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈77 tokens for metadata: the name and description. ≈755 when used: this file. ≈784 more on demand in 1 file.

Description uses 3.9% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈308 characters, or 3.9%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

video-understand

Understand video content locally using ffmpeg for frame extraction and Whisper for transcription. Fully offline, no API keys required.

Prerequisites

  • ffmpeg + ffprobe (required): brew install ffmpeg
  • openai-whisper (optional, for transcription): pip install openai-whisper

Commands

# Scene detection + transcribe (default)
python3 skills/video-understand/scripts/understand_video.py video.mp4

# Keyframe extraction
python3 skills/video-understand/scripts/understand_video.py video.mp4 -m keyframe

# Regular interval extraction
python3 skills/video-understand/scripts/understand_video.py video.mp4 -m interval

# Limit frames extracted
python3 skills/video-understand/scripts/understand_video.py video.mp4 --max-frames 10

# Use a larger Whisper model
python3 skills/video-understand/scripts/understand_video.py video.mp4 --whisper-model small

# Frames only, skip transcription
python3 skills/video-understand/scripts/understand_video.py video.mp4 --no-transcribe

# Quiet mode (JSON only, no progress)
python3 skills/video-understand/scripts/understand_video.py video.mp4 -q

# Output to file
python3 skills/video-understand/scripts/understand_video.py video.mp4 -o result.json

CLI Options

Flag Description
video Input video file (positional, required)
-m, --mode Extraction mode: scene (default), keyframe, interval
--max-frames Maximum frames to keep (default: 20)
--whisper-model Whisper model size: tiny, base, small, medium, large (default: base)
--no-transcribe Skip audio transcription, extract frames only
-o, --output Write result JSON to file instead of stdout
-q, --quiet Suppress progress messages, output only JSON

Extraction Modes

Mode How it works Best for
scene Detects scene changes via ffmpeg select='gt(scene,0.3)' Most videos, varied content
keyframe Extracts I-frames (codec keyframes) Encoded video with natural keyframe placement
interval Evenly spaced frames based on duration and max-frames Fixed sampling, predictable output

If scene mode detects no scene changes, it automatically falls back to interval mode.

Output

The script outputs JSON to stdout (or file with -o). See references/output-format.md for the full schema.

{
  "video": "video.mp4",
  "duration": 18.076,
  "resolution": {"width": 1224, "height": 1080},
  "mode": "scene",
  "frames": [
    {"path": "/abs/path/frame_0001.jpg", "timestamp": 0.0, "timestamp_formatted": "00:00"}
  ],
  "frame_count": 12,
  "transcript": [
    {"start": 0.0, "end": 2.5, "text": "Hello and welcome..."}
  ],
  "text": "Full transcript...",
  "note": "Use the Read tool to view frame images for visual understanding."
}

Use the Read tool on frame image paths to visually inspect extracted frames.

References

  • references/output-format.md -- Full JSON output schema documentation

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts11d3 checks · Risk SAFE
  • Gen Agent Trust Hub12d

    The skill is generally safe and performs local video processing using well-known tools like FFmpeg and OpenAI Whisper. It includes a feature to automatically install the Whisper package if requested. The primary security consideration is the potential for indirect prompt injection from transcribed video content.

  • Socket12d

    No alerts

  • Snyk11d

    Risk: LOW · No issues

Signed by skilld at 2e003e6. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 4 days ago.

Activeupdated 8 months ago

README badge

README badge for openmontageinc/openmontage/video-understand