/skill

VideoDB

Original skill by ECC. Credit must stay with the author.

Use VideoDB to view, understand, search, and edit media.

Main Tasks

Desktop Capture

• Start or stop a desktop session.
• Capture the screen, mic, and system sound.
• Report what is seen and heard.
• Watch for a clear event and send an alert.
• Make a session summary.
• Return time marks and playable proof links.

Desktop capture may need the capture extra. If it is not installed or not supported on the system, explain the limit. Do not say capture started unless the SDK confirms it.

Upload and Stream

• Upload a local file or public URL.
• Upload a YouTube link.
• Return a playable web stream link.
• Change the codec, bit rate, frame rate, size, quality, or shape.

Index and Search

• Build speech, word, and scene indexes.
• Search for spoken words or things shown on screen.
• Return exact time marks.
• Return playable proof when the SDK gives it.
• Make a clip from search results.

Edit and Create

• Make, translate, or burn in subtitles.
• Add text, images, captions, or logos.
• Add music, voice-over, or dubbed speech.
• Join and trim media on a timeline.
• Make images, audio, or video when the user asks.

Live Streams

• Connect to an RTSP or live stream.
• Watch for clear sight or sound events.
• Return alert data with the event, time, and proof link when available.
• Stop cleanly when the user asks or when a set time limit is reached.

Do not keep a live task running with no stop rule. Ask for a time limit or use the one in the request.

Before You Run Code

Work in the user's project folder.

Load .env before connecting:

from dotenv import load_dotenv

load_dotenv(".env")

import videodb

conn = videodb.connect()

VideoDB reads VIDEO_DB_API_KEY from:

1. The current shell.
2. The .env file in the project folder.

If the key is missing or wrong, videodb.connect() raises AuthenticationError.

Never print, copy, log, or edit the API key. Do not open .env to read the key. Ask the user to set it if it is missing.

Use a short python -c command only for one or two simple lines. Use a heredoc for longer code:

python <<'PY'
from dotenv import load_dotenv

load_dotenv(".env")

import videodb

conn = videodb.connect()
coll = conn.get_collection()
print(f"Videos: {len(coll.get_videos())}")
PY

Do not make a script file when a short one-time command is enough.

Setup

When the user asks to set up VideoDB, install:

pip install "videodb[capture]" python-dotenv

If the capture extra fails on Linux, install the base package:

pip install videodb python-dotenv

The user must set the key in one of these ways:

export VIDEO_DB_API_KEY=your-key

Or add this line to the project .env file:

VIDEO_DB_API_KEY=your-key

A free key is available at console.videodb.io.

Do not ask the user to paste the key into chat.

Safe Work Steps

For each task:

1. Check the input path or URL.
2. Connect to VideoDB.
3. Get the collection.
4. Upload or find the media.
5. Build only the index needed for the task.
6. Check all time values before editing.
7. Run the task.
8. Return IDs, time marks, and stream links.
9. Explain empty results or limits in plain words.

Do not upload the same local file again if its VideoDB ID is already known.

Do not overwrite or delete media unless the user clearly asks.

Treat private URLs, RTSP links, and stream links as private data. Do not print login details or URL tokens.

Quick Guide

Get a Collection

coll = conn.get_collection()

Keep the returned video ID. It can help avoid a second upload.

Upload Media

# Public URL
video = coll.upload(url="https://example.com/video.mp4")

# YouTube
video = coll.upload(url="https://www.youtube.com/watch?v=VIDEO_ID")

# Local file
video = coll.upload(file_path="/path/to/video.mp4")

Before a local upload:

• Check that the file exists.
• Check that it is a file, not a folder.
• Give a clear error if it cannot be read.
• Do not guess a missing path.

A private or expired URL may fail. Ask for a working URL or local file. Do not ask for a password in chat.

Speech and Subtitles

video.index_spoken_words(force=True)
text = video.get_transcript_text()
stream_url = video.add_subtitle()

force=True avoids an error when a speech index already exists.

A video with no speech may return an empty transcript. Treat that as a valid result.

Check the subtitle language before translating or burning in text. If the user did not name a target language, ask before translation.

Search Spoken Words

from videodb.exceptions import InvalidRequestError

video.index_spoken_words(force=True)

try:
    results = video.search("product demo")
    shots = results.get_shots()
    stream_url = results.compile()
except InvalidRequestError as error:
    if "No results found" in str(error):
        shots = []
        stream_url = None
    else:
        raise

Do not call compile() when there are no results.

Report an empty result as "No matching moment was found." Do not call it an error.

Search Scenes

import re

from videodb import IndexType, SceneExtractionType, SearchType
from videodb.exceptions import InvalidRequestError

try:
    scene_index_id = video.index_scenes(
        extraction_type=SceneExtractionType.shot_based,
        prompt="Describe the visual content in this scene.",
    )
except Exception as error:
    match = re.search(r"id\s+([a-f0-9]+)", str(error), re.IGNORECASE)
    if match:
        scene_index_id = match.group(1)
    else:
        raise

try:
    results = video.search(
        query="person writing on a whiteboard",
        search_type=SearchType.semantic,
        index_type=IndexType.scene,
        scene_index_id=scene_index_id,
        score_threshold=0.3,
    )
    shots = results.get_shots()
    stream_url = results.compile()
except InvalidRequestError as error:
    if "No results found" in str(error):
        shots = []
        stream_url = None
    else:
        raise

index_scenes() has no force setting.

Use score_threshold=0.3 as a good first value. If there are too many weak matches, raise it. If there are no matches, lower it a little and say that you did so.

The old scene index ID is sometimes shown in an error. Reuse it only when the error clearly says that the index already exists. Do not hide other errors.

Timeline Editing

Check every clip before making a timeline:

def check_range(start, end, length):
    if start < 0:
        raise ValueError("start must be 0 or more")
    if start >= end:
        raise ValueError("start must be less than end")
    if end > length:
        raise ValueError("end must not be past the video length")

Then build the timeline:

from videodb.asset import TextAsset, TextStyle, VideoAsset
from videodb.timeline import Timeline

check_range(10, 30, video.length)

timeline = Timeline(conn)
timeline.add_inline(
    VideoAsset(asset_id=video.id, start=10, end=30)
)
timeline.add_overlay(
    0,
    TextAsset(
        text="The End",
        duration=3,
        style=TextStyle(fontsize=36),
    ),
)
stream_url = timeline.generate_stream()

Also check that:

• The video length is known.
• Each overlay starts at or after 0.
• Each overlay ends before the final timeline ends.
• Clip ranges do not overlap by mistake.
• Text is short enough to fit on screen.

A negative start may seem to work but can make a broken stream.

Transcode Video

from videodb import AudioConfig, TranscodeMode, VideoConfig

job_id = conn.transcode(
    source="https://example.com/video.mp4",
    callback_url="https://example.com/webhook",
    mode=TranscodeMode.economy,
    video_config=VideoConfig(
        resolution=720,
        quality=23,
        aspect_ratio="16:9",
    ),
    audio_config=AudioConfig(mute=False),
)

Before starting, confirm the needed size, shape, quality, and sound setting.

A callback URL sends the result to another service. Use one only when the user gave it or clearly asked for that flow. Do not make up a callback URL.

A job ID means the job started. It does not mean the job finished.

Change the Shape

reframe() can take several minutes for a long video.

Prefer a short part:

from videodb import ReframeMode

reframed = video.reframe(
    start=0,
    end=60,
    target="vertical",
    mode=ReframeMode.smart,
)

Common shapes:

# 9:16
vertical = video.reframe(start=0, end=60, target="vertical")

# 1:1
square = video.reframe(start=0, end=60, target="square")

# 16:9
wide = video.reframe(start=0, end=60, target="landscape")

# Custom size
custom = video.reframe(
    start=0,
    end=60,
    target={"width": 1280, "height": 720},
)

For a long video, trim it first. If the user already gave a callback URL, an async job may be used:

video.reframe(
    target="vertical",
    callback_url="https://example.com/webhook",
)

This call may return None. The final result goes to the callback.

Make an Image

image = coll.generate_image(
    prompt="A sunset over mountains",
    aspect_ratio="16:9",
)

Use only the prompt and shape needed by the user. Do not add hidden tracking, watermarks, or extra text.

Full Usage Example

Task: Find a whiteboard scene in a local video and make a short clip.

python <<'PY'
import re
from pathlib import Path

from dotenv import load_dotenv
from videodb import IndexType, SceneExtractionType, SearchType
from videodb.exceptions import AuthenticationError, InvalidRequestError

load_dotenv(".env")

import videodb

source = Path("/path/to/meeting.mp4")
if not source.is_file():
    raise FileNotFoundError(f"Video not found: {source}")

try:
    conn = videodb.connect()
except AuthenticationError as error:
    raise SystemExit("Set VIDEO_DB_API_KEY and try again.") from error

coll = conn.get_collection()
video = coll.upload(file_path=str(source))

try:
    scene_index_id = video.index_scenes(
        extraction_type=SceneExtractionType.shot_based,
        prompt="Describe the people, objects, and actions in this scene.",
    )
except Exception as error:
    text = str(error)
    if "already exists" not in text.lower():
        raise

    match = re.search(r"id\s+([a-f0-9]+)", text, re.IGNORECASE)
    if not match:
        raise

    scene_index_id = match.group(1)

try:
    results = video.search(
        query="A person writes on a whiteboard",
        search_type=SearchType.semantic,
        index_type=IndexType.scene,
        scene_index_id=scene_index_id,
        score_threshold=0.3,
    )
    shots = results.get_shots()

    if not shots:
        print("No matching moment was found.")
    else:
        stream_url = results.compile()
        print(f"Matches: {len(shots)}")
        print(f"Clip: {stream_url}")
except InvalidRequestError as error:
    if "No results found" in str(error):
        print("No matching moment was found.")
    else:
        raise
PY

Expected result:

• The file is checked before upload.
• A scene index is made or reused.
• Matching shots are found.
• A playable clip link is printed.
• No match is handled without a crash.

Error Rules

from videodb.exceptions import AuthenticationError, InvalidRequestError

try:
    conn = videodb.connect()
except AuthenticationError:
    print("Set or check VIDEO_DB_API_KEY.")

try:
    video = coll.upload(url="https://example.com/video.mp4")
except InvalidRequestError as error:
    print(f"Upload failed: {error}")

Catch only errors you can handle. Raise all other errors again.

Do not use one broad except Exception block unless the SDK gives no smaller error type. If a broad catch is needed, check the message for the exact known case and raise all other errors.

Common Problems

Example Requests

• "Upload this video and give me a stream link."
• "Find each time the speaker says product demo."
• "Find the scene where someone writes on a whiteboard."
• "Make a 20-second clip from the best match."
• "Add English subtitles to this video."
• "Turn the first minute into a vertical video."
• "Start desktop capture and alert me when a password field is shown."
• "Record this session, then make a short summary with time marks."