---
name: media-depth
description: >
  Monocular depth estimation with open-source + commercial-safe models: Depth-Anything v2 (Apache 2.0, TikTok/ByteDance 2024, current SOTA small/base/large) via HuggingFace transformers, MiDaS v3.1 (MIT, Intel ISL classic with DPT/BEiT/Swin backbones). Extract depth maps from a single image or video, enable 2.5D parallax / Ken Burns effects, drive Blender/Unreal Z-depth passes, replace LiDAR capture, convert 2D → stereo 3D via depth-driven parallax. Use when the user asks to estimate depth, compute a depth map, run Depth Anything, run MiDaS, get a disparity map, animate a 2.5D parallax shot from a still, build a stereo pair from a 2D photo, extract Z-depth for compositing, drive a VFX pipeline with monocular depth, convert 2D video to side-by-side 3D, or produce a depth video for the Looking Glass / Blender camera tracker.
argument-hint: "[input] [output]"
---

# Media Depth

**Context:** $ARGUMENTS

Monocular depth estimation — estimate a per-pixel depth map from a single RGB image or video, no stereo rig or LiDAR required. Two actively maintained open-source model families with commercial-safe licenses: **Depth-Anything v2** (Apache 2.0, current SOTA, 2024) and **MiDaS v3.1** (MIT, Intel ISL, long-standing baseline).

## Quick start

- **Depth map from a photo (PNG):** → Step 3 (`depth.py image --model depthanything-v2`)
- **Depth sequence from a video:** → Step 4 (`depth.py video`)
- **2D photo → side-by-side stereo 3D pair:** → Step 5 (`depth.py stereo --baseline 6.5`)
- **Animate a 2.5D parallax / Ken Burns shot from a still:** → Step 6 (`depth.py parallax --frames 30`)
- **Pre-download model weights:** → Step 2 (`depth.py install <model>`)

## When to use

- Need a **Z-depth pass** to hand to Blender, Nuke, Fusion, After Effects, or Unreal (compositing, DoF, fog).
- Converting flat 2D footage to **side-by-side / over-under stereo 3D** without reshooting.
- Producing **parallax / 2.5D / "Ken Burns" motion** from stills for social reels or lyric videos.
- Building a **pseudo-LiDAR point cloud** for photogrammetry, volumetric capture prep, or robotics dev.
- Any workflow where "metric depth in meters" is *not* required — both default models here return **relative / inverse depth** normalized 0–1. For metric depth, read `references/models.md` on Depth-Anything v2 metric fine-tunes.
- Not for real-time (<10 ms) embedded use — pick MiDaS-small for the fastest CPU path but even that is ~50 ms on a modern laptop at 384 px.

## Step 1 — Install PyTorch + transformers

Depth-Anything v2 is distributed via HuggingFace `transformers` using the `pipeline("depth-estimation", ...)` API — no custom model code required. MiDaS also works through `transformers` (using the `DPTForDepthEstimation` class).

PEP 723 header in `scripts/depth.py` declares the full dependency set. Running via `uv run` handles an ephemeral venv. Manual install:

```bash
pip install torch transformers opencv-python numpy pillow
```

On Apple Silicon, PyTorch picks MPS automatically when you set `device="mps"`. On NVIDIA, `device="cuda"`. The helper auto-selects.

Verify the runtime:

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py install depthanything-v2 --size small
```

Downloads the `depth-anything/Depth-Anything-V2-Small-hf` weights (~100 MB) into `~/.cache/huggingface/hub/`.

## Step 2 — Pick a model

| Model                       | License       | Variants                          | Relative/Metric | Notes                                            |
|-----------------------------|---------------|-----------------------------------|-----------------|--------------------------------------------------|
| `depthanything-v2` (default)| Apache 2.0    | `small` / `base` / `large`        | Relative (inverse depth) | SOTA 2024, crisper edges than v1; metric fine-tunes exist for indoor/outdoor |
| `midas`                     | MIT           | `dpt-swinv2-tiny-256` / `dpt-beit-large-512` | Relative | Intel ISL v3.1, older, still robust             |

Pick by size vs accuracy:

- `depthanything-v2 --size small` — ~25 M params, ~40 fps 1080p on a 3090
- `depthanything-v2 --size base`  — ~97 M params, ~20 fps
- `depthanything-v2 --size large` — ~335 M params, ~8 fps, sharpest edges

Full per-model speed, VRAM, and license matrix in `references/models.md`. Read it when picking a model for batch video or embedded deployment.

## Step 3 — Depth map from a single image

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py image \
  --model depthanything-v2 --size small \
  --in photo.jpg --out depth.png
```

Outputs a 16-bit single-channel PNG where `0` = far, `65535` = near (the HuggingFace pipeline yields "inverse depth", higher = closer). Also writes `depth_preview.png` — a turbo-colormap 8-bit visualization for human inspection.

Useful flags:

- `--out-format npy` — raw `float32` numpy `.npy` (best fidelity for VFX tools)
- `--out-format exr` — 32-bit EXR (requires `opencv-python` built with OpenEXR; otherwise install `pip install OpenEXR imath` or use the `.npy` path and convert with `vfx-oiio` / `ffmpeg-ocio-colorpro`)
- `--invert` — flip so that `0` = near, `65535` = far (Blender / most DCCs expect this)

## Step 4 — Per-frame depth for video

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py video \
  --model depthanything-v2 --size small \
  --in input.mp4 --out depth.mp4
```

Writes a grayscale H.264 MP4 at the source fps where each pixel encodes depth (8-bit). For compositing fidelity prefer:

```bash
--out-format png-seq --out depth_frames/   # 16-bit PNG sequence, frame_%06d.png
--out-format exr-seq --out depth_exr/      # 32-bit EXR sequence (best for Nuke)
```

Gotcha: per-frame monocular depth is **temporally unstable** — the depth of a static object can flicker frame-to-frame. Use `--smooth temporal` for a light 3-tap EMA across frames, or hand the output to `ffmpeg-denoise-restore` (`hqdn3d` on the grayscale depth track) before consuming it downstream.

## Step 5 — 2D → stereo 3D (depth-driven parallax)

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py stereo \
  --model depthanything-v2 --size base \
  --in photo.jpg \
  --out-left left.png --out-right right.png \
  --baseline 6.5
```

`--baseline` is the interpupillary-distance shift in **pixels** (or millimeters at 96 dpi — it's an opinionated single scalar). Typical values:

- `3.0` — subtle parallax for phone viewing
- `6.5` — the physical human IPD in mm; gives a natural "window" look
- `12.0` — exaggerated for VR / anaglyph

The occlusion-aware parallax math (forward-warp with depth-sorted Z-buffer, then inpaint revealed background) is documented in `references/parallax-math.md` — read it when you need to tune the quality / hole-filling trade-off.

Pack side-by-side for a 3D TV / Meta Quest:

```bash
ffmpeg -i left.png -i right.png -filter_complex hstack sbs.png
```

## Step 6 — Animate a 2.5D parallax shot (Ken Burns style)

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py parallax \
  --image photo.jpg --depth depth.png \
  --out parallax.mp4 \
  --frames 120 --fps 30 \
  --motion orbit --amplitude 40
```

`--motion` presets:

- `orbit` — smooth circular camera orbit (best default)
- `zoom` — dolly in toward the nearest plane
- `pan-left` / `pan-right` — horizontal slide with depth-correct parallax
- `ken-burns` — slow zoom + slight pan

`--amplitude` is the max pixel shift of the nearest foreground. Values 20–80 work well for 1080p stills. Hole-fill is done via OpenCV `INPAINT_TELEA` on the revealed background — the reference doc covers why TELEA beats NS for this specific use case.

If you have both the `--image` and `--depth` already, parallax is a pure post-process (no model inference). Chain it after Step 3.

## Available scripts

- **`scripts/depth.py`** — subcommands: `image`, `video`, `stereo`, `parallax`, `install`. Each supports `--dry-run`, `--verbose`, `--device {auto,cpu,cuda,mps}`.

## Reference docs

- Read [`references/models.md`](references/models.md) when picking between Depth-Anything v2 variants, switching to MiDaS, or needing metric (in-meters) depth rather than relative.
- Read [`references/parallax-math.md`](references/parallax-math.md) when tuning the occlusion-aware warp, debugging hole-fill artifacts, or implementing a custom stereo baseline model.
- Read [`references/LICENSES.md`](references/LICENSES.md) before shipping commercial output — all models listed here are Apache 2.0 or MIT, but the commercial-safety boundary for **derivative depth maps** vs **the model weights themselves** has nuances worth knowing.

## Gotchas

- **Relative vs metric depth.** Both default models return **inverse relative depth**, normalized per image. Values are **not** in meters. Comparing two frames of the same scene means their absolute depth scale drifts. For metric-in-meters depth you need Depth-Anything v2's *metric fine-tunes* (`Depth-Anything-V2-Metric-Indoor-*` / `-Outdoor-*`), which are Apache 2.0 but trained on NC-licensed datasets (Hypersim, KITTI) — the **output** is commercial-safe; the **training data license** does not transfer. See `references/LICENSES.md`.
- **Inverse-depth convention.** HuggingFace pipelines return "depth" where **higher = closer** (inverse depth). Blender / Nuke / most DCCs expect **higher = farther**. Pass `--invert` in that case.
- **16-bit PNG vs 8-bit video.** MP4 video output is 8-bit — good enough for parallax, too coarse for compositing. Use `--out-format png-seq` or `exr-seq` for any downstream Z-depth workflow.
- **Temporal flicker on video.** Per-frame monocular depth is not temporally stable. Always smooth (`--smooth temporal`) or denoise the depth track before using it as a camera-tracking signal.
- **Input resolution matters.** Both models internally resize to their native input size (518×518 for Depth-Anything v2, 384 or 512 for MiDaS). Output is upscaled back to source resolution — don't feed 8K and expect 8K-quality edges; the model saw a 518×518 version.
- **MPS on Apple Silicon occasionally drops precision** for very large images. If output looks noisy, force `--device cpu` (slower but accurate) or downscale first.
- **`--baseline` for stereo is pixels, not cm.** Despite the hint "6.5 mm IPD," the scalar is in image pixels at the source resolution. Scale proportionally if you resize.
- **Skill-specific hole-filling.** `INPAINT_TELEA` on the revealed background beats NS for single-image parallax because it preserves edge continuity where the foreground pulled away — see `references/parallax-math.md`.
- **No prompts, no interactive choices.** Scripts are non-interactive. Pass all parameters on the command line.

## Examples

### Example 1 — Depth PNG from a JPEG

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py image \
  --model depthanything-v2 --size small \
  --in portrait.jpg --out portrait_depth.png --invert
```

Produces a 16-bit grayscale PNG where near objects are dark, far is bright (Blender convention).

### Example 2 — EXR depth sequence for Nuke

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py video \
  --model depthanything-v2 --size base \
  --in plate.mov --out depth_exr/ --out-format exr-seq \
  --smooth temporal
```

Writes `depth_exr/frame_000001.exr …` — 32-bit float Z-depth per frame with light temporal smoothing.

### Example 3 — Side-by-side stereo from a landscape photo

```bash
# 1. depth
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py image \
  --model depthanything-v2 --size base \
  --in lake.jpg --out lake_depth.png

# 2. stereo pair
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py stereo \
  --in lake.jpg --depth lake_depth.png \
  --out-left L.png --out-right R.png --baseline 8.0

# 3. pack as SBS
ffmpeg -y -i L.png -i R.png -filter_complex hstack lake_sbs.png
```

### Example 4 — 2.5D parallax reel from a single still

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py image \
  --model depthanything-v2 --size small \
  --in hero.jpg --out hero_depth.png

uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py parallax \
  --image hero.jpg --depth hero_depth.png \
  --out hero_parallax.mp4 \
  --motion orbit --frames 150 --fps 30 --amplitude 35
```

### Example 5 — Batch depth on a folder (GNU parallel)

Hand off to `media-batch`:

```bash
ls in/*.jpg | parallel --jobs 4 \
  uv run ${CLAUDE_SKILL_DIR}/scripts/depth.py image \
    --model depthanything-v2 --size small \
    --in {} --out out/{/.}_depth.png
```

## Troubleshooting

### `OSError: Can't load tokenizer/image processor for 'depth-anything/Depth-Anything-V2-Small-hf'`

Cause: No network on first run, or HF Hub outage.
Solution: Pre-download with `depth.py install depthanything-v2 --size small` when the network is up; weights cache to `~/.cache/huggingface/hub/` and work offline after.

### Depth looks inverted (near=bright, I expected near=dark)

Cause: HuggingFace convention is inverse-depth (near=bright); most DCCs want the opposite.
Solution: Re-run with `--invert`. Or post-process: `python -c "import cv2,numpy as np; d=cv2.imread('d.png',-1); cv2.imwrite('d_inv.png', 65535 - d)"`.

### Video output flickers frame-to-frame

Cause: Monocular per-frame depth is temporally unstable.
Solution: Add `--smooth temporal`. For stronger stability, hand off the depth video to `ffmpeg-denoise-restore` with `hqdn3d=4:3:6:4.5`, or upgrade to a video-aware model (none of those are Apache/MIT as of April 2026 — track updates in `references/models.md`).

### Stereo pair has obvious tearing / black slivers

Cause: Parallax baseline too large for the depth range; hole-fill can't invent enough background.
Solution: Reduce `--baseline` (try 3.0–5.0), or supply a higher-quality depth map (`--size large`).

### `RuntimeError: MPS backend out of memory`

Cause: Apple Silicon MPS allocator fragments on large images.
Solution: `--device cpu` (slower), or downscale the input (`ffmpeg -i in.jpg -vf scale=1280:-1 small.jpg`) before running depth.

### `ffmpeg` not found when writing video output

Cause: The video subcommand shells out to ffmpeg for encoding.
Solution: `brew install ffmpeg` (macOS) or your distro equivalent — the `ffmpeg-transcode` skill has install details.


---
Source attribution: damionrashford
Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-depth/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Dave AI.

## Upstream license notice

MIT License

Copyright (c) 2026 Media OS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


## Additional upstream notices

media-depth/references/LICENSES.md

# Licenses — media-depth

Every model in this skill is picked for **open source + commercial-safe** use. This file is the compliance cheat sheet — keep it current.

## Inference code / scripts

- `depth.py` (this repo) — MIT, same as the parent skills suite. Stdlib-only plus PyPI deps below.

## Runtime deps

| Package              | License              | Notes                                              |
|----------------------|----------------------|----------------------------------------------------|
| transformers (HF)    | Apache 2.0           | huggingface/transformers                           |
| torch                | BSD-3-Clause         | pytorch/pytorch                                    |
| opencv-python        | Apache 2.0           | opencv/opencv (build scripts; the lib itself is Apache 2.0) |
| numpy                | BSD-3-Clause         |                                                    |
| Pillow               | MIT-CMU (HPND)       | Effectively permissive; commercial-safe            |

## Model weights

### Depth-Anything v2 (all public variants)

- **License:** Apache 2.0
- **Covers:** `depth-anything/Depth-Anything-V2-Small-hf`, `-Base-hf`, `-Large-hf`, and the `-Metric-Indoor-*` / `-Metric-Outdoor-*` fine-tunes.
- **Commercial use:** yes.
- **Model card repos:** `https://huggingface.co/depth-anything`

**Nuance for metric fine-tunes**: The metric variants were fine-tuned on Hypersim (CC-BY-NC-SA for the dataset) and Virtual KITTI (CC-BY-NC-SA). **Model weights remain Apache 2.0** — Apache covers the final weights file the lab publishes. The NC restriction applies to redistributing *the training images* themselves, not to the output of the model or the model file. This is the consensus interpretation as of April 2026; track the Depth-Anything v2 model card for any clarifying statements.

### MiDaS v3.1

- **License:** MIT
- **Covers:** `Intel/dpt-swinv2-tiny-256`, `Intel/dpt-large-384`, `Intel/dpt-beit-large-512`.
- **Commercial use:** yes.
- **Upstream:** `https://github.com/isl-org/MiDaS`

## Explicitly DROPPED

These were considered and rejected for the commercial-safe bar of this skill:

- **ZoeDepth** — the canonical upstream repo is MIT, but some fine-tuned variants are research-licensed. To keep this skill unambiguous, we do not include it. Reconsider if and when the fine-tunes get reclassified.
- **Metric3D v2** — research license. Not distributable for commercial use.
- **Marigold** — CreativeML OpenRAIL-M (Stable Diffusion style). Usage restrictions on certain use cases; NOT a permissive license.
- **Proprietary cloud APIs** (Google Cloud Vision depth, Apple ARKit, Niantic Scaniverse) — paid, closed, out of scope.

## Derivative output (depth maps / parallax video / stereo pairs)

Output of an Apache 2.0 or MIT model is **not** encumbered by the upstream license unless the license explicitly extends to outputs (Apache and MIT do not). You may:

- Ship the depth PNG/EXR as a product asset.
- Publish the parallax MP4 on a commercial channel.
- Resell the stereo pair as part of a paid 3D download.

You must:

- Keep the NOTICE / LICENSE files if redistributing the **model weights** themselves (Apache 2.0 § 4).
- Preserve the MIT / Apache copyright notice if redistributing the **inference code** from the upstream repo (not this skill's `depth.py`, which is MIT).

## Reporting a license change

If any of the HuggingFace model cards above flips to a restricted license (CreativeML OpenRAIL-M, CC-BY-NC, RAIL, etc.), remove the model from `MODEL_REGISTRY` in `scripts/depth.py` and update this file. Do not silently keep shipping a model whose license slipped.


# Dave AI resource metadata

Creator / publisher: damionrashford
Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-depth/SKILL.md
Attribution basis: Source repository record

Fields of work: Content & Media, Data & Analytics, Design, Product Management
Roles: Data Scientist, Graphic Designer, Product Manager, UX Designer, Video Producer
Specialties: API, Testing
Tools: Hugging Face
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: abdde0be283d89dbea52f053ae0a4359f1a77baa741a6e44617c2e6f7761f947