--- name: media-interpolate description: > AI frame interpolation with open-source + commercial-safe models: RIFE (MIT, current best real-time open-source interpolator; ECCV 2022; available as rife-ncnn-vulkan prebuilt CLI on macOS/Linux/Windows), FILM (Apache 2.0, Google 2022, excels on large motion), PractCL (MIT, practical cascaded refinement). Convert 24fps to 60fps, create AI slow-motion, smooth low-framerate footage, anime frame-doubling. Superior to ffmpeg minterpolate (2015 motion-estimation filter) for clean motion without warping or jelly artifacts. Use when the user asks to interpolate frames, do AI slow-motion, boost framerate, smooth 24 to 60fps or 30 to 60fps, create motion-compensated tweens, in-between two images, or when ffmpeg minterpolate produces artifacts. argument-hint: "[model] [input] [output] [fps|factor]" --- # Media Interpolate **Context:** $ARGUMENTS ## Quick start - **24fps → 60fps:** RIFE `rife-ncnn-vulkan` → Step 2 recipe A. - **2x slow-motion (keep audio pitch unchanged):** Step 3. - **Tween between two stills** (generate N frames between A and B): Step 4. - **Anime frame-doubling:** RIFE with `-m rife-v4.6` → Step 2 recipe A. ## When to use - User says "interpolate", "smooth framerate", "AI slow-motion", "24 to 60", "frame double", "insert frames". - Source fps is lower than target display fps and you want motion-compensated tweens (not duplicated frames). - ffmpeg's `minterpolate=fps=60:mi_mode=mci` looks wobbly, jelly, or artifact-heavy. - Do NOT use for changing playback speed while keeping framerate constant — that's `ffmpeg-speed-time` (`setpts` + `atempo`). - Do NOT use for upscaling resolution — that's `media-upscale`. ## Step 1 — Pick a model | Model | License | Best for | Form | |----------|------------|---------------------------------------------|-------------------------------| | RIFE | MIT | General video, anime, real-time, default | `rife-ncnn-vulkan` prebuilt CLI + PyTorch repo | | FILM | Apache-2.0 | Large motion (sports, fast pans), stills | Python/TensorFlow inference | | PractCL | MIT | Research, cascaded refinement experiments | Python/PyTorch | **Decision rules:** 1. Default to **RIFE** via `rife-ncnn-vulkan` — Vulkan cross-platform (NVIDIA, AMD, Intel, Apple Silicon via MoltenVK), no Python, fastest install. 2. Scene with large motion (fast pans, sports, explosions, big occlusions) → **FILM**. 3. Tweening two still images with very different content → **FILM** (designed for near-duplicate photos a year apart). 4. Pure anime → **RIFE** models `rife-anime-v4.x` are tuned for it. **Do NOT use:** DAIN (research-only license). It pre-dates RIFE and is slower; RIFE supersedes it at every benchmark this skill cares about. ## Step 2 — Install the binary / package `scripts/interp.py install ` prints the exact platform-specific command. It does not auto-install. ```bash # RIFE (ncnn-vulkan prebuilt — recommended) # Download release zip from: https://github.com/nihui/rife-ncnn-vulkan/releases # Unzip; add the folder to PATH. Includes models: rife-v2.3 / v2.4 / v3.0 / v3.1 / v4 / v4.6 / anime # FILM (Python + TensorFlow) uv pip install tensorflow mediapy git clone https://github.com/google-research/frame-interpolation # PractCL git clone https://github.com/fatheral/PractCL uv pip install torch opencv-python numpy ``` ### Recipe A — RIFE 2x frame-doubling on a video ```bash # Frame pipeline: extract frames -> RIFE 2x on folder -> remux. # The script does this end-to-end: uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py video \ --model rife --in clip_24fps.mp4 --out clip_60fps.mp4 \ --fps 60 --rife-model rife-v4.6 ``` What happens under the hood: ```bash # (1) extract PNGs at source fps ffmpeg -i clip_24fps.mp4 -qscale:v 1 -qmin 1 -qmax 1 -vsync 0 in/%08d.png # (2) RIFE fills the gaps (pick 2x, or directly target -f output-frames) rife-ncnn-vulkan -i in -o out -m rife-v4.6 -n 0 # -n 0 = auto (2x). For arbitrary target fps, use the script, which computes N. # (3) remux at target fps + original audio ffmpeg -framerate 60 -i out/%08d.png -i clip_24fps.mp4 \ -map 0:v -map 1:a? -c:v libx264 -preset slow -crf 17 -pix_fmt yuv420p \ -c:a copy -movflags +faststart clip_60fps.mp4 ``` ### Recipe B — FILM on a video with big motion ```bash python frame-interpolation/eval/interpolator_cli.py \ --pattern "frames_in/*.png" \ --model_path /path/to/film_net/Style/saved_model \ --times_to_interpolate 1 \ --output_video ``` `--times_to_interpolate` is log2 of the multiplier: `1` doubles (2x), `2` quadruples (4x), `3` eight-times. ### Recipe C — PractCL ```bash python PractCL/test.py --input in/ --output out/ --factor 2 ``` ## Step 3 — Slow-motion (keep audio unmodified) `interp.py slow-mo` doubles / quadruples / 8x's the number of frames while keeping the output framerate unchanged (so content plays back slower): ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py slow-mo \ --model rife --in action.mp4 --out action_slow2x.mp4 --factor 2 ``` Implementation: extract frames at `src_fps`, RIFE by `factor`, remux at `src_fps`. Audio is **dropped** (slowing the video without slowing the audio produces a pitch mismatch). Restore audio afterwards with `ffmpeg-audio-filter` `atempo=1/factor` if desired. ## Step 4 — Tween between two stills ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py images \ --model rife before.png after.png --out middle.png # Generate N intermediate frames uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py images \ --model rife before.png after.png --out-dir sequence/ --count 8 ``` Under the hood, RIFE ncnn accepts two-image mode with `-0 first.png -1 second.png -o middle.png`. For N>1 the script repeatedly subdivides (midpoint of midpoint). ## Gotchas - **AI interpolation vs. `ffmpeg minterpolate`.** `minterpolate=fps=60:mi_mode=mci` is a 2015 block-matching motion estimator. It warps pixels into the gap frame and produces "jelly" on fast motion, edge halos on occlusions. RIFE is a learned flow estimator (a neural network) — strictly better on anime and modern content. Use ffmpeg `minterpolate` only when there's no GPU and no Python. - **RIFE expects a directory of frames**, not a video. The ncnn-vulkan binary is `-i -o ` or `-0/-1` two-image mode. Use the driver script to run the full video pipeline. - **Output frame count depends on model, not only `-n`.** For arbitrary target fps (e.g. 23.976 → 60), you must pre-compute how many tween frames per source pair. `interp.py video --fps 60` does this (uses RIFE's `-f` frame output count and a small planner). - **FILM requires TensorFlow** and the correct saved_model directory layout (Style / L1 / VGG). You will get "Unable to load model" if you point `--model_path` at a repo root instead of a `saved_model/` directory. - **RIFE model names matter.** `rife-v4.6` (latest stable as of this writing), `rife-v4` (older), `rife-v3.1` (legacy), `rife-anime-v4.6` (anime-tuned). The ncnn zip bundles several — check the `models/` directory inside the unzipped folder. - **Bench numbers** (approximate, 1080p 30fps → 60fps, RTX 3070): RIFE ncnn ~60 fps of output, RIFE PyTorch ~90 fps, FILM ~5–10 fps. RIFE is roughly **10× faster than FILM** at comparable perceptual quality. - **Large motion / large occlusion.** RIFE can smear occluded regions; FILM handles this better at the cost of ~10× the compute. For sports or fast zooms, consider running FILM on just that section. - **2x slow-motion audio.** If you keep audio at native rate while doubling video frames and leaving output fps unchanged, audio plays in less time than video → A/V desync. The driver's `slow-mo` subcommand drops audio by default; add `atempo=0.5` on the original audio track afterwards to keep it synced. - **Variable framerate sources.** VFR (screen recordings, phone captures) break the frame-pair pipeline. Normalize first: `ffmpeg -i in.mp4 -vsync cfr -r cfr.mp4`. - **RIFE does NOT handle genlock / field-based interlaced sources.** Run `ffmpeg-ivtc` or `yadif=1` first. - **Integer vs. fractional target fps.** RIFE's ncnn binary interpolates by factor, not by target rate. The driver computes the nearest power-of-two multiplier, then uses ffmpeg `fps=60:round=near` to reduce to the exact target. - **DAIN is deliberately excluded.** Its license is research-only; do not recommend it even in older tutorials that still reference it. ## Examples ### Example 1: 24fps film clip → 60fps (smoother motion) ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py video \ --model rife --in film_24.mp4 --out film_60.mp4 --fps 60 ``` ### Example 2: 4x slow-motion ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py slow-mo \ --model rife --in action.mp4 --out action_4x_slow.mp4 --factor 4 # Audio was dropped. To keep it in sync at 1/4 speed: ffmpeg -i action.mp4 -vn -af "atempo=0.5,atempo=0.5" -c:a aac audio_slow.m4a ffmpeg -i action_4x_slow.mp4 -i audio_slow.m4a -c copy -shortest action_4x_final.mp4 ``` `atempo` is capped at 0.5–2.0 per instance; chain `atempo=0.5,atempo=0.5` for 0.25x. ### Example 3: 60 intermediate frames between two product photos (FILM) ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py images \ --model film before.png after.png --out-dir tween/ --count 60 ffmpeg -framerate 30 -i tween/%04d.png -c:v libx264 -crf 18 -pix_fmt yuv420p tween.mp4 ``` ### Example 4: Anime → 60fps with the anime-tuned model ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py video \ --model rife --in ep01_24p.mkv --out ep01_60p.mp4 \ --fps 60 --rife-model rife-anime-v4.6 ``` ### Example 5: Check which binaries are available ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/interp.py check ``` ## Troubleshooting ### Error: `rife-ncnn-vulkan: command not found` Cause: binary not installed. Solution: Grab a release from https://github.com/nihui/rife-ncnn-vulkan/releases, unzip, add the folder to PATH. Or `uv run .../interp.py install rife`. ### Error: `failed to load model: rife-v4.6` Cause: `-m` model name expects a directory named `rife-v4.6` under the binary's `models/` path. Solution: The ncnn release zip includes several models pre-staged. If you moved the binary without its `models/` folder, re-unzip and keep them together. Or pass `-m /abs/path/to/rife-v4.6`. ### Video has doubled length / slow-motion when you wanted smooth-60 Cause: remuxed the 2x frames at the **source** fps instead of the **target** fps. `ffmpeg -framerate 24 -i out/...` plays 2x frames at 24 fps → 2x duration. Solution: use `-framerate 60` (or the target you asked for) on the reassembly step. The driver handles this — don't roll your own. ### Output has A/V desync after slow-mo Cause: the slow-mo path kept the original audio at native rate while the video timeline stretched. Solution: either drop audio (script default) or run `atempo=1/factor` on audio and remux. ### RIFE looks great on most frames but smears during a fast cut Cause: the two input frames span a scene change. The model tries to interpolate between unrelated content. Solution: run `media-scenedetect` to find cuts, then process each scene independently (or skip interpolation across cuts). ### FILM `Unable to load model` Cause: `--model_path` not pointing at a `saved_model` directory (must contain `saved_model.pb` + `variables/`). Solution: download from the FILM releases and point at the right style subfolder (e.g. `.../Style/saved_model`). ## Reference docs - Per-model repo URLs, license, benchmarks, GPU/CPU support → `references/models.md`. - Full license table → `references/LICENSES.md`. --- Source attribution: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-interpolate/SKILL.md License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Dave AI. ## Upstream license notice MIT License Copyright (c) 2026 Media OS Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ## Additional upstream notices media-interpolate/references/LICENSES.md # Licenses for bundled interpolation models Every model this skill recommends is open-source AND commercial-safe. DAIN (research-only) is intentionally excluded. | Model | License | Commercial use? | Closed-source-product use? | Upstream | |----------------------|-------------|-----------------|----------------------------|------------------------------------------------------------| | RIFE (code + weights)| MIT | Yes | Yes | https://github.com/hzwer/Practical-RIFE | | rife-ncnn-vulkan | MIT | Yes | Yes | https://github.com/nihui/rife-ncnn-vulkan | | FILM | Apache-2.0 | Yes | Yes | https://github.com/google-research/frame-interpolation | | PractCL | MIT | Yes | Yes | https://github.com/fatheral/PractCL | ## NOT recommended | Tool | License | Why excluded | |------|---------------|-----------------------------| | DAIN | Research-only | Not commercial-safe | ## Boilerplate attribution For a commercial product that ships outputs generated with these models, include a NOTICES / third-party licenses page with: This product uses AI frame interpolation models: RIFE (MIT) https://github.com/hzwer/Practical-RIFE FILM (Apache-2.0) https://github.com/google-research/frame-interpolation PractCL(MIT) https://github.com/fatheral/PractCL MIT requires retaining the copyright notice and license text. Apache-2.0 requires retaining the copyright, license, and any NOTICE file that accompanies the distribution. ## Model weights vs. code For all three projects, the pretrained weights are distributed alongside code under the same license. Always re-check the specific release's LICENSE file before bundling weights into a closed-source product, in case a future release changes terms. # Dave AI resource metadata Creator / publisher: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-interpolate/SKILL.md Attribution basis: Source repository record Fields of work: Content & Media, Design Roles: Graphic Designer, Motion Designer, Video Producer Specialties: Testing Tools: Film, Practcl, Rife Ncnn Vulkan Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: f44322a1d3d3f13c9558241d357d62798b40d41eed5651d69677528997d9ad5a