DEV Community

Cover image for Show DEV: I packaged my Whisper + FFmpeg Shorts pipeline into one local command
Kelvaro
Kelvaro

Posted on

Show DEV: I packaged my Whisper + FFmpeg Shorts pipeline into one local command

Disclosure: this post is about a tool I sell (29 EUR one-time). Everything below works without buying it.

Earlier I wrote about building the pipeline step by step and auto-captioning clips locally. This post shows the finished version.

What it does

python clip.py episode.mp4 produces about six 9:16 clips with burned-in word captions, silence trimmed, a hook title, and optional zooms.

The parts that were harder than expected

  1. Cut points. Cutting on Whisper word timestamps instead of on segments stopped clips from starting mid-word. Silences longer than ~0.28 s inside a clip are dropped, with a small pad (60 ms before a word, 100 ms after) so cuts don't sound clipped.
  2. Choosing clips. A plain "loudest part" heuristic kills jokes because the punchline gets cut. Scoring on 7 checks (hook, standalone, one idea, payoff, energy, comment-bait, length) worked better. The optional path sends only the transcript to an LLM, roughly 5-8k tokens for 20 minutes.
  3. Framing. A tight face crop looks great until two people talk. So the default is the full 16:9 frame on a blurred 9:16 canvas, and --crop with YuNet face-follow plus smoothing is opt-in.
  4. German captions. The default Whisper model is small for speed. If the German captions come out wrong, --model medium --lang de forces German and is more accurate, but slower on CPU.

Limits

Windows is tested and macOS/Linux are untested. It's CLI-only, so preview everything. Free open-source alternatives (e.g. OpenShorts, Docker-based) exist, and mine is a packaged no-Docker Windows version with a German guide.

Links

Top comments (0)