Long-form media transcription pipeline

Problem

Multi-hour recordings and mixed media folders are difficult to transcribe reliably in one pass. Video audio must be normalized, long inputs can exhaust memory, language may switch between French and English, and failed jobs should resume without repeating completed work. The pipeline addresses these operational constraints for both CUDA workstations and CPU-only machines.

Approach

The command-line tool recursively discovers supported audio and video files, extracts a mono audio stream when necessary, divides long recordings into deterministic chunks, and transcribes them with faster-whisper. Chunk-level segments are shifted back to the original timeline and merged into one coherent output. Existing artifacts are skipped by default, and an out-of-memory fallback retries batched inference in unbatched mode.

Technical implementation

PyAV handles decoding, resampling, extraction, and chunk creation without requiring a separate FFmpeg executable. Device selection detects available CUDA support through CTranslate2 and chooses float16 on GPU or int8 on CPU. The pipeline records manifests and metadata, supports VAD and word timestamps, and can emit plain text, timestamped text, JSON, SRT, or WebVTT. CLI switches control the model, language, task, compute type, batch size, chunk duration, overwrite behavior, and dry runs.

device, compute_type = resolve_device_and_compute("auto", "auto")
model = WhisperModel("large-v3", device=device, compute_type=compute_type)
segments, info = model.transcribe(chunk_path, vad_filter=True)
merged.extend(shift_to_source_timeline(segments, chunk_offset))