Lay out the audio front end for transcribing hour long sales calls at scale

Short answer·Medium·4.0 · 0·~3 min·Asked atCognizant Cohere Phonepe

Attempt it

You are building transcription for thousands of hour long sales calls per day. Lay out the audio front end: format and sample rate, how you chunk long recordings, how chunks are merged back, and how you keep throughput and accuracy acceptable.

Free · 2 AI evals / day

The pipeline has four stages: normalize, chunk, transcribe in parallel, stitch. **Normalize.** Telephony stacks produce audio in various codecs (mu-law, Opus, MP3) at various sample rates (8 kHz to 48 kHz). Decode everything to the format the speech model expects: usually 16 kHz mono PCM. Avoid going through a low-bitrate lossy codec twice; consonants are the first thing compression eats. **Chunk.** Speech models cap input length at 30 seconds to a few minutes. Cut the hour-long audio into chunks, but cut at silences found by voice activity detection rather than at fixed time marks. Add a 2 to 5 second overlap between consecutive chunks so words on the seam are not lost. **Transcribe in parallel.** Chunks are independent, so fan them out across multiple workers or batch them on a single GPU. Throughput scales linearly with worker count. **Stitch.** Reassemble transcripts in order, using word-level timestamps to drop duplicates in the overlap region. Add diarization as a separate pass if you need speaker labels. Track word error rate on a labeled set so regressions show up as metrics.

**Normalize first.** Decode whatever codec the telephony stack produces (mu-law, Opus, AMR, MP3) and resample to 16 kHz mono, the canonical input for Whisper, Deepgram Nova-3, AssemblyAI Universal, and Google Chirp 3. Keep a lossless or high-bitrate intermediate; double-pass lossy compression chews consonants. **Chunk on silence with overlap.** Run voice activity detection to find low-energy regions and cut there. For continuous-speech regions that exceed the model window, force a cut and add a 2 to 5 second overlap. The overlap is small relative to a 30 to 60 second chunk but enough to preserve any boundary word with bilateral context. **Stitch with timestamps.** Word-level timestamps from the model let you align overlapping transcripts and drop duplicates by picking the higher-confidence version. Without timestamps you cannot do this cleanly. **Parallelize for throughput.** Chunks are independent: fan them out to a worker pool, batch them on GPU if you self-host, or use the provider's async batch API. Reassemble in chunk order at the end. **Accuracy boosters.** Pass domain vocabulary (product names, competitor names, internal jargon) via prompt or boost lists. Run **diarization** as a separate stage (pyannote, Deepgram speakers, AssemblyAI speaker labels) if speaker turns matter; mixing diarization with transcription in a single model usually loses accuracy on both. **Govern with WER.** Maintain a labeled set of real calls and measure word error rate end-to-end. Codec changes, model swaps, and VAD threshold changes all show up as WER deltas before they show up as customer complaints.

TL;DR

Resample to 16 kHz mono, chunk at VAD silences with a few seconds of overlap, fan out chunks to parallel workers, merge with word-level timestamps, and gate the pipeline on WER measured against labeled call samples.

Memory aid

Easy to grasp

Imagine a giant restaurant where every cook can only handle small plates. A whole roast lamb (the hour-long call) has to be carved before anyone can cook it. You first wash and trim the meat so everyone is working with the same starting condition (resample to 16 kHz mono). Then you carve at natural joints rather than chopping randomly, and you leave a little meat overlapping each slice so nothing is lost at the cuts (silence-based chunking with overlap). Each cook handles one slice in parallel (workers), and a plater reassembles the meal in the right order, removing the overlap so nothing is duplicated. A taster checks a few plates each day to make sure nothing has gone wrong.

Key concepts

Multimodal AI LLM evaluation

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

A transcription pipeline for thousands of hour-long calls per day is not a single API call. It is a small distributed system with a normalization layer, a chunking strategy, a worker pool, a stitching step, and an evaluation harness that keeps the whole thing honest as models and codecs change.

This deep dive walks through each stage, the choices that matter, and the numbers you should be able to defend in an interview.

Stage 1: normalize the input

Telephony and meeting stacks produce audio in a zoo of formats: mu-law at 8 kHz from POTS lines, Opus at 16 to 24 kHz from WebRTC, AMR from mobile, MP3 from recordings. Speech encoders are mostly trained on 16 kHz mono PCM and behave best at that target.

Three normalization steps earn their place:

Resample to 16 kHz mono. Downsampling 48 kHz to 16 kHz is lossless from the model's perspective; upsampling 8 kHz to 16 kHz cannot recover lost detail but at least matches the model's expected input shape.
Avoid double-lossy. If the source is already an MP3, transcoding to a different lossy codec for storage compounds consonant loss. Keep a high-bitrate or lossless intermediate (FLAC, WAV) through the pipeline.
Loudness normalize. Quiet speakers can dip below VAD thresholds. LUFS-based loudness normalization to roughly -16 to -23 LUFS pulls the dynamic range into the band the rest of the pipeline expects.

Stage 2: chunk on silence with overlap

Stage 3: parallel transcription and the throughput math

Stage 4: stitch with timestamps and diarize separately

Stage 5: governance through WER

Production realities and 2026 model lineup

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Gong and Chorus.ai both publish that production sales-call transcription resamples to 16 kHz mono and runs VAD before any model call
Deepgram Nova-3 and AssemblyAI Universal both expose word-level timestamps explicitly so callers can stitch overlapping chunks

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you decide between self-hosting Whisper on GPU vs using a managed API like Deepgram or AssemblyAI?

Compare unit economics (cost per audio hour, GPU amortization vs API list price), accuracy on your labeled set, operational burden (queue, retry, scaling), and feature parity (diarization, word timestamps, custom vocabulary). Most teams under 5,000 hours per day pick managed; very large or compliance-sensitive workloads self-host.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Sending raw telephony audio (mu-law 8 kHz) straight to a model trained on wideband audio, then cutting it at fixed time marks with no overlap or diarization plan.

60 second bullets to scan on the way to the call.

Canonical sample rate and channel layout for modern speech encoders
Why double-pass lossy compression hurts transcription accuracy

Primary sources. Browse if you want the original framing.