Every fix below started with evidence: a line in the model’s source code, a measurement of the reference audio, or an A/B test on the GPU. Numbers marked client voice were measured on the real client reference (not published here for consent reasons; see Responsible use).
2,930 ms130 ms
Digital silence
7-chunk Hindi narration, client voice
0.9330.944
Speaker similarity
same text, same seeds
3 runs1
FFmpeg passes per upload
and zero temp files
- Problem
- The model takes speaking style from the first 6 s and timbre/room sound from the first 10 s of the reference. Whatever happened to be at the start of a 27.6 s file decided the clone, including distorted parts.
- Evidence
- Found in the model source: ENC_COND_LEN = 6 s and DEC_COND_LEN = 10 s; only the speaker embedding sees the whole file.
- Fix
- Score every 10 s window (speech density, clipping, quiet edges) and rotate the best one to the front, so the rest still feeds the speaker embedding.
- Result
- For the client voice the 8.3–18.3 s window was chosen automatically; the clip is built in memory and cached once per voice.
- Problem
- The decoder faithfully reproduces distortion and hiss from the reference.
- Evidence
- Measured: peak at 0 dBFS, 235 hard-clipped samples, constant background noise around −40 dB.
- Fix
- Repair clipping, remove low rumble (<70 Hz), apply only mild denoising (heavy denoise sounds robotic) and peak at −1 dBFS.
- Result
- 285 clipped samples repaired before conditioning; uploads are converted once, straight to 24 kHz mono.
- Problem
- exaggeration = 0.3 (default 0.5) produced a flat, careful ‘reading’ voice; cfg_weight = 0.7 forced slow, over-articulated pacing on a fast speaker. Values were hard-coded in two places with hidden English overrides.
- Fix
- Moved to configuration (0.5 / 0.4 / 0.75), removed the overrides, and tuned by measurement across 17 GPU runs.
- Result
- Stable, more natural delivery; a sweep showed 0.65+ starts to babble on this voice, so the safe range is documented.
- Problem
- Each chunk was sampled once. Mumbles, repeats, cut-off endings or 40 s babbles were never caught.
- Fix
- A GPU-free check per chunk: speech rate vs text length, the 40 s generation cap, and dead air > 1.2 s. On failure: one retry with a new seed, keep the better take.
- Result
- Normal chunks cost nothing extra; failures are caught. In the sweep, every real babble was flagged.
- Problem
- Chunks were concatenated with generated zero-silence. The voice carries room noise at about −45 dB, so every few seconds the background vanished and came back: the classic splice sound.
- Evidence
- 2,930 ms of exact digital silence in a 69 s narration.
- Fix
- Fill pauses with the recording’s own room tone, overlap-added from the quiet moments between words, with equal-power fades at every boundary.
- Result
- Digital silence fell from 2,930 ms to 130 ms; the noise floor is continuous.
- Problem
- Each chunk kept its own leading/trailing silence and then a fixed 250 ms or 600 ms was added on top.
- Fix
- Trim each chunk to its speech (+40/60 ms), then size pauses by punctuation: comma ≈160 ms, sentence 350 ms, question ≈400 ms, paragraph 750 ms, with ±15% deterministic variation.
- Result
- Rhythm follows the text instead of the chunk boundaries.
- Problem
- When a long sentence had to be split, the model appended a full stop, so the voice fell as if the sentence had ended.
- Fix
- Mid-sentence chunks get a trailing comma, keeping a continuing intonation.
- Result
- Long sentences now flow across chunk boundaries.
- Problem
- Chunks were not level-matched before joining.
- Fix
- Each chunk’s speech level is matched to the median (within ±6 dB, peak-safe).
- Result
- Even loudness across the whole narration.
- Problem
- Single-pass loudnorm rides the gain like an automatic volume control, lifting the noise in every pause.
- Fix
- Measure once, then apply one fixed gain (linear mode) to −16 LUFS, in the same encode that writes the AI-attribution tags.
- Result
- Natural dynamics, and one file rewrite instead of two.
- Problem
- torch.cuda.empty_cache() after each chunk forced the allocator to rebuild its memory pool every time.
- Fix
- Release once per job; the model stays resident and an idle janitor unloads it after 5 minutes.
- Result
- Steady ≈1.3× real-time once warm.
- Problem
- The reference file was re-read and re-hashed for every chunk; uploads took up to three FFmpeg runs and two temp files.
- Fix
- Hash once per file version; convert uploads in one pass; check function signatures once per job.
- Result
- Less disk and CPU work per job; no stray files.
- Problem
- Every test run left job folders in storage and rows in the live database.
- Fix
- The suite now runs against a throwaway database and folder that are deleted at the end.
- Result
- 80 tests pass and leave nothing behind.
Now listen to the result.