The Source Flag Behind a 2.95-Second Timestamp Shift

A six-second synthetic track moved a transcript segment's start from zero to 2.95 seconds. Passing the same samples with separated_vocals=False left the timestamp at zero. The waveform had not changed. The source flag decided whether the correction was allowed.

I tested VoiceStudio 0.5.7 at revision 4df5033e5a1a92e22936fb4d7d2d39e03cf10a91. The project has an Electron interface, Python API routes, multiple speech-engine adapters, and a dubbing pipeline. My narrower subject is its onset-alignment module, which can execute with NumPy alone. All results here use synthetic tones, silence, and noise on Linux with Python 3.12.14 and NumPy 2.3.5. No transcription, source-separation, or speech-generation model ran.

The engineering point is the admission rule around a heuristic. An energy rise can be a useful candidate for moving a subtitle or dubbed line. Whether it should change a timestamp depends on where the audio came from and what the move would skip.

Admit the Audio Before Changing the Segment

The module exposes snap_segment_starts(). It mutates segment dictionaries in place and returns the number adjusted. Its first guard returns zero when separated_vocals is false. This protects the original transcript timestamps when the caller only has a mixed track: music and ambience can have sustained energy too.

The dubbing route passes its asr_on_vocals state explicitly in both the per-chunk and whole-track correction paths:

snap_segment_starts(
    chunk_segs, audio_np, sr,
    separated_vocals=asr_on_vocals,
)

That is a code-inspection finding, not an integration test of the route. The helper's default is True, so a new caller must carry the source flag correctly. The helper does not examine samples and prove that separation succeeded.

For the first fixture, I placed a two-second, 220 Hz sine wave at second three of a six-second silent buffer. A segment initially spans the whole buffer. The detector finds the tone at 3.0 seconds; the correction retains 50 ms of pre-roll and sets the start to 2.95. The mixed-source flag disables the correction before detection:

{"separated": {"adjusted": 1, "start": 2.95, "end": 6.0}, "mixedSourceFlag": {"adjusted": 0, "start": 0.0, "end": 6.0}}

The tone makes the policy reproducible. It says nothing about identifying words, separating a score from a speaker, or maintaining real-world lip sync.

Eight Frames Out of Fifteen

detect_speech_onset() measures RMS in non-overlapping 20 ms frames. It drops a ragged final frame. The threshold is the larger of 0.001 and ten percent of the window's peak frame RMS.

A candidate needs eight above-threshold frames within the next fifteen frames at the tested 16 kHz sample rate. That corresponds to 160 ms of admitted energy within a 300 ms look-ahead. The implementation uses cumulative sums to count qualifying frames for all candidates; it avoids rescanning each window.

The eight frames need not be consecutive. I alternated an above-threshold tone frame with a silent frame for fifteen frames, beginning at second one. The detector accepted second one. This is an energy-occupancy rule, not a check for continuous phonation or semantic speech.

Two shorter fixtures exposed the threshold directly:

{"eightNonconsecutiveFrames": {"candidateOnset": 1.0}}
{"toneDurationSecondsToCandidate": {"0.14": null, "0.16": 1.0}}

At this frame alignment, a 140 ms tone supplies seven qualifying frames and fails. A 160 ms tone supplies eight and passes. These are synthetic boundary results, not a speech-recognition accuracy score.

A Candidate Can Still Be Refused

Long corrections have another guard. If moving the start would skip more than 1.5 seconds, _region_mostly_silent() checks the skipped audio. It permits the move only when at most ten percent of that span's frames are audible under its own threshold. That threshold uses the larger of the absolute floor and two percent of the skipped span's peak RMS.

I added a quiet tone, amplitude 0.01, from second 0.5 through 2.5, then a louder tone, amplitude 0.5, at second three. The quiet tone sits below the detector's peak-relative onset threshold but above the skipped-region floor. Detection still proposes second three; correction refuses to jump over the earlier audible content:

{"quietEarlyTone": {"candidateOnset": 3.0, "correction": {"adjusted": 0, "start": 0.0, "end": 6.0}}}

That guard preserves uncertainty. A later loud event should not automatically erase a quieter earlier event from the timing decision. In the silent-leading fixture, the long correction passed because the skipped region was silent. The same candidate time can therefore produce different correction decisions.

Other rules keep the move forward-only, ignore shifts below 150 ms, preserve at least 300 ms of segment duration, and leave the segment end untouched. I checked these properties over 32 clean-tone fixtures, combining eight lead-in durations with four tone durations. Those fixtures all passed; they do not establish the same outcomes on noisy recordings.

Timeline Ticks Have a Different Contract

The module also exposes detect_speech_onsets() for timeline ticks. It uses a peak-relative threshold and re-arms after roughly 150 ms below that threshold. It does not apply the eight-frame sustain check used by the correction candidate detector.

An 80 ms seeded noise burst at second one followed by the tone at second three produced:

{"transientAndTone": {"correctionCandidate": 3.0, "timelineTicks": [1.0, 3.0]}}

A separate 100 ms tone at the end of a one-second buffer gave no correction candidate but did give a timeline tick at 0.9. The tick list is useful as an editing aid. It is not interchangeable with evidence admitted for automatic timestamp correction.

Decision	Evidence checked in the inspected implementation
Show a timeline tick	Energy crossing plus re-arm hysteresis
Propose a correction candidate	Peak-relative threshold plus eight qualifying frames
Move a segment start	Source flag, minimum shift, remaining duration, and skipped-content guard

Replay the Policy Without Model Downloads

The probe verifies the module's source blob, aa7e6bdf14fbd35347beabbb26797f5e1c990239, then imports the unchanged file. It uses 16 kHz float32 waveforms and the actual exported functions.

I ran:

python public/blog/vocals-before-timestamp-correction/onset-probe.py /tmp/voice-note/services/onset_align.py

The process exited with code zero and finished with:

{"cleanToneInvariantCases": 32, "result": "passed"}
PASS: seven policy fixtures and 32 clean-tone invariant cases

For a replay from a checkout:

git clone https://github.com/debpalash/VoiceStudio VoiceStudio
git -C VoiceStudio checkout 4df5033e5a1a92e22936fb4d7d2d39e03cf10a91
python onset-probe.py ./VoiceStudio

NumPy was already installed. I also attempted the upstream synthetic test file with python -m pytest; that command failed because pytest was unavailable. The assertions reported here belong to the independent probe, not a claimed upstream-suite pass. I did not install the full application, whose dependency configuration includes PyTorch and several model backends.

The tested architecture keeps detection, provenance, and correction policy separate. Its strength is that a cheap signal-processing result must pass additional checks before it changes user-visible timing. Its limit is equally concrete: the source flag remains a caller assertion, and the energy rules remain heuristics. The replay verifies those decisions without treating a sine wave as a successful dubbing session.