VocalBurst — two-stage detect + classify
The idea: instead of classifying the whole clip at once, first find WHERE a
burst is, then name it.
1. Detector (laion/vocalburst-locator, a small model that scans the timeline) marks
every moment whose "is this a non-speech burst?" score is ≥ 0.7 → candidate time segments.
2. For each candidate segment, the multi-label classifier (VoiceCLAP → MLP) names the burst via its top-1 prediction. If it isn't confident it's a real burst, the segment is discarded — no false alarm.
Green = the named burst matches a ground-truth burst in the clip. 40 speech-with-burst clips.
GT bursts: Tsk
no segment ≥ 0.7
GT bursts: Trembling Whimper
0.0-6.1s → Trembling Whimper 0.99
GT bursts: Cough
0.5-5.9s → Cough 1.00 6.4-6.6s → discarded (no clear burst)
GT bursts: Soft Hum
0.4-6.0s → Humming 0.98
GT bursts: Hiccup
0.0-4.5s → Hiccups 0.99 4.6-4.9s → discarded (no clear burst)
GT bursts: Sobs
0.3-6.4s → Sobs 0.97
GT bursts: Hand Slaps
0.3-0.6s → Lip Smack 0.81 0.7-6.2s → Finger Snaps 0.98
GT bursts: Relief Sigh
no segment ≥ 0.7
GT bursts: Licking Sound
0.3-0.5s → Lip Smack 0.81 1.1-6.1s → Licking Sound 1.00
GT bursts: Scream
0.0-6.0s → Scream 0.99
GT bursts: Yawn
2.1-2.3s → Slap Face 0.97 5.3-5.7s → Yawn 0.92
GT bursts: Sniff
0.5-6.6s → Snort 1.00
GT bursts: Drinking Noises
0.4-0.6s → Lip Smack 0.80 0.7-4.7s → Drinking Noises 0.99
GT bursts: Fearful Gasp
0.9-6.0s → Effort Grunt 0.98
GT bursts: Gulps
0.2-4.8s → Swallows 0.99 4.9-5.2s → discarded (no clear burst) 5.4-5.8s → discarded (no clear burst) 5.9-6.1s → discarded (no clear burst) 6.2-6.5s → discarded (no clear burst) 6.7-6.8s → discarded (no clear burst)
GT bursts: Pain Moan
0.2-6.0s → Pain Moan 1.00
GT bursts: Nervous Giggle
0.2-3.8s → Nervous Giggle 0.97 3.9-4.0s → Lip Smack 0.77
GT bursts: Drinking Noises
0.1-0.3s → Lip Smack 0.83 0.3-5.9s → Swallows 0.98 6.0-6.1s → discarded (no clear burst) 7.1-7.3s → discarded (no clear burst)
GT bursts: Drinking Noises
0.2-1.3s → Slap Face 0.77 1.4-1.7s → Lip Smack 0.85 2.1-2.2s → Lip Smack 0.83 3.0-3.4s → Lip Smack 0.77 3.5-3.9s → Lip Smack 0.79 4.0-4.4s → Lip Smack 0.78 4.4-5.7s → Slap Face 0.83
GT bursts: Lip Smack
0.0-6.1s → Clicks Tongue 0.94 6.8-6.9s → discarded (no clear burst) 7.0-7.2s → discarded (no clear burst)
GT bursts: Sharp Inhale
0.5-1.9s → Slap Face 0.96 1.9-3.6s → Lip Smack 0.75 3.6-3.8s → Lip Smack 0.79 4.3-4.5s → Lip Smack 0.78 4.5-4.7s → Lip Smack 0.79 4.7-8.1s → Nervous Giggle 0.92 8.3-8.9s → Gulps 0.97
GT bursts: Nervous Giggle
1.4-4.2s → Breathy Giggle 0.99 4.3-4.7s → Clears Throat 0.89 5.0-5.2s → Lip Smack 0.64 5.2-5.4s → Yawn 0.81 5.6-5.7s → Slap Face 0.97
GT bursts: Spitting
0.1-0.3s → Lip Smack 0.81 0.4-6.3s → Snort 0.95 6.3-6.5s → discarded (no clear burst) 7.2-7.4s → discarded (no clear burst)
GT bursts: Swallows
0.2-5.9s → Swallows 0.99
GT bursts: Person Whistling Playfully
0.2-5.6s → Person Whistling Playfully 1.00
GT bursts: Finger Snaps
0.1-5.4s → Tsk 0.97 5.5-5.8s → discarded (no clear burst)
GT bursts: Scream
0.1-5.8s → Scream 0.99 6.4-6.6s → Slap Face 0.98
GT bursts: Tsk
0.2-5.8s → Tsk 0.99
GT bursts: Snicker
0.4-0.6s → Lip Smack 0.83 1.2-1.5s → Slap Face 1.00 3.5-3.7s → Lip Smack 0.83
GT bursts: Sharp Whistle
0.3-0.5s → Lip Smack 0.79 0.7-4.5s → Sharp Whistle 0.97
GT bursts: Tongue Click
0.0-6.0s → Tongue Click 1.00
GT bursts: Contented Sigh
0.0-0.2s → Slap Face 0.98 0.3-5.0s → Deep Breathing 0.90
GT bursts: Ahem
0.2-5.1s → Ahem 0.97
GT bursts: Deep Breath
0.1-6.6s → Snort 0.93
GT bursts: Soft Hum
0.0-6.0s → Soft Hum 0.98
GT bursts: Childlike Giggle
0.0-6.0s → Childlike Giggle 1.00 6.3-6.4s → discarded (no clear burst)
GT bursts: Humming
0.1-6.0s → Humming 0.98
GT bursts: Deep Breathing
0.0-5.9s → Hiss 1.00 6.0-6.1s → Lip Smack 0.91 6.2-6.4s → discarded (no clear burst)
GT bursts: Person Whistling Playfully
0.1-5.2s → Person Whistling Playfully 1.00
GT bursts: Cackle
0.0-4.8s → Guffaw 0.99