Speech reconstruction vs. SNAC, Mimi, and MioCodec over 8 LibriTTS-R clips (24 kHz mono, distinct speakers). Each codec encodes a clip and decodes it back to a waveform. Listen below; the table averages the objective metrics.
| Codec | Type | Frame/token rate | Bitrate | Trained on |
|---|---|---|---|---|
| STFT-VAE (ours) | continuous latent (STFT/ISTFT) | 3.125 Hz | continuous, 128-d | general audio |
| Mimi | RVQ, acoustic+semantic | 12.5 Hz | 1.1 kbps | speech |
| SNAC | multi-scale RVQ (3 levels) | 12 / 23 / 47 Hz | ~0.98 kbps | speech |
| MioCodec | semantic tokens + global embedding | 25 Hz | 0.34 kbps | speech |
Reading the numbers. STFT-VAE is a continuous codec at 3.125 Hz — 4–15× lower frame rate than the others — so its competitiveness is the point of interest, not a like-for-like bitrate match. MioCodec is a semantic codec: it keeps linguistic content plus a single global speaker embedding and discards fine acoustic detail, so its waveform-fidelity scores (SI-SDR) are low by design. Scores use ±50 ms cross-correlation alignment.
| Codec | SI-SDR↑ (dB) | Mel-L1↓ (dB) | PESQ↑ | STOI↑ |
|---|---|---|---|---|
| STFT-VAE (ours) | 9.39 | 3.51 | 2.56 | 0.932 |
| Mimi | 12.68 | 3.32 | 3.56 | 0.969 |
| SNAC | 4.35 | 3.46 | 2.28 | 0.921 |
| MioCodec | -15.80 | 7.98 | 1.56 | 0.770 |
Test clips: LibriTTS-R (public-domain LibriVox speech). Reproduce with
comparisons/ in the repo. Audio is lossless WAV.