εar-VAE2

Latent-diffusion music generators operate on the latent space of a VAE — whatever spectral detail, phase structure, or stereo image the decoder cannot reconstruct is unavailable to any downstream generator. At the aggressive temporal compression these systems demand, decoders show three recurring failure modes. We ask: at a fixed compression rate, which modeling choices best preserve what matters?

Stereo audio is transformed by STFT into a complex spectrogram; a 2D-convolutional Spec Encoder with delayed stereo fusion compresses it to a 25 Hz continuous latent, and a mirrored Spec Decoder with early splitting reconstructs a coarse spectrogram. The Duplex-Aware Refiner then applies band-specific complex corrections and restores the Nyquist bin. A paired iSTFT then removes the explicit analysis margin and returns exactly the original number of waveform samples. Click a stage to inspect it.

BigVGAN’s SnakeBeta activation, x + β⁻¹ sin²(αx), learns its periodicity per channel — in the spectral domain that mixes all frequencies onto one curve. Because each STFT bin corresponds to a fixed physical frequency, Spec-SnakeBeta learns one (α_f, β_f) pair per frequency bin and shares it across feature channels. A frequency-proportional initialization in log-parameter space starts near identity at low bins and becomes progressively more oscillatory toward high frequencies. Drag the frequency slider to inspect it.

From Snake to Spec-SnakeBeta.  Standard activations such as ReLU and GELU do not provide an explicit periodic basis. On bounded intervals a neural network can still approximate periodic functions; the Snake family instead supplies a periodic residual directly through three design stages:

Human localization relies on interaural timing (phase) at low frequencies and interaural level (magnitude) at high frequencies — the classical duplex theory (Rayleigh, 1907). Per-frequency error analysis of the trained decoder shows the same asymmetry: magnitude error peaks in the 1–4 kHz mid band while phase error saturates above 4 kHz. The refiner therefore constrains its residual outputs to the cue emphasized in each band. Its output layers are zero-initialized, so refinement starts as an identity mapping that preserves the frozen decoder output. Hover the bands.

Reconstruction on the 546-track Song Describer Dataset against four recent open-source audio VAEs, each at its native operating condition. The chart reports point estimates; cross-system numbers are competitive rather than strictly matched; the base-vs-full comparison within εar-VAE2 is controlled.

Musical content varies at markedly different rates in time: percussion, onsets, and rhythmic groove change fast, while sustained harmony and timbre change slowly. A latent that mirrors acoustic structure might therefore route fast and slow temporal variation to different content types. To test this, we encode a clip to its latent z ∈ ℝ^C×T, take the real FFT along the temporal axis, and split the temporal-frequency bins at the midpoint:

Each half is decoded separately and, above all, listened to per case; the schematic below traces the procedure.

Switch between versions while playing — playback position is preserved for direct A/B comparison. Gen Demos are full-length songs generated on εar-VAE2 latents; reconstruction clips are 30 s 48 kHz stereo excerpts from the Song Describer Dataset; downstream generation clips are 45 s matched-window pairs. For cross-system comparison — reconstruction and downstream generation, where εar-VAE2 is placed side by side with other systems — clips are loudness-normalized to −14 LUFS (EBU R128 linear gain, true peak ≤ −1.5 dBTP) so the A/B is level-fair. The remaining sections are presented at their native decoded levels.

The stereo polar field visualization in each player is adapted from the open-source EAR-Audio-Preview project (polar sample mode).