Research, explained.

Shorten the expensive sequence without dropping within-frame dependencies.

Change the demonstrations. Keep the downstream LLM weights fixed.

Semantic tokens can be used alone for understanding; acoustic layers retain information for reconstruction.

The same frozen backbone can use examples through the learned audio interface.

A proof of concept on simple tasks, including digit speech and denoising; broader task generalization remains the next challenge.

The system must connect meanings, acoustic detail, and task structure.

These are selected strengths, not universal superiority: MiMo-Audio is stronger on 2-shot voice-conversion WER (14.05 vs. 19.01).

The target is a structured acoustic scene, not a spoken reading of the prompt.

Schematic token states. The codec decoder and vocoder then render the waveform.

Reported generation speed: 5× the AR baseline in the paper’s inference setup.

Make-An-Audio addresses both constraints in one text-to-audio system.

The top lane constructs compositional supervision; the bottom lane describes inference.

CLAP-conditioned variant. This system comparison changes multiple components; it does not isolate prompt enhancement alone.

HiFi-Codec targets reconstruction quality with fewer codebooks.

Shown: two groups × two residual stages = four codebooks.

800 → 400 tokens/second at this frame rate, with comparable reported reconstruction scores.

Derived token rates. A codec-system comparison, not an isolated GRVQ ablation.

Optimize the interface for all three requirements, across speech, sound, and music.

Schematic compression window. Token prediction is part of the training objective, not an extra inference stage.

Large gains over reconstruction-oriented tokens; the speech-specialized GLM-4-Voice remains stronger here.

Factorize these roles while allowing the two representations to communicate.

FiLM scales and shifts reconstruction features using information from the reasoning branch.

The combined representation improves all three shown tasks over either stream alone.

Codec-system comparison; the ASR/classification ablation on the previous page separately tests branch utility.

A style request can combine emotion, energy, and speaking manner.

The strongest style relevance and strongest naturalness occur in different variants.

学长今天还说他喜欢我呢。你不珍惜我，我就跟别人跑了。

Selected project samples: a qualitative illustration of control. See the paper for systematic evaluation.

Illustrative grid. With S = 9, a bounded value 0.71 becomes round(0.71 × 9) / 9 ≈ 0.667.

Training transcripts are obtained with ASR. Sentence-level duration sets output length; word-level alignment is learned implicitly.

A representation should be evaluated through the generator that uses it.

System comparison: training data and architecture also change.

The latent space is only part of the answer; how the generator uses it also matters.

RTF 0.25 means 4 seconds of audio per second of compute in this setup.

This is a system comparison, not a claim that flow matching alone causes the entire speed gain.

Synchronized input chunks preserve the relationship between visual events and speech.

The reported <100 ms is after endpoint detection, not total end-to-end latency. Timeline is schematic.

OmniBench uses the team’s corrected scoring version. Model sizes and training differ.

Related first-author work: Omni-AutoThink → adaptive reasoning.

The choice of response mode becomes learned behavior.

Adaptive sampling/rejection changes which rollouts contribute to the update.

Overall: 73% accuracy with 47% thinking rate.

The accompanying benchmark evaluates four modality groups over five difficulty levels.

The representation connects long-range song modeling to detailed acoustic reconstruction.

Reflow shows a tradeoff; subsequent SQ-Codec finetuning improves both reported measures.