NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTS
Introduction
This is a demo for our paper NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTS. In the following, we will show some generated samples by our proposed method. We will release more samples, code and pre-trained models after the review.
The comprarison between generated sample by other SOTA models and our NoreSpeech
# Reference/Target Text: we must tread softly and talk low as we go through the land of the jolliginki
Reference (clean)
Reference (noisy)
FS2 (nosiy)
Styler
GenerSpeech (clean)
GenerSpeech (noisy)
GenerSpeech (denoise)
NoreSpeech
Reference/Target Text: a king has a great opportunity to make himself loved .
Reference (clean)
Reference (noisy)
FS2 (nosiy)
Styler
GenerSpeech (clean)
GenerSpeech (noisy)
GenerSpeech (denoise)
NoreSpeech
Reference/Target Text: maybe they were out of boys of the brand you wanted
Reference (clean)
Reference (noisy)
FS2 (nosiy)
Styler
GenerSpeech (clean)
GenerSpeech (noisy)
GenerSpeech (denoise)
NoreSpeech
Reference/Target Text: the rock was not much larger than their great bed , but of course they all knew how not to take up much room , and they were dozing , or at least lying with their eyes shut , and pinching occasionally when they thought wendy was not looking
Reference (clean)
Reference (noisy)
FS2 (nosiy)
Styler
GenerSpeech (clean)
GenerSpeech (noisy)
GenerSpeech (denoise)
NoreSpeech
Non-Parallel Transfer
In non-parallel style transfer, the TTS system must transfer prosodic style when the source and target text are completely different.
(1) Reference Text: when it was all done he put it on the table beside me , and at first i did nt know whether i was the live rabbit or the toy rabbit , we were so much alike
Reference audio
Target Text: well , his grill has a plating of gold , and his twistings are greatly admired
Styler
GenerSpeech
NoreSpeech
(2) Reference Text: chapter eight the mermaids lagoon
Reference audio
Target Text: to be more accurate , i should have said to the northeast
Styler
GenerSpeech
NoreSpeech
(3) Reference Text: some of the reverend frank milveys brethren had found themselves exceedingly uncomfortable in their minds , because they were required to bury the dead too hopefully
Reference audio
Target Text: but there was a passenger dropped off for you a little girl
Styler
GenerSpeech
NoreSpeech
(4) Reference Text: baron conrad and his men at arms sat foot in stirrup , the milk white horse that had been brought for otto stood waiting for him beside his fathers great charger.
Reference audio
Target Text: she would invoke the past , recall old recollections ; she would supplicate him by the remembrance of guilty , yet happy days.
Styler
GenerSpeech
NoreSpeech
(5) Reference Text: the morrow brought a very sober looking morning , the sun making only a few efforts to appear , and catherine augured from it everything most favourable to her wishes
Reference audio
Target Text: the store is on a corner about which coveys of ragged plumed , hilarious children play and become candidates for the cough drops and soothing syrups that wait for them inside.