training issue

[sutungpo]

The model performs exceptionally well! I have a question: how can we fine-tune the model so that paralinguistic elements—such as laughter and exclamations—can also be effectively handled in the voice conversion task?

[Alichua]

Hi @sutungpo,

Thanks for bringing this up. Preserving non-verbal nuances—like laughter, sighs, or coughs—is indeed a well-recognized challenge in VC.

In GPA’s autoregressive framework, we have the flexibility to model these paralinguistic elements, since our architecture unifies speech and symbolic representations. One practical approach is to extend the tokenizer with special tokens such as [laugh], [sigh], or [cough].

So in principle, if the training data includes audio segments annotated with such tags, GPA could learn to map non-verbal cues from the source to the target speaker.

We’re actively improving this capability—stay tuned for GPA 1.5, where we plan to provide native support for richer paralinguistic expression.

Best regards,
The GPA Team