Reference Thread: An Exhausting List of Vocal Synth Engines

Free:

UTAU by Ameya/Ayame: Free-of-charge vocal synthesizer (optional shareware version available), available since 2008; it notoriously allows users to create their own voicebanks by recording and configuring their voices. Practically infinite amount of freely-available voicebanks, probably in every available language one could think to make a voicebank for, more being made every day I'm sure. There are paid UTAU but most of them are discontinued and/or rare by now.

Synthesizer U by Ameya/Ayame: Successor/alternate editor for UTAU currently being developed by the software's original creator, similar in appearance (and name) to Synthesizer V. Looks to be more user-friendly than the original UTAU.

Sharpkey by Boxstar: Chinese vocal synthesizer similar in layout to VOCALOID2, released in 2016. Had an original editor and a "Galaxy" edition. Seven released characters, one of which was bilingual with a Chinese and Japanese voicebank.

DeepVocal by Boxstar: Chinese vocal synthesizer and successor to Sharpkey, released in 2019. Unlike Sharpkey, users are able to make their own voicebanks. Built for Chinese, and most other voicebanks are in languages such as Japanese and Korean, as languages like English are difficult to develop for with DeepVocal's voicebank creation process.

NEUTRINO by SHACHI: Neural-network singing voice synthesizer, released in 2020, one of the earliest popular AI-powered vocal synthesizers. Currently has 10 free voices, and 3 voices bundled with paid A.I.voice talk synthesizer voices. Currently Japanese-only with no native UI, meant to convert MusicXML files into singing; a fan-made UI is available on GitHub.

NNSVS by Ryuichi Yamamoto: Open-source neural-network singing voice synthesizer released in 2020, similar in function to NEUTRINO. Users are able to create their own AI voicebanks with the engine. Many use it via the UTAU plugin ENUNU as it does not natively have a user interface.

DiffSinger by MoonInTheRiver: Chinese AI singing voice synthesis released in 2022 using a shallow diffusion mechanism, users are able to train their own voicebanks for the engine like with NNSVS. Can be used via OpenUTAU.

OpenUtau by stakira: Open-source frontend for the UTAU, NNSVS/ENUNU and DiffSinger engines (possibly more engines in the future)? Released in 2021, aims to be much more user-friendly as well as developer-friendly than the original UTAU editor. Includes "phonemizers" for ease of input, which is especially invaluable for phonetically-complex languages like English.

UTSU by titinko: A Japanese frontend for UTAU.

CeVIO Creative Studio FREE by Techno-Speech: Original free version of CeVIO, released in 2013, 1 available vocal. Pitch bend and some other features are unsupported. No longer available on the website, but will work if downloaded, though it will tell you that this version is obsolete and that you should buy the full version.

NIAONiao, MUTA (discontinued), and AISingers: Released in 2011, 2015 and 2020, respectively. Chinese vocal synthesizers similar in function to UTAU, CeVIO, and...ok, well, the last one's pretty unique as it's a cloud platform, I guess it could be compared to NEUTRINO if anything? For the latter, a GUI called AISingers Studio, similar in look to NIAONiao, is available. NIAONiao allows users to create their own voicebanks. MUTA is no longer available, but five vocals were available for it, and three would have been upcoming before its discontinuation. AISingers currently has 32 vocals listed on its website, some being under testing.

Symphoneme by TechnoBrave: A web-based "converter" singing voice synthesis program released in 2023, with no UI but a story to go along with it. Unfortunately, it is infamous for poor-quality, glitchy vocals.

AquesTone by AQUEST: A VST-based MIDI vocal synthesizer; Japanese only, 32-bit only, uses artificially-generated vocals. Its speech-synthesis counterpart, AquesTalk, is the basis for UTAU's default vocal.

Sinsy: A web-based "converter" singing voice program using MusicXML files. Has 9 voices; five are HMM-based, and four are DNN/AI-based. It is one of the first known vocal synthesizers to utilize AI in this way.

RenoidPlayer: A web-based sequencer-style singing synthesizer using voices in the form of soundfont files. It does not have available pitch tuning, but it has a few adjustable parameters at the top, such as vibrato, portamento and humanization.

PaintVoice by Kumao.Works: A mobile singing synthesizer with a MIDI sequencer interface. It is very simple in quality, and simple (CV-style) UTAU can be imported.

Web Synthesizer V: A web-based demo of Synthesizer V released before Synthesizer V Studio as a test. It had two available voices.

Project Vogen: A free Cantonese singing synthesis program available on Github. I...don't know very much about this one.

SugarCape/SaltCase (discontinued): A singing voice synthesis engine for Mac.

VOCALINA (discontinued): A Korean singing synthesis engine with two vocals available: Khylin and VORA. VORA was free, and Khylin was paid, but VORA was discontinued first. It is available as abandonware from some sites.

VocalSharp: A Chinese vocal synthesizer that allows the creation of your own voicebank, similar to UTAU and DeepVocal. Released in 2021, it is one of the newest concatenative vocal synthesis engines (that I can think of anyway).

X Studio Singer: A Chinese AI vocal synthesizer by Microsoft, released in 2020. It launched with four voicebanks/ Many features, including many voicebanks and the ability to export WAV files (fans have circumvented this by using a screen recorder), are locked behind the entry of a Chinese phone number.

Geji Geji: A free Chinese AI vocal synthesizer released in 2021 with a PC and mobile edition. It allows the creation of one's own voicebanks and claims to use AI technology to help users write songs.

VOCALOID Beta Studio (exclusive and temporary): Users who register for this program can be drawn from a lottery to get this engine for free. It was designed to test future features. It is a VST that uses a DAW's MIDI input; several parameters are available to adjust. Services will end in late March/early April. It includes 12 regular vocals and one multi-vocal. Cross-lingual capabilities are supported, at least between Japanese and English (I haven't seen Chinese).

Alter/Ego by Plogue: A MIDI-based vocal synth using input from your DAW or a MIDI controller. Voices are more modern-sounding than Chipspeech but not terribly realistic in general. 3 officially available voicebanks, 2 discontinued voicebanks and 1 controversial but available voicebank.

Unsure:
ELMIRAIVE VOX/Singer by Bandai Namco: An upcoming AI singing voice synthesizer by Bandai Namco. Has 2 voices announced.

VocalWriter by KAE Labs (discontinued): An early vocal synthesizer for Mac released in 1998, with the second edition released in 2005. I assume it was paid? It is available in some places as abandonware.

can anyone confirm that SynthV is actually the best technology available?

I have tried, or own, quite a few (Emvoice, Ace Studio, Eastwest Hollywood Choirs and Hollywood Backup Singers, Realitone Ladies and Blue) and so far I find I can do much more, more quickly, and much more realistic vocals with Synthesizer V.

I keep my eye on other companies in this emerging technology, and suspect someone might come out with something even better, but I haven't found anything else yet. It's good to have this list, and I hope it gets updated in the future.

bosone said:

can anyone confirm that SynthV is actually the best technology available?

Click to expand...

It depends on what you are wanting to do. For many popular styles Synth V is great. But it doesn’t sing classical well. On the whole the women seem a bit better than the men.

jbuhler said:

For many popular styles Synth V is great. But it doesn’t sing classical well.

Click to expand...

I agree that Synth V is aimed at the commercial market (not classical) for obvious economic reasons. But there is no reason why, with a bit of programming, it can't be used to make a convincing classical vocal, with a reasonable amount of work. First you need to abandon quantisization (of both the vocal and accompaniment.) Second you need to have knowledge of, and the ability to program, classical style embellishments, glissandos, melisimas, vibrato, etc. and probably be good at Italian, Latin, German and French for that matter. Then you probably need to use RVC (voice conversion) to get the tone right, at least for the women, who don't have either the tone or range for classical.

But if you use the new Audio-to-MIDI conversion feature on great classical singers you can hear Synth-V sound totally convincing. That, of course, would be cheating, but a way to study how the program can achieve a classical style. Tell me your favorite classical vocal piece and I will send to you Synth V singing it and you will be amazed.

ThomasS said:

I agree that Synth V is aimed at the commercial market (not classical) for obvious economic reasons. But there is no reason why, with a bit of programming, it can't be used to make a convincing classical vocal, with a reasonable amount of work.

Click to expand...

If you have an example that you think works I’d love to hear it. I’ve listened to tons of SynthV examples purporting to sing classical or opera and nothing I’ve heard supports a classical tone longer than a note or two. No one can even sing a phrase and not fall out of style. Well Asterian can sing syllables ok, but that’s it.

If you really want to try this, I’ll find a short example for you. What do you need? Score, audio example, midi?

Synth V Singing in a Classical Style

With the audio-to-midi converter in Synth V, in about 5 minutes Solaria sounds like this:

View attachment Solaria - Mandoline Op 58.mp3
Solaria - Mandoline Op 58 (Gabriel Fauré)

This was done automatically, so the phonetic detection probably sounds un-intelligible to a native speaker, but that can be tweaked. Right now it is probably gibberish, but operatic in style. Or perhaps it can be substituted with English, because this song cycle has been translated.

The point is you are hearing Solaria 100%, but her pitch curves have been modeled by Leontyne Price. Everything you hear theoretically could be done by programming, but you would have to study many examples like this inside Synth V to see exactly what is being done in vibrato, melisma, note-approach, and so forth. Here is a screen grab:


Of course, once you have the right pitch curves and vibrato, you can transpose it and change the voice. For example, here is Asterian singing the same thing:

View attachment Asterian - Mandoline(2).mp3
Asterian - Mandoline