Audime vocal replacement

[Guy Rowland]

This, I guess, has been the dream for decades. What if I could sing something and then just have it magically replaced by a better voice?

Welcome to Audime. Yes, of course this is AI, so there's that. What seems interesting is that it claims to take the phrasing and performance of what is recorded, but switch the timbre to a different voice. This is a pretty good demonstration, starts in earnest at 2'07.



Not 100%, but it could be great for demos, BVs etc.

It's upload / subscription - various plans inc a free taster.

https://audimee.com/

[Mike Greene]

SynthV does this, too. (Although not many people know it, since those guys are the worst at demonstrating what their product can do.)

[Lawrence]

The Great Replacement.

[Mike Greene]

The Great Replacement.

Now there's an idea! We tell everybody that the voices being used are ... Jews!

Oooo, the tiki torch guys won't stand for that. They even warned us about it! So once we have those guys on our side, with Donald Trump in the White House, AI in music is as good as ended!

[Lawrence]

SynthV does this, too. (Although not many people know it, since those guys are the worst at demonstrating what their product can do.)

I gave Synth V, and I wasn’t aware that you could sing a track, change the vocal into “styles” ( as opposed to just vocalists) and have it pick up the lyrics as this product apparently does.

[Mike Greene]

SynthV does this, too. (Although not many people know it, since those guys are the worst at demonstrating what their product can do.)

I gave Synth V, and I wasn’t aware that you could sing a track, change the vocal into “styles” ( as opposed to just vocalists) and have it pick up the lyrics as this product apparently does.

SynthV just changes vocalists, and I get the impression that the "styles" with Audime are just the different vocalists as well, but maybe there's more to it? (I only listened at the spot Guy suggested, and the girl singer sings the same inflections/style as the YouTuber sang..)

SynthV can indeed sing your words back with the new singer's voice, although as with Audime, it's all done phonetically. In other words, they don't know what the actual "words" are, they just duplicate the phonetics on the timeline. (I tried it at NAMM, but I have no idea where a video demo might be.)

I'm not recommending (or not recommending) SynthV, by the way.

[Lawrence]

Yes, the question is-how was the Audime track inputted? The only way I know how to get a vocal lyric into Synth V is to type in the phonetics. I got the feeling that Audime takes an audio track and can convert it into another voice/style. I could be wrong

[Geoff Grace]

That’s it, Larry. I watched the whole video. He submitted his vocal track online, and chose from a variety of voices the ones he wanted to hear. It then changed his vocal to resemble the timbre, formants, etc. of the chosen singers whose voices had been analyzed by AI. The resulting audio was performed exactly as he has submitted it: the notes, lyrics, vocal inflections, etc. were all the same. Only the timbre was different.

The process is somewhat akin to creating a MIDI performance and using it to audition a selection of virtual instruments. The results sounded as if a variety of people had exactly parroted his vocal performance. He was then able to import the “performances” into his track and audition them to see which “voice” sounded best in the setting.

He advised that the resulting vocals would only be as good as the vocals that are submitted for analysis. His main use would be to pitch songs to female artists using a female version of his own vocal performance for the demo.

The biggest difficulty in using this tool is creating a good vocal track to begin with. Once you have that, the process is simple: submit it online, download the AI versions, import them into your DAW, and audition the tracks.

Best,

Geoff

[Lawrence]

Yeah, Geoff. My point was that I don’t think Synth V works quite that way, though it’s pretty amazing in its own right.

[Geoff Grace]

I think you’re right, Larry. I haven’t used Synth V, but I believe the process involves inputting notes and typing lyrics.

Audime seems to work the same way as the software that’s being used in fake kidnappings to mimic a victim’s voice and convince loved ones to pay a ransom.

Best,

Geoff

[Mike Greene]

SynthV definitely lets you can sing into it, rather than typing and entering MIDI.

I sang "Somewhere over the Rainbow" in their NAMM booth (with headphones, so as not to torture other people) and it played a different singer replicating what I sang, warts and all. You can then tweak the MIDI and phonetics, since many of us don't have great pitch when we sing.

The guy who made SynthV is incredibly inept at marketing, so I don't think they have a video about this (and if they do, it's probably aimed at engineers, instead of the public), but here's one I Googled, which seems to show the feature:
https://www.google.com/search?client=sa ... MwaW4,st:0

[Geoff Grace]

I see what you’re saying, Mike. The video in your link does a good job of showing its abilities in that area, with the additional editing features, and it’s all done locally with no need to submit it on a website.

It appears that we’re already at the point at which it’s possible to create a track sung by any human voice that’s ever been recorded: Paul McCartney, Chaka Khan, Richard Nixon, your neighbor, you name it. The only thing preventing these voices from being used is probably the fear of legal action.

I’ve mostly been unreservedly excited about advances in technology until now. AI has already been used for both wondrous and terrifying purposes. I wish we were developing it more slowly so that we could perhaps prevent it from being used to harm people. I realize I’m veering off topic, but I can foresee situations in which AI could either save the world or destroy it.

Best,

Geoff

[Mike Greene]

SynthV (and others) use marketing words like AI to describe certain things they do, since "AI" is the flavor of the month. But it's not the same sort of LLM scraping we associate with AI in recent months. Specifically with companies like Udio or MidJourney or ChatGPT, which scrape tons of data using machine learning.

SynthV is essentially just sample modeling, where they pay singers to come in and record a whole bunch of articulations. They've automated some processes for turning those recordings into samples more efficiently, and they've really improved the ability to add nuance to the performances, but it's not fundamentally different from what other sample library companies do. Their "AI" is the more traditional "rules and symbols" version of AI, which is manually coding the app to make it appear the computer is "thinking" and deciding which articulation to use.

There's still some evil to that, mind you, in the same way many sample libraries takes real musicians/singers out of the equation. (Which is something I'm still uneasy about in my own case.) But it's not scraping songs without permission and making Paul McCartney clones like Udio can.

[Geoff Grace]

That’s good to hear, Mike, and I agree that sample libraries haven’t been entirely benign.

Best,

Geoff

[Guy Rowland]

Yeah, this sounds at the benign end of things - the fact you have to have a good performance to begin with for Audime is positive...

I'd imagine for a lot of EDM this would do just fine, but for most commercial music I don't see this as an existential threat for singers. People want to hear people. It's encouraging to me that people have got instantly bored with the latest AI video tricks which bring photos (and memes) to life. It's like a silly party trick. AI video will definitely have uses, but my point here is that people are very savvy to AI - they're not impressed the moment they find out it is not real.

Like with composers, it's the low hanging fruit that is most at risk. Generic background music, generic background singers... yup, these are at risk.

[Lawrence]

SynthV definitely lets you can sing into it, rather than typing and entering MIDI.

I sang "Somewhere over the Rainbow" in their NAMM booth (with headphones, so as not to torture other people) and it played a different singer replicating what I sang, warts and all. You can then tweak the MIDI and phonetics, since many of us don't have great pitch when we sing.

The guy who made SynthV is incredibly inept at marketing, so I don't think they have a video about this (and if they do, it's probably aimed at engineers, instead of the public), but here's one I Googled, which seems to show the feature:
https://www.google.com/search?client=sa ... MwaW4,st:0

Very interesting, MG. I wonder if this is a newish feature? In any case, good to know!