Menu
What's new
By:
Filters

AI gen model to add realism to vst libraries

G

gbombadil

New Member
Just out of curiosity and eagerness :) , a question to folks working in the vst libraries industry.

Is there someone already working on training gen ai models with vst sample music and then showing them a human performance so it can enhance the realism of the vst library performance to make it more realistic?

I've been wondering about this since in the field of photography and video is already widely used in that regards, not with the intention of compose new music but instead to take a decently crafted music with vst and improving upon it....

If there are already working on it, what are the current result and why it hasn't reach the market yet?

Is the problem creating sufficiently large large training dataset?
Or the problem is with time consistency (similar to what happens in video).

Thanks in advance!!!
Click to read more...
 
gbombadil said:
Is there someone already working on training gen ai models with vst sample music and then showing them a human performance so it can enhance the realism of the vst library performance to make it more realistic?
Click to expand...
Don't know if this is happening, but it should be. This is the type of tool I've talked about in other posts. We already use humanizing algorithms in MIDI, so an AI enhanced version could be very useful.

Look, if you use AI tools like iZotope, etc., then you shouldn't have any objection to AI that makes a violin sample sound more like a realistically played violin. Where the line should be drawn is when the thing starts writing melodies, harmonies and counter melodies for you—which, of course, is something we already have with some apps that are regularly advertised on music forums, along with entire sample libraries that give you baked in phrases.

AI should assist, not create.
 
I think it would need a large set of midi performances and matching real performances and that sounds like an expensive thing to create.
 
I'm pretty sure this is already being worked on extensively. Even the spirit behind String Murmurations for example and Striiings is pointed in that direction. I'm pretty sure you can hook some basic AI up to String Murmurations, Striiings, and Scaler and just kick back and relax while the computer either composes from scratch for you or "collaborates" with some random key presses that you've made.

It's distasteful, to me personally, the direction that art is being taken by commercial interests. However, I think to look away is equally distasteful. So we must mindfully monitor the industry as well as our own motivations and principles. It's an interesting time to say the least.
 
Have any of you successfully used AI in any fee-paying context?

I’ve not. Not across any of the businesses I’m involved in.

Not only are none of them even close to being “ready for prime time” in basic ways (artefacts, unreliability, high cost, legal barriers), but moreover, none of them have produced anything like an artistically satisfying output. You cannot get AI to even follow your style: it can only interpolate amongst styles it already has in its dataset.

The whole AI craze reminds me of the dot com boom. 99% of AI companies are going to go bust. The core tech is not really that exciting. There will be one or two amazing applications of it. I highly doubt they’ll be in the “create X effortlessly” space.
 
ModalRealist said:
Have any of you successfully used AI in any fee-paying context?

I’ve not. Not across any of the businesses I’m involved in.

Not only are none of them even close to being “ready for prime time” in basic ways (artefacts, unreliability, high cost, legal barriers), but moreover, none of them have produced anything like an artistically satisfying output. You cannot get AI to even follow your style: it can only interpolate amongst styles it already has in its dataset.

The whole AI craze reminds me of the dot com boom. 99% of AI companies are going to go bust. The core tech is not really that exciting. There will be one or two amazing applications of it. I highly doubt they’ll be in the “create X effortlessly” space.
Click to expand...
Synthesizer V is a powerful counter example, and expectation-setter here. The first library that can sing your own material and often sound completely convincing--and the phrase AI is used in relation to it. As a result the vibe in here is "AI is an easy instant-realism button that developers are choosing to ignore because they hate money"
 
PancakesBear said:
....Even the spirit behind String Murmurations for example and Striiings is pointed in that direction. I'm pretty sure you can hook some basic AI up to String Murmurations, Striiings, and Scaler and just kick back and relax while the computer either composes from scratch for you or "collaborates" with some random key presses ....
Click to expand...

Sorry, but this is a baffling statement - String Murmurations has nothing to do with that. Its borderline defamation, considering that CH has explained in great depth, why and how he created it...
 
Last edited:
I'm working in MIDI AI, but on the academic side of things rather than the commercial side. One interesting stab at ideas along these lines was Google's GrooVAE, which was aimed at creating expressive drum performances from quantized drum scores. The demos on their website sound pretty impressive, but the few times I've tried it I was completely underwhelmed by the outputs I got. Link if you want to check it out: https://magenta.tensorflow.org/groovae
 
ModalRealist said:
Have any of you successfully used AI in any fee-paying context?

I’ve not. Not across any of the businesses I’m involved in.

Not only are none of them even close to being “ready for prime time” in basic ways (artefacts, unreliability, high cost, legal barriers), but moreover, none of them have produced anything like an artistically satisfying output. You cannot get AI to even follow your style: it can only interpolate amongst styles it already has in its dataset.

The whole AI craze reminds me of the dot com boom. 99% of AI companies are going to go bust. The core tech is not really that exciting. There will be one or two amazing applications of it. I highly doubt they’ll be in the “create X effortlessly” space.
Click to expand...
I think I am seeing bits of AI-generated visual artwork creep into various places - website product artwork, ads... Not entirely certain it's AI (I fully suspect artists are getting asked it should look more like Midjourney), but I think people are using this in a fee paying context. What I fear is that once the "template" for AI-generated visual art generation taking work away from human artists is established, it could lay the groundwork for a similar impact to certain sectors of commercial music.

Despite this concern, I'm incredibly happy to be able to experiment with Synthesizer V, which I find fascinating and empowering hahah. Ah well, intersting times!
 
Last edited:
tc9000 said:
Not entirely certain it's AI (I fully suspect artists are getting asked it should look more like Midjourney)
Click to expand...
I think it's pretty safe to assume that most artists would not be willing (often not even capable within the budget constraints) to emulate Midjourney and you are just seeing more and more purely AI generated artworks or - at best - AI generated artworks that had some retouching done to them to fix some of the most glaring flaws. There is no incentive for anyone still hiring real artists to ask them to make their work look more artificial. That would make no sense whatsoever.
 
You must log in or register to reply here.
Top Bottom