AI and it's practical applications for music

[Tanuj Tiku]

Yes, AI is all the rage and a source of much concern but without going into all of that in this post, I would like to discuss realistic practical applications of AI in making music.

How are you using it? Have you come across serious tools that can help us in any way? Are you using it for automation of any tasks or as a bouncing board?

To be honest, I have not used AI for any generative music/sound applications. I find the idea boring after I spent time for a few months on midjourney. It was fun but obviously boring and static very quickly as I could not do anything with it at the time. I have since seen some interesting developments from Adobe.

Where are we with music and sound tools? There is great development in noise reduction and I am currently working on a film that has a lead actor who has some sort of speech impediment, I am told. They are working with eleven labs to 'fix' that (don't ask!).

Melisma was mentioned by Linos in another thread and it looks like an interesting development with very solid results. I am going to read up on it in the coming weeks and try it.

[Linos]

Using it tentatively to check my understanding of rulebooks and stuff. NotebookLM is quite good for this as it only links to the sources you upload. I have a notebook with all the legal texts and rules that apply to the orchestra I work for. If there is a question about a rule, I check what NotebookLM says. You can't trust it to make decisions. But it can give me pointers and quickly link me to the relevant sections. It's surprisingly good for that, and makes correct connections between rules that aren't immediately obvious.

I plan to also use NotebookLM for music-related manuals. For example, you can upload the manuals of Vienna Ensemble Pro 7, Cubase and a sample library. You can then ask NotebookLM to guide you through the setup process. Or it can tell you which cc channels the library uses for what and explain the routing between VE Pro and Cubase. If you give it the necessary information about your template, you could also ask stuff like: I would like to use a consistent set of five cc channels to control the microphones of all libaries in my template. What channels are unused and suitable for that?

I haven't used the AI at all for composing or for mockups. But I have my eye on Melisma. Early results are promising. It's fascinating that a student, working on his own, could produce such promising software. I think it opens up a way forward for the sample industry, which has been stagnating and moving sideways for quite some time.

I will start a thread about Melisma soon, if nobody else is faster. I will link to that thread here.

[Luke]

Similar situation as Linos for me. Zero use for generative proliferation, but trying to employ it for sifting through huge amounts of texts. I spent much time creating cheatsheets in the past for sample libraries (such as complicated beasts like Sample Modeling Strings) in particular, so that I could quickly look up specific functions or settings without wasting too much creatively distracting time. NotebookLM is pretty good at this, as Linos described.

Picking up on Melisma (I look forward to that conversation Linos), I think it's quite exciting indeed. I will have to try it soon-ish, though there's still something about the sound that doesn't feel right - I tend to hear something akin to phasing the whole time, although it could be the instrument model used.

I've briefly tried, on a couple of occasions, to use ChatGPT to create scripts that would extend Logic's functionality via Scripter. Specifically, in the application of cleaning up MIDI recordings. I tend to prefer looking at clean and eye-pleasing data for organizations and efficiency reasons, but so far I have not succeeded in making this happen. No doubt a result of my own limitations in the interaction.

[Jaap]

I tried it a few times for newsletters and maybe improving some of my product text, but I was not happy with the results.
Also I noticed it took the fun away of doing things. Even the cumbersome jobs are somehow part of the whole process and I just want to do them and not let it be done by AI.
The satisfaction when it is all finished and done is so much better (at least for me!)

[Daryl]

AI is a very thorny topic. Suffice it to say that my labels won't accept any music that has any kind of AI involved in the compositional process.

In terms of drudgery work, I can see the advantages. Obviously many people earn their living by drudgery work, so there is that.

[RobS]

No use so far but I’m keeping an eye on the operatic voices as the publishing company I work for asks for mockups of operas every now and then. They wanted the voices to sing the actual words but that wasn’t possible…

[Tanuj Tiku]

Thank you all for your inputs.

I have not checked NotebookLM but will give it a look. Honestly, I have used zero AI in anything I do. However, I have just started messing around with Claude. It seems to be pretty good for some things but I have to keep double checking and sometimes, it makes mistakes. No music related stuff yet. Claude does claim that it can analyze scores if the sheet music is uploaded and we could 'study' it together. However, without any audio and an understanding of teaching, it could be a disaster. The legal problems, notwithstanding.

My friend however has been using Claude for coding and using it to automate a few tasks as well as write some scripts for reaper. He is also doing a lot of VFX work where he sees good value in either reducing the number of steps it takes to do something or in some cases killing entire areas for good.

I can't bring myself to use it for music. It is what I do! However, I am interested in automation of certain tasks. It is a double edged sword. On the one hand, technologies like Melisma are breakthroughs but it is also an orchestra killer. In the area of film and TV, something like this could cause a lot of damage and very soon. For concerts, it is a different thing altogether obviously.

There is of course a huge legal problem with much of this. At least in India, people use it in offices all the time (including, I hear teaching) and the legal departments have nothing to say and there are no policies for use of AI.

Daryl makes an important point of entry level work. Many assistants start off with doing this work as they get trained but this may reduce the number of jobs if more things are automated.

My view is that the companies will only make money if they sell these apps to everyone and it may actually reduce productivity in many areas as it may complicate simple tasks. It is never truly fully autonomous. The money part is substantial, especially with huge investments now being made in AI builds.

[1gc]

Off point a little but reading Rob S and remembering just reading a review in the Sound on Sound-Feb. 2025 of a vocal program name of "DREAMTONICS-FELICIA" (VoiceDatabase for Synthesizer V Studio Pro)
g.c.

[RobS]

Thank you g.c. I have SynthV will surely check that voice out

[Tobias Escher]

SynthV is the backbone of my choir work (rehearsal tracks...). Version 2 is coming end of March, and I'm super excited.
Also check out the voices from Eclipsed Sounds!

[FriFlo]

To me, AI-creation is not just a matter of whether it is as good as man-made art or not (which I am sure is the case for many people here as well ...). Melisma seems like a breakthrough, however, only listening to a few examples I can also hear some shortcomings. Also, as I understand it, you just program the note-values and basically have no means of changing the output (with direct interaction of the expression like controllers), which is a bit of a problem as soon as you want to consider yourself the "composer and conductor" so to speak. Using the modulation wheel "live" is also part of the fun to me. But I am sure that further versions and other programs will overcome these shortcomings and that is the reason I only buy very few VSTis these days as just writing down a score and get a good representation of that as an audio file surely has some appeal to me.

Coming back to my initial statement, however, it is not just a matter of how good these tools work. To me, AI is pretty much the beginning of (another) dark age of technological advancements. We surely had some dark times in the beginning of technology and the rise of capitalism. Industrialization certainly enslaved a huge number of people and made their lives miserable. But then the promise of advancements for everybody was made and partly kept (in a limited way for a limited part of the world). The promise was using the technology for the good of all. Of course, these promises where bound to fail on the long term because the necessary growth for that kind of capitalism was bound to deplete natural resources and therefore eternal growth is an illusion that cannot be kept. This is the age of failing we currently live in, but that only as a side note, as - although of great importance- it is not the topic here.

So, does AI serve that old promise of improving the life of us all? Hardly, the way I see it! It is one technology that is clearly there to take jobs away that are actually quite desirable. Of course, you can argue that manufacturing things by hand is also a quality-work that people appreciate, and therefore even the very first steam machine took away work that can be pleasurable. But undoubtedly those first machines also took away a lot of the physical burden that was not desirable! And while AI is taking away intellectual labor that most of us would consider unappealing (like doing your taxes), it mostly does intellectual labor that most people would find attractive to do!

Taking away jobs may not be the only problem here: What is to become of a society where even this kind of intellectual work no longer needs to be done/practiced? We actually have a good example: Chess masters still compete in big events. But somehow, the myth of a grand master is lost (at least to me), knowing that he could easily be defeated by a computer. In fact, the grand master of today even has to be taught by a computer. Having learned the training regiment of such a grand master, I am totally turned of by that kind of practice. I would under no circumstances ever want to be a competitive chess player. I fear the same might happen with composers of music: I appreciate the lengthy process of working on a good piece of music. That process is joy, frustration, work, more frustration, more work and finally joy and some end result. But what is to become of that process when you could basically get the same result by the push of a button? Maybe not the same result for you, but the same one for any other human being than yourself! Wouldn't that discourage most people from doing all of that? And this is not limited to music and arts! AI could take away any need for intellectual kind of work at all! I don't want to see everything in dark colours, but I clearly have a hard time seeing anything positive about that kind of development.

Coming back to what should be the promise of technology to human beings: I would like to see it like the (maybe naive) picture that Star Trek paints. There is a replicator and an eternal source of energy to produce anything you can ever think of. That is why men are no longer needed to produce material goods or collect energy. They simply can do anything they find for-filling - explore the universe with a warp drive, express themselves in different art forms and further invent. The prospect of our current advancements seems make the opposite of that more likely:
A few men controlling AI control the majority of mankind, energy and natural resources get more and more depleted and most of us are happy when we get one of the dumb jobs left that low-level humans are cheap enough to do like carrying out Pizza or whatever that will be. I am sure, these are the kinds of dark thoughts Piet was talking about, that keep us from writing music or talking about the joyous process here. And I am sure he has a point in saying that! But on the other hand, we would walk into a trap avoiding those thoughts just indulging in our art. At least I have a hard time doing that ...

[FriFlo]

To draw a bit more focus on the title of this thread: I think the practical application for AI "helping" in composing is an illusion. There may be a short period in between the age before and after AI. Composers might use tools to be able to compose faster. That will enable them to do more work in a shorter time. But it will also degrade them to pickers and choosers. There intellectual process of creating the music will no longer happen. They will end up as administrators and loose the ability to actually compose. And on the long run, they won't be needed, as the machine will certainly be able to replace them as well. Anything that will be needed for a longer time is "a face" for the public. People might want to see the DJ standing in the center selling us the AI-created music. The new job profile of the influencer already reflects that trend today.

[RobS]

SynthV is the backbone of my choir work (rehearsal tracks...). Version 2 is coming end of March, and I'm super excited.
Also check out the voices from Eclipsed Sounds!

Tobias, have you seen classical voices in the E.S. catalogue? I'll be waiting for the SynthV update with anticipation, thanks for the info

[Tobias Escher]

No, they don't have anything classical. I just try to massage the performances as good as possible, but of course it sounds nothing like "real classical". Sad

[Tanuj Tiku]

I do share some of the skepticism of AI, especially as highlighted by FriFlo.

On the one hand, I have no interest in one button composing but there could be novel uses of AI for our work. Chess is not doing too badly I think as many people still play at high levels and even recently there was a lot of attention in some competitions.

I don't think AI will replace all intellectual work because this sort of thinking does not take into account human activity linked to GDP and economics. How are these companies going to make any money unless they sell it to each and everyone of us? There is no use case of selling Instagram to businesses, you make money by somehow selling it to everyone. The same is true for phones. Blackberry was too reliant on positioned as a business use case and in came iPhone that destroyed it, including other reasons of course.

Look at the first real product from Microsoft - they integrated ChatGPT in their word document programs. If the AI will do everything how does economics as we know it work?

AI will probably enter unknown territories. There are easy to see industrial applications, co-scientist is probably useful for scientists. There are data sifting applications and much more, including but not limited to robotics.

However, it is probably going to decimate some work totally but it may not be as bad as we fear. For example, people have been worried about VFX. The total market cap of the VFX industry is about $10 billion and Open AI alone is planning an investment of $500 billion. What good is Open AI if it can replace the VFX industry? It is tiny. The same is true for music. They will probably build many products that will get integrated in workforce as well as personal use. Ultimately, I think it will be a tool that will increase some productivity in some areas but further allow new novel use cases. In some cases, I do agree that it seems to decrease productivity.

We are unable to see the positives because we just have not seen the application side of things.

For example, with music there is a threat to replace all of production music but don't we already have hundreds of thousands of tracks available already ready to license? Again, the market cap of the music industry is tiny. Personally, I feel this is an internet level invention that will have wide spread use upon which totally new and unexpected things will be built.

What do you guys think about this?

Sorry for going totally off-topic!

[RobS]

I must confess I don't have a real opinion on this... I'm not too worried anyway, Musicians will keep doing music because it's the life choice they made, it's a way to open their mind and build connections through playing/writing/publishing their work. Professionals will learn to exploit the new technology to their advantage avoiding the risk of homologation, finding the way to let their voice be heard above the uniform mediocre offerings...

[Thomas Mavian]

No, not for music but as a tool like any other. I must say that NoteBookLM is a very good "manual repository". Also, loaded a few sources on string writing and got a 27 minute podcast with 2 AI-voices sharing their "thoughts" on the sources. Quite interesting technology.

I use AI just as I use for example Google Sheets to sum up calculations or anything else.

I must admit, I struggled quite a bit to understand the benefits of AI, for me at least. Now I seem to use it more and more. For the little things "around everything else" so to speak.

[RobS]

No, they don't have anything classical. I just try to massage the performances as good as possible, but of course it sounds nothing like "real classical". Sad

too bad, the synthV plugin works very well otherwise... thank you