Menu
What's new
By:
Filters

RAM optimization / Orchestral template

B

ballagnat

Member
Hi,

The thread that I am creating here addresses subjects that have already been discussed here and there, but I believe that in view of the subtlety of the subject a redundancy will not be useless.

To put it simply, I am creating from scratch a new hardware and software configuration (on a Mac Studio M2), which will replace an old one based on a network of several computers.

The specific point I want to address here is RAM optimization, because for a whole bunch of reasons I want to "save" RAM availability.

Before exposing the different strategies considered, I should write a few words about the context and the configuration

Hardware:

Apple Mac Studio M2 / 192 GB RAM / 2TB Storage
NVME M2 8TB HARD DRIVE (for Libraries)
Apollo X6

Software:

Ventura 13.5.2
Cubase 12 Pro (not already installed due to shop closure)
VEP 7 (AS Natively compatible)
VEP 7 Server (AS Natively compatible)
Kontakt 7 (last update)
VI Pro (last update)
Vienna Synchron Player (last update)
NI Komplete 14
And A lot of 3rd Party Libraires

The configuration of my rig is based as follows:

On a single machine, a Mac Studio M2 Ultra, I run Cubase 12 Pro. The large Full Orchestral template of the latter will have the following characteristics;
12 VEP 7 Plug ins connected to 12 instances of VEP 7 (on the same machine).
4 VERBS AUXS
2500 midi tracks
290 VST STEREO OUT (returns of the VEP RACK VST)
197 Stereo Group tracks (interface between vst and audio tracks)
192 audio tracks (routing: audio tracks input = group tracks output (it prints the VEP outputs in audio))
My routing / printing system is very special but it suits me.

On the 12 instances of VEP (each dedicated to kind of instruments ex : str - ww - br etc...) a huge number of patches and samples will be present.

My problem is to be able, very quickly, to load a VEP Server template (with all the virtual instruments via 12 instances) and access certain instruments, while remaining within the limit of 192GB of RAM (less a significant quantity used by the 'OS and Apps). I specify that I want all the instruments to be present in the big VEP metaframe, correctly routed and configured, but NOT loaded into the ram (so that the template opens in the blink of an eye).

According to what I have read here and elsewhere, and according to my software architecture, two strategies are valid and coherent. They are both based on the principle of inactivating things (tracks, and/or patches and/or samples) and here is their principle:

1/ The VEP instances contain a lot of "inactive" VEP tracks housing very few instruments, which allows me to, when I wish, click on "active" in the VEP and at that moment the instruments hosted by this VEP track will load. The advantage of this strategy is that the instruments will be loaded in a very targeted way, but the disadvantage is that it requires a lot of work creating templates and a lot of kontakt instances (but this is subsidiary due to the M2 Ultra CPU capabilities).
ex: 1 VEP track = 1 kontakt plugin = Violin I (multi articulations) patch.

2/ VEP instances contain many fewer "active" VEP tracks, but these VEP tracks host many kontakt instruments which are all "purged" before saving the VEP metaframe template. The advantage is that it requiers less work to build the template but I have never tested it.

3/ a mix of the two strategies.

Well, if I'm wondering this, it's because I read on quite a few threads that some of us had problems with strategy number 2/ and that glitches occurred when loading data in real time.
Obviously I will submit this strategy 2/ to tests on my part, but I think that your experience can save me a lot of time, but also consider solutions that I would not have thought of.

Thank you for getting this far and I hope that a charitable and experienced soul will be able to provide me with some information.

Best,

Bertrand
Click to read more...
 
IMO a disabled VE Pro template is the way to go. You don't need to have 1 instance of Kontakt per instrument though, that's the road to madness and far too many audio pipelines imo which will end up consuming a ton of CPU. Every audio input from VE Pro takes CPU cycles, used or not, and they really add up over a large template so you gotta be pragmatic there and share some inputs.

I made a few videos a while ago about my setup. This method has stood the test of time for me, I use the same template to this day (with additions of course). This is a very short part 1 intro, parts 2 and 3 are linked in the description and go into the how and why I set it up like this.

(I strongly recommend you DON'T use disabled tracks in Cubase for several very practical reasons - I cover some of this in part 3)

 
Thank you both very much for your valuable advice. Here in France the day was beaten by my enthusiasm and fatigue took over. I will definitely take the time to dissect your messages very soon. Thanks anyway for the support. Best,
 
Guy, thats doing the old school multi-timbral thing. Use 1 track for 1 Kontakt instance. My template at the moment is over 1400 tracks, mostly single articulations and it's lightening fast coupled with Metagrid for Visibility and navigation. Yes the Project size is largish. To go this route you need a FAST nVme drive to save to. I personally only have 2 backups for each Project. But the straight forward routing more than makes up for that.

Not sure Junkie is doing that I'm pretty sure he's using VEP, I think you're thinking of Trevor Morris where he showcases the native Cubase Disabled route here:
 
vibrationisreality said:
Guy, thats doing the old school multi-timbral thing. Use 1 track for 1 Kontakt instance. My template at the moment is over 1400 tracks, mostly single articulations and it's lightening fast coupled with Metagrid for Visibility and navigation. Yes the Project size is largish. To go this route you need a FAST nVme drive to save to. I personally only have 2 backups for each Project. But the straight forward routing more than makes up for that.

Not sure Junkie is doing that I'm pretty sure he's using VEP, I think you're thinking of Trevor Morris where he showcases the native Cubase Disabled route here:
Click to expand...

No way for me. Project sizes in Cubase become absolutely colossal and ultimately unworkable with auto-backups.

VE Pro is cumbersome to set up, but there's no comparison with any other method I've tried in practice. And I've tried pretty much everything.
 
Salut,
2500 midi tracks !!!!! c'est de la pure folie
Utilise les key switching
Avec 120 pistes, c'est bien suffisant
C'est impossible de mixer 2500 pistes, 120 c'est déjà un sacré boulot
 
Last edited:
So here's why I use disabled tracks instead.
As many have already stated mixing and setting up is much more complicated with VEP
Kontakt 7 and Opus now allow you to unload the RAM of all instances in a single click.

Having a huge number of separate tracks is no longer a problem with Cubase 12 as you now have a very large number of VCA tracks and/or groups which allows you to mix easily.
I prefer to combine separate tracks with tracks with keyswitches which allows you to use the RAM in an optimized way.
Furthermore, the argument of a backup that is too large does not hold up.
firstly because the discs are now of large capacity and you just need to use a macro when your music is finished in order to select the disabled tracks and without data and delete.

Of course to only use Cubase better to have a powerful processor 128 gigabytes of ram and Nvme SSD
My template includes more than 4000 tracks all deactivated and thanks to the visibility PLEs I instantly access the track I want.

Having Kontakt 7 and Opus which allows you to purge the RAM quickly contributes greatly to the execution speed
and I even know that some have automated it when the slopes close
 
Freudon33 said:
So here's why I use disabled tracks instead.
As many have already stated mixing and setting up is much more complicated with VEP
Kontakt 7 and Opus now allow you to unload the RAM of all instances in a single click.

Having a huge number of separate tracks is no longer a problem with Cubase 12 as you now have a very large number of VCA tracks and/or groups which allows you to mix easily.
I prefer to combine separate tracks with tracks with keyswitches which allows you to use the RAM in an optimized way.
Furthermore, the argument of a backup that is too large does not hold up.
firstly because the discs are now of large capacity and you just need to use a macro when your music is finished in order to select the disabled tracks and without data and delete.

Of course to only use Cubase better to have a powerful processor 128 gigabytes of ram and Nvme SSD
My template includes more than 4000 tracks all deactivated and thanks to the visibility PLEs I instantly access the track I want.

Having Kontakt 7 and Opus which allows you to purge the RAM quickly contributes greatly to the execution speed
and I even know that some have automated it when the slopes close
Click to expand...
When you will have to produce a score for real orchestra you going to have fun with your 4000 tracks
Articulation switching will be much more easy, specially is you have articulation switching every few notes
 
When you will have to produce a score for real orchestra you going to have fun with your 4000 tracks
Articulation switching will be much more easy, specially is you have articulation switching every few notes
Click to expand...
I wasn't talking about using 4000 tracks at the same time but about having access to these tracks quickly
Furthermore, I don't believe that the models created with VST instruments are all systematically played in the orchestra.
And if necessary, just use tracks with keyswitches, which will greatly simplify playback.
 
cyrilblanc said:
When you will have to produce a score for real orchestra you going to have fun with your 4000 tracks
Articulation switching will be much more easy, specially is you have articulation switching every few notes
Click to expand...
It depends on what type of music you're composing. Not everyone is producing mockups for Live Orchestra.
 
vibrationisreality said:
It depends on what type of music you're composing. Not everyone is producing mockups for Live Orchestra.
Click to expand...
I write my own composition with a full orchestra, synths, drums and singer.
I could not imaging changing track because I have a legato of 3 notes then on another track a staccato of 12 notes then on another track a Sul ponticello ...... How many articulation in a violin ???
If you have the luck to write a music for a film and the film has the budget for a real orchestra you are going to be in trouble to produce a score.
It's up to you, that was just an advice !

I started like you having hundred of tracks with many instance of VE PRO SERVER and VE PRO
 
Freudon33 said:
Furthermore, the argument of a backup that is too large does not hold up.
firstly because the discs are now of large capacity and you just need to use a macro when your music is finished in order to select the disabled tracks and without data and delete.

Of course to only use Cubase better to have a powerful processor 128 gigabytes of ram and Nvme SSD
My template includes more than 4000 tracks all deactivated and thanks to the visibility PLEs I instantly access the track I want.

Having Kontakt 7 and Opus which allows you to purge the RAM quickly contributes greatly to the execution speed
and I even know that some have automated it when the slopes close
Click to expand...
Okay, you poked the bear.

If it works for you, wonderful. It didn't work for me, and here's why.

I do a lot of versions - I reach a point and save a version, move on knowing I can revert if I need to (I usually don't, but I like knowing I can). I also have auto-save, and five versions. Cubase does crash sometimes for me, again I need the safety net. I also have my project folder in Dropbox. More safety.

I found that I could approach 1gb for each session file on disabled Cubase. Let's say I have 10 versions of a cue, 5 autosaved versions - that's 50gb. For one cue. It all becomes big, clumsy, unwieldy, and slow, and you'll eventually likely reach the limit on your Dropbox plan.

Now yes, I could get a macro to clean up one project file and delete all the undisabled tracks. What about the other 49? Do I need to do all that housekeeping permanently?

Then there's the other benefits of VE Pro. It's about twice as efficient as Cubase as a host, and if you are working on different projects with the same template, you can switch between them in a snap (or as snappy as Cubase gets - the less active instruments in the DAW the better). All your active instrument tracks stay open in VEP, ready to go and connected as soon as the next project loads.

The initial load of my entire template in VE PRO is circa 10 seconds (about 1,500 instruments IIRC). Enabling an instrument is as quick or quicker than in Cubase, usually just a few seconds per instrument.

In use, it's pure joy. Small Cubase project files, I run uncoupled so autosaves are quick, and RAM use is tiny. I rarely go anywhere near 32gb, never mind 128gb. (I always save the VEP instruments purged where possible, as you say background loading in Kontakt and some other players is terrific). As for the viframe, I use the one for everything, it's pretty easy to add new instruments as I get them.

The only downside with VEP disabled - it DOES take a long time to set up in the first place. You have to put in the hours. But the payback is tremendous. They did help with some keyboard modifiers in VEP 7 for auto-incrementing midi channels etc in the automation pane.

Like I say, I'm not out to convert anyone here with their own method. If t works for you, that's all that matters. But for the OP or anyone looking to set up a serious template for the first time or unhappy with what they have, I'd definitely recommend looking at all the pros and cons with eyes wide open. There's a lot of hidden bear traps in templates, the more aware of them that you are going in, the better you'll be able to make a choice that works for you.
 
the point of having separate tracks and tracks with keyswitches disabled is simply to be able to use the PC's resources optimally
It is of course understood that if I use several articulations of the same instrument I will use the keyswitches
and the fact of optimizing these resources allows
also to consume less energy which today a wise choice
 
Freudon33 said:
the point of having separate tracks and tracks with keyswitches disabled is simply to be able to use the PC's resources optimally
It is of course understood that if I use several articulations of the same instrument I will use the keyswitches
and the fact of optimizing these resources allows
also to consume less energy which today a wise choice
Click to expand...
Rules of dumb :
- More player are loaded more cpu it is using.
- Using VE PRO is using more CPU than having instrument inserted in your DAW (with Logic)
- If you load your samples on an external SSD***, I/O are at least 5 times slower ; 500 Mbps v.s. 2 500* if not 6000** Mbps on internal SSD (Checked with Black Magic Disk Speed Test)

* My MacBook Pro 2019
** New Crucial MVE 2 SSD
*** Raid 0 of two SSD will improve your I/O 1,5 times.
 
Hi,

Wow, I'm impressed by your answers.:emoji_open_mouth:

I am very grateful for all of your experience's sharing and for all the valuable informations you bring to the table.
It's a gold mine! (by the way I took the opportunity to develop my idiom skills).

A huge thank you for all this, it’s super useful!🙏

Well, I admit that taking in all the information from the past month at once was quite intense, but it's definitely worth it.
(sorry for the late feedback guys).

The best I have to do is take a step back to put into perspective the ins and outs of this very complex project so that it becomes as "simple" and fluid as possible.

I will get back to this thread very soon and I am looking forward to the discussions we will have subsequently.

See you soon !

Best,

b.
 
Guy Rowland said:
Okay, you poked the bear.

If it works for you, wonderful. It didn't work for me, and here's why.

I do a lot of versions - I reach a point and save a version, move on knowing I can revert if I need to (I usually don't, but I like knowing I can). I also have auto-save, and five versions. Cubase does crash sometimes for me, again I need the safety net. I also have my project folder in Dropbox. More safety.

I found that I could approach 1gb for each session file on disabled Cubase. Let's say I have 10 versions of a cue, 5 autosaved versions - that's 50gb. For one cue. It all becomes big, clumsy, unwieldy, and slow, and you'll eventually likely reach the limit on your Dropbox plan.

Now yes, I could get a macro to clean up one project file and delete all the undisabled tracks. What about the other 49? Do I need to do all that housekeeping permanently?

Then there's the other benefits of VE Pro. It's about twice as efficient as Cubase as a host, and if you are working on different projects with the same template, you can switch between them in a snap (or as snappy as Cubase gets - the less active instruments in the DAW the better). All your active instrument tracks stay open in VEP, ready to go and connected as soon as the next project loads.

The initial load of my entire template in VE PRO is circa 10 seconds (about 1,500 instruments IIRC). Enabling an instrument is as quick or quicker than in Cubase, usually just a few seconds per instrument.

In use, it's pure joy. Small Cubase project files, I run uncoupled so autosaves are quick, and RAM use is tiny. I rarely go anywhere near 32gb, never mind 128gb. (I always save the VEP instruments purged where possible, as you say background loading in Kontakt and some other players is terrific). As for the viframe, I use the one for everything, it's pretty easy to add new instruments as I get them.

The only downside with VEP disabled - it DOES take a long time to set up in the first place. You have to put in the hours. But the payback is tremendous. They did help with some keyboard modifiers in VEP 7 for auto-incrementing midi channels etc in the automation pane.

Like I say, I'm not out to convert anyone here with their own method. If t works for you, that's all that matters. But for the OP or anyone looking to set up a serious template for the first time or unhappy with what they have, I'd definitely recommend looking at all the pros and cons with eyes wide open. There's a lot of hidden bear traps in templates, the more aware of them that you are going in, the better you'll be able to make a choice that works for you.
Click to expand...
This is a fantastic write-up and I wholeheartedly agree with everything here. It's also the route I took after years of experimenting with different setups. Pure joy indeed!

Post of the year candidate this one. Well done @Guy Rowland
 
My $0.02: I only use single instances of Kontakt, Musio, or anything else like that in my work and templates and for the most part, all tracks are initially disabled in my templates.

The reasons I prefer doing it that way are:

1. I never have to think about MIDI channels.
2. I can "freeze" any single instrument part at any time.
3. The template loads fast with disabled tracks and uses less RAM initially.
4. I get better CPU performance with single instances as opposed to multis.

The trade off, using a little more RAM for single instances vs. the much improved flexibility and workflow, is an easy choice for me. YMMV.
 
You must log in or register to reply here.
Top Bottom