Bulk Convert SampleMaps to WaveTables

[d.healey]

@iamlamprey said in Bulk Convert SampleMaps to WaveTables:

This next part is from a different function so I assume I can't access the result of it

PoolReference ref(chain->getMainController(), comboBoxThatHasChanged->getText(), FileHandlerBase::SampleMaps);

You'll need to use something like this to loop through all of the sample maps. Then pass the sample map file to the convert to wavetable function. Doesn't look like it should be too complicated actually, but no idea until it's tried.

[d.healey]

Ok so this is a guess (I've done no testing). You would call this function from your Bulk Convert button's callback.

	void bulkConvertSampleMapsToWavetables()
	{
		auto& spool = chain->getMainController()->getActiveFileHandler()->pool->getSampleMapPool();
		
		for (int i = 0; i < list.size; i++)
		{
			PoolReference ref(chain->getMainController(), list[i], FileHandlerBase::SampleMaps);
			currentlyLoadedMap = ref.getFile().getFileNameWithoutExtension();
			
			if (auto vData = spool.loadFromReference(ref, PoolHelpers::LoadAndCacheWeak))
			{
				converter->parseSampleMap(*vData.getData());

				if (currentMode == FFTResynthesis)
				{
					converter->refreshCurrentWavetable(getProgressCounter());
					
				}
				
				refreshPreview(); 
			}
		}	
	}

[A Former User]

@d-healey Insane. I wasn't anywhere close to that lmao

I'll try and build it tomorrow, officially past my bed time in aus :)

[orange]

@iamlamprey For a smooth interpolation between the waves, is there any limit of the wave numbers? I remember that you said that you are getting artifacts for this?

[Christoph Hart]

@orange if you enable hq mode, it already interpolates between the tables.

[orange]

@Christoph-Hart said in Bulk Convert SampleMaps to WaveTables:

@orange if you enable hq mode, it already interpolates between the tables.

Great to know that. Thanks!

[A Former User]

Okay using @d-healey 's script as a starting point, I can get it looping through the sampleMaps and everything, but it's currently not "clearing" the file each time it iterates, so the wavetables get larger and larger (and i assume sound terrible)

I also got rid of the right side since it doesn't work anyway:

void bulkConvertSampleMapsToWavetables()
	{
		auto& spool = chain->getMainController()->getActiveFileHandler()->pool->getSampleMapPool();
		auto& list = chain->getMainController()->getActiveFileHandler()->pool->getSampleMapPool().getIdList();

		for (int i = 0; i < list.size(); i++)
		{
			currentlyLoadedMap = "";

			PoolReference ref(chain->getMainController(), list[i], FileHandlerBase::SampleMaps);
			currentlyLoadedMap = ref.getFile().getFileNameWithoutExtension();

			if (currentMode == FFTResynthesis)
				converter->renderAllWavetablesFromHarmonicMaps(getProgressCounter());
			else
				converter->renderAllWavetablesFromSingleWavetables(getProgressCounter());

			if (threadShouldExit())
				return;

			auto leftTree = converter->getValueTree(true);

			auto fileL = currentlyLoadedMap + "_Left.hwt";

			auto tfl = GET_PROJECT_HANDLER(chain).getSubDirectory(ProjectHandler::SubDirectories::AudioFiles).getChildFile(fileL);

			PresetHandler::writeValueTreeAsFile(leftTree, tfl.getFullPathName());
		}
	}

I can't see any reason why it wouldn't clear the samplemap each time, I even added an empty string at the start of the loop to clear it but idk

[d.healey]

@iamlamprey I assume the renderAllWaveTables... functions are doing something to all the wavetables in the folder and creating a new wavetable, then when it loops again there are two wavetables being processed, then 3, then 4, etc.

I don't think it has anything to do with currentLoadedMap.

[A Former User]

@d-healey Yeah so close now, I moved it out of the for loop which solved the size problem, but now all of the wavetables are the same

So i think it has to do with making renderAllWaveTables reset each time or something

[d.healey]

@iamlamprey What does renderAllWaveTables do?

[A Former User]

@d-healey i think it's the actual conversion part, the rest is just writing the result to a file

void SampleMapToWavetableConverter::renderAllWavetablesFromSingleWavetables(double& progress)
{
	using Data = ResynthesisHelpers::SimpleNoteConversionData;

	for (int i = 0; i < 128; i++)
	{
		Array<Data> conversionData;

		for (auto s : sampleMap)
		{
			int loKey = s.getProperty(SampleIds::LoKey, -1);
			int hiKey = s.getProperty(SampleIds::HiKey, -1);
			int root = s.getProperty(SampleIds::Root, -1);

			if (i >= loKey && i <= hiKey)
			{
				auto thisFreq = MidiMessage::getMidiNoteInHertz(i);
				auto rootFreq = MidiMessage::getMidiNoteInHertz(root);


				auto fName = s.getProperty(SampleIds::FileName).toString();

				if (fName.isEmpty())
					s.getChild(0).getProperty(SampleIds::FileName).toString();

				PoolReference r(chain->getMainController(), fName, FileHandlerBase::Samples);

				Data data;
				data.sampleFile = r.getFile();
				data.rootRatio = thisFreq / rootFreq;
				data.noteNumber = i;
				data.veloRange.setStart((int)s.getProperty(SampleIds::LoVel, 0));
				data.veloRange.setEnd((int)s.getProperty(SampleIds::HiVel, 127) + 1);

				conversionData.insert(-1, data);
			}
		}

		progress = (double)i / 128.0;

		if (conversionData.isEmpty())
			continue;

		struct DataSorter
		{
			int compareElements(Data& first, Data& second)
			{
				if (first.veloRange.getStart() < second.veloRange.getStart())
					return -1;

				if (first.veloRange.getStart() > second.veloRange.getStart())
					return 1;

				return 0;
			}
		};

		DataSorter sorter;

		conversionData.sort(sorter);


		auto output = ResynthesisHelpers::loadAndResampleAudioFilesForVelocity(conversionData, sampleRate);

		storeData(i, output.getWritePointer(0), leftValueTree, output.getNumSamples(), conversionData.size());

		if (output.getNumChannels() == 2)
		{
			storeData(i, output.getWritePointer(1), rightValueTree, output.getNumSamples(), conversionData.size());
		}
	}
}

[d.healey]

@iamlamprey I think you might need to wait for some Christoph input

[A Former User]

@d-healey all good, ill keep noodling in the meantime :) cheers

[A Former User]

Edit

Okay I think a better way to do it is to iterate through the sampleMap ComboBox at the top of the window, since the conversion process already works on the combobox's currently selected item:

void bulkConvertSampleMapsToWavetables()
	{

		auto sampleComboBox = getComboBoxComponent("samplemap");
		int loopLength = sampleComboBox->getNumItems();

		for (int i = 0; i < loopLength; i++)
		{
			sampleComboBox->setSelectedItemIndex(i);

			if (currentMode == FFTResynthesis)
				converter->renderAllWavetablesFromHarmonicMaps(getProgressCounter());
			else
				converter->renderAllWavetablesFromSingleWavetables(getProgressCounter());

			if (threadShouldExit())
				return;

			auto leftTree = converter->getValueTree(true);

			auto fileL = currentlyLoadedMap + ".hwt";

			auto tfl = GET_PROJECT_HANDLER(chain).getSubDirectory(ProjectHandler::SubDirectories::AudioFiles).getChildFile(fileL);

			PresetHandler::writeValueTreeAsFile(leftTree, tfl.getFullPathName());
		}

Here you can see it iterating through the combobox properly, but each consecutive wavetable is larger still...

Is there a JUCE method for control.changed() ? I can't seem to find it

Edit: okay added sendNotification and tried both setSelectedItemIndex and setSelectedId and still get the same issue

[A Former User]

Hmm okay so I think the renderAllWavetablesFromSingleWavetables is going to keep "stacking" the files as long as the window is open, since manually clicking the confirmation dialog also closes the window...

Also if I load one of these "stacked" wavetables and hit a note, HISE crashes

I just need the renderAllWavetablesFromSingleWavetables to reset in between loop iterations and everything should work, I just have no idea how to do that

[A Former User]

WOOOH okay so I can clear the ValueTree (and fix the size-stacking issue) by calling...

converter->getValueTree(true).removeAllChildren(nullptr);

...at the end of each Loop iteration. I also think I had to change the comboBox notification to async for it to update properly (it was re-exporting the same wavetable under different names).

sampleComboBox->setSelectedItemIndex(i, sendNotificationAsync);

For some other weird reason I had to delete the whole HISE build folder and recompile, VS kept saying it was Up-To-Date despite me changing lines of code.

Anyway here's the "finished" function:

void bulkConvertSampleMapsToWavetables()
	{
		auto sampleComboBox = getComboBoxComponent("samplemap");
		int loopLength = sampleComboBox->getNumItems();

		for (int i = 0; i < loopLength; i++)
		{
			sampleComboBox->setSelectedItemIndex(i, sendNotificationAsync);

			if (currentMode == FFTResynthesis)
				converter->renderAllWavetablesFromHarmonicMaps(getProgressCounter());
			else
				converter->renderAllWavetablesFromSingleWavetables(getProgressCounter());

			if (threadShouldExit())
				return;

			auto tree = converter->getValueTree(true);
			auto file = sampleComboBox->getItemText(i) + ".hwt";
			auto tf = GET_PROJECT_HANDLER(chain).getSubDirectory(ProjectHandler::SubDirectories::AudioFiles).getChildFile(file);

			PresetHandler::writeValueTreeAsFile(tree, tf.getFullPathName());

			converter->getValueTree(true).removeAllChildren(nullptr);
		}
	}

There's still some clunkiness to it, mainly the bulkConvert button not being a switch/togglable thing. IDK how to do any of the github push stuff so this is about as far as I'm gonna take it

[d.healey]

@iamlamprey

Anyway here's the "finished" function:

Well done!

I had to delete the whole HISE build folder and recompile, VS kept saying...

Sometimes intermediate files get left behind between builds. You can clean the solution folder from within VS too.

IDK how to do any of the github push stuff

Time to learn

[A Former User]

@d-healey said in Bulk Convert SampleMaps to WaveTables:

Time to learn

Hahaha tomorrow :) bed time