Scriptnode pitch-shift a signal

[Christoph Hart]

@aaronventure I'm not 100% sure, but I guess it involves using a good timestretching library optimized for low latency that can handle live audio input.

[hisefilo]

@griffinboy hi mate! Faust is the way for me. Just google shimmer + faust + dsp
i.e https://github.com/azur1s/elysiera/blob/main/faust/dsp.dsp

[ps]

@Christoph-Hart 1024 samples (50ms) is what all the "realtime" pitch shifter plugins introduce.

the signal smith demo https://signalsmith-audio.co.uk/code/stretch/demo/ set to 50ms block size sounds on monophonic signals pretty much how you would expect "industry standard" rt pitch shifter to sound like .
would it be a crazy amount of work to basically replicate the web demo but with an audio stream?

the following looks even better for "realtime" processing and has a MIT license:

GitHub - jurihock/stftPitchShift: STFT based real-time pitch and timbre shifting in C++ and Python

STFT based real-time pitch and timbre shifting in C++ and Python - jurihock/stftPitchShift

GitHub (github.com)

used in here:
https://github.com/jurihock/stftPitchShiftPlugin

I tried to implement it already but failed (I think because it's not c++11)

do you want to take a look? I think it opens up really cool currently not possible opportunities straight out of the box.

[ps]

@Lindon @Christoph-Hart after posting this I started to write my own c++ code inspired by all the different algorithms available and can say that for "realtime" pitch shifting there is not really a one fits all solution but it's very doable and I got it up and running (but still optimising) . my use case right now is vocal manipulation and besides the fact that you have to really play around with window sizes, overlapping and so on - there is some other stuff that is needed for a industry standard result. I saw that autotune or little alterboy are reporting very odd latency to the host. so I figured there has to be additional sfft processing happening and it is resonance suppression. so before doing the actual shift I lower the magnitude of bins (above a certain frequency) to avoid resonances hitting the shifting algorithm. formant shifting sounds shit yet but I'm working on this right now.
Summary is that I don't see myself coming up with a node that I just use in different projects.
and if you go down the road, don't accidentally process per Frame or your computer will explode ;)

[Christoph Hart]

D'oooh, I didn't even think about trying out the signal smith algorithm...

The implementation was dead easy and the sound is way way WAY better than the naive pitch shifting from Faust or scriptnode (kind of obvious, that was the entire point of the JUCE talk of the signalsmith developer). However there is a latency of 4096 samples which is kind of huge - you can set the FFT to 2048, but then it will sound very bad on lower notes.

[aaronventure]

@Christoph-Hart will this be replacing or augmenting the timestretch for the sampler as well?

[Christoph Hart]

@aaronventure No I just took the timestretching algorithm that's already in the sampler and made it a realtime effect that works on any audio signal.

[d.healey]

@Christoph-Hart Does this mean we can have a pitch LFO in effects plugins now?

[Lindon]

@Christoph-Hart well the stftPitchShift looks pretty good - and quick and has a MIT license...

[ps]

@Lindon @Christoph-Hart
stftPitchShift is def the best out there I could find - it can do polyphonic and also formant shift.
2048samples latency is normal for pitch shifting audio (it's also normal that low signals sound bad when pitch shifted in those kind of plugins)
I'd love to see that implemented as well :)

[Christoph Hart]

@ps I played around with it yesterday but it always sounded very cheap. I‘m definitely doing something wrong maybe I‘ll try their JUCE example and go from there.

[Christoph Hart]

@d-healey said in Scriptnode pitch-shift a signal:

@Christoph-Hart Does this mean we can have a pitch LFO in effects plugins now?

A pitch lfo can be made with a simple delay buffer that modulates the delay time (a basic chorus), a pitchshifter would be overkill.

[ps]

@Christoph-Hart did you align the pitch shift with the timbre shift and set a timbre frequency to around 2.5 - 3ms?

if the formants are not shifted together with the signal it will sound strange :)

[ps]

@Christoph-Hart can you point me in the direction how you included the library? i fail drastically somehow.

[Christoph Hart]

@ps I added a third party node, included all files in there and then fought an epic battle until all compile errors went away.

You need to use a replacement for std::span (I ended up with tcb::span) as this is a C++20 feature.

However I'm really not convinced about the quality here - the CPU usage is 2-3 times as high as signalsmith and the sound is super bad. The latency is a bit better, but that's really the only thing that this library has going for them.

Also the coding style is super counterintuitive - I literally had to copy the entire plugin code into the node and write a wrapper because I couldn't figure out how it expects the different buffer sizes to work.

[ps]

@Christoph-Hart can you share the node template with me? I'm doing something wrong - I get 2000% cpu even though I'm pretty confident I do the processing in buffers haha

btw in general It would be interesting to see how you write a wrapper around a juce plugin code and integrate it as node :)

[Christoph Hart]

@ps Are you using the debug build? If you don't build the release configuration it also maxes out my CPU here with a single instance.

btw in general It would be interesting to see how you write a wrapper around a juce plugin code and integrate it as node :)

Sure, here you go:

ThirdParty.zip

This is definitely not anything I'm proud of - I just want to get this thing to work so the coding style is all over the place. If this would make it into HISE it would require a almost 100% rewrite including swapping out whatever weird thing they are using for the FFT with the proper libraries (IPP / vDSP) etc, but I'm rather sceptical at the moment that it's worth the hassle.

You might have to set the USE_JUCE_PROCESSOR preprocessor to true and manually add a few JUCE modules to the DLL Projucer project (I think audio_processor & hi_streaming are missing).

[ps]

@Christoph-Hart haha thank you - I will have a completely non-judgmental look :)

[ps]

@Christoph-Hart one last thing - what is the magic trick to make it find the files referencing each other?

'../stftPitchShift-main/cpp\StftPitchShift/RFFT.h' file not found

#include "../stftPitchShift-main/cpp\StftPitchShift/RFFT.h"

[Christoph Hart]

@ps There's a backslash in the path (after cpp. Try if changint this to a normal path helps, otherwise just make sure that the relative paths from wherever you call that works.