HISE Logo Forum
    • Categories
    • Register
    • Login

    added scriptable FFT

    Scheduled Pinned Locked Moved Scripting
    28 Posts 6 Posters 2.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Christoph HartC
      Christoph Hart
      last edited by

      @ustk said in added scriptable FFT:

      Just in order to get something better than than one sample every X samples…

      Ah, you're still in the waveform drawing stuff? Not sure if the FFT is seriously overkill for this task. Have you tried other interpolation algorithms? The next best thing is linear interpolation which should be good enough (it's good enough for the sample playback lol).

      ustkU 1 Reply Last reply Reply Quote 0
      • ustkU
        ustk @Christoph Hart
        last edited by ustk

        @christoph-hart no no as I said this is not for drawing, this one is for computational stuff.
        Actually I am taking a sample every X samples to decimate before a correlation function, and it works. But I'd like a more precise decimation model…

        What I found online always uses an fft, but the formulas are too complex for me to translate into code. Although I'm sure it's a trivial operation, I'm just not used to fft…

        I imagine this requires an inverse fft too though

        Can't help pressing F5 in the forum...

        1 Reply Last reply Reply Quote 0
        • d.healeyD
          d.healey @Christoph Hart
          last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • d.healeyD
            d.healey
            last edited by

            What about a resynthesis module? :D

            ? 1 Reply Last reply Reply Quote 1
            • ?
              A Former User @d.healey
              last edited by

              @d-healey with the results I got from DDSP I'm surprised more developers aren't looking into resynthesis more... we'd never have to sample a round robin again 😂

              d.healeyD Christoph HartC 2 Replies Last reply Reply Quote 2
              • d.healeyD
                d.healey @A Former User
                last edited by

                @iamlamprey It would be nice if we never had to sample anything again.

                1 Reply Last reply Reply Quote 2
                • Christoph HartC
                  Christoph Hart @A Former User
                  last edited by Christoph Hart

                  @iamlamprey What's DDSP? Nevermind it's more googleable than I thought.

                  ? 1 Reply Last reply Reply Quote 1
                  • ?
                    A Former User @Christoph Hart
                    last edited by

                    @christoph-hart Yep :) The style-transfer is particularly appealing, I wouldn't be surprised if you can legally train it with other people's sample libraries since it's 100% synthesis.

                    Not a lawyer but ;)

                    1 Reply Last reply Reply Quote 0
                    • d.healeyD
                      d.healey
                      last edited by

                      @Christoph-Hart I thought I'd have a go at this fft thing. Following your instructions above and the docs I load a sample from the sampler into a buffer. If I uncomment the process line in this script HISE will just crash.

                      const var Sampler1 = Synth.getSampler("Sampler1");
                      
                      reg sound = Sampler1.createSelection(".*")[0];
                      
                      const buf = sound.loadIntoBufferArray();
                      
                      const fft = Engine.createFFT();
                      fft.setEnableSpectrum2D(true);
                      fft.setWindowType(fft.BlackmanHarris);
                      
                      fft.setMagnitudeFunction(function(data, offset)
                      {
                      	var max = 0.0;
                      	
                      	for(s in data)
                      	{
                      		max = Math.max(max, s);
                      	}
                      	
                      	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
                      	
                      }, true);
                      
                      fft.prepare(1024, 2);
                      
                      //fft.process(buf);
                      
                      d.healeyD 1 Reply Last reply Reply Quote 0
                      • d.healeyD
                        d.healey @d.healey
                        last edited by

                        @d-healey Bump

                        OrvillainO 1 Reply Last reply Reply Quote 0
                        • OrvillainO
                          Orvillain @d.healey
                          last edited by

                          @d-healey Did you ever resolve this Dave?

                          d.healeyD 1 Reply Last reply Reply Quote 0
                          • d.healeyD
                            d.healey @Orvillain
                            last edited by

                            @Orvillain Not as far as I recall

                            OrvillainO 1 Reply Last reply Reply Quote 0
                            • OrvillainO
                              Orvillain @d.healey
                              last edited by

                              @d-healey

                              I'm not running into a crash, primarily because I'm not enabling the 2D Spectrum.

                              But a very simple example like this:

                              Content.makeFrontInterface(600, 600);
                              
                              const var AudioLoopPlayer1 = Synth.getAudioSampleProcessor("Audio Loop Player1");
                              const var audioFileObj = AudioLoopPlayer1.getAudioFile(0);
                              const var sampleRate = audioFileObj.getSampleRate();
                              const var audioData = audioFileObj.getContent(); // [channel][samples]
                              const var fft = Engine.createFFT();
                              fft.prepare(1024, 1); // 1 channel, 1024 sample size
                              fft.setWindowType(fft.Hann);
                              fft.setOverlap(0.75);
                              
                              fft.setMagnitudeFunction(function(data, offset)
                              {
                                  Console.print("Frame starting at sample: " + offset);
                              }, false);
                              
                              audioFileObj.setContentCallback(function()
                              {
                                  if (audioData.length == 0 || audioData[0].length == 0)
                                  {
                                      Console.print("No audio loaded.");
                                      return;
                                  }
                              
                                  var buffer = audioData[0]; // Mono: use first channel
                                  fft.process([buffer]); // Wrap in array of channels
                              });
                              
                              

                              Just isn't working. I get nothing printed in the console. It is as if the setMagnitudeFunction is not being executed.

                              I have IPP enabled for Hise.

                              @Christoph-Hart any thoughts????

                              full snippet:

                              HiseSnippet 1333.3ocyWs0SabDEdWLaTw8hZZ4g93HTevzhb7xcHBU.ClfZLXgoIUBPQi2cV6oLdlUyLKD2F9O1eJ8o9Z6Y1K16FrHNtIQ0Ojry49bNm46bnkT3QTJgzxt74CBIV1egS6Abcu58vTt0wGXY+UNMwJMQhRHs+fPrRQ7srsKcjgf8byZE+6u9o8wLL2iLhjk0KDTOxyo8o5QTas6OSYrFXex4z94jd0cO1SvqKXhHHdJ4TyJD6cMtK4DrQrYbrdFV0yx9GbVNni2xAarxlta4t0Fqtkq6Za1Y0ZK64u1Vqrx5q3Gr7l302DT5QG5S0BYaMVSTfQ2W3OncOws7DG7Bph1gQLGbsZCdNgrU8dTleqrjixxx1o0nTUojT07NMo9zgzGkx95XFnQZjOoYOyCERtuGgjctPZ1jP5wNs8jzP8HNl34ycNlCUv.LTaxGJIxZMyeWxot.jfqq1GeMogDNLTiJqWq1RH3eV7okKC0GkFcCVh1KxmJdtPD1hgGPjtncPwMNU6Rzw7Zi6GxHCCjJKDSEYTAkpyBfIGYQrgeCJibZmeCr1a6fgF1HRkZETUE6qyfDJnXd6XTp8PlUtu+N.qwiQmzzAn.5IOAcgWOLmSXWcQhiTWkyLAAZv.Gx6R4jpdRB3mFMN23JfS0PIIDKIUbqs7pKgbSrmKJ0f.EfdZ3iTzemDqjhneIk6Kt0TTqXn7LP7TKBLO8FhjgCqTq5FqYpJojah6xo5HeRiHtmlJ3UBx9vGtlKgDAAfXKV9OJifevcTIXDHDovMcgFR3UFRowRMk2Eg0og01nEP+XlpOs7cKgBvLEw32BYM0vrVcLi0Ad4Nx8YtjFfpLLsWkQ3c08P6rCpF5MuYT83hZWkm2hwplXfwD2mHRzDwD.hheUSSUlnRhNRxSNeW43+yTx5DED..Z6TvkwUllBtXaTjhfBnRn.mVmh0LobF2NW4hDSbUR87kRbHhxQXoDO.RUYpoJemonklEPB9IBM4zjzwbkuqL5sYEDLVdl7pTvXD4XYaPQkOjhU3Q86PjKAWdVDYnf.pQQnnGMYPQdIk4bBJ3GCcdmFRRO2Pv7MPLluuOvkUZeB70ubrI6avxRoAxERfNPS3Xe.4FX5QBx1bNGPTWqEgv7i6A6A.tB+HFVWDE1LeJkAjOJ.8Yf23JpdP94WuGPy0dPn4IMDerSKp1q23iwYFSLBYpOFwX5.suz4Pnq1SOJ.m0owuNsSulb2+sYteHlOIYv07N2enwCrAxeNoafDNwaf.yz7fxCIeaRbM6bIfvA.k4ckILOjig6uedxmIDZyy67a.kLTpsAsELeAoI.5tpnELXrIKAU04nsu7.3QRGwqu7nHJXfynAA0ExvH0kMoWSTWdvwUp4tX0aw2X02jdbrM1nO90lOQcbrXPbF65Ldr3.2OiucohE4YmLbghO2xI+X1cY1OVqSM6j0P9NPn9+ODSgs+lKKFaSMsUGxugvfGQww32..mA3HlNiZwWPlIdg8DbpWwlPsj1sKQlO1G6EZOsFdHLhx76dFgQvpbc6e+tOG1MBKe62QeT2Ddr0quyIIbQlm8n+K0sOosYO3XhRSzXhO3w6zOx3CSnblHxrmZSLznZ.1NIpea.01iTOc2KCX1LlkMRNWKCqqMg6Ge3efeoLcMmsSY5lw7ShO5i8jhW4krnloW9yho.2adLf+bve9MbF4ZEu7V9Zbe.U7UddEM08Tb4oUwUlVEWcZUbsoUw0mVE2XZUby2shlgV6EoE8SdxZY0r0gwa6ZaObIA6RV+KClD4JA
                              
                              OrvillainO 1 Reply Last reply Reply Quote 0
                              • OrvillainO
                                Orvillain @Orvillain
                                last edited by

                                @Orvillain

                                Even a minimal version like this doesn't work:

                                const var testBuffer = [];
                                
                                for (i = 0; i < 2048; i++)
                                    testBuffer.push(Math.sin(i * 0.1));
                                
                                const var testFFT = Engine.createFFT();
                                testFFT.prepare(1024, 1);
                                testFFT.setWindowType(testFFT.Hann);
                                testFFT.setOverlap(0.5);
                                
                                testFFT.setMagnitudeFunction(function(data, offset)
                                {
                                    Console.print("OK - frame at: " + offset);
                                }, false);
                                
                                testFFT.process([testBuffer]);
                                
                                

                                fft.process() is either silently failing, or not being called properly because of some kind of HISE setup thing, is my suspicion.

                                1 Reply Last reply Reply Quote 0
                                • OrvillainO
                                  Orvillain
                                  last edited by

                                  Okay, I rejigged Chris's example above. This actually works:

                                  // Create a dummy signal with two sine waves
                                  const var signal = Buffer.create(4096);
                                  
                                  reg uptime = 0.0;
                                  
                                  for(s in signal)
                                  {
                                      s = 0.3 * Math.sin(uptime);
                                      s += 0.7 * Math.sin(uptime * 2.0);
                                      uptime += 0.1;
                                  }
                                  
                                  // Create a FFT object. If you right click on it in the 
                                  // script watch table and view the popup you'll see a spectrogram
                                  // for the data passed in
                                  const var fft = Engine.createFFT();
                                  
                                  // creates a spectrogram image (required for 
                                  // the debug popup but might be used later to draw
                                  // on a panel. */
                                  fft.setEnableSpectrum2D(true);
                                  
                                  // Set the window type for the processing
                                  fft.setWindowType(fft.BlackmanHarris);
                                  
                                  // Give the fft a function that will be called for 
                                  // each signal chunk. `data` will contain either a 
                                  // buffer or an array of buffers containing the transformed
                                  // FFT signal and offset will contain the index of the first
                                  // sample in the chunk (here it will be 0, 1024, 2048 and 3072
                                  // because the processing size is 1024
                                  fft.setMagnitudeFunction(
                                  
                                  (function(data, offset)
                                  {
                                  	var max = 0.0;
                                  
                                  	for(s in data)
                                  	{
                                  		max = Math.max(max, s);
                                  	}
                                  	
                                  	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
                                  }), true);
                                  
                                  fft.prepare(1024, 1);
                                  
                                  // Process the buffer with the function above.
                                  fft.process(signal);
                                  
                                  

                                  Prepare needs to be called AFTER setting the magnitude function. Not before.

                                  OrvillainO 1 Reply Last reply Reply Quote 1
                                  • OrvillainO
                                    Orvillain @Orvillain
                                    last edited by

                                    Another thing to bear in mind - it seems to me that you cannot run an FFT over a typical array. It HAS to be a buffer.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post

                                    39

                                    Online

                                    1.8k

                                    Users

                                    12.1k

                                    Topics

                                    105.0k

                                    Posts