HISE Logo Forum
    • Categories
    • Register
    • Login

    HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

    Scheduled Pinned Locked Moved Bug Reports
    30 Posts 5 Posters 452 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • d.healeyD
      d.healey @aaronventure
      last edited by

      @aaronventure How is that different from HISE pinging google's server?

      Libre Wave - Freedom respecting instruments and effects
      My Patreon - HISE tutorials
      YouTube Channel - Public HISE tutorials

      A 1 Reply Last reply Reply Quote 0
      • A
        aaronventure @d.healey
        last edited by

        @d-healey It's not really, but how would you check if you're online if not by connecting to another server? 1.1.1.1 is a CloudFlare IP which uses AnyCast, so it automatically connects to the nearest operational server, vastly reducing a chance that you're making a call when the server appears to be down.

        I Imagine that's why Google was picked: the chance of downtime being so rare.

        This is supposed to be even more robust, with the benefit of the latency being almost non-existent in most countries where people buy audio plugins.

        d.healeyD 1 Reply Last reply Reply Quote 0
        • d.healeyD
          d.healey @aaronventure
          last edited by

          @aaronventure said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

          but how would you check if you're online if not by connecting to another server

          Ask the OS if the system is currently connected to a network.

          Libre Wave - Freedom respecting instruments and effects
          My Patreon - HISE tutorials
          YouTube Channel - Public HISE tutorials

          A 1 Reply Last reply Reply Quote 0
          • A
            aaronventure @d.healey
            last edited by

            @d-healey that's true, but how do you think the OS does the check? I would bet it's something like this. Maybe macOS is calling www.apple.com and Windows is calling microsoft.com, as that also confirms DNS resolution and HTTPS working.

            d.healeyD 1 Reply Last reply Reply Quote 0
            • d.healeyD
              d.healey @aaronventure
              last edited by d.healey

              @aaronventure How it does it doesn't matter, the problem is that every time you ping from in HISE and you don't reach the server you have to wait for the timeout time. I had a user in China who kept getting hit by 20 second delays every time I was calling Server.isOnline() - which I was doing in a few places in my login/download chain.

              If I could just ask the OS are you online, I get an instant yes/no. The checking, however it happens has already been done - it's probably reported by the router or access point but I'm not sure.

              My solution at the moment is I call Server.isOnline() once when the plugin loads and store the result. However if the user goes offline during the session then this will no longer be true, but I can handle that more easily than the 20 second delay.

              It's also possible to reduce the server timeout time, but you can only take it so low before you start giving users on slow connections the wrong result.

              This method would also allow for a broadcaster that reports when the system goes online/offline - I think I said this in my feature request post too.

              Libre Wave - Freedom respecting instruments and effects
              My Patreon - HISE tutorials
              YouTube Channel - Public HISE tutorials

              A 1 Reply Last reply Reply Quote 0
              • A
                aaronventure @d.healey
                last edited by aaronventure

                @d-healey Ah I see your point. Also I figured if you're making a channel strip plugin or something and a user just happens to have 100 instances in a project, it could get interesting on load.

                I did some more digging. On macOS, it appears that

                /usr/sbin/scutil -r $? 
                

                will check your internet connection without sending any packets. It returns "Reachable" or "Not Reachable".

                If you call

                /usr/sbin/scutil -r -W $? 
                

                the task will run until terminated and will respond with Reachable or Not Reachable whenever the network status changes (I just tested it), which is pretty much like a broadcaster.

                @Christoph-Hart you can implement this instead of having to clean room that module. I'm sure Windows has a similar mechanism.

                d.healeyD 1 Reply Last reply Reply Quote 1
                • d.healeyD
                  d.healey @aaronventure
                  last edited by

                  @aaronventure said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                  I did some more digging. On macOS, it appears that

                  Oh that's nice and simple.

                  I bet ChatGPT knows the magic spell for Windows too, I'll ask it tomorrow if you don't get there first :)

                  Libre Wave - Freedom respecting instruments and effects
                  My Patreon - HISE tutorials
                  YouTube Channel - Public HISE tutorials

                  A 1 Reply Last reply Reply Quote 0
                  • A
                    aaronventure @d.healey
                    last edited by aaronventure

                    @d-healey said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                    I bet ChatGPT knows the magic spell for Windows too, I'll ask it tomorrow if you don't get there first :)

                    Hah, sure, but I'm too lazy to go test it right now.

                    So here's the reply :

                    One-shot check (returns Boolean, no packets sent)

                    $nlm = [Activator]::CreateInstance([type]'Microsoft.Windows.NetworkList.NetworkListManager')
                    $nlm.IsConnectedToInternet   # $true = online, $false = offline
                    

                    Continuous monitoring (no polling)

                    $nlm  = [Activator]::CreateInstance([type]'Microsoft.Windows.NetworkList.NetworkListManager')
                    
                    Register-ObjectEvent -InputObject $nlm `
                                         -EventName   'ConnectivityChanged' `
                                         -SourceIdentifier NLM `
                                         -Action { 
                                             if ($Event.SourceEventArgs.IsConnectedToInternet) {
                                                 Write-Host 'online'
                                             } else {
                                                 Write-Host 'offline'
                                             }
                                         }
                    
                    # keep the session alive
                    while ($true) { Start-Sleep -Seconds 3600 }
                    

                    HISE-native integration of this might be better than manually launching Background Task as then it can just start a process and every instance of the plugin can read from the process instead of having every plugin instance start its own monitoring process through BackgroundTask (as I don't think BackgroundTask can connect to an existing one?)

                    1 Reply Last reply Reply Quote 1
                    • T
                      tomekslesicki @Christoph Hart
                      last edited by

                      @Christoph-Hart is this a part of the latest develop branch commint?

                      T 1 Reply Last reply Reply Quote 0
                      • T
                        tomekslesicki @tomekslesicki
                        last edited by tomekslesicki

                        Ok, a quick update on the subject - the XCode transport solution only works for some users. For others, the download callback returns this.data.finished = true && this.data.sucess = false even before anything can be downloaded. This is happening both on Windows now, and macOS, for about 0.5-1% of users, for everyone else things are working fine.

                        Here's a very simplified overview of my download logic:

                        At the beginning of the scrip, I call:

                        Server.setBaseURL("https://");
                        

                        which is used for POST calls, and then this download callback, with the url aprt being the rest of the url, after the https part.

                        inline function startDownload()
                        {
                        	// Download samples and content
                        	for (i = 0; i < plugincontent.length; i++)
                        	{
                        		local files = workingFolder.getChildFile(plugincontent[i]);
                        		downloads[i] = Server.downloadFile(url + plugincontent[i], p, files, downloadCallback);
                        	}	
                        
                        	// There's just one zip file and it's always the first in the plugincontent array
                        	zips[0] = workingFolder.getChildFile(plugincontent[0].replace(subfoldernamefull).replace(subfoldernamedemo));
                        	
                        	// Show update progress panel
                        }
                        
                        
                        function downloadCallback(obj)
                        {
                        	if (this.data.finished != true)
                        	{
                        		// Show and update download progress
                        	}
                        
                        	// Connection lost
                        	if (this.data.finished && this.data.success == false)
                        	{
                        		// Show error message
                        	}
                        
                        	if (this.data.finished && this.data.success == true)
                        	{
                        		// Update downloaded files count and extract zips
                        	}
                        }
                        

                        As said, this works 100% for POST calls for all users. The problem is for 1% of users with the download callback. I remember having a similar issue with GET when I started this project.

                        Save me, please!

                        T 1 Reply Last reply Reply Quote 0
                        • T
                          tomekslesicki @tomekslesicki
                          last edited by

                          The p part is the authorization that’s getting passed to the server. I was having an issue using that with GET - again, very inconsistent and on some systems only, and I can’t reproduce it on any fo the 3 macs I have around - as though the parameter was not being passed to the server properly. So maybe it’s about that, but then I have no clue on how to troubleshoot this further.

                          d.healeyD 1 Reply Last reply Reply Quote 0
                          • d.healeyD
                            d.healey @tomekslesicki
                            last edited by

                            @tomekslesicki Nothing in particular stands out to me. Here's my downloader if you want to compare.

                            Libre Wave - Freedom respecting instruments and effects
                            My Patreon - HISE tutorials
                            YouTube Channel - Public HISE tutorials

                            T 1 Reply Last reply Reply Quote 0
                            • T
                              tomekslesicki @d.healey
                              last edited by

                              Is there any way to print the final url with the parameter part? The docs https://docs.hise.dev/scripting/scripting-api/server/index.html#callwithget say that "The parameters have to be a non-nested JSON object and will automatically change the URL to embed these parameters". What does the final url look like after the change?

                              1 Reply Last reply Reply Quote 0
                              • d.healeyD
                                d.healey @tomekslesicki
                                last edited by d.healey

                                @tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                                The issue was only present on macOS Sequoia,

                                Have the three customers tested other versions of MacOS?

                                Libre Wave - Freedom respecting instruments and effects
                                My Patreon - HISE tutorials
                                YouTube Channel - Public HISE tutorials

                                T 1 Reply Last reply Reply Quote 0
                                • T
                                  tomekslesicki @d.healey
                                  last edited by

                                  @d-healey of course not ;-) But I also have one guy on Windows experiencing this now.

                                  d.healeyD 1 Reply Last reply Reply Quote 0
                                  • d.healeyD
                                    d.healey @tomekslesicki
                                    last edited by

                                    @tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                                    of course not ;-) But I also have one guy on Windows experiencing this now.

                                    What is the common link between all the people who have the issue?

                                    Libre Wave - Freedom respecting instruments and effects
                                    My Patreon - HISE tutorials
                                    YouTube Channel - Public HISE tutorials

                                    T 1 Reply Last reply Reply Quote 0
                                    • T
                                      tomekslesicki @d.healey
                                      last edited by

                                      @d-healey none, but I suspect the problem is with the parameter part of the url called by HISE. The reason I suspect this is that the POST calls are made to a different server, and they work fine. The download callback works fine wherever I test it and - again, works for the vast majority of users - but I guess it's possible that the parameter part is not attached to the call properly, and the server rejects it because it doesn't pass the authorization.

                                      Link Preview Image
                                      HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

                                      Is there any way to print the final url with the parameter part? The docs https://docs.hise.dev/scripting/scripting-api/server/index.html#callwithget say tha...

                                      favicon

                                      Forum (forum.hise.audio)

                                      d.healeyD 1 Reply Last reply Reply Quote 0
                                      • d.healeyD
                                        d.healey @tomekslesicki
                                        last edited by

                                        @tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                                        I suspect the problem is with the parameter part of the url called by HISE.

                                        If that was the case wouldn't it fail for all users?

                                        Libre Wave - Freedom respecting instruments and effects
                                        My Patreon - HISE tutorials
                                        YouTube Channel - Public HISE tutorials

                                        T 1 Reply Last reply Reply Quote 0
                                        • T
                                          tomekslesicki @d.healey
                                          last edited by

                                          @d-healey maybe it works on some systems but not others, for whatever reason? Is there a way to get the final url of the call affected by the parameter property?

                                          d.healeyD 1 Reply Last reply Reply Quote 0
                                          • d.healeyD
                                            d.healey @tomekslesicki
                                            last edited by

                                            @tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                                            maybe it works on some systems but not others, for whatever reason?

                                            That's why we need to find the common factor. Then you can recreate the system in a VM and debug it.

                                            What OS are they using?
                                            What DAW are they using?
                                            Are you able to make a standalone app that just does the download call so that it can be tested in isolation?
                                            Do they have a firewall?
                                            Do they use a VPN?
                                            Who is their internet provider?
                                            What country are they in?
                                            etc.

                                            Find out everything that they have in common and build a VM to match.

                                            @tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

                                            Is there a way to get the final url of the call affected by the parameter property?

                                            Only on your server I think.

                                            Libre Wave - Freedom respecting instruments and effects
                                            My Patreon - HISE tutorials
                                            YouTube Channel - Public HISE tutorials

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            19

                                            Online

                                            1.7k

                                            Users

                                            11.8k

                                            Topics

                                            102.6k

                                            Posts