HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

d.healey

@aaronventure How is that different from HISE pinging google's server?

aaronventure

@d-healey It's not really, but how would you check if you're online if not by connecting to another server? 1.1.1.1 is a CloudFlare IP which uses AnyCast, so it automatically connects to the nearest operational server, vastly reducing a chance that you're making a call when the server appears to be down.

I Imagine that's why Google was picked: the chance of downtime being so rare.

This is supposed to be even more robust, with the benefit of the latency being almost non-existent in most countries where people buy audio plugins.

d.healey

@aaronventure said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

but how would you check if you're online if not by connecting to another server

Ask the OS if the system is currently connected to a network.

aaronventure

@d-healey that's true, but how do you think the OS does the check? I would bet it's something like this. Maybe macOS is calling www.apple.com and Windows is calling microsoft.com, as that also confirms DNS resolution and HTTPS working.

d.healey

@aaronventure How it does it doesn't matter, the problem is that every time you ping from in HISE and you don't reach the server you have to wait for the timeout time. I had a user in China who kept getting hit by 20 second delays every time I was calling Server.isOnline() - which I was doing in a few places in my login/download chain.

If I could just ask the OS are you online, I get an instant yes/no. The checking, however it happens has already been done - it's probably reported by the router or access point but I'm not sure.

My solution at the moment is I call Server.isOnline() once when the plugin loads and store the result. However if the user goes offline during the session then this will no longer be true, but I can handle that more easily than the 20 second delay.

It's also possible to reduce the server timeout time, but you can only take it so low before you start giving users on slow connections the wrong result.

This method would also allow for a broadcaster that reports when the system goes online/offline - I think I said this in my feature request post too.

aaronventure

@d-healey Ah I see your point. Also I figured if you're making a channel strip plugin or something and a user just happens to have 100 instances in a project, it could get interesting on load.

I did some more digging. On macOS, it appears that

/usr/sbin/scutil -r $?

will check your internet connection without sending any packets. It returns "Reachable" or "Not Reachable".

If you call

/usr/sbin/scutil -r -W $?

the task will run until terminated and will respond with Reachable or Not Reachable whenever the network status changes (I just tested it), which is pretty much like a broadcaster.

@Christoph-Hart you can implement this instead of having to clean room that module. I'm sure Windows has a similar mechanism.

d.healey

@aaronventure said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

I did some more digging. On macOS, it appears that

Oh that's nice and simple.

I bet ChatGPT knows the magic spell for Windows too, I'll ask it tomorrow if you don't get there first :)

aaronventure

@d-healey said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

I bet ChatGPT knows the magic spell for Windows too, I'll ask it tomorrow if you don't get there first :)

Hah, sure, but I'm too lazy to go test it right now.

So here's the reply :

One-shot check (returns Boolean, no packets sent)

$nlm = [Activator]::CreateInstance([type]'Microsoft.Windows.NetworkList.NetworkListManager')
$nlm.IsConnectedToInternet   # $true = online, $false = offline

Continuous monitoring (no polling)

$nlm  = [Activator]::CreateInstance([type]'Microsoft.Windows.NetworkList.NetworkListManager')

Register-ObjectEvent -InputObject $nlm `
                     -EventName   'ConnectivityChanged' `
                     -SourceIdentifier NLM `
                     -Action { 
                         if ($Event.SourceEventArgs.IsConnectedToInternet) {
                             Write-Host 'online'
                         } else {
                             Write-Host 'offline'
                         }
                     }

# keep the session alive
while ($true) { Start-Sleep -Seconds 3600 }

HISE-native integration of this might be better than manually launching Background Task as then it can just start a process and every instance of the plugin can read from the process instead of having every plugin instance start its own monitoring process through BackgroundTask (as I don't think BackgroundTask can connect to an existing one?)

tomekslesicki

@Christoph-Hart is this a part of the latest develop branch commint?

tomekslesicki

Ok, a quick update on the subject - the XCode transport solution only works for some users. For others, the download callback returns this.data.finished = true && this.data.sucess = false even before anything can be downloaded. This is happening both on Windows now, and macOS, for about 0.5-1% of users, for everyone else things are working fine.

Here's a very simplified overview of my download logic:

At the beginning of the scrip, I call:

Server.setBaseURL("https://");

which is used for POST calls, and then this download callback, with the url aprt being the rest of the url, after the https part.

inline function startDownload()
{
	// Download samples and content
	for (i = 0; i < plugincontent.length; i++)
	{
		local files = workingFolder.getChildFile(plugincontent[i]);
		downloads[i] = Server.downloadFile(url + plugincontent[i], p, files, downloadCallback);
	}	

	// There's just one zip file and it's always the first in the plugincontent array
	zips[0] = workingFolder.getChildFile(plugincontent[0].replace(subfoldernamefull).replace(subfoldernamedemo));
	
	// Show update progress panel
}


function downloadCallback(obj)
{
	if (this.data.finished != true)
	{
		// Show and update download progress
	}

	// Connection lost
	if (this.data.finished && this.data.success == false)
	{
		// Show error message
	}

	if (this.data.finished && this.data.success == true)
	{
		// Update downloaded files count and extract zips
	}
}

As said, this works 100% for POST calls for all users. The problem is for 1% of users with the download callback. I remember having a similar issue with GET when I started this project.

Save me, please!

tomekslesicki

The p part is the authorization that’s getting passed to the server. I was having an issue using that with GET - again, very inconsistent and on some systems only, and I can’t reproduce it on any fo the 3 macs I have around - as though the parameter was not being passed to the server properly. So maybe it’s about that, but then I have no clue on how to troubleshoot this further.

d.healey

@tomekslesicki Nothing in particular stands out to me. Here's my downloader if you want to compare.

tomekslesicki

Is there any way to print the final url with the parameter part? The docs https://docs.hise.dev/scripting/scripting-api/server/index.html#callwithget say that "The parameters have to be a non-nested JSON object and will automatically change the URL to embed these parameters". What does the final url look like after the change?

d.healey

@tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

The issue was only present on macOS Sequoia,

Have the three customers tested other versions of MacOS?

tomekslesicki

@d-healey of course not ;-) But I also have one guy on Windows experiencing this now.

d.healey

@tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

of course not ;-) But I also have one guy on Windows experiencing this now.

What is the common link between all the people who have the issue?

tomekslesicki

@d-healey none, but I suspect the problem is with the parameter part of the url called by HISE. The reason I suspect this is that the POST calls are made to a different server, and they work fine. The download callback works fine wherever I test it and - again, works for the vast majority of users - but I guess it's possible that the parameter part is not attached to the call properly, and the server rejects it because it doesn't pass the authorization.

HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

Is there any way to print the final url with the parameter part? The docs https://docs.hise.dev/scripting/scripting-api/server/index.html#callwithget say tha...

Forum (forum.hise.audio)

d.healey

@tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

I suspect the problem is with the parameter part of the url called by HISE.

If that was the case wouldn't it fail for all users?

tomekslesicki

@d-healey maybe it works on some systems but not others, for whatever reason? Is there a way to get the final url of the call affected by the parameter property?

d.healey

@tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

maybe it works on some systems but not others, for whatever reason?

That's why we need to find the common factor. Then you can recreate the system in a VM and debug it.

What OS are they using?
What DAW are they using?
Are you able to make a standalone app that just does the download call so that it can be tested in isolation?
Do they have a firewall?
Do they use a VPN?
Who is their internet provider?
What country are they in?
etc.

Find out everything that they have in common and build a VM to match.

@tomekslesicki said in HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution:

Is there a way to get the final url of the call affected by the parameter property?

Only on your server I think.

HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

HEADS UP: Server.downloadFile() issue on macOS Sequoia - and a solution

26

1.7k

11.6k

101.4k