Question about queued downloads from Amazon S3

[d.healey]

@Christoph-Hart With Amazon S3 I use urls that expire after a couple of minutes. Does the built in download queue trigger each download and then add them to a waiting list, or does it only trigger the download when it's its turn to be downloaded?

I'm thinking if I have 5 urls and they expire after 10 minutes, and the HISE project is set to download up to 3 files at once, what will happen if the first 3 files take more than 10 minutes to download? The remaining two urls will have already expired unless they've been started briefly and paused.

[d.healey]

Actually I think I might be able to avoid this problem by redirecting the url through my site and have my site generate the Amazon S3 url at the time the download is started.

[Christoph Hart]

I'm pretty sure that the downloads are queried just before they are downloaded.

[Dan Korneff]

@d-healey Do you have any wisdom you cloud share on downloading S3 links in HISE?
When I download a file that's hosted on my site, all is well, but I get errors when using the S3 link provided by Woocommerce.
The Woocommerce endpoint gives me 2 links in the JSON file that look like this:

"https://MyBucketName.amazonaws.com/myfile.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=SOMELONGSTRING%2BucketSTufff%2Fs3%2Faws4_request&X-Amz-Date=20220924T040013Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=longstringwithnumbersandletters"
-AND-
"https://mywebsite.com/?download_file=productnumber&order=wc_order_ordernmber&email=me@myemail.com&key=some-long-key-1234"

I assume the first link is the one I want, but not sure how to pass in the amazon credentials.

[d.healey]

@Dan-Korneff The second one is probably the one you want. Make sure in woocomnerce you have the downloads require login checkbox unchecked.

[Dan Korneff]

@d-healey said in Question about queued downloads from Amazon S3:

Make sure in woocomnerce you have the downloads require login checkbox unchecked.

Do you recall where that setting is?
EDIT:
Found it.

Doesn't seem to cure my issue yet. When I use the 2nd link provided, the server seems to ignore everything after "https://mywebsite.com/"

When I paste the 2nd link into a web browser, it starts downloading, and the file link is actually the first one I posted.

[d.healey]

@Dan-Korneff That first link is a direct link to the resource, doesn't look like it's a link that expires to me so if someone gets hold of it and shares it your wallet would hurt.

How are you getting the link? Are you using the WooCommerce S3 plugin?

[Dan Korneff]

@d-healey Yeah, I'm using the S3 plugin. I'm gonna check out the S3 API since this might just be an amazon issue.

[d.healey]

@Dan-Korneff I'm using a custom wordpress plugin that calls $item->get_item_downloads(); https://woocommerce.github.io/code-reference/classes/WC-Order-Item-Product.html#method_get_item_downloads

[Dan Korneff]

@d-healey Thanks for this! I have half of the issue worked out. I had to set my server to "force download" so it doesn't issue a redirect.
The other half of my issue has to do with extracting the download link from JSON.
If I paste the exact link into Server.downloadFile(), it works perfectly.
If I store the download link in a variable, the link seems to add "" to the string.

for example the response from my server looks something like this:

  {
    "download_url": "https://mywebsite.com/?download_file=productnumber&order=wc_order_ordernmber&email=me@myemail.com&key=some-long-key-1234",
}

I store it in a variable like this:

downloadLink = trace(response.download_url);

if I print downloadLink to the console, it says:

"https://mywebsite.com/?download_file=productnumber&order=wc_order_ordernmber&email=me@myemail.com&key=some-long-key-1234"

So far, so good.
Then I user Server.downloadFile() to start the download.

When I look at the data from the Server Controller, the download link seems to add a " in front of the link and then times out.
So it looks like:

https://"mywebsite.com/

instead of

https://mywebsite.com/

What am I doing wrong?

[d.healey]

@Dan-Korneff said in Question about queued downloads from Amazon S3:

@d-healey Thanks for this! I have half of the issue worked out. I had to set my server to "force download" so it doesn't issue a redirect.

Set it to redirect otherwise all of the bandwidth runs through your server. Ignore the warning WooCommerce gives you about this, it doesn't apply when using S3 links - ask WooCommerce if you want to be certain, I did

What have you set your base URL to? You might need to set it to https:// and then strip that from the download URL (I think that's what I did). You can see my function here https://github.com/davidhealey/HiseDownloadManagerToolkit/blob/master/Downloader.js#L40

[Dan Korneff]

@d-healey said in Question about queued downloads from Amazon S3:

Set it to redirect otherwise all of the bandwidth runs through your server. Ignore the warning WooCommerce gives you about this, it doesn't apply when using S3 links - ask WooCommerce if you want to be certain, I did

If I set like this, downloads won't start no matter what I do.

[d.healey]

@Dan-Korneff Are you using an NGINX server?

[Dan Korneff]

@d-healey Yes

[d.healey]

@Dan-Korneff Did you follow this link and do what it says?

[Dan Korneff]

@d-healey Those settings only seem to apply to files hosted on your server, not S3

[d.healey]

@Dan-Korneff Ah well in that case I'm not sure why it's not working for you. Are you using something like hoppscotch for testing?

[Dan Korneff]

@d-healey I'm completely reckless. Testing on my live site

[d.healey]

@Dan-Korneff Living on the edge You can use hoppscotch with your live site though, it is often helpful for debugging problems as you can skip the whole UI stuff in HISE and just get straight to the web requests and responses.

[Dan Korneff]

@d-healey Did you ever find a way to use the JWT token for authentication so you can enable "downloads require login"?