why do my downloads not complete. vids cut off halfway through.

spacepirate

Video encoding is not just about the images. For starters, the keyframe image header metadata is different with streaming optimized encoded video.

Have you ever encountered, even with a video on your local machine, you could not skip to a certain position, it would always skip to a couple seconds before or after?
You need far smaller chunks when skipping through a video when it's optimized for streaming. Decreasing the delay when skipping forward or backwards. For non-streaming optimized video, every time you skip forward or backward, you would need to download more data before the part you skip to would start playing.

This in turn changes the way the video data can be transferred. You can more easily take advantage of parallel downloading instead of sequential downloading. In many cases (it's not a rule of thumb) sequential downloads are much easier to deal with for both your system and browser. You're less likely to 'choke', less likely to run into connection balancing issues (ie windows auto-tune I mentioned above.) Downloading larger 'chunks' is easier for the system to reconstruct from cache.
And so on...

You might sometimes have noticed that after downloading large files with a browser, when the download is finished (or almost finished) there is a copying job being done. This is moving and reconstructing from cache, to the download destination folder. A download manager (much like a torrent client) provisions space directly in the target folder instead of going through the system (and browser) cache. Some download managers even let you pre-allocate space in the destination folder. This will make it much less likely for your connection or network adapter to 'choke' as it writes the data away immediately instead of caching it.

'Choking' you can see as the network cache being full, without having any complete chunks to pass on. Maybe counter-intuitive, this happens much more easily with smaller chunks as the amount of simultaneous open download connections is increased. (parallel downloading). Hence download managers also have settings for max simultaneous connections.

Some websites (NAVR) don't even allow you to use download managers because they are not set up to handle parallel downloading, and it hammers their servers too much. Which I can attest to. One day after taking out an annual sub with them, they blocked my IP for using a download manager.

For what it's worth, I even asked ChatGPT4 to review the accuracy of my statements, and both validate and invalidate the statements. https://chat.openai.com/share/d4e97591-252b-455b-9a18-d9cf0fcbefc7
I'm not here to make excuses. I believe when people understand how something works, or why something is the way it is, they are more likely be able to know how to handle issues. Instead of thinking it is something we have to fix.
'Fixing' this download issue some people have, would render the streaming service inoperable.
Simultaneously, for legal reasons, I can not recommend the usage of 3rd party apps.

I'm not sure if this image makes it easier or harder to understand, but I'll try to explain the difference in encoding.

• The upper half of the illustration is streaming encoded.
• As you can see, the green frames are all neatly following each other. 1 2 3 4 5 6 ...
• 1 2 3 is a 'chunk', 4 5 6 is a 'chunk' (a packet of data that's sent from the server to you).
• Each key frame also all have their own metadata header containing the information of the frame. These basically hold the instructions on how to generate an image from the data.
• Because each keyframe has it's own metadata, the playback of the video can be jumped to very easily, and doesn't require downloading an entire GOP (see next part)

• The lower half of the image is not encoded for streaming.
• The first thing you'll notice is that they are 'smaller' because there is no metadata for each frame.
• Another thing you might notice is that here the frames are not neatly following each other, now we have 1 4 7 2 5 8 ... (3 6 9).
• This means that when you want to jump to that position in the video, you will need to download the entire GOP, all 3 packets, so the video player can put them in the correct order again (1 2 3 4 5 ...)
• The trade-off is that you don't need extra data for the frame headers. Here one metadata header is used for the entire group of pictures 1 to 9. Making the whole video file-size smaller.

This is a simplified representation.
A GOP of 250 is usually the default used, not 9. GOP literally means 'group of pictures'.
There is nothing we can do when it comes to download management on server side, that changes the construction of the video files. The most common issue for SLR users is network caching 'choke' of being full of incomplete packages. If you still run in to this issue while using a download manager, lower the amount of parallel connections in the settings.

There are still other factors too! ie buffered vs unbuffered drives, Shingled drives vs Conventional drives. But these would be edge cases.

Rakly3

Rakly3

again you are talking a lot about video encoding and gops when that has nothing to do with issues experienced when downloading a video as a file in a browser or download manager (which is what this thread is about).

those two posts read like you have some understanding of a few concepts but its very evident that you dont really understand what you are talking about because most of what you said there is straight up wrong. i was tempted to correct some of the things you said i honestly dont know where to start.

spacepirate that sentence is nonsensical

You're saying pretty much the same as me however.

Browsers have a hard time dealing with this when downloading these files in such large sizes.

I'm trying to explain why users have this problem only on SLR. Downloading from a streaming server is different from downloading from a fileserver. Like you just said, browsers are not equipped to handle large segmented files. Why are they so much more segmented than from other non-streaming sites? What makes it possible to send the data much more "random" than sequential with more simultaneous parallel connections?

When using a video player to stream the files, you will get sent the data within a certain timeframe of the file. When downloading the entire file, you're getting sent parts from all over the file, Making it much more likely to run in to caching issues.

The reason I started explaining specifically about encoding is bc you said this

spacepirate dunno why youre talking about how videos are encoded when that has nothing to do with download issues.

While all i had said up to that point was

It isn't. Our streaming service would stop working if we 'fixed' this. There are many different ways in how videos can be encoded. SLR videos are encoded specifically for streaming. Browsers have a hard time dealing with this when downloading these files in such large sizes. It's a very broad topic involving even your network card and router, but it basically comes down to how the chunks are managed and cached. Download managers handle these completely differently.

The point being, that people are downloading from a streaming server. And what the difference is, because the streaming service would not work if we would not use streaming optimized encodes.

You make it sound as if I said that it's the encoding itself that causes the issue. - Which I didn't.