The distinction for speeds is important.
the process of "downloading" a file, or web page, involves a lot of components.
The browser itself, which basically does- nothing. It opens the connection and accepts incoming data to write to the file, but nothing else. Since download speed is always much slower then the write rate, optimizations here, the only place possible in the browser, are usually not fruitful.
Once the browser starts a download, it is the network drivers job to establish a connection to the remote machine, send it the data, wait for the response, and accept it.
The connection involves, regardless of your homes configuration, thousands- maybe millions of kilometre's of wire, some of which may be deteriorating. If the facility that has this "well travelled" wire has a power outage or major disaster, the connection is achieved via an alternate route, which may be longer but have better quality wire. By using better, and shorter ethernet cables I have been able to double my download speed from the original supplied ethernet cabling.
the speed, performance, and load of the remote server is one of the single most important factors for download speed. if the server is experiencing heavy traffic or simply has an isignificant processor, the outlay of information can suffer; the subsequent recieval of the information will also be slower.