Faster Downloads with BitTorrent protocol.

[Vapor]

I'm very disappointed by download speeds.

I'm currently downloading a game update that should have finished in a couple minutes but I've been waiting over half an hour. The client says my peak speed was was about a fifth of what I consider to be my average download speed from other sites and services. On top of that, the download seems to suspend for several minutes at a time.

(I'm no novice to troubleshooting Internet connectivity but, yes, I did go over the troubleshooting tips looking for anything I might have missed. I've also tried several mirrors. Please note, I'm not posting in the Suggestions/Ideas section for troubleshooting advice.)

I am convinced that the problem lies with Steam's content servers being overwhelmed. Although I feel things could be improved, download speeds for normal game installs seem ...meh... tolerable. But, game updates are terrible. And, really, I'm not surprised. Some game updates are massive and hardware infrastructure is expensive. It isn't reasonable, from a business standpoint, for Valve to maintain the equipment needed for peak activity.

When a game update comes out, everyone I know who plays the game is downloading it at about the same time. The content servers are probably red-lining and traffic slowly tapers off as downloads finish and players move on to playing. During this time, we're all competing for the content server's bandwidth, reducing each other's speed.

If Steam were to offer "BitTorrent" as one of the download mirror options, the problem would become a feature. Using the BitTorrent protocol, more players downloading at once means faster downloads for everyone. There may be some technical hurdles but that's why I'm suggesting the option of BitTorrent rather a full replacement to the mirror system. For those that use BitTorrent to receive their games and updates, the limit to their download speed would be their own Internet connection rather than Valve's equipment. This is also beneficial for Valve because their servers will need to use far less bandwidth to distribute the same content to the same number of users.

So, that's my suggestion: A BitTorrent option.

[Vapor]

Originally posted by Ettanin:

https://github.com/arvidn/libtorrent/issues/1489

already broken by ISPs.

BitTorrent inherently doesn't support TLS because TLS requires a valid and trusted server certificate to function against Man-in-the-middle, something that's impossible to provide in a peer-to-peer scenario.

That's annoying but not necessarily a hindrance in this case. There's no reason Valve needs to adhere to conventional practices when implementing it into their own client. Plus Valve could serve as the certificate authority with the negotiation handled internally in the client. I trust their implementation would be thoughtful and thorough.

And, even if they are unable to produce a workable BitTorrent option for all users, I maintain that it could be helpful to some customers and would definitely help their bottom line. (They have to pay bills too and theirs is certainly not calculated as a monthly fee with unlimited data.) For those that can't use it, traditional mirrors would still be an option.

[Ettanin]

Originally posted by Vapor:

That's annoying but not necessarily a hindrance in this case. There's no reason Valve needs to adhere to conventional practices when implementing it into their own client.

Too much development work for edge cases.

Plus Valve could serve as the certificate authority with the negotiation handled internally in the client. I trust their implementation would be thoughtful and thorough.

IP addresses aren't necessarily static (dynamic allocation) nor unique (CGNAT/DSLite). Nor can you reliably bind certification to hardware because that hardware is invisible to the outside world.

And, even if they are unable to produce a workable BitTorrent option for all users, I maintain that it could be helpful to customers and would definitely help their bottom line. (They have to pay bills too and theirs is certainly not calculated as a monthly fee with unlimited data.)

Valve wouldn't run a CDN like they do now on their own if it didn't fit in their budget nor their revenue plans.

BitTorrent won't help against cheapskate ISPs with slow and oversaturated backbone exchanges. Most game companies using BitTorrent do so (and force to) to save traffic costs on their end. Valve doesn't need to do this, the expenditure in developing a solution like that outweighs the revenue in their current costs if they added BitTorrent.

[Traumhaft]

You would essentially be distributing software which you have no right of distributing.

[Start_Running]

Originally posted by Vapor:

Originally posted by Start_Running:

Your COnnection/ISP.
FUn story, very few residential ISP packages will guarantee given amount of bandwidth.
In all liklihood your ISP is the SPeedbump.

My bandwidth from any other source is five times faster than steam. Whatever I'm guaranteed, Steam and only Steam doesn't keep up with whatever else I might try to download. Yes, my ISP COULD be the problem here but, if that's the case, its because they're choking my connection to Steam and only Steam.

And you're showing how little you understand about network traffic. It's a trivial thing for ISPs to selectively throttle speed based on the source, destination or even the type of data being transmitted. Sop your ISP might very well be throttling data from STeam. They'd hardly be the first,

Again, I am not in Suggestions/Ideas for troubleshooting advice. I appreciate the attempt to help but that's not what I'm here to discuss. I'm here to make a suggestion for an OPTIONAL improvement to Steam which should help some people download games faster and reduce stress on Valve's servers.

We're not troubleshooting you M8. We're telling you why your assessment of the problem is likly to be incorrect ergo your suggestion is meant to fix a non-existant problem.

This is a discussion. If you can't handlyy discussion. Posti your gripe in your blog.

Satoru, you seem to be against having an option to implement BitTorrent into Steam's methods for distribution.

He does it for the same reason people would object to putting square wheels on a car. It would solve no real oproblem and intrudice no small amount of issues and soft-points.

Is the only reason you seems to be against it because you believe it would be redundant? If that's true, that's definitely a valid consideration. Can you point me to some kind of official release verifying that it is? Is there any other reason why you don't support the suggestion?

TYhe simple fact that STeam can saturate most any connections is well known and ergo proof of the redundancy. The fact that steam does indeed ultilize chunk is also known (this is why preallocation is a thing and why it can sometimes take an age.

[Tito Shivan]

Originally posted by Traumhaft:

You would essentially be distributing software which you have no right of distributing.

We're forgetting here the fact that developers are also Steam customers.
I don't think EA, Activision or every other big publisher to be happy with their software being peer distributed by Steam users.

[Ettanin]

Originally posted by Tito Shivan:

Originally posted by Traumhaft:

You would essentially be distributing software which you have no right of distributing.

We're forgetting here the fact that developers are also Steam customers.
I don't think EA, Activision or every other big publisher to be happy with their software being peer distributed by Steam users.

Let alone license validation. How should peers validate license ownership with each other and verify whether the incoming or outgoing peer even has a valid license to serve/obtain the data? That would require Steam to trust seeding peers.

Requesting ownership verification from Steam itself by the other peer would require bypassing a user's privacy settings.

[Quint the Alligator Snapper]

If you want to download games via torrent, buy games from Humble Store. Specifically, look for those products that are marked DRM-free. Because those will let you download a game installer, and Humble offers a torrent-based downloading option.

[Crashed]

To OP, is your client getting bottlenecked by disk usage?

[Vapor]

Originally posted by Ettanin:

Too much development work for edge cases.

They already have developers on staff. If a new feature increases the value of their product and decreases their costs, they have every reason to consider it.

Originally posted by Ettanin:

IP addresses aren't static (dynamic allocation) nor unique (CGNAT/DSLite). Nor can you reliably bind certification to hardware.

Can you reliably bind a certificate to a sessions, account, or unique client instance if it has access to a central authority?

Originally posted by Ettanin:

Valve wouldn't run a CDN like they do now on their own if it didn't fit in their budget nor their revenue plans.

I don't claim to know anything about Valve's budget process but I'm sure, if they like my suggestion, they'll figure it out for themselves.

Originally posted by Traumhaft:

You would essentially be distributing software which you have no right of distributing.

That's interesting! I didn't think of that. Maybe that could be a problem but I imagine there are a lot of arguments to be made there. Are the customer's distributing the software or is Valve distributing the software? A good legal team might say Steam client is just using the customer's hardware, much like a router is used to send the data over HTTP. But, you're right, that's another angle to consider and definitely could be prohibitive.

Originally posted by Start_Running:

And you're showing how little you understand about network traffic. It's a trivial thing for ISPs to selectively throttle speed based on the source, destination or even the type of data being transmitted. Sop your ISP might very well be throttling data from STeam. They'd hardly be the first,

Okay...? Anyway, you're correct that its trivial to throttle based on specific criteria. I never said it wasn't. But encrypted traffic to/from various sources is much more difficult to throttle than a list of 130 servers.

Originally posted by Start_Running:

We're not troubleshooting you M8. We're telling you why your assessment of the problem is likly to be incorrect ergo your suggestion is meant to fix a non-existant problem.

Okay, so apparently you've never experienced a situation where the Steam client downloads at a significantly slower rate than your connection is capable of? Assuming your experience represents the vast majority, that implies the ISP is throttling ergo a problem exists.

Originally posted by Start_Running:

This is a discussion. If you can't handlyy discussion. Posti your gripe in your blog.

Correct. I was hoping to have a discussion about a BitTorrent option in the Steam client. Maybe you're misreading my tone? I'm not hostile. I'm trying to stay on topic.

Originally posted by Start_Running:

TYhe simple fact that STeam can saturate most any connections is well known and ergo proof of the redundancy. The fact that steam does indeed ultilize chunk is also known (this is why preallocation is a thing and why it can sometimes take an age.

No, that's not proof. That's argumentum ad populum. Proof would be some kind of documentation from Valve which states that the bandwidth allocated to their content servers never reaches its limitations. Another form of proof (to support Satoru's statement that they use some kind of ad-hoc distribution similar to BitTorrent), would similarly be a statement from a Valve representative saying so.

I'm confident that Steam can saturate MY connection, assuming every other customer isn't downloading something at the exact same time. I'm not saying they're running services out of the back of garage or something. But I don't buy Steam's total bandwidth capabilities come close to matching the bandwidth capabilities of every customer combined. And, if they were doing that, I'd suggest BitTorrent as a way they can scale back their truly insane budget! Surely, on occasion, their content servers are under strain. A BitTorrent option would relieve that strain in the form of operational costs for them and speed benefits to their customers.

Originally posted by Crashed:

To OP, is your client getting bottlenecked by disk usage?

Nope. SSD, CPU, and network interface are almost completely idle. I routinely deal with high-volume data transfers on machines with much lower specs and I've confirmed the problem only exists with Steam during update releases.

Originally posted by Vapor:

So, that's my suggestion: A BitTorrent option.

Clearly the user support for the suggestion is negligible. I still maintain that it would be a helpful feature, should Valve find a way past any technical or legal hurdles. In the very least, it would reduce the bandwidth costs for Steam and for users on a LAN. And, at best, it would ease strain on content servers or bypass ISP throttling. That said, it sounds like the mob has weighed against the idea, deeming it unwanted for one reason or another. So, I guess I'll shut this down and go back to games, leaving any potential advocacy to others. Thanks for your feedback, Everyone.

[Crashed]

You say SSD is idle, but are you installing to the SSD that you are monitoring or could you be installing to say a hard drive that easily bottlenecks? If everything is truly idle then I'd suggest contacting your ISP in case they are having issues. There could be either routing issues to the download servers or network congestion, as mentioned multiple times the Steam download servers are easily capable of saturating just about any connection.

I personally updated Persona 4 Golden today to the beta and my connection was completely saturated as it downloaded.

For the legal aspects it would require an update in the agreement made with developers, to something similar to what Windows does for its store (Delivery Optimization).

[Radene]

No, but seriously. In the end, it might be a download server thing - and choosing the closest one to you "geographically" doesn't mean you'll get best speeds.

I got a gigabit line, and even when downloading to an SSD, and even when that's the only activity the system is doing, the download will sometimes just get all flaccid. Try changing the download server before you go complaining to your ISP.

[Crashed]

Originally posted by Radene:

No, but seriously. In the end, it might be a download server thing - and choosing the closest one to you "geographically" doesn't mean you'll get best speeds.

I got a gigabit line, and even when downloading to an SSD, and even when that's the only activity the system is doing, the download will sometimes just get all flaccid. Try changing the download server before you go complaining to your ISP.

Then everyone does that and plugs up an undersea cable?

[Ettanin]

Originally posted by Vapor:

They already have developers on staff. If a new feature increases the value of their product and decreases their costs, they have every reason to consider it.

Where's the profit in developing a peer-to-peer architecture that not only needs the blessings of all publishers on Steam, but may well cause legal issues due to indiscernibility of peer legitimacy? Just because a fraction of the user base is throttled by some possibly insignificant ISP? Not to mention that it won't help if the ISP doesn't have the peering agreements to saturate everyone or is intentionally a cheapskate to force double paid traffic agreements? (see https://wiki.hetzner.de/index.php/Double_Paid_Traffic/en [for Hetzner obsolete, but for many other hosters still a reality])

Can you reliably bind a certificate to a sessions, account, or unique client instance if it has access to a central authority?

No you can't because it can't be verified who behind all the hops is the authoritative holder of the session or account. While a session or an account can authenticate you as a legitimate user, it cannot clear all doubts about the peer being the peer it alleges to be. It might be the legitimate host, but it also might be a (malicious) proxy acting as the legitimate host. Also, it's a bad idea to give other peers than Valve's servers any hint of an identifier for authentication. (Sure, there's "sign in through steam", but this is just a user API, not a true authenticating functionality and shouldn't be used as such, never offload authentication to a third party.)

Valve could in theory apply something like PGP and act as a keyring of sorts (for envelope encryption), but this won't solve the peer legitimacy issue from the license standpoint. Yes, it's this and that peer, but does it even have the legitimacy to download or upload the game? Peer legitimacy validation would require Valve to disclose which user has which games to other peers, which potentially contradicts the privacy settings of said user.

Also, peers are outside of Valve's control and could (intentionally) break the trust chain. If it's not on their servers, it can't be trusted. Peer to Peer would actually be a weakspot to Steamworks DRM. The user cannot be absolutely certain that the source they are downloading from and/or the target they are uploading to are completely legitimate.

How will you solve CGNAT and/or Dual Stack Lite issues? Sure, go IPv6, you might say, but there are still ISPs out there that deny access to IPv6.

You see, the possible solution grows more and more complex and hence will cost more to develop.

[Washell]

From valve's point of view, everything they own, control and configure is able to saturate your connection. For them to implement a p2p protocol is opening a can of worms to solve a problem that's on your end. With no significant benefit to them or most users. Given that they haven't bothered to implement such in the 17 years of their existence and bandwidth costs have only gone down, you shouldn't have much hope.

It's not that we inherently oppose the implementation of p2p, but rather that we're trying to offer you solutions that will actually fix your problem rather than letting you keep dreaming of castles in the sky.