HUGE Upload during hosting

[GABBA GANDALF]

Hi! In short: I created a server for 24 players after the update came. Played for 2-3 hours, then my ISP banned my connection because there is a 4Gigabyte/24hr upload limit to prevent p2p sharing.
I never hit this limit when I play other online games, not WoW, not SC2, not 10hrs of playing CS or Quake or whatever.
So please fix this, the netcode still streams wayyy to many data. I cannot play the game in MP or at least host one if that doesn't get fixed.

That would be a pity, i have a superfast internet, and my server was always immidiately full because I have one of the lowest pings in EU (so said the players that were connecting to my server).

[⋋UPGRAYEDD⋌]

It certainly sounds like they are still optimizing the network code, so I'd expect it to get better.

[jannesBB]

No way the game burned through four gigs in a couple of hours.

That said, the network code will be optimized in future.

[Adenru]

You should change ISP.

[Cheese is Life]

@ OP. Stupid question but by any chance, did you have something big d/l in the background?

Edit.
Sorry, my bad. It's upload not download. :)

[GABBA GANDALF]

Originally posted by Adenru:

You should change ISP.

Lol I should change a 10 Gbit for 5$ contract because the gamecode isn't finalized? LOL

Originally posted by jannesBB:

No way the game burned through four gigs in a couple of hours.

That said, the network code will be optimized in future.

I don't have the log anymore, but it WAS the reason. I set the updaterate to high and always had 23 competitors on the server.
I will try it again someday and put the log in here if it helps. But since it was the only thing running during that time period, it can only be this or there was some p2p sharing via steam going on (which i checked and didn't find an option turned on for something like that. Blizzards launcher has that option I don't know if STEAM has it anyways).

Originally posted by Walking Tall:

@ OP. Stupid question but by any chance, did you have something big d/l in the background?

Edit.
Sorry, my bad. It's upload not download. :)

Not a stupid question :)
Its Upload and there is an UPLOAD not Download limit. Anyways I checked and there was a 500 mb Upload prior to the event ( sent my brother a fiale via skype).
I watch my upload very closely with several tools to prevent me crashing into the limit. That is why i switched off everything that produces upload, even limited some windows protocols and other things.
It was definately during the hours of playing NCG

[jannesBB]

Sorry for doubting you, after giving this a bit more thought it's indeed possible even though it does sound like a lot. If you're hosting and increase the number of clients, a single added client will increase the bandwidth requirement exponentially, meaning that by the time you have all the data moving between the host and the full grid of clients the bandwidth requirement would have skyrocketed. Thus, it's really possible to burn a few gigabytes of bandwidth in an hour. You can reduce the bandwidth requirement by lowering the server update frequency in the settings, but it can cause lag.

We can most likely reduce the bandwidth requirements somewhat, but even so the amount of transferred data will be significant when hosting a full server. Therefore it's not recommended to host with a capped connection.

[GABBA GANDALF]

Originally posted by jannesBB:

Sorry for doubting you, after giving this a bit more thought it's indeed possible even though it does sound like a lot. If you're hosting and increase the number of clients, a single added client will increase the bandwidth requirement exponentially, meaning that by the time you have all the data moving between the host and the full grid of clients the bandwidth requirement would have skyrocketed. Thus, it's really possible to burn a few gigabytes of bandwidth in an hour. You can reduce the bandwidth requirement by lowering the server update frequency in the settings, but it can cause lag.

We can most likely reduce the bandwidth requirements somewhat, but even so the amount of transferred data will be significant when hosting a full server. Therefore it's not recommended to host with a capped connection.

Hi Jannes.
Thx for answering :)

I was about to prove my point with a tool and a log by it ;-) Not needed anymore (and therefore not risking a ban again ;-) )

I understand that for high accuracy and low lagging, there needs to be a lot of bandwith. That being said, I play a lot of fast MP Games and that never occurred - not even when I was the host/server of a 32 player map of Quake, Unreal, CS:GO, Starcraft 2, ARMA 3 (with way more players, voice/teamspeak and so on) or whatever game. And as you for sure know, quake also needs high precision, low latency. That is why I wasn't expecting NCG to have such a high bandwith.
Especially in that short period of time, since I play a LOT, sometimes 10 hours a day (on a free day), games like the ones mentioned above - without any problems. I play since the 56k modem era, it was never a problem, not even for the old flatout (but I guess I had ISDN then :D )

I would like to keep up being a server for several games a day or week, my connection speed is insane, everybody who connected was so happy that this was the only lag-free server they found and so on.

I will grab some beers and talk to the administrator so maybe we could give the connection from NCG some free traffic, because this limit was strictly introduced to ban p2p filesharing (which is a good thing it is banned or prevented) but doesn't help if I produce lots of upload/traffic with "honest" things.

But the very very high bandwith and requirements for smooth online play will definately give a lot of other players trouble/headaches and therefore I hope you can take another look into the code from time to time.
Of course I know this isn't your number one priority, which (I guess) is, that you try out the new features and clear some bugs.

I wish you good luck with that and will further be looking forward to new updates and the final product.
Just maybe think about my request here from time to time. I would really like to play it, host some games and also have a small community around me (when i hosted the games and was the only playable server around, lots of players called it the "GABBA GANDALF COMMUNITY" and just waited for me to open a new game again (because every 2 starts the multiplayer-game stopped to work, as I mentioned before in a bug-report).

Cheers and good luck guys!

[GABBA GANDALF]

On another note, maybe steam found my folder for "adult content" and they are downloading it from my disk now which could lead to the high bandwith ;)
Just kidding.
I just wonder why positioning of the cars and physical objects takes so much bandwith and compared to other games (Arma, Battlefield or the games mentioned above) don't. As also mentioned the games from the 56k era did also require high precision (play CS 1.6 with voice on a high level - it would never take much bandwith).
Changing the updaterate to medium - is it comparable to Flatout 2's updaterate? Or how can I imagine it. If there is a lot of lag and delay then, I see a problem :(

If there is any thing I can contribute to help you with the problem or searching for it, I'd be glad.

[jannesBB]

Thanks, and not to worry - having the multiplayer experience polished is very important for us, and further optimization is definitely part of that. Reducing the amount of data not only serves people like you who have a capped connection and want to host but also helps ensuring that everyone will have as lag free experience as possible, and that's our goal. We're not sure how much room there is for further optimization since there's quite a lot of data involved, and especially so if we want to sync the visual damage as well in the future. In that sense, it's mostly a question of prioritization.

And thanks for wanting to host for the benefit of others, the community needs great people like you.

[Marlowe-33]

Glad i have unlimited, i was hosting for around 5-6 hours saturday, and a few hours today.

[GABBA GANDALF]

Originally posted by jannesBB:

Thanks, and not to worry - having the multiplayer experience polished is very important for us, and further optimization is definitely part of that. Reducing the amount of data not only serves people like you who have a capped connection and want to host but also helps ensuring that everyone will have as lag free experience as possible, and that's our goal. We're not sure how much room there is for further optimization since there's quite a lot of data involved, and especially so if we want to sync the visual damage as well in the future. In that sense, it's mostly a question of prioritization.

And thanks for wanting to host for the benefit of other, the community needs great people like you.

thank you. I will try to press the issue with our admin (who is also a Flatout 1 and 2 Fan btw) and maybe I can reach something there so I could be of permanent support to the community.

Glad you optimize for the ones with lower connections.

Maybe in the distant future I check by your office (GF always wants to go to Finland)to grab some beers with you cool guys and maybe someone can explain me the netcode then, especially the part why it takes such a huge ammount of traffic for the damage (how did this work in previous games or does it work in other games, why isn't it calculated on the client side and so on...)

But if that never happens, I will still support you like you support the community. Great update btw had a lot of fun so far (until i got banned :D )

[jannesBB]

The reason why we chose to implement a server based multiplayer as opposed to peer-to-peer is that the server based setup is much more robust and less prone to errors since only the host needs to communicate with each client instead of all clients communicating with each other. We also wanted to have as precise sync as possible since our game crashing into others is a significant part of the gameplay, and for that it's important that the game status is synced between the server and the clients as often as possible to prevent mismatches in positioning data. This largely contributes to the high bandwidth requirement. We do of course employ prediction strategies to reduce the amount of data, but sadly they're not a shortcut to happiness in such an unpredictable game as ours.

We're always up for a couple of beers ;-)

[GABBA GANDALF]

Originally posted by jannesBB:

The reason why we chose to implement a server based multiplayer as opposed to peer-to-peer is that the server based setup is much more robust and less prone to errors since only the host needs to communicate with each client instead of all clients communicating with each other. We also wanted to have as precise sync as possible since our game crashing into others is a significant part of the gameplay, and for that it's important that the game status is synced between the server and the clients as often as possible to prevent mismatches in positioning data. This largely contributes to the high bandwidth requirement. We do of course employ prediction strategies to reduce the amount of data, but sadly they're not a shortcut to happiness in such an unpredictable game as ours.

We're always up for a couple of beers ;-)

Ok I understand :)


Just FYI: I just reran the game for a test: Hosting for 13 players, 3 laps on Sandpit, 2 events.
588 megabyte upstream in 9 Minutes :(
Can provide the wireshark logs if you want/need them

[Adenru]

Originally posted by GABBA GANDALF:

Originally posted by Adenru:

You should change ISP.

Lol I should change a 10 Gbit for 5$ contract because the gamecode isn't finalized?

You should change because you ISP is forcing silly restrictions... Wouldn't it be better for everyone to restrict upload speen instead of traffic?

[GABBA GANDALF]

Originally posted by Adenru:

Originally posted by GABBA GANDALF:

Lol I should change a 10 Gbit for 5$ contract because the gamecode isn't finalized?

You should change because you ISP is crap forcing silly restrictions on you... Wouldn't it be better for everyone to restrict upload speen instead of traffic?

in the 8 yearhs I have the connection now, i have NEVER had that problem. So no it would be silly to change. I don't host games then or give the admin some beer to make a permanent server.
Changing the ISP would mean i change my 10 gbit network for 5$ a month to a worse connection speed (maybe 10 mbit? 1024 times slower several times the cost and not having enough upload speed to maintain good pings for the connecting players.
Currently I am on the fastest interent around in EU, and the restriction isn't stupid, I totally understand it, I just never had the problem before and 588mb upload in 9 minutes is HUGE. This wouldn't even theoretically be possible with any competitor ISP around here.

Having 10 Gbit/sec for no money and no limitations would lead to a lot of abuse and I agree with it.