Reinforcement learning

[RayneKatt]

I'm having bit of problems with these reinforcement learning tasks and was wondering if anyone could actually explain what it is we're suppose to do in them?

I managed to work out 0 and 1, but I'm stuck on 2. No matter what sliders I use or length of lidar the car will slows to a stop, change lanes and slow down infront of another car, or drive into another car and tailgate it.

Some help would be appreciated.

[GospodinNoob]

Hi.

Show your params and lidar scheme.

[muerthe_svk]

I have the same issue. Even if I do controlled learning when I show AI that it should slow down and go around the car if the lidar is red it doesnt work. The car just goes 50 and doesnt go around the car in front. Even if I evolve AI to lvl10...

[GospodinNoob]

Originally posted by muerthe_svk:

I have the same issue. Even if I do controlled learning when I show AI that it should slow down and go around the car if the lidar is red it doesnt work. The car just goes 50 and doesnt go around the car in front. Even if I evolve AI to lvl10...

Hi.

Show your params and lidar scheme.

[RayneKatt]

Originally posted by GamineBrownies:

Hi.

Show your params and lidar scheme.

Sorry for the delay in responding back. This seems like an odd question as descriptions would be better, but you did ask...

Apologies for the long response, however I was asked for screenshots of lidar and settings. It should be noted that I discovered a bug when doing this and it appears in the screenshots. Basically as I am aiming for gold (which means only 1 evolve stage) I've reloaded the game from a save before taking the RL2 job. However, the game remembers the settings and proceeds to count the some of the previous evovles even though it's a reloaded game. This can be seen in the attached video.

https://www.youtube.com/watch?v=j1GHajDRPIc&feature=youtu.be

I should add before continuing that I picked to start with 2 bars of lidar as the tutorial states (as seen in one of the screens below) that "Adding more lidars will make learning more effective" implying that more is better.

Attempt: 1
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 20/30/10/1
Result: rapidly move side to side at 50mph
Thoughts: the car moves (yay!) but badly and at the wrong speed
https://steamcommunity.com/sharedfiles/filedetails/?id=1640015366
https://steamcommunity.com/sharedfiles/filedetails/?id=1640016347

Attempt: 2
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 50/30/10/1
Result: change lanes, speed up to 65mph, ram car, slow to 30mph, speed up to 55mph ram car, tailgate at 50mph
Thoughts: the speeding up is good, but slowing down and tailgating is bad
https://steamcommunity.com/sharedfiles/filedetails/?id=1640016841
https://steamcommunity.com/sharedfiles/filedetails/?id=1640017894

Attempt: 3
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/30/10/1
Result: change lanes, speed up to 65mph, ram car, slow to 47mph, speed up to 55mph ram car, tailgate at 50mph
Thoughts: Only small change in action.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640018337
https://steamcommunity.com/sharedfiles/filedetails/?id=1640018873

Attempt: 4
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/50/10/1
Result: change lanes, speed up to 55mph, ram car, slow to 47mph, speed up to 55mph ram car, tailgate at 50mph
Thoughts: Only small change in action.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640019307
https://steamcommunity.com/sharedfiles/filedetails/?id=1640019765

Attempt: 5
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/10/1
Result: speed up to 65mph, ram car
Thoughts: Manages speed, but does nothing with obstacle.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640020158
https://steamcommunity.com/sharedfiles/filedetails/?id=1640020874

Attempt: 6
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/50/1
Result: speed up to 54mph, changes lanes, ram car
Thoughts: Creates an obstacle.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640021346
https://steamcommunity.com/sharedfiles/filedetails/?id=1640021707

Attempt: 7
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/100/5
Result: speed up to 65mph, rams car
Thoughts: Feels like one step backwards.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640022109
https://steamcommunity.com/sharedfiles/filedetails/?id=1640022582

Attempt: 8 (forgot to change a setting!)
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/100/5
Result: speed up to 65mph, rams car
Thoughts: Feels like one step backwards.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640023039
https://steamcommunity.com/sharedfiles/filedetails/?id=1640023629

Attempt: 9
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/100/10
Result: speed up to 65mph, rams car
Thoughts: Feels like one step backwards.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640024379
https://steamcommunity.com/sharedfiles/filedetails/?id=1640024379

At this point I've had to exit the game completely, restart the game, and load the save due to the bug.

Attempt: 10
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/100/10
Result: speed up to 65mph, rams car
Thoughts: the car moves but badly and at the wrong speed. Feels like the settings didn't matter as this a repeat of the results from attempt 1.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640024799
https://steamcommunity.com/sharedfiles/filedetails/?id=1640025351

Attempt: 11
Self driving: increase speed to 65mph ram car to slow to 50mph and have it found in lidar then change lead and repeat
Lidar: 2 bars
Sliders: 100/100/100/50
Result: change lanes, speed up to 65mph, ram car, slow to 30mph, speed up to 55mph ram car, tailgate at 50mph
Thoughts: the speeding up is good, but slowing down and tailgating is bad. Feels like the settings didn't matter as this a repeat of the results from attempt 2.
https://steamcommunity.com/sharedfiles/filedetails/?id=1640025772
https://steamcommunity.com/sharedfiles/filedetails/?id=1640026584

I can continue to do this if needed, but it seems a bit silly when all we need is to understand what is happening. Particularly as it can be seen from attempt 10 and 11 that the results begin to repeat despite different sliders being picked.

[GospodinNoob]

RayneKatt

Thanks for so big experiment!

Try to erase the lower lidar cell (click on it).

[Fxore]

Hi guys. I found myself in the same situation as RayneKatt. I've managed to reach RL-3, but anyway I just don't understand how the "evolution" controls work. Of course, mutation does not imply that it will be in the direction of the desired by us, but the process is at least opaque and I don't know how to proceed.

Also, I feel like tweaking the RL params is not even needed sometimes, since parameters can be good enough to find out a solution and not reach it because of the seed. However, because of how all these parameters and RL-experiment runs are shown, it is not clear what are you performing and how should you proceed.

Any clue on how to actually tackle on those experiments? Any guide or thoughts on how to understand what are we supposed to do?

[GospodinNoob]

Originally posted by Fxore:

Hi guys. I found myself in the same situation as RayneKatt. I've managed to reach RL-3, but anyway I just don't understand how the "evolution" controls work. Of course, mutation does not imply that it will be in the direction of the desired by us, but the process is at least opaque and I don't know how to proceed.

Also, I feel like tweaking the RL params is not even needed sometimes, since parameters can be good enough to find out a solution and not reach it because of the seed. However, because of how all these parameters and RL-experiment runs are shown, it is not clear what are you performing and how should you proceed.

Any clue on how to actually tackle on those experiments? Any guide or thoughts on how to understand what are we supposed to do?

Hi. Show your best lidar scheme and parameters.

[wastecan]

After watching YouTube videos of others on this and after reading this and other threads, I'm very frustrated - especially since I have the same questions and issues as so many others.

This task clearly has explanatory issues... As Fxore just asked, "Any clue on how to actually tackle on those experiments? Any guide or thoughts on how to understand what are we supposed to do?" The Developer Response, (Your Response), ignores this most crucial question. What the HECK are we supposed to do????

P.S. The answer ISN'T "Hi. Show your best lidar scheme and parameters."

The answer should be more EXPLANATORY... How should we think about coming to the answer for this task... Thanks.