Evidence based gameplay mechanics

[Oor-tael]

Hello everyone. I have always wanted to understand some gameplay mechanics about covert operations, and as I have never found serious answers to my questions anywhere, I had sworn to myself to one day roll up my sleeves and go for extensive testing.
The Force was with me these days and so that was the time !

I'll release my datas slowly but I can begin now.

Hypothesis n°1 : the more agents performing a mission with decoys, the increased chanced of the outcome "foiled".

Methodology : Imperial commandos only performing sabotage missions on civilian buildings on 100% loyal Alliance planets guarded by 4 sullustan regiments.
Comparison between 2 groups :
1/ 30 missions with 1 agent & 2 decoys
2/ 30 missions with 7 agents & 2 decoys

Outcomes : Can be foiled (FO), failed (FA) or success (SU).
1/ 20 FO ; 8 FA ; 2 SU -> 33,3% undetected
2/ 22 FO ; 1 FA ; 7 SU -> 26,7% undetected

Discussion : despite my extensive testing, the numbers are small and so light differences are likely in the margin of error. So here I think that the low difference observed between the 2 highly different groups proves the hypothesis wrong.

Conclusion : hypothesis n°1 was wrong, with decoys on a mission the number of agents performing the mission does not influence the chance of detection.

[Russian Orthodox Revolution]

Could you tell me what sort of detection the target had?

[Oor-tael]

Originally posted by Praetor Pyro:

Could you tell me what sort of detection the target had?

As written in the methodology : 4 sullustan regiments, nothing else.

OK now,

hypothesis n°2 : putting more decoys decreases the chance of the foiled outcome.

Methodology : Imperial commandos performing sabotage missions on civilian buildings on 100% loyal Alliance planets guarded by 4 sullustan regiments.
Comparison between 3 groups :
1/ 30 missions with 1 agent & 1 decoy
2/ 30 missions with 1 agent & 5 decoys
3/ 30 missions with 1 agent & 8 decoys

Outcomes :
1/ 23 FO ; 2 FA ; 5 SU -> 23,3% undetected
2/ 16 FO ; 7 FA ; 7 SU -> 46,7% undetected
3/ 17 FO ; 8 FA ; 5 SU -> 43,3% undetected

Discussion : First of all, even if not from the same "trial" I think I can add the result of the first group from hypothesis n°1 (30 missions with 1 agent & 2 decoys) ; the methodology was identical. And so I get 4 groups with an increasing number of decoys (1, 2, 5 & 8) and 4 results that seem to describe an increase then maybe a plateau (23%, 33%, 46% & 43%) within the margin of error.

Conclusion : hypothesis n°2 is partially true. Increasing the number of decoys increases the chance of being undetected... To a certain point.

That conclusion leads me to another hypothesis that I would like to test someday : putting more decoys than the number of detectors does not decrease further the chance of detection.

[Oor-tael]

Hypothesis n°3 : the more fighters garrisoned on a planet, the increased chanced to foil enemy ground missions.

Methodology : groups of 2 Noghris commandos (1 agent & 1 decoy) performing assassination missions on Luke Jedi student (169 combat) on 100% loyal Alliance planets.
Comparison between 3 groups :
1/ 20 missions with no detectors
2/ 20 missions with 18 X-Wings garrisoned
3/ 11 missions with 18 X-Wings + Luke set as commander.

Outcomes :
1/ 0 FO ; 20 FA ; 0 SU -> 100% undetected
2/ 0 FO ; 20 FA ; 0 SU -> 100% undetected
3/ 0 FO ; 11 FA ; 0 SU -> 100% undetected

Discussion : 1st group was obvious but I wanted a comparrison and a test to see if mere loyalty could foil a mission with 1 decoy. 3rd group had only 11 tests because not very useful & the process is very time consuming ! Anyway the results were obvious...

Conclusion : hypothesis n°3 was absolutely wrong. Garrisoned fighters do NOT use their detection values against ground missions !

Another hypothesis : garrisoned fighters do not detect orbital missions. They most probably don't, so not a priority...

[Oor-tael]

Hypothesis n°4 : Fighters in orbit (inside capital ships) increase the chance to foil enemy missions.

Methodology : groups of Noghris commandos (1 agent & 1 or 2 decoys) performing assassination missions on Luke Jedi student (169 combat) on 100% loyal Alliance planets.
Comparison between 3 groups :
1/ 20 missions (1 agent & 1 decoy) targeting Luke on the ground, protected by 2 empty carriers.
2/ 20 missions (1 agent & 1 decoy) targeting Luke on the ground, protected by 2 carriers full (12 X-Wings).
3/ 20 missions (1 agent & 2 decoys) targeting Luke in orbit & set to commander, inside the same fleet of 2 carriers full (12 X-Wings).

Outcomes :
1/ 5 FO ; 15 FA ; 0 SU -> 75% undetected
2/ 14 FO ; 6 FA ; 0 SU -> 30% undetected
3/ 20 FO ; 0 FA ; 0 SU -> 0% undetected

Discussion : Nothing special here... 3rd group was a bit different because I wanted to test with a commander, but leaving him on the ground seemed pointless. He was also the target of the mission so the target moved from ground in groups 1 & 2 to orbit in group 3, but in the end the result was as expected. I also put one more decoy in group 3 because I felt the missions were too weak compared to the detectors, and it appears that even so it was far from balanced.

Conclusion : No surprise here, hypothesis n°4 was correct.

[Nelyus]

Use Jedi, As decoy too.
I had Daala go all the way up to 800+ in stats from missions, Sometimes used Vader as decoy when the place was heavily guarded. Then success was closer to 99%
Make sure you have good Combat and Espionage stats for Assassination, Abduction or Sabotage, only good Espionage stats are required for Decoys.
The characters being Jedi Knight and Jedi Master has a huge increase in success rate and chances to escape capture.
If you tried a mission and foiled,
Probably put 3 or 4 decoys with 96+ Espionage, then it won`t fail again.
I found decoys with 100 or close in Espionage were extremely helpful, especially when you can use many.

[Oor-tael]

Thank you Nelyus but... You're talking case report in an evidence based thread ;-)
Unless you want to suggest a new hypothesis to test ? Like : "the higher the espionnage value of all the decoys, the increased chance to avoid detection" ? That sounds likely but worth testing. The problem is finding an acceptable methodology... The game uses a seed mechanism to simulate randomness and so it is very hard to test anything if not in parallel with huge amounts of agents.

[Oor-tael]

Hypothesis n°5 : Regiments garrisoned on a planet can detect enemy orbital missions (missions with an orbital target)

Methodology : groups of 3 commandos (imperial or noghris, depending on the mission, but always 1 agent & 2 decoys) performing sabotage or assassination missions on an orbital target (carrier or Luke aboard). The planet is 100% loyal to the Alliance and has 18 sullustan regiments garrisoned.
3 groups :
1/ 15 missions targeting the single carrier in orbit
2/ 15 missions targeting the single carrier in orbit & Luke is set to general on the ground
3/ 15 missions targeting Luke aboard the carrier

Outcomes :
1/ 2 FO (day of arrival) ; 0 FO (later) ; 1 FA ; 1 SU -> 100% undetected after day 1
2/ 1 FO (day of arrival) ; 0 FO (later) ; 0 FA ; 1 SU -> 100% undetected after day 1
3/ 1 FO (day of arrival) ; 0 FO (later) ; 14 FA ; 0 SU -> 100% undetected after day 1

Discussion : First of all, I decided to split the foiled outcomes between the foiled on the day of arrival and the foiled later, to see the difference between the orbital detection and the ground detection. That was necessary for that kind of test ! Then... Ok ok, I know, groups 1 & 2 have insufficient numbers... But the methodology was flawed ! As soon as one mission was successful, the target disappeared and all the other missions failed with "target not found"... That was stupid. So I decided to add the group n°3 with a mission that cannot succeed Anyway, the results are pretty obvious ! With such high detection levels, if the mission could be foiled it would have been on so many attempts.

Conclusion : Hypothesis n°5 was wrong ! Regiments on a planet DO NOT detect orbital missions !

[Kegbelly]

the covert ops in this game where awesome and i will never understand why the game was never remade with a focus on it . and i dont think theres anything like it in any game e outside star wars

[Oor-tael]

I confess having wet dreams of a true remake !
This is absolutely unique : all other wargames have disappointed me on this specific aspect...

[Oor-tael]

Hypothesis n°6 : Regiments inside a fleet detect enemy orbital missions (missions with an orbital target)

Methodology : 15 missions of 3 noghris commandos (1 agent & 2 decoys) performing assassination missions on Luke Jedi student (169 combat) inside au bulk transport with 6 sullustan regiments. The planet is 100% loyal to the Alliance but free of any unit.

Outcomes :
0 FO (day of arrival) ; 0 FO (later) ; 15 FA ; 0 SU -> 100% undetected

Discussion : Pretty simple... It did not need multiple groups, just enough missions to be significant, and also enough detectors to be sure not to miss anything due to poor detection. The Alliance bulk transport was helpful with its 6 regiments capacity.

Conclusion : Hypothesis n°6 was wrong ! Regiments inside a fleet DO NOT detect orbital missions !

[Oor-tael]

Hello everyone !
I've not given up on my holy mission, and 2 years later I'm back... And back testing again something I had already done :-)

Hypothesis n°2 bis : The more decoys on a mission, the fewer chances of a foiled outcome

Methodology : 3 groups of 100 sabotage missions composed of imperial commandos (1 agent & 1 decoy // 1 agent & 3 decoys // 1 agent & 6 decoys) targeting fighter squadrons on the planet (not inside a fleet). The detectors are 3 sullustan regiments, with no officer. Nothing in orbit. The planet is 100% loyal to the Alliance.

Outcomes :
*group 1A1D : 66 FO ; 16 FA ; 18 SU -> 34% undetected
*group 1A3D : 63 FO ; 16 FA ; 21 SU -> 37% undetected
*group 1A6D : 69 FO ; 18 FA ; 13 SU -> 31% undetected

Discussion : Even though I really tested 100 times each group, the small variations do not look significant. The chances of detection not even seem to increase slightly...

Conclusion : It appears that my first test 2 years ago of hypothesis n°2 was insufficient, with numbers too small. I should not have concluded by the time. Now I feel more confident in the result -> Putting more decoys on a mission does not increase the chance to avoid detection !

(Had I known...)

[Oor-tael]

Hypothesis n°7 : in order to detect enemy missions, only the sum of the detection ratings matters, not the quality of the units composing the detectors

Methodology : 2 groups of 100 sabotage missions composed of imperial commandos (1 agent & 3 decoys) targeting fighter squadrons on two different planets (not inside fleets). 1st group goes on a planet guarded by 3 sullustan regiments (3*35=105 total detection rating), 2nd group on a planet guarded by 11 alliance army regiments (11*10=110 total detection rating). No officer. Nothing in orbit. The 2 planets are 100% loyal to the Alliance.

Outcomes :
*1st group (3 sullust.) : 63 FO ; 16 FA ; 21 SU -> 37% undetected
*2nd group (11 AAR) : 41 FO ; 26 FA ; 33 SU -> 59% undetected

Discussion : First, yeah of course group 1 was used from previous hypothesis. I wasn't going to do it again ! ^_^' Now about the results : The difference seems pretty important, and with a hundred tests each, I think it is significant. So even though the 2nd group had a slight advantage in total detection value, its efficacy is way behind. This leads to another hypothesis : is the detection rating of a unit a mere probability (%) of detecting an enemy mission ? If so, each AAR would have 10% chance of foiling a mission, and each sullustan regiment 35%... Of course there must be other factors like loyalty of the planet and the espionage rating of the decoy... But now, the more I think about it, the more it looks plausible. Maybe I'll take the time to verify it one day... If the force is still with me ! :-D

Conclusion : Hypothesis n°7 was wrong... The quality of the detectors matters. The detection ratings of detectors are not values to simply add up to assess the potential to foil an enemy mission.

[DenisK1000]

Wow! I loved your testing, that is exactly what I was looing for. Will run some more tests myself and post the results.

[TruthRealm]

"Detection rating" is a 1-way/'defensive-only' Espionage rating... 1-way/defensive-only because obviously Espionage rating is used 2-way/'offensively' but otherwise the parameters are used identically

As in every other case in this incredibly-well-designed masterpiece, and unlike the overwhelming majority of games ever made where parameters' values are treated statically and evaluated independently against an arbitrary critical value, Rebellion is nearly-fully-dynamic and evaluates against a dependent variable ( another entity's designated parameter's value ) for comparative advantage

That's why all of these guides which make any static statement as "100 is the minimum rating to consider for diplomacy", or "personnel x, y, and z are your only diplomats" are automatically wrong; a character having a rating of 1 can absolutely succeed, and another character having a rating of 900 could fail...it depends ( the ultimate answer to anything and everything worth knowing or discussing is "it depends" ) on the situation and mission specifics

The dynamicism of Rebellion means there's heavy, healthy dependency on means, differences and ratios for 1st-order calculation of outcomes

[Oor-tael]

Originally posted by TruthRealm:

"Detection rating" is a 1-way/'defensive-only' Espionage rating... 1-way/defensive-only because obviously Espionage rating is used 2-way/'offensively' but otherwise the parameters are used identically

But to the best of my knowledge, we don't know the mathematical formula used, so even if the detection rating is a purely defensive Espionnage rating, that doesn't help very much. Without troops (without any detection rating) a character (= a unit with an Espionnage rating) cannot foil any mission.
The espionnage rating is only used defensively in 2 ways :
- When a character has an officer status. In this case his espionnage rating increases the chances of foiling enemy missions... If there are troops with detection rating.
- When you target one of your own planets with an espionnage mission. In this case it *appears* that the success of the (counter)espionnage is reduced when there is an enemy covert mission. Hypothesis : a counter espionnage mission fails more often if the enemy mission has a high espionnage rating... Would love to test it one day !